Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Texturing overhaul: GPU compression, sRGB sampling, swizzles, etc. #240

Merged
merged 48 commits into from
Aug 15, 2020

Conversation

Akaricchi
Copy link
Member

@Akaricchi Akaricchi commented Jul 24, 2020

Note: base branch is set to stage4bg for a cleaner diff, since this is what I've been testing against. It will be rebased onto master before merging.

Support for various GPU compression formats is achieved by integrating the Basis Universal transcoder. Unfortunately, this is C++ code. I have written a basic C API wrapper for it to avoid introducing C++ into Taisei itself. The wrapper resides in our basis_universal fork.

Rough TODO:

  • (Attempt to) clean up the code, remove debug garbage, etc.
  • Split .basis loading code into a separate file.
  • Make .basis support optional in the engine.
    • Possibly also transcode .basis files into .png/.webp at build time if support is disabled.
  • Utilize basis userdata fields to inform the loader about the nature of the content; needed for some special cases like normalmaps.
  • Document some obscure details, like the aforementioned special cases, and the reasoning behind the compressed formats priority list.
  • Improve compressed format selection heuristics, in particular for mapping 1- and 2-channel data.
  • Improve OpenGL format fuzzy matching heuristics. In particular, teach it to differentiate between classes of formats like unorm, snorm, float, etc.
  • Cache transcoded textures on disk to make subsequent loads fast.
  • Write some tooling to make creating .basis textures more straightforward.
  • Test under GLES 3 and GLES 2, ANGLE.
    • GLES 3, Mesa, Linux
    • GLES 3, ANGLE over OpenGL, Linux
    • GLES 3, ANGLE over D3D11, Windows
    • GLES 2, ANGLE over OpenGL, Linux — works but swizzles are missing
    • ANGLE in WebGL compatibility mode
  • Make it compile with Emscripten.
    • And make sure it actually works.
    • Note: the relevant texture compression extensions appear to be broken in Chromium 84.0.4147.89 (they fail conformance tests). We could work around this by disabling mipmapping for compressed textures in Chromium browsers, but I'm hoping this will be resolved before release.
  • Add "grayscale+alpha" 2-channel mode.
  • Account for size limitations for compressed textures.
    • This may differ per implementation; the GLES/WebGL extensions seem more strict.
    • Implemented partially: assumptions about size requirements are hardcoded based on GLES requirements for S3TC and BPTC textures.
  • Implement a workaround for missing texture swizzles in GLES2 and WebGL2.
  • Hunt down the source files for textures that are currently using lossy webp compression, consider converting them into .basis textures.
    • Mostly done; will probably land this separately.
  • Consider a low-quality build profile where all/most textures are compressed (might be useful for WebGL).
  • Write authoring documentation
    • Intro
    • First-time setup
    • TL;DR examples
    • In-depth encoding nuances
    • Caveats and limitations
  • Probably more stuff that I forgot about.

Sorry, something went wrong.

@Akaricchi Akaricchi added this to the v1.4 milestone Jul 24, 2020
@Akaricchi Akaricchi marked this pull request as ready for review August 13, 2020 14:32
@Akaricchi Akaricchi changed the base branch from stage4bg to master August 14, 2020 19:37
@Akaricchi Akaricchi merged commit a5fd6fe into master Aug 15, 2020
@Akaricchi Akaricchi mentioned this pull request Sep 1, 2020
@@ -495,3 +509,22 @@ uint64_t umuldiv64(uint64_t x, uint64_t multiplier, uint64_t divisor) {
uint64_t uceildiv64(uint64_t x, uint64_t y) {
return x / y + (x % y != 0);
}

int popcnt32(uint32_t x) {
#if TAISEI_BUILDCONF_HAVE_BUILTIN_POPCOUNT

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seems to be an inconsistency here between how this is defined by meson (without a value) in build_config.h, and by src/util/compat.h, which can lead to build failures (e.g. https://travis-ci.com/github/jon-turney/meson-corpus-test/jobs/384217603#L4514)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thanks; should be fixed in 0e25c3a. But I do have to wonder how you managed to trigger this condition in the first place - does the compiler really not have __builtin_popcount, or is that a Meson bug?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The compiler I tested with does have __builtin_popcount.

I think this issue here is that the behaviour of meson when using the set() method of a configuration data object with a boolean value is not what you are expecting.

https://mesonbuild.com/Configuration.html#configuring-without-an-input-file

Perhaps you want to use set10 here instead, depending on what you've decided the meaning of this define is?

https://mesonbuild.com/Reference-manual.html#configuration-data-object

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm aware of how the set method works, that's what I want. I just accidentally wrote #if in place of #ifdef.

Disregard my previous comment by the way; I just realized that the error happens when __builtin_popcount is found, since the macro just expands into an empty space. It's strange that neither GCC (10.2.1) nor clang (10.0.1) produce this error on my system however, otherwise I definitely should've caught this...

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FTR, this is a Ubuntu 20.04 (focal) VM, and the compiler is gcc (Ubuntu 9.3.0-3ubuntu1) 9.3.0.

@Akaricchi Akaricchi deleted the texture_madness branch October 8, 2020 05:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants