1. Capabilities
2. Setup for Desktop
3. Setup for Android
4. Debugger Usage
5. Architecture Overview
6. C++ Server
7. JavaScript Client
8. HTTP Requests
9. WebSocket Messages
10. Wish List
11. Screenshot
12. Material Chunks

matdbg is a library and web application that enables debugging and live-editing of Filament shaders. At the time of this writing, the following capabilities are supported.

• OpenGL: Editing GLSL
• Metal: Editing MSL
• Vulkan: Editing transpiled GLSL, displaying disassembled SPIR-V

Note that a given material can be built with multiple backends, even though only one backend is active in a particular session. For example, if the current app is using Vulkan, it is still possible to inspect the Metal shaders, as long as the material has been built with Metal support included.

When using the easy build script, include the -d argument. For example:

./build.sh -fd debug gltf_viewer

The d enables a CMake option called FILAMENT_ENABLE_MATDBG and the f ensures that CMake gets re-run so that the option is honored.

Next, set an environment variable as follows. In Windows, use set instead of export.

export FILAMENT_MATDBG_PORT=8080

Next, launch any app that links against a debug build of a Filament and point your web browser to http://localhost:8080. Skip ahead to Debugger Usage.

Rebuild Filament for Android after enabling a CMake option called FILAMENT_ENABLE_MATDBG. Note that CMake is invoked from several places for Android (both gradle and our easy build script), so one pragmatic and reliable way of doing this is to simply hack CMakeLists.txt and filament-android/CMakeLists.txt by unconditionally setting FILAMENT_ENABLE_MATDBG to ON.

After rebuilding Filament with the option enabled, ensure that internet permissions are enabled in your app by adding the following into your manifest as a child of the <manifest> element.

<uses-permission android:name="android.permission.INTERNET" />

Now launch your app as usual. The Filament Engine sets up a server that is hardcoded to listen to port 8081. Next, you will need to forward your device's TCP port 8081 to your host port of choice. For example, to forward the matdbg server on your device to port 8081 on your host machine, do the following:

adb forward tcp:8081 tcp:8081

This lets you go to http://localhost:8081 in Chrome on your host machine.

Note that we generally use a release build of Filament when running on Android, so the shaders are optimized and very unreadable. This can be avoided by modifying the build such that -g is passed to matc even in release builds.

After opening the matdbg page in your browser, the usual first step is to select a material in the upper-left pane. Sometimes you might need force your app to redraw (e.g. by resizing the window) in order make the materials selectable.

The next step is to select an active (boldface) shader variant in the lower-left pane. This allows you to view the GLSL, MSL, and SPIR-V code that was generated by matc or filamat.

In the sidebar, inactive shader variants have a disabled appearance, but they can still be examined in the shader editor. The active status of each shader program is refreshed every second.

You can also make modifications to GLSL or MSL, so long as the shader inputs and uniforms remain intact. After making an edit, click the [rebuild] button in the header. Note that your edits will be lost after closing the web page.

To save an edit, press Cmd+S (Ctrl+S on Linux/Windows) as an alternative to clicking [rebuild].

If the editor has focus, you can navigate between materials by holding Shift+Ctrl while pressing the up or down arrow. Navigation between variants is similar, just use left / right instead of up / down.

The matdbg library has two parts: a C++ server and a JavaScript client. The C++ server is responsible for instancing a civetweb context that handles HTTP and WebSocket requests. The JavaScript client is a small web app that contains a view into an in-browser database of materials.

The WebSocket server receives push-style notifications from the client (such as edits) while the HTTP server responds to material queries using simple JSON messages.

When a new WebSocket connection is established, the client asks the server for a list of materials in order to populate its in-browser database. If the connection is lost (e.g. if the app crashes), then the database stays intact and the web app is still functional. If a new Filament app is launched, the client inserts entries into its database rather than replacing the existing set.

The material database is cleared only when the web page is manually refreshed by the user.

The civetweb server is wrapped by our DebugServer class, whose public interface is comprised of a couple methods that are called from the Filament engine:

• addMaterial Notifies the debugger that the given material package is being loaded into the engine.
• setEditCallback Sets up a callback that allows the Filament engine to listen for shader edits.
• setQueryCallback Sets up a callback that allows the debugger to ask for current information.

The web app is written in simple, modern JavaScript and avoids frameworks like React or Angular. It uses the following third-party libraries which are fetched from a CDN using <script>. This allows us to avoid adding them to our git repo, and leads to good caching behavior.

• mustache Popular tiny library that converts template strings into HTML.
• monaco The engine behind Visual Studio Code.
  • We've configured this for C++ for somewhat reasonable syntax highlighting.
  • If desired we could extend the editor to better handle GLSL and SPIR-V.

All the source code for our web app is contained in a single file (script.js) and the mustache template strings are specified using <template> tags in index.html.

The web app basically provides a view over a pseudo-database which is a just a global variable that holds a dictionary that maps from material id's to objects that conform to the JSON described below.

The server responds to the following GET requests by returning a JSON blob. The {id} in these requests is a concept specific to matdbg (not Filament) which is an 8-digit hex string that hashes the entire binary content of the material package.

/api/matids

Returns an array containing the id for each known material. Example:

["e4c41141", "44ae2b62", "9dab8a03"]

/api/materials

Returns an array with all information (except shader source) for all known materials. Example:

[{
    "matid": "e4c41141",
    "name": "uiBlit",
    "version": 4,
    "shading": { "model": "unlit", "vertex_domain": "object", ... },
    "raster":  { "blending": "transparent", "color_write": "true", ... },
    "opengl": [
        { "index": " 0", "shaderModel": "gl41", "pipelineStage": "vertex  ", "variantString": "", "variant": 0 },
        { "index": " 1", "shaderModel": "gl41", "pipelineStage": "fragment", "variantString": "", "variant": 0 },
    ],
    "vulkan": [],
    "metal": [],
    "required_attributes": ["position", "color", "uv0"]
},
{
    "matid": "44ae2b62",
    ...
}]

Some of the returned data may seem redundant (e.g. the index and variantString fields) but these allow the client to be very simple by passing the raw JSON into mustache templates. Moreover it helps prevent duplication of knowledge between C++ and JavaScript.

This format of this message is also used for the in-browser "database" of materials.

/api/material?matid={id}

Returns all information (except shader source) for a specific known material. The JSON response is equivalent to one of the items in the top-level array in /api/materials.

/api/active

Returns an object that maps from material ids to their active shader variants. Example:

{"b38d4ad0": ["opengl", 5] , "44ae2b62": ["opengl", 1, 4] }

Each numeric element in the list is a variant mask. For example, at the time of this writing, Filament has 7-bit mask, so each number in the list is between 0 and 127.

/api/shader?matid={id}&type=[glsl|spirv|msl]&[glindex|vkindex|metalindex]={index}

Returns the entire shader code for the given variant. This is the only HTTP request that returns text instead of JSON.

The type field in the request selects the desired shading language, not the backend. For example, for Vulkan it can select between SPIR-V or decompiled GLSL. Note that the original GLSL that was used to create the SPIR-V is not available.

Unlike HTTP requests, WebSocket messages can be pushed at any time and can travel in either direction. In our homegrown protocol, every WebSocket message starts with a command that matches [A-Z_]+ followed by a space character. Command arguments are delimited with spaces.

Currently we support only one command. It travels from client to server.

EDIT [material id] [api index] [shader index] [shader length] [entire shader source....]

The material id is 8 hex digits.

The api index chooses between GL/VK/Metal and matches the values of the Backend enum (except that zero is invalid).

The shader index is a zero-based index into the list of variants using the order that they appear in the package, where each API (GL / VK / Metal) has its own list.

The shader length is the number of bytes required for UTF-8 encoding of the shader source string, not including the terminating null.

• Allow editing of the original GLSL, perhaps by enhancing the -g option in matc and adding new chunk types.
• Port the web side to TypeScript
  • This will clarify the structure of the pseudo-database, which is currently a total hack.
  • Allows us to use enums instead of strings in several places (e.g. getShaderAPI)
  • Try using https://github.com/basarat/typescript-script because webpack etc is painful.
  • If the above idea is too slow then use https://github.com/evanw/esbuild.
• Expose the entire engine.debug struct in the web UI.
• When shader errors occur, send them back over the wire to the web client.
• Resizing the Chrome window causes layout issues.
• The sidebar in the web app is not resizeable.
• For the material ids, SHA-1 would be better than murmur since the latter can easily have collisions.
• It would be easy to add diff decorations to the editor in our onEdit function:
  1. Examine "changes" (IModelContentChange) to get a set of line numbers.
  2. shader.decorations = gEditor.deltaDecorations(shader.decorations, ...)
  3. See these monaco docs.

This section exists only to provide a reference for the ShaderExtractor and ShaderReplacer features.

The relevant chunk types are listed here. These types are defined in the filabridge lib, in the filamat namespace.

enum UTILS_PUBLIC ChunkType : uint64_t {
    ...
    MaterialGlsl = charTo64bitNum("MAT_GLSL"),    // MaterialTextChunk
    MaterialSpirv = charTo64bitNum("MAT_SPIR"),   // MaterialSpirvChunk
    MaterialMetal = charTo64bitNum("MAT_METL"),   // MaterialTextChunk
    ...
    DictionaryGlsl = charTo64bitNum("DIC_GLSL"),  // DictionaryTextChunk
    DictionarySpirv = charTo64bitNum("DIC_SPIR"), // DictionarySpirvChunk
    DictionaryMetal = charTo64bitNum("DIC_METL"), // DictionaryTextChunk
    ...
}

These chunks have the following layout.

[u64] ChunkType magic string
[u32] Remaining chunk size in bytes
[u64] Shader count
for each shader:
    [u8]  Shader model
    [u8]  Shader variant
    [u8]  Shader stage
    [u32] Offset in bytes from (and including) "Shader count" up to "Total string size"
for each unique shader:
    [u32] Total string size (including null terminator)
    [u32] Number of line indices
    [u16 u16 u16...] Line indices

These chunks have the following layout.

[u64] ChunkType magic string
[u32] Remaining chunk size in bytes
[u64] Shader count
for each shader:
    [u8]  Shader model
    [u8]  Shader variant
    [u8]  Shader stage
    [u32] Index into the blob list in DictionarySpirvChunk

These chunks have the following layout.

[u64] ChunkType magic string
[u32] Remaining chunk size in bytes
[u32] Number of strings
for each string:
    [u8 u8 u8 u8...] include null terminator after each string

These chunks have the following layout.

[u64] ChunkType magic string
[u32] Remaining chunk size in bytes
[u32] Compression
[u32] Blob count
for each blob:
    [u64] Byte count
    [u8 u8 u8 ...]