LowLevelMesh: Triangle Colors

I am trying to follow the documentation with the beta version of visionOS with the new realitykit LowLevelMesh construct (https://developer.apple.com/documentation/realitykit/lowlevelmesh) that draws a triangle. Although the code indicates different colors for each of the 3 vertex points, the triangle renders in white.

I believe that the missing link may be a shadergraph material, but because I will be drawing millions of triangles, with colors defined at the nodes and interpolated over the area of the triangles, I want to make sure it is efficient, either with shadergraph materials or perhaps metal.

I have, with an earlier version of the app I'm working on, successfully used a shadergraph material with MeshDescriptor.primatives as polygons for tetrahedrons. However, that is inefficient for more than 1,000 tetrahedrons (and crashes) so I'm trying to use the new LowLevelMesh instead (with each tetrahedron split into 4 triangles). However, I can't get very far using the example code from the documentation (that results in the white triangles), even trying to use the default shadergraph (GridMaterial) without getting quite a few error messages. I try to fix the errors with the suggested fixes and then get new ones (whack-a-mole) until it's seems to be all broken.... So in addition to my general question of shadergraph vs metal for a LowLevelMesh, a concrete example of using a shadergraph material with LowLevelMesh would be most appreciated! Thanks.

Answered by Vision Pro Engineer in 795668022

Hi @rthart,

You're correct, the missing link required in order to get the triangle to display color is a ShaderGraphMaterial.

Start from a project that creates a triangle using the code provided in the documentation for LowLevelMesh, then apply a ShaderGraphMaterial to the triangle that will display its vertex colors using the following steps:

First, create a new Shader Graph Material in Reality Composer Pro that applies the Geometry Color node to the color input of the surface. In order to only visualize the vertex colors of the triangle, I would suggest doing this using an Unlit Surface node which ignores lighting.

In the Shader Graph editor:

  1. Add an Unlit Surface node
  2. Add a Geometry Color node
  3. Connect the output of the Geometry Color node to the "Color" input of the Unlit Surface node
  4. Connect the output of the Unlit Surface node to the "Custom Surface" input of the Outputs node

Note: If you would like your material to be affected by the lighting in your scene, use a Preview Surface or PBR Surface node in place of the Unlit Surface node above.

Note: Another way to apply vertex colors to your material is to use the Geometric Property node, which grants you access to all of the custom vertex attributes you have specified for your LowLevelMesh such as uvs, colors, normals, positions, and so on. To use the Geometric Property node to access vertex color information, set its "Type" to Geometric Property (color3f) and its "Geomprop" to GeomColor_Float before connecting it to the color input of the surface.

Next, load the ShaderGraphMaterial created above and add a triangle entity with that material applied to the scene. In my case, I define the material in a scene named "TriangleMaterial" in the "Materials" folder in Reality Composer Pro, where within that scene the actual material itself resides in the Hierarchy under "Root/TriangleMaterial". Use the name and path of the material you created to avoid any errors.

RealityView { content in
    do {
        // Load the custom ShaderGraphMaterial
        let material = try await ShaderGraphMaterial(named: "/Root/TriangleMaterial",
                                                     from: "Materials/TriangleMaterial",
                                                     in: realityKitContentBundle)
        
        // Create the triangle and apply the material to its model component
        let triangle = try triangleEntity()
        triangle.components[ModelComponent.self]?.materials = [material]
        
        // Add the triangle to the scene
        content.add(triangle)
    } catch {
        print("Unexpected error while creating triangle: \(error)")
    }
}

Finally, run the app and the triangle will now display its vertex colors!

In general, Shader Graph should play quite nicely with any LowLevelMesh you create, as all of the vertex attributes of your mesh (color, position, normal, tangent, bitangent, uv0, uv1, etc.) are accessible in Shader Graph via the Geometric Property node. Common attributes are also made accessible through their own individual nodes such as Geometry Color, Position, Normal, etc.

Accepted Answer

Hi @rthart,

You're correct, the missing link required in order to get the triangle to display color is a ShaderGraphMaterial.

Start from a project that creates a triangle using the code provided in the documentation for LowLevelMesh, then apply a ShaderGraphMaterial to the triangle that will display its vertex colors using the following steps:

First, create a new Shader Graph Material in Reality Composer Pro that applies the Geometry Color node to the color input of the surface. In order to only visualize the vertex colors of the triangle, I would suggest doing this using an Unlit Surface node which ignores lighting.

In the Shader Graph editor:

  1. Add an Unlit Surface node
  2. Add a Geometry Color node
  3. Connect the output of the Geometry Color node to the "Color" input of the Unlit Surface node
  4. Connect the output of the Unlit Surface node to the "Custom Surface" input of the Outputs node

Note: If you would like your material to be affected by the lighting in your scene, use a Preview Surface or PBR Surface node in place of the Unlit Surface node above.

Note: Another way to apply vertex colors to your material is to use the Geometric Property node, which grants you access to all of the custom vertex attributes you have specified for your LowLevelMesh such as uvs, colors, normals, positions, and so on. To use the Geometric Property node to access vertex color information, set its "Type" to Geometric Property (color3f) and its "Geomprop" to GeomColor_Float before connecting it to the color input of the surface.

Next, load the ShaderGraphMaterial created above and add a triangle entity with that material applied to the scene. In my case, I define the material in a scene named "TriangleMaterial" in the "Materials" folder in Reality Composer Pro, where within that scene the actual material itself resides in the Hierarchy under "Root/TriangleMaterial". Use the name and path of the material you created to avoid any errors.

RealityView { content in
    do {
        // Load the custom ShaderGraphMaterial
        let material = try await ShaderGraphMaterial(named: "/Root/TriangleMaterial",
                                                     from: "Materials/TriangleMaterial",
                                                     in: realityKitContentBundle)
        
        // Create the triangle and apply the material to its model component
        let triangle = try triangleEntity()
        triangle.components[ModelComponent.self]?.materials = [material]
        
        // Add the triangle to the scene
        content.add(triangle)
    } catch {
        print("Unexpected error while creating triangle: \(error)")
    }
}

Finally, run the app and the triangle will now display its vertex colors!

In general, Shader Graph should play quite nicely with any LowLevelMesh you create, as all of the vertex attributes of your mesh (color, position, normal, tangent, bitangent, uv0, uv1, etc.) are accessible in Shader Graph via the Geometric Property node. Common attributes are also made accessible through their own individual nodes such as Geometry Color, Position, Normal, etc.

Thank you, this is perfect! (It should be included in the documentation...) As a followup, will the use of the shader graph material be as efficient as metal for millions of triangles? If not, could you provide an example of how to do the same thing, starting with the example in the documentation, and blending the vertex colors over the face of the triangle, but with metal instead of the shader graph?

Hi @rthart,

LowLevelMesh should have very little overhead over rendering directly with Metal. If you find a case where rendering with Metal is more efficient than using LowLevelMesh, please provide feedback via https://feedbackassistant.apple.com.

LowLevelMesh: Triangle Colors
 
 
Q