How to use KTX file Image3D in ShaderGraph?

I've tried importing a 3D KTX file into Reality Composer Pro ShaderGraph, but got nothing output. Realitykit doesn't support Image3D yet?

First, I use pyktx to create a 3D KTX texture file and it seems correct in Finder preview.

import numpy as np
from pyktx import KtxTexture2, KtxTextureCreateInfo, KtxTextureCreateStorage, VkFormat

size = 32
image = np.zeros((size, size, size, 3), dtype=np.uint8)
for i in range(size):
    image[i, :, :, 0] = np.interp(i, [0, size], [50, 255])
    image[:, i, :, 1] = np.interp(i, [0, size], [100, 255])
    image[:, :, i, 2] = np.interp(i, [0, size], [150, 255])

info = KtxTextureCreateInfo(
    gl_internal_format=None,
    vk_format=VkFormat.VK_FORMAT_R8G8B8_SRGB,
    base_width=image.shape[2],
    base_height=image.shape[1],
    base_depth=image.shape[0],
    num_dimensions=3,
    num_levels=1,
    num_layers=1,
    num_faces=1,
    generate_mipmaps=False,
    is_array=False,
)

texture = KtxTexture2.create(info, KtxTextureCreateStorage.ALLOC)

for _ in range(1):
    texture.set_image_from_memory(0, 0, _, image[_].tobytes())

texture.write_to_named_file(f'{size}.ktx')

Then I import it into ShaderGraph like this but seems nothing could be read.

In addition, I tested 2D KTX file and it worked.

I also tested different vk_format or KTX1/KTX2 but did not work.

I also tested Image3DPixel/Image2DArray/Image3DRead and none of them work as long as it's 3D.

Answered by epw in 811223022

Temporarily solved by using TextureResource.init dimensions 3D with r16Float raw pixelFormat. Successfully set runtime dynamic volume texture parameter on ShaderGraphMaterial.

Typo: for _ in range(1) should be for _ in range(size)

Also tried create 3D texture from PNG images and set parameter on ShaderGraphMaterial. But still not work. Image3D/Image3DPixel output always white, Image3DRead output always black.

var images: [CGImage] = []
for i in 0..<32 {
    let image = UIImage(named: "\(i).png")!.cgImage!
    images.append(image)
}
let tex = try! await TextureResource.texture3D(slices: images, options: TextureResource.CreateOptions(semantic: .color))

let entity = scene.findEntity(named: "Slice")!
var material = entity.components[ModelComponent.self]!.materials.first as! ShaderGraphMaterial
try! material.setParameter(name: "Volume", value: .textureResource(tex))

So, likely the problem does not come from the KTX file, but Realitykit does not support Image3D yet?

Accepted Answer

Temporarily solved by using TextureResource.init dimensions 3D with r16Float raw pixelFormat. Successfully set runtime dynamic volume texture parameter on ShaderGraphMaterial.

How to use KTX file Image3D in ShaderGraph?
 
 
Q