Hi
I would like to manipulate blend shapes of a usdz file in VisionOS, but I can not figure out how I could do that. In SceneKit it was a piece of cake with SCNNode and morphs, but with Entity class I got stuck.
is it even possible? (I would be very disappointed if not)
RealityKit doesn't currently support blendshapes which is stated here: https://developer.apple.com/documentation/realitykit/validating-usd-files#Review-skeletons
Currently, it would only be possible to either use a custom geometry modifier to apply the vertex offsets interpolated over time through set_model_position_update or using a MeshResource update routine to transform the vertices. Most of my animations are procedural, so I haven't yet tried to get the blendshape offsets and weights from the USD file, which would then need to be uploaded through a uniform. There doesn't seem to have any exposed getter akin to jointTranforms.
https://developer.apple.com/documentation/realitykit/altering_realitykit_rendering_with_shader_functions https://developer.apple.com/documentation/realitykit/modifying-realitykit-rendering-using-custom-materials#Write-a-Geometry-Modifier https://developer.apple.com/videos/play/wwdc2021/10075 https://developer.apple.com/documentation/visionos/diorama