Creating a Realistic 3D Clothing Model with Fabric Physics for Interaction in Apple Vision Pro

Hi @MohammadM

Here's high level guidance to point you in the right direction. This is not trivial to do so if you'd like an article on the subject or an API to make it easier to simulate cloth, please file an enhancement request using feedback assistant.

One approach to creating cloth is a 2 step process:

Create a grid of entities connected by springs. Specifically, use 3 sets of springs (this is not possible today, see the explanation that follows) and:

• connect every node to four neighbors at distance 1 in the x and y direction, this resists stretching/squashing
• connect every node to four neighbors at distance 2 in the x and y direction, this resists folding
• connect every node to four diagonal neighbors at distance 1 in the xy directions, this resists shear

Next create a plane using LowLevelMesh that has a vertex for each entity. Write a system to sync the vertices' transform to match their corresponding entity and recompute the vertex normals. You will need to write a metal shader for this.

That said, RealityKit does not have springs. People have had moderate success connecting the entities with PhysicsSphericalJoint along the x and y axis. The entities need collision shapes and physics bodies. Here is some code to show you the physics/collision/size properties that I've seen work:

let entity = Entity()

// use smaller spheres
let shape = ShapeResource.generateSphere(radius: 0.013)
entity.components.set(CollisionComponent(shapes: [shape], mode: .trigger))

var physicsBodyMode:PhysicsBodyMode = .dynamic
let physicsBodyComponent = PhysicsBodyComponent(shapes: [shape],
                          mass: 0.8,
                          material: PhysicsMaterialResource.generate(friction: 0.4, restitution: 0.3),
                              mode: .dynamic)

entity.components.set(physicsBodyComponent)

The largest grid I've seen was work was 20x20 entities.

Finally there are other was to do this, but this is one approach I've seen work without requiring in depth knowledge of physics. For example, you could use Entity/Component/System to create your own physics engine / springs.