Don't understand RealityKit's entity.move in visionOS immersive space

The set up:

I am developing a visionOS app that uses an immersive space. The user sees a board with entities put onto it. My app places the board in front of the default camera and entities with a certain position and orientation relative to the board. Placement and rotation should be animated.

The problem:

If I place the entities by assigning a Transform to the transform property of the entity directly, i.e. without animation, the result is correct. However I have to use the entity's move(to: function to animate it. And move(to: works in an unexpected way. I thus wrote a little test app, based on Apple's visionOS immersive app template (below). There, the following 5 cases are treated:

  1. Set transform directly (without animation). This gives the correct result, and works as expected (without animation).
  2. Set transform using move relative to world (without animation). This gives the correct result, although it does not work as expected. I expected "relative to world" means translation and rotation is relativ to world. This seems wrong for translation and right for rotation.
  3. Set transform using move relative to parentEntity (without animation). This gives a wrong result, although translation and rotation are defined relative to the parentEntity.
  4. Set transform using move relative to world with animation. This gives also a wrong result, and without animation.
  5. Set transform using move relative to parentEntity with animation. This gives also a wrong result, and without animation.

Here are the screen shots for the cases 1...5:

Cases 1 & 2

Case 3

Cases 4 & 5

The question:

So, obviously, I don't understand what move(to: does. I would be happy to get any advice what is wrong and how to do it right.

Here is the code:

import SwiftUI
import RealityKit
import RealityKitContent

struct ImmersiveView: View {
    @Environment(AppModel.self) var appModel
    
    let boardHeight: Float = 0.1
    let boxHeight: Float = 0.3

    var body: some View {
        RealityView { content in
            let boardEntity = makeBoard()
            content.add(boardEntity)
            let boxEntity = makeBox(parentEntity: boardEntity)
            boardEntity.addChild(boxEntity)
        }
    }
    
    func makeBoard() -> ModelEntity {
        let mesh = MeshResource.generateBox(width: 1.0, height: boardHeight, depth: 1.0)
        var material = UnlitMaterial(); material.color.tint = .red
        let boardEntity = ModelEntity(mesh: mesh, materials: [material])
        boardEntity.transform.translation = [0, 0, -3]
        return boardEntity
    }
    
    func makeBox(parentEntity: Entity) -> ModelEntity {
        let mesh = MeshResource.generateBox(width: 0.3, height: boxHeight, depth: 0.3)
        var material = UnlitMaterial(); material.color.tint = .green
        let boxEntity = ModelEntity(mesh: mesh, materials: [material])
        
        // Set position and orientation of the box
        
        // To put the box onto the board, move it up by half height of the board and half height of the box
        let y_up = boardHeight/2.0 + boxHeight/2.0
        let translation = SIMD3<Float>(0, y_up, 0)
        // Turn the box by 45 degrees around the y axis
        let rotationY = simd_quatf(angle: Float(45.0 * .pi/180.0), axis: SIMD3(x: 0, y: 1, z: 0))
        let transform = Transform(rotation: rotationY, translation: translation)
        
        // Do the actual move
        
        // 1) Set transform directly (without animation)
        boxEntity.transform = transform // Translation and rotation correct
        
        // 2) Set transform using move relative to world (without animation)
//      boxEntity.move(to: transform, relativeTo: nil) // Translation and rotation correct
        
        // 3) Set transform using move relative to parentEntity (without animation)
//      boxEntity.move(to: transform, relativeTo: parentEntity) // Translation incorrect, rotation correct
        
        // 4) Set transform using move relative to world with animation
//      boxEntity.move(to: transform,
//                     relativeTo: nil,
//                     duration: 1.0,
//                     timingFunction: .linear) // Translation incorrect, rotation incorrect, no animation
        
        // 5) Set transform using move relative to parentEntity with animation
//      boxEntity.move(to: transform,
//                     relativeTo: parentEntity,
//                     duration: 1.0,
//                     timingFunction: .linear) // 5) Translation incorrect, rotation incorrect, no animation

        return boxEntity
    }

}

Hi @reinermaenner

Thanks for the detailed question and sample code. Posts like yours help us easily recreate and troubleshoot questions!

This may be a bug, but our teams will need to further investigate. Please consider filing a bug using Feedback Assistant. I think it may be related to triggering the animation during scene setup, but can't say for certain. In the meantime, I was able to work around the issue, by invoking the animation in a task after the parent child relationships are established. Here's the revised code.

import SwiftUI
import RealityKit
import RealityKitContent

struct ImmersiveView: View {
    @Environment(AppModel.self) var appModel
    
    let boardHeight: Float = 0.1
    let boxHeight: Float = 0.3

    var body: some View {
        RealityView { content in
            let boardEntity = makeBoard()
            content.add(boardEntity)
            let mesh = MeshResource.generateBox(width: 0.3, height: boxHeight, depth: 0.3)
            var material = UnlitMaterial(); material.color.tint = .green
            let boxEntity = ModelEntity(mesh: mesh, materials: [material])
            boardEntity.addChild(boxEntity)
            
            Task {
                moveBox(boxEntity, parentEntity: boardEntity)
            }
        }
    }
    
    func makeBoard() -> ModelEntity {
        let mesh = MeshResource.generateBox(width: 1.0, height: boardHeight, depth: 1.0)
        var material = UnlitMaterial(); material.color.tint = .red
        let boardEntity = ModelEntity(mesh: mesh, materials: [material])
        boardEntity.transform.translation = [0, 0, -3]
        return boardEntity
    }
    
    func moveBox(_ boxEntity:Entity, parentEntity: Entity)  {
       
        
        // Set position and orientation of the box
        
        // To put the box onto the board, move it up by half height of the board and half height of the box
        let y_up = boardHeight/2.0 + boxHeight/2.0
        let translation = SIMD3<Float>(0, y_up, 0)
        // Turn the box by 45 degrees around the y axis
        let rotationY = simd_quatf(angle: Float(45.0 * .pi/180.0), axis: SIMD3(x: 0, y: 1, z: 0))
        let transform = Transform(rotation: rotationY, translation: translation)
        
        // Do the actual move
        
        // 1) Set transform directly (without animation)
//        boxEntity.transform = transform // Translation and rotation correct
        
        // 2) Set transform using move relative to world (without animation)
//      boxEntity.move(to: transform, relativeTo: nil) // Translation and rotation correct
        
        // 3) Set transform using move relative to parentEntity (without animation)
//      boxEntity.move(to: transform, relativeTo: parentEntity) // Translation incorrect, rotation correct
        
        // 4) Set transform using move relative to world with animation
//      boxEntity.move(to: transform,
//                     relativeTo: nil,
//                     duration: 1.0,
//                     timingFunction: .linear) // Translation incorrect, rotation incorrect, no animation
        
        // 5) Set transform using move relative to parentEntity with animation
        boxEntity.move(to: transform,
                       relativeTo: parentEntity,
                       duration: 1.0,
                       timingFunction: .linear) // 5) Translation incorrect, rotation incorrect, no animation
    }

}

Thanks for confirming the problem, and for the workaround. I filed a bug report (FB15152333).

Don't understand RealityKit's entity.move in visionOS immersive space
 
 
Q