How to set followsUserLocation and define a custom camera distance dynamically using MapKit for SwiftUI ?

Hello, I've created an app that follows the user as they navigate through public transport.

I want the camera to follow the user and at the same time I want the camera distance (elevation) to change dynamically depending on speed and other factors.

I've tried a first approach using camera keyframes, but I've noticed a lot of crashes when the app comes back to the foreground.

.mapCameraKeyframeAnimator(trigger: cameraFrame) { camera in
            KeyframeTrack(\MapCamera.centerCoordinate) {
                LinearKeyframe(cameraFrame.center, duration: cameraFrame.duration, timingCurve: cameraFrame.curve)
            }
            
            KeyframeTrack(\MapCamera.distance) {
                LinearKeyframe(cameraFrame.distance, duration: cameraFrame.duration, timingCurve: cameraFrame.curve)
            }
        }

Some logs mention wrong center coordinates (nan, nan) but when I print them, it's show me valid coordinates.

I've also tried manually setting the camera for each position update, but the result is not smooth.

position = .camera(
                .init(.init(
                    lookingAtCenter: newPosition,
                    fromDistance: cameraElevation,
                    pitch: .zero,
                    heading: .zero)
                )
            )

What's the best way to achieve this?