Hi, I'm trying to personalize the Detect animal poses in Vision example (WWDC 23).
After some tests I saw that the landmarks and connection drawings work only if I do not ignore the safe area, if I ignore it (removing the toggle) or use the app on the iPad the drawings are no longer applied correctly.
In the example GeometryReader is used to detect the size of the view:
...
ZStack {
GeometryReader { geo in
AnimalSkeletonView(animalJoint: animalJoint, size: geo.size)
}
}.frame(maxWidth: .infinity)
...
struct AnimalSkeletonView: View {
// Get the animal joint locations.
@StateObject var animalJoint = AnimalPoseDetector()
var size: CGSize
var body: some View {
DisplayView(animalJoint: animalJoint)
if animalJoint.animalBodyParts.isEmpty == false {
// Draw the skeleton of the animal.
// Iterate over all recognized points and connect the joints.
ZStack {
ZStack {
// left head
if let nose = animalJoint.animalBodyParts[.nose] {
if let leftEye = animalJoint.animalBodyParts[.leftEye] {
Line(points: [nose.location, leftEye.location], size: size)
.stroke(lineWidth: 5.0)
.fill(Color.orange)
}
}
...
}
}
}
}
}
// Create a transform that converts the pose's normalized point.
struct Line: Shape {
var points: [CGPoint]
var size: CGSize
func path(in rect: CGRect) -> Path {
let pointTransform: CGAffineTransform =
.identity
.translatedBy(x: 0.0, y: -1.0)
.concatenating(.identity.scaledBy(x: 1.0, y: -1.0))
.concatenating(.identity.scaledBy(x: size.width, y: size.height))
var path = Path()
path.move(to: points[0])
for point in points {
path.addLine(to: point)
}
return path.applying(pointTransform)
}
}
Looking online I saw that it was recommended to change the property cameraView.previewLayer.videoGravity
from:
cameraView.previewLayer.videoGravity = .resizeAspectFill
to:
cameraView.previewLayer.videoGravity = .resizeAspect
but it doesn't work for me.
Could you help me understand where I'm wrong?
Thanks!
Setting the video gravity to resizeAspect preserves the video's aspect ratio, causing the video to have some extra padding along the sides when displayed in a view with a wider aspect ratio. This extra padding is not being taken into account when rendering the skeleton, which is being drawn over the full view.
One solution is to change the video gravity to resize, which will remove the extra padding.
cameraView.previewLayer.videoGravity = .resize
However, this will alter the aspect ratio of the camera preview, potentially stretching the video. If you wish to preserve the video's original aspect ratio while rendering the skeleton in the correct place, pass the video's aspect ratio along to the GeometryReader.
DisplayView(animalJoint: animalJoint)
.overlay {
GeometryReader { geo in
// draw skeleton
}.aspectRatio(..., contentMode: .fit)
}
}
This constrains the area in which the skeleton is drawn to match the bounds of the video and not its padding.
