Geometry recognition and measurement from MeshAnchor

Spatial Computing ARKit
JoonAhn OP
Created Jun ’24
Replies 5
Boosts 0
Views 647
Participants 1

FYI.

The source code of the FindSurface demo app for Apple Vision Pro (visionOS) is available now. The Swift package of FindSurface™ library is required to build the source code into the demo app. https://github.com/CurvSurf/FindSurface-visionOS

After starting the app, the floating panels (below) will appear on your right side, and you will see wireframe meshes that approximately describe your environments. Performing a spatial tap (pinching with your thumb and index finger) with staring at a location on the meshes will invoke FindSurface, with an indicator (blue disk) appearing on the surface you've gazed.

Voice commands:

  • “Tap” – Spatial tap (gazing & pinching). Invoke FindSurface.
  • “Tap plane” – Plane selection.
  • “Tap sphere” or “Tap ball” – Sphere selection.
  • “Tap cylinder” – Cylinder selection.
  • “Tap cone” – Cone selection.
  • “Tap torus” or “Tap donut” – Torus selection.
  • “Tap accuracy” or “Tap measurement accuracy” – Accuracy selection.
  • “Tap mean distance”, “Tap average distance”, or “Tap distance” – Avg. Distance selection.
  • “Tap touch radius” or “Tap seed radius” – Touch Radius selection.
  • “Tap Inlier” – “Show inlier points” toggle.
  • “Tap outline” – “Show geometry outline” toggle.
  • “Tap clear” – “Clear Scene” click.
Replies  5
Boosts  0
Views  647
Participants  1
JoonAhn OP
Jun ’24

Voice commands:

  • “Tap” – Spatial tap (gazing & pinching). Invoke FindSurface.
  • “Tap plane” – Plane selection.
  • “Tap sphere” or “Tap ball” – Sphere selection.
  • “Tap cylinder” – Cylinder selection.
  • “Tap cone” – Cone selection.
  • “Tap torus” or “Tap donut” – Torus selection.
  • “Tap accuracy” or “Tap measurement accuracy” – Accuracy selection.
  • “Tap mean distance”, “Tap average distance”, or “Tap distance” – Avg. Distance selection.
  • “Tap touch radius” or “Tap seed radius” – Touch Radius selection.
  • “Tap Inlier” – “Show inlier points” toggle.
  • “Tap outline” – “Show geometry outline” toggle.
  • “Tap clear” – “Clear Scene” click.
0
JoonAhn OP
Jul ’24

https://github.com/CurvSurf/FindSurface-RealityKit-visionOS

Once object measurement points (point cloud, mesh vertices, etc.) are collected, the measurement accuracy and average distance between points are fixed.

CurvSurf FindSurface works based on those values ​​and requires additional parameter values ​​for region growing to enlarge the length/width and increase the radius of the estimated object geometry.

After a series of experiments, CurvSurf found the optimal parameter value set for MeshAnchor generated by Apple Vision Pro:

  • Accuracy: 1.5 cm,
  • Average Distance: 10 cm,
  • Lat. Ext.: 10,
  • Rad. Exp.: 5,
  • Touch Radius: 1/4 - 1/2 of object diameter/width [cm].

As an Apple Vision Pro user, you need to adjust the Touch Radius proportional to the approximate diameter or width of your object. It is a problem of object scaling for a range of 1 - 20 meters.

If you like to get a relatively small plane or short cylinder/cone, you set the Lat. Ext. to less than 5.

0
JoonAhn OP
Jul ’24

The FindSurface demo app for Apple Vision Pro (visionOS) is now stabilized: https://github.com/CurvSurf/FindSurface-RealityKit-visionOS

  • Cone2Cylinder, Torus2Cylinder, and Torus2Sphere conversions are implemented. For example, if you try to get a cone from a real cylindrical object (with a diameter of larger than 1 meter), FindSurface will convert the resulting cone to a cylinder.
  • Cones and tori are trimmed to the boundaries of the inlier points.
0
JoonAhn OP
Aug ’24

Real-time preview for object geometry detection and measurement from MeshAnchors. Object planes, spheres, cylinders, cones, and tori can be detected and measured in real time up to 90 Hz. The source code of the app will be available on GitHub CurvSurf.

FindSurface Real-Time Preview - Apple Vision Pro https://youtu.be/CGjhfKxjpUU

0
JoonAhn OP
Aug ’24

The source code of the AVP app will be available on GitHub CurvSurf in the week of Sept. 2, 2024.

FindSurface Real-Time 1 - Apple Vision Pro

https://youtu.be/2aSMBrPTEtg

0
Geometry recognition and measurement from MeshAnchor
First post date Last post date
 
 
Q