I want to use Swift language to write code for drawing multiple polygons. I would like to find some examples as references. Can anyone provide example code or tell me where I can see such examples? Thank you!

I've been working with ARKit and Metal on the Vision Pro, but I've encountered a slight flickering issue with the mesh rendered using Metal. The flickering tends to occur around the edges of objects or on pixels with high color contrast, and it becomes more noticeable as the distance increases. Is there any way to resolve this issue?

Hello, I am trying to obtain the RealityKit URL. Can someone share please?

Technical Issue Report for Maple Tale App - Audio Format Compatibility Dear Apple Technical Support Team, I hope this message finds you well. My name is [Your Name], and I am part of the development team behind the Maple Tale app. We have encountered an issue with audio format compatibility within our app that we believe requires your assistance. The issue pertains to the audio formats supported by our app. Currently, our app only supports WAV and OGG formats, which has led to a limitation in user experience. We are looking to expand our support to include additional formats such as MP3 and AAC, which are widely used by our user base. To provide a clear understanding of the issue, I have outlined the steps to reproduce the problem: Launch the Maple Tale app. Proceed with the game normally. Upon picking up equipment within the game, a warning box pops up indicating the audio format compatibility issue. This warning box appears due to the app's inability to process audio files in formats other than WAV and OGG. We understand that this can be a significant hindrance to the user experience, and we are eager to resolve this as quickly as possible. We have reviewed the documentation available on the official Apple Developer website but are still seeking clarification on the best practices for supporting a wider range of audio formats within our app. We would greatly appreciate any official recommendations or guidelines that could assist us in this endeavor. Additionally, we are considering updating our app to inform users about the current audio format requirements and provide guidance on how to optimize their audio files for the best performance within our app. If there are any official documents or resources that we should reference when crafting this update, please let us know. We appreciate your time and assistance in this matter and look forward to your guidance on how to best implement audio format support on the iOS platform. Thank you once again for your support. Warm regards,

The update was the only change I can see. Which I did just the other day. I did log on to iCloud.com, I also looked at Apple Developer. I didn't see any additional updates of terms that needed to be accepted. I am sure to log into iCloud on the simulator. It seems to stay logged in. Until I fetchSavedGames. Where I have it exit at 20 seconds due to timing out. Then when I go back to check my account in Settings, it's asking to "Sign in to iCloud" again. It does work properly on a device. So it doesn't stay logged into iCloud on the simulator but it seems like the fetchSavedGame from GKLocalPlayer is what resets that. Any help or suggestions would be appreciated. Thanks.

Is there anywhere that the format of error/warning/other? messages that are generated by the Metal compiler are documented?

Hi, is there a way in visionOS to anchor an entity to the POV via RealityKit? I need an entity which is always fixed to the 'camera'. I'm aware that this is discouraged from a design perspective as it can be visually distracting. In my case though I want to use it to attach a fixed collider entity, so that the camera can collide with objects in the scene. Edit: ARView on iOS has a lot of very useful helper properties and functions like cameraTransform (https://developer.apple.com/documentation/realitykit/arview/cameratransform) How would I get this information on visionOS? RealityViews content does not seem offer anything comparable. An example use case would be that I would like to add an entity to the scene at my users eye-level, basically depending on their height. I found https://developer.apple.com/documentation/realitykit/realityrenderer which has an activeCamera property but so far it's unclear to me in which context RealityRenderer is used and how I could access it. Appreciate any hints, thanks!

Hey guys, is it possible to implement mirror like reflections like in this project: https://developer.apple.com/documentation/metal/metal_sample_code_library/rendering_reflections_in_real_time_using_ray_tracing for visionOS? Or is the hardware not prepared for Metal Raytracing? Thanks in advance

Hi, when I run my app, in console say: NSBundle file:///System/Library/PrivateFrameworks/MetalTools.framework/ principal class is nil because all fallbacks have failed

Given I do not understand much at all about how to write shaders I do not understand the math associated with page-curl effects I am trying to: implement a page-curl shader for use on SwiftUI views. I've lifted a shader from HIROKI IKEUCHI that I believe they lifted from a non-metal shader resource online, and I'm trying to digest it. One thing I want to do is to paint the "underside" of the view with a given color and maintain the transparency of rounded corners when they are flipped over. So, if an underside pixel is "clear" then I want to sample the pixel at that position on the original layer instead of the "curl effect" pixel. There are two comments in the shader below where I check the alpha, and underside flags, and paint the color red as a debug test. The shader gives this result: The outside of those rounded corners is appropriately red and the white border pixels are detected as "not-clear". But the "inner" portion of the border is... mistakingly red? I don't get it. Any help would be appreciated. I feel tapped out and I don't have any IRL resources I can ask. // // PageCurl.metal // ShaderDemo3 // // Created by HIROKI IKEUCHI on 2023/10/17. // #include <metal_stdlib> #include <SwiftUI/SwiftUI_Metal.h> using namespace metal; #define pi float(3.14159265359) #define blue half4(0.0, 0.0, 1.0, 1.0) #define red half4(1.0, 0.0, 0.0, 1.0) #define radius float(0.4) // そのピクセルの色を返す [[ stitchable ]] half4 pageCurl ( float2 _position, SwiftUI::Layer layer, float4 bounds, float2 _clickedPoint, float2 _mouseCursor ) { half4 undersideColor = half4(0.5, 0.5, 1.0, 1.0); float2 originalPosition = _position; // y座標の補正 float2 position = float2(_position.x, bounds.w - _position.y); float2 clickedPoint = float2(_clickedPoint.x, bounds.w - _clickedPoint.y); float2 mouseCursor = float2(_mouseCursor.x, bounds.w - _mouseCursor.y); float aspect = bounds.z / bounds.w; float2 uv = position * float2(aspect, 1.) / bounds.zw; float2 mouse = mouseCursor.xy * float2(aspect, 1.) / bounds.zw; float2 mouseDir = normalize(abs(clickedPoint.xy) - mouseCursor.xy); float2 origin = clamp(mouse - mouseDir * mouse.x / mouseDir.x, 0., 1.); float mouseDist = clamp(length(mouse - origin) + (aspect - (abs(clickedPoint.x) / bounds.z) * aspect) / mouseDir.x, 0., aspect / mouseDir.x); if (mouseDir.x < 0.) { mouseDist = distance(mouse, origin); } float proj = dot(uv - origin, mouseDir); float dist = proj - mouseDist; float2 linePoint = uv - dist * mouseDir; half4 pixel = layer.sample(position); if (dist > radius) { pixel = half4(0.0, 0.0, 0.0, 0.0); // background behind curling layer (note: 0.0 opacity) pixel.rgb *= pow(clamp(dist - radius, 0., 1.) * 1.5, .2); } else if (dist >= 0.0) { // THIS PORTION HANDLES THE CURL SHADED PORTION OF THE RESULT // map to cylinder point float theta = asin(dist / radius); float2 p2 = linePoint + mouseDir * (pi - theta) * radius; float2 p1 = linePoint + mouseDir * theta * radius; bool underside = (p2.x <= aspect && p2.y <= 1. && p2.x > 0. && p2.y > 0.); uv = underside ? p2 : p1; uv = float2(uv.x, 1.0 - uv.y); // invert y pixel = layer.sample(uv * float2(1. / aspect, 1.) * float2(bounds[2], bounds[3])); // ME<---- if (underside && pixel.a == 0.0) { //<---- PIXEL.A IS 0.0 WHYYYYY pixel = red; } // Commented out while debugging alpha issues // if (underside && pixel.a == 0.0) { // pixel = layer.sample(originalPosition); // } else if (underside) { // pixel = undersideColor; // underside // } // Shadow the pixel being returned pixel.rgb *= pow(clamp((radius - dist) / radius, 0., 1.), .2); } else { // THIS PORTION HANDLES THE NON-CURL-SHADED PORTION OF THE SAMPLING. float2 p = linePoint + mouseDir * (abs(dist) + pi * radius); bool underside = (p.x <= aspect && p.y <= 1. && p.x > 0. && p.y > 0.); uv = underside ? p : uv; uv = float2(uv.x, 1.0 - uv.y); // invert y pixel = layer.sample(uv * float2(1. / aspect, 1.) * float2(bounds[2], bounds[3])); // ME if (underside && pixel.a == 0.0) { //<---- PIXEL.A IS 0.0 WHYYYYY pixel = red; } // Commented out while debugging alpha issues // if (underside && pixel.a == 0.0) { // // If the new underside pixel is clear, we should sample the original image's pixel. // pixel = layer.sample(originalPosition); // } else if (underside) { // pixel = undersideColor; // } } return pixel; }

I have a neural network model for segmentation, I successfully integrated it and am getting a grayscale image. Next, I need to apply the segmentation mask in RealityKit to achieve the occlusion effect (like person segmentation). I tried doing it through post-processing and other methods, but none of them worked. Is there any example of how this can be done in RealityKit?

Hi, I can capture a frame on the Apple TV, but when I try to profile the capture for GPU timing information, I got "Abort Trap 6" error and with following error in the report: Exception Type: EXC_CRASH (SIGABRT) Exception Codes: 0x0000000000000000, 0x0000000000000000 Triggered by Thread: 7 Application Specific Information: abort() called Last Exception Backtrace: 0 CoreFoundation 0x18c0a99d0 __exceptionPreprocess + 160 1 libobjc.A.dylib 0x18b596d24 objc_exception_throw + 71 2 CoreFoundation 0x18bfa7308 -[__NSArrayM insertObject:atIndex:] + 1239 3 MTLReplayController 0x101f5d148 DYMTLReplayFrameProfiler_loadAnalysis + 1140 4 MTLReplayController 0x101e97f90 GTMTLReplayClient_collectGPUShaderTimelineData + 224 5 MTLReplayController 0x101e81794 __30-[GTMTLReplayService profile:]_block_invoke_4 + 288 6 Foundation 0x18eb6072c __NSOPERATION_IS_INVOKING_MAIN__ + 11 7 Foundation 0x18eb5cc1c -[NSOperation start] + 623 8 Foundation 0x18eb60edc __NSOPERATIONQUEUE_IS_STARTING_AN_OPERATION__ + 11 9 Foundation 0x18eb60bc4 __NSOQSchedule_f + 167 10 libdispatch.dylib 0x18b8d6a84 _dispatch_block_async_invoke2 + 103 11 libdispatch.dylib 0x18b8c9420 _dispatch_client_callout + 15 12 libdispatch.dylib 0x18b8cc5d0 _dispatch_continuation_pop + 531 13 libdispatch.dylib 0x18b8cbcd4 _dispatch_async_redirect_invoke + 635 14 libdispatch.dylib 0x18b8d9224 _dispatch_root_queue_drain + 335 15 libdispatch.dylib 0x18b8d9a08 _dispatch_worker_thread2 + 163 16 libsystem_pthread.dylib 0x18b6e652c _pthread_wqthread + 223 17 libsystem_pthread.dylib 0x18b6ed8d0 start_wqthread + 7 It's Xcode 16.0 + Apply TV 4K (4th Gen) tvOS 18, does anyone know what's the cause of this error and is there any solution for it? Thank you very much, Kai

Using Swift, how do I resize individual SKSpriteNodes for various iPad device sizes? Currently, I use theScene.scaleMode = .resizeFill for scaling the entire SKScene and it works as advertised. Please note that .aspectFill does not solve the challenge described below. My challenge is to resize individual SKSpriteNodes (that are components of the overall SKScene) based on size of the iOS device; for example, iPad Mini <--> iPad Pro Right now, I hard code the sizes of these Nodes. But I realize this is not the optimum approach. Again, hard coding is frowned upon. So I am looking for a more dynamic method that functions based on device size.

Hello Everyone, within the renderEncoder?.drawIndexedPrimitives(type: .line…. function, I can't render all the lines of the object. I can see approx. 80%. Do you know what could be causing this? Other game engines, like those in C++, handle this just fine. import MetalKit class Renderer: NSObject, MTKViewDelegate { var parent: ContentView var metalDevice: MTLDevice! var metalCommandQueue: MTLCommandQueue! let allocator: MTKMeshBufferAllocator let pipelineState: MTLRenderPipelineState var scene: RenderScene let mesh: ObjMesh init(_ parent: ContentView) { self.parent = parent if let metalDevice = MTLCreateSystemDefaultDevice() { self.metalDevice = metalDevice } self.metalCommandQueue = metalDevice.makeCommandQueue() self.allocator = MTKMeshBufferAllocator(device: metalDevice) mesh = ObjMesh(device: metalDevice, allocator: allocator, filename: "cube") let pipelineDescriptor = MTLRenderPipelineDescriptor() let library = metalDevice.makeDefaultLibrary() pipelineDescriptor.vertexFunction = library?.makeFunction(name: "vertexShader") pipelineDescriptor.fragmentFunction = library?.makeFunction(name: "fragmentShader") pipelineDescriptor.colorAttachments[0].pixelFormat = .bgra8Unorm pipelineDescriptor.vertexDescriptor = MTKMetalVertexDescriptorFromModelIO(mesh.metalMesh.vertexDescriptor) do { try pipelineState = metalDevice.makeRenderPipelineState(descriptor: pipelineDescriptor) } catch { fatalError() } scene = RenderScene() super.init() } func mtkView(_ view: MTKView, drawableSizeWillChange size: CGSize) { } func draw(in view: MTKView) { //update scene.update() guard let drawable = view.currentDrawable else { return } let commandBuffer = metalCommandQueue.makeCommandBuffer() let renderPassDescriptor = view.currentRenderPassDescriptor renderPassDescriptor?.colorAttachments[0].clearColor = MTLClearColorMake(0, 0.5, 0.5, 1.0) renderPassDescriptor?.colorAttachments[0].loadAction = .clear renderPassDescriptor?.colorAttachments[0].storeAction = .store let renderEncoder = commandBuffer?.makeRenderCommandEncoder(descriptor: renderPassDescriptor!) renderEncoder?.setRenderPipelineState(pipelineState) var cameraData: CameraParameters = CameraParameters() cameraData.view = Matrix44.create_lookat( eye: scene.player.position, target: scene.player.position + scene.player.forwards, up: scene.player.up ) cameraData.projection = Matrix44.create_perspective_projection( fovy: 45, aspect: 800/600, near: 0.1, far: 10 ) renderEncoder?.setVertexBytes(&cameraData, length: MemoryLayout<CameraParameters>.stride, index: 2) renderEncoder?.setVertexBuffer(mesh.metalMesh.vertexBuffers[0].buffer, offset: 0, index: 0) for cube in scene.cubes { var model: matrix_float4x4 = Matrix44.create_from_rotation(eulers: cube.eulers) model = Matrix44.create_from_translation(translation: cube.position) * model renderEncoder?.setVertexBytes(&model, length: MemoryLayout<matrix_float4x4>.stride, index: 1) for submesh in mesh.metalMesh.submeshes { renderEncoder?.drawIndexedPrimitives( type: .line, indexCount: submesh.indexCount, indexType: submesh.indexType, indexBuffer: submesh.indexBuffer.buffer, indexBufferOffset: submesh.indexBuffer.offset ) } } renderEncoder?.endEncoding() commandBuffer?.present(drawable) commandBuffer?.commit() } } ==================== import MetalKit class ObjMesh { let modelIOMesh: MDLMesh let metalMesh: MTKMesh init(device: MTLDevice, allocator: MTKMeshBufferAllocator, filename: String) { guard let meshURL = Bundle.main.url(forResource: filename, withExtension: "obj") else { fatalError() } let vertexDescriptor = MTLVertexDescriptor() var offset: Int = 0 //position vertexDescriptor.attributes[0].format = .float3 vertexDescriptor.attributes[0].offset = offset vertexDescriptor.attributes[0].bufferIndex = 0 offset += MemoryLayout<SIMD3<Float>>.stride vertexDescriptor.layouts[0].stride = offset let meshDescriptor = MTKModelIOVertexDescriptorFromMetal(vertexDescriptor) (meshDescriptor.attributes[0] as! MDLVertexAttribute).name = MDLVertexAttributePosition let asset = MDLAsset(url: meshURL, vertexDescriptor: meshDescriptor, bufferAllocator: allocator) self.modelIOMesh = asset.childObjects(of: MDLMesh.self).first as! MDLMesh do { metalMesh = try MTKMesh(mesh: self.modelIOMesh, device: device) } catch { fatalError("couldn't load mesh") } } } =============== cube.obj Blender v2.91.0 OBJ File: '' www_blender_org mtllib piece.mtl o Cube_Cube.001 v -1.000000 1.000000 -1.000000 v -1.000000 1.000000 1.000000 v 1.000000 1.000000 -1.000000 v 1.000000 1.000000 1.000000 v -1.000000 -1.000000 -1.000000 v -1.000000 -1.000000 1.000000 v 1.000000 -1.000000 -1.000000 v 1.000000 -1.000000 1.000000 vt 0.375000 0.000000 vt 0.625000 0.000000 vt 0.625000 0.250000 vt 0.375000 0.250000 vt 0.625000 0.500000 vt 0.375000 0.500000 vt 0.625000 0.750000 vt 0.375000 0.750000 vt 0.625000 1.000000 vt 0.375000 1.000000 vt 0.125000 0.500000 vt 0.125000 0.750000 vt 0.875000 0.500000 vt 0.875000 0.750000 vn 0.0000 1.0000 0.0000 vn 1.0000 0.0000 0.0000 vn 0.0000 -1.0000 0.0000 vn -1.0000 0.0000 0.0000 vn 0.0000 0.0000 -1.0000 vn 0.0000 0.0000 1.0000 usemtl None s off f 1/1/1 2/2/1 4/3/1 3/4/1 f 3/4/2 4/3/2 8/5/2 7/6/2 f 7/6/3 8/5/3 6/7/3 5/8/3 f 5/8/4 6/7/4 2/9/4 1/10/4 f 3/11/5 7/6/5 5/8/5 1/12/5 f 8/5/6 4/13/6 2/14/6 6/7/6

Leaderboard has not worked for past two days. What’s up with that?

I'm wondering if it's possible to do Parallax Occlusion Mapping in RealityKit? Does RK's metal shader API provide enough? I think it would need to be able to discard fragments and thus can't be run as a deferred pass. Not sure though!

I have attempted to use VideoMaterial with HDR HLS stream, and also a TextureResource.DrawableQueue with rgba16Float in a ShaderGraphMaterial. I'm capturing to 64RGBAHalf with AVPlayerItemVideoOutput and converting that to rgba16Float. I don't believe it's displaying HDR properly or behaving like a raw AVPlayer. Since we can't configure any EDR metadata or color space for a RealityView, how do we display HDR video? Is using rgba16Float supposed to be enough? Is expecting the 64RGBAHalf capture to handle HDR properly a mistake and should I capture YUV and do the conversion myself? Thank you

Hi everyone, I've been working with the autoAdjustmentFilters provided by Core Image, which includes filters like CIHighlightShadowAdjust, CIVibrance, and CIToneCurve. However, I’ve noticed that the results differ significantly from the "Auto" enhancement feature in the Photos app. In the Photos app, the Auto function seems to adjust multiple parameters such as contrast, exposure, white balance, highlights, and shadows in a more advanced manner. Is there an API or a framework available that can replicate the more sophisticated "Auto" adjustments as seen in the Photos app? Or would I need to manually combine filters (like CIExposureAdjust, CIWhitePointAdjust, etc.) to approximate this functionality? Any insights or recommendations on how to achieve this would be greatly appreciated. Thank you!

Hi all, I'm new to swift and I've just gotten started by making a simple pong-like game using SpriteKit. I'm trying to use the scroll wheel input to spin an object, however, nothing seems to make this work. From my googling and AI advice the way I've been doing it should, as shown in the snippet below, however debugging suggests the scrollWheel function isn't even being called. #if os(OSX) extension GameScene { override func scrollWheel(with event: NSEvent ) { print("Scroll detected: \(event.scrollingDeltaY)") let scrollDelta = event.scrollingDeltaY self.rotatePaddle(value: scrollDelta) } } #endif I have changed the build targets in Xcode to Mac os, not the designed-for-ipad version of it, and the app does launch and draw sprites correctly as well as detect other mouse or trackpad inputs like mouseDown and rightMouseDown, so it only seems to be this one specific input not working. I've also tried hardware debugging like restarting Xcode and my Mac, but no luck there either. Maybe I've missed something or am doing it completely wrong, so any help would be much appreciated. Thanks heaps

I have created a turn based game using GameKit. Everything is pretty much done, last thing left to do is the turn time out. I am passing GKTurnTimeoutDefault into the timeout argument in: func endTurn(withNextParticipants nextParticipants: [GKTurnBasedParticipant], turnTimeout timeout: TimeInterval, match matchData: Data) However when I check the .timeoutDate property of the GKTurnBasedParticipant participants, the value is always nil. What am I doing wrong? Am I checking the right property or is there another one instead that I don't know about? I have tried passing different values to the timeout parameter but the timeoutDate is always nil. Has anyone successfully implemented a timeout using GKTurnBasedMatch ?