’m using the AVFoundation Swift APIs to record a Video (CMSampleBuffers) and Audio (CMSampleBuffers) to a file using AVAssetWriter. Initializing the AVAssetWriter happens quite quickly, but calling assetWriter.startWriting() fully blocks the entire application AND ALL THREADS for 3 seconds. This only happens in Debug builds, not in Release. Since it blocks all Threads and only happens in Debug, I’m lead to believe that this is an Xcode/Debugger/LLDB hang issue that I’m seeing. Does anyone experience something similar? Here’s how I set all of that up: startRecording(...) And here’s the line that makes it hang for 3+ seconds: assetWriter.startWriting(...)

When I upload a preview video to ASC (which conforms to preview specifications), the uploaded video it uploaded correctly in the first place. During upload to ASC it shows a blurred image of the first video frame. So far so good. But, once upload is finished the video turns into a "cloud" image and says it is currently processed. The problem is that it gets stuck in the status „currently processed“ forever. I waited a few days but processing did never end. To make it worse the landscape "cloud" image turns into a portrait when I come back to the ASC media center. The problem : is reproducible occurs on different video files occurs on different appIDs occurs on all iPhone resolutions This is a serious bug. I can’t finalise my app submission. Any ideas ?

I can't figure it out. the "padding":10 either is ignored or the background extends to the left and right edge. See attached screenshot from the simulator. "role": "heading", "text": "News", "format": "markdown", "textStyle": { "textAlignment": "center", "backgroundColor": "#006FBA", "textColor": "#FFFFFF", "fontSize": 14, "lineHeight": 16, "fontWeight": "bold", "fontScaling": false }, "style": { "backgroundColor": "#006FBA" }, "layout": { "padding":5, "ignoreDocumentMargin": false, "margin": { "bottom": 15, "left":30, "right":30 } } }, { "role": "heading", "text": "News", "format": "markdown", "textStyle": { "textAlignment": "center", "backgroundColor": "#006FBA", "textColor": "#FFFFFF", "fontSize": 14, "lineHeight": 16, "fontWeight": "bold", "fontScaling": false }, "layout": { "padding":5, "ignoreDocumentMargin": false, "margin": { "bottom": 15, "left":30, "right":30 } } }, { "role": "container", "components": [ { "role": "heading", "text": "News", "textStyle": { "textAlignment": "center", "backgroundColor": "#006FBA", "textAbove": "1", "textColor": "#FFFFFF", "fontSize": 14, "lineHeight": 16, "fontWeight": "bold", "fontScaling": false }, "style": { "backgroundColor": "#006FBA" }, "layout": { "padding":5 } } ] },

I'm using mediafilesegmenter with input as a fragmented mp4 hvc1 file and got this error: Nov 23 2023 17:48:25.948: Fragmented MP4 is the only supported container format for the segmentation of HEVC content Nov 23 2023 17:48:25.948: Unsupported media type 'hvc1' in track 0 Nov 23 2023 17:48:25.948: Unable to find any valid tracks to segment. Segmenting failed (-12780).

Hi Team, Offline playback with AES-128 encryption I'm downloading HLS content that is AES-128 encrypted and using the AVAssetResourceLoaderDelegate method shouldWaitForLoadingOfRequestedResource to parse the manifest to fetch the AES key URL. After fetching the key URL, I'll download and save the AES key locally. I will use the locally saved key to start the offline playback. Since AVContentKeySession has been there for quite some time, is it okay to use the resource loader delegate method to parse and download the AES key? Is there any chance that Apple will deprecate the downloading keys through the resource loader delegate? Thanks, Deepak.N

I am following this Apple Article on how to setup an AVPlayer. The only difference is I am using @Observable instead of an ObservableObject with @Published vars. Using @Observable results in the following error: Cannot find '$isPlaying' in scope If I remove the "$" symbol I get a bit more insight: Cannot convert value of type 'Bool' to expected argument type 'Published<Bool>.Publisher' If I change by class to an OO, it works fine, although, is there anyway to get this to work with @Observable?

Dear Sir/Madam, I am reaching out as a developer working on an academic project. I am currently working on my bachelor's thesis, where I am developing a mobile application for iOS devices. For the success of this project, it is essential to have precise information about the hardware components of specific iPhone models, especially the iPhone SE with the model number MMXN3ZD/A and iOS version 17.1.1. My primary interest lies in the exact technical specifications of the LEDs and the CCD or CMOS image sensor (depending on which type is used) installed in the iPhone SE. For my project, it is crucial to understand the spectral properties of these components: LED Specifications: I require information about the spectra of the LEDs, especially which wavelengths of light they emit. This is relevant for the functionality of my app, which relies on photometric analyses. CCD/CMOS Sensor Specifications: Furthermore, it is important for me to know the wavelengths for which the sensor built into the device is sensitive. This information is critical to accurately interpret the interaction between the sensor and the illuminated environment. The results of my research and development will not only be significant for my academic work but could also provide valuable insights for the further development of iOS applications in my field of study. I would be very grateful if you could provide me with this information or direct me to the appropriate department or resource where I can obtain these specific technical details. Thank you in advance for your support and cooperation. Best regards,

Following this Apple Article, I copied their code over for observePlayingState(). The only difference I am using @Observable instead of ObservableObject and @Published for var isPlaying. We get a bit more insight after removing the $ symbol, leading to a more telling error of: Cannot convert value of type 'Bool' to expected argument type 'Published.Publisher' Is there anyway to get this working with @Observable?

is it possible to use MusicKit to fetch us apple music/iTunes analytics of a user like the number of plays per song like a lot exist for Spotify. If so can someone please guide me as I need it for a project. Thanks!

I'm trying to decode MV-HEVC videos on visionOS, but I noticed the normal API for using AVPlayer (AVPlayerVideoOutput and some new methods for setting videoOutput on AVPlayer) are not available in visionOS 1.0. They're only available in iOS 17.2 and macOS 14.2 and the header claims visionOS 1.1 but it doesn't say that in the Apple documentation anywhere. Does this mean there's really no way to work on this functionality at this time (!) This seems like a major omission given we can't even target visionOS 1.1 with the beta version of Xcode. Can you please move this API forward into visionOS 1.0.

Hi, I've started learning swiftUI a few months ago, and now I'm trying to build my first app :) I am trying to display VTT subtitles from an external URL into a streaming video using AVPlayer and AVMutableComposition. I have been trying for a few days, checking online and on Apple's documentation, but I can't manage to make it work. So far, I managed to display the subtitles, but there is no video or audio playing... Could someone help? Thanks in advance, I hope the code is not too confusing. // EpisodeDetailView.swift // OroroPlayer_v1 // // Created by Juan Valenzuela on 2023-11-25. // import AVKit import SwiftUI struct EpisodeDetailView4: View { @State private var episodeDetailVM = EpisodeDetailViewModel() let episodeID: Int @State private var player = AVPlayer() @State private var subs = AVPlayer() var body: some View { VideoPlayer(player: player) .ignoresSafeArea() .task { do { try await episodeDetailVM.fetchEpisode(id: episodeID) let episode = episodeDetailVM.episodeDetail guard let videoURLString = episode.url else { print("Invalid videoURL or missing data") return } guard let subtitleURLString = episode.subtitles?[0].url else { print("Invalid subtitleURLs or missing data") return } let videoURL = URL(string: videoURLString)! let subtitleURL = URL(string: subtitleURLString)! let videoAsset = AVURLAsset(url: videoURL) let subtitleAsset = AVURLAsset(url: subtitleURL) let movieWithSubs = AVMutableComposition() let videoTrack = movieWithSubs.addMutableTrack(withMediaType: .video, preferredTrackID: kCMPersistentTrackID_Invalid) let audioTrack = movieWithSubs.addMutableTrack(withMediaType: .audio, preferredTrackID: kCMPersistentTrackID_Invalid) let subtitleTrack = movieWithSubs.addMutableTrack(withMediaType: .text, preferredTrackID: kCMPersistentTrackID_Invalid) // if let videoTrackItem = try await videoAsset.loadTracks(withMediaType: .video).first { try await videoTrack?.insertTimeRange(CMTimeRangeMake(start: .zero, duration: videoAsset.load(.duration)), of: videoTrackItem, at: .zero) } if let audioTrackItem = try await videoAsset.loadTracks(withMediaType: .audio).first { try await audioTrack?.insertTimeRange(CMTimeRangeMake(start: .zero, duration: videoAsset.load(.duration)), of: audioTrackItem, at: .zero) } if let subtitleTrackItem = try await subtitleAsset.loadTracks(withMediaType: .text).first { try await subtitleTrack?.insertTimeRange(CMTimeRangeMake(start: .zero, duration: videoAsset.load(.duration)), of: subtitleTrackItem, at: .zero) } let playerItem = AVPlayerItem(asset: movieWithSubs) player = AVPlayer(playerItem: playerItem) let playerController = AVPlayerViewController() playerController.player = player playerController.player?.play() // player.play() } catch { print("Error: \(error.localizedDescription)") } } } } #Preview { EpisodeDetailView4(episodeID: 39288) }

I just been working on an Augmented Reality (AR) application using Unity. I am facing a critical issue with camera access when my app is published on the App Store, and I'm reaching out to seek your guidance and assistance in resolving this matter. Here's a brief overview of the problem: During development and testing in Unity, the camera functionality in my AR app works as expected. I can access and utilize the device's camera for AR features without any issues. However, when I publish the app to the App Store and users download it, they are unable to access the device's camera within the app. This is a significant problem as camera access is fundamental to the app's functionality and user experience. I have taken several steps to ensure that I have correctly configured camera access permissions both in Unity's Player Settings and within Xcode: In Unity's Player Settings, I have provided a "Camera usage Description" to explain why the app needs camera access. In Xcode, I have also included the necessary privacy descriptions for camera access in the Info.plist file. Despite these efforts, the issue still persists. Users cannot access the camera when they download the app from the App Store. I have reviewed Apple's documentation and guidelines regarding camera access permissions, and I believe I have followed them correctly. I am eager to resolve this issue promptly to ensure that my AR app provides a seamless experience for users. If there are any specific steps or configurations that I might have missed or if there are any additional requirements or changes I need to make in order to enable camera access for my app on the App Store, I would greatly appreciate your guidance. If there is any additional information or logs you require from my end to assist in diagnosing and resolving this issue, please do let me know, and I will provide them promptly.

Hi, I'm a newbie here. I've signed up for the developer program hoping to retrieve extended metadata for the Classical music that is avaliable in Classical app. For example, each performer is assigned an instrument or voice. Unfortunately, the test API call did not return extended metadata, just regular metadata from Music App or page. Does anybody knows if there are plans to make this extended metadata available thru API ?

I am running Sonoma 14.1.1 and having an issue with DNG ProRAW images when choosing to edit. I can choose to edit using the photos app, or edit with Photoshop and in both cases the image once edit is chosen becomes less clear and slightly fuzzy with loss of detail. This happens too when I choose to export the photo from photos as a DNG file and then also try to edit with Photoshop the photos. There is a drastically noticeable difference in quality of the image. This appears to be the way the image is handles in the photos app itself. Even if I save directly from the iPhone to files, it does the same thing once on the Mac. Attached are some screen shot examples, but still clear to see the difference.

Hi, I'm trying to play multiple video/audio file with AVPlayer using AVMutableComposition. Each video/audio file can process simultaneously so I set each video/audio in individual tracks. I use only local file. let second = CMTime(seconds: 1, preferredTimescale: 1000) let duration = CMTimeRange(start: .zero, duration: second) var currentTime = CMTime.zero for _ in 0...4 { let mutableTrack = composition.addMutableTrack( withMediaType: .audio, preferredTrackID: kCMPersistentTrackID_Invalid ) try mutableTrack?.insertTimeRange( duration, of: audioAssetTrack, at: currentTime ) currentTime = currentTime + second } When I set many audio tracks (maybe more than 5), the first part sounds a little different from original when it starts. It seems like audio's front part is skipped. But when I set only two tracks, AVPlayer plays as same as original file. avPlayer.play() How can I fix it? Why do audio tracks affect that don't have any playing parts when start? Please let me know.

Dear Apple Engineers, First of all, thank you for this wonderful and very necessary native solution. Question: Is it possible to use this API when processing HLS? Thank you.

Hello, Faced with a really perplexing issue. Primary problem is that sometimes I get depth and video data as expected, but at other times I don't. And sometimes I'll get both data outputs for a 4-5 frames and then it'll just stop. The source code I implemented is a modified version of the sample code provided by Apple, and interestingly enough I can't re-create this issue with the Apple sample app. So wondering what I could be doing wrong? Here's the code for setting up the capture input. preferredDepthResolution is 1280 in my case. I'm running this on an iPad Pro (6th gen). iOS version 17.0.3 (21A360). Encounter this issue on iPhone 13 Pro as well. iOS version is 17.0 (21A329) private func setupLiDARCaptureInput() throws { // Look up the LiDAR camera. guard let device = AVCaptureDevice.default(.builtInLiDARDepthCamera, for: .video, position: .back) else { throw ConfigurationError.lidarDeviceUnavailable } guard let format = (device.formats.last { format in format.formatDescription.dimensions.width == preferredWidthResolution && format.formatDescription.mediaSubType.rawValue == kCVPixelFormatType_420YpCbCr8BiPlanarFullRange && format.videoSupportedFrameRateRanges.first(where: {$0.maxFrameRate >= 60}) != nil && !format.isVideoBinned && !format.supportedDepthDataFormats.isEmpty }) else { throw ConfigurationError.requiredFormatUnavailable } guard let depthFormat = (format.supportedDepthDataFormats.last { depthFormat in depthFormat.formatDescription.mediaSubType.rawValue == kCVPixelFormatType_DepthFloat16 }) else { throw ConfigurationError.requiredFormatUnavailable } // Begin the device configuration. try device.lockForConfiguration() // Configure the device and depth formats. device.activeFormat = format device.activeDepthDataFormat = depthFormat let desc = format.formatDescription dimensions = CMVideoFormatDescriptionGetDimensions(desc) let duration = CMTime(value:1, timescale:CMTimeScale(60)) device.activeVideoMinFrameDuration = duration device.activeVideoMaxFrameDuration = duration // Finish the device configuration. device.unlockForConfiguration() self.device = device print("Selected video format: \(device.activeFormat)") print("Selected depth format: \(String(describing: device.activeDepthDataFormat))") // Add a device input to the capture session. let deviceInput = try AVCaptureDeviceInput(device: device) captureSession.addInput(deviceInput) guard let audioDevice = AVCaptureDevice.default(for: .audio) else { return } // Configure audio input - always configure audio even if isAudioEnabled is false audioDeviceInput = try! AVCaptureDeviceInput(device: audioDevice) captureSession.addInput(audioDeviceInput) deviceSystemPressureStateObservation = device.observe( \.systemPressureState, options: .new ) { _, change in guard let systemPressureState = change.newValue else { return } print("system pressure \(systemPressureState.levelAsString()) due to \(systemPressureState.factors)") } } Here's how I'm setting up the output: private func setupLiDARCaptureOutputs() { // Create an object to output video sample buffers. videoDataOutput = AVCaptureVideoDataOutput() captureSession.addOutput(videoDataOutput) // Create an object to output depth data. depthDataOutput = AVCaptureDepthDataOutput() depthDataOutput.isFilteringEnabled = false captureSession.addOutput(depthDataOutput) audioDeviceOutput = AVCaptureAudioDataOutput() audioDeviceOutput.setSampleBufferDelegate(self, queue: videoQueue) captureSession.addOutput(audioDeviceOutput) // Create an object to synchronize the delivery of depth and video data. outputVideoSync = AVCaptureDataOutputSynchronizer(dataOutputs: [depthDataOutput, videoDataOutput]) outputVideoSync.setDelegate(self, queue: videoQueue) // Enable camera intrinsics matrix delivery. guard let outputConnection = videoDataOutput.connection(with: .video) else { return } if outputConnection.isCameraIntrinsicMatrixDeliverySupported { outputConnection.isCameraIntrinsicMatrixDeliveryEnabled = true } } The top part of my delegate implementation is as follows: func dataOutputSynchronizer( _ synchronizer: AVCaptureDataOutputSynchronizer, didOutput synchronizedDataCollection: AVCaptureSynchronizedDataCollection ) { // Retrieve the synchronized depth and sample buffer container objects. guard let syncedDepthData = synchronizedDataCollection.synchronizedData(for: depthDataOutput) as? AVCaptureSynchronizedDepthData, let syncedVideoData = synchronizedDataCollection.synchronizedData(for: videoDataOutput) as? AVCaptureSynchronizedSampleBufferData else { if synchronizedDataCollection.synchronizedData(for: depthDataOutput) == nil { print("no depth data at time \(mach_absolute_time())") } if synchronizedDataCollection.synchronizedData(for: videoDataOutput) == nil { print("no video data at time \(mach_absolute_time())") } return } print("received depth data \(mach_absolute_time())") } As you can see, I'm console logging whenever depth data is not received. Note because I'm driving the video frames at 60 fps, its expected that I'll only receive depth data for every alternate video frame. Console output is posted as a follow up comment (because of the character limit). I edited some lines out for brevity. You'll see it started streaming correctly but after a while it stopped received both video and depth outputs (in some other runs, it works perfectly and in some other runs I receive no depth data whatsoever). One thing to note, I sometimes run quicktime mirroring to see the device screen to see what the app is displaying (so not sure if that's causing any interference - that said I don't see any system pressure changes either). Any help is most appreciated! Thanks.

I got code of CMIO CameraExtension by Xcode target and it is running with FaceTime. I guess this kind of Extension has lots of security limitation. I like to run command like "netstat" in Extension. Is that possible to call Process.run()? I got keep getting error like "The file zsh doesn’t exist". Same code with Process.run() worked in macOS app. I like to run DistributedNotificationCenter and send text from App to CameraExtension. Is that possible? I do not receive any message on CameraExtension. If there is any other IPC method between macOS app and CameraExtension, please let me know.

I tried the same code on ios17 and ios16 when enable address sanitizer, ios17 will crash, why? Can anyone help me？ AudioComponent comp = {0}; AudioComponentDescription compDesc = {0}; compDesc.componentType = kAudioUnitType_Output; compDesc.componentSubType = kAudioUnitSubType_RemoteIO; compDesc.componentManufacturer = kAudioUnitManufacturer_Apple; compDesc.componentFlags = 0; compDesc.componentFlagsMask = 0; comp = AudioComponentFindNext(NULL, &compDesc); if (comp == NULL) { assert(false); } AudioUnit tempAudioUnit; osResult = AudioComponentInstanceNew(comp, &tempAudioUnit); if (osResult != noErr) { assert(false); }

Hello, I am deaf-blind and I program with a braille display. Currently, I am experiencing a difficulty with one of my APPs. Basically, I'm converting AVAudioPCMBuffer for CMSampleBuffer and so far so good. I want to add several CMSampleBuffer in a video written with AVAssetWrite . The problem is that I can only add up to more or less 2 thousands CMSampleBuffer . I'm trying to create a video. In this video, I put photos that are in an array and then I put audio from CMSampleBuffer. But I can't add many CMSampleBuffer and only goes up to 2 thousand something. I do not know what else to do. Help me. Below is a small excerpt of the code: let queue = DispatchQueue(label: "AssetWriterQueue") let audioProvider = SampleProvider(buffers: audioBuffers) let videoProvider = SampleProvider(buffers: videoBuffers) let audioInput = createAudioInput(audioBuffers: audioBuffers) let videoInput = createVideoInput(videoBuffers: videoBuffers) let adaptor = createPixelBufferAdaptor(videoInput: videoInput) let assetWriter = try AVAssetWriter(outputURL: url, fileType: .mp4) assetWriter.add(videoInput) assetWriter.add(audioInput) assetWriter.startWriting() assetWriter.startSession(atSourceTime: .zero) await withCheckedContinuation { continuation in videoInput.requestMediaDataWhenReady(on: queue) { let time = videoProvider.getPresentationTime() if let buffer = videoProvider.getNextBuffer() { adaptor.append(buffer, withPresentationTime: time) } else { videoInput.markAsFinished() continuation.resume() } } } await withCheckedContinuation { continuation in audioInput.requestMediaDataWhenReady(on: queue) { if let buffer = audioProvider.getNextBuffer() { audioInput.append(buffer) } else { audioInput.markAsFinished() continuation.resume() } } }