Explore the integration of media technologies within your app. Discuss working with audio, video, camera, and other media functionalities.

All subtopics

Post

Replies

Boosts

Views

Activity

Sending '$0' risks causing data races
I had no luck to compile a sample code provided by apple with Xcode 16.0 beta 5. ScreenCaptureKit demo (https://developer.apple.com/documentation/screencapturekit/capturing_screen_content_in_macos) The part it is failling is, streamOutput.capturedFrameHandler = { continuation.yield($0) } And the error message is Sending '$0' risks causing data races Task-isolated '$0' is passed as a 'sending' parameter; Uses in callee may race with later task-isolated uses Please enlighten me why this is an issue and how to avoid? Thanks in advance!
3
2
1.1k
Aug ’24
Some use AVCaptureControl problems
Set 3 controls to the AVCaptureSession and remove them all. The number of controls in the session is indeed 0, but the camera controls button still shows the previous 3 controls. If it is only 3->2 or 3->1, it can be modified normally, 3->0 is not OK, 0->3 is OK. f (self.captureControl.zoom) { if (self.zoomScaleControl) { self.zoomScaleControl.enabled = false; [_session removeControl:self.zoomScaleControl]; } AVCaptureSlider *zoomSlider = [self.captureControl.zoom fetchCaptureSlider]; [zoomSlider setActionQueue:dispatch_get_main_queue() action:^(float zoomFactor) { @strongify(self); if ([self.dataOutputDelegate respondsToSelector:@selector(videoCaptureSession:tryChangeZoomScale:)]) { [self.dataOutputDelegate videoCaptureSession:self tryChangeZoomScale:zoomFactor]; } }]; self.zoomScaleControl = zoomSlider; } else { if (self.zoomScaleControl) { self.zoomScaleControl.enabled = false; [_session removeControl:self.zoomScaleControl]; } self.zoomScaleControl = nil; } if (self.captureControl.exposure) { if (self.exposureBiasControl) { self.exposureBiasControl.enabled = false; [_session removeControl:self.exposureBiasControl]; } AVCaptureSlider *exposureSlider = [self.captureControl.exposure fetchCaptureSlider]; [exposureSlider setActionQueue:dispatch_get_main_queue() action:^(float bias) { @strongify(self); if ([self.dataOutputDelegate respondsToSelector:@selector(videoCaptureSession:tryChangeExposureBias:)]) { [self.dataOutputDelegate videoCaptureSession:self tryChangeExposureBias:bias]; } }]; self.exposureBiasControl = exposureSlider; } else { if (self.exposureBiasControl) { self.exposureBiasControl.enabled = false; [_session removeControl:self.exposureBiasControl]; } self.exposureBiasControl = nil; } if (self.captureControl.len) { if (self.lenControl) { self.lenControl.enabled = false; [_session removeControl:self.lenControl]; } ORLenCaptureControlCustomModel *len = self.captureControl.len; AVCaptureIndexPicker *picker = [len fetchCaptureSlider]; [picker setActionQueue:dispatch_get_main_queue() action:^(NSInteger selectedIndex) { @strongify(self); if ([self.dataOutputDelegate respondsToSelector:@selector(videoCaptureSession:didChangeLenIndex:datas:)]) { [self.dataOutputDelegate videoCaptureSession:self didChangeLenIndex:selectedIndex datas:self.captureControl.len.indexDatas]; } }]; self.lenControl = picker; } else { if (self.lenControl) { self.lenControl.enabled = false; [_session removeControl:self.lenControl]; } self.lenControl = nil; } if ([_session canAddControl:self.zoomScaleControl]) { [_session addControl:self.zoomScaleControl]; } else { self.zoomScaleControl = nil; } if ([_session canAddControl:self.lenControl]) { [_session addControl:self.lenControl]; } else { self.lenControl = nil; } if ([_session canAddControl:self.exposureBiasControl]) { [_session addControl:self.exposureBiasControl]; } else { self.exposureBiasControl = nil; } if (_session.controlsDelegate == nil) { [_session setControlsDelegate:self queue:GetCaptureControlQueue()]; }
0
0
135
2w
Spatial video export fails with AVAssetExportSession
We captured a spatial video with iPhone 15 pro. When we try to export the video with AVAssetExportSession and AVAssetExportPresetMVHEVC960x960 it always go failed state and exportSession.error?.localizedDescription yield "Operation Stopped" error. Code implementation is straight forward .. other HEVC file works well.This problem occurred with only mv-hevc file. func exportSpatialVideo(videoFilePath: String, outputUrl: URL){ let url:URL? = URL(fileURLWithPath: videoFilePath) let asset: AVAsset = AVAsset(url:url!) print(asset.description) print(asset.tracks.first?.mediaType.rawValue) let preset = "AVAssetExportPresetMVHEVC960x960" let exportSession:AVAssetExportSession = AVAssetExportSession(asset: asset, presetName: preset)! exportSession.outputURL = outputUrl exportSession.shouldOptimizeForNetworkUse = true exportSession.outputFileType = AVFileType.mov exportSession.exportAsynchronously(completionHandler: { switch exportSession.status { case .unknown: print("Unknown Error") case .waiting: print( "waiting ... ") case .exporting: print( "exporting ...") case .completed: print( "completed.") case .failed: print("failed.\(String(describing: exportSession.error?.localizedDescription))") case .cancelled: } }) } is there any solution for it ?
0
0
139
2w
slow motion video playback "ramp-up" stage
In iOS, when I use AVPlayerViewController to play back a slow motion video, it has a "ramp-up" stage at the start and a "ramp-down" stage at the end, and the video plays at the normal speed (i.e. not slow motion) during these stages. My question is: are these non-slow-motion stages defined in the video file itself? (e.g. some kind of meta data?) Or, is it just a playback approach used by AVPlayerViewController ? Thanks!
0
0
132
2w
PHIImageManager cannot resize images
I am using PHImageRequestOptions and PHImageManager to load images to my app. I use version.original and resizeMode.none, version.original and resizeMode.extract. Both used to work well but since iOS18 version.original and resizeMode.extract doesn't work anymore. The images are loaded but the they are not shown. (Only the frames?) Anyone knows why? Thank you for reading.
0
0
89
2w
MusicKit currentEntry.item is nil but currentEntry exists.
I'm trying to get the item that's assigned to the currentEntry when playing any song which is currently coming up nil when the song is playing. Note currentEntry returns: MusicPlayer.Queue.Entry(id: "evn7UntpH::ncK1NN3HS", title: "SONG TITLE") I'm a bit stumped on the API usage. if the song is playing, how could the queue item be nil? if queueObserver == nil { queueObserver = ApplicationMusicPlayer.shared.queue.objectWillChange .sink { [weak self] in self?.handleNowPlayingChange() } } } private func handleNowPlayingChange() { if let currentItem = ApplicationMusicPlayer.shared.queue.currentEntry { if let song = currentItem.item { self.currentlyPlayingID = song.id self.objectWillChange.send() print("Song ID: \(song.id)") } else { print("NO ITEM: \(currentItem)") } } else { print("No Entries: \(ApplicationMusicPlayer.shared.queue.currentEntry)") } }
0
0
124
2w
AudioConverterFillComplexBuffer not working for (E)AC3 in tvOS 18
Since upgrading to tvOS 18, the above function isn't working for me in converting a stream with these formats. It does work in decoding AAC, however. https://developer.apple.com/documentation/audiotoolbox/1503098-audioconverterfillcomplexbuffer?language=objc I pass a valid ioOutputDataPacketSize in, but it always comes out as zero. Has anyone else observed this too? I wonder if this is related to the issue being discussed widely about 5.1 sound being broken for many people after upgrading to tvOS 18? https://discussions.apple.com/thread/255769102?login=true&sortBy=rank EDIT: further information; the callback gets called once, asking for 1 packet (which is ok). I give it one packet and return noErr. However, after this, the callback is never invoked again. Must be a bug? EDIT2: the same code continues to work correctly on macOS in decoding the same audio stream.
3
2
257
Oct ’24
[iPadOS18 Beta3] on iPad 7th gen, camera app cannot detect QR code
Hi, After installing iPadOS18 Beta3 on my iPad 7th gen, the default camera app no ​​longer detects QR codes. I tried updating to Beta7, but the issue remained. Also, third-party apps that use AVCaptureMetadataOutput in AVFoundation Framework to detect QR codes also no longer work. You can reproduce the issue by running default camera app or the AVFoundation sample code from the Apple developer site on iPad 7th gen (iPadOS18Beta installed). https://developer.apple.com/documentation/avfoundation/capture_setup/avcambarcode_detecting_barcodes_and_faces Has anyone else experienced this issue? I would like to know if this issue occurs on other iPad models as well. This is similar to the following issue that previously occurred with iPadOS 17.4. https://support.apple.com/en-lamr/118614 https://developer.apple.com/forums/thread/748092
5
0
623
Aug ’24
Add to Shared Photo Library with PhotoKit
I'm using PhotoKit in macOS to add photos to the user's library. Experimenting with Shared Photo Library, it seems that these new photos always end up in the Personal Library, not the Shared Library. I'd like to get them into the Shared Photo Library somehow. Is this possible? Things I've considered: A variation/option for PHAssetChangeRequest.creationRequestForAsset: doesn't seem to exist A property of PHAsset: can't find anything A special PHAssetCollection that I could add to: again, doesn't seem to exist
1
0
173
3w
GPS Coordinates with Photos
I am currently developing an application that requires access to GPS coordinates from photos on iOS. However, with the recent update to iOS 18 beta, I have encountered a challenge: I can only view photos within maps, and I am unable to access the GPS coordinates directly. Could you please provide guidance on how to enable or retrieve GPS coordinates from photos in the current iOS 18 beta version? Any insights or resources you could share would be greatly appreciated. Thank you for your assistance!
1
0
144
3w
Optimizing YOLOv8 for Real-Time Object Detection in a Specific Screen Area
I’m working on real-time object detection using YOLOv8, but I only need to detect objects in approximately 40% of the screen area. Is it possible to limit the captureOut method to focus solely on that specific region of the screen? If this isn’t feasible, I’m considering an approach where the full-screen pixel buffer is captured and then cropped to the target area before running detection. However, I’m concerned about how this might affect real-time performance. I’d appreciate any insights on how to maintain real-time performance or suggestions for better alternatives. Thank you!
2
0
227
3w
VideoPlayer crashes in Xcode Preview for macOS app
I think I have the simplest possible Mac app trying to see if I can have VideoPlayer work in an Xcode Preview. It works in an iOS app project. In a Mac app project it builds and runs. But if I preview in Xcode it crashes. The diagnostic says: | [Remote] Unknown Error: The operation couldn’t be completed. XPC error received on message reply handler | | BSServiceConnectionErrorDomain (3): | ==NSLocalizedFailureReason: XPC error received on message reply handler | ==BSErrorCodeDescription: OperationFailed The code I'm using is the exact code from the VideoPlayer documentation page. See this link. Any ideas about this XPC error, and how to work around? I'm using Xcode 16.0 on macOS 14.6.1
2
0
235
3w
Potential bug with PHAsset Creation Request with photoProxy resource type.
I have noticed a problem when a PHAsset creation request is made with the resource type PHAssetResourceType.photoProxy. let creationRequest = PHAssetCreationRequest.forAsset() creationRequest.addResource(with: .photoProxy, data: photoData, options: nil) creationRequest.location = location creationRequest.isFavorite = true After successfully saving the resulting asset through PHPhotoLibrary.shared().performChanges, I could verify it in the Photos app. I noticed that the created photo was initially marked as Favorite and that the location was added to the info as expected. The title of the image changes from "Today" to "" too. Next, the photo was refreshed, and location data was purged. However, the title remains unchanged and displays the . This refresh was also observed in the code. PHPhotoLibraryChangeObserver protocols func photoLibraryDidChange(_ changeInstance: PHChange) receives a change notification. The same asset has been changed, and there is no location information anymore. isFavorite information persists correctly. After debugging for a few hours, I discovered that changing the resource type to .photo fixes this issue. Location data is not removed in the Photos app, and no refresh callback is seen in func photoLibraryDidChange(_ changeInstance: PHChange). I initially used .photoProxy because in the AVCapturePhotoCaptureDelegate implementation class, I always get the call in func photoOutput(_ output: AVCapturePhotoOutput, didFinishCapturingDeferredPhotoProxy deferredPhotoProxy: AVCaptureDeferredPhotoProxy?, error: Error?). So here is where I am capturing the photo data as photoData = deferredPhotoProxy?.fileDataRepresentation().
2
0
141
2w
Detect when (internal or external) microphone is being used
Hello, I used kAudioDevicePropertyDeviceIsRunningSomewhere to check if an internal or external microphone is being used. My code works well for the internal microphone, and for microphones which are connected using a cable. External microphones which are connected using bluetooth are not reporting their status. The status is always requested successfully, but it is always reported as inactive. Main relevant parts in my code : static inline AudioObjectPropertyAddress makeGlobalPropertyAddress(AudioObjectPropertySelector selector) { AudioObjectPropertyAddress address = { selector, kAudioObjectPropertyScopeGlobal, kAudioObjectPropertyElementMaster, }; return address; } static BOOL getBoolProperty(AudioDeviceID deviceID, AudioObjectPropertySelector selector) { AudioObjectPropertyAddress const address = makeGlobalPropertyAddress(selector); UInt32 prop; UInt32 propSize = sizeof(prop); OSStatus const status = AudioObjectGetPropertyData(deviceID, &address, 0, NULL, &propSize, &prop); if (status != noErr) { return 0; //this line never gets executed in my tests. The call above always succeeds, but it always gives back "false" status. } return static_cast<BOOL>(prop == 1); } ... __block BOOL microphoneActive = NO; iterateThroughAllInputDevices(^(AudioObjectID object, BOOL *stop) { if (getBoolProperty(object, kAudioDevicePropertyDeviceIsRunningSomewhere) != 0) { microphoneActive = YES; *stop = YES; } }); What could cause this and how could it be fixed? Thank you for your help in advance!
3
0
833
Nov ’23
API to use for high-level audio playback to a specific audio device?
I'm working on a little light and sound controller in Swift, driving DMX lights and audio. For the audio portion, I need to play a bunch of looping sounds (long-duration MP3s), and occasionally play sound effects (short-duration sounds, varying formats). I want all of this mixed into selected channels on specific devices. That is, I might have one audio stream going to the left channel, and a completely different one going to the right channel. What's the right API to do this from Swift? Core Audio? AVPlayer stuff?
0
0
117
2w
MusicKit and sorted artist and album names?
I have an app that gets data from Music.app with both the iTunesLibrary and MusicKit. iTunesLibrary has ITLibArtist.sortName and ITLibAlbum.sortTitle and ITLibAlbum.sortAlbumArtist. I can’t seem to find an equivalent in MusicKit. How are those properties obtained using MusicKit? Thanks. FYI I have filed FB15554956 on this. You also may see my code at https://github.com/bolsinga/itunes_json
0
0
104
2w
Heic format image incompatibility issue
I am a developer working on iOS apps. In the demo, I planned to replace the local images with Heic format instead of PNG format, but the actual test results showed abnormalities on this device, while the other test devices displayed normally Heic images are converted by the built-in image conversion function on Mac. I tested multiple Heic images, but none of them were displayed and the image information returned nil,,but PNG images can be displayed normally. device information:
5
0
274
3w