I have a third party app for controlling Sony mirrorless cameras over WiFi. I’m really excited to integrate the new camera controls on the iPhone 16 Pro with the app. I’ve found the documentation around this, and seems I need an AVCaptureSession setup in order to utilise them.
func configureControls(_ controls: [AVCaptureControl]) {
// Verify the host system supports controls; otherwise, return early.
guard captureSession.supportsControls else { return }
// Begin configuring the capture session.
captureSession.beginConfiguration()
// Remove previously configured controls, if any.
for control in captureSession.controls {
captureSession.removeControl(control)
}
// Iterate over the passed in controls.
for control in controls {
// Add the control to the capture session if possible.
if captureSession.canAddControl(control) {
captureSession.addControl(control)
} else {
print("Unable to add control \(control).")
}
}
// Commit the capture session configuration.
captureSession.commitConfiguration()
}
can I just use a freshly initialised capture session for this? Or does it need to be configured in any other ways? Are there any down sides to creating a session (CPU usage etc) that I may experience from this?
Also, the scope of the controls is quite narrow. For something like shutter speed or aperture that has quite a number of possible values but requires custom labels, and a non-linear scale (so the AVCaptureIndexPicker seems to be the way to go). Will that picker support enough values to represent something like shutter speed or aperture? Is there any chance we may get non-linear float based controls in the future, which may feel more natural from a UX perspective than index-based?
Apologies, lots of edits going on here as I think about this more.
Is there any way, or would any way be considered of putting these controls in a disabled state like with other UI elements in iOS? There are times (during capture for example) that a lot of these settings can be unavailable (as communicated by the Sony camera) to be changed by the user, and managing a queue of changes when the function is unavailable to be set is going to be a challenge. If there won’t be, how will they behave if controls are removed whilst being interacted with? Presumably they will disappear entirely from the UI?
Thanks!
Photos & Camera
RSS for tagExplore technical aspects of capturing high-quality photos and videos, including exposure control, focus modes, and RAW capture options.
Post
Replies
Boosts
Views
Activity
We build a video conference app and want to turn off the gesture reaction feature by default by sepecifying NSCameraReactionEffectGesturesEnabledDefault. The menu bar FaceTime info shows that the feature was turn off but it still works!
Our app is based on electron and all of the application plist was added the key and the camera is used by a spawned process (not electron process).
By turning on and turning off gesture reaction on menu bar, it will be turely turned off. So manual value will affect whole application but default value only affect application process?
How can I use the in-app sharing extension to get the livephoto file shared by the user through the album, including the video and the picture, instead of just the URL of the picture? I got the "com.apple.live-photo" and "public.jpeg" logos, but I can't get any data through "com.apple.live-photo"
I’ve built a iOS camera app that applies many CIFilters to an image captured by the camera. Some of my users have reported that on occasion the images have large parts that are blank, see below:
Frustratingly, I can’t reproduce this myself! Does anyone know what could he causing it, is it a memory issue? I haven’t posted the code as there’s a lot to look over and I’m not sure it would help diagnose it.
Thanks for any pointers.
I am trying to retrieve all PHAssets from the Photos Library that have been edited. PHAsset contains the "hasAdjustments" boolean variable but I cannot use that in an NSPredicate to filter for edited photos.
What is the proper mechanism to filter out non-edited photos?
Here is the code I have written that crashes:
let formatString = "mediaType = %d && hasAdjustments == YES"
let allPhotosOptions = PHFetchOptions()
allPhotosOptions.sortDescriptors = [NSSortDescriptor(key: "creationDate", ascending: true)]
allPhotosOptions.predicate = NSPredicate(format: formatString, PHAssetMediaType.image.rawValue)
allPhotosOptions.includeAssetSourceTypes = [.typeUserLibrary, .typeiTunesSynced, .typeCloudShared]
return PHAsset.fetchAssets(with: allPhotosOptions)
Thanks,
Rick
I have an Electron app built for macOS, and it was distributed via 'Developer ID' for years, it worked well and I was able to access the photos in the system Photos library. Surely I already have the 'NSPhotoLibraryUsageDescription' key in Info.plist.
Recently we are trying to publish this app to Mac App Store, so I have to turn on the sandbox, after that the app starts giving XPC errors while accessing the Photos library. The errors look like:
PHAuthorizationStatus: Authorized
CoreData: XPC: sendMessage: failed #0
CoreData: XPC: Unable to sendMessage: to server
...
CoreData: XPC: sendMessage: failed #7
CoreData: XPC: Unable to connect to server with options {
NSPersistentHistoryTrackingKey = 1;
NSXPCStoreServerEndpointFactory = "<PLXPCPhotoLibraryStoreEndpointFactory: 0x7fc67e8af370>";
skipModelCheck = 1;
}
CoreData: XPC: Unable to load metadata: Error Domain=NSCocoaErrorDomain Code=134060 "A Core Data error occurred." UserInfo={Problem=Unable to send to server; failed after 8 attempts.}
CoreData: fault: Unable to create token NSXPCConnection. NSXPCStoreServerEndpointFactory 0x7fc67e8af370 -newEndpoint returned nil
CoreData: error: Failed to create NSXPCConnection
It seems the app could detect the current PHAuthorizationStatus which is Authorized, but it can't fetch the photos from the Photos library (using PhotoKit).
I learned from here that I could look for errors from the sandboxd daemon, so I did that, here is what I saw:
Sandbox: Picture Keeper(32625) deny(1) mach-lookup com.apple.photos.service
Violation: deny(1) mach-lookup com.apple.photos.service
Process: Picture Keeper [32625]
Path: /Applications/Picture Keeper.app/Contents/MacOS/Picture Keeper
Load Address: 0x103bd3000
Identifier: com.simplifieditproducts.picturekeepermas
Version: 4575 (4.5.75)
Code Type: x86_64 (Native)
Parent Process: Picture Keeper [1]
Responsible: /Applications/Picture Keeper.app/Contents/MacOS/Picture Keeper
User ID: 501
Date/Time: 2024-08-26 16:16:14.645 EDT
OS Version: macOS 14.5 (23F79)
Release Type: User
Report Version: 8
MetaData: {"process_path":["Users","Kevin","Projects","Electron","picturekeeper-electron","dist","picturekeeper","mas-dev","Picture Keeper.app","Contents","MacOS","Picture Keeper"],"apple-internal":false,"primary-filter":"global-name","policy-description":"Sandbox","flags":5,"platform-policy":false,"build":"macOS 14.5 (23F79)","process-path":"\/Applications\/Picture Keeper.app\/Contents\/MacOS\/Picture Keeper","responsible-process-path":"\/Applications\/Picture Keeper.app\/Contents\/MacOS\/Picture Keeper","primary-filter-value":"com.apple.photos.service","platform_binary":"no","responsible-process-signing-id":"com.simplifieditproducts.picturekeepermas","hardware":"Mac","target":"com.apple.photos.service","action":"deny","mach_namespace":1,"checker-pid":1,"container":"\/Users\/Kevin\/Library\/Containers\/com.simplifieditproducts.picturekeepermas\/Data","binary-in-trust-cache":false,"team-id":"LU744924UY","process":"Picture Keeper","global-name":"com.apple.photos.service","platform-binary":false,"pid":32625,"summary":"deny(1) mach-lookup com.apple.photos.service","checker":"launchd","responsible-process-team-id":"xxxxx","operation":"mach-lookup","normalized_target":["com.apple.photos.service"],"errno":1,"uid":501,"profile-flags":0,"profile-in-collection":false,"sandbox_checker":"launchd","signing-id":"com.simplifieditproducts.picturekeepermas","release-type":"User"}
I believe I already have the necessary entitlements for the Photos library, see:
codesign -d --entitlements - /Applications/Picture\ Keeper.app/Contents/MacOS/Picture\ Keeper
[Dict]
[Key] com.apple.application-identifier
[Value]
[String] xxxx.com.simplifieditproducts.picturekeepermas
[Key] com.apple.developer.team-identifier
[Value]
[String] xxxx
[Key] com.apple.security.app-sandbox
[Value]
[Bool] true
[Key] com.apple.security.application-groups
[Value]
[Array]
[String] xxxx.com.simplifieditproducts.picturekeepermas
[Key] com.apple.security.assets.movies.read-only
[Value]
[Bool] true
[Key] com.apple.security.assets.music.read-only
[Value]
[Bool] true
[Key] com.apple.security.assets.pictures.read-write
[Value]
[Bool] true
[Key] com.apple.security.cs.allow-dyld-environment-variables
[Value]
[Bool] true
[Key] com.apple.security.cs.allow-jit
[Value]
[Bool] true
[Key] com.apple.security.cs.allow-unsigned-executable-memory
[Value]
[Bool] true
[Key] com.apple.security.cs.disable-executable-page-protection
[Value]
[Bool] true
[Key] com.apple.security.cs.disable-library-validation
[Value]
[Bool] true
[Key] com.apple.security.device.usb
[Value]
[Bool] true
[Key] com.apple.security.files.bookmarks.app-scope
[Value]
[Bool] true
[Key] com.apple.security.files.bookmarks.document-scope
[Value]
[Bool] true
[Key] com.apple.security.files.downloads.read-only
[Value]
[Bool] true
[Key] com.apple.security.files.user-selected.read-write
[Value]
[Bool] true
[Key] com.apple.security.network.client
[Value]
[Bool] true
[Key] com.apple.security.network.server
[Value]
[Bool] true
[Key] com.apple.security.personal-information.location
[Value]
[Bool] true
[Key] com.apple.security.personal-information.photos-library
[Value]
[Bool] true
By the way, the Photos library related code was built into a .node file (which is a dylib), and it will be loaded by the main executable during runtime.
Anything I missed? Thank you!
Hi All , I use iPhone 15 Pro Max with ios 18 beta 5 , when i use x app I can't see Any Image ! All image show like now available
Hi,
After installing iPadOS18 Beta3 on my iPad 7th gen, the default camera app no longer detects QR codes.
I tried updating to Beta7, but the issue remained.
Also, third-party apps that use AVCaptureMetadataOutput in AVFoundation Framework to detect QR codes also no longer work.
You can reproduce the issue by running default camera app or the AVFoundation sample code from the Apple developer site on iPad 7th gen (iPadOS18Beta installed).
https://developer.apple.com/documentation/avfoundation/capture_setup/avcambarcode_detecting_barcodes_and_faces
Has anyone else experienced this issue?
I would like to know if this issue occurs on other iPad models as well.
This is similar to the following issue that previously occurred with iPadOS 17.4.
https://support.apple.com/en-lamr/118614
https://developer.apple.com/forums/thread/748092
I am using PHPickerViewController to get assets from the photo library. It works fine, but it presents the images so that the newest is first. I would like to choose the order. In some cases I prefer the oldest image to be first.
I have checked in PHPickerConfiguration with no success. I am just wondering if it is possible.
各位开发者,想问问后续是否会慢慢恢复
Hello!
We want to try the new features of DockKit on an iOS 18 device. However, I am unable to pair the DockKit-compatible device with my iOS 18 device. Is there a way to successfully pair them?
I am developing a camera extension as described here Creating a camera extension as described in Creating a camera extension with Core Media I/O.
To ensure this wasn't some issue with my code, I reverted to the sample code added when you choose File > New > Target > Camera Extension in Xcode, in other words, I am using the example code provided by Apple.
I am able to install the camera extension and see it in QuickTime Player, where I choose it as the video input. In QuickTime Player I see the white line generated by the sample code moving up and down. For some period of time, it works.
But eventually, the video in QuickTime Player freezes. The thing that's really weird is if I add some NSLog() statements at the point in the code where it returns the newly created sample:
[self->_streamSource.stream sendSampleBuffer:sbuf discontinuity:CMIOExtensionStreamDiscontinuityFlagNone hostTimeInNanoseconds:(uint64_t)(CMTimeGetSeconds(timingInfo.presentationTimeStamp) * NSEC_PER_SEC)];
the samples are still being generated and sent to the stream. But the apparently QuickTime Player has decided to stop consuming them.
I thought maybe setting the discontinuity parameter to CMIOExtensionStreamDiscontinuityFlagTime or CMIOExtensionStreamDiscontinuityFlagSampleDropped if the delta time since the last sample was generated was off by a tiny bit, but this did not improve the situation.
Finally, could this have something to do with frequently installing and uninstalling my camera extension as part of the debugging and testing process?
Thanks in advance for any advice you might have!
Dear Experts,
When I use PHImageManager's requestImageDataAndOrientationForAsset method I always seem to get JPEG data, even when the original items in the photo library are PNGs (such as screenshots) or HEICs.
Have I missed a setting somewhere that determines whether or not a "most compatible" format is used in this API?
Thanks.
I am working on an iPad app using the front camera. The camera logic is implemented using the AVFoundation framework. During a session I use central stage mode to center the face with front camera . Central stage mode works fine in all cases except when I set the session preset photo. In other presets: high, medium, low, cif352x288, vga640x480, hd1280x720, hd1920x1080, iFrame960x540, iFrame1280x720, inputPriority presets central stage mode working except photo session preset. Can you explain why this happens or maybe it is an unobvious bug?
Code snippet:
final class CameraManager {
//MARK: - Properties
private let captureSession: AVCaptureSession
private let photoOutput: AVCapturePhotoOutput
private let previewLayer: AVCaptureVideoPreviewLayer
//MARK: - Init
init() {
captureSession = AVCaptureSession()
photoOutput = AVCapturePhotoOutput()
previewLayer = AVCaptureVideoPreviewLayer(session: captureSession)
}
//MARK: - Methods Setup Camera and preview layer
func setupPreviewLayerFrame(view: UIView) {
previewLayer.frame = view.frame
view.layer.insertSublayer(previewLayer, at: 0)
setupCamera()
}
private func setupCamera() {
guard let videoCaptureDevice = AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, position: .front) else { return }
AVCaptureDevice.centerStageControlMode = .app
AVCaptureDevice.isCenterStageEnabled = true
do {
let input = try AVCaptureDeviceInput(device: videoCaptureDevice)
captureSession.addInput(input)
captureSession.addOutput(photoOutput)
/// high, medium, low, cif352x288, vga640x480, hd1280x720, hd1920x1080, iFrame960x540, iFrame1280x720 and inputPriority presets working except photo session preset
captureSession.sessionPreset = .photo
DispatchQueue.global(qos: .userInteractive).async {
self.captureSession.startRunning()
}
} catch {
print("Error setting up camera: \(error.localizedDescription)")
}
}
}
Hello,
When using a UIImagePickerController with the .camera configuration I'm currently facing an issue where the delegate function imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [UIImagePickerController.InfoKey : Any]) is not firing. But rather UIKit internals dismisses the parent view to the UIImagePickerController. I'm showing the picker controller through a UIViewControllerRepresentable.
It does not always occur however, and the behavior is very flakey, sometimes it fires when pressing the b, sometimes it does not.
When setting a breakpoint at the dismiss function when pressing the "Use Photo" button, UIKit internals dismisses the view, not my own code.
I have a LockedCameraCapture extension working well, however there is one situation I cannot find a solution to. If the user has not yet provided camera access permission then the main app will be launched rather than the LockedCameraCapture extension. I cannot find a mechanism by which my main app can detect that this was the reason for the launch and thereby request permission.
When the button is pressed from the control center without permission the app is run and the CameraCaptureIntent is called so I can prompt the user from there. However, as best I can tell the CameraCaptureIntent is not called when launched from a locked Lock Screen, the app is simply opened.
My app has a variety of functions, most of which do not involve the camera so I cannot just always prompt the user for camera access on open. Is there any mechanism by which my main app can detect it was launched for this reason so it could ask for permission? Thank you!
I integrated Visionkit into my app, to get content of an image. That works really fine. Now I realized that iPhone8 crashes on launching the app, because of the following code:
available(iOS 17, *) extension Details: ImageAnalysisInteractionDelegate {}
is there a workaround for older versions?
Hi all,
Just wondering whether anyone knows there's anyway to support iPhone connecting with an external camera (e.g., USB-C webcam), like is enabled on the iPad?
Thank you!
Ios 18 at this moment copy from photo not working.
Can the extended code created by Capture Extension call the code of the main project?
I added the control via Widget Extension and I see the perform method is called in my intent but I am missing the part where this perform method will open the UI to capture the photo.
This is my intent:
struct MyAppCaptureIntent: CameraCaptureIntent {
static var title: LocalizedStringResource = "MyAppCaptureIntent"
typealias AppContext = MyAppContext
static let description = IntentDescription("Capture photos with MyApp.")
@MainActor
func perform() async throws -> some IntentResult {
let dialog = IntentDialog("Intent result")
do {
if let context = try await MyAppCaptureIntent.appContext {
return .result()
}
} catch {
// Handle error condition.
}
return .result()
}
}
struct MyAppContext: Decodable, Encodable {
var data = ContextData()
}
struct ContextData: IntentResult, Decodable, Encodable {
var value: Never? {
nil
}
}
How can I connect this with my LockedCameraCaptureExtension?
Can you provide a complete demo?