Explore the integration of media technologies within your app. Discuss working with audio, video, camera, and other media functionalities.

Post

Replies

Boosts

Views

Activity

[Metal] 9072 by 12096 iosurface is too large for GPU
I take a picture using the iPhone's camera. The taken resolution is 3024.0 x 4032. I then have to apply a watermark to this image. After a bunch of trial and error, the method I decided to use was taking a snapshot of a watermark UIView, and drawing that over the image, like so: // Create the watermarked photo. let result: UIImage=UIGraphicsImageRenderer(size: image.size).image(actions: { _ in   image.draw(in: .init(origin: .zero, size: image.size))   let watermark: Watermark = .init(     size: image.size,     scaleFactor: image.size.smallest / self.frame.size.smallest   )   watermark.drawHierarchy(in: .init(origin: .zero, size: image.size), afterScreenUpdates: true) }) Then with the final image — because the client wanted it to have a filename as well when viewed from within the Photos app and exported from it, and also with much trial and error — I save it to a file in a temporary directory. I then save it to the user's Photo library using that file. The difference as compared to saving the image directly vs saving it from the file is that when saved from the file the filename is used as the filename within the Photos app; and in the other case it's just a default photo name generated by Apple. The problem is that in the image saving code I'm getting the following error: [Metal] 9072 by 12096 iosurface is too large for GPU And when I view the saved photo it's basically just a completely black image. This problem only started when I changed the AVCaptureSession preset to .photo. Before then there was no errors. Now, the worst problem is that the app just completely crashes on drawing of the watermark view in the first place. When using .photo the resolution is significantly higher, so the image size is larger, so the watermark size has to be commensurately larger as well. iOS appears to be okay with the size of the watermark UIView. However, when I try to draw it over the image the app crashes with this message from Xcode: So there's that problem. But I figured that could be resolved by taking a more manual approach to the drawing of the watermark then using a UIView snapshot. So it's not the most pressing problem. What is, is that even after the drawing code is commented out, I still get the iosurface is too large error. Here's the code that saves the image to the file and then to the Photos library: extension UIImage {   /// Save us with the given name to the user's photo album.   /// - Parameters:   ///  - filename: The filename to be used for the saved photo. Behavior is undefined if the filename contain characters other than what is represented by this regular expression [A-Za-z0-9-_]. A decimal point for the file extension is permitted.   ///  - location: A GPS location to save with the photo.   fileprivate func save(_ filename: String, _ location: Optional<Coordinates>) throws {           // Create a path to a temporary directory. Adding filenames to the Photos app form of images is accomplished by first creating an image file on the file system, saving the photo using the URL to that file, and then deleting that file on the file system.     //   A documented way of adding filenames to photos saved to Photos was never found.     // Furthermore, we save everything to a `tmp` directory as if we just tried deleting individual photos after they were saved, and the deletion failed, it would be a little more tricky setting up logic to ensure that the undeleted files are eventually     // cleaned up. But by using a `tmp` directory, we can save all temporary photos to it, and delete the entire directory following each taken picture.     guard       let tmpUrl: URL=try {         guard let documentsDirUrl=NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true).first else {           throw GeneralError("Failed to create URL to documents directory.")         }         let url: Optional<URL> = .init(string: documentsDirUrl + "/tmp/")         return url       }()     else {       throw GeneralError("Failed to create URL to temporary directory.")     }           // A path to the image file.     let filePath: String=try {               // Reduce the likelihood of photos taken in quick succession from overwriting each other.       let collisionResistantPath: String="\(tmpUrl.path(percentEncoded: false))\(UUID())/"               // Make sure all directories required by the path exist before trying to write to it.       try FileManager.default.createDirectory(atPath: collisionResistantPath, withIntermediateDirectories: true, attributes: nil)               // Done.       return collisionResistantPath + filename     }()     // Create `CFURL` analogue of file path.     guard let cfPath: CFURL=CFURLCreateWithFileSystemPath(nil, filePath as CFString, CFURLPathStyle.cfurlposixPathStyle, false) else {       throw GeneralError("Failed to create `CFURL` analogue of file path.")     }           // Create image destination object.     //     // You can change your exif type here.     //   This is a note from original author. Not quite exactly sure what they mean by it. Link in method documentation can be used to refer back to the original context.     guard let destination=CGImageDestinationCreateWithURL(cfPath, UTType.jpeg.identifier as CFString, 1, nil) else {       throw GeneralError("Failed to create `CGImageDestination` from file url.")     }           // Metadata properties.     let properties: CFDictionary={               // Place your metadata here.       // Keep in mind that metadata follows a standard. You can not use custom property names here.       let tiffProperties: Dictionary<String, Any>=[:]               return [         kCGImagePropertyExifDictionary as String: tiffProperties       ] as CFDictionary     }()           // Create image file.     guard let cgImage=self.cgImage else {       throw GeneralError("Failed to retrieve `CGImage` analogue of `UIImage`.")     }     CGImageDestinationAddImage(destination, cgImage, properties)     CGImageDestinationFinalize(destination)             // Save to the photo library.     PHPhotoLibrary.shared().performChanges({       guard let creationRequest: PHAssetChangeRequest = .creationRequestForAssetFromImage(atFileURL: URL(fileURLWithPath: filePath)) else {         return       }       // Add metadata to the photo.       creationRequest.creationDate = .init()       if let location=location {         creationRequest.location = .init(latitude: location.latitude, longitude: location.longitude)       }     }, completionHandler: { _, _ in       try? FileManager.default.removeItem(atPath: tmpUrl.absoluteString)     })   } } If anyone can provide some insight as to what's causing the iosurface is too large error and what can be done to resolve it, that'd be awesome.
3
0
1.9k
Jan ’23
MPMusicPlayerController on watchOS
Hi, I’m looking at using MusicKit in my watchOS app however I don’t seem to have any method of being able to play the audio though the recommended use of MPMusicPlayerController since it isn’t available on watchOS. This method works fine for iOS and iPadOS but not watchOS which seems bizarre considering we have full access to MusicKit but no way to actually play any audio. I’m trying to build an app that includes Apple Music through MusicKit but don’t have any way to actually play the audio. Is there a technical reason for this and if so is there any other way to play audio from MusicKit on watchOS. The docs for MPMusicPlayerController can be found here: https://developer.apple.com/documentation/mediaplayer/mpmusicplayercontroller
1
1
830
Feb ’23
ApplicationMusicPlayer macOS vs. Catalyst
ApplicationMusicPlayer is available on the Mac! 🎉🎉🎉 Enormous thanks to @JoeKun and the team. I've already gotten my app up and running through Catalyst, and I've successfully played music! I also got some timeouts, but that was happening on my phone a lot that day too, so maybe my local CDN was just having a bad day. I wanted to ask this question in a lab this week, but the timing didn't work out: Do you expect the experience to be the same using ApplicationMusicPlayer on a Catalyst vs a macOS target? I'm hoping to reuse much of my iPad app and go the Catalyst route, but I wanted to double check that the new support wasn't just for macOS.
2
1
628
Jun ’23
Issue with local music files on iOS 17
Hi! I'm currently developing an app that can play music stored locally. It was working fine previously, but after updating my device to iOS 17, I started getting error -54 when I try to play the file. I also noticed that when getting the list of files in MPMediaQuery.songs(), I would encounter the following errors: I suspect it might be some issue with file permissions, but I can't figure out what i am missing. I have already checked that MPMediaLibrary.authorizationStatus() is authorized. Does anyone know what the issue might be? Thank you
1
1
1.1k
Jul ’23
Terrible performance when using MusicKit's Artwork with UIKit
I'm building a UIKit app that reads user's Apple Music library and displays it. In MusicKit there is the Artwork structure which I need to use to display artwork images in the app. Since I'm not using SwiftUI I cannot use the ArtworkImage view that is recommended way of displaying those images but the Artwork structure has a method that returns url for the image which can be used to read the image. The way I have it setup is really simple: extension MusicKit.Song { func imageURL(for cgSize: CGSize) -> URL? { return artwork?.url( width: Int(cgSize.width), height: Int(cgSize.height) ) } func localImage(for cgSize: CGSize) -> UIImage? { guard let url = imageURL(for: cgSize), url.scheme == "musicKit", let data = try? Data(contentsOf: url) else { return nil } return .init(data: data) } } Now, everytime I access .artwork property (so a lot of times) the main thread gets blocked and the console output gets bombared with messages like these: 2023-07-26 11:49:47.317195+0200 Plum[998:297199] [Artwork] Failed to create color analysis for artwork: <MPMediaLibraryArtwork: 0x289591590> with error; Error Domain=NSCocoaErrorDomain Code=4099 "The connection to service named com.apple.mediaartworkd.xpc was invalidated: failed at lookup with error 159 - Sandbox restriction." UserInfo={NSDebugDescription=The connection to service named com.apple.mediaartworkd.xpc was invalidated: failed at lookup with error 159 - Sandbox restriction.} 2023-07-26 11:49:47.317262+0200 Plum[998:297199] [Artwork] Failed to create color analysis for artwork: file:///var/mobile/Media/iTunes_Control/iTunes/Artwork/Originals/4b/48d7b8d349d2de858413ae4561b6ba1b294dc7 2023-07-26 11:49:47.323099+0200 Plum[998:297013] [Plum] IIOImageWriteSession:121: cannot create: '/var/mobile/Media/iTunes_Control/iTunes/Artwork/Caches/320x320/4b/48d7b8d349d2de858413ae4561b6ba1b294dc7.sb-f9c7943d-6ciLNp'error = 1 (Operation not permitted) My guess is that the most performance-heavy task here is performing the color analysis for each artwork but IMO the property backgroundColor should not be a stored property if that's the case. I am not planning to use it anywhere and if so it should be a computed async property so it doesn't block the caller. I know I can move the call to a background thread and that fixes the issue of blocking main thread but still the loading times for each artwork are terribly slow and that impacts the UX. SwiftUI's ArtworkImage loads the artworks much quicker and without the errors so there must be a better way to do it.
4
0
1.1k
Jul ’23
Crossfade with MPMusicPlayerController.applicationQueuePlayer
Hi. In iOS 17 Apple introduced crossfading for the Music app. In my app I am using MPMusicPlayerController.applicationQueuePlayer and also want to support crossfading. In an early beta of iOS 17 my app crossfaded automatically with the same setting as for the Music app which maybe was a bug. However, I did not find any way to enable crossfade for my app. Does anybody know how to do that? Thanks, Dirk
1
0
830
Aug ’23
Share Extension - App no longer showing after Xcode/IOS Update
I recently updated to Xcode 15 and IOS 17 and now my Share Extension is no longer appearing. The share extension is meant to take the URL from a song when a user selects share in Apple Music. It was working perfectly before the update, so I'm wondering if I missed something that would cause this error or if this is happening or has happened to anyone else. If so, what's the best way to fix it? Here's my Activation Rule for reference: <dict> <key>NSExtension</key> <dict> <key>NSExtensionAttributes</key> <dict> <key>NSExtensionActivationRule</key> <dict> <key>NSExtensionActivationSupportsWebURLWithMaxCount</key> <integer>1</integer> </dict> </dict> <key>NSExtensionMainStoryboard</key> <string>MainInterface</string> <key>NSExtensionPointIdentifier</key> <string>com.apple.share-services</string> </dict> </dict>
0
0
644
Oct ’23
Beginner trying to authorize Apple Music API request
I'm a new programmer and I'm trying to work with the Apple Music API as a way to teach myself. So this is probably a dumb question, but I'm using the docs to figure out how to make a request, and I think I have the token all set with the key ID and UNIX time and all that. But I get a 401 Unauthorized back when I send a GET request with it, and I'm having trouble understanding what the docs say about authorization. It says to pass Authorization: Bearer in the header. I'm trying to do that in Insomnia but it must be over my head. I'm guessing there's something obvious that I'm doing wrong, but I'm not sure what. Hoping someone here will take one look at it and be able to tell me. I tried it a few different ways but this is the gist.
2
0
529
Oct ’23
iOS 17 Problems in getting RGB color from image
I have written and used the code to get the colors from CGImage and it worked fine up to iOS16. However, when I use the same code in iOS17, Red and Blue out of RGB are reversed. Is this a temporary bug in the OS and will it be fixed in the future? Or has the specification changed and will it remain this way after iOS17? Here is my code: let pixelDataByteSize = 4 guard let cfData = image.cgImage?.dataProvider?.data else { return } let pointer:UnsafePointer = CFDataGetBytePtr(cfData) let scale = UIScreen.main.nativeScale let address = ( Int(image.size.width * scale) * Int(image.size.height * scale / 2) + Int(image.size.width * scale / 2) ) * pixelDataByteSize let r = CGFloat(pointer[address]) / 255 let g = CGFloat(pointer[address+1]) / 255 let b = CGFloat(pointer[address+2]) / 255
1
0
728
Oct ’23
WebP image in Asset Catalog?
It appears I can't add a WebP image as an Image Set in an Asset Catalog. Is that correct? As a workaround, I added the WebP image as a Data Set. I'm then loading it as a CGImage with the following code: guard let asset = NSDataAsset(name: imageName), let imageSource = CGImageSourceCreateWithData(asset.data as CFData, nil), let image = CGImageSourceCreateImageAtIndex(imageSource, 0, nil) else { return nil } // Use image Is it fine to store and load WebP images in this way? If not, then what's best practice?
0
0
1.2k
Oct ’23
Create Media Player Application guide
I have designed an app for media player, in this app i need to implement live tv, movies and series. so url can be of any type such as .ts formate for live tv, and .mp4, .mov, etc. I am also going to work with m3u. but AVPlayer does not supports all these urls.So can i get some suggestions and solutions for that. what could be the best practice and how to work with all these kind if urls etc.
0
0
487
Oct ’23
RPScreenRecorder: Recording video
Hello everyone. I am trying to make a small utility that, in the context of digital forensics, logs the desktop. The utility is to be started via shell like this : "./nemeapp start path_to_file" and be terminated in "./nemeapp stop". The code I wrote is: import Foundation import ReplayKit let arguments = CommandLine.arguments guard arguments.count == 4 else { print("Utilizzo: nome_script start|stop percorso_file include_audio(true|false)") exit(0) } let command = arguments[1] let filePath = arguments[2] let includeAudio = arguments[3] == "true" switch command { case "start": startScreenRecording(filePath: filePath, includeAudio: includeAudio) case "stop": stopScreenRecording() default: print("Comando non riconosciuto. Utilizzo: nome_script start|stop percorso_file include_audio(true|false)") } func startScreenRecording(filePath: String, includeAudio: Bool) { if RPScreenRecorder.shared().isAvailable { RPScreenRecorder.shared().startRecording(handler: { error in if let unwrappedError = error { print("Errore durante l'avvio della registrazione: \(unwrappedError.localizedDescription)") } else { print("La registrazione dello schermo è stata avviata correttamente. Il file verrà salvato in: \(filePath)") } }) } else { print("La registrazione dello schermo non è disponibile.") } } func stopScreenRecording() { RPScreenRecorder.shared().stopRecording { previewViewController, error in if let unwrappedError = error { print("Errore durante l'arresto della registrazione: \(unwrappedError.localizedDescription)") } else { print("La registrazione dello schermo è stata interrotta correttamente.") } } } Unfortunately, the code returns no error message. Only when I give the stop command does it tell me that the registration never started. I can't even figure out if it is a permissions issue.
0
0
549
Oct ’23
MPRemoteCommandCenter likeCommand and dislikeCommand
I'm trying to show likeCommand and dislikeCommand on the Lock Screen of a music player without success. Are they still supported ? Is there any special configuration on the player or the track for them to show? My current code which works for playCommand looks like this MPRemoteCommandCenter.shared().likeCommand.addTarget { [unowned self] _ in if isPlaying { return .success } return .commandFailed }
0
0
573
Oct ’23