How to capture the audio being played by apple music in ios and combine it with fft to achieve audio visualization?

I am building an app for MacOS and I am trying to implement the code to add songs to a library playlist (which is added below). The issue I am having is that if I use Music Kit to load a users library playlists, the ID for the playlist (which is just a string of numbers) does not work with the Add tracks to a Library Playlist endpoint of Apple Music API. If I retrieve the playlists from the Apple Music API and use that playlist ID (which is different than the id I get from MusicKit) my code works fine and adds the song to the playlist. The problem is that when getting a users library playlists from Apple Music API is that it does not give me all of the library playlists that I get when using Music Kit and it also does not give me Artwork for playlists that have the collage of album covers, so I would prefer to use Music Kit to get the playlists. I have also tested trying to retrieve a single playlist using the Apple Music API with the playlist Id from Music Kit and it does not work. I get the error that the resource cannot be found. Since this is a macOs app I cannot use MusicKit to add songs to library playlists. Does anyone know a way to resolve this? Or a possible workaround? Ideally I want to use MusicKit to get the library playlists and have some way to use the playlist Id and add songs to that playlist. Below is my code for adding a song to a playlist using the Apple Music API, which works correctly only if I originally get the library playlist's id value from a playlist retrieved from the Apple Music API. Also, does anyone know why the playlist Id's are not universal and are different when using Music Kit and Apple Music API? For songs and tracks it does not seem to matter if I use music kit or Apple Music API, the Id's are in the correct format for Apple Music API to use and work with my code. Thanks everyone for any and all help! func addToPlaylist(songs: [Track], playlist: Playlist, alert: Binding<AlertItem?>) async { let tracks = AppleMusicPlaylistPostRequestBody(data: songs.compactMap { AppleMusicPlaylistPostRequestItem(id: $0.id.rawValue, type: "songs") // or "library-songs" }) let playlistID = playlist.id // Build the request URL for adding a song to a playlist guard let url = URL(string: "https://api.music.apple.com/v1/me/library/playlists/\(playlistID)/tracks") else { alert.wrappedValue = AlertItem(title: "Error", message: "Invalid URL for the playlist.") return } // Authorization Header guard let musicUserToken = try? await MusicUserTokenProvider().getUserMusicToken() else { alert.wrappedValue = AlertItem(title: "Error", message: "Unable to retrieve Music User Token.") return } do { var request = URLRequest(url: url) request.httpMethod = "POST" request.setValue("Bearer \(musicUserToken)", forHTTPHeaderField: "Authorization") request.setValue("application/json", forHTTPHeaderField: "Content-Type") let encoder = JSONEncoder() let data = try encoder.encode(tracks) request.httpBody = data let musicRequest = MusicDataRequest(urlRequest: request) let musicRequestResponse = try await musicRequest.response() // Check if the request was successful (status 201) if musicRequestResponse.urlResponse.statusCode == 201 { alert.wrappedValue = AlertItem(title: "Success", message: "Song successfully added to the playlist.") } else { print("Status Code: \(musicRequestResponse.urlResponse.statusCode)") print("Response Data: \(String(data: musicRequestResponse.data, encoding: .utf8) ?? "No Data")") // Attempt to decode the error response into the AppleMusicErrorResponse model if let appleMusicError = try? JSONDecoder().decode(AppleMusicErrorResponse.self, from: musicRequestResponse.data) { let errorMessage = appleMusicError.errors.first?.detail ?? "Unknown error occurred." alert.wrappedValue = AlertItem(title: "Error", message: errorMessage) } else { alert.wrappedValue = AlertItem(title: "Error", message: "Failed to add song to the playlist.") } } } catch { alert.wrappedValue = AlertItem(title: "Error", message: "Network error: \(error.localizedDescription)") } }

https://developer.apple.com/documentation/WidgetKit/Adding-interactivity-to-widgets-and-Live-Activities#Add-a-toggle Before explaining the situation, I referred to this document. I'm making a widget with a toggle that works with Intent. (To be precise, it's Live Activity, but Apple's literally toggling and button interaction are implemented in the same way) If you press the toggle when the device is locked, the toggle represents the state. However, as described at the top of the same document, the device must be unlocked to execute Intent, so I can't actually change the data. That's fine, but how can I return the isOn of the toggle to false? I'm blocked here because no code is executed. If it is a widget in the home screen, it will be basically unlocked, so I can proceed according to the method of the document, but if it is today's view, user can access it even if device is locked. I tested it in today's view with the Reminders app, If I don't unlock the device after tapping the toggle it returns the toggle back to false state. I wonder how I can achieve this. Is there an API that I missed? Summary. Tap the toggle of the widget while the device is locked. The device waits for unlocking, but the user cancels it without unlocking it. (Still locked) Return the isOn of the toggle to false. <- The part I want.

Hi all, I am working with MusicKit and have managed to get a user's recommendation playlists (favorites, get up, etc). Isolating just one, I am trying to access the tracks in there using .tracks but it returns nothing. Wondering if I am doing something wrong or if these don't have tracks necessarily associated with them. Code is below (not everything just kinda where I am accessing the playlist) try await requestMusicAuthorization() let request = MusicPersonalRecommendationsRequest() let response = try await request.response() guard let madeForYou = response.recommendations.first(where: { $0.title == "Made for You" }) else { throw NSError(domain: "RecommendationError", code: 0, userInfo: [NSLocalizedDescriptionKey: "No 'Made for You' recommendation found."]) } guard let firstPlaylist = madeForYou.playlists.first else { throw NSError(domain: "PlaylistError", code: 1, userInfo: [NSLocalizedDescriptionKey: "No playlists found in 'Made for You' recommendation."]) } print("Fetching tracks for playlist: \(firstPlaylist.name ?? "Unknown")") let playlistRequest = MusicCatalogResourceRequest<Playlist>(matching: \.id, equalTo: firstPlaylist.id) let playlistResponse = try await playlistRequest.response() guard let playlist = playlistResponse.items.first else { throw NSError(domain: "PlaylistError", code: 2, userInfo: [NSLocalizedDescriptionKey: "Couldn't fetch the playlist details."]) } guard let tracks = playlist.tracks else { throw NSError(domain: "PlaylistError", code: 3, userInfo: [NSLocalizedDescriptionKey: "No tracks found in the playlist."]) }

Hi, I've recently been working with the Apple Music API and have had success in loading all the playlists on my account, loading songs from each playlist, and adding songs to the ApplicationMusicPlayer.share.queue. The problem I'm running into is that not all songs from the playlist are being added to the queue, despite confirming all the songs are being based on the PlaybackView.swift I'm about to share with you. I would also like to answer other underlying questions if possible. I am also open to any other suggestions. In this scenario were also assuming isShuffled is true every time. How can I determine when a song has ended? How can I get the album title information? How can I get the current song title, album information, and artist information without pressing play? I can only seem to update the screen when I select my play meaning ApplicationMusicPlayer.shared.play() is being called. How do I get the endTime of the song? I believe it should be ApplicationMusicPlayer.shared.queue.currentEntry.endTime but this doesn't seem to work. // // PlayBackView.swift // // Created by Justin on 8/16/24. // import SwiftUI import MusicKit import Foundation enum PlayState { case play case pause } struct PlayBackView: View { @State var song: Track @Binding var songs: [Track]? @State var isShuffled = false @State private var playState: PlayState = .pause @State private var isFirstPlay = true private let player = ApplicationMusicPlayer.shared private var isPlaying: Bool { return (player.state.playbackStatus == .playing) } var body: some View { VStack { // Album Cover HStack(spacing: 20) { if let artwork = player.queue.currentEntry?.artwork { ArtworkImage(artwork, height: 100) } else { Image(systemName: "music.note") .resizable() .frame(width: 100, height: 100) } VStack(alignment: .leading) { // Song Title Text(player.queue.currentEntry?.title ?? "Song Title Not Found") .font(.title) .fixedSize(horizontal: false, vertical: true) // How do I get AlbumTitle from here?? // Artist Name Text(player.queue.currentEntry?.subtitle ?? "Artist Name Not Found") .font(.caption) } } .padding() Spacer() // Progress View // endTime doesn't work and not sure why. ProgressView(value: player.playbackTime, total: player.queue.currentEntry?.endTime ?? 1.00) .progressViewStyle(.linear) .tint(.red.opacity(0.5)) // Duration View HStack { Text(durationStr(from: player.playbackTime)) .font(.caption) Spacer() if let duration = player.queue.currentEntry?.endTime { Text(durationStr(from: duration)) .font(.caption) } } Spacer() Button { Task { do { try await player.skipToNextEntry() } catch { print(error.localizedDescription) } } } label: { Label("", systemImage: "forward.fill") .tint(.white) } Spacer() // Play/Pause Button Button(action: { handlePlayButton() isFirstPlay = false }, label: { Text(playState == .play ? "Pause" : isFirstPlay ? "Play" : "Resume") .frame(maxWidth: .infinity) }) .buttonStyle(.borderedProminent) .padding() .font(.largeTitle) .tint(.red) } .padding() .onAppear { if isShuffled { songs = songs?.shuffled() if let songs, let firstSong = songs.first { player.queue = .init(for: songs, startingAt: firstSong) player.state.shuffleMode = .songs } } } .onDisappear { player.stop() player.queue = [] player.playbackTime = .zero } } private func handlePlayButton() { Task { if isPlaying { player.pause() playState = .pause } else { playState = .play await playTrack() } } } @MainActor private func playTrack() async { do { try await player.play() } catch { print(error.localizedDescription) } } private func durationStr(from duration: TimeInterval) -> String { let seconds = Int(duration) let minutes = seconds / 60 let remainder = seconds % 60 // Format the string to ensure two digits for the remainder (seconds) return String(format: "%d:%02d", minutes, remainder) } } //#Preview { // PlayBackView() //}

Since the iOS 18 update, there's been a bug that always occurs when listening to music through Apple Music. When music is playing and the iPhone enters or exits standby mode, the music pauses by itself for 1 second. The initial conditions were the same as when this problem didn’t exist.

I have a recent post kind of outlining a similar question here. This time though I'm confident that inserting an array of Track works when inserting into the ApplicationMusicPlayer.shared.queue but now I'm not sure how I can initialize the queue to display song title and artwork for example. I'm also not sure how to get the current item in the queue's artist information and album information which I feel should be easy to do so maybe I'm missing something obvious. Hope this paints of what I'm trying to do and I'm going to post the neccessary code here to help me debug/figure out this problem. import SwiftUI import MusicKit struct PlayBackView: View { @Environment(\.scenePhase) var scenePhase @Environment(\.openURL) private var openURL // Adding Enum Here for Question Sake enum PlayState { case play case pause } @State var song: Track @Binding var songs: [Track]? @State var isShuffled = false @State private var playState: PlayState = .pause @State private var songTimer: Int = Int.random(in: 5...30) @State private var roundTimer: Int = 5 @State private var isTimerActive = false // @State private var volumeValue = VolumeObserver() @State private var isFirstPlay = true @State private var isDancing = false @State private var player = ApplicationMusicPlayer.shared private var isPlaying: Bool { return (player.state.playbackStatus == .playing) } let timer = Timer.publish(every: 1, on: .main, in: .common).autoconnect() var playPauseImage: String { switch playState { case .play: "pause.fill" case .pause: "play.fill" } } var body: some View { VStack { // Album Cover HStack(spacing: 20) { if let artwork = player.queue.currentEntry?.artwork { ArtworkImage(artwork, height: 100) } else { Image(systemName: "music.note") .resizable() .frame(width: 100, height: 100) } VStack(alignment: .leading) { /* This is where I want to display song title, album title, and artist name for example */ // Song Title Text(player.queue.currentEntry?.title ?? "Unable to Find Song Title") .font(.title) .fixedSize(horizontal: false, vertical: true) // Album Title // Text(player.queue.currentEntry ?? "Album Title Not Found") // .font(.caption) // .fixedSize(horizontal: false, vertical: true) // I don't know what the subtitle actually grabs Text(player.queue.currentEntry?.subtitle ?? "Artist Name Not Found") .font(.caption) } } .padding() // Play/Pause Button Button(action: { handlePlayButton() isFirstPlay = false }, label: { Text(playState == .play ? "Pause" : isFirstPlay ? "Play" : "Resume") .frame(maxWidth: .infinity) }) .buttonStyle(.borderedProminent) .padding() .font(.largeTitle) .tint(.red) } .padding() // Maybe I should use the `.task` modifier here? .onAppear { // I'm sure this code could be improved but don't think it'll help answer the question at the moment. Task { if let songs = songs { do { if isShuffled { let shuffledSongs = songs.shuffled() try await player.queue.insert(shuffledSongs, position: .tail) handlePlayButton() } else { try await player.queue.insert(songs, position: .tail) } } catch { print(error.localizedDescription) } } } } } private func handlePlayButton() { Task { if isPlaying { player.pause() playState = .pause isTimerActive = false } else { playState = .play await playTrack() isTimerActive = true } } } @MainActor private func playTrack() async { do { try await player.play() } catch { print(error.localizedDescription) } } } //#Preview { // PlayBackView() //}

Hi there, After upgrading to iOS 18, I noticed that ApplicationMusicPlayer.Queue behavior has broken if at least one song that is added to the queue is also in to the Apple Music Library on the device. The resulting behavior is that the queue does not accept all the items, and only items that are in the Library are playable in the queue. The expected behavior and the previous behavior on iOS 17 was that all the items would be added to the queue successfully. I confirmed this behavior on a separate test device running iOS 17.7. The items added are all being fetched via MusicCatalogResourceRequest<Song> so I would expect that a requested song being present in the library would have no effect.

I have an app on which users learn choreography. To avoid copyright infringements we currently only have audio instructions and no music on the app. Could we enable those that are subscribed to Apple Music to listen to the part of a song the corresponds to the choreography? Usually they are 60 seconds long. The app is in React Native. Would it be possible to implement it so that opening a dance video automatically triggers the playback of that song from e.g. second 32 - 95? Since the video is looping, could it then start playing from second 32 again? Also looking for devs with experience in integrating the MusicKit for this usecase if it turns out to be possible.

Hi, In my app I am using MusicLibraryRequest<Artist> to fetch all of the artists in someone's Library collection. With this response I then fetch each artists albums: artist.with([.album]). The response from this only gives albums in the users Library collection. I would like to augment it with all of the albums for an artist from the full catalogue. I'm using MusicKit and targeting iOS18 and visionOS 2. Could someone please point me towards the best way to approach this?

It's simple to reproduce. The bug is simply when you queue a bunch of songs to play, it will always queue less than what you gave it. Here, I'm attempting to play an apple curated playlist, it will only queue a subset, usually less than 15, but as low as 1 out of 100. Use the system's forward and backwards to test it out. Here is the code, just paste it in to the ContentView file and make sure you have the capibility to run it. import SwiftUI import MusicKit struct ContentView: View { var body: some View { VStack{ Button("Play Music") { Task{ await playMusic() } } } } } func getOnlySongsFromTracks(tracks:MusicItemCollection<Track>?) async throws ->MusicItemCollection<Song>?{ var songs:[Song]? if let t = tracks{ songs = [Song]() for track in t { if case let .song(song) = track { songs?.append(song) print("track is song \(track.debugDescription)") }else{ print("track not song \(track.debugDescription)") } } } if let songs = songs { let topSongs = MusicItemCollection(songs) return topSongs } return nil } func playMusic() async { // Request authorization let status = await MusicAuthorization.request() guard status == .authorized else { print("Music authorization denied.") return } do { // Perform a hardcoded search for a playlist let searchTerm = "2000" let request = MusicCatalogSearchRequest(term: searchTerm, types: [Playlist.self]) let response = try await request.response() guard let playlist = response.playlists.first else { print("No playlists found for the search term '\(searchTerm)'.") return } // Fetch the songs in the playlist let detailedPlaylist = try await playlist.with([.tracks]) guard let songCollection = try await getOnlySongsFromTracks(tracks: detailedPlaylist.tracks) else { print("no songs found") return } guard let t = detailedPlaylist.tracks else { print("no tracks") return } // Create a queue and play let musicPlayer = ApplicationMusicPlayer.shared let q = ApplicationMusicPlayer.Queue(for: t) musicPlayer.queue = q try await musicPlayer.play() print("Now playing playlist: \(playlist.name)") } catch { print("An error occurred: \(error.localizedDescription)") } }

I use Universal Product Codes (UPC) in my app to reliably identify albums after having used albumIDs for a time. AlbumIDs can change over time for no obvious reasons (see here for songIDs) so I switched to UPCs since I believed they cannot change. Well apparently they can. A few days ago I populated a JSON with UPCs including 196871067713. Today trying to perform a MusicCatalogResourchRequest for the UPC does not return anything. When using that UPC and putting it into an Apple Music link like https://music.apple.com/de/album/folge-89-im-geistergarten/1683337782?l=en-GB redirects to https://music.apple.com/de/album/folge-89-im-geistergarten/1683337782?l=en-GB so I assume the UPC has changed from 196871067713 to 1683337782. Apple Music can handle that and redirects to the new upc both in the app and as a website. But a MusicCatalogResourceRequest cannot do that. I filed a suggestion for that (FB15167146) but I need a solution quicker. Can I somehow detect where the URL is redirecting to? Is there a way MusicCatalogResourceRequest can do this? Performing a MusicCatalogSearchRequest can be an option but seems unreliable when using the title as search term. Other ideas? Thank you

Hi, I’ve encountered a bug related to including tracks as a relationship in the playlists list. The issue arises when there is more than one playlist. Specifically: Single Playlist: The functionality works as expected. Multiple Playlists: The application crashes. Please let me know if you need additional information or if there are any updates on this issue. Thank you! curl --request GET \ --url 'https://api.music.apple.com/v1/me/library/playlists?include=tracks' \ --header 'Authorization: Bearer {token}' \ --header 'Music-User-Token: {token}'

How can I search for a single song by using its song ID? I tried the following code but it's not working: MusicCatalogResourceRequest(matching: Song.self, equalTo: song.id.rawValue) I get the following errors in Xcode: Cannot convert value of type 'Song.Type' to expected argument type 'KeyPath<MusicItemType.FilterType, String>' Generic parameter 'MusicItemType' could not be inferred I have the song ID saved as a string inside song so I just want to grab the full MusicKit MusicItemType from the API. I looked at the documentation but it doesn't make any sense to me. Please help!

I want to use the Shortcuts app on macOS Monterey to modify M4A/AAC audio file metadata for files that I have ripped from CD to my local Music app library. How can I use a regular expression on track titles to replace text that matches a certain regex pattern? I have already written the regex, I just need help with the Shortcuts app. I've started my shortcut with a "Find Music" action connected to a "Repeat with Each" loop action. Within the loop, I use a "Get Details of Music" action to get the title, which is passed to a "Replace Text" action using my regex. I just don't know how to write the correct new title into the music file's title/name metadata field. I tried a "Rename File" action with type "iTunes media" to try to set the "Title" field, but the field wasn't updated either in the Music app or in the M4A/AAC file itself. I couldn't find any other actions that seemed like they might be able to modify the field. I'd prefer a solution that goes through the Music app / its APIs, rather than directly modifying the M4A on the file system, because the latter would force me to have the Music app scan for changed files after the files we're modified.

I'm trying to create an app that uses the Apple Music API and while I can fetch playlists as I desire when selecting a song from a playlist the music does not play. First I'm getting the playlists, then showing those playlists on a list in a view. Then pass that "song" which is a Track type into a PlayBackView. There are several UI components in that view but I want to boil it down here for simplicity's sake to better understand my problem. struct PlayBackView: View { @State private var playState: PlayState = .pause private let player = ApplicationMusicPlayer.shared @State var song: Track private var isPlaying: Bool { return (player.state.playbackStatus == .playing) } var body: some View { VStack { AsyncImage(url: song.artwork?.url(width: 100, height: 100)) { image in image .resizable() .frame(maxWidth: .infinity) .aspectRatio(1.0, contentMode: .fit) } placeholder: { Image(systemName: "music.note") .resizable() .frame(width: 100, height: 100) } // Song Title Text(song.title) .font(.title) // Album Title Text(song.albumTitle ?? "Album Title Not Found") .font(.caption) // Play/Pause Button Button(action: { handlePlayButton() }, label: { Image(systemName: playPauseImage) }) .padding() .foregroundStyle(.white) .font(.largeTitle) Image(systemName: airplayImage) .font(ifDeviceIsConnected ? .largeTitle : .title3) } .padding() } private func handlePlayButton() { Task { if isPlaying { player.pause() playState = .play } else { playState = .pause await playTrack(song: song) } } } @MainActor public func playTrack(song: Track) async { do { try await player.play() playState = .play } catch { print(error.localizedDescription) } } } These are the errors I'm seeing printing in the console in Xcode prepareToPlay failed [no target descriptor] The operation couldn’t be completed. (MPMusicPlayerControllerErrorDomain error 1.) ASYNC-WATCHDOG-1: Attempting to wake up the remote process ASYNC-WATCHDOG-2: Tearing down connection

Hi, I want to embed song previews in my app when a user shares an apple music link. While using MusikKit getting previews seems to require access to the iCloud Music library, even though I‘m not shure how this is related, since I‘m not acessing user data. Is there any possibility that makes the preview work while neither requiring access to the users library nor a signed jwt?

I updated to iOS 18.4 Beta for Developers today on my iPhone 14 Pro Max. It is now acting weird. At first my Apple Music would stop playing whenever I touched my phone. Now it is only doing it whenever I unlock the phone. I have tried restarting the phone. Turning Bluetooth on/off. Tried different Bluetooth connected device. Played music that is download and then streamed. I turned WiFi off and back on again. I am at a loss. I have been researching tips and tricks for hours. Am I truly only left with formatting my phone at this point so it resets to iOS 17?

I wanted a platform that combined apple music and podcasts, so I built one. I'm looking for beta testers currently, please reply if you're interested.

Hi, since the IoT cybersecurity, I need to hide the version of all sevices. For the 7000 port, AirTunes, I can't find the method to hide it. Like tomcat version hide, how can I hide the version of it?