AVSpeechSynthesizer don't speak on didReceive Notification Service Extension on Device(Background Mode)

My Code

private let synthesizer = AVSpeechSynthesizer()

override func didReceive(_ request: UNNotificationRequest, withContentHandler contentHandler: @escaping (UNNotificationContent) -> Void) {
		self.contentHandler = contentHandler
		bestAttemptContent = (request.content.mutableCopy() as? UNMutableNotificationContent)

		do {
			try AVAudioSession.sharedInstance().setCategory(AVAudioSession.Category.playback)

			let utterance = AVSpeechUtterance(string: "Hello Sim")
			utterance.voice = AVSpeechSynthesisVoice(language: "th_TH")
			utterance.rate = 1.0
			utterance.pitchMultiplier = 1.0
			utterance.volume = 1.0
			utterance.preUtteranceDelay = 0
			self.synthesizer.usesApplicationAudioSession = false
			self.synthesizer.speak(utterance)
		} catch {
			print(error.localizedDescription)
		}

		if let bestAttemptContent = self.bestAttemptContent {
			contentHandler(bestAttemptContent)
		}
}

Info.plist enable UIBackgroundModes

  • audio
  • fetch
  • processing
  • remote-notification

payloadNotification

{
"aps":{
           "alert":{
"title":"title"
"subtitle":"subtitle"
"body":"body"
}
"mutable-content":1
"content-available":1
"sound":"tester.caf"
}
}

This code can play in simulator but can't play in real device

Answered by Engineer in 805495022

You cannot play audio directly from a Notification Service Extension. I am not sure how you are able to play that sound in the simulator, but you can consider that an anomaly.

If you want the notification sound to contain your utterance, you will need to save it to a file (in your app/extension's shared container), and then change the payload of the notification so the "sound" entry is pointing to that saved file.

To do this, you would use AVSpeechSynthesizer's write(_:toBufferCallback:) function. This will call back multiple times, during which you can concatenate the buffered audio and write it as a file using the AVAudioFile's write(from:) function.

Once you have the complete audio in a file, then you can change the payload's sound entry to point to that file.

You would save this file in the /Library/Sounds directory of app’s and the extension's shared group container.

Once the new payload pointing to the synthesized file is handled by the notification system, the sound in that file will be played instead.

References: https://developer.apple.com/documentation/avfaudio/avspeechsynthesizer/3141659-write

https://developer.apple.com/documentation/avfaudio/avaudiofile/1385637-write

https://developer.apple.com/documentation/usernotifications/unnotificationsound


Argun Tekant /  DTS Engineer / Core Technologies

You cannot play audio directly from a Notification Service Extension. I am not sure how you are able to play that sound in the simulator, but you can consider that an anomaly.

If you want the notification sound to contain your utterance, you will need to save it to a file (in your app/extension's shared container), and then change the payload of the notification so the "sound" entry is pointing to that saved file.

To do this, you would use AVSpeechSynthesizer's write(_:toBufferCallback:) function. This will call back multiple times, during which you can concatenate the buffered audio and write it as a file using the AVAudioFile's write(from:) function.

Once you have the complete audio in a file, then you can change the payload's sound entry to point to that file.

You would save this file in the /Library/Sounds directory of app’s and the extension's shared group container.

Once the new payload pointing to the synthesized file is handled by the notification system, the sound in that file will be played instead.

References: https://developer.apple.com/documentation/avfaudio/avspeechsynthesizer/3141659-write

https://developer.apple.com/documentation/avfaudio/avaudiofile/1385637-write

https://developer.apple.com/documentation/usernotifications/unnotificationsound


Argun Tekant /  DTS Engineer / Core Technologies

AVSpeechSynthesizer don't speak on didReceive Notification Service Extension on Device(Background Mode)
 
 
Q