Performance issues with `AVAssetWriter`

Hey all!

I'm trying to build a Camera app that records Video and Audio buffers (AVCaptureVideoDataOutput and AVCaptureAudioDataOutput) to an mp4/mov file using AVAssetWriter.

When creating the Recording Session, I noticed that it blocks for around 5-7 seconds before starting the recording, so I dug deeper to find out why.

This is how I create my AVAssetWriter:

let assetWriter = try AVAssetWriter(outputURL: tempURL, fileType: .mov)
let videoWriter = self.createVideoWriter(...)
assetWriter.add(videoWriter)
let audioWriter = self.createAudioWriter(...)
assetWriter.add(audioWriter)
assetWriter.startWriting()

There's two slow parts here in that code:

  1. The createAudioWriter(...) function takes ages!

    This is how I create the audio AVAssetWriterInput:

    // audioOutput is my AVCaptureAudioDataOutput, audioInput is the microphone
    let settings = audioOutput.recommendedAudioSettingsForAssetWriter(writingTo: .mov)
    let format = audioInput.device.activeFormat.formatDescription
    
    let audioWriter = AVAssetWriterInput(mediaType: .audio,
                                         outputSettings: settings,
                                         sourceFormatHint: format)
    audioWriter.expectsMediaDataInRealTime = true
    

    The above code takes up to 3000ms on an iPhone 11 Pro!

    When I remove the recommended settings and just pass nil as outputSettings:

    audioWriter = AVAssetWriterInput(mediaType: .audio,
                                     outputSettings: nil)
    audioWriter.expectsMediaDataInRealTime = true
    

    ...It initializes almost instantly - something like 30 to 50ms.

  2. Starting the AVAssetWriter takes ages!

    Calling this method:

    assetWriter.startWriting()
    

    ...takes takes 3000 to 5000ms on my iPhone 11 Pro!

Does anyone have any ideas why this is so slow? Am I doing something wrong?

It feels like passing nil as the outputSettings is not a good idea, and recommendedAudioSettingsForAssetWriter should be the way to go, but 3 seconds initialization time is not acceptable.

Here's the full code: RecordingSession.swift from react-native-vision-camera. This gets called from here.

I'd appreciate any help, thanks!

Some additional information:

  1. Regarding the audio AVAssetWriter.init(...) call taking 3 seconds:

    These are the recommendedVideoSettingsForAssetWriter settings I get:

    ["AVVideoCompressionPropertiesKey": {
      AllowFrameReordering = 1;
      AllowOpenGOP = 1;
      AverageBitRate = 47848572;
      BaseLayerFrameRate = 15;
      ExpectedFrameRate = 60;
      MaxAllowedFrameQP = 41;
      MaxKeyFrameIntervalDuration = 1;
      MinAllowedFrameQP = 15;
      MinimizeMemoryUsage = 1;
      Priority = 80;
      ProfileLevel = "HEVC_Main_AutoLevel";
      RealTime = 1;
      RelaxAverageBitRateTarget = 1;
     },
     "AVVideoWidthKey": 2160,
     "AVVideoCodecKey": hvc1,
     "AVVideoHeightKey": 3840]
    

    (video initializes in ~10ms)

    And these are the recommendedAudioSettingsForAssetWriter(..) I get:

    ["AVEncoderQualityForVBRKey": 91,
     "AVEncoderBitRatePerChannelKey": 96000,
     "AVEncoderBitRateStrategyKey": AVAudioBitRateStrategy_Variable,
     "AVFormatIDKey": 1633772320,
     "AVNumberOfChannelsKey": 1,
     "AVSampleRateKey": 44100]
    

    (audio initializes in ~1500-3000ms)

    Note that subsequent calls are faster. I am also changing the AVAudioSessionCategory while configuring the audio AVAssetWriter, not sure if that has anything to do with it.

  2. Regarding the AVAssetWriter.startWriting() call taking 3-5 seconds: I still have no idea why that takes so long.

Hi mrousavy, Can you please provide a Swift code for this block please?

["AVVideoCompressionPropertiesKey": { AllowFrameReordering = 1; AllowOpenGOP = 1; AverageBitRate = 47848572; BaseLayerFrameRate = 15; ExpectedFrameRate = 60; MaxAllowedFrameQP = 41; MaxKeyFrameIntervalDuration = 1; MinAllowedFrameQP = 15; MinimizeMemoryUsage = 1; Priority = 80; ProfileLevel = "HEVC_Main_AutoLevel"; RealTime = 1; RelaxAverageBitRateTarget = 1; }, "AVVideoWidthKey": 2160, "AVVideoCodecKey": hvc1, "AVVideoHeightKey": 3840]

I have the same problem and don't know how to fix it

Performance issues with `AVAssetWriter`
 
 
Q