Server Trust Authentication with same URL Session has uncertain response time behavior

Hello Folks

I have a Custom UrlSessionDeleagte which is checking server authentication by overriding method

 func urlSession(_ session: URLSession, didReceive challenge: URLAuthenticationChallenge, completionHandler: @escaping (URLSession.AuthChallengeDisposition, URLCredential?) -> Swift.Void) {
if (challenge.protectionSpace.authenticationMethod == NSURLAuthenticationMethodServerTrust) {
    let serverTrust = challenge.protectionSpace.serverTrust
    // Applying additional validations.
    if(validated)
    {
     completionHandler(.useCredential, URLCredential(trust:serverTrust))
     }
}
else
{
  completionHandler(.performDefaultHandling, nil)
}
            

Initialized URL Session as below and reusing it in subsequent requests.

if(urlSession != nil)
{
urlSession = URLSession(configuration: URLSessionConfiguration.Default, delegate: customURLSessionDelegate, delegateQueue : nil)
}

Now the issue is the uncertainty in response time First request - say took approx 11 secs.

Second request if send immediately (< 2 secs difference from last call) - took only 0.2 secs or 1.2 secs.

Third request if send after >20 secs - took again 12 secs.

I want to know whether it is an implementation issue, or iOS behavior of handling the Server trust Authentication process in this way? Because the time it took after initializing a DataTask to checking server Auth differes. Also when call is sent immdiately it does not checkk Authentication again, but when send a after ~20 secs debugger fall on the Authentication method again, even if the URlsession instance was same.