Can't establish mTLS on iOS with WKWebView and ProxyConfiguration

I have a sample iOS app in Xcode that I run in the iOS 17.5 Simulator. It creates a WKWebView and configures a proxy via the ProxyConfiguration API, it works as expected unless the proxy tries to establish mTLS. It seems there is no way to handle the client certificate request when using a proxy. If I navigate to a page that requests mTLS without a proxy configured, it works as expected. Here is a minimal repro:

#import "ViewController.h"
#import <WebKit/WebKit.h>

@import Foundation;
@import WebKit;

@interface ViewController () <WKNavigationDelegate>
@property (nonatomic,strong) WKWebView* webView;
@property (nonatomic, strong) WKWebViewConfiguration * webConfig;
@end

@implementation ViewController

- (void)loadView {
    [super loadView];
    
    nw_protocol_options_t tls_options = nw_tls_create_options();
    sec_protocol_options_t sec_options = nw_tls_copy_sec_protocol_options(tls_options);

    sec_protocol_options_set_challenge_block(
        sec_options,
        ^(sec_protocol_metadata_t metadata, sec_protocol_challenge_complete_t challenge_complete) {
            NSLog(@"Inside of challenge block");
            challenge_complete(nil);
        },
        dispatch_get_main_queue());
    
    nw_endpoint_t proxy_endpoint =
    nw_endpoint_create_host(GetHost(), GetPort());
    
    nw_relay_hop_t relay =
    nw_relay_hop_create(nil, proxy_endpoint, tls_options);
    nw_proxy_config_t proxy_config =
    nw_proxy_config_create_relay(relay, nil);
    
    nw_proxy_config_add_match_domain(proxy_config, "api.ipify.org");
    
    self.webConfig = [[WKWebViewConfiguration alloc] init];
    self.webConfig.websiteDataStore = [WKWebsiteDataStore nonPersistentDataStore];
    self.webConfig.websiteDataStore.proxyConfigurations = @[ proxy_config ];
    
    self.webView = [[WKWebView alloc] initWithFrame:self.view.frame configuration:self.webConfig];
    
    self.webView.navigationDelegate = self;
    
    [self.view addSubview:self.webView];
}

- (void)viewDidLoad {
    [super viewDidLoad];
    NSLog(@"%s",__func__);
    NSURL* url = [[NSURL alloc] initWithString:@"https://api.ipify.org"];
    NSURLRequest* request = [[NSURLRequest alloc] initWithURL:url];
    [self.webView loadRequest:request];
}

- (void)webView:(WKWebView *)webView didStartProvisionalNavigation:(WKNavigation *)navigation {
    NSLog(@"%s",__func__);
}

- (void)webView:(WKWebView *)webView didFailProvisionalNavigation:(WKNavigation *)navigation withError:(NSError *)error {
    NSLog(@"%s. Error %@",__func__,error);
}

- (void)webView:(WKWebView *)webView didReceiveAuthenticationChallenge:(NSURLAuthenticationChallenge *)challenge completionHandler:(void (^)(NSURLSessionAuthChallengeDisposition, NSURLCredential *))completionHandler {
    NSLog(@"%s",__func__);
    NSLog(@"protection space: %@", challenge.protectionSpace.authenticationMethod);
    completionHandler(NSURLSessionAuthChallengePerformDefaultHandling, nil);
}

@end

The logs for this code show:

-[ViewController viewDidLoad]
-[ViewController webView:didStartProvisionalNavigation:]
-[ViewController webView:didFailProvisionalNavigation:withError:]. Error Error Domain=NSURLErrorDomain Code=-1206 "The server “api.ipify.org” requires a client certificate."

If we don't set up the ProxyConfiguration and navigate to a site that requires mTLS, the logs look like this:

-[ViewController viewDidLoad]
-[ViewController webView:didReceiveAuthenticationChallenge:completionHandler:]
protection space: NSURLAuthenticationMethodServerTrust
-[ViewController webView:didReceiveAuthenticationChallenge:completionHandler:]
protection space: NSURLAuthenticationMethodClientCertificate
-[ViewController webView:didStartProvisionalNavigation:]
//...

Eventually the request fails but the key difference is that didReceiveAuthenticationChallenge was invoked. When using the ProxyConfiguration neither that function nor the block we set via sec_protocol_options_set_challenge_block were run.

I also tried to provide the client identity via sec_protocol_options_set_local_identity to no avail, and I've tried configuring these options too but they had no effect

    sec_protocol_options_add_tls_application_protocol(sec_options, "h2");
    sec_protocol_options_set_max_tls_protocol_version(sec_options, tls_protocol_version_TLSv13);
    sec_protocol_options_set_peer_authentication_required(sec_options, true);

Am I missing something? Or is this a bug in the ProxyConfiguration API?

Answered by DTS Engineer in 798719022

I suspect that this is just a limitation of the Network framework proxy support. But before I sent you off to file a bug, I want to ask about this:

I have a sample iOS app in Xcode that I run in the iOS 17.5 Simulator.

Do you see the same behaviour when testing on a real device?

My guess is that you will, but I wanna be sure because the simulator uses a big chunk of the macOS networking stack. Thus, when faced with weird networking problems, it’s best to test on real hardware.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

I suspect that this is just a limitation of the Network framework proxy support. But before I sent you off to file a bug, I want to ask about this:

I have a sample iOS app in Xcode that I run in the iOS 17.5 Simulator.

Do you see the same behaviour when testing on a real device?

My guess is that you will, but I wanna be sure because the simulator uses a big chunk of the macOS networking stack. Thus, when faced with weird networking problems, it’s best to test on real hardware.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Hi Quinn, thanks for the reply. I tried it on an iPhone SE 3rd gen running iOS 17.6.1 and got the same results. Specifically these log messages:

-[ViewController viewDidLoad]
-[ViewController webView:didStartProvisionalNavigation:]
0x152019018 - [pageProxyID=7, webPageID=8, PID=865] WebPageProxy::didFailProvisionalLoadForFrame: frameID=1, isMainFrame=1, domain=NSURLErrorDomain, code=-1206, isMainFrame=1, willInternallyHandleFailure=0
-[ViewController webView:didFailProvisionalNavigation:withError:]. Error Error Domain=NSURLErrorDomain Code=-1206 "The server “api.ipify.org” requires a client certificate." UserInfo={NSErrorFailingURLKey=https://api.ipify.org/, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <93E83F6C-8958-44A7-9F3D-A9BBBE4585E7>.<2>, _kCFStreamErrorDomainKey=3, networkTaskMetricsPrivacyStance=Unknown, _NSURLErrorRelatedURLSessionTaskErrorKey=(
    "LocalDataTask <93E83F6C-8958-44A7-9F3D-A9BBBE4585E7>.<2>"
), NSLocalizedDescription=The server “api.ipify.org” requires a client certificate., _WKRecoveryAttempterErrorKey=<WKReloadFrameErrorRecoveryAttempter: 0x3004dc080>, networkTaskDescription=LocalDataTask <93E83F6C-8958-44A7-9F3D-A9BBBE4585E7>.<2>, NSErrorFailingURLStringKey=https://api.ipify.org/, NSUnderlyingError=0x300af5020 {Error Domain=kCFErrorDomainCFNetwork Code=-1206 "(null)" UserInfo={_kCFStreamPropertySSLClientCertificateState=0, _kCFNetworkCFStreamSSLErrorOriginalValue=-9829, _kCFStreamErrorDomainKey=3, _kCFStreamErrorCodeKey=-9829}}, _kCFStreamErrorCodeKey=-9829}

If I try again without using a proxy (instead navigating directly to a site that requires mTLS), I see the expected log output:

-[ViewController viewDidLoad]
-[ViewController webView:didStartProvisionalNavigation:]
-[ViewController webView:didReceiveAuthenticationChallenge:completionHandler:]
protection space: NSURLAuthenticationMethodServerTrust
-[ViewController webView:didReceiveAuthenticationChallenge:completionHandler:]
protection space: NSURLAuthenticationMethodClientCertificate

Hello Quinn, I am Piotr and I work with aurion on this issue.

Do you have any suggestions, what we can try to do/what data we should provide to move forward with this issue?

I tried it on an iPhone SE 3rd gen running iOS 17.6.1 and got the same results.

Yeah, that’s what I expected.

Looking back at your original post, I’m a bit confused about the protocol in play here. Your prose suggests that you’re trying to set up an HTTP proxy, but your code is using a relay (via nw_proxy_config_create_relay). Can you clarify the network protocol spoken by your proxy?

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

We've got an https proxy that supports both http1.1 and http2 and we're experiencing the same or similar issue trying to proxy wkwebview with establishing mTLS to the proxy.

1 URLSession with the proxy works as expected. sec_protocol_options_set_challenge_block is invoked. No issues. Both http1.1 and http2 proxyconfigurations work as expected.

2 WKWebView behaves strangely:

a. http1.1 using ProxyConfiguration(httpCONNECTProxy:tlsOptions:) leads to bad url error

b. http2 using ProxyConfiguration.RelayHop(http2RelayEndpoint:tlsOptions:additionalHTTPHeaderFields:) leads to The server “*” requires a client certificate. sec_protocol_options_set_challenge_block is not invoked.

We've got an https proxy that supports both http1.1 and http2

I’d like to keep this thread focused on aurion’s original issue here. What you’ve described sounds like a bug to me, and it’s probably best if you file it as such. Please post your bug number, just for the record.

If you have follow-up questions about that, I recommend that you start a new thread.


aurion commented:

We're using HTTPS CONNECT proxying.

Please reply as a reply, rather than in the comments. It’s easy for me to miss the latter. See Quinn’s Top Ten DevForums Tips for this and other titbits.

If it’s an HTTP proxy, you need to be using nw_proxy_config_create_http_connect. nw_proxy_config_create_relay is for relays.

As to why you’re getting the bad URL response, my advice is that you start by testing with URLSession and then move on to WKWebView. That’s because:

  • If you can’t get it working with URLSession, that’s an easier problem to investigate.

  • IF you can, and WKWebView fails, that’s clearly a bug in WKWebView.

So, if you apply the same setup to URLSession, what do you see?

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Please reply as a reply, rather than in the comments.

Understood, thanks for the reminder!

I'll give this scenario a shot with URLSession. Actually pgalisz1 will be taking over in this area for me, so he'll be updating the prototype. I'm happy to remain the POC for this issue to keep things consistent, but Piotr may be chiming in with details.

Hi! As requested I tried to implement the same scenario with NSUrlSession. It also does not work. As soon as I enable HTTPS proxy, I am not receiving neither didReceiveChallenge calls, nor completion handler. Without proxy configuration I am receiving didReceiveChallenge and I can continue.

    NSURLSessionConfiguration *config = [NSURLSessionConfiguration ephemeralSessionConfiguration];

    NSDictionary *proxyDict = @{
        (NSString *)kCFNetworkProxiesHTTPEnable: @YES,
        (NSString *)kCFNetworkProxiesHTTPProxy : GetHost(),
        (NSString *)kCFNetworkProxiesHTTPPort : GetPort(),
       
        @"HTTPSEnable" : @YES,
        @"HTTPSProxy": GetHost(),
        @"HTTPSPort": GetPort(),
    };

    config.connectionProxyDictionary = proxyDict;
    
    NSURLSession *session = [NSURLSession sessionWithConfiguration:config delegate:self delegateQueue:[NSOperationQueue mainQueue]];
    NSURLRequest *request1 = [NSURLRequest requestWithURL:[NSURL URLWithString:@"https://api.ipify.org"]];

    NSURLSessionDataTask *task = [session dataTaskWithRequest:request1 completionHandler:
                                  ^(NSData *data, NSURLResponse *response, NSError *error) {
                                   
                                      if (error) {
                                      NSLog(@"Failed === Response:%@ %@\n", response, error);
                                      }
                                      NSString* newStr = [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding];
                                      NSLog(@"%@",newStr);
                                  }];
    [task resume];

delegates:

- (void)URLSession:(NSURLSession *)session didReceiveChallenge:(NSURLAuthenticationChallenge *)challenge completionHandler:(void (^)(NSURLSessionAuthChallengeDisposition, NSURLCredential * _Nullable))completionHandler
{
    NSLog(@"%s",__func__);
    completionHandler(NSURLSessionAuthChallengePerformDefaultHandling, nil);
}

- (void)URLSession:(NSURLSession *)session task:(NSURLSessionTask *)task
                            didReceiveChallenge:(NSURLAuthenticationChallenge *)challenge
 completionHandler:(void (NS_SWIFT_SENDABLE ^)(NSURLSessionAuthChallengeDisposition disposition, NSURLCredential * _Nullable credential))completionHandler {
    NSLog(@"%s",__func__);
    completionHandler(NSURLSessionAuthChallengePerformDefaultHandling, nil);
}

I also had a look on a proxy configuration, and I believe that we should use relays as @auorion posted. Documentation for http_connect proxy says:

Creates a legacy HTTP CONNECT proxy configuration for a proxy server accessible using HTTP/1.1. This proxy will only relay TCP connections.

In our case we use HTTP/2 CONNECT proxy. Still in both cases I am not receiving any challenge calls, just direct didFailProvisionalNavigation call with Error Domain=NSURLErrorDomain Code=-1202 error

Let alone sec_protocol_options_set_challenge_block, even sec_protocol_options_set_verify_block is not being called with wkwebview and proxyconfiguration. While both functions are called with urlsession proxyconfiguration.

You’re using the old school connectionProxyDictionary here. What happens if you configure the proxy via the proxyConfigurations property on the session configuration?

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Interesting, I see now documentation for this method, but my compiler does not see it:

Property 'proxyConfigurations' not found on object of type 'NSURLSessionConfiguration *'

I am using a framework for iOS 17.5

EDIT: I see that I need #import <Network/NSURLSession+Network.h>

Ok, I tested this with the new API and it seems to be working fine.

I received callbacks in sec_protocol_options_set_challenge_block and sec_protocol_options_set_verify_block blocks, and later receive a proper error (as I did not provide certificate):

So for NSUrlSession the code seems to work. The same configuration for WKWebkit - fails

Log:

Inside of challenge block
Inside of challenge block
boringssl_context_handle_fatal_alert(2072) [C1.1.1.1.1.1:2][0x107006120] read alert, level: fatal, description: certificate required
[C1.1.1.1:3] Connection disconnected from api.ipify.org:443 without a reply
[C1.1.1.1:3] Cannot report error 1200, no proxy agent
boringssl_session_handshake_error_print(44) [C1.1.1.1.1.1:2][0x107006120] Error: 4361061696:error:1000045c:SSL routines:OPENSSL_internal:TLSV1_CERTIFICATE_REQUIRED:/Library/Caches/com.apple.xbs/Sources/boringssl_Sim/ssl/tls_record.cc:592:SSL alert number 116
Connection 1: received failure notification
Connection 1: failed to connect 3:-9829, reason -1
Connection 1: encountered error(3:-9829)
Task <38AFAEA1-DE06-4D54-8F8F-7BEC9C2931E0>.<1> HTTP load failed, 0/0 bytes (error code: -1206 [3:-9829])
Task <38AFAEA1-DE06-4D54-8F8F-7BEC9C2931E0>.<1> finished with error [-1206] Error Domain=NSURLErrorDomain Code=-1206 "The server “api.ipify.org” requires a client certificate." UserInfo={_kCFStreamErrorCodeKey=-9829, NSUnderlyingError=0x600000c705a0 {Error Domain=kCFErrorDomainCFNetwork Code=-1206 "(null)" UserInfo={_kCFStreamPropertySSLClientCertificateState=0, _kCFNetworkCFStreamSSLErrorOriginalValue=-9829, _kCFStreamErrorDomainKey=3, _kCFStreamErrorCodeKey=-9829, _NSURLErrorNWPathKey=satisfied (Path is satisfied), interface: en0[802.11], uses wifi}}, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <38AFAEA1-DE06-4D54-8F8F-7BEC9C2931E0>.<1>, _NSURLErrorRelatedURLSessionTaskErrorKey=(
    "LocalDataTask <38AFAEA1-DE06-4D54-8F8F-7BEC9C2931E0>.<1>"
), NSLocalizedDescription=The server “api.ipify.org” requires a client certificate., NSErrorFailingURLStringKey=https://api.ipify.org/, NSErrorFailingURLKey=https://api.ipify.org/, _kCFStreamErrorDomainKey=3}
Failed === Response:(null) Error Domain=NSURLErrorDomain Code=-1206 "The server “api.ipify.org” requires a client certificate." UserInfo={_kCFStreamErrorCodeKey=-9829, NSUnderlyingError=0x600000c705a0 {Error Domain=kCFErrorDomainCFNetwork Code=-1206 "(null)" UserInfo={_kCFStreamPropertySSLClientCertificateState=0, _kCFNetworkCFStreamSSLErrorOriginalValue=-9829, _kCFStreamErrorDomainKey=3, _kCFStreamErrorCodeKey=-9829, _NSURLErrorNWPathKey=satisfied (Path is satisfied), interface: en0[802.11], uses wifi}}, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <38AFAEA1-DE06-4D54-8F8F-7BEC9C2931E0>.<1>, _NSURLErrorRelatedURLSessionTaskErrorKey=(
    "LocalDataTask <38AFAEA1-DE06-4D54-8F8F-7BEC9C2931E0>.<1>"
), NSLocalizedDescription=The server “api.ipify.org” requires a client certificate., NSErrorFailingURLStringKey=https://api.ipify.org/, NSErrorFailingURLKey=https://api.ipify.org/, _kCFStreamErrorDomainKey=3}
I see that I need #import <Network/NSURLSession+Network.h>

Yep. In Swift this comes in via a cross-import overlay, but there’s no equivalent feature in Objective-C.

So for NSUrlSession the code seems to work.

Excellent news.

The same configuration for WKWebkit - fails

Bummer.

Sadly, I was kinda expecting this, which is why I suggested the URLSession test. WKWebView does all of its networking in a separate helper process, and bridging all the security stuff is quite tricky.

The only upside to this result is that you can file a very clear bug against WebKit: Challenge blocks work in the URLSession case but fail with WKWebView. Please file that bug and then post your bug number, just for the record.

Do you need to deal with challenges dynamically? Or would it be OK to use a fixed client identity? If it’s the latter, you might be able to work around this issue by calling sec_protocol_options_set_local_identity. That establishes a fixed client identity, and it’s possible that this might bridge over to the WKWebView helper process.

Note I say “possible” because I’ve not tried this myself. Fortunately, you’re in a good position to test it now, with both URLSession and WKWebView.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Is it possible to use NERelayManager instead of ProxyConfiguration to proxy wkwebview for personal devices (non managed or supervised)?

Can't establish mTLS on iOS with WKWebView and ProxyConfiguration
 
 
Q