How to add (another) default route to link layer address?

Greetings,

I am trying to mimic what the official WireGuard client (available on AppStore, source code is publicly available) does regarding the routing tables. The client uses NetworkExtension framework.

When a VPN connection is established with all traffic routed through WireGuard (AllowedIPs = 0.0.0.0/0), the routing table is amend with something like this:

Destination        Gateway            RT_IFA             Flags        Refs      Use    Mtu          Netif Expire
default            link#36            10.10.10.2         UCSg          114        0   1420          utun7
10.10.10.2         10.10.10.2         10.10.10.2         UH              0       10   1420          utun7
224.0.0/4          link#36            10.10.10.2         UmCS            0        0   1420          utun7
255.255.255.255/32 link#36            10.10.10.2         UCS             0        0   1420          utun7

Please note that another default route exists to the working Ethernet interface, but I have not mentioned it above.

I would like to do something similar for wireguard-go (open source WireGuard implementation written in Go), in particular start it, assign an IP address, then add the routes.

sudo env LOG_LEVEL=debug wireguard-go -f utun
sudo ifconfig utun5 10.10.10.2 10.10.10.2 netmask 255.255.255.255

Here is the code fragment written in C which suppose to add default route (0.0.0.0/0) to the link layer address:

void add_link_route() {
    struct {
        struct rt_msghdr hdr;
        struct sockaddr_in dest;
        struct sockaddr_dl gateway;
        struct sockaddr_in netmask;
    } rt;
    memset(&rt, 0, sizeof(rt));

    int sockfd = socket(PF_ROUTE, SOCK_RAW, 0);
    if (sockfd == -1) {
        perror("socket");
        return;
    }

    unsigned int if_index = if_nametoindex("utun5");

    rt.hdr.rtm_msglen = sizeof(rt);
    rt.hdr.rtm_version = RTM_VERSION;
    rt.hdr.rtm_type = RTM_ADD;
    rt.hdr.rtm_index = if_index;
    rt.hdr.rtm_flags = RTF_UP | RTF_STATIC | RTF_CLONING;
    rt.hdr.rtm_addrs = RTA_DST | RTA_GATEWAY | RTA_NETMASK;
    rt.hdr.rtm_seq = 1;
    rt.hdr.rtm_pid = getpid();

    rt.dest.sin_len = sizeof(struct sockaddr_in);
    rt.dest.sin_family = AF_INET;
    rt.dest.sin_addr.s_addr = INADDR_NONE;

    rt.gateway.sdl_len = sizeof(struct sockaddr_dl);
    rt.gateway.sdl_family = AF_LINK;
    rt.gateway.sdl_index = if_index;
    rt.gateway.sdl_type = IFT_PPP;

    rt.netmask.sin_len = sizeof(struct sockaddr_in);
    rt.netmask.sin_family = AF_INET;
    rt.netmask.sin_addr.s_addr = INADDR_NONE;

    if (write(sockfd, &rt, sizeof(rt)) == -1) {
        perror("write");
    }

    close(sockfd);
}

But, when executed, write() returns EEXIST (File exists) error, meaning, the default route cannot be overwritten (because another default route exists which points to the existing Ethernet interface).

At this point I have no idea how the routes could be created successfully inside NetworkExtension, and I would like to do the same.

For comparison, there is another case when all traffice is not routed through the VPN. Then, the routes are created like this:

Destination        Gateway            RT_IFA             Flags        Refs      Use    Mtu          Netif Expire
default            link#36            10.10.10.2         UCSIg           0        0   1420          utun7
10.10.10.2         10.10.10.2         10.10.10.2         UH              0        0   1420          utun7
224.0.0/4          link#36            10.10.10.2         UmCSI           0        0   1420          utun7
255.255.255.255/32 link#36            10.10.10.2         UCSI            0        0   1420          utun7

The difference is that now the scope is bound to the network interface. And in such case, my C code succeeds, providing I add RTF_IFSCOPE flag to rtm_flags.

I would appreciate if someone helped me with this problem.

I don’t have an answer for you, and I want to explain why.

DTS doesn’t support ‘ad hoc’ VPN clients on the Mac [1] [2]. We only support folks creating VPN products using Network Extension framework.

It seems like you’re building an NE packet tunnel provider and then trying to modify the routing table behind its back. This falls into the same ‘ad hoc’ VPN bucket. On macOS the routing table is owned by Apple’s networking stack [3]. If you attempt to modify it yourself, you will run into problems [4].

The correct way for an NE packet tunnel provider to configure the routing table is via the various NE-level routing parameters. See Routing your VPN network traffic for the details.


And while we’re here, I want to be clear about another thing: In an NE packet tunnel provider, the correct way to transfer packets to and from the tunnel interface is via the NEPacketTunnelFlow object. I’ve seen third-party libraries that use unsupported techniques to extract the UTUN file descriptor from this flow object and then read and write this directly. This is not supported. Code like this has broken in the past, and it may well break in the future. Don’t start down that path.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] On Apple’s other platforms, it’s not possible to create ad hoc VPN clients. Everything must go through Network Extension framework.

[2] One specific case that causes more grief than most is packet filter, and hence TN3165 Packet Filter is not API.

[3] NE, and the underlying System Configuration framework.

[4] These usually manifest in one of three ways:

  • Your changes don’t do what you think they should do.

  • The system overwrites your changes.

  • The system fails to clean up your changes.

And, as is usual for when doing unsupported stuff, things might seem to work fine on one release of the OS and then fail horribly on others.

How to add (another) default route to link layer address?
 
 
Q