Implementing Client and Server over UDP based custom protocol using Network Framework

We have an application design where, every instance (process) is acting as a UDP server as well as UDP client, using the same UDP port: to listen & respond (as a server) to multiple destinations as well to send (as a client) to multiple destinations. This considering the implicit nature of UDP being connectionless. At any given point in time, I would be, as a server, talking to many clients and as a client, talking to many servers.

We were using BSD sockets for the purpose across all our target platforms including Apple Kernel (macOS, iOS, iPadOS, tvOS, watchOS etc.). Then we learnt about limitation on watchOS, where we started exploring 'Network Framework' as an alternative to BSD sockets on watchOS (or even others on Apple Kernel).

This is to understand, how can we achieve the same (if at all) using 'Network Framework'?

Process A

  1. [To act as UDP server] We will have NWListener on inaddrany, local port X, using UDP
  • Does it implicitly work for both IPv4 and IPv6 incoming data?
    • In case of BSD sockets, we would have created two sockets - one bound on INADDR_ANY and other on in6addr_any.
    • Does in case of NWListener, also internally it creates two sockets - one for IPv4 and other for IPv6?
  • For every incoming data from a client (which may not be on Apple Kernel and hence not using NWConnection), a NWConnection would be created on this UDP server (off course, if NWConnection does not already exist for the same local and remote IP/Port). Just for our clarity:
    • An underlying socket is not created (like it would have in case of TCP)?
    • The underlying data exchange between the UDP clients and this UDP server would happen on the same socket bound on port X?
    • NWConnection for UDP is more a logical construct created to represent a “UDP flow”, that is, a sequence of datagrams, including both inbound and outbound, that share the same local IP / port and same remote IP / port tuple, where for 'local IP/Port', there would a socket bound on it internally.
  • I can use the same NWConnection to respond (send data) back to the client.
  • Since UDP is connectionless, how do we manage the lifecycle of these NWConnection(s) getting created?
    • Though there is no socket resource to be freed per NWConnection basis BUT there must be some other system resources like memory being occupied.
    • We understand that once cancelled, if we receive a datagram from the same client (actually, on the same UDP flow), the listener will create a new connection.
  1. [To act as UDP client] We will have to create a NWConnection to a UDP server
  • We would like to have that NWConnection internally use the same local port X to send data to the remote UDP server, is that possible? The interface to init NWConnection seem to only take remote endpoint as an input and protocol as an input.
  • And this we would like to do for all UDP servers we want to connect as client? Which would mean multiple NWConnection - one for each UDP Server we want to communicate to BUT same local port X is being used on the UDP Client.
  • I will receive the response from the Server also on the same NWConnection (if still active and not cancelled).
  • The client cancels the NWConnection when no more intends to talk to the same UDP Server.

Does it implicitly work for both IPv4 and IPv6 incoming data?

In case of BSD sockets, we would have created two sockets - one bound on INADDR_ANY and other on in6addr_any. Does in case of NWListener, also internally it creates two sockets - one for IPv4 and other for IPv6?

Yes, it works for both IPv4 and IPv6. There may or may not actually be a socket under the hood, but if one is created inside the listener, it will create whatever it needs to cover the interfaces that you're trying to listen on.

For every incoming data from a client (which may not be on Apple Kernel and hence not using NWConnection), a NWConnection would be created on this UDP server (off course, if NWConnection does not already exist for the same local and remote IP/Port). Just for our clarity:

An underlying socket is not created (like it would have in case of TCP)? The underlying data exchange between the UDP clients and this UDP server would happen on the same socket bound on port X? NWConnection for UDP is more a logical construct created to represent a “UDP flow”, that is, a sequence of datagrams, including both inbound and outbound, that share the same local IP / port and same remote IP / port tuple, where for 'local IP/Port', there would a socket bound on it internally.

You will have a new flow (which might be backed by a socket) and that flow will be bound to that five-tuple (local IP/port, remote IP/port, local interface). This is conceptually similar to a "connected UDP socket". To summarize, yes it is a "UDP flow" between those two endpoints.

I can use the same NWConnection to respond (send data) back to the client.

Exactly!

Since UDP is connectionless, how do we manage the lifecycle of these NWConnection(s) getting created?

Though there is no socket resource to be freed per NWConnection basis BUT there must be some other system resources like memory being occupied. We understand that once cancelled, if we receive a datagram from the same client (actually, on the same UDP flow), the listener will create a new connection.

Essentially how you've outlined here. It is good to cancel connections for logical flows of packets that you're done with, this will free up system resources like memory. Once you've closed a connection, new incoming packets should indeed go to the listener and spawn a new one.

We would like to have that NWConnection internally use the same local port X to send data to the remote UDP server, is that possible? The interface to init NWConnection seem to only take remote endpoint as an input and protocol as an input.

And this we would like to do for all UDP servers we want to connect as client? Which would mean multiple NWConnection - one for each UDP Server we want to communicate to BUT same local port X is being used on the UDP Client.

Yes, this should be possible, you'll want to set allowLocalEndpointReuse to true in the NWParameters for both the connections and the listener. You'll then use the requiredLocalEndpoint to ask for a specific port. Note that it's often better to use the system-assigned port from your listener, rather than guessing at what one might be available, as that is prone to race conditions and is otherwise often tricky to get right. You'll see failures and log messages about addresses being already in use if it's not quite right.

I will receive the response from the Server also on the same NWConnection (if still active and not cancelled).

Correct, yes.

The client cancels the NWConnection when no more intends to talk to the same UDP Server.

Yes, but note that unlike TCP, this doesn't cause anything to go out on the wire to tell the UDP server that you're done, so you are responsible for generating that message yourself and ensuring its reliable delivery. Consider something like QUIC (and/or QUIC datagrams) to get the best of both worlds here: unreliable datagram transport but reliable signaling of this type of connection state.

Finally, make sure that you've read this tech note about supported use cases for low-level networking on watchOS to make sure that what you're doing will work on watchOS.

Yes, this should be possible, you'll want to set allowLocalEndpointReuse to true in the NWParameters for both the connections and the listener. You'll then use the requiredLocalEndpoint to ask for a specific port. Note that it's often better to use the system-assigned port from your listener, rather than guessing at what one might be available, as that is prone to race conditions and is otherwise often tricky to get right. You'll see failures and log messages about addresses being already in use if it's not quite right.

As per suggestion above, I have written sample code(below) to reuse a local port (9000) for both listener and connection. Here I am creating a UDP NWListener on port 9000 and then I created a NWConnection to connect this server as a client to another UDP Listener server (on port 9001). For both listener and connection I have set 'allowLocalEndpointReuse' as true and used requiredLocalEndpoint for connection.

However, this is not working with following error logs:

Server started on INADDR_ANY:9000

Client Connection preparing

Client Connection waiting

nw_socket_connect [C1:1] connectx(5 (guarded), [srcif=1, srcaddr=<data>, dstaddr=<data>], SAE_ASSOCID_ANY, 0, NULL, 0, NULL, SAE_CONNID_ANY) failed: 48

nw_socket_connect [C1:1] connectx failed (fd 5) 48

nw_socket_connect connectx failed 48

To ensure that there are no other process using the same port I have checked the lsof command for port 9000. It is showing UDP listner server only:

nikhil.singh@tw-macoffice-01 ~ % lsof -i :9000

COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME

UDP_Serve 41362 nikhil.singh 4u IPv6 0x6febcdc1906bf539 0t0 UDP *:cslistener

Please provide inputs if this is the correct way to use the "allowLocalEndpointReuse" and "requiredLocalEndpoint" as per the context you provided. If not, how can this code be corrected to reuse local port for listner and connections. As per our application design we need to use same local port for communication.

Note: This code works if I change the local port for NWConnection to something different than 9000, for ex: 9002

import Foundation
import Network

// Create a UDP listener - set the listener's address to INADDR_ANY and the specified port
let listener = try! NWListener(using: .udp, on: 9000)

// Allow Local Endpoint Resue
listener.parameters.allowLocalEndpointReuse = true

// Handle new connections
listener.newConnectionHandler = { connection in
    print("New connection request received")
    //Adding state update handler to monitor received connection request
    connection.stateUpdateHandler = { state in
        switch state {
        case .ready:
            print("Received connection request is ready")
        case .waiting:
            print("Received connection waiting")
        case .preparing:
            print("Received connection preparing")
        case .cancelled:
            print("Received connection cancelled")
        case .failed(let error):
            print("Received connection failed: \(error)")
        default:
            break
        }
    }

    //Starting the connection
    connection.start(queue: DispatchQueue.global())
    // Start receiving data from the connection
    connection.receive(minimumIncompleteLength: 1, maximumLength: 1024) { data, context, isComplete, error in
        if let error = error {
            print("Error receiving data: \(error)")
            return
        }
         if let data = data {
            // Process received data
            print("Received data from client: \(String(data: data, encoding: .utf8)!)")

            // Optionally, send a response to the client
            let response = "Hello, client!"
            connection.send(content: response.data(using: .utf8)!, completion: .contentProcessed({ (error) in
                if let err = error {
                    // Handle error in sending
                    print("Error sending response: \(err)")
                } else {
                    // Send has been processed
                    print("Response sent successfully")
                }
            }))
        }
        if isComplete {
            print("Connection closed")
        }
    }
}
 // Start the listener
listener.start(queue: DispatchQueue.global())
print("Server started on INADDR_ANY:\(9000)")

//**************Starting NWConnection from Server1 as a client ***************//

// Allowing local port reuse for port 9000
let port = NWEndpoint.Port(rawValue: 9000)
let localEndpoint = NWEndpoint.hostPort(host: NWEndpoint.Host("127.0.0.1"), port: port!)
let parameters = NWParameters.udp
parameters.requiredLocalEndpoint = localEndpoint

// Starting connection from server to a different server as a client using NWConnection
let cli_connection = NWConnection(host: NWEndpoint.Host( "127.0.0.1"), port: NWEndpoint.Port(rawValue: 9001)!, using: parameters)
//Allowing Local Endpoint Reuse
cli_connection.parameters.allowLocalEndpointReuse = true
// Set up state update handler
cli_connection.stateUpdateHandler = { state in
    switch state {
    case .ready:
        print("Client Connection established")
        // Send data to the server
        let message = "Hello, Server2!"
        cli_connection.send(content: message.data(using: .utf8)!, isComplete: true, completion: .contentProcessed({ (error) in
            if let err = error {
                // Handle error in sending
                print("Error sending response: \(err)")
            } else {
                // Send has been processed
                print("Request sent successfully")
            }
        }))
    case .waiting:
        print("Client Connection waiting")
    case .preparing:
        print("Client Connection preparing")
    case .cancelled:
        print("Client Connection cancelled")
    case .failed(let error):
        print("Client Connection failed: \(error)")
    default:
        break
    }
}

// Start the connection
cli_connection.start(queue: DispatchQueue.global())
// Receive data from the server
cli_connection.receive(minimumIncompleteLength: 1, maximumLength: 1024) { data, context, isComplete, error in
    if let error = error {
        print("Error receiving data: \(error)")
    } else if let data = data {
        print("Received 1 data from server: \(String(data: data, encoding: .utf8)!)")
    }
    if isComplete {
        print("Connection closed")
    }
}
// Keep the server running
RunLoop.main.run()

@Frameworks Engineer Your guidance on above queries from @nikhil0n1 will help

Implementing Client and Server over UDP based custom protocol using Network Framework
 
 
Q