What is the difference between OSAllocatedUnfairLock's withLock and withLockUnchecked functions?

[Thanks for starting this thread. Sorry it was such a hassle to get you here.]

I’m gonna link to your Stack Overflow question, because Rob’s answer there is spot on.

I think that leaves me with this question:

in practice, if I did want to use withLockUnchecked, how would I "uphold the Sendability contract"?

Here’s an example I was dealing with recently: I was creating a class that works with C FILE * values, like stdout and stderr. These types have their own internal locking. You can, obviously, call fprintf on stdout from any thread. However, Swift doesn’t know that.

I wanted to provide a way for clients to atomically get a set a property of that type. Here’s my first attempt:

final class MyClass: Sendable {

    init(output: UnsafeMutablePointer<FILE>) {
        self.output = output
    }

    var output: UnsafeMutablePointer<FILE>
     // ^ Stored property 'output' of 'Sendable'-conforming class 'MyClass' is mutable
}

Swift complains, as it should. [I’m using Xcode 15.3 with Strict Concurrency Checking set and macOS 13 as my deployment target.] So, I protected this value with a lock:

final class MyClass: Sendable {

    init(output: UnsafeMutablePointer<FILE>) {
        self._output = .init(uncheckedState: output)
    }
    
    private let _output: OSAllocatedUnfairLock<UnsafeMutablePointer<FILE>>

    var output: UnsafeMutablePointer<FILE> {
        get {
            self._output.withLock { f in
                      // ^ Conformance of 'UnsafeMutablePointer<Pointee>' to 'Sendable' is unavailable
                f
            }
        }
        set {
            self._output.withLock { f in
                f = newValue
                 // ^ Capture of 'newValue' with non-sendable type 'UnsafeMutablePointer<FILE>' (aka 'UnsafeMutablePointer<__sFILE>') in a `@Sendable` closure
            }
        }
    }
}

Swift continues to complain because, while I know it’s safe to transfer C FILE * values between threads, Swift doesn’t. So, when the Sendable closure pass to withLock(_:) captures a FILE * value, the compiler right complains.

The solution is to use the unchecked variants, to tell the compile that I know what I’m doing:

final class MyClass: Sendable {

    init(output: UnsafeMutablePointer<FILE>) {
        self._output = .init(uncheckedState: output)
    }
    
    private let _output: OSAllocatedUnfairLock<UnsafeMutablePointer<FILE>>

    var output: UnsafeMutablePointer<FILE> {
        get {
            self._output.withLockUnchecked { f in
                f
            }
        }
        set {
            self._output.withLockUnchecked { f in
                f = newValue
            }
        }
    }
}

Of course, as with all unchecked (and unsafe) stuff in Swift, you better really know what you’re doing when you do this. For example, I might have code like this:

final class MyClass: Sendable {

	…
    
    func run() {
        let f = self.output
        #warning("Unsafe")
        let buf = [UInt8]("Hello Cruel World!".utf8)
        fwrite(buf, buf.count, 1, f)
    }
}

This is not safe because, my code might be interrupted at the #warning line by some other thread that does this:

let m: MyClass = …
let f = m.output
m.output = fopen(…)
fclose(f)

When my run() method continues, it’ll write to a FILE * that’s been closed and, worse yet, deallocated.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

So is it fair to summarise this as that all the Sendable conformances on this method are based on the assumption that you will be returning the lock state from this method, and the lock state itself is not explicitly marked as Sendable? And if you do not return the state, but instead just modify it, then it is safe to just use withLockUnchecked?

And, by extension, IF the lock state was required to be Sendable, then we would not need any Sendable conformances on withLock() at all? (But the usefulness of OSAllocatedUnfairLock would be slightly diminished, of course.)

So is it fair to summarise this as that all the Sendable conformances on this method are based on the assumption that you will be returning the lock state from this method … ?

No, that’s not how I look at it.

Consider this snippet:

final class S: Sendable { }
final class NS { }

func test(s: S, ns: NS) {
    var f: (@Sendable () -> Void)? = nil
    
    f = {
        print(s)
    }
    f = {
        print(ns)
           // ^Capture of 'ns' with non-sendable type 'NS' in a `@Sendable` closure
    }
    
    …
}

The @Sendable means that you can only assign sendable closures to f. The first assignment to f works because s is Sendable. The second assignment to f errors [1] because it’s capturing ns, which is non-sendable.

Now consider the declaration of the two OSAllocatedUnfairLock methods:

public func withLockUnchecked<R>(_ body:           (inout State) throws -> R) rethrows -> R
public func withLock<R>(         _ body: @Sendable (inout State) throws -> R) rethrows -> R where R : Sendable

I’ve added whitespace to highlight the differences.

In the sendable case, the body closure is decorated with @Sendable, just like the f variable in my exapmle. So, the closure you post in must be sendable. If it tries to capture a non-sendable value then you can’t call the method. And that makes sense because the only reason to have a mutex is to run code from multiple concurrency contexts, and using non-sendable values across these contexts would be bad.

Finally, there’s that where R : Sendable clause. Note that R is different from State. This clause specifically outlaws the return of a non-sendable value. You could define a method that doesn’t have that clause:

extension OSAllocatedUnfairLock {
    func withLockQ<R>(_ body: @Sendable (inout State) throws -> R) rethrows -> R {
        …
    }
}

That would allow this code:

func test2(l: OSAllocatedUnfairLock<NS>, s: S, ns: NS) {
    let nsEscaped = l.withLockQ { ns in
        return ns
    }
    print(nsEscaped)
}

This would be bad, because ns is meant to be protected from concurrent access by the lock. That badness is what your question is focused on, but it’s only one part of this story.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] Well, warns, at least in Xcode 15.3.

No, that much is clear, my question here is, WHY is withLock marking its closure as @Sendable, since it should be executed synchronously in the same actor context?

Your last point seems to support what I asked: It's marked as @Sendable because the state is NOT marked as Sendable. If the state was marked as Sendable, then the closure would not need to be @Sendable, because the Sendable conformance on the state would protect it. It seems to me that the correct solution here would have been to mark State as Sendable instead, but I assume this was not done because you might want to protect a type that did not have an explicit Sendable conformance yet, so instead this quite confusing @Sendable conformance was added to other parts of the API?

But also, if so, why would the variation of withLock() that is available when State is Void be marked as @Sendable? This conformance means that OSAllocatedUnfairLock can not conform to NSLocking, which it otherwise could, and since there is no state there should be no need for this unexpected @Sendable conformance, right?

Focusing on the checked case [1], there three types in play here:

• State, that is, the value protected by the lock

• The type of body, the closure passed to withLock(_:)

• R, the result of that closure

State doesn’t have to be sendable. Indeed, that’s the whole point of the lock.

IMO R shouldn’t need to be Sendable, but rather it needs to be ‘transferrable’. That concept is currently being discussed on Swift Evolution; see SE-0430 transferring isolation regions of parameter and result values.

Your focus is on the type of body. IMO this really does need to be Sendable [1], otherwise you can do unsafe things. Consider this code:

class Nonsendable {
    var counter: Int = 0
}

let lock = OSAllocatedUnfairLock(uncheckedState: Nonsendable())
lock.withLock { state in
    state.counter += 1
}

This seems reasonable. Nonsendable is not thread safe but it’s protected by the lock.

Note I have to use init(uncheckedState:) here but that’s because OSAllocatedUnfairLock can’t yet take advantage of transferable.

Now add another thread that does this:

let newCounter = Nonsendable()
lock.withLock { state in
    state = newCounter
         // ^ Capture of 'newCounter' with non-sendable type 'Nonsendable' in a `@Sendable` closure
}

This is obviously unsafe, and it’s only the sendability constraint on the closure that prevents you from doing it.

Note All of this was tested in Xcode 15.3 with Strict Concurrency Checking set to Complete.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] Obviously none of this matters in the unchecked case.

In the unsafe example, though, the lock still has a State. If State is Void, though, this example doesn't apply. So, it still feels to me like the State == Void variant would not need to be @Sendable?