System Keychain not available from a Daemon

I've been trying to use Keychain from a Daemon for some time now. In the end, I managed to have the System Keychain work for my application and I moved to work on other parts.

I finally went back to dealing with Keychain, but the code I wrote before stopped working. Even the application I wrote to test things out stopped working for me, and now it gives the The authorization was denied. error.

To give more perspective into what I am doing, I am running a Sandboxed Launch Daemon wrapped in an App-like structure. I register it from my main app via SMAppService API. I also have a System Extension.

My test app was structured in the same way and I used the following code to put a new key into the System Keychain and get its reference:

var err: Unmanaged<CFError>?
let access = SecAccessCreateWithOwnerAndACL(getuid(), getgid(), UInt32(kSecUseOnlyUID | kSecHonorRoot), nil, &err)
if let err = err {
    log.error("Failed to create SecAccess: \(err.takeUnretainedValue().localizedDescription)")
}

let request = [
    kSecClass: kSecClassGenericPassword,
    kSecAttrService: service,
    kSecAttrAccount: account,
    kSecValueData: passwordData,
    kSecAttrAccess: access as Any,
    kSecAttrSynchronizable: false,
    kSecUseDataProtectionKeychain: false,
    kSecReturnPersistentRef: true,
] as [String: Any]

var result: CFTypeRef?
let status = SecItemAdd(request as CFDictionary, &result)

The goal of this was to share some secrets with a System Extension.

The code above worked for me some time ago and I was able to use the System Keychain from my sandboxed daemon.

Am I missing something again? Did something change in the meantime? Or did I do something last time that I haven't noticed?

Should I cut my losses and avoid Keychain since Apple will not support it anyway?

Answered by DTS Engineer in 801823022

OK, some factoids…

Your daemon cannot use the data protection keychain. Maybe some day, but not right now (as of macOS 15, which is currently in beta).

Your daemon may or may not be sandboxed.

To run sandboxed your daemon must be have a Info.plist with a bundle ID. You can either:

I generally favour the latter because it offers more flexibility. However, the former is required if you’re installing the daemon using the legacy SMJobBless API.

If you’re using SMAppService to install your daemon and the containing app is sandboxed then the daemon must be sandboxed [1].

In general, the App Sandbox prevents you from writing to the System keychain (unless you’re an NE sysex). You can get around this using a temporary exception entitlement (com.apple.security.temporary-exception.files.absolute-path.read-write). See this thread for more context.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] This is a new security restriction added in macOS 14.2.

First up, read TN3137 On Mac keychain APIs and implementations.

Amongst other titbits, it says:

Programs that run outside of a user context, like a launchd daemon, must target the file-based keychain. The data protection keychain is only available to programs running in a user context, like an app or an app extension.

So, the data protection keychain isn’t an option for you.

The goal of this was to share some secrets with a System Extension.

It’s possible to use the System keychain [1] for this, but it generally won’t do you want. The issue is that the System keychain file is only writable by root:

% ls -l /Library/Keychains/System.keychain 
-rw-r--r--  1 root  wheel  159816 22 Jul 11:30 /Library/Keychains/System.keychain

That means your app, which isn’t running as root, can’t write items to the keychain. That kinda puts the kibosh on this whole exercise.

My recommendation here is to share state with your daemon via IPC, most commonly XPC. If your daemon wants to save its secrets to the System keychain, it can do that because it’s running as root.

Oh, and if you have both a daemon and a sysex, both of them can do this job but it’ll simplify things if you centralise the work in one or the other.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] Which, to be clear, is a file-based keychain.

So, the data protection keychain isn’t an option for you.

Yes, I do know that I can't use the Data Protection Keychain, that's why I explicitly set the kSecUseDataProtectionKeychain to false because of that.

My problem here is that my Daemon can't access the System Keychain.

Let me be more clear, here is my setup:

UI --XPC--> Daemon <--> System Keychain
                            /\
                            \/
                System Extension

I have the main application talking with my Daemon via XPC, which then it tries to use the System Keychain. The one thing that might be unusual, is that my Daemon is in an App-like structure and is sandboxed (mostly because of SMAppService).

As I already mentioned, I wrote a test Daemon to test out the System Keychain, and the code I posted worked for me then. It now does not work.

Hi,

My test app was structured in the same way and I used the following code to put a new key into the System Keychain and get its reference

What did you mean by "app" here? If you run the code you sent in a standard app, you add the key into the user keychain, not the system keychain. You can (sort of) bypass that by running the app as root, but that creates a whole lot more problems than it solves. When Quinn says this:

My recommendation here is to share state with your daemon via IPC, most commonly XPC. If your daemon wants to save its secrets to the System keychain, it can do that because it’s running as root.

...part of the goal here is to bypass session level complexity by having the component that needs the keychain entry be the same component that saves the keychain entry. A bit advantage of that architecture is that issues like this shouldn't really happen:

My problem here is that my Daemon can't access the System Keychain.

...since it's difficult for a process to create a keychain entry that it can't read. Is your daemon creating an entry that it's unable to retrieve? If so, what's the error it's getting and what API is it getting the error from?

I have the main application talking with my Daemon via XPC, which then it tries to use the System Keychain. The one thing that might be unusual, is that my Daemon is in an App-like structure and is sandboxed (mostly because of SMAppService).

You've said this a few times and I'm concerned that there might be a misunderstanding going on here that's inadvertently introduced the issue you're fighting with. SO, stepping back for a moment, I think it's important to understand what exactly SMAppService does and what it actually requires.

Conceptually, SMAppService does a few things:

  1. Defines bundle directories so that agents and daemon's can live inside app bundles instead of at fixed locations.

  2. Provides an API so that an application can manage the loading/unloading of the objects inside it.

  3. Allows "bundle ID relative" resolution of launchd objects, so that launch agents/daemons don't have to exist at a fixed location.

What confuses me here is that what you're saying here:

Daemon is in an App-like structure and is sandboxed (mostly because of SMAppService).

Simply isn't true. That is, NOTHING in SMAppService requires ANY daemon to be in a bundle structure or to be sandboxed. The basic "rules" of launchd.plist construction are exactly the same as they were before SMAppService was created. Before SMAppService, you specified the target executable by providing a fixed path in the "Program" key. With SMAppService, you specify the target path by passing in an bundle relative path in "BundleProgram". Whether or not that target is inside app app bundle is totally irrelevant to launchd. Indeed, our sample project for this doesn't use a bundle for it's agent, nor does it place the target executable in one of the directories we've actually recommended placing executable code in.

You CAN have a daemon that's inside an app bundle but, again, that's ALWAYS been true. As far a launchd is concerned, it doesn't care if the "Program" target is at:

/Library/Application Support/mytool
/Library/Application Support/mycompany/mytoool
/Library/Application Support/mycompany/mycontainer.app/Contents/MacOS/mytoool

Or the "BundleProgram" is:

Contents/Helpers/mytool
Contents/Helpers/mycompany/mytoool
Contents/Helpers/mycontainer.app/Contents/MacOS/mytoool

All it cares about is whether or not the target exists. The fact that most daemon's haven't been insides app bundles is ENTIRELY a matter historical accident/choice, NOT because of any requirement, benefit, or even formal "choice".

The sandbox side of this is even more confusing, since launchd.plist's can be other tools on the system or even shell script. You can have a daemon that's sandbox'd, but nothing in launchd requires (or even cares) about that, as the sandbox is well outside of launchd's area of responsibility.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Hi!

Thanks for the excellent insight on what SMAppService does!

All it cares about is whether or not the target exists. The fact that most daemon's haven't been insides app bundles is ENTIRELY a matter historical accident/choice, NOT because of any requirement, benefit, or even formal "choice".

This is true. I've put my Daemon in an app bundle out of choice. I technically do not need to do that, but I found that it will make a few things easier.

I just wanted to be extra clear about that, since it looks like this isn't that common or recommended.

The sandbox side of this is even more confusing, since launchd.plist's can be other tools on the system or even shell script. You can have a daemon that's sandbox'd, but nothing in launchd requires (or even cares) about that, as the sandbox is well outside of launchd's area of responsibility.

The reason why I have my Daemon sandboxed is this:
If I try to register a daemon that is not sandboxed (irrelevant if it is in an app bundle or not) from a sandboxed application I get this error:

Error Domain=SMAppServiceErrorDomain Code=1 "Operation not permitted" UserInfo={NSLocalizedFailureReason=Operation not permitted}

After looking at Console, I also see this:

-[BTMManager registerLaunchItemWithAuditToken:type:relativeURL:configuration:uid:]_block_invoke: error: sandbox required

So I think this isn't a SMAppService requirement, but BackgroundTaskManagment's.

...since it's difficult for a process to create a keychain entry that it can't read. Is your daemon creating an entry that it's unable to retrieve? If so, what's the error it's getting and what API is it getting the error from?

When trying to use the System Keychina from the Daemon, I can see this log:

2024-07-23 16:57:01.577 A  Daemon[97140:bb0991] (Security) SecItemCopyMatching
2024-07-23 16:57:01.577 A  Daemon[97140:bb0991] (Security) SecItemCopyMatching_ios
2024-07-23 16:57:01.577 Df Daemon[97140:bb0991] [com.apple.securityd:secitemratelimit] Not internal release, disabling SIRL
2024-07-23 16:57:01.577 Df Daemon[97140:bb0991] [com.apple.securityd:keychain] Enabling System Keychain Always due to platform
2024-07-23 16:57:01.577 Df Daemon[97140:bb0991] [com.apple.xpc:connection] [0x6000025c8000] activating connection: mach=true listener=false peer=false name=com.apple.securityd.xpc
2024-07-23 16:57:01.577 Df Daemon[97140:bb0991] [com.apple.xpc:connection] [0x6000025c8000] failed to do a bootstrap look-up: xpc_error=[3: No such process]
2024-07-23 16:57:01.577 Df Daemon[97140:bb0991] [com.apple.xpc:connection] [0x6000025c8000] invalidated after a failed init
2024-07-23 16:57:01.577 Df Daemon[97140:bb0991] [com.apple.securityd:SecCritical] Failed to talk to secd after 4 attempts.
2024-07-23 16:57:01.577 Df Daemon[97140:bb0991] [com.apple.xpc:connection] [0x6000025cc0f0] activating connection: mach=false listener=false peer=false name=com.apple.security.XPCKeychainSandboxCheck
2024-07-23 16:57:01.608 Df Daemon[97140:bb0991] [com.apple.xpc:connection] [0x6000025cc0f0] invalidated after the last release of the connection object
2024-07-23 16:57:01.608 A  Daemon[97140:bb0991] (CoreFoundation) Loading Preferences From System CFPrefsD
2024-07-23 16:57:01.611 Df Daemon[97140:bb0991] [com.apple.securityd:mds] Recording an MDS plugin: /System/Library/Security/ldapdl.bundle {87191ca6-0fc9-11d4-849a-000502b52122}
2024-07-23 16:57:01.611 Df Daemon[97140:bb0991] [com.apple.xpc:connection] [0x6000025f8000] activating connection: mach=true listener=false peer=false name=com.apple.analyticsd
2024-07-23 16:57:01.612 Df Daemon[97140:bb0991] [com.apple.securityd:mds] Recording an MDS plugin: /System/Library/Frameworks/Security.framework {87191ca0-0fc9-11d4-849a-000502b52122}
2024-07-23 16:57:01.615 Df Daemon[97140:bb0991] [Daemon:general] Password ref missing, setting new to: vhncUYQr
2024-07-23 16:57:01.615 A  Daemon[97140:bb0991] (Security) SecItemAdd
2024-07-23 16:57:01.615 Df Daemon[97140:bb0991] [com.apple.securityd:security_exception] MacOS error: -25307
2024-07-23 16:57:01.615 Df Daemon[97140:bb0991] [com.apple.securityd:security_exception] MacOS error: -25307
2024-07-23 16:57:01.615 Df Daemon[97140:bb0991] [com.apple.xpc:connection] [0x6000025e44b0] activating connection: mach=false listener=false peer=false name=com.apple.authd
2024-07-23 16:57:01.620 Df Daemon[97140:bb0991] [com.apple.securityd:security_exception] MacOS error: -25307
2024-07-23 16:57:01.620 Df Daemon[97140:bb0991] [com.apple.securityd:security_exception] MacOS error: -25307
2024-07-23 16:57:01.624 Df Daemon[97140:bb0991] [com.apple.securityd:security_exception] MacOS error: -60005
2024-07-23 16:57:01.628 E  Daemon[97140:bb0991] [Daemon:general] Keychain error: The authorization was denied.

The last log contains the message corresponding to the OSStatus I got back from SecItemAdd.

Adding a bit more context to this log, I first try to get the persistent reference for a constant service and account using the following code:

let request = [
    kSecClass: kSecClassGenericPassword,
    kSecAttrService: service,
    kSecAttrAccount: account,
    kSecMatchLimit: kSecMatchLimitOne,
    kSecReturnPersistentRef: true,
] as [String: Any]

var result: CFTypeRef?
let status = SecItemCopyMatching(request as CFDictionary, &result)

Which returns errSecItemNotFound, so I assume the "Password ref is missing". Then I try to add the key again with the code I attached in my first message.

It looks like I can read the System Keychain and see that there is no key with the requested service and account, but I am unable to write to it anything.

Accepted Answer

This morning I had some time to research this further on my own.

I wrote a simple command line app that reads and writes to Keychain random strings. I sandboxed it to represent better what I am doing in my other application.

From my user, it worked perfectly. It read and wrote keys to my login keychain just fine.

But from root, it did not work. It failed with the same The authorization was denied. (-60005) error.

I looked back into Console logs and found this:

Sandbox: KeychainTest(1166) deny(1) authorization-right-obtain system.keychain.create.loginkc

There were also logs from authd similarly saying that sandbox denied authorization.

From what I can understand, it looks like it tries to obtain rights to create a login keychain (create.loginkc)? This is just my guess. I'm probably wrong.

It looked to me that it was trying to access something completely different, so just for a sanity check, I decided to try to explicitly open and use the System Keychain.

This worked perfectly.

For some reason, SecItemAdd is no longer trying to access the System Keychain. To fix my issue, all I had to do, was add this code:

var keychain: SecKeychain?
let openStatus = SecKeychainOpen("/Library/Keychains/System.keychain", &keychain)
guard openStatus == errSecSuccess else {
    throw KeychainError.underlyingSecurityError(openStatus)
}

// This is an example. I handle this like the SecAccess object
request[kSecUseKeychain] = keychain as Any

Previously I didn't have to do this. I didn't have to explicitly open the System Keychain to use it. I have no idea what changed, but this put a major wrench in my work.

I understand that the file-based keychains are "deprecated" but to change how SecItem handles them on the fly like this is a bit annoying.

As a note, wrapping a Daemon in an app bundle helps it access the System Keychain. If I didn't do that, I would need to add the com.apple.security.temporary-exception.files.absolute-path.read-write entitlement for the /Library/Keychains/ directory since Sandbox would deny temporary file creation there.

TLDR

Open the System Keychain explicitly, do not let SecItem choose what to use, even if it worked previously.

I think your trouble is due to sandboxing the launch daemon. I have a launch daemon that is not sandboxed, and it can use the System keychain, but has to run as root. Not ideal but the keychain stuff works fine for creating keypairs, using them for encrypt/decrypt and for adding an internet password. I wish Apple had better support for launch daemon access to their own private keychain. They should support a launchd plist key pointing to a keychain, and an entitlement to get unlocked access to it. The system should enforce that only the launch daemon can read the keychain file/db based on developer provided code signing requirements for the keychain.

OK, some factoids…

Your daemon cannot use the data protection keychain. Maybe some day, but not right now (as of macOS 15, which is currently in beta).

Your daemon may or may not be sandboxed.

To run sandboxed your daemon must be have a Info.plist with a bundle ID. You can either:

I generally favour the latter because it offers more flexibility. However, the former is required if you’re installing the daemon using the legacy SMJobBless API.

If you’re using SMAppService to install your daemon and the containing app is sandboxed then the daemon must be sandboxed [1].

In general, the App Sandbox prevents you from writing to the System keychain (unless you’re an NE sysex). You can get around this using a temporary exception entitlement (com.apple.security.temporary-exception.files.absolute-path.read-write). See this thread for more context.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] This is a new security restriction added in macOS 14.2.

System Keychain not available from a Daemon
 
 
Q