Looks like the signed app losing entitlements

We would like to codesign up for the app that uses LuaJIT to be downloadable as the app with the identified developer on Apple silicon macOS. It means no targeting to the App Store which can be problematic due to LuaJIT usage.

Looks like there is no problem making the application run with the signature, but the performance is really bad.

All times are for running on an M2 chip, MacOS Sonoma 14.6.

Our x86_64 build works fine. Reference LuaJIT benchmark takes around 0.15 seconds (seed 2, 100 runs). Same build for arm64 with ad-hoc signature, no entitlements, and needs around 1.8 seconds (seed 2, 100 runs) to run the same benchmark code.

I created luajit_app in Xcode to investigate.

It simply opens a window, you select Lua script, and it runs it and prints output to the text area.

Signed by my developer ID, run from Xcode immediately after build:

I see the same behaviors for the x86_64 build. It needs around 0.43 seconds (seed 2, 1000 runs) to finish the benchmark code. The arm64 build without added entitlements needs around 16 seconds (seed 2, 1000 runs).

Added entitlements com.apple.security.cs.disable-executable-page-protection:

The arm64 build typically needs around 0.14 seconds (seed 2, 1000 runs).

Added entitlements com.apple.security.cs.allow-jit which fixed LuaJIT to use MAP_JIT flag:

The arm64 build typically needs around 0.14 seconds (seed 2, 1000 runs). 2nd and other app runs need around 19 seconds for benchmark.

Ad-hoc signed without developer ID and team, com.apple.security.cs.allow-jit:

Run from Xcode

The first app runs after the build/rebuild

The arm64 build typically needs around 0.14 seconds (seed 2, 1000 runs), but the first run sometimes takes around 5 seconds (seed 2, 1000 runs).

2nd and next runs of the app

The arm64 build typically needs around 19 seconds (seed 2, 1000 runs).

Bad signed, signature fix from the command line:

Signed with codesign --force --deep --sign MYID -o runtime --entitlements entitlements.plist luajit_app_bad_sign.app or AD-HOC

Behaviors are similar to Xcode runs. The first time the app runs usually takes around 5 seconds and 0.14 seconds later for benchmark script. Sometimes first benchmark runs takes 5 seconds, the second run 19 seconds and later runs take 0.14 seconds.

Later app runs typically fall to 19 seconds needed to do benchmark script.

End

I have also tried ad-hoc and the developer signature with both entitlements for the origin app, but no difference in time needs for the benchmark was observed.

Any ideas what is going on?

Answered by DTS Engineer in 805080022
It means no targeting to the App Store which can be problematic due to LuaJIT usage.

To be clear, the Mac App Store places no restrictions on the use of JIT [1]. The situation for iOS and other platforms is more nuanced.

Bad signed, signature fix from the command line:

You’re signing your code with --deep, which is a bad idea. See --deep Considered Harmful. If you’re going to sign code manually, rather than with Xcode, check out:

As to your performance questions, it’s hard to offer any insights from the ‘outside’. You are benchmarking Lua itself, and I’ve no idea what’s going on inside it. For example, how does it decide whether to run JITed or not? And what does its JIT code look like?

I will say that:

  • com.apple.security.cs.allow-jit and MAP_JIT represents the only path that I care to follow. The other hardened runtime exception entitlements are a compatibility sop, not something you want to be using long term.

  • com.apple.security.cs.disable-executable-page-protection is pointless on Apple silicon; it behaves exactly like com.apple.security.cs.allow-unsigned-executable-memory.

  • While MAP_JIT is necessary, it’s not sufficient on Apple silicon. Porting just-in-time compilers to Apple silicon has the latest news, but there’s also the older pthread_jit_write_protect_np.

IMO you need to dig into the Lua code to see how it’s using these facilities. If you need help with the Apple APIs involved, I’m happy to wade in.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] In fact, you could argue that the Mac App Store is less restrictive than direct distribution with Developer ID because it doesn’t require the hardened runtime. However, I strongly recommend that you enable the hardened runtime regardless of your deployment channel. It’s both forward looking and good for security.

It means no targeting to the App Store which can be problematic due to LuaJIT usage.

To be clear, the Mac App Store places no restrictions on the use of JIT [1]. The situation for iOS and other platforms is more nuanced.

Bad signed, signature fix from the command line:

You’re signing your code with --deep, which is a bad idea. See --deep Considered Harmful. If you’re going to sign code manually, rather than with Xcode, check out:

As to your performance questions, it’s hard to offer any insights from the ‘outside’. You are benchmarking Lua itself, and I’ve no idea what’s going on inside it. For example, how does it decide whether to run JITed or not? And what does its JIT code look like?

I will say that:

  • com.apple.security.cs.allow-jit and MAP_JIT represents the only path that I care to follow. The other hardened runtime exception entitlements are a compatibility sop, not something you want to be using long term.

  • com.apple.security.cs.disable-executable-page-protection is pointless on Apple silicon; it behaves exactly like com.apple.security.cs.allow-unsigned-executable-memory.

  • While MAP_JIT is necessary, it’s not sufficient on Apple silicon. Porting just-in-time compilers to Apple silicon has the latest news, but there’s also the older pthread_jit_write_protect_np.

IMO you need to dig into the Lua code to see how it’s using these facilities. If you need help with the Apple APIs involved, I’m happy to wade in.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] In fact, you could argue that the Mac App Store is less restrictive than direct distribution with Developer ID because it doesn’t require the hardened runtime. However, I strongly recommend that you enable the hardened runtime regardless of your deployment channel. It’s both forward looking and good for security.

@DTS Engineer

You’re signing your code with --deep, which is a bad idea. See --deep Considered Harmful. If you’re going to sign code manually, rather than with Xcode, check out:

No difference with and without --deep. The first app run works well, next runs work badly. I experimented with many codesign variants, I started with a version copied from Xcode .xcactivitylog.

However, I strongly recommend that you enable the hardened runtime regardless of your deployment channel.

Does it mean I cannot check if hardened runtime entitlements take effect with the developer/ad-hoc signature? I plan to check it locally with ad-hoc or personal developer ID if I can fix that and pay for the developer account only if I see, that it makes sense to do.

No difference with and without --deep.

… currently. One of the many problems with --deep is that it applies the same entitlements to all of your code. If at some point in the future you start using a restricted entitlement — one that must be authorised by a profile — then you can run into problems. Specifically:

  • If your app contains any helper tools, they get signed with the entitlement, which will prevent them from running because they don’t have a profile.

  • If your app contains any dynamic libraries or frameworks, the presence of the restricted entitlement on those libraries will prevent it from launching!

Signing with --deep is like digging a hole, filling it full of spikes, and covering it over with sticks and leaves. Sooner or later someone is going to fall in.

Does it mean I cannot check if hardened runtime entitlements take effect with the developer/ad-hoc signature?

No. You can test all three signing types locally with or without the hardened runtime. And on the distribution front, only Developer ID signing requires the hardened runtime (presuming that you want to notarise in order to pass Gatekeeper).

However, IMO it’s largely a distraction. Your code will eventually end up running with the hardened runtime enabled, so that’s what you need to be testing. The only reason I can see for disabling it is to establish your performance baseline.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

@DTS Engineer Thanks for the information. I believe I will use them later.

Now, back to the problem.

In Xcode, I use "Clean build folder", enable "Allow Unsigned executable memory" in Signing & Capabilities, and let Xcode automatically manage signing with my personal developer team and my Developer certificate/ID.

I use Product -> Build followed by Product -> Run, and I am happy. I click on the "Do Lua script" button and it takes 0.1 seconds to finish the script. I use Product -> Run again (no rebuild, resigning), and I am surprised, I click on the "Do Lua script" button, select the same script and it takes 18 seconds to finish it.

Sometimes 2nd, 3rd runs still work well.

So, it looks like entitlements "Allow Unsigned executable memory" work as expected but unfortunately only some short time after the app signs. If it is a problem in the application I expect a deterministic entitlement behavior if I use the same inputs. I see also the same call sequences in the debugger.

This does not look like an application problem to me but like some crazy signing issue. I did not find anything successful to fix it, so I am here.

What should I do to determine what is going on and how to fix it?

This does not look like an application problem to me but like some crazy signing issue.

That’s quite a leap. There are many potential causes for a performance problem like this.

I use Product -> Run again (no rebuild, resigning)

Instead of this, try Product > Perform Action > Run Without Building. That’s guarantee to run exactly the same code. Do you still see the massive slowdown in the second run?

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

@DTS Engineer Yes, those crazy behaviors do not depend on how I run the application. It has identical behaviors if I run it from Finder after the build.

I dig more into it. And it looks like the mmap behavior changes after signing for a short time. But not anything out from what is documented. And this change looks to be a reason for what I see.

Looks like the signed app losing entitlements
 
 
Q