Demystify explicitly built modules

Hello and welcome. Today I’m going to introduce and explain the new way that Swift and Clang modules are built in Xcode called explicitly built modules. I will start with an overview of what modules are, then explain how modules are used to build your code, then cover the differences you will see in the new Xcode, and finally cover how to optimize your build by maximizing reuse of already-built modules.

At their core, modules are units of code distribution that describe the interface of a library or framework. A Swift target contains multiple Swift files that together represent a module, and in general, all the Swift source files in a single target or framework are part of the same module. The interface they represent is explicitly marked in Swift with access specifiers.

The class and its variables are marked as public, making them visible to importers.

Modules can also import other modules, forming an acyclic module graph for the entire project.

The Swift compiler takes the external interface you wrote and summarizes it into a textual .swiftinterface file that contains just the interface.

Modules in Objective-C are represented differently. Unlike Swift, in the C family of languages, the module’s interface is hand-authored.

We start with the well-known concept of headers and add a file on the side called a module map that describes how those headers comprise a module. This is an example of the UIKit module. We start with a header from UIKit, and then add UIKit’s module map. This module map tells the compiler a few different things about the UIKit module. First, that this is a framework module, then, that it is named UIKit. This name is what allows @import to work, as the source code itself doesn’t define a name.

Then it says that UIKit.h includes all the headers that are part of this module, and finally it says that every module this module imports can be used by code that imports UIKit.

You use modules whenever your code contains an import of a named module or the inclusion of a header that belongs to a module. This is a project called ResearchKit. It contains both Swift and Objective-C code using modules.

Let’s take a look at a Swift source file from ResearchKit. It starts with an import of SwiftUI, which is a Swift module, followed by an import of the ResearchKit module, which is a Clang module implemented in Objective-C in the project itself.

In the other tab is an Objective-C file. It contains some imports of project headers, an import of UIKit, and some includes of SDK headers which Clang transforms into module imports.

Modules also allow the compiler to share the parsing of interfaces between different source files. This is done by compiling each module in isolation into a binary file to be read by the compiler when compiling project sources, then importing the public interface of that module whenever it is referenced. In Swift, this compiled form is represented as a *.swiftmodule file, and in Clang this is represented as a *.pcm or precompiled module file.

But in order for this reuse to happen, those modules first need to be found and compiled. How does that happen? When the compiler encounters an import, it first discovers which module that import refers to, and then obtains the compiled representation of that module. When the compiler encounters @import UIKit, it first finds UIKit’s module map in the SDK. Now that it knows what the UIKit module is, it then needs to find the compiled .pcm file for UIKit, but what if that file doesn’t already exist? This brings us to the two high-level ways to build modules, and what’s different about Xcode 16. The first is implicitly built modules, where compilers coordinate among themselves to manage building modules without the rest of Xcode being aware of their existence. This is how Swift and Clang have built modules since their introduction.

When a project containing Swift, C and Objective-C code uses implicitly built modules, the build often contains many long-running tasks. Each row represents a separate execution lane that Xcode is able to run build tasks on. The build system starts compilation tasks, and each separate compiler implicitly discovers, and either builds a module, if it happens to get to it first, or loads modules that are already present as it goes.

In a timeline view, this may look something like this, where the compilation tasks at the start of the build take longer than those towards the end.

With implicitly built modules, you can end up with one compilation task implicitly building a module, while another task that also needs that module is blocked, waiting for it to be built.

This can even happen across many parts of the build.

Xcode 16 changes this with explicitly built modules, where Xcode coordinates with the compilers to discover and build modules. Explicitly built modules takes the implicit work of building modules and lifts it up into explicit build system tasks.

To do this, Xcode splits up the compilation of each source file into three separate phases of scanning, building modules, and finally building the original code.

Xcode starts by scanning each source file to build up a module graph for the entire project, even sharing modules across targets. As it constructs the module graph, it can also start dispatching module compilation tasks. These are explicit tasks in the build log that are directly provided with the compiled modules they depend on. The final step is the execution of the original compilation tasks after they have been modified to add the compiled modules they depend on.

For explicitly built modules, the timeline view now contains explicit scan tasks, module compile tasks, and the original source file tasks, but now taking significantly less time.

A benefit of the build system being aware of modules is that it can now avoid filling execution lanes with tasks that are not ready to run. Instead, it waits to run tasks until the modules they need are ready. This allows the build system to make efficient use of available execution lanes.

This approach to building modules has several benefits. The first is that builds are more reliable. With precise dependencies and deterministic build graphs exposed to the build system, the compiler is run the same way every time, and any build failures can be reproduced just by running the failing task again in isolation. No implicit state is maintained elsewhere. This also means that a clean build now rebuilds modules. This also enables more efficient builds. Now that the build system is fully aware of the module graph, it can make more informed scheduling choices, rather than having execution lanes blocked by compilation tasks waiting for a module to be built. Another benefit of the build system coordinating module builds is that it now passes Swift modules to the debugger when debugging in Xcode.

When a project is built using implicitly built modules, the Xcode build and the debugger have completely separate module graphs. With explicitly built modules the debugger is now able to reuse the already built modules. This avoids building modules again when the debugger needs to know about Swift types such as when evaluating expressions using “p” or “po”.

Explicitly built modules also impact how tasks appear in the build log.

In Xcode 16, explicitly built modules are used for all C and Objective-C code, and can be enabled as a preview for Swift. I’ll start by enabling explicitly built modules for Swift by selecting the ResearchKit project in the project navigator...

selecting 'Build Settings', then typing in “explicitly built” in the filter box.

I then select the 'Explicitly Built Modules' setting, and set it to 'Yes'.

Now that explicitly built modules are enabled for both Clang and Swift, I can start a build.

The build log will contain many scan tasks. These are run once for each source file in your project and produce a module import graph for the build system. This is a built in task, and does not spawn a new process. This allows the build system to cache information between source files being scanned.

The second new task is the compile module task. You will not find these tasks attached to any specific project or target. Instead these are top-level tasks, as they can be shared between targets. For these, the build system spawns a separate compiler process for each one. This task builds the specified module into a compiled module file. This is the work that happened during normal compilation in implicitly built modules and has now been split out. Any diagnostics encountered while building modules will be attached here, rather than to whatever source file happened to build the module first. You may notice that a single module is built multiple times. In this build, the UIKit module appears multiple times because some build settings on different targets require different variants of a module to be built. This build has 2 Swift module variants and 4 Clang module variants.

Looking closer at two of these, Xcode shows that they have different hashes. This hash represents the set of command line arguments that are needed to build this module variant. These flags are often things such as language standards, feature macros, or differences in include paths. This is something you will likely encounter often, and happened with implicitly built modules as well, but was more difficult to notice as module builds were not exposed in the build system. The compiler optimizes this list of arguments during scanning to remove those that did not impact how a module is built, including things such as unused header search paths. Lets go back to ResearchKit and remove the extra variants. First, I want to collect some extra info about the modules built by my project.

I’ll clean my build folder to rebuild all of the modules in my project and then use Product => Perform Action => Build with Timing Summary to collect additional information about build performance. Multiple module variants are caused by different source files having incompatible build settings. For example, a C source file and an Objective-C source file parse some modules very differently. Unnecessary module variants create additional work that must be performed by the build.

Some common sources of module variants are additional preprocessor macros, additional language modes, such as having a single C file mixed with Objective-C, or language versions such as using Objective-C with newer C versions such as C17, and disabling automatic reference counting. Now that the build is done, I go to filter and type in 'modules report'.

Selecting the Clang modules report shows that there are 2 variants of UIKit and many other modules.

Selecting the Swift module report also shows that there are 2 variants. These together explain the 4 different variants of UIKit that we previously identified in the build log. To reduce the number of variants, I can check the build settings that commonly cause them. I select ResearchKit in the project navigator, and in the filter box I type in 'macros' to see if I have any extra macros set. There are none at the project level, so I then go to the build settings for the ResearchKit target.

This target has an extra ENABLE_FEATURE macro not present on the other targets.

I select the 'Levels' mode to also see the project level settings at the same time.

I remove this macro from the target and put it up on the ResearchKit project.

I select 'Product', 'Clean Build Folder', then 'Build with Timing Summary' again.

This time, the build log shows just 3 UIKit variants. One for Objective-C, and two for Swift. One for the Clang module, and one for the Swift module.

By unifying project settings, we’re able to take these separate graphs and merge them into one. In general, you want to move build settings as broadly as is reasonable in your project. Instead of setting the language standard at the target level, it should be moved up to the project or workspace level. That way, modules can be shared between source files as much as possible. And that’s explicitly built modules.

The key things to remember from this talk are that explicitly built modules puts the build system in control of module builds. Your build logs will now look different, with the compilation time from module builds now showing up as its own task instead of being included in compile tasks, and you can reduce the number of module variants that get built by making your settings uniform across your project, allowing modules built for a given source file to be shared across targets. Thank you for watching.