IOAudioDevice

Every audio driver based on the Audio family must have one instance of a custom subclass of IOAudioDevice. The driver’s IOAudioDevice object is the central, coordinating node of the driver’s object tree—the “root” object. All other objects ultimately depend on it or are contained by it.

Because of its status as root object, the IOAudioDevice object represents the audio hardware generally. This central position gives it several roles:

• It is the object that usually matches against the provider’s nub.

• It initializes the device, mapping hardware resources from the provider’s nub, and otherwise reads and writes to the device registers as necessary.

• It creates the IOAudioEngine objects of the driver and can create the IOAudioControl objects used by the driver.

• It usually manages synchronization of values between hardware controls and the associated software controls associated with its audio-engine objects.

• It acts as the power controller and policy maker for the audio hardware; in this role, it must respond to system sleep, system wake, and domain idleness transitions by deactivating and reactivating its audio engines as necessary.

The driver’s IOAudioDevice object fulfills other functions. It controls access to the audio hardware to ensure, in the driver’s multithreaded environment, that the hardware doesn’t get into an inconsistent state. Toward this end, the IOAudioDevice superclass provides a separate work loop (IOWorkLoop) and a command gate (IOCommandGate) to synchronize access by all objects in the driver. All other objects in the driver—IOAudioEngine, IOAudioStream, and IOAudioControl—contain references to the IOAudioDevice’s work loop and the command gate as instance variables. All Audio family classes take care of executing I/O and hardware-related code on the command gate for all calls into the driver that they know about. Generally, drivers should ensure that all I/O and hardware-specific operations are executed with the command gate closed.

IOAudioDevice also offers timer services to the audio driver. These services allow different objects within the driver to receive notifications that are guaranteed to be delivered at the requested timer interval, if not sooner. Different target objects can register for timer callbacks at a specific interval; however, IOAudioDevice makes the actual timer interval the smallest of those requested.

The idea behind this design is that there is no harm in having timed events in an audio driver occur sooner than requested. By coalescing the callback intervals, the Audio family obviates the overhead of multiple timers in a single driver.

In some cases, however, this may result in unexpected behavior if you make assumptions based on the amount of time elapsed, such as assuming that the hardware has played a certain number of samples. You should thus always make certain to test to make sure conditions are appropriate before performing such operations.

Your driver itself can have localized strings that are accessible by the Audio HAL. These strings can include such things as name, manufacturer, and input sources. Follow the OS X localization procedure for these strings, putting them in a file named Localizable.strings in the locale-specific subdirectories of your bundle. The driver should have a property named IOAudioDeviceLocalizedBundleKey, which has a value of the path of the bundle or kernel extension holding the localized strings, relative to /System/Library/Extensions. The driver’s IOAudioDevice object should set this property in its implementation of the initHardware method.

IOAudioEngine

An audio engine object represents and manages an audio device’s I/O engine. In an audio driver, the object is an instance of a custom subclass of IOAudioEngine. The audio engine has two main roles:

• To configure a hardware DMA engine to transfer audio data (in the form of a stream of sample frames) between the device and the sample buffer at a specific sampling rate. (In the absence of a hardware DMA engine, the audio engine may emulate this functionality in software.)

• To move data between the sample buffer and the mix buffer after appropriately converting the data to the format expected by the client or hardware (depending on direction).

You can find more information on this topic in The Audio I/O Model Up Close.)

An instance of IOAudioStream (described in IOAudioStream) represents and encapsulates a sample buffer in a driver (and a mix buffer for output streams). Each IOAudioEngine in a driver must create one or more IOAudioStream objects for each sample buffer required by the I/O engine. A typical driver has at least an input IOAudioStream and an output IOAudioStream.

An IOAudioEngine object may also create the audio-control objects (IOAudioControl) required by the device, although this task can be handled by the driver’s IOAudioDevice. During the initialization phase, the driver must add all created IOAudioStream instances and IOAudioControl instances to the IOAudioEngine as instance variables using the appropriate IOAudioEngine methods. This must happen before it activates the IOAudioEngine (with the activateAudioEngine method).

In addition to facilitating the transfer of audio data in and out of the sample and mix buffers, an IOAudioEngine has a number of functions:

• It must stop and start the I/O engine when requested.

• When the I/O engine is started, the IOAudioEngine object must ensure that it runs continuously and, at the end of the sample buffer, loops to the beginning of the buffer. As the engine loops, the IOAudioEngine takes a timestamp and increments a loop count.

• It must provide the current sample on demand.

• If the IOAudioEngine supports multiple stream formats or sampling rates, it must modify the hardware appropriately when a format or rate changes.

An IOAudioEngine has several attributes and structures associated with it. Most important of these is a status buffer (IOAudioEngineStatus) that it shares with the Audio HAL. This status buffer is a structure that the IOAudioEngine must update each time the I/O engine loops around to the start of the sample buffer. The structure contains four fields, three of which hold critical values:

• The number of times the audio engine has looped to the start of the sample buffer

• The timestamp of the most recent occurrence of this looping

• The current location of the erase head (in terms of sample frame)

It is important that these fields, especially the timestamp field, be as accurate as possible. The Core Audio framework (Audio HAL) uses this timing information to calculate the exact position of the audio engine at any time. The shared status buffer is thus the basis for the timer and synchronization mechanism used by the OS X audio subsystem.

The erase head mentioned in the previous paragraph is another attribute of an IOAudioEngine object. The erase head is a software construct that zeroes out the mix and sample buffers just after the sample frames have been played in an output stream. It is always moving just behind the audio engine to avoid erasing data that has not yet been played. However, it also must remain well ahead of the IOAudioEngine clipping and conversion routines that convert the audio data in the mix buffer to ensure that no stale data from a previous loop iteration is mixed or clipped.

An IOAudioEngine object performs a number of initializations to fine-tune the synchronization mechanism described above. For example, it provides methods for setting the latency of the audio engine and for varying the offset between the Audio HAL and the audio engine’s I/O head.

IOAudioStream

An IOAudioStream object represents a single, independently addressable audio input or output stream (which may include multiple channels). It contains the following (as instance variables):

• A sample buffer

• A mix buffer (for output streams)

• Supported format information (sample rate, bit depth, and number of channels)

• The starting channel ID

• The number of current clients

• All IOAudioControl objects that affect the channels of the stream

An IOAudioStream is an instance variable of the IOAudioEngine object that creates it. When the audio engine creates an IOAudioStream object, it must list all supported sample formats as well as all the supported sample rates for each format. The current format must be explicitly set.

If a sample buffer has multiple channels, the channels are typically interleaved on a frame-by-frame basis. If your hardware uses separate buffers for each channel, however, you may use separate IOAudioStream instances for different channels.

The IOAudioStream class defines the AudioIOFunction type for the callbacks (typically implemented by the owning IOAudioEngine) that clip and convert output audio data from the float mix buffer to the sample buffer in the format required by the hardware. See The Audio I/O Model Up Close for further information.

IOAudioStream includes convenience methods that permit IOAudioStream objects to be created from and saved to OSDictionary objects.

IOAudioControl

An IOAudioControl object represents a controllable attribute of an audio device, such as mute, volume, input/output selector, or master gain. It is usually associated with a specific channel in a specific stream, but can be used to control all channels in an IOAudioStream or even all channels in an IOAudioEngine.

IOAudioControl objects are typically instance variables of the owning IOAudioEngine object. However, IOAudioControl objects associated with a specific stream may also be stored in the relevant IOAudioStream object.

Usually an instance of an IOAudioEngine subclass creates its IOAudioControl instances when it initializes the hardware (in the initHardware method). However, the driver’s IOAudioDevice object may be the object that creates the necessary IOAudioControl objects. In either case, the driver must add the control objects to the appropriate IOAudioEngine using the addDefaultAudioControl method.

Thus an IOAudioControl object is associated with an IOAudioEngine object, an IOAudioStream object, and a channel of the stream. All of its attributes are stored in the I/O Registry. It is known as a “default control” because the Audio HAL recognizes and uses it based on its attributes, which are discovered through the I/O Registry.

When the IOAudioEngine (or IOAudioDevice) object creates IOAudioControl objects, it must obtain from the audio device the starting channel identifier (an integer) for the audio stream. When the driver creates the first IOAudioControl for the stream, it assigns this channel ID to it. When it creates IOAudioControl objects for any other channels of the stream (based on the number of channels the stream supports), it simply increments the ID of the channel associated with the control.

For audio devices with more than one audio stream, each stream should start at the next free ID beyond the highest numbered ID that the previous stream could contain. This can be obtained by adding the maximum number of channels in any given stream format to the starting ID.

The Audio family assigns enum identifiers to channel IDs in IOAudioTypes.h; these include identifiers for left, right, center, left-rear, and right-rear channels, as well as an identifier for all channels.

In addition to channel ID, the Audio family uses a multitier classification scheme (defined by enums in IOAudioTypes.h) to identify IOAudioControl types:

• Audio types: output, input, mixer, pass-through, and processing

• Audio subtypes:

  • For output: internal speaker, external speaker, headphones, line, and S/PDIF

  • For input: internal microphone, external microphone, CD, line, and S/PDIF

• Control types: level and selector

• Control subtypes: volume, mute, input, output, clock services

• Usage type: input, output, and pass-through

When you create an IOAudioControl object, you specify control type, control subtype, usage type, and channel name (in addition to channel ID).

The level and selector control types correspond to subclasses of IOAudioControl, described in Table 2-1.

Some objects in an audio driver—typically the IOAudioDevice object because of its central role—must implement what is known as “value change handlers.” A value change handler is a callback routine that conforms to one of three prototypes defined in IOAudioControl.h based on the type of value (integer, OSObject, or void * data). When invoked, a value change handler should write the change in value to the audio hardware.

Changes to control values that originate with clients of the Audio HAL—for example, a user moving the volume slider in the menu bar—initiate a long series of actions in the Audio HAL and the Audio family:

1. The Audio HAL goes through the I/O Registry to determine the property or properties associated with the value change.

2. Via the IOAudioEngineUserClient object, the IOAudioControl superclass’s implementation of setProperties is invoked.

3. Using the dictionary of properties passed into setProperties, IOAudioControl locates the target control object and calls setValueAction on it.

4. The setValueAction method calls setValue on the driver’s work loop while holding the driver’s command gate.

5. The setValue method first calls performValueChange, which does two things:

   1. It calls the value change handler for the IOAudioControl (which must conform to the appropriate function prototype for the callback).

   2. It sends a notification of the change to all clients of the IOAudioControl (sendValueChangeNotification).

6. Finally, setValue calls updateValue to update the I/O Registry with the new value.

When a change is physically made to audio hardware—for example, a user turns a volume dial on an external speaker—what must be done is much abbreviated. When the driver detects a control-value change in hardware, it simply calls hardwareValueChanged on the driver’s work loop. This method updates the value in the IOAudioControl instance and in the I/O Registry, and then sends a notification to all interested clients.

User Client Classes

The Audio family provides two user-client classes, IOAudioEngineUserClient and IOAudioControlUserClient. The Audio family automatically instantiates objects of each class for each IOAudioEngine object and each IOAudioControl in a driver. These objects enable the communication of audio data and notifications between the driver and the Audio HAL. You should not have to do anything explicitly in your code to have the default user-client objects created for, and used by, your driver.

For further details, see User Client Objects.

IOAudioPort

The IOAudioPort class instantiates objects that represent a logical or physical port, or a functional unit in an audio device. An IOAudioPort object represents an element in the signal chain in the audio device and may contain one or more IOAudioControl objects through which different attributes of the port can be represented and adjusted.

The IOAudioPort class is deprecated and may eventually be made obsolete. The class is currently public to maintain compatibility. Driver writers are discouraged from using IOAudioPort objects in their code.

Interpolation

Before going further, it is worthwhile to consider some of the theory behind this design. The OS X audio system makes the assumption that a hardware I/O engine, as it processes audio data in the sample buffer, is proceeding continuously at a more or less constant rate. The “more or less” qualification is important here because, in reality, there will be slight variations in this rate for various reasons, such as imperfections in clock sources. So the mechanism by which the Audio HAL continually uses timestamps to calculate and predict a wake-up time for each of its client I/O threads can be considered an interpolation engine. It is a highly accurate predictive mechanism that “smooths out” these slight variations in engine rate, building in some leeway so that there is no discernible effect on audio quality.

Client Buffers and I/O Procedures

As noted earlier, each client of the Audio HAL can define the size of its audio buffer. There are no restrictions, except that the buffer can be no larger than the size of the hardware sample buffer. For performance reasons, almost all clients prefer buffer sizes that are considerably smaller. Buffer sizes are typically a power of two. The Audio HAL takes the buffer sizes of its clients into account when it calculates the next I/O cycle for those clients.

Each client of the Audio HAL must also implement a callback function conforming to the type AudioDeviceIOProc. When the Audio HAL wakes a sleeping client I/O thread, it calls this function, passing in the buffers (input and output) whose sizes were specified by the client. It is in this implementation of the AudioDeviceIOProc routine that the client gives audio data to the hardware or receives it from the hardware.

The following section, A Walk Through the I/O Model, discusses what happens next in detail.

Output Streams

Let’s begin with an output stream. Figure 2-4 illustrates the relationship between the buffers of Audio HAL clients and the buffers of the audio driver during an output cycle. Refer to this diagram during the following discussion.

For each of its clients, the Audio HAL calculates intervals that are based on the accumulated timestamps and loop counts associated with an I/O engine as well as client buffer sizes. The Audio HAL sleeps the I/O threads of its clients for these intervals, waking each thread when it’s time for the client to give the hardware its data. In waking the thread, it calls the AudioDeviceIOProc routine implemented by the client, passing in a number of buffers and timestamps:

• A list of input buffers along with a timestamp that indicates when the data was recorded

• A list of output buffers along with a timestamp that indicates when the data will be played

• A timestamp to be used for “now” rather than the device clock

The input and output timestamps allow the client to make various calculations, such as how much time it has before the data is played. The inclusion of both input and output parameters enables clients that are both producers and consumers of audio data (for example, a recording unit with playback capabilities) to process both streams at the same time. In this case, the client first takes the data in the list of input buffers before filling the output buffers with 32-bit floating-point samples.

When the client returns in its AudioDeviceIOProc routine, the Audio HAL puts the I/O thread to sleep until the next time data is required from the client. The Audio HAL gives the samples in the output buffer to the associated IOAudioEngineUserClient object, which calls the appropriate IOAudioStream object to have the samples moved from the client buffer to the appropriate frames in the engine’s mix buffer. Other clients can also deposit data in the same locations in the mix buffer. If another client already has deposited data in those frames, the new client’s floating-point values are simply added to the existing values.

Clients can contribute output data to a frame almost until the I/O engine is ready for that data. The IOAudioEngine object containing the mix buffer knows how many clients it has and when each has contributed its share of data to any one frame of the mix buffer (for the current loop through it). In addition, the driver (through the Audio family) maintains a “watchdog” timer that tracks the current location of each client relative to the I/O engine. If a client has not provided audio data by the time the I/O engine needs to accesses it, the watchdog timer fires and clips all of the currently mixed samples into the sample buffer.

Because some time is needed to perform this clip operation, the watchdog actually fires a short amount of time before the data is needed. It is possible that a “late” client could attempt to put data in the location of the mix buffer after the watchdog has fired but before the I/O engine has processed the data. To accommodate this situation, the driver backs up and remixes and clips the data in an attempt to get the “late” samples to the I/O engine in time.

Next, the driver’s clip routine, clipOutputSamples, is invoked. In its implementation of this method, the driver must clip any excess floating-point values under –1.0 and over 1.0 —which can happen when multiple clients are adding their values to existing values in the same frame—and then convert these values to whatever format is required by the hardware. When clipOutputSamples returns, the converted values have been written to the corresponding locations in the sample buffer. The DMA engine grabs the frames as it progresses through the sample buffer and the hardware plays them as sound.

Since a picture is worth a thousand words, the interaction of these processes is described in Figure 2-5.

Erase Heads and Timer Services

The Audio family includes a further refinement to the synchronized actions described in the preceding paragraphs. Between the I/O engine and the clipping and converting done by the driver, it runs parallel “erase heads” in both the mix and sample buffers. These erase heads simply zero-fill the corresponding frames at the same time. This precaution reduces the possibility that any frame could become polluted with leftover bits.

The erase heads are run in a separate thread and have their own timer. They are programmed to run four times per sample-buffer cycle. They do not erase the entire range of frames between the current locations of the DMA engine and the driver’s clip routine, allowing a little space for the remixing of data from tardy clients.

The erase head’s timer is run using IOAudioDevice’s timer services. Its interval is, of course, closely tied to the rate of the I/O engine and the timestamps taken by the IOAudioEngine.

Input Streams

With input audio streams, the picture is much simpler. There is no mix buffer and there are no erase heads. The Audio HAL clients are consumers of the data in this case, and not producers of it, so there is no need for these things.

Neither is there any need for a clip routine. The driver has to convert the integer data coming from the hardware to the 32-bit floating point required by the Audio HAL. But in the input direction, the driver is in a position to fit the converted data within the –1.0 to 1.0 floating-point maximum range.

So the simplified sequence is this: shortly after the I/O engine writes the input data into the sample buffer, the driver—in its implementation of the IOAudioEngine method convertInputSamples—converts that data to 32-bit floating point. Then the data is given, via the IOAudioEngineUserClient interface, to each Audio HAL client in that client’s AudioDeviceIOProc callback routine.