Firmware
Sound Open Firmware
Sound Open Firmware is an open source audio DSP firmware and SDK intended to provide a generic audio firmware infrastructure and development tools to the community. The firmware is BSD licensed and is platform and architecture agnostic. The SDK is also open source and provides a toolchain, debugger, emulator and firmware image creation tools.
The firmware and SDK are intended for developers who are interested in audio or signal processing on modern DSPs or for developers who are interested in micro kernels running on small but powerful processors.
The SDK comprises of five source components :-
1) Sound Open Firmware source code. The firmware is written in C with some architecture specific assembler and does not link to any external dependencies.
2) Sound Open Firmware tools. These are bespoke tools required to convert firmware from the ELF file format to formats understood by the kernel drivers and tools to assist with debugging running firmware images.
3) ASoC Linux kernel drivers. An ASoC kernel driver is required to register the DSP and firmware as a kernel audio device and expose PCMs, kcontrols etc. This driver will also load any topology data.
4) Crosstool-NG tool chain. Crosstool-NG is used to build a GNU cross tool chain (gcc, gdb, binutils, etc.) that is used to build the firmware binaries. Other compilers and tool chains can also be used to build the firmware.
5) Qemu emulator used to emulate the DSP and host. Qemu is used to provide a functional emulator to trace and debug driver and DSP firmware code simultaneously.
This page is currently work in progress so please expect frequent updates and corrections.
Installing the SDK
1. Install the firmware image tools.
These tools are used to convert ELF binaries into the file format used by the kernel drivers when loading the firmware.
git clone git://git.alsa-project.org:sound-open-firmware-tools.git soft.git cd soft.git ./autogen.sh ./configure make make install
This will install the 'rimage' and 'rmbox' tools which are used for firmware image creation and firmware debugging respectively.
2. Build and install the GNU cross compiler tool chain using Crosstool-NG.
A cross tool chain is required to build firmware binaries on x86. This tool chain can either be built from source (e.g. GCC/LLVM) or supplied by the DSP vendor. Luckily crosstool-NG takes all the effort out of building a GNU cross compiler and makes the process quite simple. Sound open Firmware currently just needs the xtensa cross compiler and a modified crosstool-NG is used to build the xtensa cross tool chain.
git clone https://github.com/01org/osadsp-crosstool-ng.git ct-ng.git cd ct-ng.git
Build crosstool-NG
./bootstrap ./configure --prefix=`pwd` make make install
Now build the xtensa cross compiler.
./ct-ng xtensa-byt-elf ./ct-ng build
Finally add the Xtensa cross compiler to your PATH.
PATH=$PWD/builds/xtensa-byt-elf/bin:$PATH
Todo: We are missing some headers like stdint.h from cross gcc tarball. Add note on adding them.
Please note that the xtensa GCC compiler only generates instructions using the core xtensa ISA and does not vectorize or use media instructions when generating Xtensa binaries (i.e. no automatic SIMD or media instructions generated without hand coding assembler). Cadence does however provide a free download of their optimising compiler tool chain for the Baytrail target here (todo add link).
3. Get the firmware source code.
The firmware source code is hosted using git on alsa-project.org
git clone git://git.alsa-project.org:sound-open-firmware.git sof.git cd sof.git ./configure --with-arch=xtensa --with-platform=baytrail --with-tool-dir=~/path/to/xtensa/gcc/xtensa-byt-elf --host=xtensa-byt-elf make make bin
This will create a binary firmware image in src/arch/xtensa/ called reef-byt.ri. This image can now be copied to /lib/firmware/intel where it will be loaded by the Baytrail audio DSP driver.
4. Get the Linux driver updates.
This firmware requires some driver updates and they can be found here :-
git://git.kernel.org/pub/scm/linux/kernel/git/lrg/asoc.git #topic/reef-v4.8
TODO: add instructions to enable the correct kconfig options
5. Optional. Build the Qemu DSP emulator from source.
Qemu needs some extra patches to support heterogeneous virtualization of the guest x86 OS and xtensa firmware. We are using the latest qemu-2.7 release with the support for heterogeneous virtualization and audio DSP hardware on top.
sudo apt-get install libgtk-3-dev libsdl-dev libspice-protocol-dev libspice-server-dev libusb-1.0-0-dev libusbredirhost-dev libtool-bin iasl valgrind texinfo virt-manager kvm libvirt-bin virtinst git clone https://github.com/01org/osadsp-qemu.git cd osadsp-qemu.git ./configure --prefix=. --target-list=xtensa-softmmu,x86_64-softmmu --enable-gtk --enable-sdl --enable-spice --audio-drv-list=alsa --enable-libusb --enable-usb-redir --enable-coroutine-pool --disable-opengl --enable-fdt make
Please do not install this version of Qemu as it will overwrite your distro version of Qemu.
Running the Firmware
The firmware image files should be copied to the Linux firmware directory in order for them to be loaded by the ALSA driver. This is usually /lib/firmware but some distributions may use other locations. The Intel platforms use the Intel sub directory, so firmware should be copied to /lib/firmware/intel for Intel platforms.
The drivers are responsible for making sure the firmware boots once it's been loaded. The firmware will send a "boot complete" IPC message to the driver once it has successfully booted otherwise the driver will time out the firmware boot and report this in the kernel logs.
If the firmware fails to boot then it can be made to dump boot tracing information to the host. Please see firmware debugging.
Firmware Architecture
The diagram below shows the high level firmware architecture with the Baytrail platform integration. The firmware is split into four main sections :-
1) Generic micro kernel. The micro kernel manages and abstracts the DSP hardware for the rest of the system. It also exports C APIs for memory allocation, scheduling work, event notifications and power management.
2) Audio components. The audio components can be used to form an audio processing pipeline from host DMA buffer to DSP digital audio interface. Audio components will have a source and sink buffer where they will usually transform or route audio data as part of their processing.
3) Audio task. The audio task manages the audio pipelines at run time. i.e. it manages the transportation of data from source to sink component within the pipeline. The pipelines are currently statically defined in the firmware, but infrastructure is now in place to allow the dynamic creation of pipelines from Linux userspace.
4) Platform drivers. The platform drivers are used to control any external IP to the DSP IP. This will usually be things like DMA engines or DAI (Digital Audio Interface) controllers. These drivers are used by the audio components and pipelines to send/receive data to/from the host and external codecs.
The left hand side of the diagram shows the Linux kernel ASoC driver and the mailbox/doorbell mechanism for IPC between the host and DSP.
Driver Architecture
The ASoC driver architecture for Sound Open Firmware is shown in the diagram below. The driver architecture is also split into four layers, like a protocol stack, each with a different purpose.
1) Machine driver. The ASoC machine driver does all the machine/board audio hardware integration. It also glues the platform driver and drivers for any codec(s) together so they appear as a single ALSA sound card. Sound Open Firmware can reuse existing upstream machine drivers (as only the platform name need changed) or can have bespoke machine drivers.
2) Generic PCM Driver. The PCM driver creates ALSA PCMs, DAPM and kcontrols based on the topology data loaded at run time. The PCM driver also allocates buffers for DMA and registers with run time PM. It is architecture and platform generic code.
3) Generic IPC driver. The IPC driver is the messaging bridge between the host and DSP and defines the messaging ABI and protocol. It is architecture and platform generic code.
4) DSP Platform Driver. The platform driver is a platform specific driver that abstracts the low level platform DSP hardware into a common generic API that is used by the upper layers. This includes code that will initialise the DSP and boot the firmware.
The right hand side of the diagram shows the mailbox/doorbell mechanism and the DSP.
It's intended that the PCM and IPC drivers will be reused without modification on every platform. The platform differentiation will occur via the topology data and firmware. There is also scope for differentiation via the machine driver and platform driver. i.e. ACPI or Device Tree could be used to specify HW configuration.
Debugging Firmware
Most small DSP devices that run firmware have limited debug facilities compared to debugging programs on a PC. i.e. there are usually no UARTs, network, GUI or even GPIOs/LEDs to use as a debugging aid when things go wrong. DSP development can sometimes require custom hardware debuggers (e.g via JTAG) or proprietary DSP emulators. Sound Open Firmware provides several free methods to develop and debug DSP firmware.
1) DSP boot tracing.
DSP boot problems can be difficult to debug since the DSP hardware wont be in a fully initialised state. Sound Open Firmware provides a simple platform specific method to trace each step of the boot process. The boot trace can be enabled in include/reef/trace.h by setting TRACE to 1
#define TRACE 1
The firmware trace implementations are platform specific as the IO on each platform may be different. The boot trace method used on the Intel platforms writes to a shared DSP register that is always visible to both host and DSP.
/* Platform defined trace code */ #define platform_trace_point(__x) \ shim_write(SHIM_IPCDH, ((shim_read(SHIM_IPCDH) & 0xc0000000) |\ ((__x) & 0x3fffffff)))
The firmware boot trace value codes are :-
/* general trace init codes - only used at boot when main trace is not availble */ #define TRACE_BOOT_START 0x1000 #define TRACE_BOOT_ARCH 0x2000 #define TRACE_BOOT_SYS 0x3000 #define TRACE_BOOT_PLATFORM 0x4000 /* system specific codes */ #define TRACE_BOOT_SYS_WORK (TRACE_BOOT_SYS + 0x100) #define TRACE_BOOT_SYS_CPU_FREQ (TRACE_BOOT_SYS + 0x101) /* platform/device specific codes */ #define TRACE_BOOT_PLATFORM_MBOX (TRACE_BOOT_PLATFORM + 0x100) #define TRACE_BOOT_PLATFORM_SHIM (TRACE_BOOT_PLATFORM + 0x101) #define TRACE_BOOT_PLATFORM_PMC (TRACE_BOOT_PLATFORM + 0x102) #define TRACE_BOOT_PLATFORM_TIMER (TRACE_BOOT_PLATFORM + 0x103) #define TRACE_BOOT_PLATFORM_CLOCK (TRACE_BOOT_PLATFORM + 0x104) #define TRACE_BOOT_PLATFORM_SSP_FREQ (TRACE_BOOT_PLATFORM + 0x105) #define TRACE_BOOT_PLATFORM_IPC (TRACE_BOOT_PLATFORM + 0x106) #define TRACE_BOOT_PLATFORM_DMA (TRACE_BOOT_PLATFORM + 0x107) #define TRACE_BOOT_PLATFORM_SSP (TRACE_BOOT_PLATFORM + 0x108)
These values should be written by the firmware platform boot trace code where the host driver can read then at any firmware boot failure. The boot failure can then be traced in the kernel log.
2) DSP run time tracing.
The run time tracing is more powerful than boot tracing as it includes timestamps and more verbose information. It is enabled in trace.h like the boot trace.
# define TRACE 1
The run time tracing again has a platform specific implementation like the boot tracing. The Intel platforms write their trace data to a circular buffer in the mailbox area that is shared with the host OS. The trace data can then be read from userspace by using the rmbox tool.
rmbox -o dump
This will display a dump of the current trace buffer :-
trace.io: timestamp 0x3d760104 (41245748.00 us) delta 0x0000782e (1230.64 us) comp DPi trace.io: timestamp 0x3d760134 (41245748.00 us) delta 0x00000030 (1.92 us) value 0x000000e0 trace.io: timestamp 0x3d760238 (41245760.00 us) delta 0x00000104 (10.40 us) pipe PWs trace.io: timestamp 0x3d7602c6 (41245764.00 us) delta 0x0000008e (5.68 us) comp VPi trace.io: timestamp 0x3d760910 (41245828.00 us) delta 0x0000064a (64.40 us) comp HPO trace.io: timestamp 0x3d760a0f (41245840.00 us) delta 0x000000ff (10.20 us) pipe PWe trace.io: timestamp 0x3d760b0b (41245848.00 us) delta 0x000000fc (10.08 us) comp HPo trace.io: timestamp 0x3d768338 (41247080.00 us) delta 0x0000782d (1230.60 us) comp DPo trace.io: timestamp 0x3d76836a (41247084.00 us) delta 0x00000032 (2.00 us) value 0x000001e0 trace.io: timestamp 0x3d76846e (41247092.00 us) delta 0x00000104 (10.40 us) pipe PWs trace.io: timestamp 0x3d7684f9 (41247100.00 us) delta 0x0000008b (5.56 us) comp VPo trace.io: timestamp 0x3d768b47 (41247164.00 us) delta 0x0000064e (64.56 us) comp HPI trace.io: timestamp 0x3d768c44 (41247172.00 us) delta 0x000000fd (10.12 us) pipe PWe trace.io: timestamp 0x3d768d41 (41247184.00 us) delta 0x000000fd (10.12 us) comp HPi trace.io: timestamp 0x3d77056f (41248416.00 us) delta 0x0000782e (1230.64 us) comp DPi trace.io: timestamp 0x3d77059f (41248416.00 us) delta 0x00000030 (1.92 us) value 0x000000e0
The trace above shows the trace event, time of the event and delta time to the last event.
3) DSP exception oops.
Sound Open Firmware has a similar crash oops feature to the Linux kernel. It will attempt to dump the DSP registers and stack to a place where the host drivers can read this information upon a DSP firmware crash or exception. The Intel platforms dump the exception data to the mailbox area where it can be read from userspace with the rmbox tool.
rmbox -o dump
The dump data can then be used to check the exception type and addresses (this is DSP ISA specific - please check your DSP documentation). The exception addresses can then used to look up the code region that caused the exception for further debug. Sound Open Firmware builds a disassembly map file as part of the build that can be used to lookup symbols. e.g. if the exception address was 0xff2c1053 then reef-byt.map shows :-
ff2c1050 <clock_get_freq>: ff2c1050: 004136 entry a1, 32 ff2c1053: ffb581 l32r a8, ff2c0f28 <clock_disable+0xc> <<< exception was here. ff2c1056: 902220 addx2 a2, a2, a2 ff2c1059: 0888 l32i.n a8, a8, 0 ff2c105b: a02280 addx4 a2, a2, a8 ff2c105e: 0228 l32i.n a2, a2, 0 ff2c1060: f01d retw.n
This means clock_get_freq() can be looked up in the source where additional debug can be performed.
4) Dumping firmware MMIO space, memory and objects to host userspace.
5) DSP emulator.
Using the Qemu DSP emulator
TODO
FAQ
What license does the firmware and SDK use ?
The firmware is released using a standard BSD license with some parts MIT. The SDK is GPL.
Do I need to open source my firmware code changes ?
No. The firmware BSD and MIT licensed code means you can keep code changes private. Patches are always welcomed if do decide to open source work.
What DSP architectures are supported ?
Sound Open Firmware currently supports the Cadence/Tensilica Xtensa audio DSP architecture and ISA.
What host platforms are supported ?
Sound Open Firmware currently supports the Intel Baytrail and Cherrytrail based platforms. This includes devices like the Minnowboard MAX and the ASUS T100 laptop, but should also include any Baytrail or Cherrytrail based devices that have the audio DSP enabled in the BIOS.
The code has also been designed to easily port to other host platform architectures like ARM, MIPS etc.
How can I get involved ?
Please join the developer mailing where new development features and patches are discussed. http://alsa-project.org/mailman/listinfo/sound-open-firmware
What is the development model ?
Sound Open Firmware has a similar development model to the Linux kernel. Patches are discussed and posted on the mailing list before being merged. The release cadence will likely be every 6 - 8 weeks. There will be a stable release tagged after passing QA then development will continue for the next release.
Who is working on Sound Open Firmware ?
Currently Intel is sponsoring development work on the Minnowboard MAX and other Intel based platforms.
How do I add support for DSP architecture X ?
It's straight forward enough to add support for a new DSP architecture. New architectures usually require support in the GNU tool chain, although other tool chains can be used too. It also helps to have qemu support for the architecture in order to provide an emulator.
The main work in adding the new architecture is duplicating and porting the src/arch directory to your new architecture. The code in the architecture directory mainly deals with architecture abstraction and initialisation of any architecture IP like MMU, IRQs and caches alongside providing optimised version of common C functions (memcpy, memset, etc) for that architecture. Adding a new architecture also usually means adding a new host platform too.
How do I add support for host platform X ?
Adding a new host platform is a lot simpler than adding a new DSP architecture. A new host platform consists of adding a new src/platform/ directory, together with mappings for memory, IRQs, GPIOs and peripheral devices in the DSP memory space. New drivers may also have to be added (e.g. for DMA, I2S) to the drivers directory.
How do I port to other OSes ?
There is nothing stopping the firmware working with non Linux based OSes providing a driver exists or can be written for that OS. The main area for potential optimisation in porting to another OS is aligning the IPC (Inter processor Communication) mechanism to the audio driver flow for that OS. Sound Open Firmware has IPC optimised for the ALSA driver flow, but it's easy enough to reuse this IPC on other OSes or add a completely new IPC for the OS of your choice. The IPC ABI is defined in src/include/uapi/ and the IPC IO logic lives in src/ipc/.
The current upstream IPC uses a memory mapped doorbell and mailbox to pass messages between the host and DSP. Non memory mapped IO (like I2C) can also be supported by adding new doorbell and mailbox driver in your platform code.
What audio components are supported ?
Firmware currently supports mixers, volume, DAIs and Host PCMs in the upstream code base. More components are in progress...
How do I create my own pipelines ?
The current upstream supports creating statically defined pipelines in src/audio/static-pipeline.c. This default pipeline can be changed in this file and in the driver to match any new pipeline topology.
Dynamic pipeline topology will be supported upstream soon. This will allow pipelines to be defined at run time in the firmware and driver by using the alsa topology framework.
Can I add my own media encoder/decoders ?
Yes.
Can I add non audio functions ?
Yes, the instruction sets used by DSPs are also good at non audio processing tasks too. e.g. low power sensor signal processing. Providing your DSP has physical IO ports to connect other non audio devices then it's possible to process data from these devices too.