.. _cmake-details: Build System (CMake) ******************** CMake is used to build your application together with the Zephyr kernel. A CMake build is done in two stages. The first stage is called **configuration**. During configuration, the CMakeLists.txt build scripts are executed. After configuration is finished, CMake has an internal model of the Zephyr build, and can generate build scripts that are native to the host platform. CMake supports generating scripts for several build systems, but only Ninja and Make are tested and supported by Zephyr. After configuration, you begin the **build** stage by executing the generated build scripts. These build scripts can recompile the application without involving CMake following most code changes. However, after certain changes, the configuration step must be executed again before building. The build scripts can detect some of these situations and reconfigure automatically, but there are cases when this must be done manually. Zephyr uses CMake's concept of a 'target' to organize the build. A target can be an executable, a library, or a generated file. For application developers, the library target is the most important to understand. All source code that goes into a Zephyr build does so by being included in a library target, even application code. Library targets have source code, that is added through CMakeLists.txt build scripts like this: .. code-block:: cmake target_sources(app PRIVATE src/main.c) In the above :file:`CMakeLists.txt`, an existing library target named ``app`` is configured to include the source file :file:`src/main.c`. The ``PRIVATE`` keyword indicates that we are modifying the internals of how the library is being built. Using the keyword ``PUBLIC`` would modify how other libraries that link with app are built. In this case, using ``PUBLIC`` would cause libraries that link with ``app`` to also include the source file :file:`src/main.c`, behavior that we surely do not want. The ``PUBLIC`` keyword could however be useful when modifying the include paths of a target library. Build and Configuration Phases ============================== The Zephyr build process can be divided into two main phases: a configuration phase (driven by CMake) and a build phase (driven by Make or Ninja). .. _build_configuration_phase: Configuration Phase ------------------- The configuration phase begins when the user invokes *CMake* to generate a build system, specifying a source application directory and a board target. .. figure:: ../../../../../../../zephyr/doc/build/cmake/build-config-phase.svg :align: center :alt: Zephyr's build configuration phase :figclass: align-center :width: 80% CMake begins by processing the :file:`CMakeLists.txt` file in the application directory, which refers to the :file:`CMakeLists.txt` file in the Zephyr top-level directory, which in turn refers to :file:`CMakeLists.txt` files throughout the build tree (directly and indirectly). Its primary output is a set of Makefiles or Ninja files to drive the build process, but the CMake scripts also do some processing of their own, which is explained here. Note that paths beginning with :file:`build/` below refer to the build directory you create when running CMake. Devicetree :file:`*.dts` (*devicetree source*) and :file:`*.dtsi` (*devicetree source include*) files are collected from the target's architecture, SoC, board, and application directories. :file:`*.dtsi` files are included by :file:`*.dts` files via the C preprocessor (often abbreviated *cpp*, which should not be confused with C++). The C preprocessor is also used to merge in any devicetree :file:`*.overlay` files, and to expand macros in :file:`*.dts`, :file:`*.dtsi`, and :file:`*.overlay` files. The preprocessor output is placed in :file:`build/zephyr/zephyr.dts.pre`. The preprocessed devicetree sources are parsed by :zephyr_file:`gen_defines.py ` to generate a :file:`build/zephyr/include/generated/devicetree_generated.h` header with preprocessor macros. Source code should access preprocessor macros generated from devicetree by including the :zephyr_file:`devicetree.h ` header, which includes :file:`devicetree_generated.h`. :file:`gen_defines.py` also writes the final devicetree to :file:`build/zephyr/zephyr.dts` in the build directory. This file's contents may be useful for debugging. If the devicetree compiler ``dtc`` is installed, it is run on :file:`build/zephyr/zephyr.dts` to catch any extra warnings and errors generated by this tool. The output from ``dtc`` is unused otherwise, and this step is skipped if ``dtc`` is not installed. The above is just a brief overview. For more information on devicetree, see :ref:`dt-guide`. Kconfig :file:`Kconfig` files define available configuration options for the target architecture, SoC, board, and application, as well as dependencies between options. Kconfig configurations are stored in *configuration files*. The initial configuration is generated by merging configuration fragments from the board and application (e.g. :file:`prj.conf`). The output from Kconfig is an :file:`autoconf.h` header with preprocessor assignments, and a :file:`.config` file that acts both as a saved configuration and as configuration output (used by CMake). The definitions in :file:`autoconf.h` are automatically exposed at compile time, so there is no need to include this header. Information from devicetree is available to Kconfig, through the functions defined in :zephyr_file:`kconfigfunctions.py `. See :ref:`the Kconfig section of the manual ` for more information. Build Phase ----------- The build phase begins when the user invokes ``make`` or ``ninja``. Its ultimate output is a complete Zephyr application in a format suitable for loading/flashing on the desired target board (:file:`zephyr.elf`, :file:`zephyr.hex`, etc.) The build phase can be broken down, conceptually, into four stages: the pre-build, first-pass binary, final binary, and post-processing. Pre-build +++++++++ Pre-build occurs before any source files are compiled, because during this phase header files used by the source files are generated. Offset generation Access to high-level data structures and members is sometimes required when the definitions of those structures is not immediately accessible (e.g., assembly language). The generation of *offsets.h* (by *gen_offset_header.py*) facilitates this. System call boilerplate The *gen_syscall.py* and *parse_syscalls.py* scripts work together to bind potential system call functions with their implementations. .. figure:: ../../../../../../../zephyr/doc/build/cmake/build-build-phase-1.svg :align: center :alt: Zephyr's build stage I :figclass: align-center :width: 80% Intermediate binaries +++++++++++++++++++++ Compilation proper begins with the first intermediate binary. Source files (C and assembly) are collected from various subsystems (which ones is decided during the configuration phase), and compiled into archives (with reference to header files in the tree, as well as those generated during the configuration phase and the pre-build stage(s)). .. figure:: ../../../../../../../zephyr/doc/build/cmake/build-build-phase-2.svg :align: center :alt: Zephyr's build stage II :figclass: align-center :width: 80% The exact number of intermediate binaries is decided during the configuration phase. If memory protection is enabled, then: Partition grouping The *gen_app_partitions.py* script scans all the generated archives and outputs linker scripts to ensure that application partitions are properly grouped and aligned for the target’s memory protection hardware. Then *cpp* is used to combine linker script fragments from the target’s architecture/SoC, the kernel tree, optionally the partition output if memory protection is enabled, and any other fragments selected during the configuration process, into a *linker.cmd* file. The compiled archives are then linked with *ld* as specified in the *linker.cmd*. Unfixed size binary The unfixed size intermediate binary is produced when :ref:`usermode_api` is enabled or :ref:`devicetree` is in use. It produces a binary where sizes are not fixed and thus it may be used by post-process steps that will impact the size of the final binary. .. figure:: ../../../../../../../zephyr/doc/build/cmake/build-build-phase-3.svg :align: center :alt: Zephyr's build stage III :figclass: align-center :width: 80% Fixed size binary The fixed size intermediate binary is produced when :ref:`usermode_api` is enabled or when generated IRQ tables are used, :kconfig:option:`CONFIG_GEN_ISR_TABLES` It produces a binary where sizes are fixed and thus the size must not change between the intermediate binary and the final binary. .. figure:: ../../../../../../../zephyr/doc/build/cmake/build-build-phase-4.svg :align: center :alt: Zephyr's build stage IV :figclass: align-center :width: 80% Intermediate binaries post-processing +++++++++++++++++++++++++++++++++++++ The binaries from the previous stage are incomplete, with empty and/or placeholder sections that must be filled in by, essentially, reflection. To complete the build procedure the following scripts are executed on the intermediate binaries to produce the missing pieces needed for the final binary. When :ref:`usermode_api` is enabled: Partition alignment The *gen_app_partitions.py* script scans the unfixed size binary and generates an app shared memory aligned linker script snippet where the partitions are sorted in descending order. .. figure:: ../../../../../../../zephyr/doc/build/cmake/build-postprocess-1.svg :align: center :alt: Zephyr's intermediate binary post-process I :figclass: align-center :width: 80% When :ref:`devicetree` is used: Device dependencies The *gen_device_deps.py* script scans the unfixed size binary to determine relationships between devices that were recorded from devicetree data, and replaces the encoded relationships with values that are optimized to locate the devices actually present in the application. .. figure:: ../../../../../../../zephyr/doc/build/cmake/build-postprocess-2.svg :align: center :alt: Zephyr's intermediate binary post-process II :figclass: align-center :width: 80% When :kconfig:option:`CONFIG_GEN_ISR_TABLES` is enabled: The *gen_isr_tables.py* script scans the fixed size binary and creates an isr_tables.c source file with a hardware vector table and/or software IRQ table. .. figure:: ../../../../../../../zephyr/doc/build/cmake/build-postprocess-3.svg :align: center :alt: Zephyr's intermediate binary post-process III :figclass: align-center :width: 80% When :ref:`usermode_api` is enabled: Kernel object hashing The *gen_kobject_list.py* scans the *ELF DWARF* debug data to find the address of the all kernel objects. This list is passed to *gperf*, which generates a perfect hash function and table of those addresses, then that output is optimized by *process_gperf.py*, using known properties of our special case. .. figure:: ../../../../../../../zephyr/doc/build/cmake/build-postprocess-4.svg :align: center :alt: Zephyr's intermediate binary post-process IV :figclass: align-center :width: 80% When no intermediate binary post-processing is required then the first intermediate binary will be directly used as the final binary. Final binary ++++++++++++ The binary from the previous stage is incomplete, with empty and/or placeholder sections that must be filled in by, essentially, reflection. The link from the previous stage is repeated, this time with the missing pieces populated. .. figure:: ../../../../../../../zephyr/doc/build/cmake/build-build-phase-5.svg :align: center :alt: Zephyr's build final stage :figclass: align-center :width: 80% Post processing +++++++++++++++ Finally, if necessary, the completed kernel is converted from *ELF* to the format expected by the loader and/or flash tool required by the target. This is accomplished in a straightforward manner with *objdump*. .. figure:: ../../../../../../../zephyr/doc/build/cmake/build-build-phase-6.svg :align: center :alt: Zephyr's build final stage post-process :figclass: align-center :width: 80% .. _build_system_scripts: Supporting Scripts and Tools ============================ The following is a detailed description of the scripts used during the build process. .. _gen_syscalls.py: :zephyr_file:`scripts/build/gen_syscalls.py` -------------------------------------------- .. include:: ../../../../../../../zephyr/scripts/build/gen_syscalls.py :start-after: """ :end-before: """ .. _gen_device_deps.py: :zephyr_file:`scripts/build/gen_device_deps.py` ----------------------------------------------- .. include:: ../../../../../../../zephyr/scripts/build/gen_device_deps.py :start-after: """ :end-before: """ .. _gen_kobject_list.py: :zephyr_file:`scripts/build/gen_kobject_list.py` ------------------------------------------------ .. include:: ../../../../../../../zephyr/scripts/build/gen_kobject_list.py :start-after: """ :end-before: """ .. _gen_offset_header.py: :zephyr_file:`scripts/build/gen_offset_header.py` ------------------------------------------------- .. include:: ../../../../../../../zephyr/scripts/build/gen_offset_header.py :start-after: """ :end-before: """ .. _parse_syscalls.py: :zephyr_file:`scripts/build/parse_syscalls.py` ---------------------------------------------- .. include:: ../../../../../../../zephyr/scripts/build/parse_syscalls.py :start-after: """ :end-before: """ .. _gen_idt.py: :zephyr_file:`arch/x86/gen_idt.py` ---------------------------------- .. include:: ../../../../../../../zephyr/arch/x86/gen_idt.py :start-after: """ :end-before: """ .. _gen_gdt.py: :zephyr_file:`arch/x86/gen_gdt.py` ---------------------------------- .. include:: ../../../../../../../zephyr/arch/x86/gen_gdt.py :start-after: """ :end-before: """ .. _gen_relocate_app.py: :zephyr_file:`scripts/build/gen_relocate_app.py` ------------------------------------------------ .. include:: ../../../../../../../zephyr/scripts/build/gen_relocate_app.py :start-after: """ :end-before: """ .. _process_gperf.py: :zephyr_file:`scripts/build/process_gperf.py` --------------------------------------------- .. include:: ../../../../../../../zephyr/scripts/build/process_gperf.py :start-after: """ :end-before: """ :zephyr_file:`scripts/build/gen_app_partitions.py` -------------------------------------------------- .. include:: ../../../../../../../zephyr/scripts/build/gen_app_partitions.py :start-after: """ :end-before: """ .. _check_init_priorities.py: :zephyr_file:`scripts/build/check_init_priorities.py` ----------------------------------------------------- .. include:: ../../../../../../../zephyr/scripts/build/check_init_priorities.py :start-after: """ :end-before: """