Modem trace module

To enable the tracing functionality, enable the CONFIG_NRF_MODEM_LIB_TRACE Kconfig in your project configuration. The module is implemented in nrf/lib/nrf_modem_lib/nrf_modem_lib_trace.c and consists of a thread that initializes, deinitializes, and forwards modem traces to a backend. The trace backend can be selected in one of the following ways:

To reduce the amount of trace data sent from the modem, a different trace level can be selected. Complete the following steps to configure the modem trace level at compile time:

  1. Enable the CONFIG_NRF_MODEM_LIB_TRACE_LEVEL_OVERRIDE option in your project configuration.

  2. Enable any one of the following Kconfig options by setting it to y in your project configuration:

The application can use the nrf_modem_lib_trace_level_set() function to set the desired trace level. Passing NRF_MODEM_LIB_TRACE_LEVEL_OFF to the nrf_modem_lib_trace_level_set() function disables trace output.

Note

The modem stores the current trace level on passing the AT+CFUN=0 command. If the trace level stored in the modem is NRF_MODEM_LIB_TRACE_LEVEL_OFF, the application must enable traces in the modem using the nrf_modem_lib_trace_level_set() function or by enabling any of the aforementioned Kconfig options.

During tracing, the integration layer ensures that modem traces are always flushed before the Modem library is re-initialized (including when the modem has crashed). The application can synchronize with the flushing of modem traces by calling the nrf_modem_lib_trace_processing_done_wait() function.

For trace backends that support storing of trace data, the application can be notified using the nrf_modem_lib_trace_callback() function if the trace storage becomes full. The nrf_modem_lib_trace_callback() must be defined in the application if the CONFIG_NRF_MODEM_TRACE_FLASH_NOSPACE_SIGNAL Kconfig option is enabled. In this case, the application is responsible for reading the trace data with the nrf_modem_lib_trace_read() function if required, before clearing the trace backend storage by calling the nrf_modem_lib_trace_clear() function. It is not necessary to turn off modem tracing. However, it is expected that the modem will drop traces when the backend becomes full.

To enable the measurement of the modem trace backend bitrate, enable the CONFIG_NRF_MODEM_LIB_TRACE_BACKEND_BITRATE Kconfig in your project configuration. After enabling this Kconfig option, the application can use the nrf_modem_lib_trace_backend_bitrate_get() function to retrieve the rolling average bitrate of the modem trace backend, measured over the period defined by the CONFIG_NRF_MODEM_LIB_TRACE_BACKEND_BITRATE_PERIOD_MS Kconfig option. To enable logging of the modem trace backend bitrate, enable the CONFIG_NRF_MODEM_LIB_TRACE_BACKEND_BITRATE_LOG Kconfig option. The logging happens at an interval set by the CONFIG_NRF_MODEM_LIB_TRACE_BACKEND_BITRATE_LOG_PERIOD_MS Kconfig option. If the difference in the values of the CONFIG_NRF_MODEM_LIB_TRACE_BACKEND_BITRATE_PERIOD_MS and CONFIG_NRF_MODEM_LIB_TRACE_BACKEND_BITRATE_LOG_PERIOD_MS Kconfig options is very high, you can sometimes observe high variation in measurements due to the short period over which the rolling average is calculated.

To enable logging of the modem trace bitrate, use the CONFIG_NRF_MODEM_LIB_TRACE_BITRATE_LOG Kconfig option.

Modem trace flash backend

The flash backend stores modem traces to the external flash storage on the nRF91 Series DK.

First, set up the external flash for your application. You can then set the following configuration options for the application to decide how to handle when the flash is full:

You can also increase heap and stack sizes when using the modem trace flash backend by setting values for the following configuration options:

The modem trace flash backend has some additional configuration options:

  • CONFIG_FCB - Required for the flash circular buffer used in the backend.

  • CONFIG_NRF_MODEM_LIB_TRACE_BACKEND_FLASH_PARTITION_SIZE - Defines the space to be used for the modem trace partition. In order to improve the modem trace write performance, this partition is erased during system boot. This might lead to a significant increase in the boot time on the nRF9160 DK. The external flash size on the nRF9160 DK is 8 MB (equal to 0x800000 in HEX) and 32 MB on the nRF9161 DK (equal to 0x2000000 in HEX).

It is also recommended to enable high drive mode and high-performance mode in devicetree. High drive is to ensure that the communication with the flash device is reliable at high speed. High-performance mode is a feature in the flash device that allows it to write and erase faster than in low-power mode. See the external flash documentation for more details. The trace backend needs to handle trace data at ~1 Mbps to avoid filling up the buffer in the modem. If the modem buffer is full, the modem drops modem traces until the buffer has space available again.

Sending traces over UART on an nRF91 Series DK

To send modem traces over UART on an nRF91 Series DK, configuration must be added for the UART device in the devicetree and Kconfig. This is done by adding the modem trace UART snippet when building and programming.

Use the Cellular Monitor app for capturing and analyzing modem traces.

TF-M logging must use the same UART as the application. For more details, see shared TF-M logging.

Sending traces over UART on a custom board

To send modem traces over UART on a custom board, configuration must be added for the UART device in the devicetree. This is done by adding the following code snippet to the board devicetree or overlay file, where the pin numbers (0, 1, 14, and 15) must be updated to match your board. The snippet uses UART1. However, any free UART instance can be selected.

&pinctrl {
   uart1_default: uart1_default {
      group1 {
         psels = <NRF_PSEL(UART_TX, 0, 1)>,
            <NRF_PSEL(UART_RTS, 0, 14)>;
      };
      group2 {
         psels = <NRF_PSEL(UART_RX, 0, 0)>,
            <NRF_PSEL(UART_CTS, 0, 15)>;
         bias-pull-up;
      };
   };

   uart1_sleep: uart1_sleep {
      group1 {
         psels = <NRF_PSEL(UART_TX, 0, 1)>,
            <NRF_PSEL(UART_RX, 0, 0)>,
            <NRF_PSEL(UART_RTS, 0, 14)>,
            <NRF_PSEL(UART_CTS, 0, 15)>;
         low-power-enable;
      };
   };
};

&uart1 {
   status = "okay";
   current-speed = <1000000>;
   pinctrl-0 = <&uart1_default>;
   pinctrl-1 = <&uart1_sleep>;
   pinctrl-names = "default", "sleep";
};

/ {
   chosen {
      nordic,modem-trace-uart = &uart1;
   };
};

This is in addition to selecting the CONFIG_NRF_MODEM_LIB_TRACE, CONFIG_NRF_MODEM_LIB_TRACE_BACKEND_UART, CONFIG_UART_ASYNC_API, and CONFIG_SERIAL Kconfig options.

Modem tracing with RTT

Note

Modem tracing with RTT is experimental.

Following are the requirements to perform tracing with RTT:

  • An nRF91 Series DK with SEGGER J-Link on-Board or an external SEGGER J-Link

  • J-Link RTT logger software application

To enable modem traces with RTT, enable the CONFIG_NRF_MODEM_LIB_TRACE_BACKEND_RTT and CONFIG_USE_SEGGER_RTT Kconfig options, with the CONFIG_NRF_MODEM_LIB_TRACE Kconfig option.

The traces can be captured using the J-Link RTT logger software. This produces a RAW binary trace file with a .log extension. The RAW binary trace file can be converted to PCAP with the Open trace file in Wireshark option in the Cellular Monitor app of nRF Connect for Desktop. By default, files with the .log extension are not shown.

Adding custom trace backends

You can add custom trace backends if the existing trace backends are not sufficient. At any time, only one trace backend can be compiled with the application. The value of the CONFIG_NRF_MODEM_LIB_TRACE_BACKEND Kconfig option determines which trace backend is compiled. The Cellular: Modem trace backend sample demonstrates how you can add a custom trace backend to an application.

Complete the following steps to add a custom trace backend:

  1. Place the files that have the custom trace backend implementation in a library or an application you create. For example, the implementation of the UART trace backend (default) can be found in the nrf/lib/nrf_modem_lib/trace_backends/uart/uart.c file.

  2. Add a C file implementing the interface in the nrf/include/modem/trace_backend.h header file.

    /* my_trace_backend.c */

#include <modem/trace_backend.h>

int trace_backend_init(void)
{
     /* initialize transport backend here */
     return 0;
}

int trace_backend_deinit(void)
{
     /* optional deinitialization code here */
     return 0;
}

int trace_backend_write(const void *data, size_t len)
{
     /* forward or store trace data here */
     /* return the number of bytes written or stored, or a negative error code on failure */
     return 0;
}

size_t trace_backend_data_size(void)
{
   /* If trace data is stored when calling `trace_backend_write()`
    * this function returns the size of the stored trace data.
    *
    * If not applicable for the trace backend, set to NULL in the `trace_backend` struct.
    */
}

int trace_backend_read(uint8_t *buf, size_t len)
{
   /* If trace data is stored when calling `trace_backend_write()`
    * this function allows the application to read back the trace data.
    *
    * If not applicable for the trace backend, set to NULL in the `trace_backend` struct.
    */
}

int trace_backend_clear(void)
{
   /* This function allows the backend to clear all stored traces in the backend. For instance
    * this can be erasing a flash partition to prepare for writing new data.
    *
    * If not applicable for the trace backend, set to NULL in the `trace_backend` struct.
    */
}

int trace_backend_suspend(void)
{
   /* This function allows the trace module to suspend the trace backend. When suspended,
    * the backend cannot be used by the trace module until it is resumed by calling
    * `trace_backend_resume()`.
    *
    * If not applicable for the trace backend, set to NULL in the `trace_backend` struct.
    */
}

int trace_backend_resume(void)
{
   /* This function allows the trace module to resume the trace backend after it is suspended.
    *
    * If not applicable for the trace backend, set to NULL in the `trace_backend` struct.
    */
}

struct nrf_modem_lib_trace_backend trace_backend = {
   .init = trace_backend_init,
   .deinit = trace_backend_deinit,
   .write = trace_backend_write,
   .data_size = trace_backend_data_size, /* Set to NULL if not applicable. */
   .read = trace_backend_read, /* Set to NULL if not applicable. */
   .clear = trace_backend_clear, /* Set to NULL if not applicable. */
   .suspend = trace_backend_suspend, /* Set to NULL if not applicable. */
   .resume = trace_backend_resume, /* Set to NULL if not applicable. */
};

  3. Create or modify a Kconfig file to extend the choice CONFIG_NRF_MODEM_LIB_TRACE_BACKEND with another option.

    if NRF_MODEM_LIB_TRACE

# Extends the choice with another backend
choice NRF_MODEM_LIB_TRACE_BACKEND

config NRF_MODEM_LIB_TRACE_BACKEND_MY_TRACE_BACKEND
        bool "My trace backend"
        help
          Optional description of my
          trace backend.

endchoice

endif

  4. Create or modify a CMakeLists.txt file, adding the custom trace backend sources only if the custom trace backend option has been chosen.

    if(CONFIG_NRF_MODEM_LIB_TRACE)

zephyr_library()

# Only add 'custom' backend to compilation when selected.
zephyr_library_sources_ifdef(
  CONFIG_NRF_MODEM_LIB_TRACE_BACKEND_MY_TRACE_BACKEND
  path/to/my_trace_backend.c
)

endif()

  5. Include the Kconfig file and the CMakeLists.txt file to the build.

  6. Add the following Kconfig options to your application’s prj.conf file to use the custom modem trace backend:

    CONFIG_NRF_MODEM_LIB_TRACE=y
CONFIG_NRF_MODEM_LIB_TRACE_BACKEND_MY_TRACE_BACKEND=y

Modem trace shell command

Shell command: modem_trace

You can use the modem trace commands to control the trace functionality in the modem when the CONFIG_NRF_MODEM_LIB_SHELL_TRACE Kconfig option is enabled. A trace backend that can store modem traces is required to send modem traces to the cloud. The modem_trace dump_uart command is a simple demonstration of how trace data can be read out from storage using the nrf_modem_lib_trace_read() function.

Be aware that the default trace level is CONFIG_NRF_MODEM_LIB_TRACE_LEVEL_FULL and starts modem traces during system boot automatically. In case you want to start modem tracing using shell commands instead, set the CONFIG_NRF_MODEM_LIB_TRACE_LEVEL_OFF Kconfig option to y.

Following are some examples of modem tracing:

  • To trace everything (LTE, IP, and GNSS):

    modem_trace start full
<use the modem by triggering functionality in the app or use at-commands>
modem_trace stop

  • To read out the size of stored modem traces:

    modem_trace size

  • To read out stored traces and send over UART:

    modem_trace dump_uart

  • To delete all stored modem traces:

    modem_trace clear