Sensing Subsystem

Overview

Sensing Subsystem is a high level sensor framework inside the OS user space service layer. It is a framework focused on sensor fusion, client arbitration, sampling, timing, scheduling and sensor based power management.

Key concepts in Sensing Subsystem include physical sensor and virtual sensor objects, and a scheduling framework over sensor object relationships. Physical sensors do not depend on any other sensor objects for input, and will directly interact with existing zephyr sensor device drivers. Virtual sensors rely on other sensor objects (physical or virtual) as report inputs.

The sensing subsystem relies on Zephyr sensor device APIs (existing version or update in future) to leverage Zephyr’s large library of sensor device drivers (100+).

Use of the sensing subsystem is optional. Applications that only need to access simple sensors devices can use the Zephyr Sensors API directly.

Since the sensing subsystem is separated from device driver layer or kernel space and could support various customizations and sensor algorithms in user space with virtual sensor concepts. The existing sensor device driver can focus on low layer device side works, can keep simple as much as possible, just provide device HW abstraction and operations etc. This is very good for system stability.

The sensing subsystem is decoupled with any sensor expose/transfer protocols, the target is to support various up-layer frameworks and Applications with different sensor expose/transfer protocols, such as CHRE, HID sensors Applications, MQTT sensor Applications according different products requirements. Or even support multiple Applications with different up-layer sensor protocols at the same time with it’s multiple clients support design.

Sensing subsystem can help build a unified Zephyr sensing architecture for cross host OSes support and as well as IoT sensor solutions.

The diagram below illustrates how the Sensing Subsystem integrates with up-layer frameworks.

Unified Zephyr sensing architecture.

Configurability

  • Reusable and configurable standalone subsystem.

  • Based on Zephyr existing low-level Sensor API (reuse 100+ existing sensor device drivers)

  • Provide Zephyr high-level Sensing Subsystem API for Applications.

  • Separate option CHRE Sensor PAL Implementation module to support CHRE.

  • Decoupled with any host link protocols, it’s Zephyr Application’s role to handle different protocols (MQTT, HID or Private, all configurable)

Main Features

  • Scope

    • Focus on framework for sensor fusion, multiple clients, arbitration, data sampling, timing management and scheduling.

  • Sensor Abstraction

    • Physical sensor: interacts with Zephyr sensor device drivers, focus on data collecting.

    • Virtual sensor: relies on other sensor(s), physical or virtual, focus on data fusion.

  • Data Driven Model

    • Polling mode: periodical sampling rate

    • Interrupt mode: data ready, threshold interrupt etc.

  • Scheduling

    • single thread main loop for all sensor objects sampling and process.

  • Buffer Mode for Batching

  • Configurable Via Device Tree

Below diagram shows the API position and scope:

Sensing subsystem API organization.

Sensing Subsystem API is for Applications. Sensing Sensor API is for development sensors.

Major Flows

  • Sensor Configuration Flow

Sensor Configuration Flow (App set report interval to hinge angel sensor example).

  • Sensor Data Flow

Sensor Data Flow (App receive hinge angel data through data event callback example).

Sensor Types And Instance

The Sensing Subsystem supports multiple instances of the same sensor type, there’re two methods for Applications to identify and open an unique sensor instance:

  • Enumerate all sensor instances

    sensing_get_sensors() returns all current board configuration supported sensor instances’ information in a sensing_sensor_info pointer array .

    Then Applications can use sensing_open_sensor() to open specific sensor instance for future accessing, configuration and receive sensor data etc.

    This method is suitable for supporting some up-layer frameworks like CHRE, HID which need to dynamically enumerate the underlying platform’s sensor instances.

  • Open the sensor instance by devicetree node directly

    Applications can use sensing_open_sensor_by_dt() to open a sensor instance directly with sensor devicetree node identifier.

    For example:

sensing_open_sensor_by_dt(DEVICE_DT_GET(DT_NODELABEL(base_accel)), cb_list, handle);
sensing_open_sensor_by_dt(DEVICE_DT_GET(DT_CHOSEN(zephyr_sensing_base_accel)), cb_list, handle);

This method is useful and easy use for some simple Application which just want to access specific sensor(s).

Sensor type follows the HID standard sensor types definition.

See include/zephyr/sensing/sensing_sensor_types.h

Sensor Instance Handler

Clients using a sensing_sensor_handle_t type handler to handle a opened sensor instance, and all subsequent operations on this sensor instance need use this handler, such as set configurations, read sensor sample data, etc.

For a sensor instance, could have two kinds of clients: Application clients and Sensor clients.

Application clients can use sensing_open_sensor() to open a sensor instance and get it’s handler.

For Sensor clients, there is no open API for opening a reporter, because the client-report relationship is built at the sensor’s registration stage with devicetree.

The Sensing Subsystem will auto open and create handlers for client sensor to it’s reporter sensors. Sensor clients can get it’s reporters’ handlers via sensing_sensor_get_reporters().

Sensor Reporting Topology.

Note

Sensors inside the Sensing Subsystem, the reporting relationship between them are all auto generated by Sensing Subsystem according devicetree definitions, handlers between client sensor and reporter sensors are auto created. Application(s) need to call sensing_open_sensor() to explicitly open the sensor instance.

Sensor Sample Value

Device Tree Configuration

Sensing subsystem using device tree to configuration all sensor instances and their properties, reporting relationships.

See the example samples/subsys/sensing/simple/boards/native_sim.overlay

API Reference

group sensing_sensor_types

Sensor Types Definition.

Sensor types definition followed HID standard. https://usb.org/sites/default/files/hutrr39b_0.pdf

TODO: will add more types

Defines

SENSING_SENSOR_TYPE_LIGHT_AMBIENTLIGHT

sensor category light

SENSING_SENSOR_TYPE_MOTION_ACCELEROMETER_3D

sensor category motion

SENSING_SENSOR_TYPE_MOTION_GYROMETER_3D
SENSING_SENSOR_TYPE_MOTION_MOTION_DETECTOR
SENSING_SENSOR_TYPE_OTHER_CUSTOM

sensor category other

SENSING_SENSOR_TYPE_MOTION_UNCALIB_ACCELEROMETER_3D
SENSING_SENSOR_TYPE_MOTION_HINGE_ANGLE
SENSING_SENSOR_TYPE_ALL
group sensing_datatypes

Data Types.

struct sensing_sensor_value_header
#include <sensing_datatypes.h>

sensor value header

Each sensor value data structure should have this header

Here use ‘base_timestamp’ (uint64_t) and ‘timestamp_delta’ (uint32_t) to save memory usage in batching mode.

The ‘base_timestamp’ is for readings[0], the ‘timestamp_delta’ is relation to the previous ‘readings’. So, timestamp of readings[0] is header.base_timestamp + readings[0].timestamp_delta. timestamp of readings[1] is timestamp of readings[0] + readings[1].timestamp_delta.

Since timestamp unit is micro seconds, the max ‘timestamp_delta’ (uint32_t) is 4295 seconds.

If a sensor has batched data where two consecutive readings differ by more than 4295 seconds, the sensor subsystem core will split them across multiple instances of the readings structure, and send multiple events.

This concept is borrowed from CHRE: https://cs.android.com/android/platform/superproject/+/master:\ system/chre/chre_api/include/chre_api/chre/sensor_types.h

Public Members

uint64_t base_timestamp

base timestamp of this data readings, unit is micro seconds

uint16_t reading_count

count of this data readings

struct sensing_sensor_value_3d_q31
#include <sensing_datatypes.h>

Sensor value data structure types based on common data types.

Suitable for common sensors, such as IMU, Light sensors and orientation sensors.

Sensor value data structure for 3-axis sensors. struct sensing_sensor_value_3d_q31 can be used by 3D IMU sensors like: SENSING_SENSOR_TYPE_MOTION_ACCELEROMETER_3D, SENSING_SENSOR_TYPE_MOTION_UNCALIB_ACCELEROMETER_3D, SENSING_SENSOR_TYPE_MOTION_GYROMETER_3D, q31 version

struct sensing_sensor_value_uint32
#include <sensing_datatypes.h>

Sensor value data structure for single 1-axis value.

struct sensing_sensor_value_uint32 can be used by SENSING_SENSOR_TYPE_LIGHT_AMBIENTLIGHT sensor uint32_t version

struct sensing_sensor_value_q31
#include <sensing_datatypes.h>

Sensor value data structure for single 1-axis value.

struct sensing_sensor_value_q31 can be used by SENSING_SENSOR_TYPE_MOTION_HINGE_ANGLE sensor q31 version

Related code samples

Sensing subsystem

Get high-level sensor data in defined intervals.

group sensing_api

Sensing Subsystem API.

Defines

SENSING_SENSOR_VERSION(_major, _minor, _hotfix, _build)
SENSING_SENSOR_FLAG_REPORT_ON_EVENT

Sensor flag indicating if this sensor is on event reporting data.

Reporting sensor data when the sensor event occurs, such as a motion detect sensor reporting a motion or motionless detected event.

SENSING_SENSOR_FLAG_REPORT_ON_CHANGE

Sensor flag indicating if this sensor is on change reporting data.

Reporting sensor data when the sensor data changes.

Exclusive with SENSING_SENSOR_FLAG_REPORT_ON_EVENT

SENSING_SENSITIVITY_INDEX_ALL

SENSING_SENSITIVITY_INDEX_ALL indicating sensitivity of each data field should be set.

Typedefs

typedef void *sensing_sensor_handle_t

Define Sensing subsystem sensor handle.

typedef void (*sensing_data_event_t)(sensing_sensor_handle_t handle, const void *buf, void *context)

Sensor data event receive callback.

Param handle:

The sensor instance handle.

Param buf:

The data buffer with sensor data.

Param context:

User provided context pointer.

Enums

enum sensing_sensor_state

Sensing subsystem sensor state.

Values:

enumerator SENSING_SENSOR_STATE_READY = 0
enumerator SENSING_SENSOR_STATE_OFFLINE = 1
enum sensing_sensor_attribute

Sensing subsystem sensor config attribute.

Values:

enumerator SENSING_SENSOR_ATTRIBUTE_INTERVAL = 0
enumerator SENSING_SENSOR_ATTRIBUTE_SENSITIVITY = 1
enumerator SENSING_SENSOR_ATTRIBUTE_LATENCY = 2
enumerator SENSING_SENSOR_ATTRIBUTE_MAX

Functions

int sensing_get_sensors(int *num_sensors, const struct sensing_sensor_info **info)

Get all supported sensor instances’ information.

This API just returns read only information of sensor instances, pointer info will directly point to internal buffer, no need for caller to allocate buffer, no side effect to sensor instances.

Parameters:

  • num_sensors – Get number of sensor instances.

  • info – For receiving sensor instances’ information array pointer.

Returns:

0 on success or negative error value on failure.

int sensing_open_sensor(const struct sensing_sensor_info *info, struct sensing_callback_list *cb_list, sensing_sensor_handle_t *handle)

Open sensor instance by sensing sensor info.

Application clients use it to open a sensor instance and get its handle. Support multiple Application clients for open same sensor instance, in this case, the returned handle will different for different clients. meanwhile, also register sensing callback list

Parameters:

  • info – The sensor info got from sensing_get_sensors

  • cb_list – callback list to be registered to sensing, must have a static lifetime.

  • handle – The opened instance handle, if failed will be set to NULL.

Returns:

0 on success or negative error value on failure.

int sensing_open_sensor_by_dt(const struct device *dev, struct sensing_callback_list *cb_list, sensing_sensor_handle_t *handle)

Open sensor instance by device.

Application clients use it to open a sensor instance and get its handle. Support multiple Application clients for open same sensor instance, in this case, the returned handle will different for different clients. meanwhile, also register sensing callback list.

Parameters:

  • dev – pointer device get from device tree.

  • cb_list – callback list to be registered to sensing, must have a static lifetime.

  • handle – The opened instance handle, if failed will be set to NULL.

Returns:

0 on success or negative error value on failure.

int sensing_close_sensor(sensing_sensor_handle_t *handle)

Close sensor instance.

Parameters:

  • handle – The sensor instance handle need to close.

Returns:

0 on success or negative error value on failure.

int sensing_set_config(sensing_sensor_handle_t handle, struct sensing_sensor_config *configs, int count)

Set current config items to Sensing subsystem.

Parameters:

  • handle – The sensor instance handle.

  • configs – The configs to be set according to config attribute.

  • count – count of configs.

Returns:

0 on success or negative error value on failure, not support etc.

int sensing_get_config(sensing_sensor_handle_t handle, struct sensing_sensor_config *configs, int count)

Get current config items from Sensing subsystem.

Parameters:

  • handle – The sensor instance handle.

  • configs – The configs to be get according to config attribute.

  • count – count of configs.

Returns:

0 on success or negative error value on failure, not support etc.

const struct sensing_sensor_info *sensing_get_sensor_info(sensing_sensor_handle_t handle)

Get sensor information from sensor instance handle.

Parameters:

  • handle – The sensor instance handle.

Returns:

a const pointer to sensing_sensor_info on success or NULL on failure.

struct sensing_sensor_version
#include <sensing.h>

Sensor Version.

struct sensing_sensor_info
#include <sensing.h>

Sensor basic constant information.

Public Members

const char *name

Name of the sensor instance.

const char *friendly_name

Friendly name of the sensor instance.

const char *vendor

Vendor name of the sensor instance.

const char *model

Model name of the sensor instance.

const int32_t type

Sensor type.

const uint32_t minimal_interval

Minimal report interval in micro seconds.

struct sensing_callback_list
#include <sensing.h>

Sensing subsystem event callback list.

struct sensing_sensor_config
#include <sensing.h>

Sensing subsystem sensor configure, including interval, sensitivity, latency.

Public Members

int8_t data_field

SENSING_SENSITIVITY_INDEX_ALL

group sensing_sensor

Sensing Sensor API.

Defines

SENSING_SENSOR_INFO_NAME(node, idx)
SENSING_SENSOR_INFO_DEFINE(node, idx)
SENSING_CONNECTIONS_NAME(node)
SENSING_SENSOR_SOURCE_NAME(idx, node)
SENSING_SENSOR_SOURCE_EXTERN(idx, node)
SENSING_CONNECTION_INITIALIZER(source_name, cb_list_ptr)
SENSING_CONNECTION_DEFINE(idx, node, cb_list_ptr)
SENSING_CONNECTIONS_DEFINE(node, num, cb_list_ptr)
SENSING_SUBMIT_CFG_NAME(node, idx)
SENSING_SENSOR_IODEV_NAME(node, idx)
SENSING_SENSOR_IODEV_DEFINE(node, idx)
SENSING_SENSOR_NAME(node, idx)
SENSING_SENSOR_DEFINE(node, prop, idx, reg_ptr, cb_list_ptr)
SENSING_SENSORS_DEFINE(node, reg_ptr, cb_list_ptr)
SENSING_SENSORS_DT_DEFINE(node, reg_ptr, cb_list_ptr, init_fn, pm_device, data_ptr, cfg_ptr, level, prio, api_ptr, ...)

Like SENSOR_DEVICE_DT_DEFINE() with sensing specifics.

Defines a sensor which implements the sensor API. May define an element in the sensing sensor iterable section used to enumerate all sensing sensors.

Parameters:

  • node – The devicetree node identifier.

  • reg_ptr – Pointer to the device’s sensing_sensor_register_info.

  • cb_list_ptr – Pointer to devices callback list.

  • init_fn – Name of the init function of the driver.

  • pm_device – PM device resources reference (NULL if device does not use PM).

  • data_ptr – Pointer to the device’s private data.

  • cfg_ptr – The address to the structure containing the configuration information for this instance of the driver.

  • level – The initialization level. See SYS_INIT() for details.

  • prio – Priority within the selected initialization level. See SYS_INIT() for details.

  • api_ptr – Provides an initial pointer to the API function struct used by the driver. Can be NULL.

SENSING_SENSORS_DT_INST_DEFINE(inst, ...)

Like SENSING_SENSORS_DT_DEFINE() for an instance of a DT_DRV_COMPAT compatible.

Parameters:

Enums

enum [anonymous]

Values:

enumerator EVENT_CONFIG_READY
enum [anonymous]

Values:

enumerator SENSOR_LATER_CFG_BIT

Functions

int sensing_sensor_get_reporters(const struct device *dev, int type, sensing_sensor_handle_t *reporter_handles, int max_handles)

Get reporter handles of a given sensor instance by sensor type.

Parameters:

  • dev – The sensor instance device structure.

  • type – The given type, SENSING_SENSOR_TYPE_ALL to get reporters with all types.

  • max_handles – The max count of the reporter_handles array input. Can get real count number via sensing_sensor_get_reporters_count

  • reporter_handles – Input handles array for receiving found reporter sensor instances

Returns:

number of reporters found, 0 returned if not found.

int sensing_sensor_get_reporters_count(const struct device *dev, int type)

Get reporters count of a given sensor instance by sensor type.

Parameters:
Returns:

Count of reporters by type, 0 returned if no reporters by type.

int sensing_sensor_get_state(const struct device *dev, enum sensing_sensor_state *state)

Get this sensor’s state.

Parameters:

  • dev – The sensor instance device structure.

  • state – Returned sensor state value

Returns:

0 on success or negative error value on failure.

struct sensing_sensor_register_info
#include <sensing_sensor.h>

Sensor registration information.

Public Members

uint16_t flags

Sensor flags.

uint16_t sample_size

Sample size in bytes for a single sample of the registered sensor.

sensing runtime need this information for internal buffer allocation.

uint8_t sensitivity_count

The number of sensor sensitivities.

struct sensing_sensor_version version

Sensor version.

Version can be used to identify different versions of sensor implementation.

struct sensing_connection
#include <sensing_sensor.h>

Connection between a source and sink of sensor data.

struct sensing_sensor
#include <sensing_sensor.h>

Internal sensor instance data structure.

Each sensor instance will have its unique data structure for storing all it’s related information.

Sensor management will enumerate all these instance data structures, build report relationship model base on them, etc.

struct sensing_submit_config
#include <sensing_sensor.h>