Persistent Storage Manager

Introduction

A typical Bluetooth low energy (BLE) example is when a bonded peer configuration needs to be cached to store the bond identification, GATT Server configuration, and/or GATT Server attribute handles. This is just one example of several reasons why persistent data storage is needed.

The Persistent Storage Manager has a generic API (Persistent Storage Interface) used by SDK modules for common storage access. A generic API allows for SDK modules to be reused directly on other platforms. All modules using the Persistent Storage Manager are decoupled from accessing the nRF51822 flash directly. This allows for improved re-usability of SDK modules. For example, the Bond Manager in s110_serialization can be used on processors other than nRF51822. This can be achieved by implementing a storage module according to the Persistent Storage Manager API for the targeted platform.

To summarize, the application interface of this module is as defined in the header and should not be altered to have zero impact on other SDK modules. However, the implementation of the interface is expected to vary based on storage solution identified for the system. Hence, pstorage.h defines the application interface, while pstorage_platform.h is use case and system specific file that can contain defines and routines needed for a specific implementation of the identified interface. The source file, pstorage.c shall implement the needed APIs. SDK examples include implementation of the interface using the SoftDevice APIs for flash access on the chip.

Multiple SDK modules and application itself may be require storing data, each module may have its own data size requirements. Therefore, this module allows registering with it with various block size and block count, each with a separate event handler to be notified of result of flash access.

Figure 1: Multiple blocks requested storage blocks.

Application Interface Overview

Any SDK or application component that requires storing its data registers with the module using the pstorage_register, requesting at the time of registry number of blocks of storage it needs and how many such blocks. The interface is designed to be asynchronous and application is also expected to register a callback to know the result of a storage access operation. Identified storage access operations include load, store and clear. Once application is successfully registered, application is assigned a handle for all future operations needed for accessing these storage blocks. pstorage_handle_t abstracts this handle.

Warning

Before accessing any of the APIs of the module, the module shall be initialized using the Initialization. This initialization should be performed once.

Note

For implementation of interface included in the example, SoftDevice should be enabled and scheduler (if used) should be initialized prior to initializing this module.

Application does not have to remember a handle or an identifier for each of the blocks, instead it just needs to remember one identifier for all the blocks. When, reference to a specific block for flash access is required, pstorage_block_identifier_get API shall be used to get a block specific handle. Base handle and block number counting from 0 are provided as input to the API.

As mentioned earlier, an asynchronous interface is defined for storage access as storing data or clearing data can take time. However no copy of data to be stored is made by the implementation included in the SDK. Therefore, data source provided for store operation using the Store Data API, expects resident memory, meaning that the memory pointed to by data source for this API should not be reused or freed unless client is notified of completion of store operation using the asynchronous notification callback registered with the module by the client.

Initialization

The storage module must be initialized before using any other API of the module.

uint32_t retval;

retval = pstorage_init();

if(retval == NRF_SUCCESS)
{
    // Module initialization successful.
}
else
{
   // Initialization failed, take corrective action.
}

Registration

A module that requires storage must register with the storage module in order to allocate storage blocks for data. The application must register the asynchronous event notification handler, number of blocks, and block size which should be in range of PSTORAGE_MIN_BLOCK_SIZE and PSTORAGE_MAX_BLOCK_SIZE. A reference handle is given to the application once registration is successful and is remembered by the application to reference the storage blocks.

pstorage_handle_t       handle;
pstorage_module_param_t param;
uint32_t                retval;
      
param.block_size  = 100;
param.block_count = 10;
param.cb          = example_cb_handler;
    
retval = pstorage_register(&param, &handle);
if (retval == NRF_SUCCESS)
{
    // Registration successful.
}
else
{
    // Failed to register, take corrective action.
}

Note

The application is provided here with a single handle for all blocks and dedicated handle for each block. This saves the application from having to remember too many handles. The handle provided to the application is referred to as the base handle for the blocks allocated. The application shall use pstorage_block_identifier_get to acquire a specific block reference.

Get Block Identifier

This API provides a specific block reference to the application in allocated blocks. Allocated blocks are identified by the base block identifier provided at the time of registration. Block offset are indexed starting from zero to (number of blocks allocated - 1).

This API shall be called before a load or store operation to a specific block.

pstorage_handle_t base_handle;
pstorage_handle_t block_handle;
uint32_t          retval;

.
.
.
// Registration successfully completed, base_handle is identifier for allocated blocks.
.
.
.

// Request to get identifier for 3rd block. 
retval = pstorage_block_identifier_get(&base_handle, 2, &block_handle);

if (retval == NRF_SUCCESS)
{
    // Get Block Identifier successful.
}
else
{
    // Failed to get block id, take corrective action.
}

Load Data

This API is used to read data from a storage block. It is permitted to read a part of the block using the offset field. The application should ensure the destination has enough memory to copy data from the storage block to the destination pointer provided in the API.

Note

Block size and offset in load and store should be a multiple of word size (4 bytes). Get Block Identifier API shall be used before this API to get block specific identifier.

pstorage_handle_t block_handle;
uint8_t           dest_data[4];
uint32_t          retval;    

// Request to read 4 bytes from block at an offset of 12 bytes.
retval = pstorage_load(dest_data, &block_handle, 4, 12);

if (retval == NRF_SUCCESS)
{
    // Load successful. Consume data.
}
else
{
    // Failed to load, take corrective action.
}

Note

The SDK implementation requires the offset and size to be a multiple of word size (4 bytes) and will not generate a success or failure event. API return value determines determines the result of this operation.

Store Data

This API is used to write data to a storage block. It is permitted to write only a part of the block using the offset field. Application cannot free or reuse the memory that is the source of data until this operation is complete. The event notified using registered callback will indicate when this operation is complete. Event result indicates whether the operation was successful or not.

Note

Block size and offset in load and store should be a multiple of word size (4 bytes). Get Block Identifier API shall be used before this API to get a specific block identifier.

pstorage_handle_t block_handle;
uint8_t           source_data[4];
uint32_t          retval;    

// Request to write 8 bytes to block at an offset of 20 bytes.
retval = pstorage_store(&block_handle, source_data, 8, 20);

if (retval == NRF_SUCCESS)
{
    // Store successfully requested. Wait for operation result.
}
else
{
    // Failed to request store, take corrective action.
}

.
.
.

// Event Notification Handler.
static void example_cb_handler(pstorage_handle_t  * handle,
                               uint8_t              op_code,
                               uint32_t             result,
                               uint8_t            * p_data,
                               uint32_t             data_len)
{
    switch(op_code)
    {
       .
       .
       .
       case PSTORAGE_LOAD_OP_CODE:
           if (result == NRF_SUCCESS)
           {
               // Store operation successful.
           }
           else
           {
               // Store operation failed.
           }
           // Source memory can now be reused or freed.
           break;
       .
       .
       .
    }
}

Note

Flash memory is unreliable when writing to a block already containing data, a clear operation is needed. The SDK implementation does not clear the blocks before writing to them. The application, not the storage module, must perform a clear operation before new data is written to the block.

Update Data

This API is used to update data in storage blocks. Application cannot free or reuse the memory that is the source of data until this operation is complete. The event notified using registered callback will indicate when this operation is complete. Event result indicates whether the operation was successful or not.

pstorage_handle_t base_handle;
uint8_t           source_data[16];
uint32_t          retval;    

// Request update of one blocks. Block size is 16 bytes.
retval = pstorage_update(&base_handle, source_data, 16, 0);

if (retval == NRF_SUCCESS)
{
    // Update successfully requested. Wait for operation result.
}
else
{
    // Failed to request update, take corrective action.
}

.
.
.

// Event Notification Handler.
static void example_cb_handler(pstorage_handle_t  * handle,
                               uint8_t              op_code,
                               uint32_t             result,
                               uint8_t            * p_data,
                               uint32_t             data_len)
{
    switch(op_code)
    {
       .
       .
       .
       case PSTORAGE_UPDATE_OP_CODE:
           if (result == NRF_SUCCESS)
           {
               // Update operation successful.
           }
           else
           {
               // Update operation failed.
           }
           break;
       .
       .
       .
    }
}

Clear Data

This API is used to clear data in storage blocks. The event notified using registered callback will indicate when this operation is complete. Event result indicates whether the operation was successful or not.

The size requested to be erased has to be within or equal the size of a block. If a size less than block size is given, the API will clear the requested number of bytes beginning at the first memory location in the block. It is not possible to request a size which spans to the next block in the module.

pstorage_handle_t base_handle;
uint32_t          retval;    

// Request clearing of all blocks in the module. 32 blocks each with 16 bytes in size.
retval = pstorage_clear(&base_handle, 32 * 16);

// Request clearing of one block where block size is 16 bytes.
retval = pstorage_clear(&base_handle, 16);

if (retval == NRF_SUCCESS)
{
    // Clear successfully requested. Wait for operation result.
}
else
{
    // Failed to request clear, take corrective action.
}

.
.
.

// Event Notification Handler.
static void example_cb_handler(pstorage_handle_t  * handle,
                               uint8_t              op_code,
                               uint32_t             result,
                               uint8_t            * p_data,
                               uint32_t             data_len)
{
    switch(op_code)
    {
       .
       .
       .
       case PSTORAGE_CLEAR_OP_CODE:
           if (result == NRF_SUCCESS)
           {
               // Clear operation successful.
           }
           else
           {
               // Clear operation failed.
           }
           break;
       .
       .
       .
    }
}

Get Status

The Persistent Storage Manager allows application to get a count of how many storage access operations are pending with it using the pstorage_access_status_get API. This is particularly useful when you want to enter power off mode or want to enter a radio intense operation, but before doing so, want to ensure that the storage operations have been completed.

uint32_t    retval;
uint32_t    count;   

// Request clearing of blocks
retval = pstorage_access_status_get(&count);

if (count == 0)
{
    // No pending operations, safe to power off or enter radio intense operations.
}
else
{
    // Storage access pending, wait!
}

Raw Mode

Certain use cases require complete control of the entire flash region and do not have typical storage requirements. The storage module then provisions for one application to be registered with it in the 'raw' mode. In raw mode, the application is responsible for conceptualizing the flash region as blocks and their management. Dedicated APIs, register, store, and clear, are provided to distinguish raw mode from the normal mode. Raw mode APIs have a similar signature to the normal mode.

As this is not a typical use case, and is included for a targeted few applications like the DFU, the raw mode by default is disabled and is included only if PSTORAGE_RAW_MODE_ENABLE is defined in the pstorage_platform.h header.

Specifics and limitations of the SDK implementation

Additional requirements and a few limitations exist when implementing the example included. Some have already been mentioned but the following is a summarized list with more detailed information.

General:

• The SoftDevice and scheduler must be initialized first before initializing the Persistent Storage Module. Modules that use this storage module shall then be initialized after the storage module.
• Block size and offset in load, store and update shall be a multiple of word size (4 bytes).
• Module APIs are not thread-safe or re-entrant.
• In case blocks span across multiple flash pages update and clear of blocks might not work as expected.
• The application is expected to ensure that when a System OFF is issued, flash access is not ongoing or queued with the module.
• In addition to the error codes listed in the header file for pstorage the SoC flash API error codes can also be returned if API call does not succeed.
• Loading, storing, updating and clearing of each block is supported. A single operation can not be requested for multiple blocks. However, clear is permitted on requested blocks. This is the only exception.
• Power off is not handled by the module. Hence power off when a flash operation is on-going or pending results in lose of data.
• Registering for system events and passing them on to pstorage module using the pstorage_sys_event_handler is mandatory for the module to function as expected. Code snippet below demonstrates what application needs to do for this.

  /**@brief Function for dispatching a system event to interested modules.
    
     @details This function is called from the System event interrupt handler after a system
              event has been received.
    
     @param[in]   sys_evt   System stack event.
   */
  static void sys_evt_dispatch(uint32_t sys_evt)
  {
      pstorage_sys_event_handler(sys_evt);
  }
  
  /**@brief BLE stack initialization.
    
     @details Initializes the SoftDevice and the stack event interrupt.
   */
  static void ble_ant_stack_init(void)
  {
      // Initialize SoftDevice
      SOFTDEVICE_HANDLER_INIT(NRF_CLOCK_LFCLKSRC_XTAL_20_PPM, false);
      
      // Subscribe for BLE events.
      uint32_t err_code = softdevice_ble_evt_handler_set(ble_evt_dispatch);
      APP_ERROR_CHECK(err_code);
      
      // Register with the SoftDevice handler module for System events.
      err_code = softdevice_sys_evt_handler_set(sys_evt_dispatch);
      APP_ERROR_CHECK(err_code);
  }

Registration:

• Registration of a new module shall use a block size which is a multiple of word size (4 bytes).

Store:

• Writing data to blocks that already have data stored is unreliable. The application must clear the blocks before writing fresh data to it.
• No intermediate copy of source data to be stored in flash is made. Application cannot free or reuse the memory that is the source of data until this operation is complete. Completion of this operation is notified through the event notification callback.

Update:

• When using the update routine the Persistent Storage Manager will use a swap area to conserve the blocks not affected. The data page backed up in the swap area, the data page is erased, the non affected blocks are copied back, then the updated block is stored. It is not possible to update multiple blocks in one API call. Module detects if backing up the page is needed or not and optimizes accordingly.
• No intermediate copy of source data to be stored in flash is made. Application cannot free or reuse the memory that is the source of data until this operation is complete. Completion of this operation is notified through the event notification callback.
• Update operation is a time intensive operation, hence developer descretion is required on usage of this API. Application should be tolerant to this.

Clear:

• Clear operation results in clearing all storage blocks allocated to the application. Clearing a single storage block or certain blocks is not implemented. The size parameter of pstorage_clear is currently unused, any value provided will not have any impact on the behaviour of this API.
• The Persistent Storage Module interface allows clearing individual blocks, but the SDK implementation does not. When a clear operation is requested, the current implementation clears all blocks allocated for the module.
• When using the clear operation the Persistent Storage Manager will use a swap area to conserve the blocks not affected. The data page backed up in the swap area, the data page is erased, the non affected blocks are copied back.
• The size shall be multiple of block size expressed in number of bytes. The implementation does not allow clearing a region within a block.
• Update operation is a time intensive operation, hence developer descretion is required on usage of this API. Application should be tolerant to this.