Cryptography tests
Cryptography tests verify the functionality of the Nordic Security Module by using known test vectors approved by the National Institute of Standards and Technology (NIST) and others.
Requirements
The tests support the following development kits:
Hardware platforms |
PCA |
Board name |
Build target |
|---|---|---|---|
PCA10095 |
|
||
PCA10090 |
|
||
PCA10056 |
|
Note
Nordic Semiconductor devices such as nRF51, nRF52810, or nRF52811 cannot run the full test suite because of limited flash capacity. A recommended approach in such case is to run subsets of the tests one by one.
Overview
Cryptography tests use Zephyr Test Framework (Ztest) to run the tests. See Test Framework for details. The tests do not use the standard Ztest output but provide custom output for the test reports. See Ztest custom log formatting for details.
The tests are executed if the cryptographic functionality is enabled in Kconfig.
Make sure to configure Nordic Security Module and all available hardware or software backends to enable the tests.
See CONFIG_NORDIC_SECURITY_BACKEND.
Cryptographic mode |
Sub-mode |
Link to standard |
Test Vector Source |
|---|---|---|---|
AES |
CBC |
||
CFB |
|||
ECB |
|||
CTR |
|||
CBC MAC |
CBC MAC |
No official test vectors |
|
CMAC |
|||
AEAD |
CCM |
||
CCM* |
Formal Specification of the CCM* Mode of Operation - September 9, 2005 |
Formal Specification of the CCM* Mode of Operation - September 9, 2005 |
|
EAX |
|||
GCM |
|||
ChaCha-Poly |
|||
ECDH |
secp160r1 |
GEC 2: Test Vectors for SEC 1 |
|
secp160r2 |
No test vectors |
||
secp192r1 |
|||
secp224r1 |
|||
secp256r1 |
|||
secp384r1 |
|||
secp521r1 |
|||
secp160k1 |
No test vectors |
||
secp192k1 |
No test vectors |
||
secp224k1 |
No test vectors |
||
secp256k1 |
No test vectors |
||
bp256r1 |
|||
bp384r1 |
|||
bp512r1 |
|||
Curve25519 |
|||
ECDSA |
secp160r1 |
No test vectors |
|
secp160r2 |
No test vectors |
||
secp192r1 |
No test vectors |
||
secp224r1 |
|||
secp256r1 |
|||
secp384r1 |
|||
secp521r1 |
|||
secp160k1 |
No test vectors |
||
secp192k1 |
No test vectors |
||
secp224k1 |
No test vectors |
||
secp256k1 |
No test vectors |
||
bp256r1 |
RFC 5639 - ECC Brainpool Standard Curves and Curve Generation |
No test vectors |
|
bp384r1 |
No test vectors |
||
bp512r1 |
No test vectors |
||
EdDSA |
RFC 8032 - Edwards-Curve Digital Signature Algorithm (EdDSA) |
RFC 8032 - Edwards-Curve Digital Signature Algorithm (EdDSA) |
|
Hash |
SHA256 |
||
SHA512 |
|||
HMAC |
HMAC SHA256 |
||
HMAC SHA512 |
|||
HKDF |
HKDF SHA256 |
RFC 5869 - HMAC-based Extract-and-Expand Key Derivation Function (HKDF) |
RFC 5869 - HMAC-based Extract-and-Expand Key Derivation Function (HKDF) |
HKDF SHA512 |
RFC 5869 - HMAC-based Extract-and-Expand Key Derivation Function (HKDF) |
||
EC-JPAKE |
secp256r1 |
Custom |
Building and running
This test can be found under tests/crypto/ in the nRF Connect SDK folder structure.
See Building on the command line for information about how to build and program the tests.
There are multiple ways to build the tests. See Nordic Security Module for additional information about configuring the Nordic Security Module. You can use the following configuration files to build the test in a specific setup:
overlay-cc3xx.confuses hardware acceleration using the Arm CryptoCell accelerator (for cryptography and entropy for random number generation).overlay-cc3xx-oberon.confuses a combination of hardware acceleration, using the Arm CryptoCell, and the Oberon software library, that adds key sizes and algorithms not supported in the CryptoCell. This setup uses hardware acceleration as much as possible.overlay-oberon.confuses only the Oberon software library for all cryptographic operations.overlay-vanilla.confis for software only, except for a hardware-accelerated module to generate entropy for random number generation.overlay-multi.confuses a combination of hardware acceleration, using the Arm CryptoCell, and vanilla Mbed TLS and Oberon software implementations to support functionalities not supported by the CryptoCell. This setup uses hardware acceleration as much as possible.
You can use one of the listed overlay configurations by adding the -- -DOVERLAY_CONFIG=<overlay_config_file> flag to your build. Also see Providing CMake options for instructions on how to add this option.
Ztest custom log formatting
Cryptography tests replace the standard Ztest formatting to assure more efficient reporting of running tests and test results.
Set the configuration option CONFIG_ZTEST_TC_UTIL_USER_OVERRIDE to replace the Ztest macros TC_START and Z_TC_END_RESULT with versions more suited for reporting results of cryptographic tests.
CONFIG_ZTEST_TC_UTIL_USER_OVERRIDE uses tests/crypto/include_override/tc_util_user_override.h to define the custom formatting.
Testing
Compile and program the application.
Observe the result of the different test vectors in the log using RTT Viewer or a terminal emulator. The last line of the output indicates the test result:
PROJECT EXECUTION SUCCESSFUL
Additional test cases and test vectors
You can add test cases and test vectors to the test suite either by including additional source files or by extending the existing files.
Test case
A test case is a function designed to verify parts of the functionality of a cryptographic operation. Most cryptographic operations, like hash calculations and ECDH, have multiple test cases to be able to cover all features. A typical test case is called by looping over the registered test vectors and calling the test case. The execution logs the verdict for each test vector.
Registering a test case
A new test case must be registered to the test_case_data section using ITEM_REGISTER, which registers it with the named section test_case_hmac_data:
ITEM_REGISTER(test_case_hmac_data, test_case_t test_hmac) = {
.p_test_case_name = "HMAC",
.setup = hmac_setup,
.exec = exec_test_case_hmac,
.teardown = hmac_teardown,
.vector_type = TV_HMAC,
.vectors_start = __start_test_vector_hmac_data,
.vectors_stop = __stop_test_vector_hmac_data,
};
Note
The macro call to ITEM_REGISTER must be done in a .c file.
Setting up a test case
As part of the test case setup, any previously used buffers are cleared.
The next test vector is fetched using the ITEM_GET macro.
The macro requires the following parameters:
test_vector_hmac_data- The section to fetch the test vector from (HMAC in this example).test_vector_hmac_t- Information about which type of test vector to expect in the given section. In the example,test_vector_hmac_tis expected. It is the same type that is used when registering HMAC test vectors.Information about which index to fetch a test vector from.
The fetched test vector is then unhexified.
Test vector data is stored as strings of hexadecimal characters. To use them, they must be parsed to binary, which is also done in the setup procedure.
The following example shows a test vector setup:
void hmac_setup(void)
{
hmac_clear_buffers();
p_test_vector = ITEM_GET(test_vector_hmac_data, test_vector_hmac_t,
hmac_vector_n);
unhexify_hmac();
}
void exec_test_case_hmac(void)
...
On teardown, the test vector index is incremented, so that the next call to hmac_setup by the Ztest framework fetches the next test vector:
void hmac_combined_teardown(void)
{
hmac_combined_vector_n++;
}
Executing a test case
An example of an HMAC test case in a simplified form is shown below:
void exec_test_case_hmac(void)
{
int err_code = -1;
/* Initialize the HMAC module. */
mbedtls_md_init(&md_context);
const mbedtls_md_info_t *p_md_info =
mbedtls_md_info_from_type(p_test_vector->digest_type);
err_code = mbedtls_md_setup(&md_context, p_md_info, 1);
if (err_code != 0) {
LOG_WRN("mb setup ec: -0x%02X", -err_code);
}
TEST_VECTOR_ASSERT_EQUAL(err_code, 0);
start_time_measurement();
err_code = mbedtls_md_hmac_starts(&md_context, m_hmac_key_buf, key_len);
TEST_VECTOR_ASSERT_EQUAL(err_code, 0);
err_code =
mbedtls_md_hmac_update(&md_context, m_hmac_input_buf, in_len);
TEST_VECTOR_ASSERT_EQUAL(err_code, 0);
/* Finalize the HMAC computation. */
err_code = mbedtls_md_hmac_finish(&md_context, m_hmac_output_buf);
stop_time_measurement();
TEST_VECTOR_ASSERT_EQUAL(p_test_vector->expected_err_code, err_code);
/* Verify the generated HMAC. */
TEST_VECTOR_ASSERT_EQUAL(expected_hmac_len, hmac_len);
TEST_VECTOR_MEMCMP_ASSERT(m_hmac_output_buf, m_hmac_expected_output_buf,
expected_hmac_len,
p_test_vector->expected_result,
"Incorrect hmac");
mbedtls_md_free(&md_context);
}
Test vectors
A test vector is a set of inputs and expected outputs to verify the functionality provided in a test case:
typedef const struct {
const uint32_t digest_type; /**< Digest type of HMAC operation. */
const int expected_err_code; /**< Expected error code from HMAC operation. */
const uint8_t expected_result; /**< Expected result of HMAC operation. */
const char *p_test_vector_name; /**< Pointer to HMAC test vector name. */
const char
*p_input; /**< Pointer to input message in hex string format. */
const char *p_key; /**< Pointer to HMAC key in hex string format. */
const char *p_expected_output; /**< Pointer to expected HMAC digest in hex string format. */
} test_vector_hmac_t;
Registering a test vector
Test vectors are added by registering them for a section defined in the test case code.
The test vector is registered in the section test_vector_hmac_data, which is defined in the test case example exec_test_case_hmac.
The test vector can reuse the already defined hash test vector structure test_vector_hmac_t, as shown in the code block below:
/* HMAC - Custom test vector */
ITEM_REGISTER(test_vector_hmac_data,
test_vector_hmac_t test_vector_hmac256_min_key_min_message_0) = {
.digest_type = MBEDTLS_MD_SHA256,
.expected_err_code = 0,
.expected_result = EXPECTED_TO_PASS,
.p_test_vector_name = TV_NAME("SHA256 key_len=1 message_len=1 zeros"),
.p_input = "00",
.p_key = "00",
.p_expected_output =
"6620b31f2924b8c01547745f41825d322336f83ebb13d723678789d554d8a3ef"
};
Output logging
The test project generates a test log using RTT or UART output.
Executing exec_test_case_hmac with its registered test vectors adds the following output to the test log:
Running test suite HMAC
357: SHA256 key_len=131 message_len=152 -- PASS -- [../test_cases/test_vectors_hmac.c:259]
358: SHA256 key_len=131 message_len=54 -- PASS -- [../test_cases/test_vectors_hmac.c:240]
359: SHA256 key_len=25 message_len=50 -- PASS -- [../test_cases/test_vectors_hmac.c:225]
360: SHA256 key_len=20 message_len=50 -- PASS -- [../test_cases/test_vectors_hmac.c:210]
361: SHA256 key_len=4 message_len=28 -- PASS -- [../test_cases/test_vectors_hmac.c:197]
362: SHA256 key_len=20 message_len=8 -- PASS -- [../test_cases/test_vectors_hmac.c:184]
363: SHA256 key_len=74 message_len=128 -- PASS -- [../test_cases/test_vectors_hmac.c:164]
364: SHA256 key_len=64 message_len=128 -- PASS -- [../test_cases/test_vectors_hmac.c:145]
365: SHA256 key_len=45 message_len=128 -- PASS -- [../test_cases/test_vectors_hmac.c:127]
366: SHA256 key_len=40 message_len=128 -- PASS -- [../test_cases/test_vectors_hmac.c:109]
367: SHA256 key_len=1 message_len=1 non-zeros -- PASS -- [../test_cases/test_vectors_hmac.c:96]
368: SHA256 key_len=1 message_len=1 zeros -- PASS -- [../test_cases/test_vectors_hmac.c:82]
369: SHA256 invalid - signature changed -- PASS -- [../test_cases/test_vectors_hmac.c:64]
370: SHA256 invalid - key changed -- PASS -- [../test_cases/test_vectors_hmac.c:46]
371: SHA256 invalid - message changed -- PASS -- [../test_cases/test_vectors_hmac.c:28]
Test suite HMAC succeeded