Cryptography tests

Cryptography tests verify the functionality of the nRF Security by using known test vectors approved by the National Institute of Standards and Technology (NIST) and others.

Requirements 

The tests support the following development kits:

Hardware platforms	PCA	Board name	Build target
nRF5340 DK	PCA10095	nrf5340dk_nrf5340	`nrf5340dk_nrf5340_cpuapp`
nRF9160 DK	PCA10090	nrf9160dk_nrf9160	`nrf9160dk_nrf9160`
nRF52840 DK	PCA10056	nrf52840dk_nrf52840	`nrf52840dk_nrf52840`

Note

Nordic Semiconductor devices such as nRF51, nRF52810, or nRF52811 cannot run the full test suite because of limited flash capacity. A recommended approach in such case is to run subsets of the tests one by one.

Overview 

Cryptography tests use Zephyr Test Framework (Ztest) to run the tests. See Test Framework for details. The tests do not use the standard Ztest output but provide custom output for the test reports. See Ztest custom log formatting for details.

The tests are executed if the cryptographic functionality is enabled in Kconfig. Make sure to configure nRF Security and all available hardware or software backends to enable the tests. See CONFIG_NORDIC_SECURITY_BACKEND.

Cryptographic mode	Sub-mode	Link to standard	Test Vector Source
AES	CBC	NIST - AES	NIST - AES
	CFB		NIST - AES
	ECB		NIST - AES
	CTR	NIST SP 800-38A	NIST SP 800-38A
	CBC MAC	CBC MAC	No official test vectors
	CMAC	NIST SP 800-38B	NIST SP 800-38B
AEAD	CCM	NIST - CCM	NIST - CCM
	CCM*	Formal Specification of the CCM* Mode of Operation - September 9, 2005	Formal Specification of the CCM* Mode of Operation - September 9, 2005
	EAX	The EAX Mode of Operation	The EAX Mode of Operation
	GCM	NIST - GCM	NIST - GCM
	ChaCha-Poly	RFC 7539 - ChaCha20 and Poly1305 for IETF Protocols	RFC 7539 - ChaCha20 and Poly1305 for IETF Protocols
ECDH	secp160r1	NIST - ECDH	GEC 2: Test Vectors for SEC 1
	secp160r2		No test vectors
	secp192r1		NIST - ECDH
	secp224r1		NIST - ECDH
	secp256r1		NIST - ECDH
	secp384r1		NIST - ECDH
	secp521r1		NIST - ECDH
	secp160k1		No test vectors
	secp192k1		No test vectors
	secp224k1		No test vectors
	secp256k1		No test vectors
	bp256r1	RFC 7027 - ECC Brainpool Curves for TLS	RFC 7027 - ECC Brainpool Curves for TLS
	bp384r1		RFC 7027 - ECC Brainpool Curves for TLS
	bp512r1		RFC 7027 - ECC Brainpool Curves for TLS
	Curve25519	RFC 7748 - Elliptic Curves for Security	RFC 7748 - Elliptic Curves for Security
ECDSA	secp160r1	NIST - ECDSA	No test vectors
	secp160r2		No test vectors
	secp192r1		No test vectors
	secp224r1		NIST - ECDSA
	secp256r1		NIST - ECDSA
	secp384r1		NIST - ECDSA
	secp521r1		NIST - ECDSA
	secp160k1		No test vectors
	secp192k1		No test vectors
	secp224k1		No test vectors
	secp256k1		No test vectors
	bp256r1	RFC 5639 - ECC Brainpool Standard Curves and Curve Generation	No test vectors
	bp384r1		No test vectors
	bp512r1		No test vectors
	EdDSA	RFC 8032 - Edwards-Curve Digital Signature Algorithm (EdDSA)	RFC 8032 - Edwards-Curve Digital Signature Algorithm (EdDSA)
Hash	SHA256	NIST - SHA	NIST - SHA
	SHA512		NIST - SHA
HMAC	HMAC SHA256	NIST - HMAC	NIST - HMAC
	HMAC SHA512		NIST - HMAC
HKDF	HKDF SHA256	RFC 5869 - HMAC-based Extract-and-Expand Key Derivation Function (HKDF)	RFC 5869 - HMAC-based Extract-and-Expand Key Derivation Function (HKDF)
	HKDF SHA512		RFC 5869 - HMAC-based Extract-and-Expand Key Derivation Function (HKDF)
EC-JPAKE	secp256r1	J-PAKE: Password-Authenticated Key Exchange by Juggling	Custom

Building and running 

This test can be found under tests/crypto/ in the nRF Connect SDK folder structure.

See Programming an application for information about how to build and program the tests.

There are multiple ways to build the tests. See nRF Security for additional information about configuring the nRF Security subsystem. You can use the following configuration files to build the test in a specific setup:

overlay-cc3xx.conf uses hardware acceleration using the Arm CryptoCell accelerator (for cryptography and entropy for random number generation).
overlay-cc3xx-oberon.conf uses a combination of hardware acceleration, using the Arm CryptoCell, and the Oberon software library, that adds key sizes and algorithms not supported in the CryptoCell. This setup uses hardware acceleration as much as possible.
overlay-oberon.conf uses only the Oberon software library for all cryptographic operations.
overlay-vanilla.conf is for software only, except for a hardware-accelerated module to generate entropy for random number generation.
overlay-multi.conf uses a combination of hardware acceleration, using the Arm CryptoCell, and vanilla Mbed TLS and Oberon software implementations to support functionalities not supported by the CryptoCell. This setup uses hardware acceleration as much as possible.

You can use one of the listed overlay configurations by adding the -- -DEXTRA_CONF_FILE=<overlay_config_file> flag to your build. Also see Providing CMake options for instructions on how to add this option.

Ztest custom log formatting 

Cryptography tests replace the standard Ztest formatting to assure more efficient reporting of running tests and test results. Set the configuration option CONFIG_ZTEST_TC_UTIL_USER_OVERRIDE to replace the Ztest macros TC_START and Z_TC_END_RESULT with versions more suited for reporting results of cryptographic tests.

CONFIG_ZTEST_TC_UTIL_USER_OVERRIDE uses tests/crypto/include_override/tc_util_user_override.h to define the custom formatting.

Testing 

Compile and program the application.
Connect the kit to the computer using a USB cable. The kit is assigned a COM port (Windows) or ttyACM device (Linux), which is visible in the Device Manager.
Connect to the kit with a terminal emulator (for example, nRF Connect Serial Terminal). See Testing and optimization for the required settings and steps.
Observe the result of the different test vectors in the terminal emulator log. The last line of the output indicates the test result:
```
PROJECT EXECUTION SUCCESSFUL
```

Additional test cases and test vectors 

You can add test cases and test vectors to the test suite either by including additional source files or by extending the existing files.

Test case

A test case is a function designed to verify parts of the functionality of a cryptographic operation. Most cryptographic operations, like hash calculations and ECDH, have multiple test cases to be able to cover all features. A typical test case is called by looping over the registered test vectors and calling the test case. The execution logs the verdict for each test vector.

Registering a test case

A new test case must be registered to the test_case_data section using ITEM_REGISTER, which registers it with the named section test_case_hmac_data:

ITEM_REGISTER(test_case_hmac_data, test_case_t test_hmac) = {
.p_test_case_name = "HMAC",
.setup = hmac_setup,
.exec = exec_test_case_hmac,
.teardown = hmac_teardown,
.vector_type = TV_HMAC,
.vectors_start = __start_test_vector_hmac_data,
.vectors_stop = __stop_test_vector_hmac_data,
};

Note

The macro call to ITEM_REGISTER must be done in a .c file.

Setting up a test case

As part of the test case setup, any previously used buffers are cleared. The next test vector is fetched using the ITEM_GET macro. The macro requires the following parameters:

test_vector_hmac_data - The section to fetch the test vector from (HMAC in this example).
test_vector_hmac_t - Information about which type of test vector to expect in the given section. In the example, test_vector_hmac_t is expected. It is the same type that is used when registering HMAC test vectors.
Information about which index to fetch a test vector from.

The fetched test vector is then unhexified.

Test vector data is stored as strings of hexadecimal characters. To use them, they must be parsed to binary, which is also done in the setup procedure.

The following example shows a test vector setup:

void hmac_setup(void)
{
   hmac_clear_buffers();
   p_test_vector = ITEM_GET(test_vector_hmac_data, test_vector_hmac_t,
                hmac_vector_n);
   unhexify_hmac();
}
void exec_test_case_hmac(void)
...

On teardown, the test vector index is incremented, so that the next call to hmac_setup by the Ztest framework fetches the next test vector:

void hmac_combined_teardown(void)
{
   hmac_combined_vector_n++;
}

Executing a test case

An example of an HMAC test case in a simplified form is shown below:

void exec_test_case_hmac(void)
{
int err_code = -1;

/* Initialize the HMAC module. */
mbedtls_md_init(&md_context);

const mbedtls_md_info_t *p_md_info =
        mbedtls_md_info_from_type(p_test_vector->digest_type);
err_code = mbedtls_md_setup(&md_context, p_md_info, 1);
if (err_code != 0) {
        LOG_WRN("mb setup ec: -0x%02X", -err_code);
}
TEST_VECTOR_ASSERT_EQUAL(err_code, 0);

start_time_measurement();
err_code = mbedtls_md_hmac_starts(&md_context, m_hmac_key_buf, key_len);
TEST_VECTOR_ASSERT_EQUAL(err_code, 0);

err_code =
        mbedtls_md_hmac_update(&md_context, m_hmac_input_buf, in_len);
TEST_VECTOR_ASSERT_EQUAL(err_code, 0);

/* Finalize the HMAC computation. */
err_code = mbedtls_md_hmac_finish(&md_context, m_hmac_output_buf);
stop_time_measurement();

TEST_VECTOR_ASSERT_EQUAL(p_test_vector->expected_err_code, err_code);

/* Verify the generated HMAC. */
TEST_VECTOR_ASSERT_EQUAL(expected_hmac_len, hmac_len);
TEST_VECTOR_MEMCMP_ASSERT(m_hmac_output_buf, m_hmac_expected_output_buf,
                          expected_hmac_len,
                          p_test_vector->expected_result,
                          "Incorrect hmac");

mbedtls_md_free(&md_context);
}

Test vectors

A test vector is a set of inputs and expected outputs to verify the functionality provided in a test case:

typedef const struct {
        const uint32_t digest_type;        /**< Digest type of HMAC operation. */
        const int expected_err_code;    /**< Expected error code from HMAC operation. */
        const uint8_t expected_result;  /**< Expected result of HMAC operation. */
        const char *p_test_vector_name; /**< Pointer to HMAC test vector name. */
        const char
                *p_input;                   /**< Pointer to input message in hex string format. */
        const char *p_key;              /**< Pointer to HMAC key in hex string format. */
        const char *p_expected_output;  /**< Pointer to expected HMAC digest in hex string format. */
} test_vector_hmac_t;

Registering a test vector

Test vectors are added by registering them for a section defined in the test case code. The test vector is registered in the section test_vector_hmac_data, which is defined in the test case example exec_test_case_hmac. The test vector can reuse the already defined hash test vector structure test_vector_hmac_t, as shown in the code block below:

/* HMAC - Custom test vector */
ITEM_REGISTER(test_vector_hmac_data,
              test_vector_hmac_t test_vector_hmac256_min_key_min_message_0) = {
        .digest_type = MBEDTLS_MD_SHA256,
        .expected_err_code = 0,
        .expected_result = EXPECTED_TO_PASS,
        .p_test_vector_name = TV_NAME("SHA256 key_len=1 message_len=1 zeros"),
        .p_input = "00",
        .p_key = "00",
        .p_expected_output =
                "6620b31f2924b8c01547745f41825d322336f83ebb13d723678789d554d8a3ef"
};

Output logging

The test project generates a test log using RTT or UART output. Executing exec_test_case_hmac with its registered test vectors adds the following output to the test log:

Running test suite HMAC
SHA256 key_len=131 message_len=152 -- PASS -- [../test_cases/test_vectors_hmac.c:259]
SHA256 key_len=131 message_len=54 -- PASS -- [../test_cases/test_vectors_hmac.c:240]
SHA256 key_len=25 message_len=50 -- PASS -- [../test_cases/test_vectors_hmac.c:225]
SHA256 key_len=20 message_len=50 -- PASS -- [../test_cases/test_vectors_hmac.c:210]
SHA256 key_len=4 message_len=28 -- PASS -- [../test_cases/test_vectors_hmac.c:197]
SHA256 key_len=20 message_len=8 -- PASS -- [../test_cases/test_vectors_hmac.c:184]
SHA256 key_len=74 message_len=128 -- PASS -- [../test_cases/test_vectors_hmac.c:164]
SHA256 key_len=64 message_len=128 -- PASS -- [../test_cases/test_vectors_hmac.c:145]
SHA256 key_len=45 message_len=128 -- PASS -- [../test_cases/test_vectors_hmac.c:127]
SHA256 key_len=40 message_len=128 -- PASS -- [../test_cases/test_vectors_hmac.c:109]
SHA256 key_len=1 message_len=1 non-zeros -- PASS -- [../test_cases/test_vectors_hmac.c:96]
SHA256 key_len=1 message_len=1 zeros -- PASS -- [../test_cases/test_vectors_hmac.c:82]
SHA256 invalid - signature changed -- PASS -- [../test_cases/test_vectors_hmac.c:64]
SHA256 invalid - key changed -- PASS -- [../test_cases/test_vectors_hmac.c:46]
SHA256 invalid - message changed -- PASS -- [../test_cases/test_vectors_hmac.c:28]
Test suite HMAC succeeded