This is a draft of a test plan for testing third party PKCS#11 tokens with our QA test suite. Currently, the test suite only works with the NSS internal softoken, either with or without FIPS140-2, and it tests some limited aspects of the NSS root certificate module.
We want the test suite to be as generic as possible. It shouldn't be specific to one particular token - be it a smartcard, accelerator or HSM. It should be reusable with all PKCS#11 tokens, such as Sun's Niagara 1, Niagara 2, older SCA cards, any nCipher and Rainbow devices, or smartcards. In order to achieve that device-independence from the QA standpoint, I propose that we isolate all the information about the other tokens that the QA test suite will need outside the scripts. This includes :
- a PKCS#11 shared library filename . eg. /usr/lib/libpkcs11.so for Solaris Crypto Framework
- the name of one or more tokens to use from that module (eg. "Sun Metaslot")
- the passwords for each of these tokens, if needed
- the list of mechanisms we want to test from the module, as well as the supported key sizes for each. Normally this information is reported by the module and processed automatically by NSS, but it isn't currently available in human-readable or script-readable form unfortunately. But it should be OK to configure this manually.
- whether the token is a FIPS140 token or not, as this will change the expected results of some tests
- potentially, any limitations about the tokens such as a token being read-only, or having limited capacity (1 cert and key max, for example, for some smartcards). I would suggest having 3 modes of operation - normal (full read/write access), single cert/key, and unlimited
We'll need to somehow make that information available to the QA scripts. Defining environment variables for this purpose is probably the easiest way to go. A configuration file is another option but possibly overkill.
Here is a list of foreseen changes, in no particular order :
- run modutil -add to add the module to secmod.db, and modutil -default to enable selected mechanisms
- for SSL tests, the module should be added only to the secmod.db for one side, either the client side or server side, if they are using a client cert or server cert from the token. The module should not be in both the client and server sides.
- if the token is writeable, specify the "-h token" to certutil when generating keys and signing certificates
- if the token has a limited capacity (1 cert/key), generate all keys and certificates in softoken, export them to PKCS#12 with pk12util, and import them to the token with pk12util only when needed . Remove certs and keys from the token between each test.
- skip ECC or RSA testing depending on token supported mechanisms/key sizes
- delete dead functions that are hardcoded for particular tokens (eg. hw_acc in cert.sh)
- add code to cleanup the token at the end of tests - delete all certs and keys that were used
- parameterize the certificate subject names, to allow multiple simultaneous NSS QA test runs on the same token on the same machine.
- for pk12util, expect failures on FIPS140 tokens, and success only for import for non-FIPS tokens
- for cipher tests, bltest only tests the lowest level of our softoken, not 3rd party tokens. rsaperf can work on third party tokens, but is only for RSA. We may want to enhance other tools to allow using PKCS#11.
- in smime.sh, cmsutil should be able to use nicknames of the form token:nickname
- fips.sh does not apply to 3rd party tokens
- should we skip modifying crmf.sh ?