Interoperability Testing – DEV

Interoperability Testing is a required step in the Certification Process.

The following types of Interoperability Testing are supported for Certification:

Interoperability Testing Events

Interoperability testing events are a forum for implementers to gather and validate that their implementations are compatible with each other. This means that each implementer will test out their implementation with those of other implementers. For example, a FIDO UAF Client will test with all UAF Servers and all UAF Authenticators present at a UAF interoperability event; likewise, a FIDO U2F Server would need to test with all U2F Authenticators at a U2F interoperability event. For each combination of participants, the corresponding UAF and U2F testing procedures listed below will be performed. Implementations showing that they can pass the testing procedures with all the other implementations at the event, or that can show that any failed testing procedures are not due to their implementations being non-conformant with the FIDO specifications, will pass the interoperability event and be eligible for certification.

Interoperability Events are held at least once every 90 days, the schedule of which can be found below.  Prior to registering for an interoperability event, an implementation must pass conformance self‐validation and the implementation must not be changed before the interoperability event. Participants must register at least 14 days prior to the event. A non-disclosure agreement is required to protect all test participants’ confidential information.  The non-disclosure agreement is available here.

In the event that there are not enough implementations to hold an interoperability testing event, the event will be cancelled and potential participants will be notified 12 days before the event.

As one of the options during interoperability event registration, implementers will be presented with the choice of whether or not to participate in pre-testing. The intent of pre-testing is to give the companies that will be participating in the interoperability event the ability to exchange software, metadata, and test with each other ahead of the event. For those implementers that opt-in to pre-testing, their contact information will be shared with other implementers ahead of the event so that they may communicate with each other to share the appropriate information and perform pre-testing.

Implementers are encouraged to attend interoperability events in person to help facilitate interaction and problem resolution. For participants unable to travel to interoperability events, remote access will be provided; however, remote participation is not preferred since it makes communication and participation during the interoperability event much more difficult. Remote participants should be prepared to set up both a web camera and screen sharing capability so that the interoperability steps can be visually verified as they are performed.

UAF Interoperability Events

May 25-26, 2016
Synaptics Headquarters
1251 McKay Dr.
San José, CA 95131
Register

August 17-18, 2016
November 9-10, 2016

U2F Interoperability Events

May 24, 2016
Google
Building SB55
1355 Shorebird Way
Mountain View, CA 94043
Register

August 16, 2016
November 8, 2016

On Demand Testing

On Demand Testing has been introduced as an alternative to attending Interoperability Events. On Demand Testing is available year-round.

On Demand testing uses FIDO® Certified reference implementations to complete testing. An implementation is eligible for On Demand testing if the number of reference implementations is equal to the minimum requirements for testing (two implementations from two unique vendors in each implementation class). Vendors can view the Reference Implementation Library.

FIDO members with certified implementations can donate their implementations to the FIDO Reference Implementation Library by filling out the Donation Form.

There are three different options available for On Demand Testing:

  1. Virtual
  2. Shipped
  3. In-Person

Virtual

Virtual On Demand Certification requires the vendor submitting for On Demand testing to provide the Certification Secretariat access to, and instructions for, the operation of their implementation. The Certification Secretariat will facilitate the On Demand Interoperability Testing Process. Contact information for a representative from the vendor company must be provided, and the representative must be available during their testing slot if any questions or issues come up during testing.

Vendors interested in Virtual Testing can use the On Demand Registration Calendar to schedule testing.

Shipped

Shipped On Demand Certification requires the vendor submitting for On Demand testing to ship the implementation, and operating instructions, to our Certification Secretariat P.O. Box in the United States. The Certification Secretariat will facilitate the On Demand Interoperability Testing Process. A company representative should be available if any questions or issues come up during testing. The company will be notified of the result when complete. Once complete, the Certification Secretariat will ship the implementation back to the company.

Vendors interested in Virtual Testing can use the On Demand Registration Calendar to schedule testing.

In-Person

In-Person On Demand testing requires a company representative to be present during the testing procedure. In order to apply for In-Person Testing, the vendor submitting for On Demand Testing must complete an In-Person Testing Request  for each implementation. Requests received will be evaluated on a case-by-case basis by the Certification Secretariat. In-Person testing has an additional $10,000 USD fee per implementation, and the company is responsible for all travel and expenses associated with In-Person testing.

The Certification Secretariat will travel to the vendor’s desired located to perform testing. Each approved In-Person Certification Request entitles the vendor to up to three testing sessions, each lasting a maximum of three days, for a total of nine days. Completing In-Person testing in less than three sessions is possible if the implementation passes all certification requirements.

UAF Interoperability Testing Procedures

Following the policies above, UAF testing will iterate through the prescribed combination of Authenticators / ASMs, Clients and Servers. This will require the following configuration for each set of tests:

  • The Client, Authenticator and Server’s Relying Party Application (RP App) will be loaded on to a single implementation
  • The Server will install the metadata for the Authenticator with corresponding policies and permissions

For each prescribed combination, the following tests will be performed in front of a facilitator:

  1. Register: perform valid registration with the server.
  2. Authenticate: perform valid authentication with the server.
  3. Transaction: perform a transaction with the server. The test must show a text or image indicator of the transaction that is being performed and confirmation that the transaction was successful.
  4. De-Register: remove the registration from the device. Confirm that de-registration was successful by attempting an authentication with the server and confirming that it fails.

Note that due to the time required for configuration of each test and the potential number of combinations for UAF interoperability testing, each test event will span three days. Implementers are expected to attend each day, even if their implementations have already passed all of their designated tests, to facilitate any necessary re-testing. If it is determined that fewer days of testing are required for the interoperability test, participants will be notified at minimum 7 days prior to the event or as soon as reasonably possible.

U2F Interoperability Testing Procedures

Following the policies above, U2F testing will iterate through the prescribed combination of Authenticators and Server. Interoperability testing is performed with the Chrome browser as the U2F Client. Testing is performed with the native U2F functionality of the browser and the U2F Chrome Extension will not be allowed in testing. This policy may be changed when other U2F Clients become available. Each combination of Authenticator and Server will be required to perform the following tests for a facilitator:

  1. Register: The U2F Authenticator will be required to register itself with the U2F Server.
  2. Authenticate: The U2F Authenticator, after being registered with the server, will be required to demonstrate that it can authenticate with the server.

Per the specifications, human interaction is required for each of these steps, such as the touch of a button; the insertion or removal of a device; etc. If the insertion of a device is being used as the form of human interaction, it should require being re-inserted each time a test step is performed.

Implementations may also perform the following test steps:

  1. Negative Register: Register with an invalid certificate in a way that should be rejected by the server.
  2. Negative Authentication: After valid registration, authenticate with invalid credentials in a way that may be rejected by the server.

These optional steps are optional for a client to implement, since some implementations may have difficulty implementing invalid certificates or the other mechanisms required for performing these test steps. However, for clients that do perform these optional steps, servers are required to pass the interoperability testing.