Your browser does not appear to support Javascript, please update your browser or contact your system administrator to enable Javascript on your Internet browser. Thank you. Chapter 5: Test Plan — U.S. Election Assistance Commission
Skip to content

U.S. Election Assistance Commission

Personal tools
You are here: Home TGDC Recommended Guidelines Part 2: Documentation Requirements Chapter 5: Test Plan
TGDC Recommended
Guidelines

VVSG Navigation
 

Chapter 5: Test Plan (test lab)

This chapter defines required content for the test plan, which is to be prepared by the test lab. It does not specify an overall organization for the test plan, nor does it enumerate all of the content that would be reasonable and customary for a test lab to include. Test labs are encouraged to apply relevant external standards, such as [IEEE97] and [IEEE98] or their logical successors, to determine the organization and content of test plans, provided that the information described in this chapter does appear in the result.

The purpose of the test plan is to document the test lab's development of the complete or partial test suites. To some extent, the test plan is determined by the Testing Requirements (Part 3). The test plan must document the test suites so that the results of testing are reproducible.

Prior to development of any test plan, the test lab must obtain the Technical Data Package (TDP) from the manufacturer submitting the voting system for conformity assessment. The TDP contains information necessary to the development of the test plan, such as the manufacturer's hardware specifications, application logic specifications, operating manual, and maintenance manual.

5.1 Test plan contents

5.1-A Test plan references

The test lab SHALL list all documents that contain material used in preparing the test plan.

Applies To: Voting system

Source: [VVSG2005] II.A.1.1

5.1-B Test plan, implementation statement

The test lab SHALL include a copy of the implementation statement provided by the manufacturer.

Applies To: Voting system

Source: Revision of [VVSG2005] II.A.1

5.1-B.1 Test plan, clarifications to implementation statement

The test lab SHALL document any interpretations made by the test lab to fully identify the implementation under test and the scope of assessment that is desired.

Applies To: Voting system

5.1-C Test plan, inventory of materials delivered

The test lab SHALL enumerate the materials delivered by the manufacturer to the test lab to enable conformity assessment to occur.

Applies To: Voting system

DISCUSSION

Materials include hardware, software, the TDP, evidence of prior certifications, test ballots, test data, etc.

Source: [VVSG2005] II.A.3

5.1-C.1 Test plan, specificity of inventory

Materials SHALL be identified by specific version, serial number, etc., if they are versioned or numbered, and the quantity of each SHALL be noted.

Applies To: Voting system

5.1-D Test plan, previous work

The test lab SHALL document all prior certifications, reviews, tests, or other conditions that impact the test lab's determination of the scope of conformity assessment, and document said impact.

Applies To: Voting system

DISCUSSION

The test lab may recognize certifications, reviews, and tests conducted by other labs, whether they are accredited for voting system conformity assessment or not, as making some portions of the voting system test campaign redundant. For example, a COTS computer should already have been certified to comply with the Rules and Regulations of the Federal Communications Commission, Part 15, Subpart B requirements for both radiated and conducted emissions and need not be retested for that. Also, if a slightly modified system is submitted for reassessment, the test lab's finding that some or all of the test campaign need not be repeated would be documented under this requirement.

Sometimes new systems use a combination of new devices interfaced with the devices of a previously tested system. For example, a manufacturer can submit a voting system for conformity assessment that has a new DRE voting device, but that integrates the election management subsystem from a previously tested system. In this situation, the accredited test lab may design and perform a test procedure that draws on the results of testing performed previously on reused subsystems. However, irrespective of previous testing performed, the scope of testing is expected to cover:

  1. All functionality performed by new devices;
  2. All functionality performed by modified devices;
  3. Functionality that is accomplished using any interfaces to new devices, or that shares inputs or outputs from new devices;
  4. All functionality related to vote tabulation and election results reporting; and
  5. All functionality related to audit trail maintenance.

Source: [VVSG2005] II.3.2.4, II.A.2, II.B.1.2

5.1-E Test plan, reproducible testing

The test lab SHALL provide the information needed to reproduce the testing that it performs, including facility requirements, test set-up, test sequence, test operations procedures, data recording requirements, and pass criteria.

Applies To: Voting system

Source: Condensed from [VVSG2005] II.A.5 and 6

5.1-E.1 Test plan, standard test suites

For applicable tests that are specified in Part 3, the test lab SHALL document the implementation details that determine how the standard tests are realized for the implementation under test.

Applies To: Voting system

Source: New requirement

5.1-E.2 Test plan, public test suites

For tests that the test lab is adopting from publicly available test suites, the test lab SHALL identify the public reference and document the implementation details that determine how the public tests are realized for the implementation under test.

Applies To: Voting system

Source: New requirement

5.1-E.3 Test plan, other test suites

For all other tests, the test lab SHALL incorporate all relevant information into the test plan as needed to reproduce the testing.

Applies To: Voting system

Source: New requirement

5.1-F Test plan, responsible parties

The test lab SHALL identify the parties responsible for conducting the conformity assessment, including subcontracted test labs and engineers assigned to the task.

Applies To: Voting system

Source: New requirement