Skip to main content

Test Plan Types

At a high level test plans come in two flavors:

  1. Zero Configuration (zero conf)
  2. Data Driven

Zero Configuration vs. Data Driven

Zero ConfigurationData Driven
Time to ValueInstantRequires One Time Configuration
Test Category CoverageReducedComprehensive
Efficacy of Test ResultsModerateSuperior
Horizontal Authorization Abuse Coverage (BOLA / IDOR)NoYes
Vertical Authorization Abuse Coverage (Privilege Escalation)YesYes
Coverage for API Methods (GET, POST, etc.)All Methods
  • All Methods.
  • PUT, POST, PATCH, & DELETE require additional configuration to support stateful operations on resources.
Primary Use CaseRapid AssessmentsDeep / Comprehensive Assessments

Zero Configuration

Zero Configuration test plans are instantly runnable, and only require specifying user authentication information.

These test plans use type information specified in the API schema to autogenerate data for various parameters required by the API.

For example is a string field in a POST body is of format type email, the test plan will autogenerate syntactically valid email addresses.

However there is no guarantee that the generated values are truly valid for the API endpoint being tested (the values may not exist in the backend database, etc.). Since parameter data is autogenerated, these test plans provide less security test coverage, and lower efficacy that Data Driven test plans.

For example, APIs often return 404 Not Found in lieu of 401 Unauthorized or 403 Forbidden during authentication/authorization failures. This is prevent hackers from enumerating resources.

Zero Configuration test plans cannot truly distinguish between an authentication/authorization failure and the absence of the requested resource in the API's backend database.

This results in lower efficacy and reduced test coverage (results might have false negatives).

Data Driven

As the name suggests, parameter data for APIs is end user supplied via fixtures.

By configuring parameter data, end users have complete control of what data is send when making API calls.

This dramatically increases efficacy of test results and test coverage, as the test plans can clearly distinguish between, authentication/authorization failures and the absence of the requested resource.