Skip to main content

Simple API Testing Experiments

By Updated

This page is set of suggested experiments. You might use this as a basic API Test Plan or API Test Approach - in reality all test plans are a really a set of experiments, and we adjust them based on the results of our experimentation.

Use it when you want to start practicing testing without a formal set of challenges. Remember you don't need a rigid set of tests scripts in order to test a system. You need domain knowledge to identify ideas, technical knowledge to spot issues, and a set of ideas.

This set of experiments is just an initial set of ideas. Each experiment gives you something to try, what to look for, and what to vary next.

The real key is not to be limited by the experiments listed, and think of new ideas as you test, or when you finish and think about: what else you could do, or what you could do better.

The main endpoints for the Simple API are:

  • /simpleapi/items
  • /simpleapi/items/{id}
  • /simpleapi/randomisbn

The item fields are:

Field Notes
id generated by the API
type required enum: book, blu-ray, cd, dvd
isbn13 required, unique, 13 digits with optional hyphens
price required number from 0 to 50000
numberinstock integer from 0 to 100, default 0

How To Experiment

References: HTTP requests and responses, API testing coverage, document your testing.

Work in small loops:

  1. Start with a request that should work.
  2. Examine the full response: status, headers, body, and any important ids.
  3. Change one thing.
  4. Send the request again.
  5. Compare the new response with the previous response and the documentation.

Useful evidence includes the request, response status, Content-Type, Location, error messages, generated id, and a follow-up GET that provides evidence of the stored state.

Helpful pages:

Baseline Collection Experiment

References: HTTP GET, HTTP headers, JSON response bodies, status codes.

Start with a simple read:

GET /simpleapi/items
Accept: application/json
GET /simpleapi/items as a JSON baseline

Look for:

  • status 200
  • Content-Type: application/json
  • a body with an items collection
  • items with the documented fields
  • field types that match the documentation
  • no unexpected private or user-supplied free text

Vary:

  • remove the Accept header
  • send Accept: */*
  • ask for Accept: application/xml
  • compare collection and single-item response shapes

This gives you a known-good response before you start changing data.

Create And Verify Experiment

References: HTTP POST, HTTP GET, REST CRUD, HTTP headers.

Create a fresh item and then check that it exists.

First get a unique ISBN:

GET /simpleapi/randomisbn
GET /simpleapi/randomisbn to generate an ISBN

Then create an item:

POST /simpleapi/items
Content-Type: application/json
Accept: application/json

{
  "type": "book",
  "isbn13": "123-4-56-789012-3",
  "price": 2.99,
  "numberinstock": 3
}
POST /simpleapi/items to create a known item
GET /simpleapi/items/{id} to check the created item

Look for:

  • status 201
  • generated id
  • Location header for the new item
  • response body values matching the request
  • no client-supplied id required for creation

Vary:

  • create each allowed type
  • use hyphenated and unhyphenated isbn13 values
  • omit numberinstock and check the default
  • try boundary values such as price: 0 and numberinstock: 0
  • retrieve the Location URL and compare the stored item with the create response

Questions:

  • what is the minimum set of parameters you need in order to create an item?

Validation Experiment

References: Data risks, HTTP status codes, Content-Type header.

Use invalid data to check whether the API rejects bad requests clearly and without side effects.

Try variations such as:

  • missing type
  • unknown type
  • missing isbn13
  • duplicate isbn13
  • malformed isbn13
  • price below 0
  • price above 50000
  • numberinstock below 0
  • numberinstock above 100
  • numberinstock as a string
  • an extra field the schema does not describe
  • check for field lengths - might it be possible that only the first few chars are used?
  • check for spaces padding, try other padding - might it be doing partial matching?
POST /simpleapi/items and experiment with invalid data

Look for:

  • a suitable 4xx status
  • useful error messages
  • consistent error response format
  • no item created when validation fails
  • the failed request does not change an existing item

Vary:

  • change one invalid field at a time
  • combine multiple invalid fields and inspect whether all errors are reported
  • compare JSON validation with XML validation
  • repeat a failed create and confirm it fails consistently
  • Accept and Content-Type headers

ISBN Uniqueness Experiment

References: Data risks, REST CRUD, HTTP POST.

The isbn13 field must be unique. Hyphens are supposed to be ignored for uniqueness, so these should represent the same ISBN:

1234567890123
123-4-56-789012-3

Try:

  1. Create an item with a generated ISBN.
  2. Create another item with the same ISBN.
  3. Create another item with the same digits but different hyphen placement.
  4. Check the collection for duplicates.
POST /simpleapi/items and experiment with duplicate ISBNs

Look for:

  • the first create succeeds
  • duplicate creates are rejected
  • hyphen-only variations do not bypass uniqueness
  • only one item exists with the same ISBN digits

Vary:

  • duplicate during POST
  • duplicate during PUT
  • duplicate during PATCH
  • case is not supposed to be relevant because the ISBN is numeric, so you might choose to focus on separators and length
  • check that case is actually not relevant, otherwise your earlier testing might be invalid

Update Experiment

References: HTTP PUT, HTTP PATCH, payloads vs body, REST CRUD.

Use PUT for full replacement and PATCH for partial change.

Try a full replacement:

PUT /simpleapi/items/{id}
Content-Type: application/json
Accept: application/json

{
  "type": "dvd",
  "isbn13": "123-4-56-789012-3",
  "price": 4.56,
  "numberinstock": 8
}

Then try a partial update:

PATCH /simpleapi/items/{id}
Content-Type: application/json
Accept: application/json

{
  "price": 9.99
}
PUT /simpleapi/items/{id} to replace an item
PATCH /simpleapi/items/{id} to amend values
GET /simpleapi/items/{id} after update

Look for:

  • PUT changes the full item to match the submitted representation
  • PATCH changes only the submitted fields
  • unchanged fields remain unchanged after PATCH
  • follow-up GET confirms the stored state
  • invalid updates do not partially corrupt the item

Vary:

  • omit fields in PUT
  • include the id in the request body and compare with the URL id
  • update isbn13 to a duplicate value
  • send unknown fields
  • use JSON Merge Patch and JSON Patch to check the API supports the documented media types

Delete Experiment

References: HTTP DELETE, common HTTP status codes, REST CRUD.

Delete an item and then prove it has gone.

DELETE /simpleapi/items/{id}
DELETE /simpleapi/items/{id} to remove an item
GET /simpleapi/items/{id} after delete

Look for:

  • expected success status, usually 204
  • no response body when the status says no content
  • follow-up GET /simpleapi/items/{id} returns a 404 missing-resource response
  • collection count or search result reflects the deletion
  • remember this a multi-user system so someone might also be testing

Vary:

  • delete the same id twice
  • delete an id that never existed
  • delete with different Accept headers
  • try DELETE /simpleapi/items on the collection and check whether it is rejected

Unsupported Method Experiment

References: HTTP methods, HTTP OPTIONS, HTTP TRACE, common HTTP status codes.

Check that endpoints reject methods they do not support.

Try:

TRACE /simpleapi/items
DELETE /simpleapi/items
POST /simpleapi/randomisbn
PUT /simpleapi/randomisbn
PATCH /simpleapi/randomisbn
DELETE /simpleapi/randomisbn
Experiment with unsupported Simple API methods

Look for:

  • appropriate 405 Method Not Allowed or documented status
  • Allow header listing supported methods where applicable
  • no state change from rejected methods
  • consistent error response format

Vary:

  • compare collection endpoint methods with single-item endpoint methods
  • send OPTIONS and compare the Allow header with the documentation
  • try unsupported methods with and without request bodies

Query And Selection Experiment

References: query strings, HTTP GET, HTTP QUERY, coverage of query parameters.

Use the collection endpoint to practise retrieving subsets of data.

Try documented QUERY and GET filter approaches against:

  • type
  • isbn13
  • price
  • numberinstock
  • id
GET /simpleapi/items? to filter the collection
QUERY /simpleapi/items with a form body

Look for:

  • response contains only matching items
  • unknown query fields are handled predictably
  • numeric comparisons behave as numbers, not strings
  • empty result sets are valid responses
  • filters do not mutate data

Vary:

  • exact match filters
  • boundary values
  • values that match no items
  • URL encoding around special characters
  • combining filters, if documented

Content Negotiation Experiment

References: Accept header, HTTP headers, requesting formats.

/simpleapi/items is useful for experimenting with response formats because the data is simple and safe to change.

Try GET /simpleapi/items with different Accept headers:

Accept: application/json
Accept: text/xml
Accept: application/vnd.apichallenges.item+xml
Accept: application/*+xml
Accept: application/xml;q=0.5, application/json;q=1
Accept: application/problem+json
GET simpleapi accept headers

Look for:

  • status code
  • exact response Content-Type
  • whether the body is valid for the returned media type
  • whether unsupported high-priority types fall back to a lower-priority supported type
  • whether q=0 excludes a media type

Vary:

  • switch the q-values so XML is preferred over JSON
  • use Accept: application/problem+json, application/json;q=0.5
  • use Accept: application/json;q=0, application/xml;q=0
  • compare application/xml, text/xml, vendor +xml, and application/*+xml
  • try application/*+json and confirm it does not automatically match plain application/json

application/problem+xml, application/problem+json, and similar error-document media types should not be treated as normal item representations unless the API explicitly documents them.

  • What other headers make a difference?

Request Body Format Experiment

References: Content-Type header, requesting formats, JSON body format, XML body format.

For write requests, compare the request Content-Type with the response Accept header.

Try:

Content-Type: application/json
Accept: application/json

Content-Type: application/xml
Accept: application/xml

Content-Type: text/xml
Accept: application/json

Content-Type: application/vnd.apichallenges.item+xml
Accept: application/json
POST /simpleapi/items with XML and Accept JSON
POST /simpleapi/items with vendor item XML

Look for:

  • the request body is parsed according to Content-Type
  • the response format follows Accept
  • unsupported request body types return 415 Unsupported Media Type
  • malformed JSON or XML returns a clear client error
  • successful creates and updates store equivalent data from JSON and XML bodies

Vary:

  • send JSON with Content-Type: application/xml
  • send XML with Content-Type: application/json
  • omit Content-Type on a request with a body
  • use vendor +xml for item bodies
  • request JSON back after sending XML

Error Response Experiment

References: HTTP status codes, common API issues, data risks, Accept header.

Error responses are part of the API contract.

Try:

  • GET /simpleapi/items/999999
  • malformed JSON body
  • unsupported Accept
  • unsupported Content-Type
  • invalid field type
  • missing mandatory field
Experiment with requests to trigger errors

Look for:

  • status code matches the problem
  • response body explains what went wrong
  • error body has a predictable structure
  • response Content-Type matches the body
  • no server stack traces or internal implementation details leak to the client
  • repeated invalid requests behave consistently

Vary:

  • JSON and XML Accept headers for errors
  • missing headers
  • invalid ids with different shapes such as numbers, words, and very long values
  • invalid data on POST, PUT, and PATCH
  • anything that you think should trigger an error response

Documentation Comparison Experiment

References: OpenAPI for testing, viewing an OpenAPI file, standard and permissive files.

Use the documentation as an oracle, then use the API to challenge it.

Compare:

Experiment with Simple API requests to compare with the docs

Look for:

  • all documented endpoints are reachable
  • documented methods match OPTIONS and real behavior
  • schemas match actual response fields and types
  • status codes are complete and accurate
  • examples can be sent without hidden fixes
  • standard and permissive OpenAPI files differ only where intended

Vary:

  • use a browser-based client, a command-line client, and an API client
  • export the OpenAPI file into another tool
  • compare generated client behavior with hand-written requests
  • tooling: do the different representations of the documentation trigger different ideas? Do they reveal more information? Or do they hide information? Do the editable clients help you more than the read only clients?

Multi-User And Data Lifecycle Experiment

References: REST guidance, data risks, coverage driven testing.

The Simple API is shared and safe to change. Data refreshes automatically when low, and there is a maximum number of items.

Try:

  • create several items with recognizable values
  • delete items you created
  • watch whether ids increase over time
  • observe what happens when the collection is near its maximum size
  • refresh the data explorer and compare it with API responses
Experiment with Simple API data lifecycle requests

Look for:

  • other users' changes do not break your test assumptions
  • tests create their own data instead of relying on fixed existing data
  • tests clean up after themselves when possible
  • the API recovers when data is low
  • limits return a clear status and message

Vary:

  • run the same create flow twice
  • run two clients against the API
  • use unique ISBNs for every test
  • avoid relying on collection order unless the API documents it

Good Automated Execution Candidates

References: coverage of what, document your testing, HTTP methods.

After you have explored manually, automate the checks that are stable and valuable.

Good first automated experiments:

  • GET /simpleapi/items returns a valid collection
  • POST /simpleapi/items creates an item with a unique ISBN
  • GET /simpleapi/items/{id} retrieves the created item
  • duplicate isbn13 values are rejected
  • invalid field types are rejected
  • PUT replaces an item
  • PATCH changes only the intended field
  • DELETE removes an item
  • unsupported methods are rejected
  • Accept and Content-Type negotiation behaves as documented

Keep exploratory notes as well as automated coverage. Automating is good for repeatable assertions, but exploration helps you notice confusing behavior, weak documentation, and risks you did not know to script yet.

How are you protecting your automated execution against multiple users working at the same time as your automated execution runs? Remember the application can be run locally. Are you controlling the data? Or are you making your assertions internally consistent rather than based on external setup expectations?

Continue Experimenting

References: what would we test?, coverage driven testing, HTTP clients, HTTP proxies.

The scope of Testing possibilities is rarely every complete. There is a potentially infinite amount of inputs you can send to the API - many will not provide you any additional information because they are equivalent variations on a theme.

Identify new experiments to try e.g.:

  • use different tools
  • put a proxy in the middle and view the raw requests - what features does the proxy have to help you test?
  • use browser clients and use the Network tab to view the raw responses - do you see differences from the Proxy?
  • read the standards, and compare the API against the standards
  • look for common API faults - is this API vulnerable to them?
  • write your own plan and experiment set from scratch - what would you cover? would you vary the order? if so, why?
  • try AI tools, how can they help you?

This is intended to be a practical thinking experiment - perform it multiple times.

And use it as a basis for other APIs - explore the API Challenges todo api based on what you learn from this experiment set, also the buggy API, and try your ideas using the practice APIs we've listed.

Experiment with API

Keep Experimenting:

  • make notes on what you observe
  • reflect on these and think about what they are telling you
  • interrogate and vary that endpoint and input data set more carefully
  • build a coverage model and see what you missed
  • what risks can you identify? experiment and see if they manifest.

 

Copyright Compendium Developments Ltd 2020 - 2026