How To Perform JSON Schema Validation in API Testing Using Rest-Assured Java

Have you ever come across a situation in automated API testing where you were not able to identify the issue in the test failure and after debugging for multiple hours, you noticed that the data type of the value supplied in the response of the API had changed. Did you then notice that this was the core reason for the test failure?

This type of scenario can generally happen when you have third-party APIs integrated into your application. A real-time example of such a scenario would be integrating with the Bank APIs for making a payment for your e-commerce application or integrating with third-party API which provides the registration and login functionality using two-factor authentication.

In such a situation, though, you would be provided with detailed documentation of the APIs and functionality if it happens that there is a change in the API response from the third-party application since they cater to multiple clients and might have updated their API, maybe for a bug fix or a new feature requirement which you are unaware of.

One of the data type of the field received in response may be changed to integer from String or vice-versa. Or there is a new field/Object added in the response.

Thanks to JSON Schema Validation, these changes can now be caught easily and can save a lot of your efforts and time in debugging and finding the issue that leads to the failure of your system.

Before we begin with discussing the JSON Schema validation, let’s first understand what JSON is, and then continue with the JSON Schema Validation.

What Is JSON?

JSON stands for JavaScript Object Notation. It was originally specified by Douglas Crockford. It is a lightweight format for storing and transporting data and is often used when data is sent from the server to webpages. It is self-describing and easy to understand.

The following are important syntax rules for JSON:

To understand further, let's take the following JSON file as an example:

JSON
  
{
 "page": 1,
 "total_pages": 2,
 "employee_data": [{
   "id": 5,
   "first_name": "Michael",
   "last_name": "Doe",
   "designation": "QA",
   "location": "Remote"
  },
  {
   "id": 6,
   "first_name": "Johnny",
   "last_name": "Ford",
   "designation": "QA",
   "location": "NY,US"
  }
 ],
 "company": {
  "name": "QA Inc",
  "Address": "New Jersey, US"
 }
}


Understanding the JSON File

The above-mentioned file begins with a curly brace {which means the file holds a JSON object. Inside the JSON object, data is stored in multiple data types as follows:

1. The root level itself is a JSON Object as it has a curly bracket to start with and has data stored in a key/value pair

JSON
  
{
 "page": 1,
 "total_pages": 2
}


2. JSON Array

JSON Array stores data inside the JSON file in a block with square bracket []. If we take the example of the JSON file mentioned above,employee_data JSON array has 2 JSON Objects inside it.

JSON
  
"employee_data": [{
   "id": 5,
   "first_name": "Michael",
   "last_name": "Doe",
   "designation": "QA",
   "location": "Remote"
  },
  {
   "id": 6,
   "first_name": "Johnny",
   "last_name": "Ford",
   "designation": "QA",
   "location": "NY,US"
  }
 ]


3. JSON Object

As mentioned earlier, data stored within curly braces are JSON Objects and have multiple key/value pairs in them.

The company JSON Object holds the data for company details:

JSON
  
"company": {
  "name": "QA Inc",
  "Address": "New Jersey, US"
 }


It can also be referred as company key holding the company details record in its value.

What Is JSON Schema?

JSON Schema is a specification for JSON-based format for defining the structure of JSON data.

JSON Schema helps us describe the existing data format and provides clear, human and machine-readable documentation.

As JSON Schema provides complete structural validation, it helps in automated tests and also validating the client-submitted data for verification.

How Do I Generate JSON Schema for the JSON Request of an API?

Consider the following example of Post Response from a restful-booker website where the following data is returned in response once the user hits the post API for creating a new booking:

JSON
  
{
    "bookingid": 1,
    "booking": {
        "firstname": "Jim",
        "lastname": "Brown",
        "totalprice": 111,
        "depositpaid": true,
        "bookingdates": {
            "checkin": "2018-01-01",
            "checkout": "2019-01-01"
        },
        "additionalneeds": "Breakfast"
    }
}


To generate the JSON Schema, we would be using an online JSON schema generator tool from extendsclass.com. Using this tool is very simple, you just need to copy and paste the JSON data for which you need to generate the JSON schema and click on the Generate Schema from JSON button on the web page and it will provide you with the JSON schema for the respective JSON data provided.

How To Perform JSON Schema Validation in API Testing Using Rest-Assured Java

Here is the JSON Schema generated for the above JSON data for creating a new booking:

JSON
  
{
 "definitions": {},
 "$schema": "http://json-schema.org/draft-07/schema#", 
 "$id": "https://example.com/object1661496173.json", 
 "title": "Root", 
 "type": "object",
 "required": [
  "bookingid",
  "booking"
 ],
 "properties": {
  "bookingid": {
   "$id": "#root/bookingid", 
   "title": "Bookingid", 
   "type": "integer",
   "examples": [
    1
   ],
   "default": 0
  },
  "booking": {
   "$id": "#root/booking", 
   "title": "Booking", 
   "type": "object",
   "required": [
    "firstname",
    "lastname",
    "totalprice",
    "depositpaid",
    "bookingdates",
    "additionalneeds"
   ],
   "properties": {
    "firstname": {
     "$id": "#root/booking/firstname", 
     "title": "Firstname", 
     "type": "string",
     "default": "",
     "examples": [
      "Jim"
     ],
     "pattern": "^.*$"
    },
    "lastname": {
     "$id": "#root/booking/lastname", 
     "title": "Lastname", 
     "type": "string",
     "default": "",
     "examples": [
      "Brown"
     ],
     "pattern": "^.*$"
    },
    "totalprice": {
     "$id": "#root/booking/totalprice", 
     "title": "Totalprice", 
     "type": "integer",
     "examples": [
      111
     ],
     "default": 0
    },
    "depositpaid": {
     "$id": "#root/booking/depositpaid", 
     "title": "Depositpaid", 
     "type": "boolean",
     "examples": [
      true
     ],
     "default": true
    },
    "bookingdates": {
     "$id": "#root/booking/bookingdates", 
     "title": "Bookingdates", 
     "type": "object",
     "required": [
      "checkin",
      "checkout"
     ],
     "properties": {
      "checkin": {
       "$id": "#root/booking/bookingdates/checkin", 
       "title": "Checkin", 
       "type": "string",
       "default": "",
       "examples": [
        "2018-01-01"
       ],
       "pattern": "^.*$"
      },
      "checkout": {
       "$id": "#root/booking/bookingdates/checkout", 
       "title": "Checkout", 
       "type": "string",
       "default": "",
       "examples": [
        "2019-01-01"
       ],
       "pattern": "^.*$"
      }
     }
    }
,
    "additionalneeds": {
     "$id": "#root/booking/additionalneeds", 
     "title": "Additionalneeds", 
     "type": "string",
     "default": "",
     "examples": [
      "Breakfast"
     ],
     "pattern": "^.*$"
    }
   }
  }
}
}


Understanding the JSON Schema

If you check the JSON data, the following two fields are the main records:

The following block generated in JSON Schema talks about these 2 fields that in root, these two fields are required as an Object type.

JSON
  
"title": "Root", 
"type": "object",
 "required": [
  "bookingid",
  "booking"
 ],


Next, let's talk about the properties block inside the JSON Schema. The following block states that bookingid should be in the root object and its type should be integer . Hence, in response, it is expected that the value in this field should be an integer only. So, in case this type is changed to any other data type like String ,Object ,longor float, schema validation will fail and we would be able to identify the issue in the schema right away.

JSON
  
"properties": {
  "bookingid": {
   "$id": "#root/bookingid", 
   "title": "Bookingid", 
   "type": "integer",
   "examples": [
    1
   ],
   "default": 0
  },
  "booking": {
   "$id": "#root/booking", 
   "title": "Booking", 
   "type": "object",
   "required": [
    "firstname",
    "lastname",
    "totalprice",
    "depositpaid",
    "bookingdates",
    "additionalneeds"
   ],
   "properties": {
    "firstname": {
     "$id": "#root/booking/firstname", 
     "title": "Firstname", 
     "type": "string",
     "default": "",
     "examples": [
      "Jim"
     ],
     "pattern": "^.*$"
    },
    "lastname": {
     "$id": "#root/booking/lastname", 
     "title": "Lastname", 
     "type": "string",
     "default": "",
     "examples": [
      "Brown"
     ],
     "pattern": "^.*$"
    },
    "totalprice": {
     "$id": "#root/booking/totalprice", 
     "title": "Totalprice", 
     "type": "integer",
     "examples": [
      111
     ],
     "default": 0
    },
    "depositpaid": {
     "$id": "#root/booking/depositpaid", 
     "title": "Depositpaid", 
     "type": "boolean",
     "examples": [
      true
     ],
     "default": true
    },
    "bookingdates": {
     "$id": "#root/booking/bookingdates", 
     "title": "Bookingdates", 
     "type": "object",
     "required": [
      "checkin",
      "checkout"
     ],
     "properties": {
      "checkin": {
       "$id": "#root/booking/bookingdates/checkin", 
       "title": "Checkin", 
       "type": "string",
       "default": "",
       "examples": [
        "2018-01-01"
       ],
       "pattern": "^.*$"
      },
      "checkout": {
       "$id": "#root/booking/bookingdates/checkout", 
       "title": "Checkout", 
       "type": "string",
       "default": "",
       "examples": [
        "2019-01-01"
       ],
       "pattern": "^.*$"
      }
     }
    }
,
    "additionalneeds": {
     "$id": "#root/booking/additionalneeds", 
     "title": "Additionalneeds", 
     "type": "string",
     "default": "",
     "examples": [
      "Breakfast"
     ],
     "pattern": "^.*$"
    }
   }
  }
}
}


Likewise, you can notice the data types and required field values mentioned for the other fields in the JSON Schema.

Performing the JSON Schema Validation Using Rest-Assured Framework

What Is Rest-Assured?

REST-Assured is a Java library that provides a domain-specific language (DSL) for writing powerful, maintainable tests for RESTful APIs. One thing I really like about rest assured is its BDD style of writing tests and one can read the tests very easily in a human-readable language.

Getting Started

The project is created using Maven. Once the project is created we need to add the dependency for rest-assured in pom.xml file. TestNG is used as a test runner.

The following dependencies are mandatorily required to be added in pom.xml

How To Perform JSON Schema Validation in API Testing Using Rest-Assured Java

rest-assured dependency is required for running the API tests and json-schema-validator dependency is required for validating the JSON Schema.

How To Perform JSON Schema Validation in API Testing Using Rest-Assured Java

Validating the JSON Schema

Step 1: First of all, we need to generate the JSON Schema for the response JSON data which we need to validate.

As we are using the restful-booker create booking API, we would be copy-pasting the JSON response and generating the JSON schema using the JSON Schema Validator. In the screenshot below, on the left-hand side we have the JSON response. On clicking the Generate Schema from JSON button , JSON Schema would be generated on the right-hand section.

How To Perform JSON Schema Validation in API Testing Using Rest-Assured Java

The following is the response received from create booking API in restful-booker.

JSON
  
{
    "bookingid": 1,
    "booking": {
        "firstname": "Jim",
        "lastname": "Brown",
        "totalprice": 111,
        "depositpaid": true,
        "bookingdates": {
            "checkin": "2018-01-01",
            "checkout": "2019-01-01"
        },
        "additionalneeds": "Breakfast"
    }
}


The following is the JSON Schema for the above JSON response.

JSON
  
{
 "definitions": {},
 "$schema": "http://json-schema.org/draft-07/schema#", 
 "$id": "https://example.com/object1661586892.json", 
 "title": "Root", 
 "type": "object",
 "required": [
  "bookingid",
  "booking"
 ],
 "properties": {
  "bookingid": {
   "$id": "#root/bookingid", 
   "title": "Bookingid", 
   "type": "integer",
   "examples": [
    1
   ],
   "default": 0
  },
  "booking": {
   "$id": "#root/booking", 
   "title": "Booking", 
   "type": "object",
   "required": [
    "firstname",
    "lastname",
    "totalprice",
    "depositpaid",
    "bookingdates",
    "additionalneeds"
   ],
   "properties": {
    "firstname": {
     "$id": "#root/booking/firstname", 
     "title": "Firstname", 
     "type": "string",
     "default": "",
     "examples": [
      "Jim"
     ],
     "pattern": "^.*$"
    },
    "lastname": {
     "$id": "#root/booking/lastname", 
     "title": "Lastname", 
     "type": "string",
     "default": "",
     "examples": [
      "Brown"
     ],
     "pattern": "^.*$"
    },
    "totalprice": {
     "$id": "#root/booking/totalprice", 
     "title": "Totalprice", 
     "type": "integer",
     "examples": [
      111
     ],
     "default": 0
    },
    "depositpaid": {
     "$id": "#root/booking/depositpaid", 
     "title": "Depositpaid", 
     "type": "boolean",
     "examples": [
      true
     ],
     "default": true
    },
    "bookingdates": {
     "$id": "#root/booking/bookingdates", 
     "title": "Bookingdates", 
     "type": "object",
     "required": [
      "checkin",
      "checkout"
     ],
     "properties": {
      "checkin": {
       "$id": "#root/booking/bookingdates/checkin", 
       "title": "Checkin", 
       "type": "string",
       "default": "",
       "examples": [
        "2018-01-01"
       ],
       "pattern": "^.*$"
      },
      "checkout": {
       "$id": "#root/booking/bookingdates/checkout", 
       "title": "Checkout", 
       "type": "string",
       "default": "",
       "examples": [
        "2019-01-01"
       ],
       "pattern": "^.*$"
      }
     }
    }
,
    "additionalneeds": {
     "$id": "#root/booking/additionalneeds", 
     "title": "Additionalneeds", 
     "type": "string",
     "default": "",
     "examples": [
      "Breakfast"
     ],
     "pattern": "^.*$"
    }
   }
  }
}
}


Project Folder Structure

We can copy the JSON Schema create a new JSON file and put it in the src\test\resources folder inside the project.

How To Perform JSON Schema Validation in API Testing Using Rest-Assured Java

Writing JSON Schema Validation Test

The following test script will allow us to test the JSON Schema validation using the Rest-Assured framework.

Java
  
@Test
public void testCreateBookingJsonSchema() {

    InputStream createBookingJsonSchema = getClass().getClassLoader()
            .getResourceAsStream("createbookingjsonschema.json");
    BookingData newBooking = getBookingData();
    bookingId = given().body(newBooking)
            .when()
            .post("/booking")
            .then()
            .statusCode(200)
            .and()
            .assertThat()
            .body(JsonSchemaValidator.matchesJsonSchema(createBookingJsonSchema))
            .and()
            .extract()
            .path("bookingid");
}


It is pretty simple to write automation tests using Rest-Assured. We need to write the assertion for validating the JSON Schema inside the body() method after the assertThat() method. But before we move to the assertion, we need to read the JSON file we posted inside the src\test\resources folder. To do that we would be using the InputStream class. The following line of code will help us in reading the JSON Schema file createbookingjsonschema.json

Java
  
InputStream createBookingJsonSchema = getClass ().getClassLoader ()
    .getResourceAsStream ("createbookingjsonschema.json");


Next, we need to hit the post API and check the JSON Schema in response by using JsonSchemaValidator.matchesJsonSchema() method and pass the createBookingJsonSchema InputStream instance in it.

The data required in the post-request payload will be generated using Builder pattern + Data Faker.  The following is the implementation of getBookingData() method that is available in the BookingData class.

Java
  
public class BookingDataBuilder {

    private static final Faker FAKER = new Faker ();

    public static BookingData getBookingData () {
        SimpleDateFormat formatter = new SimpleDateFormat ("YYYY-MM-dd");
        return BookingData.builder ()
            .firstname (FAKER.name ()
                .firstName ())
            .lastname (FAKER.name ()
                .lastName ())
            .totalprice (FAKER.number ()
                .numberBetween (1, 2000))
            .depositpaid (true)
            .bookingdates (BookingDates.builder ()
                .checkin (formatter.format (FAKER.date ()
                    .past (20, TimeUnit.DAYS)))
                .checkout (formatter.format (FAKER.date ()
                    .future (5, TimeUnit.DAYS)))
                .build ())
            .additionalneeds ("Breakfast")
            .build ();

    }
}


Once the payload data is generated, it is very easy to write the JSON Schema validation test.

The following lines of code will help us in validating the JSON Schema in the response. Interpreting the lines of code given below, we are sending a post request with the body as required for Post API after which we are checking the status code returned in response is 200 and that the body has the JSON Schema as provided in the createBookingJsonSchema instance.

Java
  
given ().body (newBooking)
    .when ()
    .post ("/booking")
    .then ()
    .statusCode (200)
    .and()
    .assertThat ()
    .body (JsonSchemaValidator.matchesJsonSchema (createBookingJsonSchema));


Running the Tests

Ite time now to run the test and check if the Schema validation happens correctly. Here is the Screenshot of testng.xml file.

XML
  
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Restful Booker Test Suite">
    <test name="Restful Booker JSON Schema Validation tests">
        <classes>
           <class name="com.restfulbooker.JsonSchemaValidationTest">
                <methods>
                    <include name="testCreateBookingJsonSchema"/>
                </methods>
            </class>
        </classes>
    </test>
</suite> <!-- Suite -->


Let’s run the tests now and validate the JSON Schema. We would be running the tests using TestNG by right-clicking on the testng.xml file.

How To Perform JSON Schema Validation in API Testing Using Rest-Assured Java

The JSON Schema received in the Response matches with the JSON Schema provided in the src\test\resources folder passing the test.How To Perform JSON Schema Validation in API Testing Using Rest-Assured Java
Now, let’s make some changes in the JSON Schema in the createbookingjsonschema.json file provided in the src\test\resources

In bookingid field, value of type integer is required however it has been updated to string just to check if the validation is actually working fine.

JSON
  
"properties": {
  "bookingid": {
    "$id": "#root/bookingid",
    "title": "Bookingid",
    "type": "string",
    "examples": [
      1
    ],
    "default": 0
  },


Let’s run the test again by right-clicking on the testng.xml file.

How To Perform JSON Schema Validation in API Testing Using Rest-Assured Java

The following error log was generated and displayed in the console, which says that the value received in response was an integer whereas the value expected was string

JSON
  
error: instance type (integer) does not match any allowed primitive type (allowed: ["string"])
    level: "error"
    schema: {"loadingURI":"#","pointer":"/properties/bookingid"}
    instance: {"pointer":"/bookingid"}
    domain: "validation"
    keyword: "type"
    found: "integer"
    expected: ["string"]


You See! How easy it is to identify such kind of schema-related errors which if not done could have taken a lot of your effort as well as time to find it.

Conclusion

Running automated tests for checking the JSON Schema validation could prove to be a fruitful exercise and help in detecting the schema-level issues before they slip into production. It is recommended to add these checks in the automated pipeline and run them as regression tests in the nightly build. 

Happy Testing!

 

 

 

 

Top