A Simple Introduction to Property Based Testing in JavaScript

Recently I have been exploring some other folks blog posts around a topic that really piqued my interest; property based testing. I wanted to take the time to explore it more as a concept, having experimented with it a little in the JavaScript ecosystem.

Property Based Testing Overview #

Firstly, I want to give an outline of property-based testing. Property based testing is slightly different from software testing approaches you may be familiar with such as unit testing. In a unit test a common approach would be to set up a series of inputs for a function/system and check the output corresponds to what is expected.

Property based testing is slightly different - instead of specifying individual test inputs and expected outputs, property-based tests specify a set of 'properties' that the code should satisfy for any input. Properties are attributes of the programmes behaviour, for example that it always returns a number, as a basic example. Once we have setup the expected property of the programme, it is common for the test framework then generates a large number of random inputs and checking that the properties hold for all of them.

Property Based Testing Libraries and JavaScript Ecosystem #

The main two property based testing libraries for JavaScript from my research appear to be fast-check (3.4k stars) and jsverify (1.6k stars). I'll show a very minimal example of doing some testing with fast-check to give you an idea of how property based testing works.

Below I am going to show a very contrived example using fast-check but hopefully it sets the scene to understand the premise of property based testing. We are going to test the function addPositiveNumbers which aims to take two positive numbers, add them together and return the result.

Property Based Testing with fast-check #

Before we dive into the property based testing approach, lets look at how we traditionally might unit test this in a framework like jest:

test("returns 2 when arguments 1 and 1 are passed", () => {
  expect(() => addPositiveNumbers(1, 1)).toBe(2);
});

test("returns 2 when arguments 2 and 3 are passed", () => {
  expect(() => addPositiveNumbers(2, 3)).toBe(5);
});

This works well and checks that the function, for this specific input returns exactly what we expect it to return. Let's have a look at how we might approach this from a property based testing approach using fast-check.

The main things you will have to wrap you head around with fast-check:

For fast-check it's worth checking out the documentation to understand how to take this all further.

Here, instead of thinking about specific inputs and outputs, property based testing prompts us to think about the 'properties' of the function. What do we know should always hold true for this function?

Let's think out loud:

Let's think about how we might express these in fast-check alongside popular testing framework jest. First we need to understand how to express the arbitraries (the values we want to test against) and the runners (how we want to check that the property holds true). In fast-check we can use the fc.assert runner to check a property holds true consistently. In our case we will want to use the fc.integer arbitrary to ensure our addPositiveNumbers holds true to our properties.

import fc from "fast-check";
import { addPositiveNumbers } from "./example";

test("should throw when both values are negative", () => {
  fc.assert(
    fc.property(fc.integer({ max: 0 }), fc.integer({ max: 0 }), (a, b) => {
      expect(() => addPositiveNumbers(a, b)).toThrowError();
    })
  );
});

test("should throw when one value is negative", () => {
  fc.assert(
    fc.property(fc.integer({ max: 0 }), fc.integer({ min: 0 }), (a, b) => {
      expect(() => addPositiveNumbers(a, b)).toThrowError();
    })
  );
});

test("should always greater than first argument", () => {
  fc.assert(
    fc.property(fc.integer({ min: 0 }), fc.integer({ min: 0 }), (a, b) => {
      expect(addPositiveNumbers(a, b)).toBeGreaterThanOrEqual(a);
    })
  );
});

test("should always be greater than second argument", () => {
  fc.assert(
    fc.property(fc.integer({ min: 0 }), fc.integer({ min: 0 }), (a, b) => {
      expect(addPositiveNumbers(b, a)).toBeGreaterThanOrEqual(b);
    })
  );
});

These properties are arguably 'basic' and we could probably think of more advanced properties that exist in relation to the function. This blog post explores some great ideas around how to reason about your programme and come up with good properties to test.

Metamorphic Relations #

Once you've wrapped your head around the above, you can go a level deeper there is a more advanced concept called metamorphic relations. The basic idea is to use the output of one test case as the input for another test case, where the developer understands the relationship between the two functions under test and can use them to check against each other. The relationship between the outputs is called a metamorphic relationship, and it can be used to generate new test cases that are likely to uncover defects in the software.

We could say for example:

test("addPositiveNumbers can be inverted with minusTwoPositiveNumbers", () => {
  fc.assert(
    fc.property(fc.integer({ min: 0 }), fc.integer({ min: 0 }), (a, b) => {
      const result = addPositiveNumbers(a, b);
      const minusResultOne = minusTwoPositiveNumbers(result, a);
      const minusResultTwo = minusTwoPositiveNumbers(minusResultOne, b);

      expect(minusResultTwo).toEqual(0);
    })
  );
});

Here these functions are simple, but metamorphic relationships are particularly useful for testing complex software systems that are hard to test using traditional methods, for example where it is hard for the developer to determine what the output should be. They can also be used to generate test cases for systems that have multiple outputs, such as image processing or compression algorithms.

Published