Sinon assert vs Chai assert

Main idea

One day I was covering my code with tests and take note that Sinon also has an assertion as Chai has. I investigated it a little deeper and looked like it makes sense to use Sinon instead of Chai in some cases.

Implementation

The example of the test is based on class which shout down the instances on AWS but it really doesn’t matter such as code is pretty simple.

const AWS = require('aws-sdk');

class SpotScheduler {

  constructor(awsRegion = null) {
    if (awsRegion) {
      AWS.config.update({ region: awsRegion });
    }
    this.ec2 = new AWS.EC2();
  }

  async run() {
    let params = {
      Filters: [{
        "Name": "state",
        "Values": ["open", "active", "disabled"],
      }]
    };

    let response = await this.ec2.describeSpotInstanceRequests(params).promise();
    for (let request of response.SpotInstanceRequests) {
      let params = { InstanceIds: [request.InstanceId] };
      await this.ec2.stopInstances(params).promise();
    }
  }

}

module.exports = SpotScheduler;

So, the test cases are:

• stopInstances should be called once;
• stopInstances should be called with certain parameters;

Create test for these. Don’t afraid the beginning of the test file, it just a preparation for testing AWS SDK which actually doesn’t have real matter in this post.

const AWSMock = require('aws-sdk-mock');
const AWS = require('aws-sdk');
const chai = require('chai');
const sinon = require('sinon');
const SpotScheduler = require('spot-scheduler');

describe('AWS EC2 Spot Instances Lambda Scheduler', async () => {
  it('run: "stopInstances" action should be called once', async() => {
    // Important creating the spy/stub in such way if there are several calls to AWS under the hood
    let stopInstancesSpy = sinon.spy((params, callback) => {
      callback(null, { SpotInstanceRequests: [{ InstanceId: "TEST-SPOT-ID-123" }]});
    })

    AWSMock.setSDKInstance(AWS);

    AWSMock.mock('EC2', 'stopInstances', stopInstancesSpy);
    AWSMock.mock('EC2', 'describeSpotInstanceRequests', async (param, callback) => {
      callback(null, { SpotInstanceRequests: [{ InstanceId: "TEST-SPOT-ID-123", Type: "persistent" }] });
    });

    let spotScheduler = new SpotScheduler('eu-central-1');
    await spotScheduler.run();

    // Assert on your Sinon spy as normal
    sinon.assert.calledOnce(stopInstancesSpy);
    sinon.assert.calledWith(stopInstancesSpy, { InstanceIds: ['TEST-SPOT-ID-123'] });
    sinon.assert.calledWith(stopInstancesSpy.firstCall, { InstanceIds: ['TEST-SPOT-ID-123'] });

    // Assert on your Sinon spy through Chai as normal
    chai.assert.isTrue(stopInstancesSpy.calledOnce);
    chai.assert.deepEqual(stopInstancesSpy.getCall(0).args[0], { InstanceIds: ['TEST-SPOT-ID-123'] });
    chai.expect(stopInstancesSpy.getCall(0).args[0]).to.deep.equal({ InstanceIds: ['TEST-SPOT-ID-123'] });

    // Important! Restore AWS SDK
    AWSMock.restore('EC2');
  });
});

As you can see the assertions seems very similar and it’s true and when test passed with success you don’t see any inconvenience.

However, let’s break our first assertion for Sinon

//sinon.assert.calledOnce(stopInstancesSpy);
sinon.assert.calledTwice(stopInstancesSpy);

and Chai

//chai.assert.isTrue(stopInstancesSpy.calledOnce);
chai.assert.isTrue(stopInstancesSpy.calledTwice);

Run your test, you should get the next error for the first assertion:

   run: "stopInstances" action should be called once:
     expected spy to be called twice but was called once
     spy({ InstanceIds: ["TEST-SPOT-ID-123"] }, function callback() {}) ...

and this one for the second:

   run: "stopInstances" action should be called once:

      AssertionError: expected false to be true
      + expected - actual

      -false
      +true

As you can see the first one is clear for understand without any explanation, but the second one is fully unclear. Yes, I know, we can add a message for Chai assertion but it require additional work. We are lazy, aren’t?

Ok, if you insist on that, I’ll add the message

chai.assert.isTrue(stopInstancesSpy.calledTwice, 'should call stopInstance once via AWS SDK');

Run your test once again:

   run: "stopInstances" action should be called once if an instance is "persistent":

      should call stopInstance once via AWS SDK
      + expected - actual

      -false
      +true

I agree the message is more clear, but anyway it’s not detailed as with Sinon. We can continue to improve this case but it requires additional work, which I don’t want to do.

The other assertions work in the same way.

Conclusion

It is more convenient to use Sinon in some cases, as example for spy/stub assertions. The Sinon assertions are self-explained and more readable when using them with spy/stub, but you shouldn’t consider them as replacement for Chai.

Read more about:

• Sinon assertions
• Chai assertions