Articles

INTEGRATION TESTING IN NESTJS USING TESTCONTAINERS

A developer starring into the abyss
Photo by Andrea Piacquadio on Pexels.com

Automated tests are scripts that are executed against your code to ensure you get what you ordered. They are bouncers at your deployment gate that prevent all code Karens from making it to the production club.

Although all forms of testing are tedious - hopefully we can all agree, shipping buggy software to our end users is not an option and neither is hiring a full-time QA team especially if you run a startup. In any case, it means you ship value to your users at a much slower pace. Still, it’s better to have slow deployment cycles with 100% working software than quickly shipping buggy software and have the same users destroy you on Twitter, X, or whatever it’s called lately. (I’d like to give a shoutout to my Kenyan friends. Asante sana✌🏻).

Unit Tests

At the bottom of the Testing Pyramid is unit tests. Unit tests are scripts that test isolated portions of your code according to their public API. You typically run a function or method, and assert that the given input produces the expected output. You'd usually mock your inputs and functions to abstract away any interaction with real services and environments in your unit tests. This makes unit tests run faster with quicker feedback loops compared to the other types of tests we’ll examine. I must confess right out of the gate stating that I do not think unit tests are of much value to me. Many components need to be mocked during unit testing. However, in my opinion, writing automated tests that nearly resemble the production environment situation gives me more confidence; making the giant monster bugs disappear in my sleep at night. Having mentioned that, I still write unit tests for complex logic that I need to confirm behaves as expected. I only tend to lean more on integration and end-to-end tests.

Integration And End-to-end Tests

While integration tests test only a subset of the modules in a system, end-to-end tests verify the entire software system from start to end. Both of these tests are particularly time-consuming and flaky since unlike unit tests where you typically mock out your services and environments, you include how the software will interact with the real world in these. The scope, focus, effort, time, cost, complexity, and benefits are more pronounced in integration and end-to-end tests. I’ll be focusing on integration testing a Nestjs app for the remainder of this article.

Test Environments

A major pain point that deters many developers from writing integration tests or end-to-end tests for that matter is setting up test environments that closely mimic production environments. I used to provision “test” versions of all services of my application. I’d set up a testing database, testing Redis instance, testing messaging server, testing - you get the idea. This required me to install a lot of software on my local machine or provision these on a testing instance of my cloud service. At the mention of a separate cloud instance, you can almost hear cost bing bing bing all over the place. It’s no good. Neither is spending time wrestling with software installation, configuration, managing compatibility, and updates on your local machine. These wear you out even before writing a single test code line. Let’s examine some drawbacks of testing in this manner.

  • Slow tests: remote communication is slower than local communication and suffers from instability or unpredictable latency.
  • Cost: you usually may need to provision a separate database instance to test against for instance. This costs more money.
  • Environment Differences: Even if you pull off installing a bunch of software on your machine to create a test environment, we all run into the eventual “it works on my machine and not on - fill in the blanks” situation.
  • Changing requirements leads to updating/changing software frequently.

If only the test dependencies hurdle could be jumped, we could make integration tests great again(MITGA, join me, would you?).

Enter Testcontainers

Testcontainers is an open-source testing library that provides easy, lightweight APIs for bootstrapping integration tests with real services wrapped in Docker containers. Using Testcontainers, you can write tests that talk to the same types of services you use in production, without mocks or in-memory services. The idea is to provision your dependencies as code while testcontainers handles the heavy lifting.

Testcontainers allow you to run databases, message brokers, headless browsers; and essentially anything that can run in a Docker container. It gives you APIs to manage the lifecycle and configuration of these services. The developer has the full capacity to create an environment that the tests require. You don’t need to provision testing services, and you don’t need to install it locally on your machine. All you need is Docker. Testcontainers has been around for quite some time, starting shortly after docker development started. It was moved in 2022 by ThoughtWorks to the “adopt” stage of their tech radar. It supports many languages and frameworks including Java, Go,.NET,Node.js, Python, Rust, Haskell, Ruby, Clojure, and Elixir. Anything you can containerize will work with Testcontainers. I will focus on how to use Testcontainers to write integration tests within the context of a NestJs app. But for simplicity, we’ll use a Testcontainer Postgres database alone.

How To Use Testcontainers To Write Integration Tests in a Nestjs App

You must have docker installed and running to proceed. Since I'm not teaching how to write NestJS APIs in this article, I have prepared a demo repository for a sample REST API. I also assume you already know how to use Jest to write tests. Most of Jest is already setup in NestJS projects. Therefore, I will focus only on how to use test containers. I used Typeorm as the ORM and Postgres as the database for a fictional e-commerce API.

Install Testcontainers

We'll install the postgres testcontainers module as the demo application uses postgres. Here's the full list of other modules you can install.

yarn add @testcontainers/postgresql

# OR

npm install @testcontainers/postgresql

Typeorm Setup

It is common to see Typeorm being setup by retrieving database-related environment variables with configService as shown below.

// app.module.ts

@Module({
  imports: [
    TypeOrmModule.forRootAsync({
      imports: [ConfigModule],
      inject: [ConfigService],
      useFactory: (configService: ConfigService) => ({
        type: "postgres" as const,
        host: configService.get<string>("DB_HOST"),
        port: configService.get<number>("DB_PORT"),
        username: configService.get<string>("DB_USERNAME"),
        password: configService.get<string>("DB_PASSWORD"),
        database: configService.get<string>("DB_NAME"),
        autoLoadEntities: true,
        synchronize: IS_DEV,
        logging: false,
      }),
      dataSourceFactory: async (options) => {
        const dataSource = await new DataSource(options).initialize();
        return dataSource;
      },
    }),
    AuthModule,
  ],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}

This is great but we'll make a little adjustment in this case as shown below. We’ll retrieve environment variables from the process.env object instead. This is because "MutaScript" here allows us to mutate the process.env object hence we might as well take advantage of this mutability and update the process.env object with database credentials from the postgres testcontainer instead of our local database credentials so that the tests will be run against the testcontainer database. This will be explained better as we continue.

// app.module.ts

TypeOrmModule.forRootAsync({
  useFactory: () => ({
    type: "postgres" as const,
    host: process.env.DB_HOST,
    port: +process.env.DB_PORT,
    username: process.env.DB_USERNAME,
    password: process.env.DB_PASSWORD,
    database: process.env.DB_NAME,
    autoLoadEntities: true,
    synchronize: IS_DEV,
    logging: false,
  }),
  dataSourceFactory: async (options) => {
    const dataSource = await new DataSource(options).initialize();
    return dataSource;
  },
});

The starting point of your app.e2e-spec.ts file will look close to this:

// app.e2e-spec.ts

describe("(e2e) Tests", () => {
  let app: INestApplication;

  beforeEach(async () => {
    const moduleFixture: TestingModule = await Test.createTestingModule({
      imports: [AppModule],
    }).compile();

    app = moduleFixture.createNestApplication();
    await app.init();
  });

  it("/ (GET)", () => {
    return request(app.getHttpServer())
      .get("/")
      .expect(200)
      .expect("Hello World!");
  });
});

We'll update beforeEach to beforeAll because it's not efficient to setup the testcontainers before each test is run. We'll do it once. We'll also declare some variables to hold the supertest request object and the postgres SQL container as show below.

// app.e2e-spec.ts

import {
  PostgreSqlContainer,
  StartedPostgreSqlContainer,
} from "@testcontainers/postgresql";

import * as request from "supertest";

describe("(e2e) Tests", () => {
  jest.setTimeout(1000 * 60 * 1);

  let app: INestApplication;
  let postgreSqlContainer: StartedPostgreSqlContainer;
  let server: request.SuperTest<request.Test>;

  beforeAll(async () => {
    const moduleFixture: TestingModule = await Test.createTestingModule({
      imports: [AppModule],
    }).compile();

    app = moduleFixture.createNestApplication();
    await app.init();
  });

  it("/ (GET)", () => {
    return request(app.getHttpServer())
      .get("/")
      .expect(200)
      .expect("Hello World!");
  });
});

Have you noticed this line: jest.setTimeout(1000 * 60 * 1);? It's vital because without it, jest will timeout even before the postgres image is pulled and the container starts. We've given it enough time to setup the testcontainer. If you run into a timeout error, simply increase the time.

We'll then assign the postgres container to the variable we declared and then update the process.env object with database credentials from the testcontainer postgres database right after that.

// app.e2e-spec.ts

postgreSqlContainer = await new PostgreSqlContainer()
  .withName("testcontainer-demo-api")
  .withDatabase("testcontainer_demo")
  .start();

//update database environment variables for typeorm to connect to the testcontainer DB

process.env.DB_HOST = postgreSqlContainer.getHost();
process.env.DB_PORT = postgreSqlContainer.getPort().toString();
process.env.DB_USERNAME = postgreSqlContainer.getUsername();
process.env.DB_PASSWORD = postgreSqlContainer.getPassword();
process.env.DB_NAME = postgreSqlContainer.getDatabase();

Finally we'll assign the supertest request to the variable we declared and cleanup the container after all tests have been run.

// app.e2e-spec.ts

server = request(app.getHttpServer());

afterAll(async () => {
  await postgreSqlContainer.stop();
  await app.close();
});

Your final setup should look like this:

// app.e2e-spec.ts

describe("(e2e) Tests", () => {
  jest.setTimeout(1000 * 60 * 1);

  let app: INestApplication;
  let postgreSqlContainer: StartedPostgreSqlContainer;
  let server: request.SuperTest<request.Test>;

  beforeAll(async () => {
    postgreSqlContainer = await new PostgreSqlContainer()
      .withName("testcontainer-demo-api")
      .withDatabase("testcontainer_demo")
      .start();

    //update database environment variables for typeorm to connect to the testcontainer DB

    process.env.DB_HOST = postgreSqlContainer.getHost();
    process.env.DB_PORT = postgreSqlContainer.getPort().toString();
    process.env.DB_USERNAME = postgreSqlContainer.getUsername();
    process.env.DB_PASSWORD = postgreSqlContainer.getPassword();
    process.env.DB_NAME = postgreSqlContainer.getDatabase();

    const moduleFixture: TestingModule = await Test.createTestingModule({
      imports: [AppModule],
    }).compile();

    app = moduleFixture.createNestApplication();
    await app.init();
    server = request(app.getHttpServer());
  });

  afterAll(async () => {
    await postgreSqlContainer.stop();
    await app.close();
  });

  it("/ (GET)", () => {
    return server.get("/").expect(200).expect("Hello World!");
  });
});

This is a minimum setup for the purpose of simplicity. You can introduce testcontainers for any other component your application requires. Proceed to add the test cases required by your application.

Running The Test

Run the following command:

yarn test:e2e --detectOpenHandles

Note that running yarn test will run unit tests, and not e2e tests. If you run into the error: “Could not find a working container runtime strategy”, it means docker has not been started on your machine. Install and run docker before executing the test.

I hope you enjoyed the article. Kindly contact me via my socials if you have further questions. Thank you.