API-Tests with Postman

Postman is a popular API development platform that simplifies the process of testing, documenting and sharing APIs. One of the standout features of Postman is the ability to create collections – a group of requests that can be saved and easily run together.

However, manually running each request in a collection can be time-consuming. This is where Newman comes in.

Newman

Newman is Postman’s command-line collection runner that allows you to execute Postman collections directly from the terminal. It is built on top of Node.js and can be installed using npm.

Newman provides several advantages over the Postman app. It can be integrated into your local build process, which means you can run tests without opening the Postman app. It also supports continuous integration and allows you to automate the testing process by integrating with tools like Jenkins or GitHub Actions.

In this article, we will look at how to use Newman to run API-tests on GitHub Actions. This is a very simple example, but it can be extended to run tests against different environments, with different parametrization, etc.

Installing Newman

Newman can be installed using npm. If you don’t have npm installed, you can download it from here .

Once you have npm installed, you can install Newman using the following command:

npm install -g newman

Running a collection

To run a collection, you need to export it as a JSON file. You can do this by clicking on the three dots next to the collection name and selecting “Export”.

Finally, run the collection using the newman run command followed by the path to the exported JSON file.

Newman generates a detailed report with information on the number of tests run, the time taken, and the number of assertions passed or failed. This helps developers to quickly identify issues with their APIs and make necessary adjustments.

GitHub Actions

GitHub Actions is a CI/CD tool that allows you to automate your software development workflow. It is built into GitHub and can be used to run tests, deploy applications, and perform other tasks.

Newman can be integrated with GitHub Actions to run API-tests on every commit. This ensures that any changes made to the codebase do not break existing functionality.

Creating a workflow

To create a workflow, you need to create a .github/workflows directory in your repository. This directory should contain a YAML file with the name of the workflow and the steps to be executed.

We’ll use the matt-ball/newman-action to run our API-tests without having to deal with the installation of Node.js and Newman. The action is available on the GitHub Marketplace.

Please note that the Newman action is not officially supported by Postman. The recommended way to generate a GitHub Action for Postman collections is described here .

The following example shows how to create a workflow that runs API-tests for a spring-rest-app built with gradle on every push on the main branch:

name: Newman Run
on:
  push:
    branches:
      - main

jobs:
  build-and-test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: Set up Java
        uses: actions/setup-java@v2
        with:
          distribution: 'zulu'
          java-version: '17'

      - name: Setup Gradle
        uses: gradle/gradle-build-action@v2
        with:
          gradle-version: 8.0.2
          arguments: build

      - name: Start Spring app
        run: java -jar build/libs/rest-0.0.1-SNAPSHOT.jar &

      - name: Wait for Spring app to start
        run: sleep 7s

      - uses: matt-ball/newman-action@master
        with:
          collection: postman/mytests.postman_collection.json
          environment: postman/environment.json

If a test fails, the workflow will fail. In any case, the Newman action will generate a detailed report with information on the number of tests run, the time taken, and the number of assertions passed or failed.

A typical report generated by Newman looks like this:

→ Read book by id
  GET http://localhost:8080/books/3 [200 OK, 241B, 10ms]
  ✓  Successful GET request
  ✓  The Name of the book is book_one_title

→ Change book
  PUT http://localhost:8080/books/3 [200 OK, 245B, 9ms]
  ✓  Successful PUT request
  ✓  The Author of the book has changed to book_another_author

→ Delete book
  DELETE http://localhost:8080/books/3 [204 No Content, 112B, 4ms]
  ✓  Status code is 204

→ Read deleted book - error
  GET http://localhost:8080/books/3 [404 Not Found, 296B, 26ms]
  ✓  Status code is 404
  ✓  The book with id 3 does no longer exist

┌─────────────────────────┬───────────────────┬──────────────────┐
│                         │          executed │           failed │
├─────────────────────────┼───────────────────┼──────────────────┤
│              iterations │                 1 │                0 │
├─────────────────────────┼───────────────────┼──────────────────┤
│                requests │                 7 │                0 │
├─────────────────────────┼───────────────────┼──────────────────┤
│            test-scripts │                14 │                0 │
├─────────────────────────┼───────────────────┼──────────────────┤
│      prerequest-scripts │                 8 │                0 │
├─────────────────────────┼───────────────────┼──────────────────┤
│              assertions │                14 │                0 │
├─────────────────────────┴───────────────────┴──────────────────┤
│ total run duration: 575ms                                      │
├────────────────────────────────────────────────────────────────┤
│ total data received: 598B (approx)                             │
├────────────────────────────────────────────────────────────────┤
│ average response time: 48ms [min: 4ms, max: 219ms, s.d.: 72ms] │
└────────────────────────────────────────────────────────────────┘

You can generate reports in different formats by using the reporters option. Available reporters are json, junit, progress and emojitrain.

Conclusion

In conclusion, Newman is an excellent tool for developers who want to streamline their API testing process and save time. With its seamless integration with Postman collections and powerful reporting capabilities, it is a must-have tool in any developer’s arsenal.

The newman-github-action makes it easy to integrate Newman with GitHub Actions and helps developers automate their API testing process.