An Overview of JavaScript Testing in 2020

This guide is intended to catch you up with the most important reasoning, terms, tools, and approaches to JavaScript testing as for the year 2020.

It combines information from the best articles recently released (they are referenced at the bottom) and adds from my own experience at Welldone Software, where we have implemented different testing solutions for different products over many years.

It?s a little long but whoever reads and understands this guide, can safely assume they know the overall state of JavaScript testing in the web development community.

Image for post

Just a couple of years ago website testings had very little documentation, were slow, hard to implement, not deterministic, and in general not too fun to work on.

Nowadays, the website testing field has stabilized. Several testing tools receive very positive feedbacks from their users, well known testing best practices has been created, and testing became a more integral part of the job of many developers.

Today?s cutting edge website testing tools are fast, informative, and easy to work with. These tools would make your testing and developing experience much more enjoyable.

According to The State of JavaScript. Jest, the leading unit test framework that we would discuss later in details, has it?s satisfaction rates at 96%!

Looking forward, I forecast a large entrance of AI into the field of automated testing. Some such tools already exist, and actively improving the workflow and experience of thousands of developers.

If you see anything I missed or got wrong, leave a comment and I?ll address it right away.
I want to keep this guide updated so if you see anything outdated, let me know by commenting below.
Notice the links at the bottom of the page. Reading them might provide you with a lot of extra information.

Enjoy 🙂

Test Types

You can read about different types of tests in more depth here and here and here.

In general, the most important test types for a website are:

Unit Tests- Testing of individual units like functions or classes by supplying input and making sure the output is as expected:

Integration Tests- Testing processes across several units to achieve their goals, including their side effects:

End-to-end Tests (also known as e2e tests or functional tests)- Testing how scenarios function on the product itself, by controlling the browser or the website. These tests usually ignore the internal structure of the application entirety and look at them from the eyes of the user like on a black box.

Running Tests

Tests can run in the browser by creating an HTML page with the test libraries and test files included as JS scripts.
Tests can run in a headless browser which is way to launch browsers where they run without actually rendering on the screen. This way you can run them even in a command line environment and very fast.
Tests can also be executed in Node.js by simply importing the test files and dependent libraries. jsdom is commonly used in Node.js to simulate a browser-like environment using pure JavaScript. Jsdom simulates whatever you get when you run your JS inside the browser like window, document, body, location, cookies, selectors but it renders nothing real.

Out of the three, we suggest using the third method (Node.js + jsdom) when possible, because it?s much faster than the other two. The first and second methods may be more reliable because you?re using the the same exact software which will render your site in real life.

Test Tools Types

Test tools can be divided into the following functionalities. Some provide us with only one functionality, and some provide us with a combination.

To achieve the most flexible set functionality, it?s common to use a combination of several tools.

Test launchers are used to launch your tests in the browser or Node.js with user config. (Karma, Jasmine, Jest, TestCafe, Cypress, webdriverio)
Testing structure providers help you arrange your tests in a readable and scalable way. (Mocha, Jasmine, Jest, Cucumber, TestCafe, Cypress)
Assertion functions are used to check if a test returns what you expect it to return and if its?t it throws a clear exception. (Chai, Jasmine, Jest, Unexpected, TestCafe, Cypress)
Generate and display test progress and summary. (Mocha, Jasmine, Jest, Karma, TestCafe, Cypress)
Mocks, spies, and stubs to simulate tests scenarios, isolate the tested part of the software from other parts, and attach to processes to see they work as expected. (Sinon, Jasmine, enzyme, Jest, testdouble)
Generate and compare snapshots to make sure changes to data structures from previous test runs are intended by the user?s code changes. (Jest, Ava)
Generate code coverage reports of how much of your code is covered by tests. (Istanbul, Jest, Blanket)
Browser Controllers simulate user actions for Functional Tests. (Nightwatch, Nightmare, Phantom, Puppeteer, TestCafe, Cypress)
Visual Regression Tools are used to compare your site to its previous versions visually by using image comparison techniques.(Applitools, Percy, Wraith, WebdriverCSS)

Let?s explain some of the terms mentioned above:

Test launchers

Launch a list of tests based on a configuration you provide (what browsers to run in, what babel plugins to use, how to format the output, etc).

Testing Structure

Refers to the organization of your tests. Nowadays, tests are usually organized in a BDD structure that supports behavior-driven development (BDD). It often looks like this:

Assertion Functions

Are used to make sure that tested variables contain the expected value. They usually look like one of these:

TIP: Here is a nice article about advanced Jasmine and Jest assertions.

Spies

Attach themselves to functions to provide us extra information about them. For example, how many times were they called, in which cases, by whom, what was passed to them in each call?

Spies are used in integration tests to make sure that the side effects of a process are as expected. For example, how many times was a calculation function like execute in this case called?

Stubbing or Dubbing

Replaces selected methods of existing modules with user-supplied functions in order to ensure expected behavior during the test.

To ensure user.isValid() will always return true during a test, where you test something completely different, you can use:

Mocks or Fakes

Used to fake certain modules or behaviors to test different parts of a processes.

Sinon can, for example, fake a server to ensure offline, fast and expected responses when testing a certain flow.

Snapshot Testing

Allows you to compare a data structure to what it was in older releases.

The following example, from the official Jest documentation, shows a snapshot test of a certain Link component.

It doesn?t actually renders and takes a picture of the component, but it saves its internal data in a separate file like this:

When the test runs, and the new snapshot differs from the last one, the developer is prompted to confirm that the change is intended.

Image for post

Notice: Snapshots are usually made to compare component representation data but they can also compare other types of data, like redux stores states or the inner structure of different units in the application.

Browsers Controllers

There are several ways to control browsers to simulate user behavior, like clicking the mouse, drag and dropping, typing, and navigating.

Some tools install drivers that are installed on top of them and control the browser using different methods. This is how selenium works:

Your code on Node.js <> WebDriver <> FF/Chrome/Safari Driver <> Browser

Another way would be to inject scripts of a JS code that has access to the whole application environment: DOM, network, cookies etc? to raise events that simulate user behavior. For example, here is how you simulate a click:

document.getElementByID(‘someButton’).dispatchEvent(clickEvent)

Lately, a new way became very popular. Browser maintainers, exposed their own native APIs that lets you control the browsers directly. For example puppeteer:

Your code on Node.js <> Browser’s API

Putting it All Together

To start testing you need to choose a testing structure that suits you, choose the style of assertion functions you like, and decide how do you want to run the tests.

We will discuss the tools you can choose from later in the article.

Some frameworks like Jest, Jasmine, TestCafe, and Cypress provide all of these out of the box. Some of them provide only some of the functionality and a combination of libraries can be used. A famous combinations of tools would be: mocha + chai + sinon.

We also suggest creating two different processes. One for running unit and integration tests and another one for Functional Tests. This is because functional tests usually take longer, especially when running the test suite on several different browsers.

Unit and integration tests can run on the fly, as you code, by using a ?watch mode?. E2E tests, usually need much more time and are usually launched before merges and releases.

Unit Tests

Should cover all the small pure units of an application- utils, services and helpers. Provide all these units with simple and edge case inputs and make sure their outputs are as expected using the assertion functions.Also make sure to use a code coverage reporting tool to know which units are covered.

Unit tests are one of the reasons to use functional programming and pure functions as much as possible.

The purer your application is, the easier you can test it.

Integration Tests

Old school tests were focused on unit testing and resulted in applications where many small parts were working but the processes as a whole kept on failing.Integration tests, on the other hand, detect cases where a unit is refactored and passes its tests but a process that depends on it fails.

Image for post

The best demonstration of why testing more than each part of a system separately is important can be seen in this great GIF.

I created this GIF based on a slide in this great lecture byViv Richards that I recommend to watch.

It?s also important to remember that in the real world, for the reasons of imperfect design and the widespread use of black boxes, not all units are pure and not all units are testable- some units can be tested only as part of a bigger process.

Integration tests should cover important cross-module processes. Sometimes they extend to processes across several classes and sometimes to testing different systems like Front-End-Back-End interactions.

Comparing to unit tests, you would probably benefit from using spies to ensure expected side effects instead of just asserting the output and stubs to mock and modify the parts of the process that are not in the specific test.

Component snapshot tests fall into this category as well. They provide us with a way to test how processes affect selected component structures and data structure without actually rendering them into a browser or browser-like environment.

Functional Tests

Sometimes the quick and effective unit and integration tests are not enough.

Functional tests control browsers and simulate user behavior on these environments (clicking, typing, scrolling etc?) and make sure these scenarios actually work from the point of view of an end user.

Also it is worth noting that many services provide you with devices and browsers to run these tests on. Here are even more services like these.

Visual regression testing tools can also be set up to verify that the different screens of your applications are ok visually by a smart comparison of screenshots. These screenshots are usually taken as part of your Functional Tests or by executing a separate session of browser automation.

Image for post Applitools smart visual regression tools

List of General Prominent Testing Tools

There are dozens of great tools out there. I couldn?t include all of them here but I tried to include the most important to know, best maintained , and the most adopted tools in the following list:

jsdom

jsdom/jsdom

jsdom ? A JavaScript implementation of the WHATWG DOM and HTML standards, for use with node.js

github.com

jsdom is a JavaScript implementation of the WHATWG DOM and HTML standards. In other words, jsdom simulates a browser?s environment without running anything but plain JS.

As mentioned before, in this simulated browser environment, tests would run really fast. The drawback of jsdom is that not everything can be simulated outside a real browser (you can?t take a screenshot for example) so using it will limit your test?s reach.

It?s worth mentioning that the JS community rapidly improves jsdom and the current version is very close to support whatever exists on a real browser.

Storybook

Storybook: UI component explorer for frontend developers

“Storybook is a powerful frontend workshop environment tool that allows teams to design, build, and organize UI?

storybook.js.org

While not, strictly speaking, a testing tool, Storybook lets you write components in special ?stories? which enable developing and interacting with components in isolation.

Not only it encourages you to write your components in a more ?testable? fashion, you can actually test the components in storybook, using Chromatic that we would discuss below.

Testing Library

Testing Library Simple and complete testing utilities that encourage good testing practices

The Testing Library family of libraries is a very light-weight solution for testing without all the implementation?

testing-library.com

Simple and complete testing utilities that encourages good testing practices. It is developed by Kent C. Dodds which is testing guru. I suggest to have a look at his work to learn more about best practices in testing websites.

The library provides special tools for different frameworks like React, Preact, React Native, Marko, Angular, Vue, and Svelte. The most famous of them is React Testing Library which is very widely adopted.

It also helps with E2E testing tools like Cypress, Puppeteer, Testcafe, and Nightwatch that would all be discussed next.

It helps with convenient selectors, firing events, configuration, dealing with asynchronous code, and many more.

Electron

electron/electron

Electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS

github.com

The Electron framework lets you write cross-platform desktop applications using JavaScript, HTML and CSS. It also has a headless mode.

It has a huge community and many important applications are built on top of it, so it is supposed to stay up to date:Atom, Slack, Skype, GitHub Desktop and many more.

Testing tools like Cypress.io use Electron to launch tests with maximum control of the browser.

Istanbul

Istanbul, a JavaScript test coverage tool.

Istanbul instruments your ES5 and ES2015+ JavaScript code with line counters, so that you can track how well your?

istanbul.js.org

Istanbul will tell you how much of your code is covered with unit tests. It will report on statement, line, function and branch coverage in percentages so you will understand better what is left to cover.

Karma

Karma ? Spectacular Test Runner for Javascript

The main goal for Karma is to bring a productive testing environment to developers. The environment being one where?

karma-runner.github.io

Karma hosts a test server with a special web page to run your tests in the page?s environment. This page can be run across many browsers and browser-like environments including jsdom.

Chai

chaijs/chai

chai ? BDD / TDD assertion framework for node.js and the browser that can be paired with any testing framework.

github.com

Chai is the most popular assertion library. It has many plugins and extensions.

Unexpected

unexpectedjs/unexpected

unexpected ? Unexpected ? the extensible BDD assertion toolkit

github.com

Unexpected is an assertion library with a slightly different syntax from Chai. It is also extensible so assertions can be more advanced with libraries that are based on it like unexpected-react that you can read about more in depth here.

Sinon.JS

Sinon.JS ? Standalone test spies, stubs and mocks for JavaScript. Works with any unit testing framework.

Standalone test spies, stubs and mocks for JavaScript. Works with any unit testing framework.

sinonjs.org

Sinon has very powerful standalone test spies, stubs and mocks for JavaScript that works with any unit testing framework.

testdouble.js

testdouble.js ? A minimal test double library for TDD with JavaScript

github.com

testdouble is a less popular library that does what Sinon does, and claims to do it better, with a few differences in design, philosophy, and features that could make it useful in many cases. You can read about it here, here and here.

Wallaby

Wallaby ? Integrated Continuous Test Runner for JavaScript

Smart and fast integrated continuous test runner for JavaScript, TypeScript, CoffeeScript. It enables continuous?

wallabyjs.com

Wallaby is another tool worth mentioning. It is not free, but many users recommend buying it. It runs on your IDE (it supports all major ones) and runs tests that are relevant to your code changes and indicates if anything fails in real time alongside your code.

Image for post

Cucumber

cucumber/cucumber-js

cucumber-js ? Cucumber for JavaScript

github.com

Cucumber help with writing tests in BDD by dividing them between the acceptance criteria files using the Gherkin syntax and the tests that correspond to them.

Tests can be written in a variety of languages that are supported by the framework, including JS, which we are focusing on:

Many teams will find this syntax more convenient than TDD.

Choose Your Unit and Integration Tests Framework

The first choice you should probably make is which framework you want to use.

* you can?t go wrong with Jest. It has nice approval ratings, it?s very fast, clear and has many features in case you need to cover complex scenarios. If you want to ?just get started?, go with Jest.

* If you want a very flexible and extendable configuration, go with Mocha.

* If you are looking for simplicity go with Ava.

* If you want to be really low-level, go with tape.

Here is a list of the most prominent tools with some of their characteristics:

Jest

Jest ? Painless JavaScript Testing

? Painless JavaScript Testing

facebook.github.io

Jest is the testing framework created and maintained by Facebook. It spiked in popularity and became the most popular library throughout 2017 (Judging by State of Javascript 2017 vs 2018)

It is based on Jasmine which we will discuss later. Over time, Facebook replaced most of its functionality and added a lot of features on top of it.

Performance- First of all Jest is considered to be faster for big projects with many test files by implementing a clever parallel testing mechanism.
UI- Clear and convenient.
Ready-To-Go- Comes with assertions, spies, and mocks that are equivalent to libraries that do the same like Sinon. Libraries still can easily be used in case you need some unique features.
Globals- Like in Jasmine, it creates test globals by default so there is no need to require them. This can be considered bad since it makes your tests less flexible and less controllable, but in most cases it just makes your life easier:

Snapshot testing- jest-snapshot is developed and maintained by Facebook, although it can be used in almost any other framework as part of the framework?s integration of the tool or by using the right plugins.
Great modules mocking- Jest provides you with an easy way to mock heavy modules to improve testing speed. For example a service can be mocked to resolve a promise instead of making a network request.
Code coverage- Includes a powerful and fast built-in code coverage tool that is based on Istanbul.
Reliability- Since it has a huge community, and used in many very complex projects, is is considered very reliable.
Support- It is currently supported by all the major IDEs and tools.
Development- jest only updates the files updated, so tests are running very fast in watch mode.

jasmine

jasmine/jasmine

Simple JavaScript testing framework for browsers and node.js

github.com

Jasmine is the testing framework that Jest is based on. Why would you prefer Jasmine over Jest? It has been around for longer and it has a huge amount of articles, tools, and questions answered in various forums that were all created by the community over many years.

Also, Angular still suggests using it over Jest, although Jest is perfectly suitable to run Angular tests as well, and many people do it.

Ready-To-Go- Comes with everything you need to start testing.
Globals- Comes with all the important testing features in the global scope as well.
Community- It has been on the market since 2009 and gathered a vast amount of articles, suggestions and tools that are based on it.
Angular- Has widespread Angular support for all its versions and it is recommended in the official Angular documentation.

mocha

mochajs/mocha

simple, flexible, fun javascript test framework for node.js & the browser

github.com

Mocha is the most used library. Unlike Jasmine, it is used with third party assertions, mocking, and spying tools (usually Sinon and Chai).

This means Mocha is a little harder to set up and divided into more libraries but it is more flexible and open to extensions.

For example, if you want special assertion logic, you can fork Chai and replace only Chai with your own assertion library. This can also be done in Jasmine but in Mocha this change will be more clear and obvious.

Community- Has many plugins and extension to test unique scenarios.
Extensibility- Very extensible, to the point where plugins, extensions and libraries are designed only to run on top of it.
Globals- Creates test structure globals by default, but obviously not assertions, spies and mocks like Jasmine. some people are surprised by this seemingly inconsistency of globals.

AVA

avajs/ava

Futuristic JavaScript test runner

github.com

Ava is a minimalistic testing library that runs tests in parallel.

Ready-To-Go- Comes with everything you need to start testing (besides spying and dubbing that you can add easily). Uses the following syntax for test structure and assertions, and runs in Node.js:

Globals- As you can see above, it does not create any test globals, so you have more control over your tests.
Simplicity- Simple structure and assertions without a complex API while supporting many advanced features.
Development- Ava only updates the files updated so tests will run fast in watch mode.
Speed- Runs tests in parallel as separate Node.js processes.
Snapshot testing is supported as part of the framework.

tape

substack/tape

tape ? tap-producing test harness for node and browsers

github.com

Tape is the simplest of them all. It?s just a JS file you run with node with a very short and ?to-the-point? API.

Simplicity- Minimalistic structure and assertions without a complex API. Even more than Ava.
Globals- Does not create any test globals so you have more control over your tests.
No Shared State between tests- Tape discourages the use of functions like ?beforeEach? to ensure test modularity and maximum user control over the testsing cycle.
No CLI is needed- Tape will simply run anywhere JS can run.

Functional Testing Tools

First of all, as mentioned above, here and here you can find great articles about service providers that will host the machines where tests run and help you run these tests on different devices and browsers.

The tools for the purpose of functional testing differ very much from each other in their implementation, philosophy, and API, so it is strongly suggested to invest time in understanding the different solutions and testing them on your product.

* In short, if you want to ?just get started? with a simple to set-up cross-browser all-in-one tool, go with TestCafe.

* For a convenient UI, clear documentation, cool tools and overall fun all-in-one tool Functional Testing experience go with Cypress.io.

* If you prefer older and more time-proven tools, you can ?just get started? with Nightwatch.js.

* If you prefer older and more time-proven tools, with the maximum community support and flexability, WebdriverIO is the way to go.

* If you want the most reliable and Angular friendly solution, use Protractor.

selenium

SeleniumHQ/selenium

selenium ? A browser automation framework and ecosystem.

github.com

Selenium and tools that rely on it dominated the market of Functional Tests for years. It is not written specifically for testing and can control a browser for many purposes by exposing a driver that controls browsers using add-ins and browser extensions.

Node.js <=> WebDriver <=> FF/Chrome/IE/Safari drivers <=> browser

Selenium WebDriver can be accessed in many different ways and using a variety of programming languages, and with some tools even without any real programming.

The WebDriver can be imported into your testing framework and tests can be written as part of it:

The WebDriver itself might be sufficient for you and indeed some people suggest using it as is, but various libraries were created to extend it either by forking and altering it or by wrapping it.

And indeed wrapping the WebDriver might add redundant code and could make debugging harder, whereas forking it might diverge it from the very active ongoing development of the WebDriver.

Still, some people prefer to not use it directly. Let?s look at some of the libraries for selenium:

Protractor

angular/protractor

protractor ? E2E test framework for Angular apps

github.com

Protractor is a library that wraps Selenium and provides us with improved syntax and special built-in hooks for Angular.

Angular- Has special hooks, although it can successfully be used with other JS frameworks too. Angular official documentation suggests using this tool.
Error reporting- Good mechanism.
Support- TypeScript support is available and the library is operated and maintained by the huge Angular team.

WebdriverIO

WebdriverIO ? Selenium 2.0 bindings for NodeJS

A nodejs bindings implementation for selenium 2.0/webdriver.

webdriver.io

WebdriverIO has its own implementation of the selenium WebDriver.

Syntax- very easy and readable.
Flexible- A very simple, flexible, and extensible library.
Community- It has good support and enthusiastic developer community.
Applitools support, which is a visual regression library we will discuss later.

Nightwatch

Nightwatch.js

Write efficient and straightforward end-to-end tests in Node.js which run against a Selenium/WebDriver server.

nightwatchjs.org

Nightwatch has its own implementation of the selenium WebDriver. And provides its own testing framework with a test server, assertions, and tools.

Framework- Can be used with other frameworks too, but can be especially useful in case you want to run functional tests not as part of another framework.
Syntax- looks the easiest and the most readable.
Support- No typescript support and in general, this library seems to be slightly less supported than the others.

Appium

appium/appium

appium ? :iphone: Automation for iOS, Android, and Windows Apps.

github.com

Apium provides an API similar to Selenium for testing websites on a mobile device using the following tools:

iOS 9.3+: Apple?s XCUITest
Before iOS 9.3: Apple?s UIAutomation
Android 4.2+: Google?s UiAutomator/UiAutomator2
Android 2.3+: Google?s Instrumentation. (Instrumentation support is provided by bundling a separate project, Selendroid)
Windows Phone: Microsoft?s WinAppDriver

So if you use Selenium or Selenium based tools, you can also use Apium to test on mobile devices.

TestCafe

Automated browser testing for the modern web development stack | TestCafe

TestCafe is a simple and powerful framework for testing websites and apps. It allows you to easily create, maintain and?

devexpress.github.io

TestCafe is a great alternative to Selenium-Based tools. It was rewritten and open-sourced at the end of 2016.

TestCafe had also a paid version that offered non-programming testing tools. It has been deprecated and got replaced by the new TestCafe Studio.

TestCafe injects itself into the website as JavaScript scripts instead of controlling the browsers themselves like Selenium does. This allows it to run on any browser, including on mobile devices, and have full control over the JavaScript execution loop.

Fast to set up- Npm install and run your first test on any browser you want.
Cross Browser and Devices- Supports many browsers and devices and can be used with SauceLabs or BrowserStack which provide devices and browsers for your tests. This includes running of tests in Headless Chrome and Headless Firefox which will discussed later.
Parallel Testing- TestCafe can run your tests on several browser instances at once. This practice can significantly decrease your testing time.
Convenient Error Reporting
Own Ecosystem- TestCafe uses its own test structure. It could be quite convenient, especially because UI tests usually run separately from other tests, but some people dislike it.

Cypress

JavaScript End to End Testing Framework | Cypress.io

Fast, easy and reliable testing for anything that runs in a browser. Install Cypress in seconds and take the pain out?

www.cypress.io

Cypress is a direct competitor of TestCafe. They are doing relatively the same, which is injecting tests into a website, but they try to do it in a more modern, flexible and convenient way.

The difference between them is that Cypress.io runs itself in the browser and controls your tests from there where TestCafe runs in Node.js and controls the tests through a serialized communication with its injected script in the browser.

The library is relatively new (moved from closed beta to public beta at October 2017) but they already have many adopters and enthusiasts.

Parallel testing was introduced in version 3.10.
Documentation- Solid and clear.
Native access to all your application?s variables without serialization (TestCafe on the other hand turns objects to JSON, sends them to Node.js as text and then parses them back to objects).
Very convenient running and debugging tools- Easy debugging and logging of the test process.
Cross-browser Support- since version 4.0.
Some use-cases are missing but in constant development such as lack of HTML5 drag-n-drop.
Using Mocha as the test structure provider makes its use pretty standard and allow your functional tests to be built in the same structure as the rest of your tests.

Puppeteer

GoogleChrome/puppeteer

puppeteer ? Headless Chrome Node API

github.com

Puppeteer is a Node.js library, developed by Google. It provides a convenient Node.js API to control Chrome or Headless Chrome.

Headless Chrome is just a regular Chrome v59+ that is launched with the–headless flag. When Chrome is run in headless mode, it exposes an API to control it, and as said before, Puppeteer is the JavaScript tool that Google provides to control it.

Here it is worth mentioning that Firefox has also released their headless mode at the end of 2017.

Notice that different testing tools can also use Headless Chrome and Firefox. For example: TestCafe, Karma, Cypress.

Puppeteer is maintained by google and has a big community that uses and develops tools and wrappers around it.
Since it is native and uses the latest Chrome engine, it is very fast.
One major drawback of Headless Chrome (thus of Puppeteer as well) is that it doesn?t supports extensions like Flash and probably won?t in the near future.
Notice that this tool doesn?t come with build in testing system. Tools like mocha and chai can be used to create the necessary testing environment and structure around this library.

Playwright

microsoft/playwright

Playwright is a Node library to automate the Chromium, WebKit and Firefox browsers with a single API. It enables?

github.com

Playwright is a exactly like Puppeteer, but it is developed by Microsoft (launched at January 2020 by a team that originally developed Puppeteer) and strives to be cross-platform. It also has a slightly different API and new features like waiting for elements to be available before clicking on them.

It?s worth mentioning that since the library is pretty fresh, breaking changes may be made in the near future and some things might not work or be documented as expected at the moment.

Here is a nice comparison of the library with Pupetter, and here is an article about it with an example of how to use it.

PhantomJS

PhantoJs is a headless WebKit scriptable with JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG.

phantomjs.org

Phantom implements the chromium engine to create a controllable Chrome-like headless browser. It was a great tool to run in headless mode until Google announcement of ?Puppeteer?.

It?s main maintainer, Vitaliy Slobodin, no longer works on it, and its development was suspended and its repository archived.

Nightmare

segmentio/nightmare

nightmare ? A high-level browser automation library.

github.com

Nightmare is a Functional Testing library that offers a very simple test syntax.

It uses Electron which uses Chromium to control the browser?s behavior.

Doesn?t seem to be maintained recently. Probably because of the introduction of ?Puppeteer? as well which provides you with the same features out of the box.

CodeceptJS

Codeception/CodeceptJS

CodeceptJS ? Modern Era Acceptance Testing Framework for NodeJS

github.com

Like CucumberJS which was discussed above, Codecept provides another abstraction over different libraries? API?s to make your interactions with tests use a slightly different philosophy that focuses on user behavior.

Here is how it looks:

And here is the list of libraries that can be executed using this code. All discussed above.

WebDriverIO, Protractor, Nightmare, Appium, Puppeteer, and even the new Playwrigh.

If you believe this syntax is better for your needs, give it a shot.

Visual Regression Testing

The visual regression testing tools are consists roughly of the following:

Techniques and integrations to automate the browser or to run as part of Functional Testing tools discussed above, including in the CLI for CI.
Taking smart screenshots as images and as DOM snapshots.
Images and DOM comparison techniques to spot differences sometimes even using advanced AI.
UI for humans to approve, reject and improve the comparison mechanism to only show what?s relevant for the user.

There are a lot of tools of this type in the market, but it feels like this field still has a long way to do.

Also I noticed that paid tools in the visual regression testing category are much better than the free ones.

Applitools

Applitools Eyes

Applitools is an application visual management and AI-powered visual UI testing and monitoring software. Use our visual?

applitools.com

Easy to set up.
Uses AI to make the comparison technique and human input regarding differences and negation of false positives very powerful.
It can be integrated conveniently with many of the tools discussed above.
Has a free and paid flexible plans, including special pricing for startup companies and non-profits.

Percy

Continuous visual integration for web apps.

github.com

Easy to set up.
Uses smart comparison techniques.
Convenient human input regarding differences.
It can be integrated conveniently with many of the tools discussed above.
It has great integrations with great tools.
Has free and paid flexible plans.

Happo

Happo.io ? Cross-browser screenshot testing

Happo is a cross-browser screenshot testing service. Prevent visual regressions, visualize your UI changes, and keep a?

happo.io

Happo is a paid visual regression testing tool. It hooks into the CI to compare the visual appearance of UI components before and after a change.

Screenshots can be taken in different browsers and across different screen sizes to ensure consistent cross-browser and responsive styling of your application.

Paid with a free plan for open source projects.

LooksSame

gemini-testing/looks-same

Node.js library for comparing images. Contribute to gemini-testing/looks-same development by creating an account on?

github.com

Yandex created this library alongside with the now deprecated Gemini, that was a great simple-to-use visual regression testing tool.

Yandex now migrated to hermione that runs tests using WebdriverIO v4 and Mocha.js and uses LooksSame for visual regrations. It is more simple and limited than the paid tools mentioned above, but for simple websites it can be enough.

LooksSame can also be used on it?s own as long as you generate screenshots in any way you like.

BackstopJS

garris/BackstopJS

Catch CSS curve balls. Contribute to garris/BackstopJS development by creating an account on GitHub.

github.com

An open source visual regression utility that runs on Chrome Headless with Puppeteer and CI support.

AyeSpy

newsuk/AyeSpy

A visual regression testing tool.

github.com

An open source utility by the Times Tooling team at News UK.

Uses selenium docker to create visual regression tests on Chrome / Firefox.

reg-suit

reg-viz/reg-suit

recycle: Visual Regression Testing tool. Contribute to reg-viz/reg-suit development by creating an account on GitHub.

github.com

An open source library that compares images, generates reports and saves them on the cloud. Very convenient if you want to add visual regression tests to an existing functional test. Simply add steps of taking screenshots to your existing test flow and use it to compare these screenshots.

Differencify

NimaSoroush/differencify

Differencify is a library for visual regression testing ? NimaSoroush/differencify

github.com

Another open source Chrome Headless using Puppeteer testing tool with nice integrations with Jest snapshots. Can run in docker and generates convenient reports.

No Coding Functional Testing Tools

testim

Testim.io | Agile, Self-Healing, Autonomous Testing Solution

Testim uses machine learning automation testing to improve the testing around your software by integrating with all?

www.testim.io

Opens your application in a separate window and uses a browser extension to record your manual interactions with the application as test scenarios.

Uses machine learning to help you record and validate test scenarios. Cross Browser and has some nice integrations with many CI and Collaboration tools.

Has free and paid flexible plans.

Chromatic

https://www.chromaticqa.com/

Chromatic is a visual testing tool for Storybook made by Storybook maintainers.

Diff Detector renders the UI of each story and takes a visual snapshot. If anything changes, you are prompted to accept the visual changes locally and as part of the project?s pipelines.

Screener

Screener.io: Automated Visual Testing

Screener is a cloud-based automation platform for visual and functional UI testing. Test your web applications and UI?

screener.io

Lets you record your tests using a chrome extension, has in depth visual regression reporting. Has some nice integrations like with storybook different CI tools and BrowserStack and Sauce Labs.

Not free.

Other tools of this type (you are free to suggest more in the comments):

Screenster (not Screener that was mentioned above)
Chromatic
Ghost Inspector

Conclusion

We reviewed the most trending testing strategies and tools in the web development community and hopefully made it easier for you to test your sites.

In the end, the best decisions regarding application architecture today are made by understanding general patterns that are trending in the very active community of developers, and combining them with your own experience and the characteristics of your application.

Oh, and writing, and rewriting, and rewriting, and rewriting, and testing different solutions 🙂

Happy testings 🙂

Thanks 🙂