Understanding Node.js Async Flows: Parallel, Serial, Waterfall and Queues

Understanding Node.js Async Flows: Parallel, Serial, Waterfall and Queues

Image for post

Introduction

Promises in Javascript has been around for a long time now. It helped solve the problem of callback hell. But as soon as the requirements get complicated with control flows, promises start getting unmanageable and harder to work with. This is where async flows come to the rescue. In this blog, let?s talk about the various async flows which are used frequently rather than raw promises and callbacks.

Async Utility Module

Async is a utility module which provides straight-forward, powerful functions for working with asynchronous JavaScript. Although it is built on top of promises, it makes asynchronous code look and behave a little more like synchronous code, making it easier to read and maintain.

Async utility has a number of control flows. Let?s discuss the most popular ones and their use cases:

1. Parallel

When we have to run multiple tasks independent of each other without waiting until the previous task has completed, parallel comes into the picture.

Tasks: A collection of functions to run. It can be an array, an object or any iterable.

Callback: This is the callback where all the task results are passed and executed once all the task execution has completed.

In case an error is passed to a function?s callback, the main callback is immediately called with the error. Although parallel is about starting I/O tasks in parallel, it?s not about parallel execution since Javascript is single-threaded.

An example of Parallel is shared below :

2. Series

When we have to run multiple tasks which depend on the output of the previous task, series comes to our rescue.

Tasks: A collection of functions to run. It can be an array, an object or any iterable.

Callback: This is the callback where all the task results are passed and executed once all the task execution has completed.

The callback function receives an array of result objects when all the tasks have completed. If an error is encountered in any of the tasks, no more functions are run but the final callback is called with the error value.

An example of Series is shared below :

3. Waterfall

When we have to run multiple tasks which depend on the output of the previous task, Waterfall can be helpful.

Tasks: A collection of functions to run. It can be an array, an object or any iterable structure.

Callback: This is the callback where all the task results are passed and is executed once all the task execution has completed.

It will run one function at a time and pass the result of the previous function to the next one.

An example of Waterfall is shared below :

4. Queue

When we need to run a set of tasks asynchronously, a queue can be used. A queue object based on an asynchronous function can be created which is passed as a worker.

Task: Here, it takes two parameters, first ? the task to be performed and second ? the callback function.

Concurrency: It is the number of functions to be run in parallel.

async.queue returns a queue object that supports few properties:

  • Push: Adds tasks to the queue to be processed.
  • Drain: The drain function is called after the last task of the queue.
  • Unshift: Adds tasks in front of the queue.

An example of a Priority Queue is shared below :

5. Priority Queue

It is the same as the queue, the only difference being that a priority can be assigned to the tasks which are considered in ascending order.

Task: Here, it takes three parameters:

  • First ? The task to be performed.
  • Second ? Priority, it is a number that determines the sequence of execution. For an array of tasks, the priority remains the same for all of them.
  • Third ? Callback function.

The async.priorityQueue does not support ?unshift? property of the queue.

An example of a Priority Queue is shared below :

6. Race

It runs all the tasks in parallel, but as soon as any of the function completes its execution or passes an error to its callback, the main callback is immediately called.

Task: Here, it is a collection of functions to run. It is an array or any iterable.

Callback: The result of the first complete execution is passed. It may be the result or error.

An example of Race is shared below:

Combining Async Flows

In complex scenarios, the async flows like parallel and series can be combined and nested. This helps in achieving the expected output with the benefits of async utilities.

However, the only difference between Waterfall and Series async utility is that the final callback in series receives an array of results of all the task whereas, in Waterfall, the result object of the final task is received by the final callback.

Conclusion

Async Utilities has an upper hand over promises due to its concise and clean code, better error handling and easier debugging. It makes us realize how simple and easy asynchronous code can be without the syntactical mess of promises and callback hell.

*****************************************************************

This post was originally published on Velotio Blog.

Velotio Technologies is an outsourced software product development partner for technology startups and enterprises. We specialize in enterprise B2B and SaaS product development with a focus on artificial intelligence and machine learning, DevOps, and test engineering.

Interested in learning more about us? We would love to connect with you on ourWebsite, LinkedIn or Twitter.

*****************************************************************

25