AWS Lambda and the Node.js event loop

• 25 min read

Lambda can freeze and thaw its execution context, which can impact Node.js event loop behavior.


One of the more surprising things I learned recently while working with AWS Lambda is how it interacts with the Node.js event loop.

Lambda is powered by a virtualization technology. And to optimize performance it can freeze and thaw the execution context of your code so it can be reused.

This will make your code run faster, but can impact the “expected” event loop behavior. We’ll explore this in detail. But before we dive in, lets quickly refresh the Node.js concurrency model.

If you’re already familiar with the event loop, you can jump straight to the AWS Lambda section.

Concurrency model

Node.js is single threaded and the event loop is the concurrency model that allows non-blocking I/O operations to be performed1.

How? Well, we’ll have to discuss the call stack and the task queue first.

Call stack

Function calls form a stack of frames, where each frame represents a single function call.

Every time a function is called, it’s pushed onto the stack (i.e. added to the stack). And when the function is done executing, it’s popped off the stack (i.e. removed from the stack).

The frames in a stack are popped off in LIFO order:

Call stack with 2 frames.
A call stack with 2 frames.

Each frame stores information about the invoked function. Like the arguments the function was called with and any variables defined inside the called function’s body.

When we execute the following code:

"use strict"

function work() {
  console.log("do work")
}

function main() {
  console.log("main start")
  work()
  console.log("main end")
}

main()

We can visualize the call stack over time like this:

Call stack progression over time for steps 1, 2 and 3.

Call stack progression over time for steps 1, 2 and 3.

  1. When the script starts executing, the call stack is empty.

  2. main() is called, and pushed onto the call stack:

"use strict"

function work() {
  console.log("do work")
}

function main() {
  console.log("main start")
  work()
  console.log("main end")
}

main() // Step 2
  1. While executing main, console.log("main start") is called, and pushed onto the call stack:
"use strict"

function work() {
  console.log("do work")
}

function main() {
  console.log("main start") // Step 3
  work()
  console.log("main end")
}

main()
Call stack progression over time for steps 4, 5 and 6.

Call stack progression over time for steps 4, 5 and 6.

  1. console.log executes, prints main start, and is popped off the call stack.

  2. main continues executing, calls work(), and is pushed onto the call stack:

"use strict"

function work() {
  console.log("do work")
}

function main() {
  console.log("main start")
  work() // Step 5
  console.log("main end")
}

main()
  1. While executing work, console.log("do work") is called, and pushed onto the call stack:
"use strict"

function work() {
  console.log("do work") // Step 6
}

function main() {
  console.log("main start")
  work()
  console.log("main end")
}

main()
Call stack progression over time for steps 7, 8 and 9.

Call stack progression over time for steps 7, 8 and 9.

  1. console.log executes, prints do work, and is popped off the call stack.

  2. work finishes executing, and is popped off the call stack.

  3. main continues executing, calls console.log("main end") and is pushed onto the call stack:

"use strict"

function work() {
  console.log("do work")
}

function main() {
  console.log("main start")
  work()
  console.log("main end") // Step 9
}

main()
Call stack progression over time for steps 10 and 11.
Call stack progression over time for steps 10 and 11.
  1. console.log executes, prints main end, and is popped off the call stack.

  2. main finishes executing, and is popped off the call stack. The call stack is empty again and the script finishes executing.

This code didn’t interact with any asynchronous (internal) APIs. But when it does (like when calling setTimeout(callback)) it makes use of the task queue.

Task queue

Any asynchronous work in the runtime is represented as a task in a queue, or in other words, a message queue.

Each message can be thought of as a function that will be called in FIFO order to handle said work. For example, the callback provided to the setTimeout or Promise API:

A queue with 2 tasks (i.e. messages).
A queue with 2 tasks (i.e. messages).

Additionally, each message is processed completely before any other message is processed. This means that whenever a function runs it can’t be interrupted. This behavior is called run-to-completion and makes it easier to reason about our JavaScript programs.

Messages get enqueued (i.e. added to the queue) and at some point messages will be dequeued (i.e. removed from the queue).

When? How? This is handled by the Event Loop.

Event loop

The event loop can be literally thought of as a loop that runs forever, and where every cycle is referred to as a tick.

On every tick the event loop will check if there’s any work in the task queue. If there is, it will execute the task (i.e. call a function), but only if the call stack is empty.

The event loop can be described with the following pseudo code2:

while (queue.waitForMessage()) {
  queue.processNextMessage()
}

To summarize:

  • When code executes, function calls are added to the call stack.
  • Whenever calls are made via asynchronous (internal) APIs (like setTimeout or Promise) the corresponding callbacks are eventually added to the task queue.
  • When the call stack is empty and the task queue contains one or more tasks, the event loop will remove a task on every tick and push it onto the call stack. The function will execute and this process will continue until all work is done.
The event loop can be visualized as a queue of tasks, which are picked up by a loop.
The event loop visualized.

With that covered, we can explore how the AWS Lambda execution environment interacts with the Node.js event loop.

AWS Lambda

AWS Lambda invokes a Lambda function via an exported handler function, e.g. exports.handler. When Lambda invokes this handler it calls it with 3 arguments:

handler(event, context, callback)

The callback argument may be used to return information to the caller and to signal that the handler function has completed, so Lambda may end it. For that reason you don’t have to call it explicitly. Meaning, if you don’t call it Lambda will call it for you3.

Baseline

From here on we’ll use a simple script as a “baseline” to reason about the event loop behavior. Create a file called timeout.js with the following contents:

"use strict"

function timeout(ms) {
  console.log("timeout start")

  return new Promise((resolve) => {
    setTimeout(() => {
      console.log(`timeout cb fired after ${ms} ms`)
      resolve()
    }, ms)
  })
}

async function main() {
  console.log("main start")
  timeout(5e3)
  console.log("main end")
}

main()

When we execute this script locally (not via Lambda) with node timeout.js, the following will print:

main start
timeout start
main end
timeout cb fired after 5000 ms

The last message takes 5 seconds to print, but the script does not stop executing before it does.

What happens in Lambda, stays in Lambda

Now lets modify the code from timeout.js so it’s compatible with Lambda:

"use strict"

function timeout(ms) {
  console.log("timeout start")

  return new Promise((resolve) => {
    setTimeout(() => {
      console.log(`timeout cb fired after ${ms} ms`)
      resolve()
    }, ms)
  })
}

async function main() {
  console.log("main start")
  timeout(5e3)
  console.log("main end")
}

exports.handler = main // This change allows Lambda to run `main`

You can create a new function in the AWS Lambda console and paste in the code from above. Run it, sit back and enjoy:

Lambda console first run. The log message from the timeout callback is not printed.

Lambda console first run. The log message from the timeout callback is not printed.

Wait, what? Lambda just ended the handler function without printing the last message timeout cb fired after 5000 ms. Lets run it again:

Lambda console second run. The log message of the timeout callback from the previous run is printed first.

Lambda console second run. The log message of the timeout callback from the previous run is printed first.

It now prints timeout cb fired after 5000 ms first and then the other ones! So what’s going on here?

AWS Lambda execution model

AWS Lambda takes care of provisioning and managing resources needed to run your functions. When a Lambda function is invoked, an execution context is created for you based on the configuration you provide. The execution context is a temporary runtime environment that initializes any external dependencies of your Lambda function.

After a Lambda function is called, Lambda maintains the execution context for some time in anticipation of another invocation of the Lambda function (for performance benefits). It freezes the execution context after a Lambda function completes and may choose to reuse (thaw) the same execution context when the Lambda function is called again (but it doesn’t have to).

In the AWS documentation we can find the following regarding this subject:

Background processes or callbacks initiated by your Lambda function that did not complete when the function ended resume if AWS Lambda chooses to reuse the Execution Context.

As well as this somewhat hidden message:

When the callback is called (explicitly or implicitly), AWS Lambda continues the Lambda function invocation until the event loop is empty.

Looking further, there’s some documentation about the context object. Specifically about a property called callbackWaitsForEmptyEventLoop. This is what it does:

The default value is true. This property is useful only to modify the default behavior of the callback. By default, the callback will wait until the event loop is empty before freezing the process and returning the results to the caller.

Okay, so with this information we can make sense of what happened when we executed the code in timeout.js before. Lets break it down and go over it step by step:

Event loop visualized when executing step 1 of the timeout js script.
  1. Lambda starts executing the code in timeout.js. The call stack is empty.
Event loop visualized when executing step 2 of the timeout js script.
  1. main is called, and pushed onto to the call stack:
"use strict"

function timeout(ms) {
  console.log("timeout start")

  return new Promise((resolve) => {
    setTimeout(() => {
      console.log(`timeout cb fired after ${ms} ms`)
      resolve()
    }, ms)
  })
}

async function main() {
  console.log("main start")
  timeout(5e3)
  console.log("main end")
}

exports.handler = main // Step 2
Event loop visualized when executing step 3 of the timeout js script.
  1. While executing main, console.log("main start") is called, and pushed onto the call stack:
"use strict"

function timeout(ms) {
  console.log("timeout start")

  return new Promise((resolve) => {
    setTimeout(() => {
      console.log(`timeout cb fired after ${ms} ms`)
      resolve()
    }, ms)
  })
}

async function main() {
  console.log("main start") // Step 3
  timeout(5e3)
  console.log("main end")
}

exports.handler = main
Event loop visualized when executing step 4 of the timeout js script.
  1. console.log executes, prints main start, and is popped off the call stack.
Event loop visualized when executing step 5 of the timeout js script.
  1. main continues executing, calls timeout(5e3), and is pushed onto the call stack:
"use strict"

function timeout(ms) {
  console.log("timeout start")

  return new Promise((resolve) => {
    setTimeout(() => {
      console.log(`timeout cb fired after ${ms} ms`)
      resolve()
    }, ms)
  })
}

async function main() {
  console.log("main start")
  timeout(5e3) // Step 5
  console.log("main end")
}

exports.handler = main
Event loop visualized when executing step 6 of the timeout js script.
  1. While executing timeout, console.log("timeout start") is called, and pushed onto the call stack:
"use strict"

function timeout(ms) {
  console.log("timeout start") // Step 6

  return new Promise((resolve) => {
    setTimeout(() => {
      console.log(`timeout cb fired after ${ms} ms`)
      resolve()
    }, ms)
  })
}

async function main() {
  console.log("main start")
  timeout(5e3)
  console.log("main end")
}

exports.handler = main
Event loop visualized when executing step 7 of the timeout js script.
  1. console.log executes, prints timeout start, and is popped off the call stack.
Event loop visualized when executing step 8 of the timeout js script.
  1. timeout continues executing, calls new Promise(callback), and is pushed onto the call stack:
"use strict"

function timeout(ms) {
  console.log("timeout start")

  // Step 8
  return new Promise((resolve) => {
    setTimeout(() => {
      console.log(`timeout cb fired after ${ms} ms`)
      resolve()
    }, ms)
  })
}

async function main() {
  console.log("main start")
  timeout(5e3)
  console.log("main end")
}

exports.handler = main
Event loop visualized when executing step 9 of the timeout js script.
  1. While new Promise(callback) executes, it interacts with the Promise API and passes the provided callback to it. The Promise API sends the callback to the task queue and now must wait until the call stack is empty before it can execute.
Event loop visualized when executing step 10 of the timeout js script.
  1. new Promise finishes executing, and is popped of the call stack.
Event loop visualized when executing step 11 of the timeout js script.
  1. timeout finishes executing, and is popped off the call stack.
Event loop visualized when executing step 12 of the timeout js script.
  1. main continues executing, calls console.log("main end"), and is pushed onto the call stack:
"use strict"

function timeout(ms) {
  console.log("timeout start")

  return new Promise((resolve) => {
    setTimeout(() => {
      console.log(`timeout cb fired after ${ms} ms`)
      resolve()
    }, ms)
  })
}

async function main() {
  console.log("main start")
  timeout(5e3)
  console.log("main end") // Step 12
}

exports.handler = main
Event loop visualized when executing step 13 of the timeout js script.
  1. console.log executes, prints main end, and is popped off the call stack.
Event loop visualized when executing step 14 of the timeout js script.
  1. main finishes executing, and is popped off the call stack. The call stack is empty.
Event loop visualized when executing step 15 of the timeout js script.
  1. The Promise callback (step 9) can now be scheduled by the event loop, and is pushed onto the call stack.
Event loop visualized when executing step 16 of the timeout js script.
  1. The Promise callback executes, calls setTimeout(callback, timeout), and is pushed onto the call stack:
"use strict"

function timeout(ms) {
  console.log("timeout start")

  return new Promise((resolve) => {
    // Step 16.
    setTimeout(() => {
      console.log(`timeout cb fired after ${ms} ms`)
      resolve()
    }, ms)
  })
}

async function main() {
  console.log("main start")
  timeout(5e3)
  console.log("main end")
}

exports.handler = main
Event loop visualized when executing step 17 of the timeout js script.
  1. While setTimeout(callback, timeout) executes, it interacts with the setTimeout API and passes the corresponding callback and timeout to it.
Event loop visualized when executing step 18 of the timeout js script.
  1. setTimeout(callback, timeout) finishes executing and is popped of the call stack. At the same time the setTimeout API starts counting down the timeout, to schedule the callback function in the future.
Event loop visualized when executing step 18 of the timeout js script.
  1. The Promise callback finishes executing and is popped off the call stack. The call stack is empty again.

At this point the call stack and task queue are both empty. At the same time a timeout is counting down (5 seconds), but the corresponding timeout callback has not been scheduled yet. As far as Lambda is concerned, the event loop is empty. So it will freeze the process and return results to the caller!

The interesting part here is that Lambda doesn’t immediately destroy its execution context. Because if we wait for +5 seconds and run the Lambda again (like in the second run) we see the console message printed from the setTimeout callback first.

This happens because after the Lambda stopped executing, the execution context was still around. And after waiting for +5 seconds, the setTimeout API sent the corresponding callback to the task queue:

The setTimeout callback is sent to the task queue after 5 seconds.

The setTimeout callback is sent to the task queue after 5 seconds.

When we execute the Lambda again (second run), the call stack is empty with a message in the task queue, which can immediately be scheduled by the event loop:

The setTimeout callback is scheduled by the event loop.

The setTimeout callback is scheduled by the event loop.

This results in timeout cb fired after 5000 ms being printed first, because it executed before any of the code in our Lambda function:

The setTimeout callback is executed before any other code in the Lambda.

The setTimeout callback is executed before any other code in the Lambda.

Doing it right

Obviously this is undesired behavior and you should not write your code in the same way we wrote the code in timeout.js.

Like stated in the AWS documentation, we need to make sure to complete processing all callbacks before our handler exits:

You should make sure any background processes or callbacks (in case of Node.js) in your code are complete before the code exits.

Therefore we’ll make the following change to the code in timeout.js:

- timeout(5e3);
+ await timeout(5e3);

This change makes sure the handler function does not stop executing until the timeout function finishes:

"use strict"

function timeout(ms) {
  console.log("timeout start")

  return new Promise((resolve) => {
    setTimeout(() => {
      console.log(`timeout cb fired after ${ms} ms`)
      resolve()
    }, ms)
  })
}

async function main() {
  console.log("main start")
  await timeout(5e3) // Use await
  console.log("main end")
}

exports.handler = main

When we run our code with this change, all is well now:

Lambda console third run. Awaiting the timeout prints the log messages in expected order.

Lambda console third run. Awaiting the timeout prints the log messages in expected order.

In closing

I intentionally left out some details about the the task queue. There are actually two task queues! One for macrotasks (e.g. setTimeout) and one for microtasks (e.g. Promise).

According to the spec, one macrotask should get processed per tick. And after it finishes, all microtasks will be processed within the same tick. While these microtasks are processed they can enqueue more microtasks, which will all be executed in the same tick.

For more information see this article from RisingStack where they go more into detail.

This post was originally published on Medium.

Footnotes

  1. The event loop is what allows Node.js to perform non-blocking I/O operations (despite the fact that JavaScript is single-threaded) by offloading operations to the system kernel whenever possible.

  2. Taken from MDN.

  3. When using Node.js version 8.10 or above, you may also return a Promise instead of using the callback function. In that case you can also make your handler async, because async functions return a Promise.

Webmentions

Loading...


Thanks for reading! If you have any remarks or ideas on how to improve this post, please let me know on GitHub .

Post last updated .