Community Post

MongoDB and Express - When should the webserver start?

Ivan Sieder

First off - the code

The code used in this example is available in my repository on GitHub. Feel free, to check it out. It might be easier to follow the tutorial, if you have the code on your pc, so you can modify it and experiment around.

Why should I care, when my express webserver starts?

Imagine the following situation: You have a huge, distributed application running dozens of instances of one specific service (let's say, a service, which returns a list of posts from a user to keep it simple). Your service gets a few thousand requests per second and is also being consumed by external APIs.

For this example, imagine we are running those dozens of instances inside Docker containers.

You would usually have a code similar to this one (really simplified):

const express = require('express');
const MongoClient = require('mongodb').MongoClient;

const app = express();
let db;
const port = process.env.PORT || 3000;

app.get('/users', (req, res) => {
    db.collection('users').find({}).toArray((err, docs) => {
        if (err) throw err;

        res.status(200).json(docs);
    });
});

MongoClient.connect('mongodb://localhost:27017/mydb', (err, database) => {
    if (err) throw err;

    console.log('Connected to the database.');
    db = database;
});

app.listen(port, () => {
    console.log(`Webserver is online at http://localhost:${port}`);
});

The following happens in the code above:

  • All dependencies are being required (Express web framework and MongoDB driver)
  • An instance of express and an empty database-holding variable db is being created and the port for the web server is being defined (either from the environment variable PORT or if that variable is not present or not set, the port will be set to a default value of 3000).
  • Mongoclient.connect(/**/) initializes the connection to the MongoDB database (This happens asynchronously). Inside the callback (which will be executed, as sonn as we get an answer from MongoDB), we get a database object passed in (assuming, no error arose) which we assign to the db variable. Inside the callback we are also logging to the console, that we are now connected to the database).
  • Next in the code we define a basic Express route to retrive users from a MongoDB database using our previously assigned db variable (again, really simplified, no filters, projections etc.).
  • The last part starts the express webserver at the specified port and logs to the console, that the webserver is now running.

Copy the above code into a file (for example server.js), save it, open a command prompt in the same directory and start the application with node server.js.

The output to the console will most probably be in the following order:

Webserver is online at http://localhost:3000
Connected to the database

Why doesn't the "Connected to the database" message show up before the "Webserver is online at http://localhost:3000" one even if it comes first in the code?

Asynchronous Database Connection

The reason the webserver starts before the application is connected to the database is the asynchronous behaviour of the MongoClient library.

As soon as we ask the MongoClient to establish a connection to the database (with MongoClient.connect(/**/)), NodeJS sends the command to a background task - libuv, to be more precise - and proceeds with the rest of the code.

Therefore, the callback inside MongoClient.connect(/**/) will only be called, as soon as libuv returns, telling us, that the connection is established or the connection failed.

While NodeJS waits for the MongoDB database to respond, it executes the rest of the code in his execution path. This also includes starting the webserver.

Most probably after NodeJS has reached the end of the code, the connection has been established and libuv (remember, the background process) will tell NodeJS, that the connection is ready and NodeJS will then execute the callback inside the MongoClient.connect(/**/) function.

That's a simplified explanation, why the database connection won't be established before the webserver is started, even if it comes first in the code.

Why is this important at all?

You might ask yourself, why this is important at all, because establishing the connection is taking just a few seconds (if not only milliseconds).

Remember, that I told you our application is responding to a few thousand requests per second? That's the reason, why this is important.

With Docker it's possible to do updates of containers in a more fluent way (updating only a bunch of containers at a time), so we are never really offline. Let's imagine, that we have a load balancer in front of our containers, which checks, if our container responds to /api/heartbeat (a simple express endpoint, which returns an empty response, just to make sure that express is started) before adding it to it's load balancing endpoints.

If we start or update a container, which starts the webserver before a db connection is established, the container gets added to the load balancer, before a database connection is ready and if traffic is immediately being redirected to that container, it could return invalid/broken data, as our db object is undefined, until the connection is ready.

How can we fix that?

Fixing this issue is pretty simple. For this reason, it can and sould also be applied to small applications from the beginning.

Move the app.listen(/**/) part to the end of the Mongoclient.connect(/**/) block, as that's the place, where the connection is being established, the db variable is ready to be use and the webserver can safely be started.

The modified code now should look similar to this:

const express = require('express');
const MongoClient = require('mongodb').MongoClient;

const app = express();
let db;
const port = process.env.PORT || 3000;

app.get('/users', (req, res) => {
    db.collection('users').find({}).toArray((err, docs) => {
        if (err) throw err;

        res.status(200).json(docs);
    });
});

MongoClient.connect('mongodb://localhost:27017/mydb', (err, database) => {
    if (err) throw err;

    console.log('Connected to the database.');
    db = database;

    app.listen(port, () => {
        console.log(`Webserver is online at http://localhost:${port}`);
    });
});

Now, in our console, we will see the output in the expected order, which is first the connection to the database and then the webserver:

Connected to the database.
Webserver is online at http://localhost:3000

Ivan Sieder

NodeJS, Docker, MongoDB and Vue.js enthusiast. Writing about (hopefully) useful topics. For you, to learn