Building a Slack Clone in Meteor.js (Part 5): Meteor Deployment

Free Course

Build Your First Node.js Website

Node is a powerful tool to get JavaScript on the server. Use Node to build a great website.

This is the fifth, and last, of a five-part series on building a Slack clone using Meteor. The aim of these tutorials are not just for you to blindly follow instructions, but it’s our hope that you’ll understand the thought process and reasoning behind the architecture.

So far, we have been working on our local machine, chatting to ourselves. We need to deploy it on a server so other people can chat with us! So in the last article of the series, we’ll focus on how deploy our awesome application for the world to see.

You can host your application on Meteor’s own infrastructure using meteor deploy, or on your own infrastructure.

meteor deploy

If you’ve tried out our demo you’d have noticed the URL is something.meteor.com – on the meteor.com domain. This is because Meteor provides their own infrastructure to us for free, allowing us to deploy quickly without having the bother with setting up servers, configuring databases, configuring email settings etc.


    $ meteor deploy d4nyll-slack.meteor.com
    Deploying to d4nyll-slack.meteor.com.         
    Now serving at http://d4nyll-slack.meteor.com 

Deployment requires you to have a Developer account. If you don’t have one, it will prompt you to create one. If you are not logged into the Meteor tool, login using meteor login.

It’ll take some time while Meteor minify and upload your files. But once it’s done, you can now access it on http://d4nyll-slack.meteor.com

Once you’ve deployed on that subdomain using your Developer Account, no one else can deploy to that domain.


    $ meteor deploy scotch-slack1.meteor.com
    Sorry, that site belongs to a different user. 
    You are currently logged in as d4nyll.        
    
    Either have the site owner use 'meteor authorized --add' to add you as
    an authorized developer for the site, or switch to an authorized account
    with 'meteor login'.

To see if the name is taken, just go to the URL and if you see the following screen, you’re in luck!

Screen saying there is no site deployed at this address

Own Domain

Meteor also allows you to deploy the application on their server, but make it reachable through a custom domain. For example, if I want to have http://slack.danyll.com to house my Slack clone.

First, I’d need to go and edit my DNS settings so slack.danyll.com points to origin.meteor.com.

I’m using DigitalOcean to host my sites, and they have a handy interface to change my DNS settings, so I will be using that. You should find something similar with your own hosting provider.

cname-record

Notice the period (.) in the ‘hostname’. That is very important – it signifies the root DNS server.

After that’s done, your zone file should have a new entry:


    slack.danyll.com. 1800 IN CNAME origin.meteor.com.

You might need to wait a short time before trying it out as the DNS settings need some time to propagate. So in the mean time, let’s deploy our application on Meteor’s infrastructure, but using our own domain name.


    $ meteor deploy slack.danyll.com
    Deploying to slack.danyll.com.                
    Now serving at http://slack.danyll.com

Fantastic! If you go to http://slack.danyll.com, you can actually see it being deployed.

Updating

If you’ve made an update you can simply deploy it again by running the same command. The database will be kept.


    $ meteor deploy d4nyll-slack.meteor.com

To delete your application, just pass the --delete flag.


    $ meteor deploy --delete d4nyll-slack.meteor.com

meteor deploy is part of the meteor tool commands, make sure you have that installed and are logged in to your Developer Account

Development vs Production

Run the application on your local environment and take a look at the one deployed. They look the same, but look under the hood we find a very different story!

Minification and Concatenation

Open up the developer’s tools and see what’s inside the <head>.

development-vs-production

In our localhost:3000 application, the <head> is filled with many files. In fact, all the files from pacakges and our own application can be found there individually. This is because we are in development mode.

When we deploy our application using meteor deploy, the environment is automatically set to production mode. One of the effects of this is that files are minified and concatenated together. So we get only one CSS stylesheet, and one JavaScript file, and both are minified.

When we are deploying the application on our own infrastructure, it’s important we remember to put it in production mode.

Memory

Background processes that enables features like Hot Code Push and resolving package dependencies takes up a lot of memory, and you don’t need this in production. Running on development mode when the application is live can lead to Out-Of-Memeory (OOM) errors.

Own Infrastructure

While Meteor provides free hosting, it’s not recommened to use it for live applications, since we have no control over the server or infrastructure. It’s there as a free service so there are no guarantees.

That’s why for production applications, we’d need to deploy on your own servers.

meteor build

The most no-nonsense way to deploy a Meteor application is using the meteor build command. This will build from the files in the application, bundle it into a tarball (.tar.gz), which you can extract from and run.

Let’s do that now. Inside the root directory of your Meteor application, run:


    $ meteor build .

You can now find a file named src.tar.gz inside the root application directory. This is the file we’d need to upload to our remote machine. So the next step is to set up Virtual Private Hosting (VPS).

Setting Up VPS

Setting up a server and securing it is an entire topic on its own, so we won’t go too in-depth into it, but just enough so you can get it running.

Again, I will be using Digital Ocean, but you can use your own hosting provider. Just to level the playing field, there are other VPS providers out there – Linode, Amazon EC2, Modulus (which is a PaaS, like Heroku), and we will also take a look at Galaxy – “the managed cloud platform for deploying Meteor apps.”

Let’s create a new server (which are called ‘Droplets’ in Digital Ocean).

new-droplet

We have included our SSH key in here so we can access it using SSH from my machine. If you don’t have one, generate one now (press for everything)


    $ ssh-keygen -t rsa

You can now find your key at ~/.ssh/id_rsa.pub. Copy it in its entirety into the box.

Click ‘Create Droplet’ to continue.

server-details

From this screen, we know that our server can be reached via the IP address 128.199.170.132. Since it’s our first time on this freshly installed server, we will ssh into it as root.


    $ ssh root@128.199.170.132
    The authenticity of host '128.199.170.132 (128.199.170.132)' can't be established.
    ECDSA key fingerprint is 96:46:56:e5:33:71:45:12:f3:bd:b1:0e:65:e8:fd:03.
    Are you sure you want to continue connecting (yes/no)? yes
    Warning: Permanently added '128.199.170.132' (ECDSA) to the list of known hosts.
    Welcome to Ubuntu 14.04.2 LTS (GNU/Linux 3.13.0-52-generic x86_64)
    
     * Documentation:  https://help.ubuntu.com/
    
      System information as of Sun Jun  7 09:10:16 EDT 2015
    
      System load: 0.0               Memory usage: 9%   Processes:       51
      Usage of /:  7.5% of 19.56GB   Swap usage:   0%   Users logged in: 0
    
      Graph this data and manage this system at:
        https://landscape.canonical.com/
    
    root@Production:~# 
Creating New User

For security reasons, we don’t use our root user. Instead we create a new user with root priviledges.


    # adduser d4nyll

After filling in some details about the user, assign it to the sudo group


    # gpasswd -a d4nyll sudo
    Adding user d4nyll to group sudo

Now if we exit, we can login again as the user d4nyll.


    $ ssh d4nyll@128.199.170.132

From here on, if you get an permission-related error while running the commands, try prepending the command with sudo, which will run the command as the root user.

Dependencies

We have our application code ready; we have our server set up. Next we need to deploy it onto our server. To follow along, you’d need to have to get access to a server. I will be using Ubuntu 14.04 in this tutorial.

Ubuntu 15.04 was released but since “MongoDB only provides packages for long-term support Ubuntu releases”, it’s best we stick to 14.04.

There are tools that we need to set up to serve our application. Here’s the stack we will be using:

  • nginx as a reverse proxy web server which passes requests to our application
  • mongodb
  • pm2 – A production process manager that keeps an eye on our application, and restart it if it crashes. There are alternatives – forever, supervisor and Upstart for Ubuntu, which you can explore in your own time.
Installing Dependencies

Now let’s install all the software we need. First, nginx


    $ apt-get install nginx

Next up is Mongo:


    $ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10
    $ echo "deb http://repo.mongodb.org/apt/ubuntu "$(lsb_release -sc)"/mongodb-org/3.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.0.list
    $ sudo apt-get update
    $ sudo apt-get install -y mongodb-org

You can find more in-depth explanation of the purpose of each step on this official Guide.

Alternatively, you can use apt-get install mongodb-server to install MongoDB, but using the official releases are generally more up-to-date. You cannot run both the Ubuntu-package and the official versions concurrently, so I’d advice with sticking with the official version.

And of course – Node


    $ curl -sL https://deb.nodesource.com/setup_0.12 | sudo bash -
    $ sudo apt-get install -y nodejs

Note on Reverse Proxy

We will be using nginx as a reverse proxy. This means our Meteor application will be running on port 3000, nginx will take in requests and pass it to the Meteor application; our application then replies with a response, which will be sent to the client through nginx.

The reason why it’s called a reverse proxy is because the client doesn’t know that it’s not in fact nginx that is providing the responde, but in fact our Meteor application. For all they know, nginx is serving the files to them; when in fact, all nginx is doing is relay the message.

Contrast this with a forward proxy, or what most of you will think of when the term proxy is used. In a forward proxy, the client access the server via another server, and the server has no idea who the final recipient is.

Running Our Application

MongoDB should already be running. If it’s not run

$ sudo service mongod start

First, we need to upload our tarball onto the server, we will use good old sftp for this.


    $ sftp -P 3825 d4nyll@128.199.170.132 
    Connected to 128.199.170.132.
    sftp> put src.tar.gz 
    Uploading src.tar.gz to /home/d4nyll/src.tar.gz
    src.tar.gz                             100% 2330KB 166.4KB/s   00:14

Then move the src.tar.gz file to the location where we put our application files. For me (and it is very much a personal preference), I put my files under /srv/www/.


    $ mv src.tar.gz /srv/www/
    $ cd /srv/www/

Unpack the files:


    $ tar -zvxf src.tar.gz
    $ ls -al
    total 2344
    drwxr-xr-x 3 root   root      4096 Jun  7 11:30 .
    drwxr-xr-x 3 root   root      4096 Jun  7 11:21 ..
    drwxr-xr-x 4 d4nyll d4nyll    4096 Jun  7 10:53 bundle
    -rw-rw-r-- 1 d4nyll d4nyll 2385695 Jun  7 11:24 src.tar.gz

We have generated a bundle directory containing all the files we need to run our application. Feel free to rename it however you want, especially when you have many applications on the same server. We can now delete our tarball and see what’s inside the bundle.


    $ rm src.tar.gz
    $ cd bundle
    $ ls -al
    total 28
    drwxr-xr-x 4 d4nyll d4nyll 4096 Jun  7 10:53 .
    drwxr-xr-x 3 root   root   4096 Jun  7 11:34 ..
    -r--r--r-- 1 d4nyll d4nyll  485 Jun  7 10:53 main.js
    drwxr-xr-x 4 d4nyll d4nyll 4096 Jun  7 10:53 programs
    -r--r--r-- 1 d4nyll d4nyll  543 Jun  7 10:53 README
    drwxr-xr-x 2 d4nyll d4nyll 4096 Jun  7 10:53 server
    -r--r--r-- 1 d4nyll d4nyll  392 Jun  7 10:53 star.json

The important file here is the main.js file, which we can run using


    $ node main.js

But when we do, it comes up with the following error.


    module.js:338
        throw err;
              ^
    Error: Cannot find module 'fibers'
        at Function.Module._resolveFilename (module.js:336:15)
        at Function.Module._load (module.js:278:25)
        at Module.require (module.js:365:17)
        at require (module.js:384:17)
        at Object.<anonymous> (/srv/www/bundle/programs/server/boot.js:1:75)
        at Module._compile (module.js:460:26)
        at Object.Module._extensions..js (module.js:478:10)
        at Module.load (module.js:355:32)
        at Function.Module._load (module.js:310:12)
        at Module.require (module.js:365:17)

We actually need to go into the /programs/server/ and install the packages there. This requires g++ and make, which you can install along with other useful tools as part of the build-essential package.


    $ cd programs/server/
    $ apt-get install build-essential
    $ npm install

Now if we run node main.js again, we get another error:


    /srv/www/bundle/programs/server/node_modules/fibers/future.js:245
                            throw(ex);
                                  ^
    Error: Must pass options.rootUrl or set ROOT_URL in the server environment
        at Object.Meteor.absoluteUrl (packages/meteor/url_common.js:21:1)
        at Object.WebAppInternals.generateBoilerplate (packages/webapp/webapp_server.js:543:1)
        at Object.main (packages/webapp/webapp_server.js:739:1)
        at /srv/www/bundle/programs/server/boot.js:255:27
Environment Variables

This is because there are environment variables we need to set up – namely ROOT_URL and MONGO_URL. (If you’re using the email package, you’d also need to set up a MAIL_URL environment variable)

ROOT_URL

The ROOT_URL environment variable is used by some packages, and should be set to the URL that the client will enter to reach your application. In our case, it would be http://128.199.170.132. If you have your own domain name, it’d be http://mydomain.com.

MONGO_URL

The MONGO_URL environment variable is a little more self-explanatory, and needs to be the address for your MongoDB database.

Logged in as the user who will be running our Meteor application, edit / create a ~/.bash_profile file and export these environment variables:

~/.bash_profile


    export ROOT_URL=http://128.199.170.132
    export MONGO_URL=mongodb://localhost:27017/myapp

If you now run echo $ROOT_URL you won’t see the newly exported variables because these settings weren’t there when this terminal session started. To make these settings come into effect for the current session, run


    $ source ~/.bash_profile

And now the environment variables are in effect.

Now let’s try running our application again:


    $ node /srv/www/bundle/main.js

This time no errors.

Keeping Your Application Running

As mentioned earlier, we will be using pm2 to monitor our application and keep it running. Let’s install it globally (using the -g flag), run


    $ npm install -g pm2

And now, instead of running node main.js, we simply run


    $ pm2 start main.js

Run pm2 list to see a list of applications, their uptime, and the number of times it’s been restarted.


    ┌──────────┬────┬──────┬──────┬────────┬─────────┬────────┬─────────────┬──────────┐
    │ App name │ id │ mode │ pid  │ status │ restart │ uptime │ memory      │ watching │
    ├──────────┼────┼──────┼──────┼────────┼─────────┼────────┼─────────────┼──────────┤
    │ main     │ 0  │ fork │ 8268 │ online │ 0       │ 8m     │ 48.844 MB   │ disabled │
    └──────────┴────┴──────┴──────┴────────┴─────────┴────────┴─────────────┴──────────┘

You can also get the real-time status of the application, specifically the memory usage, by running pm2 monit.

With pm2, you’d need to environment variales inside a file instead. Convention calls it process.json.


    {
      "apps": [
        {
          "name": "app-name",
          "script": "./main.js",
          "log_date_format": "YYYY-MM-DD",
          "exec_mode": "fork_mode",
          "env": {
            "PORT": 3000,
            "MONGO_URL": "mongodb://127.0.0.1:27017/meteor",
            "ROOT_URL": "http://128.199.170.132/",
            "BIND_IP": "127.0.0.1"
          }
        }
      ]
    }

And then run pm2 start process.json

Keeping pm2 running

pm2 can only keep our application running if it is itself running. We need to make sure pm2 is automatically restarted after a server restart. To do this, pm2 actually provides the pm2 startup command, which in turns output a command you need to run to set it up.


    $ pm2 startup
    [PM2] Spawning PM2 daemon
    [PM2] PM2 Successfully daemonized
    [PM2] You have to run this command as root. Execute the following command:
          sudo env PATH=$PATH:/usr/bin pm2 startup linux -u d4nyll
    
    $ sudo env PATH=$PATH:/usr/bin pm2 startup linux -u d4nyll
    [PM2] Generating system init script in /etc/init.d/pm2-init.sh
    [PM2] Making script booting at startup...
    [PM2] -linux- Using the command:
          su -c "chmod +x /etc/init.d/pm2-init.sh && update-rc.d pm2-init.sh defaults"
     Adding system startup for /etc/init.d/pm2-init.sh ...
       /etc/rc0.d/K20pm2-init.sh -> ../init.d/pm2-init.sh
       /etc/rc1.d/K20pm2-init.sh -> ../init.d/pm2-init.sh
       /etc/rc6.d/K20pm2-init.sh -> ../init.d/pm2-init.sh
       /etc/rc2.d/S20pm2-init.sh -> ../init.d/pm2-init.sh
       /etc/rc3.d/S20pm2-init.sh -> ../init.d/pm2-init.sh
       /etc/rc4.d/S20pm2-init.sh -> ../init.d/pm2-init.sh
       /etc/rc5.d/S20pm2-init.sh -> ../init.d/pm2-init.sh
    [PM2] Done.

And now pm2 and our Meteor application will be kept alive.

Web Server Set-up

We have our application and it’s running. The last thing we need to do is to point our web server at it. Remember we are using nginx as a reverse proxy, which means it’s not server the files itself, but rather simply acting as a middle-man between the client and our Meteor application.

We must configure nginx first.

Configuration

Configuration files for services on a Linux machine are usually stored inside the /etc/ directory; nginx is no exception.

If you are unfamiliar with conventional Ubuntu or Linux file directories, take at look at this blog post I wrote last year.


    $ cd /etc/nginx/
    $ ll
    total 72
    drwxr-xr-x  5 root root 4096 Jun  7 09:48 ./
    drwxr-xr-x 92 root root 4096 Jun  7 09:56 ../
    drwxr-xr-x  2 root root 4096 Feb 11 11:26 conf.d/
    -rw-r--r--  1 root root  911 Mar  4  2014 fastcgi_params
    -rw-r--r--  1 root root 2258 Mar  4  2014 koi-utf
    -rw-r--r--  1 root root 1805 Mar  4  2014 koi-win
    -rw-r--r--  1 root root 2085 Mar  4  2014 mime.types
    -rw-r--r--  1 root root 5287 Mar  4  2014 naxsi_core.rules
    -rw-r--r--  1 root root  287 Mar  4  2014 naxsi.rules
    -rw-r--r--  1 root root  222 Mar  4  2014 naxsi-ui.conf.1.4.1
    -rw-r--r--  1 root root 1601 Mar  4  2014 nginx.conf
    -rw-r--r--  1 root root  180 Mar  4  2014 proxy_params
    -rw-r--r--  1 root root  465 Mar  4  2014 scgi_params
    drwxr-xr-x  2 root root 4096 Jun  7 09:48 sites-available/
    drwxr-xr-x  2 root root 4096 Jun  7 09:48 sites-enabled/
    -rw-r--r--  1 root root  532 Mar  4  2014 uwsgi_params
    -rw-r--r--  1 root root 3071 Mar  4  2014 win-utf

nginx can serve multiple applications on the same server in different virtual hosts, these different applications can be reached via different domain names. So we should create a new virtual host for each application we are running on our server.

Open up nginx.conf, which is the main configuration file for nginx. At the bottom of the http {} block, you will find these lines:


    ##
    # Virtual Host Configs
    ##
    
    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;

This means nginx will look inside these directories for configuration for each virtual host.

We will use the /etc/nginx/sites-enabled/ to house all our virtual host configuration files.

Actually, we will create all our virtual host configuration files inside the sites-available directory and then symbolic link them to the sites-enabled directory. This means we can have many configurations available, but only enable the onces we want to be public.


    $ cd /etc/nginx/sites-available
    $ ls -al
    total 12
    drwxr-xr-x 2 root root 4096 Jun  7 09:48 .
    drwxr-xr-x 5 root root 4096 Jun  7 09:48 ..
    -rw-r--r-- 1 root root 2593 Mar  4  2014 default

You’ll find there is a default configuration file prepared for us already. Let’s copy that and modify it.


    $ cp default myapp

If we take away all the comments, we are left with this:


    server {
      listen 80 default_server;
      listen [::]:80 default_server;
    
      root /var/www/html;
      index index.html index.htm;
    
      server_name _;
    
      location / {
        try_files $uri $uri/ =404;
      }
    }

The listen lines tells us what port the server is listening out for. The default_server tells us this is the virtual host that requests will default to if the domain name doesn’t match any defined in the server_name property in any configuration files.

The root tells nginx where to look for files, and index index.html index.htm defines which file it should look for if no filename is specified, and the location block simply say “Try to serve the requests as a file, then as a directory, and if none is found, return with status code 404

But that doesn’t matter to us, since we don’t want nginx to serve files, we want it to be a reverse proxy. So replace the configuration with below:


    server {
      listen 80 default_server;
      listen [::]:80 default_server;
    
      location / {
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host $http_host;
            proxy_set_header X-NginX-Proxy true;
            proxy_pass http://127.0.0.1:3000/;
            proxy_redirect off;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
            proxy_set_header   X-Forwarded-Proto $scheme;
        }
    }

At the moment, if you go to http://128.199.170.132, you’ll see the NGINX Welcome page.

nginx-landing

That’s because in our /etc/nginx/sites-available directory, the default configuration file is used. We need to remove that and symbolic link to our new configuration file.


    $ cd /etc/nginx/sites-available
    $ rm default
    $ ln -s /etc/nginx/sites-available/myapp /etc/nginx/sites-enabled/myapp

Now, all we have to do is restart nginx and you will see our app running.


    $ service nginx restart

Setting up your own server is error-prone, so if you run into any errors, test your nginx configuration file using nginx -t. It will help you determine whether the problem is with nginx or something else.

meteor-up

In this article, we’ve dealt with deployment the hard way, because then you get to appreciate the process involved. But once you understand the process, using a tool like meteor-up will make your workflow much quicker.

This article is long enough already, so here’s a quick overview:

Install meteor-up on your local / development environment.


    $ npm install -g mup

Inside the application directory, run


    $ mup init

This creates two files – mup.json and settings.json.

Open mup.json and change the settings to fit your project and server configuration. Things are commented and quite self-explanatory. If you’re using a fresh server, leave most of the settings alone and just update the servers, app and env blocks.


    "servers": [
        {
          "host": "hostname",
          "username": "d4nyll",
          "pem": "~/.ssh/id_rsa"
        }
      ],

I am using SSH to connect to the server here. With meteor-up, your private key cannot be password-protected.

Set app to . because the mup.json file is already inside our application root directory. Set the env variables as described before.

Lastly, run mup setup followed by mup deploy. To update your application, just run mup deploy again.

Galaxy

As with every open-source framework funded by investors, there will be a time when they roll out some commercial product to complement the open-source work.

Galaxy is Meteor’s commercial product.

Modulus already provides a PaaS for Meteor, and has garnered some good reviews. But the MDG is coming up with the commercial platform which is tailored-made for Meteor alone.

You can keep up-to-date with updates on the Meteor Roadmap

ngrok

If you don’t want to actually deploy your application, but just want to provide your co-workers or client with a demo for a short period of time, then you don’t need to take up a meteor.com subdomain, or bother with deployment at all. Instead, check out ngrok!

ngrok basically opens up a tunnel from your machine and allow access through a subdomain of ngrok.io.

So all you have to do is run Meteor as normal (on port 3000), and run the command,


    ngrok http -subdomain=d4nyll 3000

And now you can access your site through d4nyll.ngrok.io. Neat.

Conclusion

As you’d appreciate, deploying your Meteor application isn’t so easy. Deployment and server configuration is an art of its own, and there are too many different distributions, variations and minor details for me to cover all of them. Tools like meteor-up deals with many of these complexities for you.

After you have your server set up, you’d still need to consider creating backups of the database, caching, logging, monitoring, load-balancing, seting up firewalls, security and encryption using SSL plus many more.

This is just a very basic guide so you know all the components that are required to run a Meteor application. Now that you have an understanding of the process, you might understand better what is happening behind the scenes when using tools like meteor-up.

Daniel Li

I am a full-stack web developer in Hong Kong. Do check out my profile, blog and projects, many of which are open-source! I get a real buzz from hearing words like Meteor and Node, if you do too, throw me a tweet!