Express File Uploads with Multer

Jecelyn Yeen
👁️ 103,323 views
💬 comments

File upload is a common feature that almost every website needs. We will go through step by step on how to handle single and multiple file(s) upload with Express, save it to database (LokiJs), and retrieve the saved file for viewing.

The complete sourcecode is available here: https://github.com/chybie/file-upload-express.

We will be using Typescript throughout this tutorial.

Table of Contents

    Install Required Dependencies

    I am using Yarn for package management. However, you can use npm if you like.

    Dependencies

    Run this command to install required dependencies

    // run this for yarn
    yarn add express cors multer lokijs del
    
    // or using npm
    npm install express cors multer lokijs del --save

    Notes:-

    • express: We will develop our API using ExpressJs
    • cors: A node.js package that provides an Express/Connect middleware to enable Cross Origin Resource Sharing (CORS)
    • multer: Node.js middleware for handling multipart/form-data.
    • loki: LokiJs, a fast, in-memory document-oriented datastore for node.js, browser and cordova
    • del: Delete files and folders

    Development Dependencies

    Since we are using Typescript, we need to install typings files in order to have auto-complete function (intellesense) during development.

    // run this for yarn
    yarn add typescript @types/express @types/multer @types/lokijs @types/del --dev
    
    // or using npm
    npm install typescript @types/express @types/multer @types/lokijs @types/del --save-dev

    Setup

    A couple of setup steps to go before we start.

    Typescript Configuration

    Add a typescript configuration file. To know more about Typescript configuration, visit https://www.typescriptlang.org/docs/handbook/tsconfig-json.html.

    // tsconfig.json
    
    {
        "compilerOptions": {
            "module": "commonjs",
            "moduleResolution": "node",
            "target": "es6",
            "noImplicitAny": false,
            "sourceMap": true,
            "outDir": "dist"
        }
    }

    Notes:-

    1. The compiled javascript code will be output to dist folder.
    2. Since Node Js 7.5+ support ES6 / 2015, we will set the target as es6.

    Start Script

    Add the following scripts.

    // package.json
    
    {
        ...
        "scripts": {
            "prestart": "tsc",
            "start": "node dist/index.js"
        }
        ...
    }

    Later on we can run yarn start or npm start to start our application.

    Notes:-

    1. When we run yarn start, it will trigger prestart script first. The command tsc will read the tsconfig.json file and compile all typescript files to javascript in dist folder.
    2. Then, we will run the compiled index file dist/index.js.

    Starting Express Server

    Let's start creating our Express server.

    // index.ts
    
    import * as express from 'express'
    import * as multer from 'multer'
    import * as cors from 'cors'
    import * as fs from 'fs'
    import * as path from 'path'
    import * as Loki from 'lokijs'
    
    // setup
    const DB_NAME = 'db.json';
    const COLLECTION_NAME = 'images';
    const UPLOAD_PATH = 'uploads';
    const upload = multer({ dest: `${UPLOAD_PATH}/` }); // multer configuration
    const db = new Loki(`${UPLOAD_PATH}/${DB_NAME}`, { persistenceMethod: 'fs' });
    
    // app
    const app = express();
    app.use(cors());
    
    app.listen(3000, function () {
        console.log('listening on port 3000!');
    });
    

    The code is pretty expressive itself. We allow Cross-Origin Resource Sharing (CORS), set the connection port to 3000, and start the server.

    Upload Single File

    Let's create our first route. We will create a route to allow users to upload their profile avatar.

    Route

    // index.ts
    ...
    import {
        loadCollection
    } from './utils';
    ...
    
    app.post('/profile', upload.single('avatar'), async (req, res) => {
        try {
            const col = await loadCollection(COLLECTION_NAME, db);
            const data = col.insert(req.file);
    
            db.saveDatabase();
            res.send({ id: data.$loki, fileName: data.filename, originalName: data.originalname });
        } catch (err) {
            res.sendStatus(400);
        }
    })
    

    Notes:

    1. This is a HTTP POST function.
    2. upload.single('avatar') is Multer middleware. It means we accept a single file with the field name avatar. File upload will be handled by Multer.
    3. Multer will add a file property for request when it's a single file upload.
    4. We will then load the LokiJs collection/table(we will create this function next), and insert the request file req.file to the collection.

    Load LokiJs Collection

    A generic function to retrieve a LokiJs collection if exists, or create a new one if it doesn't.

    // utils.ts
    
    import * as del from 'del';
    import * as Loki from 'lokijs';
    
    const loadCollection = function (colName, db: Loki): Promise<LokiCollection<any>> {
        return new Promise(resolve => {
            db.loadDatabase({}, () => {
                const _collection = db.getCollection(colName) || db.addCollection(colName);
                resolve(_collection);
            })
        });
    }
    
    export { loadCollection }
    

    Run Our Application

    You may run the application with yarn start. I try to call the locahost:3000/profile API with (Postman)[https://www.getpostman.com/apps], an GUI application for API testing.

    When I upload a file, you can see that a new file created in uploads folder and the database file db.json is created as well.

    When I issue a call without passing in avatar, error will be returned.

    Upload single file

    Filter File Type

    We can handle file upload successfully now. Next, we need to limit the file type to image only. To do this, let's create a filter function that will test the file extensions.

    // utils.ts
    ...
    
    const imageFilter = function (req, file, cb) {
        // accept image only
        if (!file.originalname.match(/\.(jpg|jpeg|png|gif)$/)) {
            return cb(new Error('Only image files are allowed!'), false);
        }
        cb(null, true);
    };
    
    ...
    
    export { imageFilter, loadCollection }
    

    Apply the Image Filter

    We need to tell the Multer to apply our image filter function. Add it in upload variable.

    // index.ts
    ...
    import { imageFilter, loadCollection } from './utils';
    ...
    // setup
    ...
    const upload = multer({ dest: `${UPLOAD_PATH}/`, fileFilter: imageFilter }); // apply filter
    
    ...
    

    Restart the application, try to upload a non-image file and you should get error.

    Upload Multiple Files

    Let's proceed to handle multiple files upload now. We will create a new route to allow user upload their photos.

    Route

    ...
    
    app.post('/photos/upload', upload.array('photos', 12), async (req, res) => {
        try {
            const col = await loadCollection(COLLECTION_NAME, db)
            let data = [].concat(col.insert(req.files));
    
            db.saveDatabase();
            res.send(data.map(x => ({ id: x.$loki, fileName: x.filename, originalName: x.originalname })));
        } catch (err) {
            res.sendStatus(400);
        }
    })
    
    ...
    

    The code is similar to single file upload, except we accept a field photos instead of avatar, limit the total file to 12, accept an array of files as input and reply result as array.

    Retrieve List of Images

    Next, create a route to retrieve all images.

    // index.ts
    ...
    
    app.get('/images', async (req, res) => {
        try {
            const col = await loadCollection(COLLECTION_NAME, db);
            res.send(col.data);
        } catch (err) {
            res.sendStatus(400);
        }
    })
    
    ...

    The code is super easy to understand.

    Retrieve Image by Id

    Next, create a route to retrieve an image by id.

    // index.ts
    ...
    
    app.get('/images/:id', async (req, res) => {
        try {
            const col = await loadCollection(COLLECTION_NAME, db);
            const result = col.get(req.params.id);
    
            if (!result) {
                res.sendStatus(404);
                return;
            };
    
            res.setHeader('Content-Type', result.mimetype);
            fs.createReadStream(path.join(UPLOAD_PATH, result.filename)).pipe(res);
        } catch (err) {
            res.sendStatus(400);
        }
    })
    
    ...

    Notes:-

    1. We will return 404 if image not exist in database.
    2. We will stream the file as output, set the content-type correctly so our client or browser know how to handle it.

    Run the Application

    Now restart the application, upload a couple of images, and retrieve it by id. You should see the image is return as image instead of json object.

    Get image by id

    Clear All Data When Restart

    Sometimes, you might want to clear all the images and database collection during development. Here's a helper function to do so.

    // utils.ts
    
    ....
    
    const cleanFolder = function (folderPath) {
        // delete files inside folder but not the folder itself
        del.sync([`${folderPath}/**`, `!${folderPath}`]);
    };
    
    ...
    
    export { imageFilter, loadCollection, cleanFolder }
    

    Use it in our application.

    // index.ts
    
    ...
    import { imageFilter, loadCollection, cleanFolder } from './utils';
    ...
    
    // setup
    ...
    
    // optional: clean all data before start
    cleanFolder(UPLOAD_PATH);
    
    ...

    Summary

    Handle file(s) upload with Express is easy with the help of Multer middleware. Multer is very flexible and configurable. You can go though the documentation and add more configuration as you need.

    The complete sourcecode is available here: https://github.com/chybie/file-upload-express.

    That's it. Happy coding.

    Jecelyn Yeen

    21 posts

    Coder. Diver. Board Game Lover.

    Speak English, Mandarin, JavaScript, Typescript, C# and more.

    GDE | Angular | Web Technologies.