Build a RESTful API with Flask – The TDD Way

Jee Gikera

Great things are done by a series of small things brought together – Vincent Van Gogh

This article's intention is to provide a easy-to-follow project-based process on how to create a RESTful API using the Flask framework.

Why Flask?

A bit of context – I've written a bunch of articles on Django-driven RESTful APIs. Though a great resource for Django enthusiasts, not everyone wants to code in Django. Besides, it's always good to acquaint yourself with other frameworks.

Learning Flask is easier and faster. It's super easy to setup and get things running. Unlike Django (which is heavier), you'll never have functionality lying around that you aren't using.

Typical of all our web apps, we'll use the TDD approach. It's really simple. Here's how we do Test Driven Development:

  • Write a test. – The test will help flesh out some functionality in our app
  • Then, run the test – The test should fail, since there's no code(yet) to make it pass.
  • Write the code – To make the test pass
  • Run the test – If it passes, we are confident that the code we've written meets the test requirements
  • Refactor code – Remove duplication, prune large objects and make the code more readable. Re-run the tests every time we refactor our code
  • Repeat – That's it!

What we'll create

We're going to develop an API for a bucketlist. A bucketlist is a list of all the goals you want to achieve, dreams you want to fulfill and life experiences you desire to experience before you die (or hit the bucket). The API shall therefore have the ability to:

  • Create a bucketlist (by giving it a name/title)
  • Retrieve an existing bucketlist
  • Update it (by changing it's name/title)
  • Delete an existing bucketlist

Prerequisites

  • Python3 - A programming language that lets us work more quickly (The universe loves speed!).
  • Flask - A microframework for Python based on Werkzeug, Jinja 2 and good intentions
  • Virtualenv - A tool to create isolated virtual environments

Let's start with configuring our Flask app structure!

Virtual Environment

First, we'll create our application directory. On the terminal, create an empty directory called bucketlist with mkdir bucketlist. Then, Cd into the directory. Create an isolated virtual environment:

$ virtualenv  venv

Install Autoenv globally using pip install autoenv Here's why – Autoenv helps us to set commands that will run every time we cd into our directory. It reads the .env file and executes for us whatever is in there.

Create a .env file and add the following:

source env/bin/activate
export FLASK_APP="run.py"
export SECRET="some-very-long-string-of-random-characters-CHANGE-TO-YOUR-LIKING"
export APP_SETTINGS="development"
export DATABASE_URL="postgresql://localhost/flask_api"

The first line activates our virtual environment venv that we just created. Line 2, 3 and 4 export our FLASK_APP, SECRET, APP_SETTINGS and DATABASE_URL variables. We'll integrate these variables as we progress through the development process.

Run the following to update and refresh your .bashrc:

$ echo "source `which activate.sh`" >> ~/.bashrc
$ source ~/.bashrc

You will see something like this on your terminal

Sometimes autoenv might not work if you have zsh installed. A good workaround would be to simply source the .env file and we are set.

$ source .env

Conversely, if you don't want to automate things for the long run, you don't have to use autoenv. A simple export directly from the terminal would do.

$ export FLASK_APP="run.py"
$ export APP_SETTINGS="development"
$ export SECRET="a-long-string-of-random-characters-CHANGE-TO-YOUR-LIKING"
$ export DATABASE_URL="postgresql://localhost/flask_api"

Inside our virtual environment, we'll create a bunch of files to lay out our app directory stucture. Here's what it should look like:

β”œβ”€β”€ bucketlist (this is the directory we cd into)
    β”œβ”€β”€ app
    β”‚   β”œβ”€β”€ __init__.py
    β”‚   └── models.py  
    β”œβ”€β”€ instance
    β”‚   └── __init__.py
    β”œβ”€β”€ manage.py
    β”œβ”€β”€ requirements.txt
    β”œβ”€β”€ run.py
    └── test_bucketlist.py

After doing this, install Flask using pip.

(venv)$ pip install flask

Environment Configurations

Flask needs some sought of configuration to be available before the app starts. Since environments (development, production or testing) require specific settings to be configured, we'll have to set environment-specific things such as a secret key, debug mode and test mode in our configurations file.

If you haven't already, create a directory and call it instance. Inside this directory, create a file called config.py and also init.py. Inside our config file, we'll add the following code:

# /instance/config.py

import os

class Config(object):
    """Parent configuration class."""
    DEBUG = False
    CSRF_ENABLED = True
    SECRET = os.getenv('SECRET')
    SQLALCHEMY_DATABASE_URI = os.getenv('DATABASE_URL')

class DevelopmentConfig(Config):
    """Configurations for Development."""
    DEBUG = True

class TestingConfig(Config):
    """Configurations for Testing, with a separate test database."""
    TESTING = True
    SQLALCHEMY_DATABASE_URI = 'postgresql://localhost/test_db'
    DEBUG = True

class StagingConfig(Config):
    """Configurations for Staging."""
    DEBUG = True

class ProductionConfig(Config):
    """Configurations for Production."""
    DEBUG = False
    TESTING = False

app_config = {
    'development': DevelopmentConfig,
    'testing': TestingConfig,
    'staging': StagingConfig,
    'production': ProductionConfig,
}

The Config class contains the general settings that we want all environments to have by default. Other environment classes inherit from it and can be used to set settings that are only unique to them. Additionally, the dictionary app_config is used to export the 4 environments we've specified. It's convenient to have it so that we can import the config under its name tag in future.

A couple of config variables to note:

  • The SECRET_KEY – is a random string of characters that's used to generate hashes that secure various things in an app. It should never be public to prevent malicious attackers from accessing it.
  • DEBUG – tells the app to run under debugging mode when set to True, allowing us to use the Flask debugger. It also automagically reloads the application when it's updated. However, it should be set to False in production.

Configuring the Database

Installation requirements

The tools we need for our database to be up and running are:

We might have used a easy-to-setup database such as SQLite. But since we want to learn something new, powerful and awesome, we'll go with PostgreSQL.

SQLAlchemy is our Object Relational Mapper (ORM). Why should we use an ORM, you ask? An ORM converts the raw SQL data (called querysets) into data we can understand called objects in a process called serialization and vice versa (deserialization). Instead of painstakingly writing complex raw SQL queries, why not use a tested tool developed just for this purpose?

Let's install the requirements as follows:

(venv)$    pip install flask-sqlalchemy psycopg2 flask-migrate

Ensure you have installed PostgresSQL in your computer and it's server is running locally on port 5432

In your terminal, create a Postgres database:

(venv) $ createdb test_db
(venv) $ createdb flask_api

Createdb is a wrapper around the SQL command CREATE DATABASE. We created

  • test database test_db for our testing environment.
  • main database flask_api for development environment.

We've used two databases so that we do not interfere with the integrity of our main database when running our tests.

Create The App

It's time to right some code! Since we are creating an API, we'll install Flask-API extension.

(venv)$ pip install Flask-API

Flask API is an implementation of the same web browsable APIs that Django REST framework provides. It'll helps us implement our own browsable API.

In our empty app/__init__.py file, we'll add the following:

# app/__init__.py

from flask_api import FlaskAPI
from flask_sqlalchemy import SQLAlchemy

# local import
from instance.config import app_config

# initialize sql-alchemy
db = SQLAlchemy()

def create_app(config_name):
    app = FlaskAPI(__name__, instance_relative_config=True)
    app.config.from_object(app_config[config_name])
    app.config.from_pyfile('config.py')
    app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
    db.init_app(app)

    return app

The create_app function wraps the creation of a new Flask object, and returns it after it's loaded up with configuration settings using app.config and connected to the DB using db.init_app(app).

We've also disabled track modifications for SQLAlchemy because it'll be deprecated in future due to it's significant performance overhead. For debugging enthusiasts, you can set it to True for now.

Now, we need to define an entry point to start our app. Let's edit the run.py file.

import os

from app import create_app

config_name = os.getenv('APP_SETTINGS') # config_name = "development"
app = create_app(config_name)

if __name__ == '__main__':
    app.run()

Run it!

Now we can run the application on our terminal to see if it works:

(venv)$   flask run

We can also run it using python run.py. We should see something like this:

Data Model

It's time to create our bucketlist model. A model is a representation of a table in a database. Add the following inside the empty models.py file:

# app/models.py

from app import db

class Bucketlist(db.Model):
    """This class represents the bucketlist table."""

    __tablename__ = 'bucketlists'

    id = db.Column(db.Integer, primary_key=True)
    name = db.Column(db.String(255))
    date_created = db.Column(db.DateTime, default=db.func.current_timestamp())
    date_modified = db.Column(
        db.DateTime, default=db.func.current_timestamp(),
        onupdate=db.func.current_timestamp())

    def __init__(self, name):
        """initialize with name."""
        self.name = name

    def save(self):
        db.session.add(self)
        db.session.commit()

    @staticmethod
    def get_all():
        return Bucketlist.query.all()

    def delete(self):
        db.session.delete(self)
        db.session.commit()

    def __repr__(self):
        return "<Bucketlist: {}>".format(self.name)

Here's what we've done in the models.py file:

  • We imported our db connection from the app/__init__.py.
  • Next, we created a Bucketlist class that inherits from db.Model and assigned a table. name bucketlists (it should always be plural). We've therefore created a table to store our bucketlists.
  • The id field contains the primary key, the name field will store the name of the bucketlist.
  • The __repr__ method represents the object instance of the model whenever it is queries.
  • The get_all() method is a static method that'll be used to get all the bucketlists in a single query.
  • The save() method will be used to add a new bucketlist to the DB.
  • The delete() method will be used to delete an existing bucketlist from the DB.

Making Migrations

Migrations is a way of propagating changes we make to our models (like adding a field, deleting a model, etc.) into your database schema. Now that we've a defined model in place, we need to tell the database to create the relevant schema.

Flask-Migrate uses Alembic to autogenerate migrations for us. It will serve this purpose.

The migration script

A migration script will conveniently help us make and apply migrations everytime we edit our models. It's good practice to separate migration tasks and not mix them with the code in our app.

That said, we'll create a new file called manage.py.

Our directory structure should now look like this:

β”œβ”€β”€ bucketlist
    β”œβ”€β”€ app
    β”‚   β”œβ”€β”€ __init__.py
    β”‚   └── models.py  
    β”œβ”€β”€ instance
    β”‚   β”œβ”€β”€ __init__.py
    β”‚   └── config.py
    β”œβ”€β”€ manage.py
    β”œβ”€β”€ requirements.txt
    β”œβ”€β”€ run.py
    └── test_bucketlist.py

Add the following code to manage.py:

# manage.py

import os
from flask_script import Manager # class for handling a set of commands
from flask_migrate import Migrate, MigrateCommand
from app import db, create_app
from app import models

app = create_app(config_name=os.getenv('APP_SETTINGS'))
migrate = Migrate(app, db)
manager = Manager(app)

manager.add_command('db', MigrateCommand)

if __name__ == '__main__':
    manager.run()

The Manager class keeps track of all the commands and handles how they are called from the command line. The MigrateCommand contains a set of migration commands. We've also imported the models so that the script can find the models to be migrated. The manager also adds the migration commands and enforces that they start with db.

We will run migrations initialization, using the db init command as follows:

(venv)$   python manage.py db init

You'll notice a newly created folder called migrations. This holds the setup necessary for running migrations. Inside of β€œmigrations” is a folder called β€œversions”, which will contain the migration scripts as they are created.

Next, we'll run the actual migrations using the db migrate command:

 (venv)$   python manage.py db migrate

  INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
  INFO  [alembic.runtime.migration] Will assume transactional DDL.
  INFO  [alembic.autogenerate.compare] Detected added table 'results'
    Generating /bucketlist/migrations/versions/63dba2060f71_.py
    ...done

You’ll also notice that in your versions folder there is a migration file. This file is auto-generated by Alembic based on the model.

Finally, we’ll apply the migrations to the database using the db upgrade command:

(venv)$   python manage.py db upgrade

INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade  -> 536e84635828, empty message

Our DB is now updated with our bucketlists table. If you jump into the psql prompt, here's a screenshot on how you can confirm if the table exists:

Time To Test!

Inside our tests directory, let's create tests. Creating tests that fail is the first step of TD.(Failing is good). These tests will help guide us in creating our functionality. It might seem daunting at first to write tests but it's really easy once you get practicing.

On the parent directory, create a test file called test_bucketlist.py. This file will contain the following:

  • Test Case class to house all our API tests.
  • setUp() methods to initialize our app and it's test client and create our test database within the app's context.
  • tearDown() method to tear down test variables and delete our test database after testing is done.
  • tests to test whether our API can create, read, update and delete a bucketlist.
# test_bucketlist.py
import unittest
import os
import json
from app import create_app, db

class BucketlistTestCase(unittest.TestCase):
    """This class represents the bucketlist test case"""

    def setUp(self):
        """Define test variables and initialize app."""
        self.app = create_app(config_name="testing")
        self.client = self.app.test_client
        self.bucketlist = {'name': 'Go to Borabora for vacation'}

        # binds the app to the current context
        with self.app.app_context():
            # create all tables
            db.create_all()

    def test_bucketlist_creation(self):
        """Test API can create a bucketlist (POST request)"""
        res = self.client().post('/bucketlists/', data=self.bucketlist)
        self.assertEqual(res.status_code, 201)
        self.assertIn('Go to Borabora', str(res.data))

    def test_api_can_get_all_bucketlists(self):
        """Test API can get a bucketlist (GET request)."""
        res = self.client().post('/bucketlists/', data=self.bucketlist)
        self.assertEqual(res.status_code, 201)
        res = self.client().get('/bucketlists/')
        self.assertEqual(res.status_code, 200)
        self.assertIn('Go to Borabora', str(res.data))

    def test_api_can_get_bucketlist_by_id(self):
        """Test API can get a single bucketlist by using it's id."""
        rv = self.client().post('/bucketlists/', data=self.bucketlist)
        self.assertEqual(rv.status_code, 201)
        result_in_json = json.loads(rv.data.decode('utf-8').replace("'", "\""))
        result = self.client().get(
            '/bucketlists/{}'.format(result_in_json['id']))
        self.assertEqual(result.status_code, 200)
        self.assertIn('Go to Borabora', str(result.data))

    def test_bucketlist_can_be_edited(self):
        """Test API can edit an existing bucketlist. (PUT request)"""
        rv = self.client().post(
            '/bucketlists/',
            data={'name': 'Eat, pray and love'})
        self.assertEqual(rv.status_code, 201)
        rv = self.client().put(
            '/bucketlists/1',
            data={
                "name": "Dont just eat, but also pray and love :-)"
            })
        self.assertEqual(rv.status_code, 200)
        results = self.client().get('/bucketlists/1')
        self.assertIn('Dont just eat', str(results.data))

    def test_bucketlist_deletion(self):
        """Test API can delete an existing bucketlist. (DELETE request)."""
        rv = self.client().post(
            '/bucketlists/',
            data={'name': 'Eat, pray and love'})
        self.assertEqual(rv.status_code, 201)
        res = self.client().delete('/bucketlists/1')
        self.assertEqual(res.status_code, 200)
        # Test to see if it exists, should return a 404
        result = self.client().get('/bucketlists/1')
        self.assertEqual(result.status_code, 404)

    def tearDown(self):
        """teardown all initialized variables."""
        with self.app.app_context():
            # drop all tables
            db.session.remove()
            db.drop_all()

# Make the tests conveniently executable
if __name__ == "__main__":
    unittest.main()

A bit of testing explanation. Inside the test_bucketlist_creation(self) we make a POST request using a test client to the /bucketlists/ url. The return value is obtained and its status code is asserted to be equal to a status code of 201(Created). If it's equal to 201, the test assertion is true, making the test pass. Finally, it checks whether the returned response contains the name of the bucketlist we just created. This is done using self.assertIn(a, b) If the assertion evaluates to true, the test passes.

Now we'll run the test as follows:

(venv)$   python test_bucketlist.py

All the tests must fail. Now don't be scared. This is good because we have no functionality to make the test pass. Now's the time to create the API functionality that will make our tests pass.

API Functionality

Our API is supposed to handle four HTTP requests

  • POST – Used to create the bucketlist
  • GET – For retrieving one bucketlist using its ID and many bucketlists
  • PUT – For updating a bucketlist given its ID
  • DELETE – For deleting a bucketlist given its ID

Let's get this done straight away. Inside our app/__init__.py file, we'll edit it as follows:

# app/__init__.py

# existing import remains

from flask import request, jsonify, abort

def create_app(config_name):
    from api.models import Bucketlist

    #####################
    # existing code remains #
    #####################

    @app.route('/bucketlists/', methods=['POST', 'GET'])
    def bucketlists():
        if request.method == "POST":
            name = str(request.data.get('name', ''))
            if name:
                bucketlist = Bucketlist(name=name)
                bucketlist.save()
                response = jsonify({
                    'id': bucketlist.id,
                    'name': bucketlist.name,
                    'date_created': bucketlist.date_created,
                    'date_modified': bucketlist.date_modified
                })
                response.status_code = 201
                return response
        else:
            # GET
            bucketlists = Bucketlist.get_all()
            results = []

            for bucketlist in bucketlists:
                obj = {
                    'id': bucketlist.id,
                    'name': bucketlist.name,
                    'date_created': bucketlist.date_created,
                    'date_modified': bucketlist.date_modified
                }
                results.append(obj)
            response = jsonify(results)
            response.status_code = 200
            return response

    return app

We've imported

  • request for handling our requests.
  • jsonify to turn the JSON output into a Response object with the application/json mimetype.
  • abort which will abort a request with an HTTP error code early.

We've also added an import from api.models import Bucketlist immediately inside the create_app method so that we get access to the Bucketlist model while preventing the horror of circular imports. Flask provides an @app.route decorator on top of the new function def bucketlists() which enforces us to only accepts GET and POST requests. Our function first checks the type of request it receives. If it's a POST, it creates a bucketlist by extracting the name from the request and saves it using the save() method we defined in our model. It consequently returns the newly created bucketlist as a JSON object. If it's a GET request, it gets all the bucketlists from the bucketlists table and returns a list of bucketlists as JSON objects. If there's no bucketlist on our table, it will return an empty JSON object {}.

Now let's see if our new GET and POST functionality makes our tests pass.

Running our Tests

Run the tests as follows:

(venv)$ python test_bucketlists.py

2 out of 5 tests should pass. We've now handled the GET and POST requests successfully.

At this moment, our API can only create and get all the bucketlists. It cannot get a single bucketlist using its bucketlist ID. Also, it can neither edit a bucketlist nor delete it from the DB. To complete it, we'd want to add these functionalities.

Adding PUT and DELETE functionality

On our app/__init__.py file, let's edit as follows:

# app/__init__.py

# existing import remains

def create_app(config_name):

    #####################
    # existing code remains #
    #####################

    ###################################
    # The GET and POST code is here
    ###################################

    @app.route('/bucketlists/<int:id>', methods=['GET', 'PUT', 'DELETE'])
    def bucketlist_manipulation(id, **kwargs):
     # retrieve a buckelist using it's ID
        bucketlist = Bucketlist.query.filter_by(id=id).first()
        if not bucketlist:
            # Raise an HTTPException with a 404 not found status code
            abort(404)

        if request.method == 'DELETE':
            bucketlist.delete()
            return {
            "message": "bucketlist {} deleted successfully".format(bucketlist.id) 
         }, 200

        elif request.method == 'PUT':
            name = str(request.data.get('name', ''))
            bucketlist.name = name
            bucketlist.save()
            response = jsonify({
                'id': bucketlist.id,
                'name': bucketlist.name,
                'date_created': bucketlist.date_created,
                'date_modified': bucketlist.date_modified
            })
            response.status_code = 200
            return response
        else:
            # GET
            response = jsonify({
                'id': bucketlist.id,
                'name': bucketlist.name,
                'date_created': bucketlist.date_created,
                'date_modified': bucketlist.date_modified
            })
            response.status_code = 200
            return response

    return app

We've now defined a new function def bucketlist_manipulation() which uses a decorator that enforces it to only handle GET, PUT and DELETE Http requests. We query the db to filter using an id of the given bucketlist we want to access. If there's no bucketlist, it aborts and returns a 404 Not Found status. The second if-elif-else code blocks handle deleting, updating or getting a bucketlist respectively.

Run the tests

Now, we expect all the tests to pass. Let's run them and see if all of them actually pass.

(venv)$ python test_bucketlist.py

We should now see all the test passing.

Test using Postman and Curl

Fire up Postman. Key in http://localhost:5000/bucketlists/ and send a POST request with a name as the payload. We should get a response like this:

We can play around with Curl as well to see it working from our terminal:

Conclusion

We've covered quite a lot on how to create a test-driven RESTful API using Flask. We learnt about configuring our Flask environment, hooking up the models, making and applying migrations to the DB, creating unit tests and making tests pass by refactoring our code.

In Part 2 of this Series, we'll learn how to enforce user authentication and authorization on our API with a continued focus on unit testing.

Jee Gikera

5 posts

A full-stack software engineer, Jee believes in doing it today, not tomorrow. He enjoys gaming, lively hangouts with friends and family, and when away – the quest for rare coins.