Real-Time Graph Database as a Service with Appbase

Introduction

As developers, we all know that our industry evolves at a very fast pace. So fast that it’s often hard to keep up with all the new libraries, frameworks, and new versions of the tools we use daily. In some roles, it’s important to stay on top of industry news and trends. This post brings to you a new database concept for you to get started with and make your learning experience more effective and easier.

A trend that has already started taking shape is the concept of database-as-a-service (DBaaS). There are many hosted services out there.

In this article, I’ll cover a realtime graph database, which is worth keeping an eye on. It’s a DBaaS product that handles database management and optimization for users: AppBase.

appbase

As a service, Appbase handles your database backend and does the dirty work of database management for you.

They use a REST (representational state transfer) API that can be used from any platform. The REST API basically acts as a two-way street to get data in and out. Also this platform is naturally designed to handle multi tenancy, meaning the ability to handle separate groups of users who share access to the databases.

Concepts

Several popular databases can be classified as below:

AppBase is a realtime Directed Graph Database, built on top of a graph schema, where objects are called Vertices (a graph term for entities), and the links (relationships) between the objects are called Edges.

We can not actually compare these concepts with other kinds of databases, but it helps to understand if we compare the usage practices. For instance, SQL has concepts for table, row, column, relation and MongoDB has collection, Document, and fields, here are the key concepts in AppBase:

  1. Vertex — is similar to a Document in MongoDB, it stores data.

    • key — A string that is used to access the vertex as an entry point.
    • properties — A JSON object representing the data inside the vertex.
  2. Namespace — is similar to a Collection in MongoDB or type in ElasticSearch. A vertex always belongs to a namespace and security rules can be applied on namespaces.

  3. Edge — is a directional link to another vertex.

    • priority — An edge can optionally have a priority, a Number (a secondary index).
    • outVertex — is the vertex which the edge points to.
    • inVertex — the vertex from which the edge starts.
    • i.e. an edge is pointed from inVertex to an outVertex.
  4. Path — it is a powerful way to point to the exact data you want to interact with. A typical path looks like <namespace>/<v1>/<v2>/…/<vn>/.

  5. URL — It is just another representation of a Path prefixed with the Appbase API base URL, application’s name and the API version. A way to point to a Path in the REST API. Eg. https://api.appbase.io/<app_name>/v2/<path>/.

Example

This example will use a graph visualization tool for drawing directed graphs and use AppBase as a realtime database for storing and fetching data.

There are some editing tools for building directed graphs, for instance, Force Editor includes pan and zoom functions. Also it enables the user to drag from an existing node to add a new node/link and drag the nodes to move the graph dynamically.

Force Editor is based on another open source D3.js.The D3.js library offers many ways to visualize the relationships between nodes. The purpose is to position the nodes of a graph in two-dimensional space in a simple, intuitive, and aesthetically pleasing way.

image that ilustrates Force Editor

Drag from an existing node to add a new node or link. Click to select/deselect nodes/links. Hit the DELETE key to remove the selected node or link. Drag to pan. Scroll to zoom.

How to set up AppBase

It can be used either on the backend or on the client side. To use it in the browser, use this link: https://cdn.appbase.io/latest/appbase.js. To use it with nodejs, do npm install appbasejs. It will be exposed as a global object Appbase and has the following four methods: credentials(), ns(), uuid(), and serverTime(). Each methods will be described later.

First step to getting started with is log in to Dashboard, and create a new app. To set it up, you have to authorize the app using credentials method and providing the appname and appsecret identifiers, as they appear in the Dev Console.


Appbase.credentials('aphrodite',
'4d8d0072580912343cd74aa0015cd217',
function(error, isAuthorized) {
  if (!error && isAuthorized)
    console.log("app successfully registered");
});

Note: you will wanna change the parameters of the Appbase.credentials() method to your own app name, app secret combination, as seen in the Dashboard.

OAuth is another manner to make authentication easy with completely client-side code. It has built-in functionality for third-party providers such as Facebook, LinkedIn, GitHub, Dropbox, and Google. When a user authenticates to, the information about the user is returned in callbacks on the client device. Once authenticated, the user session will be managed, ensuring that the user is remembered across page refreshes and browser restarts.

To enable OAuth login is needed few initial steps. These steps will not be covered here, you can read further details at this link.

After enabling it, the following code shows a popup and retrieves the user’s credentials for Appbase, and Facebook.


Appbase.authPopup('facebook', function(error, authObj, requestObj) {
  if(error) {
    console.log('Error occured:', error);
  } else {
    console.log('Logged in as:', authObj.uid); 
  }
});

requestObj returned in the authentication can be used to perform API calls to a provider. It contains methods that wrap the usual HTTP methods: .get(), .post(), .put(), .patch(), and .del().

Once the user logs in with a provider, the credentials are cached in localStorage automatically, and this helps staying logged in across browser restarts.

Setting up our example

This example is basically client side code. The authentication uses Facebook provider. You can grab the entire example files from my repository. The server in the repository is just to server static files over HTTP.

In JS folder, you can find login.js, file responsible to authenticate via OAuth. Let’s take a look on that and see how it’s set up.


define(['jquery', 'forceView'], function($, forceView) {
  return function(e, callback) {
    Appbase.credentials("sample_app_with_d3");
    Appbase.authPopup('facebook', function(error, authObj, requestObj) {
      if(error) {
        console.log('Error occured:', error);
      } else {
        console.log('Logged in as:', authObj.uid);
        $(".row.login").hide();
        $('.row.force-view').removeClass('hidden');
        forceView.init(authObj.name);
        require(['observeNS'], function(Observe) {
          Observe.init(authObj);
        });
      }
      callback && callback(error, authObj, requestObj);
    });
  };
});

Once the user log in, OAuth callback will be triggered and do the following:

  1. Hide login form
  2. Show Force Editor
  3. Init all nodes from user namespace
  4. Observe and watch for further new user namespaces

To see a live example in action, go to https://sample-appbase-with-d3.herokuapp.com/ and click on Enter via Facebook Auth or Enter via appsecret identifier button.

LogIn page

After entering in, open a new window and see how both windows are synced.

after the logging in

The graph drawing is up to D3.js but notice the vertice content and edges are the same on both windows.

sync cross windows

Storing and retrieving data

Let’s learn how to create two vertices under a namespace user.


var authorRef = Appbase.ns('user').v('igor');
var readerRef = Appbase .ns('user').v(Appbase.uuid());

Namespaces are containers that do not hold actual data, but help with data logic separation. A vertex can only be created using a ns reference object.

As seen here, vertex takes a String as an identifier. Appbase.uuid() is a handy way for creating a unique String id.

A vertex can store data as a JSON object using the setData() method.


authorRef.setData({ status: 'geek', location: 'Belo Horizonte, Brazil' });
readerRef.setData({ status: 'learning', message: 'Remember Red, tweet this blog is a good thing.' });

Vertices can be connected via edges. Now let’s add readerRef as an edge to the authorRef vertex.


authorRef.setEdge('friendship', readerRef);

Now, the readerRef vertex can also be accessed using the path /user/igor/friendship. To retrieve a vertex reference at a given path, ns.v(path) is a handy method.

Retrieval objects is supported using realtime streams. Just add listeners, which return existing data stored, and keeps returning new data as it keeps changing. Similarly let’s add a listener for retrieving edges on the vertex.


authorRef.on('properties', function(error, ref, authorSnap) {
  console.log(authorSnap.properties().status);
  console.log(authorSnap.properties().location);
});
authorRef.on('edge_added', function(error, outVertexRef, readerSnap) {
  console.log(readerSnap.properties().message);
});

Simple as that! Now you can be confident to come back to the example and understand how Appbase and Force Editor are being integrated on our example.

Appbase and Force Editor integration

The integration uses the topic-based publish/subscribe library PubSubJS. It has synchronisation decoupling, so topics are published asynchronously. This helps keep the integration in our example predictable.

There two cases in that integration. One for listening changes from DBaaS and then apply these changes in Force Editor. And the second case is send changes from Force Editor to DBaaS.

Take a look on the first case below. There is a listener on the Appbase namespace to observe when a vertex is added. Once it detected a new vertex, it will publish a message to Force Editor and the node will be visually added.

appbaseSync.js


nsref.on('vertex_added', function(err, vertexRef, obj) {
  /** … **/
  PubSub.publish('forceView:addNode', $.extend(obj.properties() || {}, {
    id: vertexRef.name()
  }));
  /** **/
});

force_view.js


PubSub.subscribe('forceView:addNode', function(msg, node) {
  var selected_node = searchNode(node);
  if (!selected_node) {
    nodes.push(node);
    redraw();
  }
});

Let’s take a look on the second case when the user add a node in Force Editor and it sends a message to store this node accordingly in the DB.

force_view.js


function mouseup() {
  /** … **/
  PubSub.publish('forceView:addedNode', node);
  /** … **/
  redraw();
}

appbaseSync.js


$('button.add-node').on('click', function() {
  PubSub.publish('forceView:addedNode', {});
});

PubSub.subscribe('forceView:addedNode', function(msg, node) {
  var id = Appbase.uuid(), vref = nsref.v(id);
  node.id = id;
  vref.setData(node);
});

Running the example locally

You can access the full example application code on Github, just clone the repository git clone git@github.com:igorlima/sample_appbase_with_d3.git (or download the code), execute npm install from a console to install all dependencies, then execute node server.js to start the application! Head to http://localhost:5000 to see the application in action!

NOTE: Remeber you are using my application sample_app_with_d3. Appbase is completely free up to 100 thousand API calls per month. Feel free to use it while you’re learning. After that, create your own application’s name, then new learners can use my API calls left. Thanks.

Conclusion

That’s all there is to it! Just remember the Appbase is a cloud service that lets developers build realtime apps with awesome collaboration and search features. Whether or not it’s a new option you have as DBaaS and it can be used either on the backend or on the client side; Also, you now have an additional nugget of knowledge in your tool belt for mastering the data storage.

Appbase is not open-source, yet. They are thinking about the best way to balance their business interests with developer freedom. If you think Appbase should be open source, feel free and write them at info@appbase.io

Igor Ribeiro Lima

A lucky guy who loves his wife, family, friends and coding. Enjoy learning new things, working on personal projects and contributing on open source.