Community Post

Build ultra-speed autocomplete with Go and jQuery [Part 1]

Danilo Polani

Autocomplete is an important focus of the User Experience, so a speed hint is required while user is typing.

After trying many technologies such as PHP (I'm a PHP developer), Python and Ruby, I found the perfect solution in Go, the language created by Google. Its performance are awesome, near C/C++ speed and better than Java.

Quick performance information

With Laravel5 + Cache the request is about 500ms long. GoLang + Cache is about 70ms.

I'll skip the installation of GoLang, you can find it here: https://golang.org/doc/install also with a really great tour about the language and how to write it. We'll focus on code.

1. Create project folder

The first thing is to create the project folder. Go to your src folder inside your Go workspace (or simply do cd $GOPATH) or create it if it doesn't exist.

Go (or create) a structure folder inside your src as following: github.com//, for example github.com/DaniloPolani/autocomplete (hint: you can do this executing mkdir -p github.com/<username>/<project>. The "p" flag stands for "parents", it will create the folders hierarchical).

2. Install dependencies

We'll use two external (by users) packages for this project: Gin Tonic, a web framework for our routes, and Go Cache, an in-memory key-value cache. They will be saved in src folder.

$ go get gopkg.in/gin-gonic/gin.v1
$ go get https://github.com/patrickmn/go-cache

After this, we move on our "project" folder (autocomplete) and we create a new file called autocomplete.go.

$ cd github.com/DaniloPolani/autocomplete
$ nano autocomplete.go

With nano, we create the new file and we are ready to write-in it.

3. Time to work

The first thing is to declare the package, the imports and the main function.

package main

import (
    "gopkg.in/gin-gonic/gin.v1"
    "github.com/patrickmn/go-cache"
    "net/http"
    "io/ioutil"
    "time"
)

func main() {}

As you can see, we import also net/http, io/ioutil and time that are built-in in Go. We'll use net/http for make Get requests, io/ioutil for reading responses of the calls and time to set properly the cache.

Inside the main function, we declare our gin instance and the cache.

// [...]
    // Routes
    //gin.SetMode(gin.ReleaseMode) Set this without comment when you'll go live
    r := gin.Default()

    // Set cache default expiring to 30days and which purges expired items every 24h
    c := cache.New((24  time.Hour)  30  6, 24  time.Hour)
// [...]

We keep commented the line of gin.SetMode to show app information, when you are ready to go live remove the comment.

The cache settings are a bit weird, I suggest you to leave the settings as they are, but you can check them out here on the Github page of Go-cache.

Now we write our route poiting to /autocomplete/company/ and check if the term results are cached.

// [...]
    r.GET("/autocomplete/company/:term", func(g *gin.Context) {
        term := g.Param("term")

        // Check if in cache
        item, cached := c.Get("autocomplete-company-" + term)

        if cached {
                g.String(200, item.(string))
                return
        }
// [...]

c.Get() returns two values: a prototype (the cache value) and a boolean, true if the key is stored. So if cached is true (note that in Go you don't need brackets) the item is already stored and we return the prototype value (use .(string) to access the value) and break the script.

If it isn't in cache, we retrieve the results from a GET request. In this example I use an endpoint to search companies offered free by clearbit.com, but you can use whatever you want, for example Google Maps API.

// [...]
    res, err := http.Get("https://autocomplete.clearbit.com/v1/companies/suggest?query=" + term)
    if err != nil {
        panic(err.Error())
    }

    defer res.Body.Close()
    body, err := ioutil.ReadAll(res.Body)
    results := string(body)
// [...]

With ioutil.ReadAll we fetch the response body that will be bytes and we convert it in a string with the string() function.

Now that we have results, we can store them in cache and send them in output. We'll send them as string because it's already a JSON-Encoded string, but if you want to JSON-Encode and send as output, Gin Tonic offers a way to do this.

// [...]  
// Save in cache  
    c.Set("autocomplete-company-" + term, results, cache.DefaultExpiration)
    g.String(200, results)
// [...]  

c.Set() is the method to store an item in cache and it need three params: a key, a value and an expiration time. with cache.DefaultExpiration we tell to Go-cache to use the default value set in the initialization (30 days). You can find out more on the expiration in the usage section of the package.

Now we're ready. The last thing to do is to run Gin Tonic a specific port. In this case we use the port 1234.

// [...]
r.Run(":1234")
// [...]

4. Ready!

Our simple script is finished and now it looks like this (you can get it also on this Pastebin):

package main

import (
    "gopkg.in/gin-gonic/gin.v1"
    "net/http"
    "io/ioutil"
    "github.com/patrickmn/go-cache"
    "time"
)

func main() {

    // Routes
    // gin.SetMode(gin.ReleaseMode)
    r := gin.Default()

    // Set cache default expiring to 30days and which purges expired items every 24h
    c := cache.New((24 * time.Hour) * 30 * 6, 24 * time.Hour)

    r.GET("/autocomplete/company/:term", func(g *gin.Context) {
        term := g.Param("term")

        // Check if in cache
        item, cached := c.Get("autocomplete-company-" + term)

        if cached {
            g.String(200, item.(string))
            return
        }

        res, err := http.Get("https://autocomplete.clearbit.com/v1/companies/suggest?query=" + term)
        if err != nil {
            panic(err.Error())
        }

        defer res.Body.Close()
        body, err := ioutil.ReadAll(res.Body)
        results := string(body)

        // Save in cache
        c.Set("autocomplete-company-" + term, results, cache.DefaultExpiration)
        g.String(200, results)
    })

    r.Run(":1234")
}

Now we need to build our project with the go install command.

$ go install github.com/DaniloPolani/autocomplete

This will create an executable script in the bin folder, so we can execute it:

$ $GOPATH/bin/autocomplete

This generates a view with a message about the debug mode and when we try to reach our script at localhost:1234/autocomplete/company/goo we can see the results JSON-Encoded in the browser, and in the terminal the request called.

When you're ready to go live, uncomment the gin.SetMode() line.

If you want to access it with the standard port, you'll need to create a reverse proxy. Don't worry, it's just a line with some command to install things.

Guide for Apache: http://www.microhowto.info/howto/configure_apache_as_a_reverse_proxy.html

Guide for NGINX: https://www.nginx.com/resources/admin-guide/reverse-proxy/

Check out part two!

{{-- intercom --}}