Introduction to Computer Vision in JavaScript using OpenCV.js

OpenCV is a powerful library used for image processing and image recognition. The library, Open-Source Computer Vision, has a massive community and has been used extensively in many fields, from face detection to interactive art. It was first built in C++ but bindings have since been created for different languages, such as Python and Java. It is even available in JavaScript (OpenCV.js), which is what we'll be using for this tutorial.

In this project, we will create a simple webpage where a user can upload an image in order to detect all the circles contained in it. We will highlight them with a black outline and the user will be able download the modified image. Thanks to OpenCV.js, this is a very easy task.

Throughout this tutorial, I won't discuss much about styling the page so feel free to add your own. Here is my Github repo with all the code in this post, plus my own styling using Bootstrap CSS.

Set up

Create a folder opencvjs-project and add an index.html file with the following basic template.

<!DOCTYPE html>
<html>
<head>
    <title>OpenCV.js</title>
</head>
<body>
    <!-- Our HTML will go here-->

<script type="text/javascript">
    // Our JavaScript code will go here
</script>

</body>
</html>

We will need to pull in the OpenCV.js library. The latest version can be built following the instructions found here, or you can copy the file for v3.3.1 from here (directly from the official OpenCV website) and save it locally as opencv.js. Add a script tag to the index.html file which references the local 'opencv.js' file. The script is quite large and takes a bit of time to load, so it is better load it asynchronously. This can be done by adding async to the script tag:

<script async src="opencv.js" type="text/javascript"></script>

As OpenCV.js won't be ready immediately, we can provide a better user experience by showing that the content is being loaded. I decided to add a loading spinner to my page (credit to Sampson), though you may prefer something else. In brief, I added a div tag at the bottom of the body and the following CSS into a separate style tag at the top of the page. The spinner is invisible by default (thanks to display: none;):

<body>
...
<div class="modal"></div>
<body>
/* display loading gif and hide webpage */
.modal {
    display:    none;
    position:   fixed;
    z-index:    1000;
    top:        0;
    left:       0;
    height:     100%;
    width:      100%;
    background: rgba( 255, 255, 255, .8) 
                url('http://i.stack.imgur.com/FhHRx.gif') 
                50% 50% 
                no-repeat;
}

/* prevent scrollbar from display during load */
body.loading {
    overflow: hidden;   
}

/* display the modal when loading class is added to body */
body.loading .modal {
    display: block;
}

To show the loading gif, we can add "loading" class to the body. Add the following to the top of the empty script.

document.body.classList.add("loading");

When Opencv.js loads, we'll want to hide the gif. Modify the script tag to add an onload event listener:

<script async src="opencv.js" onload="onOpenCvReady();" type="text/javascript"></script>

This allows us to remove the "loading" class:

// previous code is here

function onOpenCvReady() {
  document.body.classList.remove("loading");
}

Open the HTML page in your browser and check that OpenCV.js loads as expected.

Uploading the Image

Next step is to add an input tag so the user can upload an image:

<input type="file" id="fileInput" name="file" />

If we just want to display the source image, we'll also need to add an image tag and an event listener which responds to change on the input element. Copy the following element and place it under the input tag:

<img id="imageSrc" alt="No Image" />

Get both the image element and the input element using their IDs:

// previous code is here

let imgElement = document.getElementById('imageSrc');
let inputElement = document.getElementById('fileInput');

Now add the event listener which triggers when the input changes (i.e. when a file is uploaded). From the change event, it's possible to access the uploaded file (event.target.files[0]), and convert it into a URL using URL.createObjectURL. The image's src attribute can be updated to this URL:

// previous code is here

inputElement.onchange = function() {
  imgElement.src = URL.createObjectURL(event.target.files[0]);
};

Next to the original image, we can show a second image which we're going to modify. The image will be displayed with a canvas element which are used for drawing graphics with JavaScript:

<canvas id="imageCanvas" ></canvas>

We can add another event listener which updates the canvas with the uploaded image:

// previous code is here

imgElement.onload = function() {
  let image = cv.imread(imgElement);
  cv.imshow('imageCanvas', image);
  image.delete();
};

Detecting Cirlces

This is where the power of OpenCV is evident, as detecting circles becomes a very simple task. We want to find the circles when the user clicks a button, so we'll need to add the button and an event listener:

<button type="button" id="circlesButton" class="btn btn-primary">Circle Detection</button>
// previous code is here

document.getElementById('circlesButton').onclick = function() {
      // circle detection code
};

Depending on the image, circle detection may take a while so it is a good idea to disable the button to prevent the user from spamming it. It could also be useful to show a loading spinner on the button. I have simply reused the loading gif from the initial script load:

// previous code is here

document.getElementById('circlesButton').onclick = function() {
    this.disabled = true;
    document.body.classList.add("loading");

      // circle detection code

    this.disabled = false;
    document.body.classList.remove("loading");
};

The first step to detecting the circles is reading the image from the canvas.

In OpenCV, images are stored and manipulated as Mat objects. These are essentially matrices which hold values for each pixel in the image. For our circle detection, we're going to need three Mat objects. One to hold the source image (from which detect the circles) srcMat, one to store the circles we detect circlesMat and one to display to the user (on which we will draw our highlighted circles) displayMat. For the final Mat, we can make a copy of the first using the clone function:

let srcMat = cv.imread('imageCanvas');
let displayMat = srcMat.clone();
let circlesMat = new cv.Mat();

The srcMat needs to be converted to grayscale. This makes circle detection easier by simplifying the image. We can use cvtColor function to do this, which requires the source Mat (srcMat), the destination Mat (in this case the source and the destination Mat will be the same srcMat), and a value which refers to the colour conversion. cv.COLOR_RGBA2GRAY is the constant for grayscale:

cv.cvtColor(srcMat, srcMat, cv.COLOR_RGBA2GRAY);

The cvtColor function, like other OpenCV.js functions, accepts more parameters but these are not required and so will be set to the default. You can look at the documentation for better customisation.

Once the image is converted to grayscale, it's possible to use the HoughCircles function to detect the circles. This function needs a source Mat, srcMat, from where it'll find the circles and a destination Mat, circlesMat, where it'll store the circles. The other parameters required for the HoughCircles function are the method to detect circles (cv.HOUGH_GRADIENT), the inverse ratio of the accumulator resolution (1), and the minimum distance between the center point of circles (45). There are more parameters, thresholds for the algorithm (75 and 40), which can be played with to improve accuracy for your images. It is also possible to limit the range of the circles you want to detect by setting a minimum (0) and maximum radius (0).

cv.HoughCircles(srcMat, circlesMat, cv.HOUGH_GRADIENT, 1, 45, 75, 40, 0, 0);

Drawing Cirlces

All the circles which were detected can now be highlighted. We want to make an outline around each circle to show to the user. To draw a circle with OpenCV.js, we need the center point and the radius. These values are stored inside circlesMat and so we can retrieve it by looping through the matrix's columns:

for (let i = 0; i < circlesMat.cols; ++i) {
    // draw circles
}

The circlesMat stores the x and y values for the center point and the radius sequentially. So for the first circle, it would be possible to retrieve the values as follows:

let x = circlesMat.data32F[0];
let y = circlesMat.data32F[1];
let radius = circlesMat.data32F[2];

To get all the values for each circle, we can do the following:

for (let i = 0; i < circlesMat.cols; ++i) {
    let x = circlesMat.data32F[i * 3];
    let y = circlesMat.data32F[i * 3 + 1];
    let radius = circlesMat.data32F[i * 3 + 2];

    // draw circles
}

Finally, with all these values, we are able to draw outlines around the circles. We create a new Point for the center using the x and y values. To draw circles in OpenCV.js, we need a destination Mat (the image we're going to display to the user displayMat), the center Point, the radius value, and a scalar (an array of RBG values). There are also additional parameters which can be passed into circles, such as the line thickness which for this example is 3:

let center = new cv.Point(x, y);
cv.circle(displayMat, center, radius, [0, 0, 0, 255], 3);

All the code for drawing circles is as follows:

for (let i = 0; i < circlesMat.cols; ++i) {
    let x = circlesMat.data32F[i * 3];
    let y = circlesMat.data32F[i * 3 + 1];
    let radius = circlesMat.data32F[i * 3 + 2];
    let center = new cv.Point(x, y);
    cv.circle(displayMat, center, radius, [0, 0, 0, 255], 3);
}

Once we're done drawing all the circles on displayMat, we can show it to the user:

cv.imshow('imageCanvas', displayMat);

Finally, it's good practice to clean up the Mat objects which we'll no longer be needing. This is done to prevent memory problems:

srcMat.delete();
displayMat.delete();
circlesMat.delete();

Altogether the circle detection and drawing code looks like this:

// previous code is here

document.getElementById('circlesButton').onclick = function() {
    this.disabled = true;
    document.body.classList.add("loading");

    let srcMat = cv.imread('imageCanvas');
    let displayMat = srcMat.clone();
    let circlesMat = new cv.Mat();

    cv.cvtColor(srcMat, srcMat, cv.COLOR_RGBA2GRAY);

    cv.HoughCircles(srcMat, circlesMat, cv.HOUGH_GRADIENT, 1, 45, 75, 40, 0, 0);

    for (let i = 0; i < circlesMat.cols; ++i) {
        let x = circlesMat.data32F[i * 3];
        let y = circlesMat.data32F[i * 3 + 1];
        let radius = circlesMat.data32F[i * 3 + 2];

        let center = new cv.Point(x, y);
        cv.circle(displayMat, center, radius, [0, 0, 0, 255], 3);
    }

    cv.imshow('imageCanvas', displayMat);

    srcMat.delete();
    displayMat.delete();
    circlesMat.delete();

    this.disabled = false;
    document.body.classList.remove("loading");
};

Downloading the Image

After the image has been modified, the user may want to download it. To do this, add a hyperlink to your index.html file:

<a href="#" id="downloadButton">Download Image</a>

We set the href to the image URL and the download attribute to the image file name. Setting the download attribute indicates to the browser that the resource should be downloaded rather than navigating to it. We can create the image URL from the canvas using the function toDataURL(). Add the following JavaScript to the bottom of the script:

// previous code is here

document.getElementById('downloadButton').onclick = function() {
    this.href = document.getElementById("imageCanvas").toDataURL();
    this.download = "image.png";
};

Conclusion

Detecting circles is a very simple task with OpenCV. Once you get accustomed to manipulating images as Mat objects, there is so much more you can do. The HoughCircles algorithm is one of many provided by OpenCV to make image processing and image recognition that much easier. You can find more tutorials, including face recognition and template matching, on the OpenCV website. OpenCV has a lot of support in the different languages and I hope this tutorial shows just how easy it is to use in JavaScript.