Image of a computer. Image of a robot. Image of a blue piece. Image of a green piece.
UNDER CONSTRUCTION

I am currently revamping the website. Please come back later for projects and other content.

Deep Image Prior with ONNX.js

2020-12-26

Introduction

I have been hearing a lot about the Open Neural Network Exchange format (ONNX), which defines a format that allows for interoperability between multiple deep learning frameworks, such as PyTorch, Tensorflow, and MXNet. What got me really excited was ONNX.js, which basically allows neural networks to be run in the browser like Tensorflow.js (without the Tensorflow limitation), removing the need to host expensive servers in order to share some of my fun side projects.

This post was transplanted from my old website with the original post dated some time in 2018, where I experimented with Tensorflow.js in the original post, and I am swapping it out for ONNX.js for this updated version, because I favor PyTorch more than Tensorflow because the former provides a much more intuitive (and Pythonic) user experience.

Background

Neural networks are all the rage these days, and with techniques like batch normalization and residual connections that address the vanishing and exploding gradient problem, they are getting deeper and more effective ever.

However, one of the problems I keep running into is that it is not practical to expose these networks as services for personal projects, as the hardware required for reasonably speedy inference is too much. As a specific example, I tried hosting a convolutional neural network (CNN) for dog breed classification - the computation would timeout on both Heroku and AWS Lambda (API Gateway has a 30-second timeout), and I was only able to get it to work with a dedicated EC2 instance with 2+ vCPUs, which required a sizable chunk of change (for a student).

So, for the first interactive demo on my website, I thought it would be fun to implement Deep Image Prior by Ulyanov et all, which TL;DR, will de-noise an image of your choosing with a convolutional neural network in an encoder-decoder setup.

WARNING! If your laptop does not have a reasonable powerful dedicated GPU, the browser might freeze while this program is running!

Demo Time

CURRENTLY UNDER CONSTRUCTION