Getting Tensorflow 1.4 working with NVIDIA DIGITS in Ubuntu 16.04

The steps to follow are here.

In this post, I am not explaining all the bits and pieces on how to install, I am trying to avoid the confusion regarding what to follow.

Here is the main dependency, If you need to train tensorflow model with NVIDIA DIGITS, you need to get DIGITS 6. If you have DIGITS 5 installed, It won’t detect tensorflow. At the time of writing, unlike installing DIGITS 5, there are no binaries provided by NVIDIA to install DIGITS 6. You need to either install using docker or compile and install from source. I tried installing from docker, later I figured out that unless you are already familiar with docker, you are going to spend hell a lot of time trying to understand docker itself and its usage. Then there is nvidia-docker, which is the one actually needed for NVIDIA DIGITS 6. I tried for some time and realised that it is not required for me since I own the machine and I am the only person using it. I really am not ready to spend time on going through docker pain at this point of time.

Even though I am not a fan of compiling and installing, looks like that’s the only way. It is going to take some time and, you may need to fix some build failures, dependencies, stack-overflowing, googling etc. I followed the installation instructions from DIGITS github page.

Long story short,  you need to,

  1. Remove DIGITS 5 ( check here how to list and remove packages)
  2. compile and install Caffe (can not skip this, it is a dependency for DIGITS 6)
  3. compile and install Torch (not a requirement but let’s keep it)
  4. compile and install tensorflow_gpu (I had this already, so I skipped)
  5. compile and install DIGITS 6

Make sure you add this variables to ~/.bashrc

export DIGITS_ROOT=~/digits
export CAFFE_ROOT=~/caffe
export TORCH_ROOT=~/torch

The digits server can be invoked by ‘digits-devserver &‘. By default the service will be active at http://localhost:5000/
If everything goes fine, when you create a new model in DIGITS you can see the frameworks.tensorflow on NVIDIA digits 6

 

Video inferencing on neural network trained using NVIDIA DIGITS with opencv

I have been playing with the inferencing code for some time. Here is a real time video inferencing using opencv to capture video and slice through the frames. The overall frame rate is low due to the system slowness. In the video, ‘frame’ is the normalised image caffe network sees after reducing mean image file . ‘frame2’ is the input image.

Caffe model is trained in NVIDIA DIGITS using goolgleNet(SGD, 100 epoch), it reached 100% accuracy by 76 epoch.
NVIDIA DIGITS goolgleNet caffe inferencing

Here is the inferencing code.


import numpy as np
import matplotlib.pyplot as plt
import caffe
import time
import cv2
cap = cv2.VideoCapture(0)
from skimage import io

MODEL_FILE = './deploy.prototxt'
PRETRAINED = './snapshot_iter_4864.caffemodel'
MEAN_IMAGE = './mean.jpg'
#Caffe
mean_image = caffe.io.load_image(MEAN_IMAGE)
caffe.set_mode_gpu()
net = caffe.Classifier(MODEL_FILE, PRETRAINED,
channel_swap=(2,1,0),
raw_scale=255,
image_dims=(256, 256))
#OpenCv loop
while(True):
    start = time.time()
    ret, frame = cap.read()
    resized_image = cv2.resize(frame, (256, 256)) 
    cv2.imwrite("frame.jpg", resized_image)
    IMAGE_FILE = './frame.jpg'
    im2 = caffe.io.load_image(IMAGE_FILE)
    inferImg = im2 - mean_image
    #print ("Shape------->",inferImg.shape)
    #Inferencing
    prediction = net.predict([inferImg])
    end = time.time()
    pred=prediction[0].argmax()
    #print ("prediction -> ",prediction[0]) 
    if pred == 0:
       print("cat")
    else:
       print("dog")
    #Opencv display
    cv2.imshow('frame',inferImg)
    cv2.imshow('frame2',im2)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
cap.release()
cv2.destroyAllWindows()

 

 

Inferencing on the trained caffe model from NVIDIA DIGITS

With this post I will explain how to do inferencing on the trained network created with NVIDIA DIGITS through command line. link to the previous post here

In DIGITS UI, we have to upload a file into the model webpage to do inferencing. This is time consuming and not practical for real world appications. We need to deploy trained model as a standalone python application.
To achieve this we need to download the trained model from NVIDIA DIGITS model page. This will download a .tgz file to your computer. Open the .tgz file using this command

tar -xvzf filename.tgz
caffe model NVIDIA DIGITS
 

Save the ‘Image mean’ image file from datasets page of NVIDIA DIGITS in to your computer.

NVIDIA DIGITS inferencing

Provide path for,

'Image mean' file    -> eg:'/home/catsndogs/mean.jpg'
deploy.prototext ->eg:'/home/catsndogs/deploy.prototxt'
caffemodel ->eg:'/home/catsndogs/snapshot_iter_480.caffemodel'
input image to test ->eg:'/home/catsndogs/image_to_test.jpg'

in the below python script.

import numpy as np
import matplotlib.pyplot as plt
import caffe
import time
from PIL import Image

MODEL_FILE = '/home/catsndogs/deploy.prototxt'
PRETRAINED = '/home/catsndogs/snapshot_iter_480.caffemodel'
MEAN_IMAGE = '/home/catsndogs/mean.jpg'
# load the mean image
mean_image = caffe.io.load_image(MEAN_IMAGE)
#input the image file need to be tested
IMAGE_FILE = '/home/catsndogs/image_to_test.jpg'
im1 = Image.open(IMAGE_FILE)
# Tell Caffe to use the GPU
caffe.set_mode_gpu()
# Initialize the Caffe model using the model trained in DIGITS
net = caffe.Classifier(MODEL_FILE, PRETRAINED,
channel_swap=(2,1,0),
raw_scale=255,
image_dims=(256, 256))
# Load the input image into a numpy array and display it
plt.imshow(im1)
# Iterate over each grid square using the model to make a class prediction
start = time.time()
inferImg = im1.resize((256, 256), Image.NEAREST)
inferImg -= mean_image
prediction = net.predict([inferImg])
end = time.time()
print(prediction[0].argmax())
pred=prediction[0].argmax()
if pred == 0: 
  print("cat")
else: 
  print("dog")
# Display total time to perform inference
print 'Total inference time: ' + str(end-start) + ' seconds'

Run the file with

python catsndogs.py

for inferencing.

Train a neural network with custom images using NVIDIA DIGITS on your machine

It was a surprise when NVIDIA DIGITS framework was up and running in my 10 year old machine in no time. So I decided I start my blog with a tutorial about it.

Until now I was hand coding neural networks in tensorflow/tflearn and feeding data directly into the network. There is less time to play with the network and optimise while majority of the time will be spend to make it run.

NVIDIA DIGITS, is a framework which gives a nice UI to create your network, without even touching the code, unless u really want to. It does the heavy lifting like parsing test/train data and creating input database. I have no idea how it arranges the data, If you have used sklearn, you used to do the hard work arranging data in to arrays, so you will be clear of each and every bit. I will extend the tutorial as I figure out things. DIGITS then create models/network from existing models and this can be customised.

I will explain here, how to create/train a network on your machine which can detect cat and dogs from images using data from kaggle, download the data here. Extract the tar file and arrange the data according to Step2.

Step 1: Signup for NVIDIA developer network and download and install NVIDIA DIGITS.

I downloaded DIGITS 5 since I had a previous installation of CUDA 8. At the time of writing this blog, there was no local installer for DIGITS 6. A follow up tutorial will be there when I move to DIGITS on Docker. That will happen when I upgrade my machine. Currently I use an old machine with Pentium D desktop paired with a GTX 1050 Ti running on Ubuntu 16.04.3. I am waiting to upgrade to Intel kabylake, So I have to upgrade motherboard, RAM and CPU. When this happens, I have to have a fresh installation of all(CUDA/CuDNN/Python3/OpenCV/tensorflow/Anaconda/Arduino etc). So there will be more tutorials to follow.

NVIDIA DIGITS

This will download a deb file, install it using the below commands.
sudo dpkg -i nv-deep-learning-repo-ubuntu1604-ga-cuda8.0-digits5.0_1-1_amd64.deb
sudo apt-get update
sudo apt-get install digits

This will start DIGITS on a local webserver, which can be accessed by going to http://localhost

NVIDIA DIGITS framework

 

Step 2: Create database for training from the images you have.

There should be train and validate directories. Under train/validate, there should be directories for each separate class which contain the images. Here, cats and dogs are the two classes.

Click on DIGITS webpage -> Datasets->Images dropdown->Classification
Fill in, path to test images and validation images, give a name and click create. This will do parsing of images and creating a database for feeding it into the neural network model.

NVIDIA DIGITS new dataset creation

Step3: Create a model and train, thats all!!

Click on DIGITS webpage -> Models->Images dropdown->Classification
Choose Alexnet or Googlenet, give a name and click create. This will create your neural network in caffe framework and it will start training. The parameters in ‘Solver options’ and ‘Data transformations’ can be changed to try with different optimizers. the network layers can be edited by clicking ‘Customize’.

NVIDIA DIGITS new model creation

I got an ERROR for Out of memory, but it got resolved when I recreated the model. If anyone knows about the reason please let me know in the comments. My system swap space was changed to 30GB so it should not fail, I assume it was because of GPU memory availability at 4GB. Maybe we should be able to redirect to system memory or I have to upgrade my GPU. Anyway it worked for my surprise.

NVIDIA DIGITS Error out of memory

The CPU utilisation shoots up to 164%!!, I have no idea why, may be the CPU is a severe bottleneck.

NVIDIA DIGITS cats and dogs training

 

Step4: Test the network with a random image.

While training, we can observe the accuracy and loss in the graph. Once the training is over, upload a random image and click on ‘Classify one’ to inference.

NVIDIA DIGITS inference

And It works!

NVIDIA DIGITS inference

This is so fast considering the amount of debugging time spent  when I write by myself. The whole process took me about 20 mins till the output. In the following post I will try to explain how to make it standalone python program.