Linux

Removing noise from video file in Linux using Audacity and kdenlive

December 23, 2017February 4, 2018 Posted in Editing, Ubuntu. 1 Comment

Note: This is in Ubuntu 16.04

Removing the static noise picked up by the microphone is essential before uploading youtube videos since youtube does not allow editing audio track after the upload. It stopped allowing annotations in videos too, so the only way to remove noise after uploading is to take down the video and upload a new video. Unfortunately I had to do the same, just explaining here how to do the editing using freeware in ubuntu.

To remove noise, we need two software,

1)Audacity – Install it from Ubuntu Software

2)kdelive

sudo add-apt-repository ppa:sunab/kdenlive-release
sudo apt-get update
sudo apt-get install kdenlive

Audacity:

Initial step is two create a audio file which is free from noise from the video file using Audacity.
Open Audacity-> File->Open (open video file)

Select noise as shown in fig above; Effect->Noise Reduction: Step 1 – Get Noise Profile (this will tell audacity which is the noise signature)

Deselect the noise (click anywhere outside the selection); Effect->Noise Reduction: Step 2 – OK (keep default settings or change Noise reduction settings in db for more attenuation)

File->Export Audio->Save as noise_free.mp3

kdelive:

We use kdelive to merge the noise free audio track to the video track.

Open kdelive,
menu->Projects->Add clip->Add the original video with noisy audio
menu->Projects->Add clip->Add the noise free audio file (noise_free.mp3)

Drag the video to Video 1, drag noise_free.mp3 to Audio 1, mute Video 1 audio track. Go to project Monitor as press play button to check the new audio.

To save the new video Click on Render-> Render into a file (keep default settings or change settings whichever needed for file format)

See the comparisons below.

<br />	<br />
with noise	noise removed

Getting Tensorflow 1.4 working with NVIDIA DIGITS in Ubuntu 16.04

December 11, 2017December 11, 2017 Posted in Caffe, Machine Learning, NVIDIA DIGITS, Tensorflow, Ubuntu. Leave a comment

The steps to follow are here.

In this post, I am not explaining all the bits and pieces on how to install, I am trying to avoid the confusion regarding what to follow.

Here is the main dependency, If you need to train tensorflow model with NVIDIA DIGITS, you need to get DIGITS 6. If you have DIGITS 5 installed, It won’t detect tensorflow. At the time of writing, unlike installing DIGITS 5, there are no binaries provided by NVIDIA to install DIGITS 6. You need to either install using docker or compile and install from source. I tried installing from docker, later I figured out that unless you are already familiar with docker, you are going to spend hell a lot of time trying to understand docker itself and its usage. Then there is nvidia-docker, which is the one actually needed for NVIDIA DIGITS 6. I tried for some time and realised that it is not required for me since I own the machine and I am the only person using it. I really am not ready to spend time on going through docker pain at this point of time.

Even though I am not a fan of compiling and installing, looks like that’s the only way. It is going to take some time and, you may need to fix some build failures, dependencies, stack-overflowing, googling etc. I followed the installation instructions from DIGITS github page.

Long story short, you need to,

Remove DIGITS 5 ( check here how to list and remove packages)
compile and install Caffe (can not skip this, it is a dependency for DIGITS 6)
compile and install Torch (not a requirement but let’s keep it)
compile and install tensorflow_gpu (I had this already, so I skipped)
compile and install DIGITS 6

Make sure you add this variables to ~/.bashrc

export DIGITS_ROOT=~/digits
export CAFFE_ROOT=~/caffe
export TORCH_ROOT=~/torch

The digits server can be invoked by ‘digits-devserver &‘. By default the service will be active at http://localhost:5000/
If everything goes fine, when you create a new model in DIGITS you can see the frameworks.

Speech detection with Tensorflow 1.4 on Raspberry Pi 3 – Part 2: Live audio inferencing using PyAudio

December 10, 2017December 23, 2017 Posted in Artificial intelligence, Embedded, Inferencing, Linux, Machine Learning, Raspberry Pi, Robotics, Tensorflow. 7 Comments

Here is link to Part 1

Now we know, how to loop around the inferencing function, capture a voice for a fixed time and process it. What we need now is a program to listen to the input stream and measure the audio level. This will help us to take a decision if we need to capture the audio data or not.

File1: audio_intensity.py
The following code, reads a CHUNK of data from the stream and measure average intensity, prints it out so that we will know how much ambient noise is there in the background. First we need to figure out the average intensity level (INTENSITY) so that we will get a threshold number to check for.

import pyaudio
import wave
import math
import audioop
import time
 
p = pyaudio.PyAudio() 
FORMAT = pyaudio.paInt16
CHANNELS = 1
RATE = 16000
CHUNK = 512 
RECORD_SECONDS = 1
WAVE_OUTPUT_FILENAME = "file.wav"
INTENSITY=11000
 
def audio_int(num_samples=50):
    print ('Getting intensity values from mic.')
    p = pyaudio.PyAudio()

    stream = p.open(format=FORMAT,
                    channels=CHANNELS,
                    rate=RATE,
                    input=True,
                    frames_per_buffer=CHUNK)
    #----------------------checks average noise level-------------------------
    cur_data = stream.read(CHUNK)
    values = [math.sqrt(abs(audioop.avg(cur_data, 4)))
                for x in range(num_samples)]
    values = sorted(values, reverse=True)
    r = sum(values[:int(num_samples * 0.2)]) / int(num_samples * 0.2)
    #---------------------prints out avg noise level--------------------------
    print (' Average audio intensity is r', r)
    time.sleep(.1)

    stream.close()
    p.terminate()
    return r


if(__name__ == '__main__'):
    while (True):
    audio_int()  # To measure your mic levels

File 2: audio_intensity_trigger.py
In this program, I have added an infinite loop and a check for INTENSITY level before printing the average audio level. If the room is silent or just background noise nothing is triggered. I have kept is as ‘11000’. Make sure that you change it according to output of audio_intensity.py. If its output is, say 8000, keep the intensity as 9000 or 10000.

......
......
......
while True:
  cur_data = stream.read(CHUNK)
  values = [math.sqrt(abs(audioop.avg(cur_data, 4)))
            for x in range(num_samples)]
  values = sorted(values, reverse=True)
  r = sum(values[:int(num_samples * 0.2)]) / int(num_samples * 0.2)
  #print " Finished "
  if (r > INTENSITY):
    print (' Average audio intensity is r', r)

stream.close()
......
......

File 3: audio_trigger_save_wav.py
This one will wait for the threshold and once triggered, it will save 1 second of audio to a file in wave format together with 5 frames of previous voice chunks. This is important, otherwise our recording will not contain the starting of words or the words will be biased towards first half of 1 second and remaining half will be empty. The spectrogram when generated by tensorflow will looked chopped off.

......
......
......
    prev_data0=[]
    prev_data1=[]
    prev_data2=[]
    prev_data3=[]
    prev_data4=[]
    while True:
      #reading current data
      cur_data = stream.read(CHUNK)
      values = [math.sqrt(abs(audioop.avg(cur_data, 4)))
                for x in range(num_samples)]
      values = sorted(values, reverse=True)
      r = sum(values[:int(num_samples * 0.2)]) / int(num_samples * 0.2)
      if (r > INTENSITY):
        #-------------------------------------------------if triggered; file.wav = 5 previous frames + capture 1 sec of voice-------------------------------
        print (' Average audio intensity is r', r)
        frames = []
        frames.append(prev_data0)
        frames.append(prev_data1)
        frames.append(prev_data2)
        frames.append(prev_data3)
        frames.append(prev_data4)
        frames.append(cur_data)
        #---------------getting 1 second of voice data-----------------
        for i in range(0, int(RATE / CHUNK * RECORD_SECONDS)):
          data = stream.read(CHUNK)
          frames.append(data)
        print ('finished recording')
        #-------------     ---saving wave file-------------------------
        waveFile = wave.open(WAVE_OUTPUT_FILENAME, 'wb')
        waveFile.setnchannels(CHANNELS)
        waveFile.setsampwidth(p.get_sample_size(FORMAT))
        waveFile.setframerate(RATE)
        waveFile.writeframes(b''.join(frames))
        waveFile.close()
      #------------------------------------------------------if not triggered; saving previous values to a FIFO of 5 levels----------------------------------
      prev_data0=prev_data1
      prev_data1=prev_data2
      prev_data2=prev_data3
      prev_data3=prev_data4
      prev_data4=cur_data
     stream.close()
......
......
......

File 4: wav_trigger_inference.py
This is the modified tensorflow inference file (label_wav.py). I have fused the program audio_trigger_save_wav.py to label_wav.py. The usage is,

cd /tensorflow/examples/speech_commands
touch file.wav ; to create a dummy file for the first pass
python3 wav_trigger_inference.py --graph=./my_frozen_graph.pb --labels=./conv_labels.txt --wav=file.wav

The while loop is around run_graph(). If the audio is detected and is above threshold; wave file is captured and given for inferencing. Once the results are printed out, it continue listening for the next audio.

....
....
....
      waveFile.writeframes(b''.join(frames))
      waveFile.close()
      with open(wav, 'rb') as wav_file:
        wav_data = wav_file.read()
      run_graph(wav_data, labels_list, input_name, output_name, how_many_labels)
    prev_data0=prev_data1
    prev_data1=prev_data2
....
....
....
 parser.add_argument(
      '--how_many_labels',
      type=int,
      default=1,# -------------------this will make use that, it prints out only one result with max probability------------------------
      help='Number of results to show.')
....
....
....

Here is the result. There are some errors while processing since the graph is not accurate. I could train it only till 88% accuracy. More data argumentation is needed for improving the accuracy and I may need to fiddle around with all the switches that is provided by tensorflow for training. But this is good enough to create a speech controlled device using raspberry pi.

Setup and installation for machine learning with CUDA-8.0.61 and cuDNN-6 on Ubuntu 16.04 LTS- Part 2

November 24, 2017November 24, 2017 Posted in CUDA, cuDNN, Machine Learning, Tensorflow, Ubuntu. Leave a comment

Tensorflow installation is as simple as running few commands if you have the correct version of CUDA and cuDNN.

To start with I will explain how to uninstall the previous version of CUDA/cuDNN which is installed in Part-1. It is important to know how to configure the installation since this utils can break anytime due to version changes and frequent updates. So every time reinstalling OS is not a solution.

To remove nvidia drivers use this command,

sudo /usr/bin/nvidia-uninstall
sudo apt-get remove --purge nvidia-*
sudo apt-get --purge remove nvidia-cuda*

Just to make sure, try listing out the packages,


apt list --installed | grep cuda

and uninstall each package by,

sudo apt-get remove <package>

Disable nouveau driver(free driver for nvidia cards comes with ubuntu) for nvidia driver installation.

Edit this file,

vi /etc/modprobe.d/blacklist-nouveau.conf

with this content,

blacklist nouveau
options nouveau modeset=0

Regenerate the kernel initramfs:(initramfs is used to mount root file system / while boot)

sudo update-initramfs -u
sudo reboot

Reboot system.

Install CUDA-8.0 and cuBLAS patch form .deb file downloaded from NVIDIA CUDA archives.

sudo dpkg -i cuda-repo-ubuntu1604-8-0-local-ga2_8.0.61-1_amd64.deb
sudo apt-get update
sudo apt-get install cuda-8.0
sudo dpkg -i cuda-repo-ubuntu1604-8-0-local-cublas-performance-update_8.0.61-1_amd64.deb
sudo apt-get update
sudo apt-get upgrade cuda-8.0

Once the installation is done, install cuDNN 6. Download .deb file form cuDNN download page and install. Install, runtime library, development library and code samples.


sudo dpkg -i libcudnn6_6.0.21-1+cuda8.0_amd64.deb
sudo dpkg -i libcudnn6-dev_6.0.21-1+cuda8.0_amd64.deb
sudo dpkg -i libcudnn6-doc_6.0.21-1+cuda8.0_amd64.deb

Add this to .bashrc

export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

Go to cuDNN samples directory and compile the sample program.

cp /usr/src/cudnn_samples_v6 ~/.
cd ~/cudnn_samples_v6/mnistCUDNN
make clean
make
./mnistCUDNN

If you get this error,

cudnnGetVersion() : 6021 , CUDNN_VERSION from cudnn.h : 6021 (6.0.21)
Host compiler version : GCC 5.4.0
There are 1 CUDA capable devices on your machine :
device 0 : sms 6 Capabilities 6.1, SmClock 1417.5 Mhz,
 MemSize (Mb) 4035, MemClock 3504.0 Mhz, Ecc=0, boardGroupID=0
Using device 0

Testing single precision
CUDNN failure
Error: CUDNN_STATUS_INTERNAL_ERROR
mnistCUDNN.cpp:394
Aborting...

just run as root,

Test Passed!! You now have CUDA-8.0 with cuDNN-6

To install tensorflow, execute this commands. For python 2.7,

sudo apt-get install libcupti-dev

sudo apt-get install python-pip python-dev

pip install tensorflow-gpu

or for python 3.5,

sudo apt-get install libcupti-dev

sudo apt-get install python3-pip python3-dev

pip3 install tensorflow-gpu

After installation, test by calling sample program,

# Python
import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))

If everything is installed correctly, this will print out the GPU device tensorflow is running.

Setup and installation for machine learning with CUDA and cuDNN on Ubuntu 16.04 LTS- Part 1

November 24, 2017November 24, 2017 Posted in CUDA, cuDNN, Machine Learning, Ubuntu. Leave a comment

Important: This is to install CUDA 9.0 with CuDNN 7, this will not work with tensorflow 1.4(at the time of writing). I realized this after the installation. I will go through tensorflow 1.4 with CUDA 8 and cuDNN 6 in the next post.

Tensorflow 1.4 release notes

All our prebuilt binaries have been built with CUDA 8 and cuDNN 6. We anticipate releasing TensorFlow 1.5 with CUDA 9 and cuDNN 7.

Step 1: Download ubuntu .iso from ubuntu/downloads. This will download a .iso image file to your PC. In my case, the file is ubuntu-16.04.03-desktop-amd64.iso

Step 2: Create bootable USD stick or burn a dvd from the image for installation. Install Ubuntu on the PC. I have downloaded .iso in Windows 10 and used the default dvd writer program to burn to a disk.

Step 3: Boot into the fresh installation. Open a terminal in ubuntu, update the installation.

sudo apt-get update

Step 4: Download NVIDIA drivers.

I have updated the drivers through additional drivers menu in ubuntu.
Make sure that you have, NVIDIA graphics driver 384.81 or newer for CUDA 9.

Step 5: Download and Install CUDA.

Method 1 : Download the .run file (cuda_9.0.176_384.81_linux.run for ubuntu 16.04).

Press ctrl+alt+f1 to stop X server and go to tty mode, execute the command.

sudo sh cuda_9.0.176_384.81_linux.run

Accept the licence terms, skip install driver which comes with it (at least it did not work for me), install OpenGL driver, allow permission to manage x server configuration, accept all default paths.

If the driver installation fails, got to /tmp and remove the X server lock files, retry the installation.

cd /tmp
rm -rf .X*

Press ctrl+alt+f7 to return to login screen, once the installation completes.
Install third party lib for building CUDA samples,

sudo apt-get install g++ freeglut3-dev build-essential libx11-dev libxmu-dev libxi-dev libglu1-mesa libglu1-mesa-dev

Go to the samples directory, eg:/usr/local/cuda/samples/5_Simulations/particles and try,

sudo make

if everything goes well it will compile and create an executable, run it by,

./particles

this will show the demo application

To check device status,

/usr/local/cuda-8.0/samples/1_Utilities/deviceQuery$ ./deviceQuery

To uninstall CUDA, if something goes wrong, got to /usr/local/cuda/bin and run the uninstall script.

The default installation path will be /usr/local/cuda/

Method 2: installing from .deb


sudo dpkg -i cuda-repo-ubuntu1604-9-0-local_9.0.176-1_amd64.deb
sudo apt-get update
sudo apt-get install cuda

Try compiling the sample program to check if CUDA is installed fine.

Add this to .bashrc

export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

Step 6: Install cuDNN.

Method 1: Download .deb file form cuDNN download page and install. Install, runtime library, development library and code samples.


sudo dpkg -i libcudnn7_7.0.3.11-1+cuda9.0_amd64.deb
sudo dpkg -i libcudnn7-dev_7.0.3.11-1+cuda9.0_amd64.deb
sudo dpkg -i libcudnn7-doc_7.0.3.11-1+cuda9.0_amd64.deb

Method 2 : install for downloaded tar file, cudnn-9.0-linux-x64-v7.tgz

If there is any error associated with running cuDNN, check the libcudnn*.so* files are present in /usr/local/cuda/lib64 and cudnn.h file is present in /usr/local/cuda/include

If you are installing form a tar file, cuDNN can be installed by simply copying these files to respective folder of CUDA installation.


tar -xzvf cudnn-9.0-linux-x64-v7.tgz
sudo cp cuda/include/cudnn.h /usr/local/cuda/include
sudo cp cuda/lib64/libcudnn* /usr/local/cuda/lib64
sudo chmod a+r /usr/local/cuda/include/cudnn.h /usr/local/cuda/lib64/libcudnn*

Go to cuDNN samples directory and compile the sample program.


cd /usr/src/cudnn_samples_v7/conv_sample

sudo make clean

sudo make

This will compile the code and show the result and we can verify the cuDNN installation.


$ sudo ./conv_sample
Testing single precision
Testing conv
^^^^ CUDA : elapsed = 4.41074e-05 sec,
Test PASSED
Testing half precision (math in single precision)
Testing conv
^^^^ CUDA : elapsed = 4.00543e-05 sec,
Test PASSED

Cool !! CUDA 9.0 with cuDNN 7 is installed in your system.

Support and documentation.

CUDA developer zone

NVIDIA Linux Display driver archive

I got an error while compiling the mnist code sample, not sure what is the issue, just pasting the error below,

 
/usr/src/cudnn_samples_v7/mnistCUDNN$ sudo make
/usr/local/cuda/bin/nvcc -ccbin g++ -I/usr/local/cuda/include -IFreeImage/include  -m64    -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_53,code=sm_53 -gencode arch=compute_53,code=compute_53 -o fp16_dev.o -c fp16_dev.cu
g++ -I/usr/local/cuda/include -IFreeImage/include   -o fp16_emu.o -c fp16_emu.cpp
g++ -I/usr/local/cuda/include -IFreeImage/include   -o mnistCUDNN.o -c mnistCUDNN.cpp
In file included from /usr/local/cuda/include/channel_descriptor.h:62:0,
                 from /usr/local/cuda/include/cuda_runtime.h:90,
                 from /usr/include/cudnn.h:64,
                 from mnistCUDNN.cpp:30:
/usr/local/cuda/include/cuda_runtime_api.h:1683:101: error: use of enum ‘cudaDeviceP2PAttr’ without previous declaration
  __cudart_builtin__ cudaError_t CUDARTAPI cudaDeviceGetP2PAttribute(int *value, enum cudaDeviceP
                                                                                      ^
/usr/local/cuda/include/cuda_runtime_api.h:2930:102: error: use of enum ‘cudaFuncAttribute’ without previous declaration
 __cudart_builtin__ cudaError_t CUDARTAPI cudaFuncSetAttribute(const void *func, enum cudaFuncAtt
                                                                                      ^
In file included from /usr/local/cuda/include/channel_descriptor.h:62:0,
                 from /usr/local/cuda/include/cuda_runtime.h:90,
                 from /usr/include/cudnn.h:64,
                 from mnistCUDNN.cpp:30:
/usr/local/cuda/include/cuda_runtime_api.h:5770:92: error: use of enum ‘cudaMemoryAdvise’ without previous declaration
  __host__ cudaError_t CUDARTAPI cudaMemAdvise(const void *devPtr, size_t count, enum cudaMemoryA
                                                                                      ^
/usr/local/cuda/include/cuda_runtime_api.h:5827:98: error: use of enum ‘cudaMemRangeAttribute’ without previous declaration
 t__ cudaError_t CUDARTAPI cudaMemRangeGetAttribute(void *data, size_t dataSize, enum cudaMemRang
                                                                                      ^
/usr/local/cuda/include/cuda_runtime_api.h:5864:102: error: use of enum ‘cudaMemRangeAttribute’ without previous declaration
 cudaError_t CUDARTAPI cudaMemRangeGetAttributes(void **data, size_t *dataSizes, enum cudaMemRang
                                                                                      ^
Makefile:200: recipe for target 'mnistCUDNN.o' failed
make: *** [mnistCUDNN.o] Error 1

Video inferencing on neural network trained using NVIDIA DIGITS with opencv

November 6, 2017November 24, 2019 Posted in Artificial intelligence, Caffe, Inferencing, Machine Learning, NVIDIA DIGITS, python, Ubuntu. Leave a comment

I have been playing with the inferencing code for some time. Here is a real time video inferencing using opencv to capture video and slice through the frames. The overall frame rate is low due to the system slowness. In the video, ‘frame’ is the normalised image caffe network sees after reducing mean image file . ‘frame2’ is the input image.

Caffe model is trained in NVIDIA DIGITS using goolgleNet(SGD, 100 epoch), it reached 100% accuracy by 76 epoch.

Here is the inferencing code.


import numpy as np
import matplotlib.pyplot as plt
import caffe
import time
import cv2
cap = cv2.VideoCapture(0)
from skimage import io

MODEL_FILE = './deploy.prototxt'
PRETRAINED = './snapshot_iter_4864.caffemodel'
MEAN_IMAGE = './mean.jpg'
#Caffe
mean_image = caffe.io.load_image(MEAN_IMAGE)
caffe.set_mode_gpu()
net = caffe.Classifier(MODEL_FILE, PRETRAINED,
channel_swap=(2,1,0),
raw_scale=255,
image_dims=(256, 256))
#OpenCv loop
while(True):
    start = time.time()
    ret, frame = cap.read()
    resized_image = cv2.resize(frame, (256, 256)) 
    cv2.imwrite("frame.jpg", resized_image)
    IMAGE_FILE = './frame.jpg'
    im2 = caffe.io.load_image(IMAGE_FILE)
    inferImg = im2 - mean_image
    #print ("Shape------->",inferImg.shape)
    #Inferencing
    prediction = net.predict([inferImg])
    end = time.time()
    pred=prediction[0].argmax()
    #print ("prediction -> ",prediction[0]) 
    if pred == 0:
       print("cat")
    else:
       print("dog")
    #Opencv display
    cv2.imshow('frame',inferImg)
    cv2.imshow('frame2',im2)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
cap.release()
cv2.destroyAllWindows()

Recreate GRUB in ubuntu, to fix boot error: vfs unable to mount root fs on unknown-block

November 6, 2017 Posted in Linux, Ubuntu. Leave a comment

Most of the time I encounter boot issue in ubuntu and tend to loose data because of it. This is very time consuming and a hindrance to the work. There is a nifty technique to get over ‘ERROR vfs unable to mount root fs on unknown-block’ by reinstalling GRUB boot-loader.

To reinstall, you need a live CD or download and write a bootable USB drive for the same version of ubuntu. Once booted in to the OS, open a terminal and install boot-repair.

sudo add-apt-repository ppa:yannubuntu/boot-repair
sudo apt-get update
sudo apt-get install -y boot-repair && boot-repair

Open boot-repair by typing boot-repair in terminal.

According to majority of posts, you can get over the issue by trying ‘Recommended repair’ and reboot the machine. I tried it and it did not work for me. So I navigated to Advanced options->GRUB options->Reset GRUB to its most recent version

Even though upgrading GRUB installs newest version of GRUB irrespective of the current version of OS(ubuntu 17.10). After reboot, the system booted to the existing ubuntu 16.04.3 installation.

December 2019
S	M	T	W	T	F	S
« May
1	2	3	4	5	6	7
8	9	10	11	12	13	14
15	16	17	18	19	20	21
22	23	24	25	26	27	28
29	30	31