Implementing Keras model in Tensorflow

Deep learning is the latest trend in the field of artificial intelligence. Deep learning based models have produced incredible results in the fields such as computer vision, natural language processing, robotics etc. Tensorflow and Keras are two popular frameworks used for building the deep neural networks used in deep learning. Tensorflow is one of the advanced software libraries used for implementing deep learning models whereas Keras is a high level abstract library that runs on top of tensorflow.

In this blog, we will see a simple implementation of using a Keras model with Tensorflow backend in a Tensorflow serving framework. The Jupyter notebook implementation of the code can be found in github.

First of all import the libraries required.

## Imports
import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data
from keras.layers import Dense, Dropout, Flatten, MaxPooling2D, Conv2D, Input
from keras.models import Model
import numpy as np
import pylab as plt
import os

Next define values for the variables used in the program.

## Defining variables
batch_size=512
nb_classes=10
lr_rate=.0005
height=28
width=28
channel=1
input_shape=[height,width,channel]
iterations=500

Then load the MNIST dataset.

mnist = input_data.read_data_sets('MNIST_data', one_hot=True)

Now the model has to be defined in Keras. For that, first the placeholder variables are defined in tensorflow. The data placeholder variable will be used for serving the input data and the target placeholder variable will be used for feeding the expected target values.

## Defining placeholders in tf
data = tf.placeholder(tf.float32, [None, height,width,channel]) #step_size=No: of frames in video sequence
target = tf.placeholder(tf.float32, [None, nb_classes])

Next the model has to be defined in Keras. A simple CNN is used as an example in this program. The model is defined using Keras functional API.

## Defining the model in keras using functional layers
input_layer = Input(shape=input_shape)
layer1= Conv2D(32, kernel_size=(3, 3),activation='relu')(input_layer)
layer2=Conv2D(64, (3, 3), activation='relu')(layer1)
layer3=MaxPooling2D(pool_size=(2, 2))(layer2)
layer4=Flatten()(layer3)
layer5=Dense(128, activation='relu')(layer4)
output_layer=Dense(nb_classes, activation='linear')(layer5)
model=Model(input_layer,output_layer)
print(model.summary())
out=model(data)

After that the remaining part is defined as normal in tensorflow. This includes defining the cross entropy loss function, Adam optimiser for optimisation of the network and defining accuracy.

## Making optimisation method, loss function and calculating accuracy
predictions= tf.nn.softmax(out)
cross_entropy=tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=target,logits=out))
optimizer = tf.train.AdamOptimizer(lr_rate)
minimize = optimizer.minimize(cross_entropy)
mistakes = tf.equal(tf.argmax(target, 1), tf.argmax(predictions, 1))
accuracy = tf.reduce_mean(tf.cast(mistakes, tf.float32))

Once the model and the related operations are defined, the model has to be trained. The training is done using the tensorflow serving mechanism. First a session is started, the variables are initialised and then the network is trained by feeding the the input and target data batch by batch.

## The keras model is trained as a tensorlfow graph
init_op = tf.global_variables_initializer()
sess = tf.InteractiveSession()
sess.run(init_op)

for i in range(iterations):
    batch_x, batch_y = mnist.train.next_batch(batch_size)
    batch_x = batch_x.reshape(batch_x.shape[0], height, width, channel)
    sess.run([minimize],{data: batch_x, target: batch_y})
    print('Iteration {}'.format(i))

And finally the trained model is tested and session is closed.

mnist_test = mnist.test.images.reshape(mnist.test.images.shape[0], height, width, channel)
acc=sess.run(accuracy, feed_dict={data:mnist_test ,target: mnist.test.labels})
print('accuracy on test set: {}'.format(acc))

There are also other ways of using Keras alongside tensorflow. Refer to this blog for further reading.

Identifying Coloured Objects Using OpenCv

Hey everyone , in this blog we will be learning how to create a mask for a particular colour  and extract the particular coloured object from an image.

Algorithm used:

1) Read the BGR image
2) Convert the image into HSV color space
3) Create a color mask by setting the upper and lower thresholds from the hsv image
4) Filter the original BGR  image using the color mask

Sample Code :

import cv2
import numpy as np

img = cv2.imread("tomato.jpg",-1)
cv2.imshow("orginal",img)

blur=cv2.GaussianBlur(img,(5,5),0)
cv2.imshow("blurred image",blur)
hsv=cv2.cvtColor(blur,cv2.COLOR_BGR2HSV)

lower=np.array([0,100,100])
upper=np.array([20,255,255])
mask=cv2.inRange(hsv,lower,upper)
cv2.imshow("mask",mask)
out=cv2.bitwise_and(img,img,mask=mask)
cv2.imshow("masked image",out)

cv2.waitKey(0)

Explanation:

The sample code above separates all the red coloured objects from your image.

import cv2
import numpy as np

img = cv2.imread("tomato.jpg",-1)
cv2.imshow("orginal",img)

The above lines are for reading the original image with the name tomato.jpg stored in the working directory.In case the image is stored in some other directory you have to give the complete path.

blur=cv2.GaussianBlur(img,(5,5),0)
cv2.imshow("blurred image",blur)
hsv=cv2.cvtColor(blur,cv2.COLOR_BGR2HSV)

The first two lines removes the noise in the image using a gaussian filter and displays the blurred image. Next line converts the image into into hsv color space.For more details of the conversion visit my blog.

lower=np.array([0,100,100])
upper=np.array([20,255,255])
mask=cv2.inRange(hsv,lower,upper)
cv2.imshow("mask",mask)

The lines #1,#2 sets the lower and upper threshold for the mask.The hue value is selected to be in between 0-20 since we want to get the red objects.If we want to select some other colour you can do it by just changing the hue value here.The value for saturation and value are set to be 100-255 to include all tones and shades of red.The line #3 creates a mask from the hsv image.Line #4 displays the mask.

out=cv2.bitwise_and(img,img,mask=mask)
cv2.imshow("masked image",out)
cv2.waitKey(0)

The line 1 use a bitwise 'and' operation to filter the original BGR image using the mask and return a masked image in which only the red objects will be shown.

Original Image

Blurred Image

Masked Image

Mask

 Hoping the blog was helpful.Take care & Continue Hacking ;-)

Colour Spaces and their conversions in opencv

In this blog we will be dealing with various color spaces and their conversions using opencv functions.

Introduction to Color spaces

According to definitions given in wikipedia the colorspace of an image is defined as: 

"Color space, also known as the color model (or color system), is an abstract mathematical model which simply describes the range of colors as tuples of numbers, typically as 3 or 4 values or color components "

For example in a RGB image every color is defined as a combination of three basic colors: red,green and blue.The color of a point changes as the value of red,green and blue component changes.(0,0,0) represents perfect black (255,255,255) represents white.One thing to be taken into account always is that in opencv the images are represented in the BGR format.So (0,0,255) represents perfect red and not blue as in a ordinary RGB image.

Another important color space used in image processing projects is HSV color space.Here the color value of each point is represented by three parameters Hue,Saturation and value.You can find more details here.Hue is the color component.In opencv it ranges from 0-180 unlike the normal image handling softwares such as Gimp or photoshop were the hsv value of hue varies from 0-360.So dont forget to divide the value by to before applying to opencv.

There are more than 150 color-space conversion methods available in OpenCV. But we will look into only two which are most widely used ones, BGR  Gray and BGR  HSV.

The syntax of the function used for color space conversion is :

cv2.cvtColor(input_image, flag) where flag determines the type of conversion.

For BGR  Gray conversion we use the flags cv2.COLOR_BGR2GRAY. Similarly for BGR  HSV, we use the flag cv2.COLOR_BGR2HSV. 

To get other flags, just run following commands in your Python terminal :

>>> import cv2
>>> flags = [i for i in dir(cv2) if i.startswith('COLOR_')]
>>> print flags

Example Code for BGR-> HSV conversion:

import cv2

img=cv2.imread('image.jpg')  # A BGR image is taken

hsv=cv2.cvtColor(img,cv2.COLOR_BGR2HSV) #image is converted to hsv color space

Example Code for BGR-> GRAY conversion:

import cv2

img=cv2.imread('image.jpg')    # A BGR image is taken

gray=cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)  #image is converted to gray sacle image

Similarly by changing the flags the image can be converted to various color spaces.Different color spaces have their own advantages and disadvantages.Each of them is selected according to their suitability to the application.