Updated date:

How to Trace an Object in Opencv

Victor is a registered member of Data Science Nigeria and an Executive tutor in Data Science Ife, Obafemi Awolowo University.A mastermind.

The Trace

Trace My Pen

Welcome back folks to another episode of the tutorial on opencv. Permit me to share with you a quick story. Sometimes ago, my colleagues tried to build a device using a gyro-meter which could be used to write directly into a PC, the idea was to write on any surface and get corresponding results on the PC. Cool right, it's like writing instead of typing. Yes, they made the device, but on the day of the project defence, Dr Akintade (One of the conducting lecturers) disagreed with the idea. He claimed that the gyroscope should have a point of origin on the (x,y,z) Axis so the software back-end could take reasonable measurements and convert to traces. This eventually brings us to this quick trace project, where we detect the color of my object(Pen) And track it's movement.

The Pipeline

The pipeline is quite easy, first is to get the frame, mask out all other color regions asides our color of interest, and lastly draw a bounding circle for the contour as well as a line between the position of where the contour was found last and where is is now.

Import Packages

Firstly, we import the packages needed to run the algorithm. If you've been following my previous tutorials, the only package new to you may be deque. The deque is just a form of list with super-fast edits, like fast appends, fast pops, and more. It even has an append-left feature which will be very useful in this project.

Time is another package available in python which helps us keep track of real time.

from collections import deque # import deque a far faster list
import numpy as np # import numpy as needed
import imutils # we definately need imutils convienience functions.
import time # time is time
import cv2 # The major library, opencv

Set Parameters

In the short code below, I defined the upper and lower boundary for the color of my object (Red pen), I actually used trial by error method to get a good span for the upper and lower color boundary, I left the tip to do this down in the bonus section.

Having defined these, I initialized points to be a deque (Think of it like a speed list) With maximum length of 64, the max length there pops out the item in the list when I append a new one. Finally I start the video stream giving it some 2 seconds to warm up.

lower_color_boundary = (161, 155, 84) # define the lower boundary of the colour of the object in HSV
upper_color_boundary = (179, 255, 255)# define the upper region of the colour of the object in HSV
points        = deque(maxlen = 64) # the Deque with a maximum hold size of 64 contents.

print("Booting the Video stream,")
vs = cv2.VideoCapture("video_file.mp4")#start the video stream.
time.sleep(2.0) #set sleep time to 2.0 seconds

The Big Deal

In this section, we do everything. First, I placed the whole process in a while loop since a video is nothing more than continuous moving frame of pictures.

In the next line I read off the frame from the video master, next, I separated the ret and frame. The ret is simply just a boolean regarding whether there was a return at all from the vs.Read(). Next if there is nothing in the frame, break stop the process.Another way could be if ret is false, break. In the next line, I used imutils to resize the frame. The imutils resize saves a lot of stress in preserving the aspect ratio than stressful standard opencv methods of resizing.

Following that, I decided to flip the image. I did this to obtain the mirror view of my frame which is absolutely quite cool for direct streaming from the camera.

Processing the Frame.

The frame is ready, first I blurred the frame using the popular Gaussian blur method, this helps to reduce noise in the image. Next, I converted the color from the BGR to HSV. The HSV is a color band which means hue, saturation and value. It is a color representation closer to the human vision(Our way of seeing).

In the next step, I created a mask by using the image threshold such that I save out only colors within my upper and lower boundary, which in this case is the color of my red pen. After masking out other parts of the image, I performed erosion, and dilation to further reduce high frequency noise caused by taking images to their threshold.

All Said and Done

Now that I've gotten a view of the object (My red pen), I found the contours, grabbed them using imutils. In the following if statement, I picked out the contour with the maximum area, which is definitely the contour of my object, the remaining contours I'm quite positive is just noise. Unless if you plan to track two objects of the same color. In the next line, I prepared parameters required to construct a bounding circle on the object. The first is the radius and the x,y co-ordinates, next I used cv2.Moments to obtain it's center of mass. The last if loop was written such that if the radius is reasonable enough, I draw a bounding circle around my object as well as a filled in dot at the center of the object.


Very importantly, i append-left the center of my object to the deque. Remember that my deque has a max-length of 64. I'll explain why i did this shortly.

Draw Traces

The last and final step is to draw the traces. To do this I decided to connect all my points in order with a single line, like in a chain form. I also specified that if the value in the list is none, it should ignore and start the while loop all over. I did this so that I avoid drawing a point at (0,0) Which is (None,none) On my frame when the object is not detected.

On the other hand, if I find my object, it connects all 64 dots together and traces it out. I used a max-length of 64 to avoid my screen from getting all messed up with traces. See the effects yourself. Run the code.

while True:
    frame = vs.read() #Read off the frame from the video stream
    ret,frame  = frame # Use this if you want to load in your video
    if frame is None: # If there is no frame, save my pc from going through any stress at all
        break
    # otherwise, if we have a frame, we proceed with the following code
    frame = imutils.resize(frame, width = 700) # so much easier than open cv, keeping aspect ratio intact
    frame = cv2.flip(frame,1) # i want the mirror view, it's very helpful especially if i'm streaming
    
    #processing the frame 
    blurred = cv2.GaussianBlur(frame, (11,11),0) # blurr helps to reduce high frequency noise, definately helps model
    hsv = cv2.cvtColor(blurred, cv2.COLOR_BGR2HSV) # convert my color to the HSV format
    
    # Create a mask
    mask = cv2.inRange(hsv, lower_color_boundary, upper_color_boundary) # mask other regions except colors in range of upper to lower (thresholding)
    mask = cv2.erode(mask, None, iterations =2) # Reduce noise caused by thresholding
    mask = cv2.dilate(mask, None, iterations =2) # foreground the found object i.e futher reduce noise.
    
    
    contours = cv2.findContours(mask.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE) # find contours
    contours = imutils.grab_contours(contours) # Grab the contours using imutils
    center = None # center is initially set to none
    if len(contours) > 0: # if the contours list is not empty proceed
        contour = max(contours, key = cv2.contourArea) # select contour with maximum Area, most likely our object
        ((x,y), radius) = cv2.minEnclosingCircle(contour) # pick up co-ordinates for drawing a circle around the object
        M = cv2.moments(contour) # Extract moments from the contour.
        center = (int(M["m10"] / M["m00"]), int(M["m01"] / M["m00"])) # Obtain the centre of mass of the object.
        if radius > 10: # if we have a reasonable radius for the proposed object detected
            cv2.circle(frame, (int(x), int(y)), int(radius), (0,255,255), 2) # Draw a circle to bound the Object
            cv2.circle(frame, center, 5, (0,0,225), -1) # Draw a filled in dot at the centre of the circle

    points.appendleft(center) # Tell my deque to keep the position of the centre of that frame.
    
    
    # Speedily draw all the in the deque.
    
    for i in range (1, len(points)): #for all the points in the deque
        if points[i-1] is None or points is None: # if we have none as the current point or previous
            continue # Start the while loop all over again.
        thickness = int(np.sqrt(64 / float(i+1)) * 2.5) # let's make thickness vary with point in deque
        cv2.line(frame, points[i-1], points[i], (0,0,225), thickness) #draw a line between all 64 points in the deque
    cv2.imshow("Frame by Frame makes Video", frame) # let's see the frame X frame
    
    
    # Closing a video frame
    key = cv2.waitKey(1) #wait for the cv key
    if key == ord("x"): # If the x button is pressed
        break # Break from the loop
vs.release() # Let opencv release the video loader
cv2.destroyAllWindows() # Destroy all windows to close it

The End

This concludes the tutorial on tracing my red pen with opencv.

Bonus Tip

How on earth did i find out the HSV values for my red pen. I am no magician, i'm a programmer and i definitely do not know all things. You can look up on google for the HSV values for your color of interest and test them down here with this short code like i did(trial by error). I have put the code in a RAWNB format.

img = cv2.imread('frame.jpg') #load in your image of intrest

RED_MIN = np.array([161, 155, 84],np.uint8) # minimum color for reddish
RED_MAX = np.array([179, 255, 255],np.uint8) # max color reddish 

hsv_img = cv2.cvtColor(img,cv2.COLOR_BGR2HSV) #convert the image color format to HSV

frame_threshed = cv2.inRange(hsv_img, RED_MIN, RED_MAX) #Threshold the image
cv2.imshow('Thresholded image', frame_threshed) # show the thresholded image
cv2.imshow('original image', img) #Show the original image
cv2.waitKey(0) # close convieniently.
how-to-trace-an-object-in-opencv
how-to-trace-an-object-in-opencv

Comments

Olufemi Victor (author) from Nigeria on August 13, 2020:

Download the full code and run on your pc from my github repository.

https://github.com/osinkolu/Tutorial-6-Object-dete...