User:Medelen8/ENES100/Vision-Based Object Tracking Tutorial

Create a tutorial that future students can use to learn the basics of object tracking for their projects. Try to use existing hardware (cameras, Arduino Uno, Raspberry Pi, etc) and software (MATLAB, Arduino, etc) found in CL158/159/160.

Tutorials are at the bottom of this page

Decision Matrix
This was our first Decision Matrix, before we even knew about processing:

Arduino with OpenCV
This is a very complex project that involves an Android Phone and Ardiuno with OpenCv software: file:///C:/Users/DH304-08U/Downloads/Android_Arduino_Robot.pdf

This is a simpler project than the one above that is aimed more for Enes100: http://aleksandarkrstikj.com/tracking-a-ball-and-rotating-camera-with-opencv-and-arduino/

This project is very similar to the project above just different in some ways: http://myrobotlab.org/content/second-mrl-tutorial-tracking-service-wiring-diagram-0

OpenCV
This is where to download OpenCV: http://opencv.org/downloads.html

This program is written for OpenCV 2.4.3 and is a camera tracking objects outside: http://mateuszstankiewicz.eu/?p=189

This is one of the best tutorials to use but is not very detailed because he says you need previous C++ knowledge: http://www.youtube.com/watch?v=bSeFrPrqZ2A

Rasberry Pi with OpenCV
This is a great outline for the Rasberry Pi with OpenCV, I have not used the Raspberry Pi yet, so do not know how well it works: http://thinkrpi.wordpress.com/2013/05/

Short video of what is maybe the future goal for this project if done with the Rasberry Pi: http://www.youtube.com/watch?v=EEajP-dGTLY

This is a detailed page for the Rasberry Pi with OpenCV: http://programmaticponderings.wordpress.com/2013/02/09/opencv-and-cvblob-with-raspberry-pi/

Also a detailed page for the Rasberry Pi with Open Cv: http://letsmakerobots.com/node/36947

Jake’s Approach
Goal: http://www.youtube.com/watch?v=EEajP-dGTLY

To learn OpenCV: http://www.youtube.com/user/khounslow?feature=watch

To learn Raspberry Pi with OpenCV: http://thinkrpi.wordpress.com/2013/05/22/opencv-and-camera-board-csi/

This is the new tutorial I am using for the project: http://www.18f4550.com/OpenCV_and_Emgu_CV/OpenCV_tut_1/OpenCV_tut_1.html

Processing
This approach we found works the best

Download Processing 2.1 (The other versions might have errors, we used the 64-bit): https://code.google.com/p/processing/downloads/list

This is the website that has the code that tells the user what resolution will work in the Processing Code: http://processing.org/reference/libraries/video/Capture_list_.html

This is the site where we found the blob detection code: http://www.v3ga.net/processing/BlobDetection/

Below is the code that needs to be put into Processing 2.1



// - Super Fast Blur v1.1 by Mario Klingemann  // - BlobDetection library

import processing.video.*; import blobDetection.*;

Capture cam; BlobDetection theBlobDetection; PImage img; boolean newFrame=false;

// ================================================== // setup // ================================================== void setup {	// Size of applet size(640, 480); // Capture cam = new Capture(this, 640, 480, 30); // Comment the following line if you use Processing 1.5 cam.start; // BlobDetection // img which will be sent to detection (a smaller copy of the cam frame); img = new PImage(80,60); theBlobDetection = new BlobDetection(img.width, img.height); theBlobDetection.setPosDiscrimination(true); theBlobDetection.setThreshold(0.2f); // will detect bright areas whose luminosity > 0.2f; }

// ================================================== // captureEvent // ================================================== void captureEvent(Capture cam) {	cam.read; newFrame = true; }

// ================================================== // draw // ================================================== void draw {	if (newFrame) {		newFrame=false; image(cam,0,0,width,height); img.copy(cam, 0, 0, cam.width, cam.height, 				0, 0, img.width, img.height); fastblur(img, 2); theBlobDetection.computeBlobs(img.pixels); drawBlobsAndEdges(true,true); } }

// ================================================== // drawBlobsAndEdges // ================================================== void drawBlobsAndEdges(boolean drawBlobs, boolean drawEdges) {	noFill; Blob b;	EdgeVertex eA,eB; for (int n=0 ; n<theBlobDetection.getBlobNb ; n++) {		b=theBlobDetection.getBlob(n); if (b!=null) {			// Edges if (drawEdges) {				strokeWeight(3); stroke(0,0,400);//this makes the blob line change color (r,g,b) for (int m=0;m<b.getEdgeNb;m++) {					eA = b.getEdgeVertexA(m); eB = b.getEdgeVertexB(m); if (eA !=null && eB !=null) line(							eA.x*width, eA.y*height, 							eB.x*width, eB.y*height							); }			}

// Blobs if (drawBlobs) {				strokeWeight(1); stroke(0,0,400);//this changes the color of the box (r,g,b) rect(					b.xMin*width,b.yMin*height,					b.w*width,b.h*height					); }

}

} }

// ================================================== // Super Fast Blur v1.1 // by Mario Klingemann //  // ================================================== void fastblur(PImage img,int radius) { if (radius<1){ return; } int w=img.width; int h=img.height; int wm=w-1; int hm=h-1; int wh=w*h; int div=radius+radius+1; int r[]=new int[wh]; int g[]=new int[wh]; int b[]=new int[wh]; int rsum,gsum,bsum,x,y,i,p,p1,p2,yp,yi,yw; int vmin[] = new int[max(w,h)]; int vmax[] = new int[max(w,h)]; int[] pix=img.pixels; int dv[]=new int[256*div]; for (i=0;i<256*div;i++){ dv[i]=(i/div); }

yw=yi=0;

for (y=0;y>16; gsum+=(p & 0x00ff00)>>8; bsum+= p & 0x0000ff; }   for (x=0;x>16; gsum+=((p1 & 0x00ff00)-(p2 & 0x00ff00))>>8; bsum+= (p1 & 0x0000ff)-(p2 & 0x0000ff); yi++; }   yw+=w; }

for (x=0;x<w;x++){ rsum=gsum=bsum=0; yp=-radius*w; for(i=-radius;i<=radius;i++){ yi=max(0,yp)+x; rsum+=r[yi]; gsum+=g[yi]; bsum+=b[yi]; yp+=w; }   yi=x; for (y=0;y<h;y++){ pix[yi]=0xff000000 | (dv[rsum]<<16) | (dv[gsum]<<8) | dv[bsum]; if(x==0){ vmin[y]=min(y+radius+1,hm)*w; vmax[y]=max(y-radius,0)*w; }     p1=x+vmin[y]; p2=x+vmax[y];

rsum+=r[p1]-r[p2]; gsum+=g[p1]-g[p2]; bsum+=b[p1]-b[p2];

yi+=w; } }

}

Check this code if you want to draw a center point. WE COULDN'T FIGURE THE CORRECT FORMAT FOR THE CODE, BUT WE FOUND A WAY WHERE YOU CAN DRAW A POINT AT THE TOP LEFT EDGE OF EVERY RECTANGLE.

YOU WOULD PUT IN THE CODE RIGHT AFTER WHERE IT SAYS

 strokeWeight(1); stroke(0,0,400);//this changes the color of the box (r,g,b) rect(b.xMin*width,b.yMin*height,				b.w*width,b.h*height);

YOU WOULD PLAY WITH THIS CODE TO MOVE TO THE CENTER OF THE RECTANGLES

strokeWeight(3) stroke(400) point(b.xMin*width,b.yMin*height); point(b.w*width,b.h*height);

We tried with the code so many different times, but we never got to a correct code that works to put the point in the center of the rectangles.

Processing Tutorials
Tutorial on How to download Processing 2.1

http://www.youtube.com/watch?v=9bVXOAmWJf8

Tutorial: How to download "Blob Detection" into the Processing 2.1 Library

https://www.youtube.com/watch?v=G0erHWGftUc

Another Tutorial: How to download "Blob Detection" into the Processing 2.1 Library

http://www.youtube.com/watch?v=WboB_mHWiYs

Tutorial: What kind of Camera Resolution codes can you use

http://www.youtube.com/watch?v=25xXQR51iVI