User:Mir07/mywork2

''This is designed for a person working on a project for 4 weeks as part of a 3 credit class. Please copy this form to your user page. Then delete everything in italics when filling out this form.''

Write problem/project Goal
To use the kinect Gestures to control a toy car.

My First Task
Get the Kinect and arduino to work together. Get the arduino to get the kinect skeleton points and convert them into control data which then can be used to move or rotate servos or certain parts of the car ie right left movement and acceleration.

Summary of actual work over first weekend
My task was to get the kinect gesture feature to work with the arudino. The gestures work as controller points which the arudino passes to any object ie car and moves it, either right or left. At this point I just wanted to get the kinect gesture feature to work with the arduino to pass parameters to the servos. Using left and right hand movements in Kinect to control the rotation of two servos tied to the Arduino, which in turn moves the left and right servos. The code for this is written in Visual Basic with gets the gesture points from the kinect skeleton and converts them into arduino parameters that moves the servos.

Week1 Narrative
The following steps will show how the kinect and arduino are able to communitcate with each other to make the servos move, based on gestures alone.

Hardware Required: 1- Microsoft Kinect 2- Arduino 3- Micro Servos/or stepper motor 4- Breadboards and wires

Setup: Drivers and software (some drivers and software are required to get the kinect and the arduino to work together with the kinect gesture feature). 1-download and install the Microsoft development kit from http://www.microsoft.com/en-us/kinectforwindows/develop/ 2-download and install the arduino software from http://www.arduino.cc/en/Main/software 3-download and install the coding4fun kinect toolkit from http://c4fkinect.codeplex.com/releases/view/76271#DownloadId=300233 4-download and install Microsoft wisual basic Express from http://www.microsoft.com/visualstudio/en-us/products/2010-editions/visual-basic-express 5- install .NET Framework 4 from http://msdn.microsoft.com/en-us/netframework/aa569263 6- Microsoft DirectX SDK - June 2010 or later version from http://www.microsoft.com/download/en/details.aspx?id=8109 7- Install Runtime for Microsoft DirectX 9 http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=35 8- Microsoft Kinect Code4Fun Quickstart VB code (some parts of the code are modified) http://files.ch9.ms/coding4fun/KinectForWindowsSDKQuickstarts.zip

Wiring: The servos will be powered by the Arduino connected to the PC via USB. One servo will be left and the other right. If the right and left are interchanged just change a line of code instead of rewiring Left = Arduino Digital Pin 9 Right = Arduino Digital Pin 11

Running the code to move servos with kinect gestures: 1- Download the "ArduinoXYZ.ino" file from http://www.instructables.com/id/Kinect-controls-Arduino-wired-Servos-using-Visual-/ and load it into the Arduino UNO via the Arduino IDE 1.0.

2- Download KinectForWindowsSDKQuickstarts.zip from http://www.instructables.com/id/Kinect-controls-Arduino-wired-Servos-using-Visual-/

3- Find and extract the SkeletalTrackingVB folder and place it under the Projects folder the Visual Basic 2010 Express or visual studio 2010.

4- From Visual Basic 2010 IDE explorer, double click on MyProject. There will be a with a project configuration page.

5- Click on the References tab.

6- Search for Microsoft.Research.Kinect and delete it.

7- Search for Code4Fun.Kinect.wpf and delete it.

8- Click on Add.

9- Click the Browse tab.

10- Search for and add the newer versions of those files which you have downloaded part of the latest Microsoft Kinect SDK and Coding4Fun Kinect Toolkit zip files.

11- Download MainWindow.xaml and MainWindow.xaml.vb.

With the Kinect and Arduino connected, run the test the program.Important: The COM port field on the main application window is set to 16. Change this to match the Arduino's COM port on your PC.

With the code the servos connected to the arduino can move with hand gestures using kinect.

My Second Task
For my second task I want apply the techniques I applied to moving the servos to a RC toy car. I want to control all the movements and acceleration of a car based on gestures from the kinect. Possibly reverse engineer a remote controller to work with the RC car.

Summary of actual work over second weekend
Researched as to how I can move a car based on kinect gestures. I was not able to reverse engineer an RC car controller at this point, however If was able to get some idea as to how this could work. I tried a few things, which I will cover below, and I think I may be able to communicate between the arduino and the RC contorller through serial communication. This will allow me to communicate with the kinect which is plugged to the computer; with the ardunio which is also connected to the computer. Serial communication allows hardware to communicate with each other through a medium (pc).

Week2 Narrative
Main idea: Kinect Controls RC Car 1) Kinect and arduino are connected to the PC via USB. 2) C# or python code on pc interprets kinect’s gesture based inputs then commands arduino to pulse the controls within the RC car, i.e. right/left motors and acceleration is controlled with the kinect gestures. 3) Arduino just waits for the serial input from the PC to control RC car movements. 4) The Arduino receives commands from its Serial interface and toggles its output to control the car’s remote controller. The Arduino receives commands from its Serial interface and toggles its output to control the car Sketch that the Arduino runs in conjuction with the C# code which converts gesture based data from the kinect and sends it to the controller. Important: the digitalWrite(pin, LOW); always precedes the digitalWrite(pin, HIGH); command? This is to prevent conflicting commands from being sent to the car(e.g. Forwards and Backwards simultaneously).

Hardware: 1) Arduino 2) Breadboarded

Software: 1) Everything from week 1 2) Arduino Sketch

My Third task
Reverse engineer RC Controller and sync it with the arduino so that the data from the kinect can be sent to the controller rather than the car. This would increase the car allowing it to move without being plugged to the pc via usb.

Summary of actual work over third weekend
Researched on how to hook up an RC Controller with arduino

Week3 Narrative
Each output pin controls current passing through an opto-isolator, which isolates the circuit of the Arduino from that of the car’s controller. This allows the Arduino to control the car, despite both circuits having different electrical potentials. The switches at the top of the above diagrams are placeholders for the actual control mechanism of the car. A current-limiting resistor is used so as to provide a current within the operating parameters of the opto-isolator. The breadboarded circuit looks like this:

The jumper cable and pin information can gotten from the is code.

int pinRight = 11; int pinLeft = 10; int pinForward = 9; int pinReverse = 8;

void setup Serial.begin(9600); pinMode(pinRight, OUTPUT); pinMode(pinLeft, OUTPUT); pinMode(pinForward, OUTPUT); pinMode(pinReverse, OUTPUT); } void loop { if(Serial.available > 0){ int tmpByte = Serial.read; switch(tmpByte){ case 'w': // Move car FORWARDS digitalWrite(pinReverse, LOW); digitalWrite(pinForward, HIGH); break; case 's': // Move car in REVERSE digitalWrite(pinForward, LOW); digitalWrite(pinReverse, HIGH); break; case 'a': // Turn steering wheels LEFT digitalWrite(pinRight, LOW); digitalWrite(pinLeft, HIGH); break; case 'd': // Turn steering wheels RIGHT digitalWrite(pinLeft, LOW); digitalWrite(pinRight, HIGH); break; case '_': // STOP ALL motion digitalWrite(pinReverse, LOW); digitalWrite(pinForward, LOW); // The missing break; here is entirely intentional. case 'x': // Move steering wheels STRAIGHT digitalWrite(pinRight, LOW); digitalWrite(pinLeft, LOW); break; default: break; } } The Kinect data is received and processed in the C# application that dispatches instructions over the serial port.

My Fourth task
Find C# code to convert gestures to data that the arduino can interpret and send to the RC controller

Summary of actual work over fourth weekend
Used a different approach to make the rc-controller work with the arduino and kinect. Instead of working with RC-Car, I started working with RC-Helicopter that need no calibration of controller. However what each frequency does needs to be determined ie speed, right-left, up-down, all depend on the frequency of the controller. This part build on the previous weeks so the arduino and kinect serial communication works in the same way, except for the addition of IR LED's

Week4 Narrative
Hardware:
 * 1-kinect
 * 2-Arduino Board
 * 3-resistors
 * 4-resistors
 * 5-transistors
 * 6-IR LEDs

Software: Can be download from this web site, this is the code that gets the RC- helicopter flying
 * 1-http://codingfun.collinboy.net/S107_IR_Code.ino
 * 2.Microsoft Kinect SDK software: to determine the location of hands relative to head and hips. This gaves a y-coordinate distance that could be scaled from 0 to 127.

Process:
 * 1- Get the Arduino to communicate with the helicopters controller, the frequency of the controller is 38 khz signal(signals are transmitted over a 38 khz signal)

'''Note:Channel "A" uses Throttle Values 0 to 127. Channel "B" uses Throttle Values 128 to 255.'''

Format: This process makes the arduino code work with the transmission.
 * 1.Start Signal: 2000 us
 * 2.Pause: 2000 us
 * 3.Rudder Signal: 8 bits of transmission (for example 127 is "01111111", 63 is "00111111")
 * 4.Elevator Signal: 8 bits of transmission
 * 5.Throttle Signal: 8 bits of transmission
 * 6.Trim Signal: 8 bits of transmission
 * 7.End Signal: 300 us

In the code provided all it does is feed the void Transmit with integer values between 0 and 127 (or 128 to 255 for Channel "B"

Complete Team Page
Fill out the Team Form (should have already copied the form, created the team page, linked to it and started filling it out).