Computer Vision
I worked as an Undergraduate Research Assistant in the ReNeu, or Rehabilitation and Neuro-engineering, lab on UT campus.
I worked with graduate student to research and conduct experiments focused on translating human movement into robotic manipulation. My role consisted of design and manufacturing of the experiment space, assisting with human experimentation, and post-processing algorithms.
Goal of Research
The goal of the research I worked on was to convert EMG signals from human neck movement into robotic manipulation of a Yaskawa arm. The idea was for specific movements of the neck, such as extension, compression, twist left, and twist right, the EMG voltage signals can be recorded and mapped through an algorithm to move the end effector of a robotic arm on a 2D plane accurately and precisely.
This could ultimately used for spinal cord injury patients who often only have motor control of their body from the neck up.
Framework for Experimentation
To ensure we had a good workspace to gauge the validity of our research and work, my task was to design a framework to be able to test how well the EMG voltages could control the end effector.
To portray that the end effector was being controlled precisely, I designed am obstacle course of sorts. Using balls as targets with thick wire and 80/20s to hold the pieces up.. I placed red balls in strategic areas to test motor control, full range of motion, and speed of the end effector. In addition I also used a green ball as a target to avoid.
This allowed our experiment to be quantified under the parameters of how many targets were hit in 2 minutes, and if the green target was avoided.
End Effector
To further make the experiment more logical for the subjects, I designed an end effector that mimicked a cursor, or arrow of sorts to aptly portray which side of the end effector was the "front", and which was the "back"., as the end effector was not programmed to be multidirectional and moved similar to a car
Computer Vision
A key part of the process was to accurately post process the human trails we were doing to be able to show our findings in a professional and tru manner. The way we were able to check the ground truth values of the moving end effector in relation to the obstacle course we made was using computer vision. Computer vision is an algorithm that takes in a video input and is able to use different, distinct characteristics to map and track certain phenomena. In our case we used the sharp colors of the targets and the end effector to contrast with a black background to track where the targets were at every moment, where the end effector was at every moment, and if the end effector did in fact hit the targets.
I used MATLAB to code this software and utilized an existing toolbox with functions that allowed me to read through each frame of a video and distinguish color. The code essentially worked by taking the video file, separating it frame by frame, and tracking every instance of a certain color, in this case blue, for each frame. It then uses a grid format with the origin in the middle and maps the position of the tracked object and outputs these as CSV files to convert the videos to planar kinetic motion.
Challenges and Specifications
Some of the challenges that went into ensuring the computer vision was accurate was to keep the camera recording the video in the same spot. The tripod was not moved in between trials.
In addition, the software could only track one moving object or multiple stationary objects. To remedy this, I took one video with blue magnetic. 3D printed covers over the red and green targets, and another video with the blue end effector moving. These videos were then overlain.
Another critical component of the software was that it only tracked the centroid of the shape it was tracking. This mean, to accurately see if the end effector was touching the targets, I used the radius of the target to map out the circle in MATLAB.
Lastly, the csv files were converted to a map as a visual aid to show the robot paths for each trial.