Introduction to GERM&N Engineering
An Introduction to Our Project: Simon Says “Exercise with Me!”, a Low-Cost, Open-Source, Socially Assistive Robot
About Our Project
The current cost of physical therapy is exorbitant for simple, exercise-based treatments.
Meanwhile, the existing, therapy-based socially assistive robots (SARs) are either unavailable
to the general public or costly with limited options for customization.
We will bring the expensive, research-focused world of socially assistive robot therapy from the lab to the home through a cost-effective, open-source alternative — Simon.
Simon Says "Exercise with Me!"
Simon, assembled from the open-source, 3D-printed InMoov, will guide patients through simple physical therapy exercises, complete with intuitive gestures, facial tracking, and natural language responses.
Simon’s estimated price point, including the costs of base InMoov, 3D printing, and materials will be $1,000. The software suite for Simon’s functionality will be completely free of charge and open-source, bringing the focus to consumers.
Competitors in the Current SARS Market
The current landscape is limited due to a large focus on research, with a multitude of
applications to social therapy.
Simon does not have direct competitors in the socially assistive robot industry. The prototype robots, like USC’s “Bandit,” are research-focused and unavailable to the general public. The prebuilt robots, like SoftBank’s “NAO,” cost several thousands of dollars while still lacking features included in Simon.
Socially Assistive Robots double as friendly companions. We wanted to implement humanlike features,
such as facial tracking, to allow easier interaction with end users. Additionally, the team wanted Simon
to be capable of adapting to individual users with facial recognition through the robot’s neck and head movement.
We developed facial recognition software to identify a user's face. Next, to maintain continuous eye contact, we created an algorithm tracking user movement by calculating where Simon should relocate to in real time.
We tested our robot’s facial tracking functionality with a member standing within view of the mounted camera, facing the robot directly. Our test subject moved randomly to various spots within the camera's range of vision. We were able to complete accuracy, resolution, and repeatability testing to create a more friendly, humanlike robot.
Facial Recognition Open Source Code with OpenCV and Raspberry Pi 4.
Simon knows a large library of words borrowed and learned
from Google's Voice API. Simon is able to listen and even converse with users, who
can ask Simon to assist them with their daily dose of exercise! This library is constantly evolving!
Check out Simon's voice recognition software here.
Joint Motor Control
We have implemented fine joint motor control in the shoulder, elbow, wrist, and finger joints via Arduino. We are also able to individually control the motors and debug existing mechanical issues. These arm movements can be accurately and fluidly repeated upon commands to mimic five distinct arm exercises — arm lifts, bicep curls, fist clenches, hand flips, and reverse raises. Check out our software here!
Remote Robotic Control
We had to pivot our project in response to the challenges of COVID-19. With GERM&N Engineering separated across hundreds of miles, each subteam (Face Team and Arm Team) took Simon’s respective components to work remotely. Under the mentorship of our instructors, we are transitioning to Internet of Things (IoT) and remotely sending commands from the head to the arm through an online database. This is achieved with Google Drive API, as it is readily available and easy to use! Here is our open source code if you would like to try it out at home too!
Simon will not be going through the FDA. GERM&N Engineering’s overall design lies within Simon’s software component, which does not fall under the FDA’s regulation of medical devices.
Simon’s target market is the online socially assistive robot community. These may be individuals who have loved ones living alone. SARs assist users through social, non-contact interactions.
Our software is hosted as an open-source repository on GitHub, widening the field and reaching those without academic or financial resources. This will also shift SARs from the lab to the home or clinic.
We are currently working on redesigning Simon’s head and 3D-printed parts to be friendlier via LED panels to depict real-time facial reactions. We are also refining Simon’s speech response. Check in with us in the future for updates!
- Facial Recognition
- Joint Movements
- Arm Raise Exercise
- Arm Rotation Exercise
Facial TrackingThis is an example of how the robot is able to recognize and move to face a user.
Elbow Joint MovementA command is sent from a computer (not shown here) and the robot's arm is instructed to move. This mimics an arm curl exercise.
Shoulder Joint Movement (Top)The shoulder joint is programmed to move left and right with restrictions to prevent breaking the motor.
Facial Tracking Demo (Left-Right)The robot smoothly tracks the user in real time as they move side to side.
Shoulder Joint Movement (Top)The shoulder joint is programmed to move up and down with restrictions to prevent breaking the motor.
Shoulder Joint Movement (Bottom)The robot is controlled with Arduino and Raspberry Pis. Most parts were 3D printed except for motors.
Fine Finger Joint MovementThe fingers were programmed to be able to complete finger curl exercises.
Shoulder Movement (Bottom)The robot is controlled with Arduino and Raspberry Pis. Most parts were 3D printed except for motors.
Facial Tracking Demo (Up-Down)The robot smoothly tracks the user in real time as they move up and down.
We are a group of biomedical engineering students at University of California, Irvine working with Dr. David Reinkensmeyer's Biorobotics Laboratory.
Dr. David ReinkensmeyerFaculty Mentor
Nathan ChiangTeam Lead
Edward FahmanMarketing Advisor
Ryan FalkBusiness Advisor
Interested in joining our efforts or have any questions about the project? Contact us here!
UCI School of Engineering & BioEngine
Irvine, CA 92612