Welcome to GERM&N Engineering

We are a team of dedicated engineers developing open-source software for socially assistive robots

Explore More

Introduction to GERM&N Engineering

An Introduction to Our Project: Simon Says “Exercise with Me!”, a Low-Cost, Open-Source, Socially Assistive Robot

About Our Project

The current cost of physical therapy is exorbitant for simple, exercise-based treatments. Meanwhile, the existing, therapy-based socially assistive robots (SARs) are either unavailable to the general public or costly with limited options for customization.

We will bring the expensive, research-focused world of socially assistive robot therapy from the lab to the home through a cost-effective, open-source alternative — Simon.

Simon Says "Exercise with Me!"

Simon, assembled from the open-source, 3D-printed InMoov, will guide patients through simple physical therapy exercises, complete with intuitive gestures, facial tracking, and natural language responses.

Simon’s estimated price point, including the costs of base InMoov, 3D printing, and materials will be $1,000. The software suite for Simon’s functionality will be completely free of charge and open-source, bringing the focus to consumers.


Competitors in the Current SARS Market

The current landscape is limited due to a large focus on research, with a multitude of applications to social therapy.

Simon does not have direct competitors in the socially assistive robot industry. The prototype robots, like USC’s “Bandit,” are research-focused and unavailable to the general public. The prebuilt robots, like SoftBank’s “NAO,” cost several thousands of dollars while still lacking features included in Simon.


Facial Tracking

Socially Assistive Robots double as friendly companions. We wanted to implement humanlike features, such as facial tracking, to allow easier interaction with end users. Additionally, the team wanted Simon to be capable of adapting to individual users with facial recognition through the robot’s neck and head movement.

We developed facial recognition software to identify a user's face. Next, to maintain continuous eye contact, we created an algorithm tracking user movement by calculating where Simon should relocate to in real time.

We tested our robot’s facial tracking functionality with a member standing within view of the mounted camera, facing the robot directly. Our test subject moved randomly to various spots within the camera's range of vision. We were able to complete accuracy, resolution, and repeatability testing to create a more friendly, humanlike robot.

Facial Recognition Open Source Code with OpenCV and Raspberry Pi 4.

rasp pi

inside-simon's head

facial-tracking image

Voice Recognition

Simon knows a large library of words borrowed and learned from Google's Voice API. Simon is able to listen and even converse with users, who can ask Simon to assist them with their daily dose of exercise! This library is constantly evolving!
Check out Simon's voice recognition software here.

voice demo giphy

Joint Motor Control

We have implemented fine joint motor control in the shoulder, elbow, wrist, and finger joints via Arduino. We are also able to individually control the motors and debug existing mechanical issues. These arm movements can be accurately and fluidly repeated upon commands to mimic five distinct arm exercises — arm lifts, bicep curls, fist clenches, hand flips, and reverse raises. Check out our software here!

simon arm

Remote Robotic Control

We had to pivot our project in response to the challenges of COVID-19. With GERM&N Engineering separated across hundreds of miles, each subteam (Face Team and Arm Team) took Simon’s respective components to work remotely. Under the mentorship of our instructors, we are transitioning to Internet of Things (IoT) and remotely sending commands from the head to the arm through an online database. This is achieved with Google Drive API, as it is readily available and easy to use! Here is our open source code if you would like to try it out at home too!

Market Strategy


Simon will not be going through the FDA. GERM&N Engineering’s overall design lies within Simon’s software component, which does not fall under the FDA’s regulation of medical devices.

Target Market

Simon’s target market is the online socially assistive robot community. These may be individuals who have loved ones living alone. SARs assist users through social, non-contact interactions.

Open-Source Code

Our software is hosted as an open-source repository on GitHub, widening the field and reaching those without academic or financial resources. This will also shift SARs from the lab to the home or clinic.

Projected Impact

There is currently no dominant company holding market share for socially assistive robot therapy, allowing Simon to affect significant change.

Upcoming Features

We are currently working on redesigning Simon’s head and 3D-printed parts to be friendlier via LED panels to depict real-time facial reactions. We are also refining Simon’s speech response. Check in with us in the future for updates!

Photo Gallery

  • All
  • Facial Recognition
  • Joint Movements
  • Arm Raise Exercise
  • Arm Rotation Exercise

Facial Tracking

This is an example of how the robot is able to recognize and move to face a user.

Elbow Joint Movement

A command is sent from a computer (not shown here) and the robot's arm is instructed to move. This mimics an arm curl exercise.

Shoulder Joint Movement (Top)

The shoulder joint is programmed to move left and right with restrictions to prevent breaking the motor.

Facial Tracking Demo (Left-Right)

The robot smoothly tracks the user in real time as they move side to side.

Shoulder Joint Movement (Bottom)

The robot is controlled with Arduino and Raspberry Pis. Most parts were 3D printed except for motors.

Fine Finger Joint Movement

The fingers were programmed to be able to complete finger curl exercises.

Facial Tracking Demo (Up-Down)

The robot smoothly tracks the user in real time as they move up and down.

Our Team

We are a group of biomedical engineering students at University of California, Irvine working with Dr. David Reinkensmeyer's Biorobotics Laboratory.

Dr. David Reinkensmeyer

Faculty Mentor

Nathan Chiang

Team Lead

Grant Barnes


Edward Fahman

Marketing Advisor

Ryan Falk

Business Advisor

Melany Yeung


Contact Us

Interested in joining our efforts or have any questions about the project? Contact us here!

UCI School of Engineering & BioEngine
Irvine, CA 92612