Rock, Paper, Scissors AI

Rock, Paper, Scissors AI

An AI that detects whether you're holding rock, paper, or scissors and plays a hand back


Created with Sketch.


We decided what our product would be during our first three days. Everyone came up with different ideas, and we decided to make a rock paper scissors detector because it’s a fun topic which would help us learn about AI and programming without spending lots of time trying to find good data for the model.

Data collection

After we finalized what our project would be, the next step was to collect the data that would be used to train the AI. We downloaded a set of over 2000 images of hands with rock, paper, or scissor hand gestures, and used labelbox to label the images as rock, paper, or scissors and drew bounding boxes around the hands in the image.

Data organization

The group split into three teams. 
The labeled data was downloaded as a csv file from labelbox. The first group downloaded all of the images from the csv file. The second team used the file to describe the dimensions of the bounding boxes and quantify the data in a text file. The third team used the image and text files created by the first two teams to create three folders for the data called train, test, and valid. These folders were used to train the neural network.

Training AI

When the data organization was completed, the neural network for our model was set up. The organized data was used to train our neural network.


After training the AI, our group split up into two teams to complete the website. One team designed the website, and the other team made sure that the website would work with the AI.


The backgrounds of our images were all the same, which caused the AI to have a difficult time identifying hands with other backgrounds. This was fixed by changing the backgrounds of the images to different colors for variation. The images were also difficult to label at first in labelbox.

Our Team