An investigation into computer vision applied to Rock-PaperScissors Isaac Winston Abstract: In this project, I trained a computer to play rock-paper-scissors through gesture recognition from a webcam feed. I did this by segmenting skin from the video, and then feeding the result through a classifier that decides whether the current image is rock, paper or scissors. The computer randomly chooses a gesture itself and decides the winner. Provisional results show that the images are classified with an accuracy of ~91%. In the future, I want to add a tactics engine to infer what the user is likely to pick next based on past actions. Adding a robotic arm to show the computer’s choice would make the game more interactive. Introduction Rock-paper-scissors Rock-paper-scissors (RPS) is a two-player game, where players secretly choose rock, paper or scissors before revealing their choice using a hand gesture. Each of the items (rock, paper or scissors) beats one other specific item, loses to one other specific item, or draws if the opponent picks the same item. In this project, I aim to use computer vision to recognise the RPS hand gestures so that a computer can act as the second player. Computer vision/AI Computer vision is a subset of Artificial Intelligence (AI) techniques. It aims to let computers process (“look at”) images/videos and interpret them for further processing such as decisionmaking. Here, I aim to train a computer to look at a video feed and decide if a person is playing rock, paper, or scissors. If I imagine my brain playing RPS, I first look at an opponent, then separate the hand from the rest of the image, work out what gesture the hand is showing, and finally work out who won. On a computer, we will use a similar set of processes. Neural Networks Neural networks are a type of Machine Learning, which is a type of AI technique as illustrated in Figure 1. Artificial Intelligence Machine Learning Neural Networks
Figure 2: Terminology in the hierarchy of AI techniques16
16
Image idea source: Neural Networks from Scratch (nnfs.io), by Harrison Kinsley and Daniel Kukiela P. 11, Published 2020
59
