USF's First Brain-Controlled Drone Race

By Russell Nay

Photo: Sandra Roa

Students, faculty and USF community members watched as USF students faced off at the USF Yuengling Center during the university’s first-ever brain-controlled drone race.

The crowd of more than 300 filtered into the Corral of the event center, where the USF Volleyball team court was cleared and surrounded by nets with two sets of tables at each end. For the next four hours, groups of two student competitors would go head-to-head on the court to see who could move a drone the fastest with nothing but their minds.

The students, who represented the U.S., Brazil, India, Dominican Republic, Venezuela, Bangladesh, Kuwait and Japan, were the 16 students with the quickest times who passed a qualifying round in January to compete in the race. A total of 60 students registered for the qualifier.

The event’s announcer counted down each race, and on “go” the racers would focus on a computer screen window with a multicolored cube in the middle. By focusing on pushing the cube while wearing their electroencephalography (EEG) headbands, students could coax their drones to lift off their starting pads and hover toward the finish line.

Brain-Drone Racing League members and volunteers watch a drone pull ahead as racers at the other end of the court prepare for their match.

Photo: Ryan Wakefield

Reaching the other side of the court, however, was no easy task. Some drones refused to budge, while others stopped halfway across the court as their drivers had a momentary lapse in focus. Racers needed to focus completely on their drone moving forward to win their matches.

Representing Kuwait, computer science and engineering graduate student Khaled Alshatti won all four of his matches to be declared the champion of the competition and said his key to winning was blocking out distractions.

“During the race I tried not to focus on anything around me and just focus on the simulation itself,” Alshatti said. “It was mainly distracting whenever I saw the person racing against either moving their drone or getting close to my drone.”

While the prize for winning the competition included a Tello drone and title of USF Brain-Drone Race champion, Alshatti said the research behind the race motivated him to sign up.

Organized by the Neuro-Machine Interaction Lab led by USF Department of Computer Science Professor Marvin Andujar, Ph.D., and Brain-Drone Racing League volunteers, this brain-drone race was the second that Andujar has planned. The first, held at the University of Florida (UF) in 2016 when he was a Ph.D. student there, was the world’s first brain-drone race at a university.

“I’ve always wanted to work with (Andujar) and do research with him,” Alshatti said. “Seeing the (registration) email and knowing it was part of his research really got me interested in it as well as me watching drone racing at home.”

Computer science and engineering senior Emily Cardella, one of the competitors representing the United States, said she enjoyed getting to use the brain-controlled drone technology in a race and would encourage more of her classmates to sign up for future potential races on campus.

A brain-controlled drone takes off during USF’s first Brain-Controlled Drone Race.

Photo: Ryan Wakefield

“It was fun just to be part of making USF history,” Cardella said. “The whole marriage between hardware and software was what got me interested in computer engineering in the first place, so something like this that combines the two is perfect … Now I want to join Dr. Andujar’s lab just to see how everything works.”

Before the race at UF, Andujar was already studying brain-computer interfaces (BCIs) and brain-controlled drones at Clemson University with his colleague Chris Crawford in 2012.

A BCI like the EEG headband worn by competitors allows users to send electrical signals to a computer or other device like a surgical implant. These signals can then be programmed to trigger a certain task, like the fingers of a prosthetic arm closing or a drone lifting off the ground.

Andujar said that holding an event like the brain-drone race is a great way to introduce BCI technology to the public with an application of the tech that anyone can be a part of. This includes users with physical disabilities. Unlike traditional drone racing which requires a controller, brain-drone racing requires only brain activity.

Brain-Drone Racing League members prepare for the next race.

Photo: Sandra Roa

When he interned at Intel in 2013, he said that Intel team members would ask him what he thought the “killer app” for the technology was. That is, what BCI application would be most popular and contribute to its widespread adoption. He and Crawford thought brain-controlled drones were a good choice.

“We saw drones and especially drone racing becoming more popular,” Andujar said. “When we got to UF in 2014 … to finish our Ph.D., that’s when we came up with the idea of brain-drone racing.”

BCI technology was first developed for biomedical applications — like assistive devices and prosthetics that help users regain lost movement and communication abilities — but has since contributed to research in a wide variety of fields from security and neuromarketing to entertainment and gaming.

The Neuro-Machine Interaction Lab began in the summer of 2018 and currently consists of a team of four Ph.D. students, one master’s student and about 20 undergraduate students. Andujar said there’s also a waiting list of 30 undergraduate students waiting to join.

“We’re not bigger because we don’t have the space,” he said. “I get students every other day and from all over who tell me they want to join the lab as a Ph.D. student.”

One of the lab’s earliest projects was a simulation of a brain-drone race, which lets players use a real EEG headband to fly a virtual drone through a simulated race track. The lab demoed the simulation at the 2018 and 2019 Synapse Summit tech events in Tampa.

Computer science and engineering graduate student Khaled Alshatti is presented with a trophy after winning all four of his matches at the USF Brain-Drone Race.

Photo: Sandra Roa

The lab team is currently working to research how a drone’s camera can be used to analyze emotion and also on an app that can help students monitor their brain activity to see how attentive they were while studying. Andujar said he plans to expand work on this app to study how it can be used specifically by individuals with Attention Deficit Hyperactivity Disorder.

Lab collaborators include Intel, which donated three drones to use during the USF Brain-Drone Race, Tampa video production company Diamond View - which created a promotional video for the race - and Ybor City-based U.S. Special Operations Command collaborator SOFWERX, which is working with lab researchers on projects for the U.S. Air Force.

Andujar said that he still collaborates with Crawford — who now researches human-computer interaction at the University of Alabama — on further developing brain-controlled drone technology. At the time of the USF Brain-Drone Race, Andujar had not heard of another lab at an American university that researches brain-controlled drones. He said that he and Crawford are two of only a handful of researchers currently researching the subject in the U.S.

Dante Tezza, a second-year Ph.D. computer science and engineering student in the Neuro-Machine Interaction Lab, will likely soon join them. Tezza worked with unmanned aerial vehicles (UAVs) for the Brazilian Air Force, contributed greatly to the first fully autonomous takeoff and landing system in Brazilian aviation history and spent nearly four years in the UAV industry after getting his undergraduate degree.

He said he decided to join the lab last year after hearing about the lab’s work with brain-controlled drones. He’s currently researching new methods people can use to interact with drones including via a person’s emotions. Tezza and fellow lab Ph.D. student Sarah Garcia also had a paper accepted to appear at the July 2019 Human-Computer Interaction International Conference regarding the use of BCIs for gaming and brain-controlled drones.

“I think the biggest advantage of graduate school is that you can really pick the area that you want to work in,” Tezza said. “Basically everything that I work with here, I want to keep working with after I graduate. It’s great experience and lets me work in projects that I really enjoy, so it’s definitely aligned with my goals.”

Blanche Pinto, who will be starting her computer science and engineering Ph.D. program at USF in the fall, said she first joined the lab while taking a class on BCIs taught by Andujar. Talking with Tezza and Andujar about plans to host a brain-drone race at USF piqued her interest, and in the spring of 2018 she became president of the Brain Computer Interface Club. The club is a student organization made to teach students about BCI tech and promote brain-drone racing on campus.

As the lab’s event manager and BCI club president, Pinto played a large role in planning and organizing the USF Brain-Drone Race. In previous years, she was on the USF Engineering Expo student board, which she said was great experience for organizing events at USF. It also helped her navigate the obstacles of planning an event involving drones, including space and netting requirements, special permits and approval from three different offices on campus.

Pinto currently works on the lab’s brain-controlled virtual reality painting project, which would allow users to control brushstrokes in a VR art program. The lab plans to publish a paper on the project in the future and currently has a prototype in the works.

Pinto said that further developing the BCI club is also on the lab’s to-do list this year.

“Our main goal was to host the race, but we also want to tell people about the tech, and we’re planning on hosting workshops in the future on how to build drones and how brain-controlled drones work,” she said.

Angela Rodriguez, a USF computer science and engineering student and Brain-Drone Racing League volunteer, said she loved taking Andujar’s course on BCI and its applications. She also enjoyed seeing her son Angel Romeu get to pilot a brain-controlled drone himself at the league’s community table where audience members could line up and put on EEG headbands of their own.

“I think it’s a new technology that will succeed because there’s so many possibilities for using it to help people in various ways and for entertainment,” Rodriguez said.

Romeu, a Tampa middle school student, said that while he became familiar with the concept of brain-drone racing at last year’s USF youth robotics competition Roboticon, seeing the Brain-Drone Racing League’s first official competition at USF and getting to fly a brain-controlled drone himself was a completely different experience.

Romeu went to Roboticon through the College of Engineering’s Bulls-EYE Mentoring program, which provides underrepresented middle school students in the USF Tampa community with STEM and engineering experiences through a summer program at USF.

“I think it’s really cool and extraordinary how the technology is getting better these days,” Romeu said. “To be honest, this is something I didn’t think would be possible until like the 2030’s.”

In the future, Andujar said he hopes to send USF students to brain-drone races hosted at universities all over the world. He said the Brain-Drone Racing League, of which the University of Alabama and the University of Florida are currently members, has also received interest from schools including the University of Cambridge and the University of Tokyo. Ideally, he said, this year’s race will also serve as a pilot for races that would welcome students from other universities around the world in future years.