Brain-Controlled Shark Attack!

Visiting my friends at OpenBCI HQ, we got together to do some hacking.  Since I'm always looking to control new things with my brain, I was really excited to see that someone had brought a remote-controlled shark-shaped balloon (an "Air Swimmers").  This is a very cool toy -- it swims through the air in a wondrous way.  But, I can't just leave a good things alone.  So, after a few hours of hacking, Joel and I were able to turn this simple toy into a 5-person, brain-controlled, SHARK ATTACK!

Approach:  Our approach to this hack is extremely similar to the approach that we used for our multi-person control of a toy robot.  As shown in the figure below, the idea is that you get multiple players hooked up to a single EEG system (OpenBCI, in my case).  The computer processes the EEG data looking for each person's eyes-closed Alpha waves.  Depending upon which person's Alpha waves are detected, the computer sends commands to the shark.  The commands are conveyed to the shark via an Arduino, which is driving the shark's remote control.  The end result is that the shark swims because of one player's brain waves. I think that's pretty cool.

Two-Person Demo:  As Joel and I were pulling this hack together, we started to test it using just the two of us.  Being just two people, we could only do shark two commands, not all five.  It was still pretty fun, though.  I love the sense of excitement that happens when a hack first starts to work.

Hacking the Remote Control:  To make this shark controllable from the computer, we needed to hack into the shark's remote control.  Like when I hacked the remote for the toy robot, Joel found that the remote for the shark was simply a few push buttons that were wired to pull one side of the switch down to ground whenever the button was pushed.  So, to make this controllable from my computer, Joel soldered some wires to the circuit board (to the high side of each switch) to allow an Arduino to pull it down to ground instead of having to push it with your finger.  As a result, we can now send a command to the Arduino and cause the shark to move.  Our Arduino code for this hack is on GitHub here.

We modified the shark's remote control by adding a wire to the non-grounded side of each push button.
We brought the wires out and connected them to an Arduino.

An Arduino drives the shark's remote control.

EEG Electrode Setup:  It's quite easy to record a person's eyes-closed Alpha waves.  You need three electrodes.  Put one electrode on the back of your head (O1 or O2, if you know the 10-20 system), put the EEG reference electrode onto your earlobe, and put the EEG bias electrode on your other earlobe.  You can see some examples in the photo below, where we had three people controlling the shark.  We used the gold cup electrodes and Ten20 electrode paste that came with the OpenBCI kit.

Three-Man "Team Alpha!" Controlling the Shark.  You can also see
where we put the electrodes -- back of head and both earlobes.

OpenBCI Setup:  We are going to wire up multiple people to control this shark.  And to be clear, it is not normal to hook multiple people to one EEG system.  But that is what we are going to do.  This is definitely using EEG in a non-traditional way.  That's why this is called "hacking".  The trick to making it work is to tell the EEG system (in this case, OpenBCI) that each person has his own EEG reference electrode.  OpenBCI enables this by allowing each EEG channel to be run in "differential mode", where you use each channel's the "P" and "N" inputs as a differential pair.  This is in contrast to the more-usual "common reference mode", where we use one of the SRB inputs as a common EEG reference for all EEG channels.  To change OpenBCI to differential mode, you use the OpenBCI GUI, via the "Chan Set" tab, to change each channel's "SRB1" and "SRB2" setting to "off".  Then, in the main window, turn off all of the other channels that you are not using.

Screenshot Showing How to Configure for Five Channels in
Differential Mode...simply Turn Off SRB2.

Plugging Into OpenBCI:  Once you've got the EEG electrodes on the individual players, you've got to hook them into the OpenBCI board.  Because we're in "differential mode", each player will get one "P" input and one "N" input.  For this hack, we put the electrode from back of the head into the "N" input.  We then put the left earlobe into the corresponding "P" input.  Finally, the right earlobe was connected to a bias pin.  Because we had five players, there aren't enough bias pins available on the OpenBCI board.  I used a 16-channel OpenBCI board because it has 4 bias pins, so that covered four players.  The fifth player simply plugged into the analog ground pin ("AGND"), which is not as good as using a bias pin, but it worked well enough.

Wiring Electrodes to the OpenBCI Board for Five Players.  Note that the board only
has 4 bias connections, so one player is attached to AGND instead.

EEG Processing Algorithms:  As mentioned earlier, the PC does all of the EEG processing...no processing occurs on the OpenBCI board itself.  For the software on the PC, we started with the stock OpenBCI GUI.  Then, I extended it by (1) adding the Alpha detection algorithms and by (2) adding code to send shark commands to the Arduino.  This variant of the OpenBCI GUI is currently saved here on GitHub as a branch of the main repository.  As you can tell from the class names shown in the code, the code is based heavily upon the previous work with the HexBug robot...which you may find confusing since we're controlling a shark and not a HexBug.  Sorry for the confusion!

Three-Person Testing:  After Joel and I did our two-person testing, we roped in a couple of other players to join Joel.  That got us to a three-man shark attack!

Three-Person Brain-Controlled Shark

Shark Food:  While we were all hacking the shark to make it brain-controlled, Conor was busy doing his own hacking.  Once we finally got our brain-controlled shark into fighting condition, we couldn't resist swimming it over to harass Conor.  Conor was pretty sure that his teeth were sharper than the shark's, so he wasn't much afraid.

Conor Faces Off Against the Shark.

Four-Person Shark Control:  Having successfully used three people to control the shark, we wired up a fourth person.  Have you ever tried to get four people doing anything in a smooth and coordinated fashion?  It's hard!  But, we did have success...

Five-Person Shark Attack:  OK, if four people working together is hard, five people is just chaos. In case you can't seem him, notice below that the fifth guy is in the center of the crowd, kneeling so that you just see his head popping above the bench.  The wiring on the OpenBCI electrodes seems generously long when you're just attaching one person.  With 5 people, though, you really need longer wires...or you simply need a little creativity on how you pack the people together.

Five-Person Brain-Control of the Swimming Shark.  The fifth person is kneeling
and you can only see his head.  We need longer wires!

Swimming Away:  By the time that we got this all working, it was really late at night.  The time stamps on the pictures show that it was about midnight, and we'd been at OpenBCI HQ since about 10AM.  So, between the fatigue and all the caffeine, I think that we were getting some funny brain wave behavior.  For example, one guy was making some weak Alpha even with his eyes open.  Wacky!  Regardless, we were able to get the shark to swim around...until we swam the shark too far to one end of the OpenBCI HQ...

Swimming the Shark Off Into the Sunset.

...at which point the shark's IR remote control could no longer communicate with the shark.  Stranded Shark!  And so our night of EEG hacking ended.  Still, it was mighty fine work,  Go Team Alpha!

Update 2015-03-09: I just saw that someone has already made a BCI for controlling this very same swimming shark (Chen et al.  "Recreational devices controlled using an SSVEP-based Brain Computer Interface (BCI)").  Note that they used one person to control the shark via SSVEP, which is exactly what I did with my brain-controlled Hex Bug!

Update 2015-09-24: Wow!  I was given an opportunity to write an article for IEEE Spectrum for their Oct 2015 issue.  How cool is that?  You can check it out here: "OpenBCI: Control An Air Shark With Your Mind".

Update 2015-11-08: I see that Wired (magazine) posted their nicely-done video on our shark hacking.  It's quite an enjoyable piece.  Good work, Wired!.

Two Brains - One Robot

After my success with sharing the brain-controlled hex bug with Conor and Joel, we brainstormed on how we could make this hack even more fun.  We decided that the main problem with this hack is that only one person gets to participate -- the person driving the robot.  The solution?  Let's hook up multiple people at the same time to control the one robot.  It'll be like that 3-legged race, where you tie your leg to the leg of another person, and then you stumble together in slapstick hilarity until you both get to the finish line.  We are going to do the same thing, but with brain-controlled robots.  Here's how far we've gotten so far...

The Plan:  Our goal is to have multiple people control one robot via their brain waves.  To do this, we aimed to connect multiple people to a single OpenBCI board.  I have never connected multiple people to one EEG system before, so this was pretty exciting for me.  As shown in the figure below, the idea is that each player is responsible for just one of the robot's actions -- one player is responsible for "Turn Left", another for "Turn Right", etc.  Since the robot has four actions (Left, Right, Forward, Fire), we can have up to four players.

The Hexbug robot has four commands (Left, Right, Forward, Fire), so for multi-player fun,
connect four people to one OpenBCI board and work cooperatively!

Commanding the Robot:  In setting up this hack, I wanted to make it as easy as possible for the players to command the robot with their brain waves.  The easiest brain waves to generate and the easiest brain waves to detect are Alpha rhythms (ie, 10 Hz oscillations), specifically the Alpha rhythm that naturally occurs when you close your eyes.  So, with the setup above, we have the computer looking for Alpha waves in each person's EEG signal.  If the computer sees Alpha waves from Player 1, the computer issues a "Turn Left" command to the robot.  If the computer sees Alpha waves from Player 2, it issues a "Forward" command.  And so on...

EEG Setup:  To detect these eyes-closed Alpha waves, we put one electrode on the back of a player's head over the visual cortex (position "O1" in the 10-20 system).  We put another electrode on one ear lobe to act as the EEG reference.  Finally, we put a third electrode on the other ear lobe to act as the EEG Bias.

Individual Reference:  To allow each person to use their own reference electrode, we configured the software on the OpenBCI board to put the ADS1299 EEG chip into per-channel differential mode.  Unlike our normal mode of operation, which uses a common reference electrode via SRB1 or SRB2, this differential mode allows each channel (ie, each player) to have its own reference.  This is what we want!  We simply plug the O1 electrode into the channel's "P" input and the ear lobe reference into the channel's "N" electrode.

Common Bias:  The only tricky part is that we want all four players to be connected to the OpenBCI Bias.  This is tricky because the OpenBCI board does not have four Bias pins.  Well, as you can see below, all it takes is a soldering iron and you can connect a piece of pin header to turn the single Bias pin into four Bias pins.  Now we're hacking!

OpenBCI V3 Board With Extra Pins Soldered to the Bias Output

Connecting the Pieces:   The picture below shows all the connections to the OpenBCI board assuming three players.  On the lower left, we've got three pairs of wires (one pair for each player) plugged into the "P" and "N" inputs of three different channels.  Then, in the upper-left, you see three wires plugged into three of the four new Bias pins.  Finally, in the upper-right, you see five wires that go off to command the hacked Hexbug remote control.

OpenBCI Board with Connections Ready for Three Players

Making It Happen:  Being a rare thing that me and Joel and Conor are all together, it was really fun that we could work together to make this hack happen.  Joel worked the soldering iron to attach the pins and he modified the Arduino code running on the OpenBCI board to enable the per-channel differential mode.  Conor further modified the Arduino code as well as the Processing GUI to enable slower turning of the robot (originally, it was turning WAY too fast).  Then, I modified the Processing GUI to enable Alpha detection on the four individual players.  We did all this in parallel.  I'd never really done group-hacking before.  It was definitely fun.

Conor and Joel working through the details of the connecting the Hexbug remote control.

Testing It:  Once we pulled together all of the pieces, Conor and I began to test the complete setup (see pic below).  After a little tweaking, we got the whole system working, as shown in the video at the top of this post.  It was a group effort that worked out.  Pretty sweet.

Conor and Chip -- Two Brains, One Robot.

Breaking Robots:  So our original vision was to get this hack working so that we could have *two* 4-person teams, with each team controlling their own robot.  Luckily, we had multiple robots and multiple OpenBCI boards, so we thought that we could make it happen.  Unfortunately, as soon as Conor and I made our video, the robots started to break.  They don't like being stuffed in suitcases, I guess.  So, we were left with just one working robot.  Bummer.

Recruiting a Team:  At the AF LabHack, there were lots of folks doing their own hacking.  By the time we got our system working (with the one healthy robot), the other teams were scrambling to get there last results prior to presenting to the group...so we had a tough time recruiting volunteers for being part of a robot-control team.  In the short time we had left, we did get three enthusiastic folks step up.  We got them all equipped with EEG electrodes, tuned the system a bit and let them play!

Our Fine Volunteers.  Three Brains, One Robot.  

No Video:  At this point, we should be presenting a triumphant video.  Unfortunately, we don't have one.  If we did, what you'd see is that two of the three players could easily and repeatably use their eyes-closed Alpha waves to command the robot.  It was cool to see.

No Alpha:  The third player, though, did not have much luck controlling his part of the robot.  At first, I assumed that it was a problem with our system, but after a little debugging, I came to the conclusion that his brain simply wasn't generating eyes-closed Alpha.  He could have been trying too hard (you must be relaxed, without concentrating or being overly focused), or he could have been part of the 11% of the normal, healthy population that simply does not generate Alpha upon closing their eyes [Ref 1].  For these folks, I've got to come up with an alternate robot-control methodology...perhaps by the concentration signature of counting-backwards-by-three.

Next Steps:  The next steps are clear -- I have to get a bunch of people together, hook them up, and enjoy the shenanigans of many brains trying to control a single robot.  Should be fun!

Ref [1]: Gibbs FA, Gibbs EL, Lennox WG. Electroencephalographic classification of epileptic patients and control subjects. Arch Neurol Psychiatry. 1943;50:111–28, as referenced by http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3927247/

Follow-Up:  We used a similar approach to get a 5-person team to brain-control a swimming shark balloon.  It's cool.  Check it out here.

Controlling a Hex Bug with my Brain Waves

Ever since my effort with OpenBCI began, I've been looking to control something with my brain.  Sure, a while back, I was successful in lighting an LED with my brain waves, but that's pretty simple.  I wanted something more.  And now I can do it.  I can control a robot with my mind!  Yes!

Approach:  My robot has just a few actions that it can do...turn left, turn right, walk forward, and fire.  To make this brain-controlled, I need a way to invoke these commands using signals from my brain.  Ideally, I'd just think the word "Fire!" and the robot would respond.  Unfortunately, those kinds of brain waves are too hard to detect.  Instead, I need to use brain waves that are easy to detect.  For me, "easy" brain waves include the Alpha waves (10 Hz oscillations) that occur when I close my eyes, as well as the brain waves that occur when I watch my blinking movies (a.k.a. visual entrainment).  So, my approach is to use OpenBCI to record my brainwaves, to write software to detect these specific types of brain waves, and to issue commands to the robot based on which brain waves are detected.

Here are all the pieces that you see in the video

Hardware Setup:  The core hardware for this hack is similar to my usual OpenBCI setup: EEG electrodes, an OpenBCI board, an Arduino Uno, and my computer. Added to this setup is the Hex Bug itself and its remote control, which I hacked so that the remote can be controlled by an Arduino.  So, as shown below, my brain wave signals go from my head all the way to the PC.  The PC processes the EEG data looking for the Alpha waves or the visually-entrained waves.  If any are detected, it decides what commands to give the robot.  The commands are conveyed back to the Arduino, which then drives the remote control, which the Hex Bug receives over its usual IR link.

Here is the schematic of how the pieces work together.

EEG Setup:  I'm going to be measuring my Alpha waves and I'm going to be measuring the brain waves induced through visual entrainment.  Based on my previous experience, I know that both are best recorded using an electrode on the back of the head (at the "O1" position, if you're into your 10-20 electrode placement standard).  I do not need electrodes all over my head.  That's the only sensing electrode that I'm using.  That's it.  Of course, EEG also requires a reference electrode, which I put on my left earlobe.  And, finally, EEG often has a third electrode ("bias" or "driven ground"), which I placed on my right earlobe.

Looking at the Frequency of my Brain Waves:  As mentioned above, my approach is to control my robot by detecting Alpha waves and by detecting visually-entrained brain waves.  These are easily detectable because they occur at specific frequencies.  Alpha occur around 10 Hz and the visually-entrained brain waves occur at the blink rate(s) of whatever movies I use (my best results were from 5 Hz and 7.5 Hz movies). So, to control my robot, I will be looking for EEG signals at these frequencies: 5 Hz, 7.5 Hz, and 10 Hz.  I'm going to "look" for these frequencies by writing some EEG processing software that'll look at the frequency content of my EEG signal to see if these frequencies are present.

EEG Processing:  The flow chart above shows the steps that I use to process the EEG signal (my software is here).  Once the PC gets EEG data from the OpenBCI board, the first step is to compute the spectrum of the signal, which tells me the content of the EEG signal as a function of frequency.  I then search through the relevant part of the spectrum (4-15 Hz) to find the peak value.  I note both its frequency value and its amplitude.  In parallel, I also compute the average EEG amplitude across the 4-15Hz frequency band.  This average value is my baseline for deciding whether my peak is tall (strong) or short (weak).  By dividing the amplitude of my peak by this baseline value,  I get the signal-to-noise ratio (SNR) of the peak.  The SNR is my measure of the strength of the peak.  The output of the EEG processing, therefore, are two values: the frequency of the peak and the SNR of the peak.

Deciding My Robot's Action:  Once my EEG processing finds the frequency and SNR of the peak in my EEG spectrum, I now have to decide how to act on that information.  After some trial and error, I settled on the algorithm shown in the flow chart above.  It's got three steps:

• SNR Check:  First, I decide whether the current peak in the spectrum is legitimate, or if it is likely to be just noise.  I don't want to issue a command if it is just noise because then my robot will be taking all sorts of actions that I didn't intend.  That is not what I want.  So, to decide if the peak is likely to be legitimate, I look at the SNR of the peak.  If it has a big SNR, I'll accept it as a legitimate peak.  If it is too small, I'll take no further action.  Right now, my threshold for this decision is at 6 dB.  Setting a higher threshold results in fewer false commands (which would be good), but it also makes the system less sensitive to legitimate commands (which is bad).  This 6 dB threshold resulted in an OK (but not great) balance.

• Frequency Check:  If the peak seems legitimate, I decide how to command the robot based on the frequency of the peak.  If the peak is between 4.5-6.5 Hz, I must be looking at the right-side of my 2-speed blinking movie (ie, the portion that blinks at 5 Hz), so the computer prepares the "Turn Right" command.  Alternatively, if the EEG peak is 6.5-8.5 Hz, I must be looking at the left-side of my 2-speed blinking movie (ie, the portion that blinks at 7.5 Hz), so it prepares the "Turn Left" command.  Finally, if the EEG peak is 8.5-12 Hz, it must be my eyes-closed Alpha waves, so the computer prepares the "Move Forward" command.

• New Command Check:  Before issuing the command, I check to see whether this command is the same as the last command that was extracted from my brain waves.  If the latest command is different, I hijack the command and, instead, issue the "Fire!" command.  If the latest command is the same, I go ahead and issue the left / right / forward command like normal.  The reason for this hijack is that I have no other type of easily-detected brain wave that I can use for commanding the robot to fire.  This approach of issuing "Fire!" on every change in command seemed like a decent way of getting a 4th command out of 3 types of brain waves.

Putting It All Together:  As you can see in the movie, I eventually able to get all of these pieces working together to allow me to command the Hex Bug using just my brain waves.  Of course, it didn't work the first time.  Even once I got all the hardware working, I still needed to tune a bunch of the software parameters (FFT parameters and the detection threshold) until I got something that worked somewhat reliably.  To help with this tuning process, I used the spectrum display that is in my Processing GUI.  Some screen shots are below.

Example EEG spectrum when I stared at the right side of my two-speed blinking
movie.  It induced 5 Hz brain waves.  I programmed 5 Hz to mean "Turn Right".
The SNR here is between 6 and 7 dB.

Here's an example EEG spectrum when I stared at the left side of my two-speed
blinking movie.  It induced 7.5 Hz brain waves.  When the GUI detected 7.5 Hz,
it issued a "Turn Left" command to the Hex Bug.  The SNR is only 6-7 dB.

Finally, here's an example EEG spectrum with my eyes closed so that I was
exhibiting Alpha waves, which are near 10 Hz.  When it detected 10 Hz, I
programmed it to issue a"Forward" command.  The SNR is > 8 dB.

Weak Peaks:  In the screenshots above, the red line shows the current EEG spectrum.  The heavy black circle shows the spectral peak that my software algorithms have detected.  The black dashed line is the "background noise" from which the SNR is computed.  To be declared a legitimate detection, the peak must be 6 dB higher than the black dashed line (unfortunately, I don't show this on the plot...sorry!).  As can be seen, the 5 Hz and 7.5 Hz examples are not very strong (the SNR is only 6-7 dB).  Other peaks within the plots are very close to being the same size, which would cause false commands to be sent to the robot.  In my movie at the top of this post, there were several false commands.

Balancing Sensitivity with False Commands:  To reduce the number of false commands, I could raise my detection threshold above 6 dB. Unfortunately, as see in the first two spectrum plots above, my 5 Hz and 7.5 Hz peaks are usually pretty weak (<  7 dB).  Therefore, any attempt to raise my detection threshold above 6 dB would cause me to no longer detect my legitimate brain waves.  I know because this is exactly the tuning process that I tried.  Bummer!  So, if I want more reliable performance, I'll need to develop a fancier signal processing beyond this simple FFT-threshold approach.  Future challenges!

Wrapping Up:  Even with the false commands seen in my movie, I was still able to command the robot to move around the table.  I could get it to go (roughly) where I wanted it to go.  And, I did it all with just my brain waves.  I think that this is pretty exciting!  Yay!  What are the next steps?  Well, maybe now that I have this under my belt, I can move on to control flying fish, or maybe a quadcopter!  Do you have any other cool ideas for things I can control with my brain?

Coolness:  This hack got picked up by IEEE Spectrum as part of an article on OpenBCI.  Cool!  Check it out here.

More Coolness:  This hack also got picked up by Wired.  Fun!

Follow-Up: I got to share this hack with Joel and Conor of OpenBCI.  You can see their luck with controlling the robot here.

Follow-Up: Follow-Up:  We used a similar approach to get a 5-person team to brain-control a swimming shark balloon.  It's cool.  Check it out here.

OpenBCI Alpha Wave Detector

Part of what excites me so much about EEG hacking is the idea that maybe I can control things with just my mind.  Once you gain any experience at all with EEG signals, however, you realize just how hard a task that can be.  So, start simple.  Start with what's easy.  Build from there.  With EEG, "easy" are Alpha waves.  Alpha waves is nice a simple EEG waveform that shows up fairly strongly around 10 Hz.  It is a great target for making one's first brain controlled hack.  And I just did it myself...check it out!  It's completely computer-free and feels like magic!

Electronics Setup

For this hack, I used the parts shown in the picture below.  It's an OpenBCI board sitting on an Arduino Uno (the Uno is completely hidden in this photo under the white OpenBCI board), a cheap peizo buzzer, and a basic red LED.  I put a 330 ohm resistor in series with the LED and in series with the buzzer to limit the current (a practice recommended in all of the Arduino learning examples).  Because I wanted to be completely computer-free, the whole thing is powered by a 9V battery adapter.

OpenBCI plus Arduino Uno plus LED, Piezo Buzzer, and 9V Battery.

EEG Setup

This hack uses two off-the-shelf reusable EEG electrodes.  I plug them into my Open BCI board using the adapter cable that I made.  I placed the first electrode (the reference) on the top of my head, a little towards the front ("Fz").  I placed the second electrode on the back of my head, just above the bump on the back of the head (the "inion") and a little to my left (aka "O1").  I worked the electrodes under my hair and stuck them in place using ten20 conductive paste.  This time, I used the small jar of it instead of the tube...the jar is much easier to work with.

My setup, including the colorful homemade adapter cable for
attaching my two EEG electrodes.

Arduino Processing Software

Unlike all of my previous experience with EEG hacking, where a PC was involved for doing the signal processing, today I will be doing all of the EEG processing on the Arduino itself.  The Arduino is not a computational power house, so we have to be reasonable in our expectations when doing signal processing on the Arduino.

To detect Alpha waves in EEG signals, there are several things that you need to do:

1. Filter to remove strong interfering signals (60 Hz and DC drift)
2. Filter to focus on the frequencies of interest (in this case, Alpha are ~10 Hz)
3. Decide if there is enough Alpha (and just Alpha) to activate the LED and buzzer

The full details of my signal processing chain is a long story that will get its own post.  The Arduino's limited bit depth (32-bits is the maximum data type) and limited speed with floating point operations means that you cannot use sharp filters.  Instead, you have to use relatively gentle filters such as "biquad" filters (which are a form of 2nd order IIR filter).  To generate the filter coefficients, I used code that I found here.

After doing my filtering, I then compute the RMS power of the signal.  I simply square the single, apply a low-pass filter (another biquad from the code above), and take the square root.  This gives you a running RMS estimate of the signal amplitude.  Because of my filtering, this gives me a running estimate of the Alpha amplitude.  I then light my LED and sound my buzzer based on the amount of Alpha energy.

Thinking Things to Action

So, with this setup, you've got yourself a self-contained device that responds to your brain waves.  It doesn't have to be just an LED or buzzer, either.  It could be a robotic arm swinging a sword (to fight off pirates, of course), or a cool flashing hat for dance parties, or an animatronic flower that blooms with your thoughts.  Or maybe you like the idea of getting this kind of neurofeedback to see what is happening when you meditate (change the filter from 10 Hz Alpha up to 20 Hz Beta).  Or, maybe you could use it simply for weirding-out your friends and co-workers...what with the strange wires coming out of your head and all.  There are so many possibilities.

To me, controlling things with your brain in this way seems a bit magical...but I made this thing myself, so I know that it's not magic...it's hacking!

Follow-Up: A long time has passed, but I think that I found the code for this hack.  This was prompted by folks on the OpenBCI forum trying to do it themselves!  Cool!

Follow-Up:  I extended this work to control a six-legged robot with my brain waves!

Mu Rhythms for BCI

As discussed previously, there are several approaches that are currently being pursued for Brain Computer Interfaces (BCI).  One approach is to perform an EEG and to measure the "Mu Waves" or "Mu Rhythms" from the sensorimotor portion of one's brain.  The Mu Waves are associated with you moving your body -- either by actually moving your body, or by you *thinking* about moving your body.  Sounds like a great way to command a computer, eh?  Let's dig in a little more...

What Are Mu Waves?  The first paragraph on Mu waves in Wikipedia seems decent enough.  As it says, Mu Waves are a type of oscillating electrical rhythm within the brain that can be seen in an EEG.  Specifically, they occur in the sensorimotor cortex, which is the portion of the brain associated with coordinating muscle motion and the perception of ones muscle and joint motion.  Looking at the image below, the sesnorimotor cortex as the areas labeled "Primary Motor Cortex" and "Primary Somatosensory Cortex".   It is relatively narrow strip going from one ear, up over the top of the head, to the other ear.  This is where Mu Waves seem to occur.

Illustration of Sections of the Brain (via UIC)

When Do they Appear?  It is my understanding that Mu Waves appear naturally when your body is physically relaxed.  The appearance of the Mu Waves are an indication that the sensorimotor portion of your brain is "idling".  When you move a major body part, those portions of your brain stop "idling", they get down to real work, and the Mu Waves go away (are "suppressed") during the motor activity.  Amazingly, this portion of your brain exhibits the same Mu Wave suppression simply by imagining the motion of a body part.  Even better, the specific portion of your cortex where the Mu Waves are suppressed is linked to the body part that you're imagining moving.  Now that's cool!

From BCI2000.  The different regions of the sensorimotor cortex, *roughly* correspond to different body parts.  Feet and legs are near the top of the head.  Hands are near the middle.  Face and tongue are near the bottom of the cortex, which on your scalp is located just above your ears.

Difference From Similar Rhythms:  The Mu rhythm occurs in the frequency range commonly referred to as Alpha waves (8-12 Hz).  There are several sources of activity in the Alpha range.  The most common trigger for Alpha waves is simply to close your eyes.  In most people, closing your eyes idles the visual cortex (the whole back portion of your brain...the "occipital" region), which causes Alpha waves to appear throughout the rear portion of the brain.  This called the Posterior Dominant Rhythm.  The Mu Waves, by contrast, are associated with the sensorimotor portion of your brain, so they should only appear in the signals from the electrodes over that part of the brain.

How Can We Measure Our Mu Waves?  Theoretically, if you hook up an EEG sensor system to your scalp, and if you put some electrodes exactly over the sensorimotor portion of your brain, you should be able to see Mu Waves when you relax your body.  I have yet to be successful with this, though I will try again.  In preparation, I have been reading the tutorial from BCI2000 to get a better idea of where to put my electrodes and which electrodes to use for reference and bias.

Mu Waves in EEG Traces:  Below is a cool video that shows what Mu Waves look like in raw EEG traces. Being localized to just the sensorimotor cortex, they appear most strongly in the F4-C4 trace.  This link also shows Mu waves in a raw EEG trace...in this montage, they're seen most strongly in the F3-C3 trace.  In my own trials, I have not specifically plotted these two combinations of electrodes.  I will.

What Could We Do with Mu Waves?  In the EEG traces above, it appears that the presence or absence of Mu Waves is pretty easy to see...we can probably get a computer to detect their presence pretty easily.  Once the computer sees that they're present, we can imagine moving our body, which should make them go away.  The computer can see that they went away and can take some action (like moving a robotic limb). It would only be a simple on/off control, but it still would be cool!

Using Mu Wave for Fine Control of a BCI:  Mu waves are compelling for BCI, though, because we don't have to be satisfied with simple on-off control.  Take, for example, the fact that our bodies and brains are sided -- the left side of your brain controls the right side of your body, and vice versa.  So, if by imagining motion with the left side of your body, the Mu waves should only be suppressed on the *right* side of your brain.  The converse is true as well -- imagining motion on the right side of your body should suppress the Mu waves on the left side of your brain.  As a result, you should be able to use a Mu wave reading BCI to control a robot to move in two ways...say, left or right.  Now it's getting useful!

Using Different Body Parts:  But we're not done.  Mu waves are quite local.  If you imagine moving just your feet, the Mu waves are only suppressed in a small portion of your sensorimotor cortex that, for the feet, is near the top of your head.  Imagining moving your hands suppresses the Mu waves in a different part of the cortex (down closer to the ears).  So, with more electrodes -- electrodes that are carefully placed over the different regions of the sensorimotor cortex -- we should be able to distinguish between thoughts of moving your hands versus moving your feet.   The movie below shows an example of a group who built a BCI that achieves this.  Fantastic.

The Future:  In theory, more electrodes on the scalp could maybe yield an even finer distinction between body parts, though I've only seen BCIs that do hands versus feet.  Maybe now is the time for a break-through!

Follow Up:  Check out my Mu waves!

Brain Control Interfaces - Different Approaches

An important question to be able to answer is "Why do you spend all this time hacking with EEG?".  For me, there are a number of answers.  My first answer, though, is that I'm really interested in brain-computer interfaces (BCIs).  I want to be able to control things with my brain.  Why?  Because, when successful, it's like magic.  It's like THE FORCE from Star Wars.  It's the kind of thing that, when demonstrated in real life, gets a heart-felt "Whoa!" from unsuspecting on-lookers.   It's cool.

There are a number of different methods of implementing a brain computer interface.  The first major division in approaches is whether the BCI is invasive or non-invasive.  In this context, "invasive" means that a surgeon cuts open you head, saws open your skull, and implants electrodes directly in your brain.  If you're a quadriplegic, you might be willing to have this done in order to get your best chance at a BCI that works well.

For the rest of us, though, we might be more interested in a non-invasive BCIs that sense your brain waves by electrodes on the scalp.

How do BCIs listen to the signals from your brain (via your scalp) and do something useful?  To my understanding there are three approaches: Frequency Analysis, Mu Wave Detection, and Event-Related Potentials.

Frequency Analysis

The simplest approach is simply to look at the frequency content of the EEG signals recorded from the scalp.  Since nearly everyone produces alpha waves when they close their eyes, a straight-forward example of a frequency-based BCI would be to program the computer to move a motor in proportion to the alpha waves measured in the EEG signals.  I've done it.  It's fun!  More complex control schemes can be developed by looking at more frequency bands (theta, alpha, beta, etc) and by looking at different or multiple locations on the scalp.  With this added range of variables, you can do more complex things.  The video below shows an example of this kind of setup.

The hard part is that most people cannot easily control the frequency content of the signals in their head.  Usually, you're asked to control wishy-washy aspects of your mental/emotional state such as "alertness", "relaxation", "focus", etc.  How do you do that?  Well, it requires much practice and, to date, has yielded unreliable results for most people.  But, it is easy to implement on the computer, so it's a good starting place for people hacking their own BCI system.

Mu Waves (Mu Rhythms)

A special case of the "Frequency Analysis" methods is a method based on looking for "Mu Waves".  Mu waves are special because they occur in the motor cortex (or, more precisely, in the combined sensorimotor cortex).  If you can get your scalp electrodes in the right place, you will see Mu waves whenever your body is physically relaxed.  When you contract the muscles in a relevant body part (or, even if you just visualize yourself contracting the body part), the Mu waves in that part of your brain get suppressed.  So, the EEG setup is a little harder, but one's ability to actually control these brain waves is much better.

To get more information on how to do a Mu wave BCI, the BCI2000 folks have some great information
http://www.bci2000.org/wiki/index.php/User_Tutorial:Mu_Rhythm_BCI_Tutorial

For another example, check out the video below.  They built a BCI for playing World of Warcraft.  If you skip to 0:46, you see how they put together the system and how, through the subject moving his feet and hands, they trained the computer to understand his brain waves.  This use of physical motions is almost certainly training the system to look for the subject's Mu waves.  Furthermore, note that the only electrodes that are wired-up on his EEG cap are the ones over his motor cortex.   It's gotta be a mu-wave system.

Follow Up: Here's more discussion of using Mu waves for my BCI.

Event-Related Potential (ERP)

A third way to do a BCI is to measure event-related potentials (ERPs).  ERPs are EEG measurements in response to a particular sensory stimulus, which then causes a particular response in the brain.  Often visual stimuli are used via a computer screen.  This is useful for BCI because, if the user is consciously paying attention to the visual stimuli, his brain gives one type of response (that is detectable via EEG), while if he ignores the stimuli, it gives a different response.  This means that the human subject can consciously interact with the computer simply through selectively focusing (or not) on the visual stimuli.

The video below presents a typical setup.  Here, the computer presents a grid of letters on the screen.  The human subject wants to spell a word, so he focuses his attention on a letter on the screen...the letter "S", for example.  The computer then randomly highlights the letters on the computer screen.  Whenever the letter "S" is highlighted, the human recognizes that his letter was highlighted and his cognitive response causes a quick and temporary change in his EEG signals (the "P300" feature appears).   Unfortunately, the P300 is a very subtle change, so the whole process has to be repeated many times so that the recordings can be averaged together to make the P300 detectable.  If the computer has to flash through the whole keyboard, you can imagine how slow this is.  The video below illustrates the slowness...he gets about one letter every 40 seconds.

Still, even though it is slow, ERP interfaces allow for a very rich interaction with the computer that can be more complex than the simple "left", "right", "forward" commands seen in the World of Warcraft video above.  Plus, the system used in the video is not the be-all and end-all in ERP interfaces.  This is a very new field and many advances are possible.

If you want to learn more about (or try!) a P300 ERP system, the BCI2000 folks also have some tutorials:
http://www.bci2000.org/wiki/index.php/User_Tutorial:P300_BCI_Tutorial