Wednesday, October 30, 2013

Conor and his Brain Hat and Android App

One of my collaborators on OpenBCI is Conor.  He's done some really cool EEG hacks.  This one is really sweet:

It uses a Neurosky Mindflex, an Arduino, and a Sparkfun bluetooth module.  It monitors his brainwaves, analyzes the energy content in different frequency bands, and sends the data out over Bluetooth to his Android app.  The Android app that he's using (did he write it or use an existing one?) looks like:

With this app he can log the EEG data along with his mood and activity at the time.  Through post-test analysis he can look for correlations.

Overall, this is a sweet project.  I'm looking forward to stealing...I mean, borrowing...I mean stealing his ideas for my own hacks.

Thanks Conor!

Sparkfun EEG Hacker-in-Residence

I just saw this post from Sparkfun regarding their latest "hacker in residence".  Apparently, Sophi is going to be hacking a Neurosky "Mindwave" EEG headset to create a brainwave interface to other hackable electronics.  Cool!  Here's the quote:

Sophi: I’ll be working with Neurosky’s brainwave headset to create a hardware interface that makers, engineers and educators can use to create projects that trigger on brainwaves. I am very curious about the brainwave headset (it’s called a Mindwave). The Mindwave uses one electrode to measure attention, concentration and eye blinks. I want to see if anything usable can be registered from one sensor intended to get a signal from an object (the brain), that sits inside of a bunch of fluid and then has a skull and hair and skin on top of it.
A couple of years ago I got to ride The Ascent, which uses the Mindwave headset to control the ride. The Rider is strapped into a harness, which is connected to a computer-controlled winch. As the Rider concentrates while wearing the Mindwave, the winch pulls the Rider as high as 30' up towards the ceiling. It was an incredible experience!
I’d like to explore the different applications for this technology, such as using an eye blink to fast-forward your music while running, or wagging a fake tail at ComicCon. Do you have an idea for an application? Put it in the comments!

My favorite part is that this is exactly the kind of stuff that we're looking to enable with our OpenBCI device. I'm always happy when I find folks out there like me who are into brainwaves and into mind-control over their hacks.  Good luck Sophi!

Tuesday, October 22, 2013

Finding My Mu Waves

I'm trying to control things with my brain.  As discussed in my previous post, I think that Mu waves are the best approach that I know about right now.  I tried to get my Mu waves before, but failed.  Now I'm trying again.  This time, I've done a little more learning, so I think that I know how to do it better.  Let's go!

Listening to My Brain
Approach:  The idea with a Mu wave brain-computer interface (BCI) is that I'll use my EEG system to listen to my brain waves.  Mu waves are simple but strong signals in the Alpha band that appear in the sensorimotor (SM) cortex when you relax parts of your body.  The Mu waves go away when you move (or *think* about moving!) those parts of your body.  Supposedly, different regions of the SM cortex correspond to different portions of your body.  So, with careful use of EEG, I might be able to separate thoughts of moving my legs from thoughts of moving my arms.  Seeing other folks do this on the web, this is pretty exciting stuff.

Goal:  My goal today is not to make a full Mu wave BCI.  That's too big a leap.  Today, I just want to see if I can pick up my Mu waves.

Equipment: To do this test, I need some EEG electrodes and an EEG system:

  • For the EEG electrodes, I'm using re-usable gold-plated cup electrodes from Biopac (EL160, see pic below).  This style of electrode is often used in hospitals and in research settings.  I'm using this style of electrode because I can easily slip them within my hair and stick them to the skin.  To use the electrodes, you need some electrode paste, which is both conductive and sticky.  I used Ten20 paste, also available from Biopac.  Just swab some paste into the electrode cup and stick it firmly onto your scalp, with as little hair as possible between the skin and the electrode.

Re-Usable Gold-Plated Electrodes from Biopac
  • For the EEG system, I'm using the OpenBCI system that I'm helping to develop.  This is an open source EEG system that mates to a microcontroller, such as an Arduino.  I'm using an Arduino Uno, which is simply pumping the EEG data from the OpenBCI board to a PC.  On the PC, I'm running some software to capture the data from the serial port.  I can view the data in real time, but really, I'm going to do most of the processing afterwards.
Using OpenBCI as My EEG System

EEG Montage:  I think that a key challenge with measuring Mu waves is to get the electrodes in the right place.  They must be over the sensorimotor cortex or you're not going to see them.  How do you find the sensorimotor cortex?  Well, you have a good chance of getting it simply by drawing a line from the front of your ear (specifically, your tragion) up over the top of your head.  Really, though, you should follow the directions for finding EEG locations C3 or C4 according to the proper layout of the 10-20 system.  On my head, I put electrodes at C3 (left side), C4 (right size), Cz (top), and Oz (back).  I used Fz (front-top) as my reference electrode.  In the end, though I really only needed C3 and Oz along with the Fz reference.

Electrode Locations Used in My Testing
Another View Showing the Electrode Locations Used in My Testing

Test Plan:  So we need a test plan that will help me see Mu waves, which means that part of my test needs to have me be physically relaxed.  Then I need to make the Mu waves go away, which means that part of my test needs to have me move my body (I'll move my hand).  Actually, let's get fancy...I'm just going to *think* about moving my hand.  Finally, if I do see Mu waves, I need to make sure that I'm not just seeing alpha waves from the back of my head, so a third part of my test will be to close my eyes to induce my Posterior Dominant Rhythm, which are Alpha waves from the back of my head that expresses the idling of my visual cortex.  OK, I've got three parts of my test: relaxed, thinking about moving my hand, and relaxing but with my eyes closed.  I'm going to do this all while seated in a chair.

Data from the Oz, my Visual Cortex:  The data that I collected is messy.  This is true of most data that is ever collected from a living, breathing human being.  To help make sense of the data, I'm going to talk about the easier results first...and those are the results from the electrode at location Oz, on the back of my head.  The plot below is a spectrogram of the signal recorded at Oz.  Time is on the horizontal axis and signal frequency on the vertical axis.  A pixel's color indicates the intensity of the signal at that pixel's time and frequency.  The reason that I'm showing this plot first is because of the obviousness of the red horizontal lines that appear when my eyes are closed.  These red lines indicate strong sinusoidal signals around 9-10 Hz that are sustained when my eyes are closed.  Because it only occurs when my eyes are closed, this is certainly my Posterior Dominant Rhythm (PDR).  These are not Mu waves.  Therefore, moving forward, any signal that occurs at these times and these frequencies in my other electrodes will simply by this PDR being detected from afar.  Again, these are not Mu waves and can be ignored.

Horizontal Lines are my Posterior Dominant Rhythm
Data from the C3, my Sensorimotor Cortex:  Now that we know what to ignore, the plot below shows a spectrogram of the data from electrode C3, with is located over my SM cortex.  Probably the easiest thing to see (though it is generally messy all around) are weaker versions of the horizontal lines that appear when I close my eyes.  As discussed above, these are my PDR and should be ignored.  They are not Mu waves. What is more interesting in this plot are the faint horizontal lines a little higher in frequency (~12 Hz) that only seem to occur when I'm relaxed.  Note that, when I think about moving my hand, they go away.  These must be Mu waves!  I found them!

Some Mu Waves Are Occasionally Seen Around 12 Hz When Relaxed.

Plan for Live Feedback:  OK, so with this one test, I've shown that I can detect my Mu waves when I'm relaxed and that I can make them go away when I think about moving a body part.  That's pretty exciting. The signals are really weak, though.  It's hard to see them and it'll be hard to get the computer to detect them reliably.  Apparently, I'm not good at getting sufficiently relaxed.  My next step, therefore, is to generate some sort of live feedback based on the strength of my Mu waves.  I to do this, I will modify my PC software to process the data and make a dinging sound or something in proportion to the strength of the signal at 12 Hz.  This live feedback will tell my I'm doing the right thing.   As a result, I should be able to train myself to do a better job making Mu waves.

Plan for BCI:  Once I get better at making Mu waves, it will be easier for the computer to differentiate between my relaxed state (strong Mu waves) and my "thinking about my hand" state (no Mu waves).  Once the computer can differentiate between these two states, I will have my magical brain-controlled interface.  Moreover, since the brain is sided (left side vs right side), the computer should be able to tell the difference between thinking about the left hand versus thinking about the right hand.  With two controls (left and right) we can start doing more complex moving robot arms back and forth...or like changing the television channel up and down...or like driving a rover forward and back.

Yeah, this is gonna be cool...

Follow-Up: I recorded more Mu waves using some homemade electrodes
Follow-Up: I can now view my Mu waves using a real-time spectrogram in my Processing GUI

Tuesday, October 15, 2013

Mu Rhythms for BCI

As discussed previously, there are several approaches that are currently being pursued for Brain Computer Interfaces (BCI).  One approach is to perform an EEG and to measure the "Mu Waves" or "Mu Rhythms" from the sensorimotor portion of one's brain.  The Mu Waves are associated with you moving your body -- either by actually moving your body, or by you *thinking* about moving your body.  Sounds like a great way to command a computer, eh?  Let's dig in a little more...

What Are Mu Waves?  The first paragraph on Mu waves in Wikipedia seems decent enough.  As it says, Mu Waves are a type of oscillating electrical rhythm within the brain that can be seen in an EEG.  Specifically, they occur in the sensorimotor cortex, which is the portion of the brain associated with coordinating muscle motion and the perception of ones muscle and joint motion.  Looking at the image below, the sesnorimotor cortex as the areas labeled "Primary Motor Cortex" and "Primary Somatosensory Cortex".   It is relatively narrow strip going from one ear, up over the top of the head, to the other ear.  This is where Mu Waves seem to occur.

Illustration of Sections of the Brain (via UIC)

When Do they Appear?  It is my understanding that Mu Waves appear naturally when your body is physically relaxed.  The appearance of the Mu Waves are an indication that the sensorimotor portion of your brain is "idling".  When you move a major body part, those portions of your brain stop "idling", they get down to real work, and the Mu Waves go away (are "suppressed") during the motor activity.  Amazingly, this portion of your brain exhibits the same Mu Wave suppression simply by imagining the motion of a body part.  Even better, the specific portion of your cortex where the Mu Waves are suppressed is linked to the body part that you're imagining moving.  Now that's cool!

From BCI2000.  The different regions of the sensorimotor cortex, *roughly* correspond to different body parts.  Feet and legs are near the top of the head.  Hands are near the middle.  Face and tongue are near the bottom of the cortex, which on your scalp is located just above your ears.
Difference From Similar Rhythms:  The Mu rhythm occurs in the frequency range commonly referred to as Alpha waves (8-12 Hz).  There are several sources of activity in the Alpha range.  The most common trigger for Alpha waves is simply to close your eyes.  In most people, closing your eyes idles the visual cortex (the whole back portion of your brain...the "occipital" region), which causes Alpha waves to appear throughout the rear portion of the brain.  This called the Posterior Dominant Rhythm.  The Mu Waves, by contrast, are associated with the sensorimotor portion of your brain, so they should only appear in the signals from the electrodes over that part of the brain.

How Can We Measure Our Mu Waves?  Theoretically, if you hook up an EEG sensor system to your scalp, and if you put some electrodes exactly over the sensorimotor portion of your brain, you should be able to see Mu Waves when you relax your body.  I have yet to be successful with this, though I will try again.  In preparation, I have been reading the tutorial from BCI2000 to get a better idea of where to put my electrodes and which electrodes to use for reference and bias.

Mu Waves in EEG Traces:  Below is a cool video that shows what Mu Waves look like in raw EEG traces. Being localized to just the sensorimotor cortex, they appear most strongly in the F4-C4 trace.  This link also shows Mu waves in a raw EEG this montage, they're seen most strongly in the F3-C3 trace.  In my own trials, I have not specifically plotted these two combinations of electrodes.  I will.

What Could We Do with Mu Waves?  In the EEG traces above, it appears that the presence or absence of Mu Waves is pretty easy to see...we can probably get a computer to detect their presence pretty easily.  Once the computer sees that they're present, we can imagine moving our body, which should make them go away.  The computer can see that they went away and can take some action (like moving a robotic limb). It would only be a simple on/off control, but it still would be cool!

Using Mu Wave for Fine Control of a BCI:  Mu waves are compelling for BCI, though, because we don't have to be satisfied with simple on-off control.  Take, for example, the fact that our bodies and brains are sided -- the left side of your brain controls the right side of your body, and vice versa.  So, if by imagining motion with the left side of your body, the Mu waves should only be suppressed on the *right* side of your brain.  The converse is true as well -- imagining motion on the right side of your body should suppress the Mu waves on the left side of your brain.  As a result, you should be able to use a Mu wave reading BCI to control a robot to move in two ways...say, left or right.  Now it's getting useful!

Using Different Body Parts:  But we're not done.  Mu waves are quite local.  If you imagine moving just your feet, the Mu waves are only suppressed in a small portion of your sensorimotor cortex that, for the feet, is near the top of your head.  Imagining moving your hands suppresses the Mu waves in a different part of the cortex (down closer to the ears).  So, with more electrodes -- electrodes that are carefully placed over the different regions of the sensorimotor cortex -- we should be able to distinguish between thoughts of moving your hands versus moving your feet.   The movie below shows an example of a group who built a BCI that achieves this.  Fantastic.

The Future:  In theory, more electrodes on the scalp could maybe yield an even finer distinction between body parts, though I've only seen BCIs that do hands versus feet.  Maybe now is the time for a break-through!

Follow Up:  Check out my Mu waves!

EEG Frequency Bands - Jorge Ochoa

Following my own introductory hack at describing different approaches to Brain Computer Interfaces (BCI), I found this link, "EEG Signal Classification for Brain Computer Interface Applications" by Jorge Ochoa, which is a great summary of EEG and of applying EEG for BCI.  It's from 2002, but its fundamentals are still nicely explained.

One of the parts that I really liked was Section 2.2, where he discusses the typical meaning of different frequency bands (or "wave groups", as he calls them).  Sure, this information is all over the internet, but I liked his description quite a bit.  Here it is, quoted:
BETA. Primarily between 13 and 30 Hz, and usually has a low voltage between 5-30 µV (Fig. 2-6). Beta is the brain wave usually associated with active thinking, active attention, focus on the outside world or solving concrete problems. It can reach frequencies near 50 hertz during intense mental activity. 
ALPHA. Primarily between 8 and 13 Hz, with 30-50 µV amplitude (Fig 2-4). Alpha waves have been thought to indicate both a relaxed awareness and also inattention. They are strongest over the occipital (back of the head) cortex and also over frontal cortex. Alpha is the most prominent wave in the whole realm of brain activity and possibly covers a greater range than has been previously thought of. It is frequent to see a peak in the beta range as high as 20 Hz, which has the characteristics of an alpha state rather than a beta, and the setting in which such a response appears also leads to the same conclusion. Alpha alone seems to indicate an empty mind rather than a relaxed one, a mindless state rather than a passive one, and can be reduced or eliminated by opening the eyes, by hearing unfamiliar sounds, or by anxiety or mental concentration.
THETA. Theta waves lie within the range of 4 to 7 Hz, with an amplitude usually greater than 20 µ V. Theta arises from emotional stress, especially frustration or disappointment. Theta has been also associated with access to unconscious material, creative inspiration and deep meditation. The large dominant peak of the theta waves is around 7 Hz.
DELTA. Delta waves lie within the range of 0.5 to 4 Hz, with variable amplitude. Delta waves are primarily associated with deep sleep, and in the waking state, were thought to indicate physical defects in the brain. It is very easy to confuse artifact signals caused by the large muscles of the neck and jaw with the genuine delta responses. This is because the muscles are near the surface of the skin and produce large signals whereas the signal which is of interest originates deep in the brain and is severely attenuated in passing through the skull. Nevertheless, with an instant analysis EEG, it is easy to see when the response is caused by excessive movement. 
GAMMA. Gamma waves lie within the range of 35Hz and up. It is thought that this band reflects the mechanism of consciousness - the binding together of distinct modular brain functions into coherent percepts capable of behaving in a re-entrant fashion (feeding back on themselves over time to create a sense of stream-of-consciousness). 
MU. It is an 8-12 Hz spontaneous EEG wave associated with motor activities and maximally recorded over motor cortex (Fig. 2-8). They diminish with movement or the intention to move. Mu wave is in the same frequency band as in the alpha wave (Fig. 2-7), but this last one is recorded over occipital cortex. 
Following his description of the different wave groups, he's got this nice tidbit, speaking to BCI:
Most attempts to control a computer with continuous EEG measurements work by monitoring alpha or [beta] waves, because people can learn to change the amplitude of these two waves by making the appropriate mental effort. A person might accomplish this result, for instance, by recalling some strongly stimulating image or by raising his or her level of attention. 
In the quote above, he fails to mention Mu Waves as a source of conscious control (simply think about moving a major body part and the Mu Waves go away), but he does include it later in his document.

Returning to the subject of alpha and beta waves, consciously affecting the balance of these two waves seems like an interesting (though difficult) target for driving a BCI.  I've attempted this, but with the exception of closing my eyes to induce big alpha waves, I'm currently unable to affect my alpha to beta balance.  I need more practice!

Sunday, October 13, 2013

Brain Control Interfaces - Different Approaches

An important question to be able to answer is "Why do you spend all this time hacking with EEG?".  For me, there are a number of answers.  My first answer, though, is that I'm really interested in brain-computer interfaces (BCIs).  I want to be able to control things with my brain.  Why?  Because, when successful, it's like magic.  It's like THE FORCE from Star Wars.  It's the kind of thing that, when demonstrated in real life, gets a heart-felt "Whoa!" from unsuspecting on-lookers.   It's cool.

There are a number of different methods of implementing a brain computer interface.  The first major division in approaches is whether the BCI is invasive or non-invasive.  In this context, "invasive" means that a surgeon cuts open you head, saws open your skull, and implants electrodes directly in your brain.  If you're a quadriplegic, you might be willing to have this done in order to get your best chance at a BCI that works well.

For the rest of us, though, we might be more interested in a non-invasive BCIs that sense your brain waves by electrodes on the scalp.

How do BCIs listen to the signals from your brain (via your scalp) and do something useful?  To my understanding there are three approaches: Frequency Analysis, Mu Wave Detection, and Event-Related Potentials.

Frequency Analysis

The simplest approach is simply to look at the frequency content of the EEG signals recorded from the scalp.  Since nearly everyone produces alpha waves when they close their eyes, a straight-forward example of a frequency-based BCI would be to program the computer to move a motor in proportion to the alpha waves measured in the EEG signals.  I've done it.  It's fun!  More complex control schemes can be developed by looking at more frequency bands (theta, alpha, beta, etc) and by looking at different or multiple locations on the scalp.  With this added range of variables, you can do more complex things.  The video below shows an example of this kind of setup.

The hard part is that most people cannot easily control the frequency content of the signals in their head.  Usually, you're asked to control wishy-washy aspects of your mental/emotional state such as "alertness", "relaxation", "focus", etc.  How do you do that?  Well, it requires much practice and, to date, has yielded unreliable results for most people.  But, it is easy to implement on the computer, so it's a good starting place for people hacking their own BCI system.

Mu Waves (Mu Rhythms)

A special case of the "Frequency Analysis" methods is a method based on looking for "Mu Waves".  Mu waves are special because they occur in the motor cortex (or, more precisely, in the combined sensorimotor cortex).  If you can get your scalp electrodes in the right place, you will see Mu waves whenever your body is physically relaxed.  When you contract the muscles in a relevant body part (or, even if you just visualize yourself contracting the body part), the Mu waves in that part of your brain get suppressed.  So, the EEG setup is a little harder, but one's ability to actually control these brain waves is much better.

To get more information on how to do a Mu wave BCI, the BCI2000 folks have some great information

For another example, check out the video below.  They built a BCI for playing World of Warcraft.  If you skip to 0:46, you see how they put together the system and how, through the subject moving his feet and hands, they trained the computer to understand his brain waves.  This use of physical motions is almost certainly training the system to look for the subject's Mu waves.  Furthermore, note that the only electrodes that are wired-up on his EEG cap are the ones over his motor cortex.   It's gotta be a mu-wave system.

Follow Up: Here's more discussion of using Mu waves for my BCI.

Event-Related Potential (ERP)

A third way to do a BCI is to measure event-related potentials (ERPs).  ERPs are EEG measurements in response to a particular sensory stimulus, which then causes a particular response in the brain.  Often visual stimuli are used via a computer screen.  This is useful for BCI because, if the user is consciously paying attention to the visual stimuli, his brain gives one type of response (that is detectable via EEG), while if he ignores the stimuli, it gives a different response.  This means that the human subject can consciously interact with the computer simply through selectively focusing (or not) on the visual stimuli.

The video below presents a typical setup.  Here, the computer presents a grid of letters on the screen.  The human subject wants to spell a word, so he focuses his attention on a letter on the screen...the letter "S", for example.  The computer then randomly highlights the letters on the computer screen.  Whenever the letter "S" is highlighted, the human recognizes that his letter was highlighted and his cognitive response causes a quick and temporary change in his EEG signals (the "P300" feature appears).   Unfortunately, the P300 is a very subtle change, so the whole process has to be repeated many times so that the recordings can be averaged together to make the P300 detectable.  If the computer has to flash through the whole keyboard, you can imagine how slow this is.  The video below illustrates the slowness...he gets about one letter every 40 seconds.

Still, even though it is slow, ERP interfaces allow for a very rich interaction with the computer that can be more complex than the simple "left", "right", "forward" commands seen in the World of Warcraft video above.  Plus, the system used in the video is not the be-all and end-all in ERP interfaces.  This is a very new field and many advances are possible.

If you want to learn more about (or try!) a P300 ERP system, the BCI2000 folks also have some tutorials:

EEG Hacking Begins

So you're interested in doing some EEG Hacking?  Me, too!  I've been doing EEG hacking for a few months now, and I'd like to share what I've learned.  I'm not quite sure where to begin, so I'll start with some of my favorite links on how I learned about EEG.  None of these links is entirely satisfying, so I'll eventually replace these links with my own content...but until I'm able to write it, we're stuck with what we find on the Internet.

A Sporty Blue EEG Cap 

Of course, everyone starts with the wikipedia page.  It's an OK page.  It's too long to be a good introduction.  I do find its discussion of the history of EEG to be interesting (it's 90 years old!) and I like the discussion of the different EEG frequency bands.  Sadly, it's not too helpful on how to actually do an EEG.

EEG Setup:

There's no good, simple description of how to do a very basic, hacker-style EEG.  I'm going to have to fix that.  Until that time, we're stuck with others' descriptions.

Here's a decent link that talks about a number of aspects of EEG, but I like the pictures of the electrodes, of the electrode placement, and of typical EEG artifacts...especially of eye motion.

The BCI2000 folks also have a lot of good information.  Here's a basic on electrodes, placement, and typical signal artifacts.

Typical EEG Signals:

The links above talked about about some typical signals, especially of "artifacts", which are the undesirable signals that get picked up by the EEG system.  These artifacts are usually not associated with brain activity, which is why they are considered bad.

Actual brain signals in EEG are often discussed and analyzed based on their frequency content.  These are the so-called "Alpha waves", "Beta waves", "Theta waves", and such.  The Wikipedia page above discusses these different frequency band.  So does the link below:

Another way of discussing EEG signals is by the shape of the waves, not just their frequency content.  This is the "morphology" of the waves.  These are the "Sleep Spindles" and "K Complexes".  Some of the morphologic features in EEG signals:

Finally, here's a more formal (and detailed!) discussion of some normal types of signals that you might see in an EEG.  It spends a lot of time discussing alpha waves.