Brain Got Beats -- Not Yet

I like controlling things with my mind.  That's why I do this brain-computer interface (BCI) thing.  The tough part of BCIs, though, is finding brain signals that are simple enough for the computer to detect, yet are also something that I can consciously control.  So far, I can do eyes-closed Alpha waves, concentration-controlled Beta/Gamma, and steady-state visual evoked potential (SSVEP).  I need more options.  Today, I'm going to try to do auditory steady state response (ASSR).  Or, more colloquially, does my brain got beats?

Can I use beating tones to entrain brainwaves?

Auditory Steady-State Response (ASSR)

The idea with ASSR is that we are looking for EEG signals from my brain that are driven by sounds presented to my ears.  When doing an ASSR, you use an audio tone whose amplitude is varied ("modulated") at a fixed rate such as 40 Hz.  Then, when you play that sound in your ears, you look in the EEG signals for a strong 40 Hz component.  Easy, eh?

Note that this is very similar to the steady-state visual evoked potential (SSVEP) that I used previously, where I'd make my computer screen blink at 8 Hz and 8 Hz signals would appear in my EEG.

Attention-Based ASSR?

If I want to use ASSR for a brain-computer interface (ie, for controlling robots!), there needs to be some way to consciously control my response to the sound.  For the SSVEP, where stimulation was my blinking computer screen, my response was much stronger if I consciously paid attention to the blinking screen.  This attention-based response was the key to being able to exploit it for a BCI.

Does ASSR have a similar attention-based component?  Until yesterday morning, I didn't know.  But then I came across this paper:  Do-Won Kim et al.  "Classification of selective attention to auditory stimuli: Toward vision-free brain–computer interfacing".  Journal of Neuroscience Methods 197 (2011) 180–185.  PDF here.

Kim's ASSR Setup

In the paper by Kim, they used two loudspeakers to present tones to the test subject.  The setup is shown below.  The subjects were sitting down in a comfy chair listening to the tones while wearing a small montage of EEG electrodes (Cz, Oz, T7, T8, ref at left mastoid, ground at right mastoid).

Test Setup as used by Kim (2011) for Evoking Auditory Steady-State Response (ASSR)

For the audio tones, they used a 2500 Hz tone from one speaker and a 1000 Hz tone from another speaker.  The key feature of ASSR, though, is the modulation of these tones.  For one of the tones, they varied the amplitude of the tone (ie, they alternately made it quiet and loud) at a rate of 37 Hz, while the other tone they modulated at a rate of 43 Hz.  These frequencies are the "beat rates" for the audio.  It is the 37 Hz or 43 Hz beat rate that they are looking for in the EEG (hence, "brain got beats?").

Below is what they saw in the EEG signals (Cz) for one of their subjects when the subject gave their attention to the 37 Hz modulated signal (red) or the 43 Hz modulated signal (blue).  There is clearly a difference.  This makes me happy.  This is what I want to recreate with my own testing.

Spectral Results for One Subject from Kim (2011) In Response to Steady-Pitch
Tones that were Amplitude Modulated at 37 Hz or 43 Hz.

My Test Setup

I want to recreate their results.  I'm going to create some audio files with the amplitude modulated signals, I'm going to play them into my ears via headphones, and I'm going to record my EEG signals (OpenBCI!) to look for my ASSR.

EEG Setup:  Reading more details from the paper, they said that they got the strongest response from the electrode at Cz, so I decided to start there.  I put one electrode at the top of my head (Cz) with the reference on my left ear lobe and the OpenBCI "bias" on my right ear lobe.  I used the gold electrodes and the Ten20 EEG paste that came with the OpenBCI kit.  Without really trying, I happened to get an electrode impedance of 20-30 kOhm at both Cz and at the reference, which are probably good enough.

My EEG Setup, Cz Only.  Also, unlike Kim, I used ear buds (headphones)
stead of loudspeakers to present my tones.

OpenBCI EEG System:  For this test, I happened to use my 16-channel OpenBCI system.  I'm only using one channel of EEG data, though, so I could have used the 8-channel systems (or even other systems, like OpenEEG) just as well.  I wired up my OpenBCI unit as shown below.  Starting from the left, the white wire is the "bias" (aka, driven ground) going to my right ear lobe, the brown wire is the electrode at the top of my head, and the black wire is the reference electrode on my left ear lobe.  Note that they are all plugged into the lower row of pins (the "N" inputs) on the lower board.  The system is being powered by four AA batteries and is sending its data wirelessly back to the PC.  I'm using the OpenBCI GUI in Processing.

Here's How I Plugged into the OpenBCI Board.

Audio Files:  I created my audio files in Audacity.  I created two sets of files, based on the frequencies used in the Kim paper: one set of files using a 1000 Hz tone and the other set using a 2500 Hz tone.  The Kim paper said that the strongest ASSR generally occurs for a beat frequency of 40 Hz.  I wanted to see my response at different beat frequencies, so for each tone I created three versions: one beating at 38 Hz, one at 40 Hz, and one at 42 Hz.  I made each version 20 seconds long.  I used a square wave (ie, on/off) amplitude modulation, though next time I might try sine wave modulation instead.

I Created My Amplitude-Modulated (AM) Test Tones in Audacity.  First, "generate" the
tone.  Then, to do the AM, go under "Effect" and select "Tremolo". 

Data and Analysis Files:  My audio files, my data files, and my analysis files are all on my GitHub here.  Note that I did my analysis using an IPython Notebook (see it here).  My specific Python installation is described here.

My ASSR Response

My goal is to see if I exhibit the ASSR response with this test setup.  To do the test, I wired myself up as discussed above, I queued up all six audio files (the three at 1000 Hz followed by the three at 2500 Hz), put in my ear buds, and started recording.

Eyes Closed:  The spectrogram below shows my Cz EEG signal when I did this test with my eyes closed.  That strong red stripe at 10 Hz is my Alpha response simply due to having my eyes closed.  What I do not see here are horizontal stripes of energy at 38, 40, or 42 Hz.  In other words, I do not see any brain waves entraining with the audio stimulation.  This is disappointing.

Spectrogram of EEG Signal from Cz with AM Auditory Stimulation Near 40 Hz.

My eyes were closed, hence the strong response at 10 Hz.

There is no signature of the 38-42 Hz AM Audio Stimulation.

Eyes Open:  I also performed this test with my eyes open.  A spectrogram of my EEG signal at Cz is shown below.  I started and ended the test with my eyes closed for 10 seconds, which you can see as 10 Hz Alpha waves at the start and end.  What I really want to see, though, is something corresponding to the audio stimulation at 38 Hz, 40 Hz, or 42 Hz.  Again, I see nothing.

Spectrogram of EEG Signal from Cz with AM Auditory Stimulation Near 40 Hz.

My eyes were open, except at the beginning at end.

There is no signature of the 38-42 Hz AM Audio Stimulation.

Average Spectrum:  To most closely mimic the plot from the Kim paper (ie, the graph that I copied earlier), I plotted the average spectrum.  In the Kim plot, there were clear peaks at his two beat frequencies (37 and 43 Hz).  In my equivalent plot below, there are no peaks at the three beat frequencies that I studied (38, 40, and 42 Hz).

Mean Spectrum During the Test Period.  There is no evidence of my brain waves entraining

with the 38, 40, and 42 Hz AM auditory signals.  Bummer.

Conclusion:  So, it is clear that i did not see any ASSR in my EEG recordings.  This is very disappointing to me.

Comparison to Kim

Why did Kim see ASSR and I did not?  I'm not sure.  Maybe my test setup or my audio files were sufficient different to prevent the response.  Or, maybe I'm reading too much into his results...

In looking back at his plot with the spectrum from one of his subjects (copied earlier in this post), I see that the y-axis is a linear axis, whereas I always do dB.  What might his values look like when converted to dB?

As an example, I see that his first peak is 0.40 uV^2, relative to a baseline of about 0.30 uV^2.  Converted to dB (re: 1 uV^2), this would be -4.0 dB and -5.2 dB.  Comparing to my own spectrum plot above, where my baseline is about -10 dB, any peak at -4.0 dB should be easily seen.  Therefore, if my own response were as strong has Kim's subject's response, I would think that I would see the response in my plots.  I don't see the peak, so I guess that I didn't have the response as strongly as Kim's subject.

Perhaps the "gotcha" here is that the difference in Kim's data between the peak (-4.0 dB) and the baseline (-5.2 dB) is only 1.2 dB.  That is a really small difference.  For reliable detection, I generally like to see 6-10 dB of difference.  It might be too much to hope to reliably see only a 1.2 dB difference.

Next Steps

I'm not going to give up yet.  I'm going to try again.  I'm going to try using the additional EEG electrodes as used by Kim and I'm going to try to use sine-wave modulation instead of square-wave modulation.  I want to see this response!

Controlling a Hex Bug with my Brain Waves

Ever since my effort with OpenBCI began, I've been looking to control something with my brain.  Sure, a while back, I was successful in lighting an LED with my brain waves, but that's pretty simple.  I wanted something more.  And now I can do it.  I can control a robot with my mind!  Yes!

Approach:  My robot has just a few actions that it can do...turn left, turn right, walk forward, and fire.  To make this brain-controlled, I need a way to invoke these commands using signals from my brain.  Ideally, I'd just think the word "Fire!" and the robot would respond.  Unfortunately, those kinds of brain waves are too hard to detect.  Instead, I need to use brain waves that are easy to detect.  For me, "easy" brain waves include the Alpha waves (10 Hz oscillations) that occur when I close my eyes, as well as the brain waves that occur when I watch my blinking movies (a.k.a. visual entrainment).  So, my approach is to use OpenBCI to record my brainwaves, to write software to detect these specific types of brain waves, and to issue commands to the robot based on which brain waves are detected.

Here are all the pieces that you see in the video

Hardware Setup:  The core hardware for this hack is similar to my usual OpenBCI setup: EEG electrodes, an OpenBCI board, an Arduino Uno, and my computer. Added to this setup is the Hex Bug itself and its remote control, which I hacked so that the remote can be controlled by an Arduino.  So, as shown below, my brain wave signals go from my head all the way to the PC.  The PC processes the EEG data looking for the Alpha waves or the visually-entrained waves.  If any are detected, it decides what commands to give the robot.  The commands are conveyed back to the Arduino, which then drives the remote control, which the Hex Bug receives over its usual IR link.

Here is the schematic of how the pieces work together.

EEG Setup:  I'm going to be measuring my Alpha waves and I'm going to be measuring the brain waves induced through visual entrainment.  Based on my previous experience, I know that both are best recorded using an electrode on the back of the head (at the "O1" position, if you're into your 10-20 electrode placement standard).  I do not need electrodes all over my head.  That's the only sensing electrode that I'm using.  That's it.  Of course, EEG also requires a reference electrode, which I put on my left earlobe.  And, finally, EEG often has a third electrode ("bias" or "driven ground"), which I placed on my right earlobe.

Looking at the Frequency of my Brain Waves:  As mentioned above, my approach is to control my robot by detecting Alpha waves and by detecting visually-entrained brain waves.  These are easily detectable because they occur at specific frequencies.  Alpha occur around 10 Hz and the visually-entrained brain waves occur at the blink rate(s) of whatever movies I use (my best results were from 5 Hz and 7.5 Hz movies). So, to control my robot, I will be looking for EEG signals at these frequencies: 5 Hz, 7.5 Hz, and 10 Hz.  I'm going to "look" for these frequencies by writing some EEG processing software that'll look at the frequency content of my EEG signal to see if these frequencies are present.

EEG Processing:  The flow chart above shows the steps that I use to process the EEG signal (my software is here).  Once the PC gets EEG data from the OpenBCI board, the first step is to compute the spectrum of the signal, which tells me the content of the EEG signal as a function of frequency.  I then search through the relevant part of the spectrum (4-15 Hz) to find the peak value.  I note both its frequency value and its amplitude.  In parallel, I also compute the average EEG amplitude across the 4-15Hz frequency band.  This average value is my baseline for deciding whether my peak is tall (strong) or short (weak).  By dividing the amplitude of my peak by this baseline value,  I get the signal-to-noise ratio (SNR) of the peak.  The SNR is my measure of the strength of the peak.  The output of the EEG processing, therefore, are two values: the frequency of the peak and the SNR of the peak.

Deciding My Robot's Action:  Once my EEG processing finds the frequency and SNR of the peak in my EEG spectrum, I now have to decide how to act on that information.  After some trial and error, I settled on the algorithm shown in the flow chart above.  It's got three steps:

• SNR Check:  First, I decide whether the current peak in the spectrum is legitimate, or if it is likely to be just noise.  I don't want to issue a command if it is just noise because then my robot will be taking all sorts of actions that I didn't intend.  That is not what I want.  So, to decide if the peak is likely to be legitimate, I look at the SNR of the peak.  If it has a big SNR, I'll accept it as a legitimate peak.  If it is too small, I'll take no further action.  Right now, my threshold for this decision is at 6 dB.  Setting a higher threshold results in fewer false commands (which would be good), but it also makes the system less sensitive to legitimate commands (which is bad).  This 6 dB threshold resulted in an OK (but not great) balance.

• Frequency Check:  If the peak seems legitimate, I decide how to command the robot based on the frequency of the peak.  If the peak is between 4.5-6.5 Hz, I must be looking at the right-side of my 2-speed blinking movie (ie, the portion that blinks at 5 Hz), so the computer prepares the "Turn Right" command.  Alternatively, if the EEG peak is 6.5-8.5 Hz, I must be looking at the left-side of my 2-speed blinking movie (ie, the portion that blinks at 7.5 Hz), so it prepares the "Turn Left" command.  Finally, if the EEG peak is 8.5-12 Hz, it must be my eyes-closed Alpha waves, so the computer prepares the "Move Forward" command.

• New Command Check:  Before issuing the command, I check to see whether this command is the same as the last command that was extracted from my brain waves.  If the latest command is different, I hijack the command and, instead, issue the "Fire!" command.  If the latest command is the same, I go ahead and issue the left / right / forward command like normal.  The reason for this hijack is that I have no other type of easily-detected brain wave that I can use for commanding the robot to fire.  This approach of issuing "Fire!" on every change in command seemed like a decent way of getting a 4th command out of 3 types of brain waves.

Putting It All Together:  As you can see in the movie, I eventually able to get all of these pieces working together to allow me to command the Hex Bug using just my brain waves.  Of course, it didn't work the first time.  Even once I got all the hardware working, I still needed to tune a bunch of the software parameters (FFT parameters and the detection threshold) until I got something that worked somewhat reliably.  To help with this tuning process, I used the spectrum display that is in my Processing GUI.  Some screen shots are below.

Example EEG spectrum when I stared at the right side of my two-speed blinking
movie.  It induced 5 Hz brain waves.  I programmed 5 Hz to mean "Turn Right".
The SNR here is between 6 and 7 dB.

Here's an example EEG spectrum when I stared at the left side of my two-speed
blinking movie.  It induced 7.5 Hz brain waves.  When the GUI detected 7.5 Hz,
it issued a "Turn Left" command to the Hex Bug.  The SNR is only 6-7 dB.

Finally, here's an example EEG spectrum with my eyes closed so that I was
exhibiting Alpha waves, which are near 10 Hz.  When it detected 10 Hz, I
programmed it to issue a"Forward" command.  The SNR is > 8 dB.

Weak Peaks:  In the screenshots above, the red line shows the current EEG spectrum.  The heavy black circle shows the spectral peak that my software algorithms have detected.  The black dashed line is the "background noise" from which the SNR is computed.  To be declared a legitimate detection, the peak must be 6 dB higher than the black dashed line (unfortunately, I don't show this on the plot...sorry!).  As can be seen, the 5 Hz and 7.5 Hz examples are not very strong (the SNR is only 6-7 dB).  Other peaks within the plots are very close to being the same size, which would cause false commands to be sent to the robot.  In my movie at the top of this post, there were several false commands.

Balancing Sensitivity with False Commands:  To reduce the number of false commands, I could raise my detection threshold above 6 dB. Unfortunately, as see in the first two spectrum plots above, my 5 Hz and 7.5 Hz peaks are usually pretty weak (<  7 dB).  Therefore, any attempt to raise my detection threshold above 6 dB would cause me to no longer detect my legitimate brain waves.  I know because this is exactly the tuning process that I tried.  Bummer!  So, if I want more reliable performance, I'll need to develop a fancier signal processing beyond this simple FFT-threshold approach.  Future challenges!

Wrapping Up:  Even with the false commands seen in my movie, I was still able to command the robot to move around the table.  I could get it to go (roughly) where I wanted it to go.  And, I did it all with just my brain waves.  I think that this is pretty exciting!  Yay!  What are the next steps?  Well, maybe now that I have this under my belt, I can move on to control flying fish, or maybe a quadcopter!  Do you have any other cool ideas for things I can control with my brain?

Coolness:  This hack got picked up by IEEE Spectrum as part of an article on OpenBCI.  Cool!  Check it out here.

More Coolness:  This hack also got picked up by Wired.  Fun!

Follow-Up: I got to share this hack with Joel and Conor of OpenBCI.  You can see their luck with controlling the robot here.

Follow-Up: Follow-Up:  We used a similar approach to get a 5-person team to brain-control a swimming shark balloon.  It's cool.  Check it out here.

Controlling Entrainment Through Attention

In a previous post, I showed that I could induce (entrain) brain waves at different frequencies simply by staring at blinking movies playing on my computer.  Having demonstrated this basic feasibility, my goal now is to exploit this phenomenon to make a brain-computer interface (BCI) to control future hacks.  My idea is to play two blinking movies simultaneously -- one at a slow speed and one at a fast speed.  I'm hoping that my brainwaves will only entrain with the blinking from the one movie that I choose to focus on.  Does my brain work this way?  Will my brain successfully reject the blinking from the movie that I'm ignoring?  Let's find out!

Simultaneous Blinking at Two Speeds:  Previously, I made some blinking movies where the whole screen would blink black or white at a given speed.  To make this idea work for a BCI, I want my screen to blink at two different rates at the same time.  So, as you can see in the video above, I made the left side of my screen blink at one rate while the right side of my screen blinks at a different rate.  I'm hoping that, if I focus my attention on the left side of my screen, by brainwaves will only become entrained at the left-side blink frequency, whereas if I were to focus on the right side of the screen, my brainwaves would follow the right-side blink frequency.

Swapping Sides:  To help with this test, I wanted to remove any effect of turning my head to change my gaze between the two sides of my screen.  So, in creating my dual-rate blinking movie, I had the movie automatically swap sides every 20 seconds.  As a result, it starts with fast blinking on the left and slow blinking on the right.  After 20 seconds, it swaps so that slow is on the left and fast is on the right.  It does this swap a few times.  The Matlab code that I used to make these movies is here.

I created a movie where the left and right sides blink independently -- left is fast and right is slow.
For this test, the two blink rates swapped sides every 20 seconds.

Choosing my Blink Rates:  Based on my previous results, it looks like my brain (coupled with my computer's limited ability to blink steadily) is most easily entrained in the 6-10 Hz frequency range. So, for this dual-rate movie, I chose "slow" to toggle between black and white at 10 Hz (ie, a 5 Hz white-white rate) and "fast" to toggle at 15 Hz (ie, a 7.5 Hz white-white rate).  In truth, I made a bunch of movies at different rates, but the the 10/15Hz movie worked the best, so I'll only show its results.

EEG Setup:  With my movies prepared, I gathered up my EEG stuff.  Like usual, I used my OpenBCI board and a few cup electrodes with Ten20 paste. I put one electrode on the left side of my forehead (Fp1), on one the left side of the back of my head (O1) and one on the right side of the back of my head (O2).  Using the impedance measuring feature, my impedances were 11 kOhm, 67 kOhm, and 28 kOhm (I seem to have an on-going problem getting a low impedance at O1).  My reference electrode was on my left ear lobe and my bias electrode was on my right ear lobe.  My OpenBCI board was connected to the PC via USB and I was logging data using my OpenBCI GUI in Processing.  For this test, I also used my photocell to confirm that my computer's blinking was sufficiently steady.

I used my OpenBCI V1 board to record my EEG into the computer.  I also attached a
photocell to confirm that the screen was blinking at the right rate.

Results:  After setting everything up, I started recording my EEG data and then I started playing the dual-rate blinking movie.  It was night time, so my room was pretty dark.  I focused my attention at the center of the left-hand movie. As described above, the left movie toggled fast-slow-fast-slow every 20 seconds, while the right movie played the opposite -- slow-fast-slow-fast.  Spectrograms of the EEG signals from my head are shown in the figure below.  As you can see, there was no entrainment seen in the signals from my forehead (as expected) but there was entrainment in the back of my head (also as expected).  The best entrainment was seen on the left side of my head.

Spectrograms of my EEG signals recorded while watching my dual-rate blinking movie.  The left-back
of my head exhibited the strongest entrainment to the blinking of my movie.  

Only Seeing the Left Blink Rate:  Because the left-back of my head (O1) gave the best entrainment, let's just focus on its results. The figure below shows just the results for the left-back of my head.  Note how,once the movie starts playing, my EEG signals seem to toggle between a fast blink rate (~7.5 Hz) and a slow link rate (~5 Hz).  This exactly follows the white-white blink rate of the left movie. So, my brainwaves successfully entrained to the movie that I was watching. Most importantly, there seems to be no signature in my EEG data from the blinking of the right movie.  This is success!

Spectrogram of my EEG data from the left-back of my head (O1) while watching my dual-rate
blinking movie.  I was focused just on the left movie.  Because of this focus, my brainwaves
appear to have entrained only with the left movie's blink rate.

Purposely Shifting My Attention:  OK,so I've demonstrated that my mind can successfully ignore one of the movies.  That's really good.  But, maybe I'm just biased to looking left. To really make this work for a BCI, I need to be able to shift my attention to either movie and have my brainwaves follow.  So, for my 2nd test, I started the same movie playing back.  But, this time, when the movies swapped sides every 20 seconds, I switched my attention to follow the movie that blinked faster. This means that I started by watching the left movie, then I watched the right, then left, then right.  My EEG response is shown below. Note that I showed strong entrainment and,most importantly, that my brainwaves only show the fast blink rate (7.5 Hz). So, by shifting my attention to follow the faster movie, I successfully rejected the effect of the slower blinking movie. Success again!

Spectrogram of my EEG data from the left-back of my head (O1) while watching my dual-rate blinking
movie.  While watching the movie, I switched my attention between left and right to follow the movie
 that blinked faster. Because of this focus, my brainwaves remained entrained only at the faster rate.

All the Elements are In Place:  It looks like I now have the elements in place for a 3-state BCI.  If I don't look at the movie at all, I get State 1: "Nothing". If I watch the blinking of the left movie, I get State 2: "Left".  If I watch the blinking of the right movie, I get State 3: "Right".  It may be possible to further divide my screen to get more blinking regions to add more BCI states. Maybe that's a good experiment for the future. Right now, though, I think that I'm going to turn my attention to a little robot that I got (thanks for the pointer Conor!) to see if I can control it with visual entrainment.  This is gonna be fun!

Follow-Up:  Interested in getting the EEG data from this post?  Try downloading it from my github!

Follow-Up:  I successfully used visual entrainment to control a six-legged robot!

Inducing Brain Waves with Visual Entrainment

A while back, I had a friend come over and I measured his EEG in response to staring at a blinking light.  We saw (as we hoped) that his brainwaves oscillated in sync with the blinking of the light.  I thought that this visual entrainment (aka "steady-stead visual evoked potential") was pretty cool.  Since then, I've learned that it can be used as the basis for a brain-computer interface (BCI).  Because I'm still searching for a good BCI paradigm, I decided to return to my exploration of visual entrainment.  Today, I'm going to show how I successfully used visual stimuli to induce brainwaves at different frequencies.  As a result, I can now see an good avenue for an EEG-based BCI.  Yes!  Let's go!

Inducing SSVEP Using a Toggling Checkerboard Pattern on my Computer Screen

Goal:  My goal today is to use visual stimuli to induce brainwaves across a range of frequencies.  Because I want to use this for a BCI, I'm trying to determine what kind of visual stimuli I should use and what EEG frequencies I can induce.  What does it take to make this work reliably?

Visual Setup:  In my previous post, my visual stimulation was simply a blinking head-lamp.  It was effective (and really bright!), but I had no control over its blink frequency.  As a result, I also had no control over the frequency of the brainwaves that it induced.  So, for today's test, I needed to get fancier.  I ditched the head-lamp and, instead, created a series of blinking movies that I could playback on my computer.  I controlled the "blinking rate" by saving my movies at different frame rates.  What exactly did the movie look like?  Well, at first, because of a paper that I read in the VEP literature, my movie used the checkerboard pattern shown in the picture above.  The movie toggled back-and-forth between this image and the inverse image (swap blacks and whites).  While this worked OK, I later switched to a simpler movie (code here) where the screen was simply all-white or all-black.  That seemed to work better.

EEG Setup:  Once I made my movies, I set myself up with my EEG system (OpenBCI ).  I my usual gold cup electrodes with Ten20 EEG paste.  I put one electrode on the back of my head (near O1) and I put another electrode on my forehead.  My reference electrode was on my left ear lobe and my bias electrode was on my right ear lobe.  Using the impedance-measuring feature of OpenBCI, the electrode on my forehead had an impedance of 24 kOhm and the one on the back of my head was about 65 kOhm.  I couldn't seem to get the back electrode to a lower value.

Channel 1 was on my Forehead, Channel 2 was on the back of my head.
My left ear lobe was my reference.  My right ear lobe was the bias.

I used my OpenBCI V1 board with Ten20 Paste.
I think that those owl napkins are fun!

Test Method:  Once I got everything setup, I launched the OpenBCI GUI in Processing and started an EEG recording.  To playback my homemade blinking movies, I opened up Windows Media Player and set it to full-screen mode.  I had ten movies, with each movie blinking at a different rate.  I had WMP play all 10 movies continuously in sequence.  Each movie was 20 seconds long, so the whole test took about 200 seconds.  It was nighttime when I did this test and my room was dimly lit.  I tried to stare at the screen and I tried to only blink my eyes at the transition between the different blinking rates.

Results, Checkerboard:  As usual, my preferred way to view the data is to make spectrograms.  In the figure below, the top plot is the data from my forehead and the bottom plot is the data from the back of my head.  From my forehead, there is nothing interesting except my eye blinks.  From the back of my head, we see several interesting features, which I've marked with blue and white arrows.  Note that these interesting features change every 20 seconds, which is the same as my 20-second movies.  It seems clear to me that these features are my brainwaves responding to changing of the blink rate in my movies.  Excellent!

Spectrogram of EEG Signal Recorded While Watching the Blinking Checkerboard Pattern.
The top plot is the signal from my forehead.  The bottom plot is from the back of my head.
"W-B" is the rate at which the movie switched from either white-to-black or black-to-white.
"W-W" is the rate if you measure just from white-to-white.

Entrained with the Blink Rate?  Looking at the three blue arrows, it appears that I have entrained brainwaves at 2 Hz, 4 Hz, and 6 Hz.  At these times, any given square in my movie was blinking at 1 Hz, 2 Hz, and then 3 Hz, if you count from white period to white period ("W-W").  Because I have entrained brainwaves as 2x the white-to-white frequency, it suggests that it is NOT white-to-white that matters, but that it is the transition between white/black or black/white that matters.  At least, that is what is implied for these three (out of 10) cases for the checkerboard stimuli.

Complications:  While that would be a fine conclusion, why does this rule not continue through the other 7 cases in this checkerboard test?  Why does it only work for the three cases with the blue arrows?  The cases with the white arrows do show some sort of EEG response, but not at any frequency that makes sense given the speed of my movies.  What is going on?  I've got two possible explanations: (1) either my movies are not playing back reliably during these other cases, or (2) the checkerboard pattern is too complicated to be a good starting point for learning about my brainwaves.

Modifying the Test:  Of these two possible explanations, it's easier for me to simplify the checkerboard than it is for me to fix the reliability of my movie playback.  So, I changed my movies so that the whole screen is either all all black or all white.  Hopefully, this simpler visual stimuli will make my EEG response easier to understand.

Results, Whole-Screen Blinking:  After recording my EEG while staring at the new movies, the spectrograms of my data are shown below.  Again, all of the interesting action is in the back of my head.  The bottom plot shows that I got good entrainment of my brainwaves for nearly *all* of the new movies.  I'm very pleased.  I'm also very curious about the jump between the cases marked with blue arrows versus the cases marked with red arrows.  What is happening here?

Spectrogram of EEG Signal Recorded While Watching the Whole Screen Toggle White or Black.
The top plot is the signal from my forehead.  The bottom plot is from the back of my head.

What Blink Rate Matters?  Looking at the first half of this plot, the blue arrows indicate cases that have results similar to the checkerboard data shown earlier.  Here, my brain seems to respond to every transition from white to black and from black to white (aka, the "W-B-W" speed).  But, for the second half of this plot, when the blinking is faster, it looks like my brainwaves follow the slower rate resulting from just the white-to-white frequency ("W-W").  Based on this weird result, I'm thinking that my brain doesn't actually care so much about whether the stimuli is W-B-W or W-W...it is simply sensitive to rhythmic visual stimuli in a certain frequency range.  I'm thinking that, whatever rhythmic stimuli falls in this frequency range, my brain will become entrained with it.

Quantifying Entrainment vs Frequency:  If it's simply the frequency that matters, it would be good to see which frequencies yield the strongest entrainment.  Sure, the spectrograms above suggest which frequencies are best, but I took the next step and actually measured the EEG response at each of the stimulation frequencies.  The plot below shows the EEG amplitude that I measured for each of the visual blinking frequencies.  Note that there are two lines, one for if you count based on the white-to-white frequency (blue line) or whether you're counting based on all the white/black and black/white transitions (red line).  This graph suggests that I seem to yield decent responses in the 6-10 Hz frequency range.  So, if I'm looking to use visual entrainment for a BCI, I should focus on the 6-10 Hz band.

Amplitude of EEG Signals Induced by Visual Entrainment.
My best responses seem to be in the 6-10 Hz band.
That could be a good target frequency range for use in a BCI.

Computer Could be Limiting my Performance:  As mentioned earlier, all of these results could be confounded by the possibility that my computer cannot reliably and steadily refresh my screen.  Perhaps it can reliably handle the frequencies at 10 Hz and below, but is not steady above 10 Hz.  Perhaps that's why my apparent response above 10 Hz falls off.  Sure, my computer claims that the screen has a 60 Hz refresh rate, but that doesn't mean that Windows or that Windows Media Player can keep up.  So, any next steps should include some method of assessing whether the computer is actually displaying my movies smoothly at the rate that I expect.

Entrainment for BCI:  My overall goal is to make a cool brain-computer interface (BCI).  Because I am showing that I can successfully measure visual entrainment, I would like to further explore how visual entrainment could be exploited for a BCI.  One idea is that I could simultaneously show two movies side-by-side, each blinking at its own rate.  Perhaps, if I'm lucky, my brainwaves will only respond to the one movie that I'm actually watching.  If that's the case, then I would have conscious control over my brainwaves (and, therefore, the BCI) simply by selecting which of the two movies that I watch.  That could be very cool.


Follow-Up:  I setup a photocell and my Arduino to measure the actual blink rate of the movies on my computer.  In my results, I found that I can't generate steady blinking faster than 10 Hz.  This is probably a strong reason why my EEG recordings exhibited no entrainment above 10 Hz...how can I entrain to signals that aren't there?!?

Follow-Up: I extended this work by having one movie blink at two different rates.  I found that I could control my entrained brainwaves by choosing which of the blink rates I focused on.  Pretty cool!  If you're interested, you can see the results in this post.

Follow-Up:  Interested in getting the EEG data from this post?  Try downloading it from my github!

Blinky Lights - Visual Entrainment

In talking up my EEG hacking with some friends, I found a buddy who was really interested.  In particular, he was interested these smartphone apps that claim to affect your sleep state.  My friend wanted to know if these apps actually did anything to the brain.  That's a pretty cool question, and very similar to the question that I had about meditators (see their results here and here).  To figure out if his sleep-modifying apps were doing anything to his brain wave, he volunteered to be my guinea pig.  What a guy!

A Willing Guinea Pig Meets the Red EEG Cap

This post shows some of the data that I collected...though not yet when subject to the sleep app.  I decided to start simple and record how his particular brain responds to sensory entrainment.  Entrainment is how these sleep apps work, so if we understand how he responds to entrainment in general, we'll be well-positioned to understand his response to the sleep apps.  So

Background:  It is my understanding that the sleep apps work by playing specially-constructed sounds into your ears via headphones.  They're trying to induce certain brain rhythms (Delta, Theta, Alpha, Beta, etc) by playing audio into your ears at the same frequency as the desired brain rhythm.  Put most simply, they play a 10 Hz tone into your ears and hope to get brain waves at 10 Hz (ie, Alpha rhythm).  This is called entrainment and is a long-known phenomenon in EEG.  Personally, I'm not too familiar with this type of auditory entrainment, but I do know that visual entrainment, so I'm going to start there.

Setup:  I'm using the same setup as I used for my recordings of meditators.  I used an EEG electrode cap (this is the first time using the red-colored cap, though...exciting!) with the EEG electrode gel that came with the electrode cap kit (ECI Electro-Gel).  We used the same electrode montage (see figures below), the same reference electrode (near FPz/AFz) and the same ground/bias electrode (right mastoid).  For electronics, I used an OpenBCI V1 board with an Arduino streaming data to my PC running our full GUI that was written in Processing.

Baseline, Eyes-Closed Alpha:  Since I had never recorded my friend's EEG before, I decided to start with the most basic recording -- I had him close his eyes so that we could see his Alpha-wave posterior dominant rhythm (PDR).  The spectrograms in the montage below show his response...it is very normal.  Note the energy in the Alpha band (~10 Hz) that shows up most strongly in the back of his head and not at all in the front of his head.  As I said, very normal.

Spectrograms of EEG Signals Recorded With the Eyes Closed.
Notice the Strong (and typical) Energy in the Alpha Frequencies.
Click to Zoom.

In the figure below, I summarize this PDR Alpha response across the eight electrodes.  It shows that his Alpha peaks at about 10.25 Hz.  His Alpha are a bit stronger on the left side of his head (channel 7, green) than on the right (channel 8, blue).  That's also what happens with me.  I've always wondered if this asymmetric Alpha response is related to handedness.  I'm right handed.  I don't know handedness my friend is.  It would be interesting to record a lefty and see what happens!

Average EEG Amplitude Recorded With Eyes Closed and Relaxing.
Notice the Strong Peak in the Alpha Band (~10 Hz).

Finally, the last thing that I'd like to examine with his eyes-closed Alpha data is the spectral coherence of the EEG signals from neighboring electrodes.  This is a quantity that I first analyzed in this post on my second meditator.  It shows how strongly related (how correlated) are the signals between two electrodes.  I use this type of analysis to estimate whether the different physical areas of the brain are working together or independently.

Below are the cross-channel coherence plots for my friend sitting with his eyes closed.  Like with my meditating friend, he shows very little coherence in the front of they head (those areas must be acting independently relative to each other) and more coherence towards the back of the head.  Looking specifically at the Alpha band, it looks like the Alpha seen between electrodes 5 and 7 (ie, back left) are strongly related to each other.  Same with the Alpha seen between electrodes 6 and 8 (ie, back right).  In the very back of the head (7 and 8), the 10 Hz energy is not very coherent between the two hemispheres, even though they are physically closer together that 5/7 or 6/8.  This is so interesting to me.  It is also the same result that we saw with my meditator friend when he was not meditating.

Spectral Coherence Between Neighboring Electrodes.   Strong coherence (red) implies coordinated
EEG activity whereas low coherence (blue) implies independent EEG activity.
Click to Zoom.

Visual Entrainment:  Now we start to do something new.  To see how entrainment works, I started with the easiest sensory entrainment that I know about -- visual entrainment.  The idea here is that you blink a light at a certain speed and you look for brain rhythms at that same frequency.  Truth-be-told, I wasn't actually planning on doing this test, so I didn't have a good light prepared.  But I do have a nice new, really-bright hiking headlamp that has a blink setting.  I don't know exactly what speed it is, but I counted blinks and it's less than 5 Hz.  Sadly, it's blinking rate isn't as steady as I might like.  But, when you're EEG hacking, sometimes you gotta be quick and dirty.

[WARNING!  Be careful doing this kind of test at home!  Blinking lights like this can induce seizures!  Proceed at your own risk!]

To do my visual entrainment test, I darkened the room and had my friend sit in a chair, like before.  I held the blinking light about a foot and a half from his face (see picture below).  We did part of a recording where his eyes were open and looking at the blinking light (so bright!), then he closed his eyes while the blinking continued, then he opened his eyes again.  It turns out that only the eyes-closed portion gave decent results, so that data is what I'm going to focus on.

Attempting Visual Entrainment Using a Blinking LED Hiking Headlamp

If we start with the spectrograms (below, you might want to click on the figure to see it bigger), you'll see that we got a nice line of energy down at the low frequencies (~4 Hz).  The line only appears when both the light was blinking and when his eyes were closed.  Note that it shows up in all EEG channels, but it appears to be a bit stronger on the right side of his head.  These lines in the spectrograms mean that his brain waves were indeed being induced to oscillate at the same rate as the blinking light.  It's a well known effect, but I still think that's kinda cool.

Spectrograms of EEG Signals Recording With Eyes Closed with a Bright Blinking Light.
Click to Zoom.

These spectrograms are summarized in the single spectrum plot below.  It shows a peak at 3.9 Hz, which is most likely the blinking rate of my head lamp.  The amplitude of the entrained waves is quite strong --  note that it is similar in amplitude as the eyes-closed baseline Alpha waves that we recorded earlier.  This graph also confirms that the entrained waves are a bit stronger on the right side (channel 8, blue) versus the left (channel 7, green).  If you remember from above, his baseline eyes-closed alpha waves were the opposite -- they were stronger on the left.  Finally, perhaps most surprising of all is that there are no Alpha waves at all.  Remember, his eyes are closed just like before.  Yet, there are no Alpha waves.  The presence of the blinking light apparently suppresses his natural rhythms (the Alpha) and entrains a rhythm at its own blink rate (the 3.9 Hz signal).

Average EEG Amplitude Recorded With Eyes Closed and A Bright Light Blinking
Notice the Strong Peak at 3.9 Hz (the Blink Rate) and the Absence of Alpha Waves.

Finally, let's look at the spectral coherence across neighboring EEG channels.  The plot below shows strong coherence at these low frequencies (3.9 Hz) across all pairs of channels except for the 1/3 pair (front left) and the 2./4 pair (front right).  Why are these not coherent yet the others are?  I don't know.  The 1/3 pair and the 2/4 pair do have the largest physical spacing of any of the pairs, but I still find it surprising.  I mean, even the cross-hemisphere pairs of electrodes (the 1/2 pair in front and the 7/8 pair in back) show good coherence, but not these 1/3 and 2/4 pairs.  I'm not sure what it means (the front's response is independent of the whole rest of the brain?) but I'll be sure to keep an eye on the 1/3 and 2/4 coherence in the future to see if there is a trend.

Spectral Coherence Between Neighboring Electrodes During the Eyes-Closed Blinking Light Test.
Click to Zoom.

Conclusion:  OK, what have we learned?  We learned that my buddy looks pretty cool in that red EEG cap.  And we learned that his brain is a mysterious place that emanates lots of cool signals.  His willingness to be my guinea pig gave me lots of data from which I have made lots of nerdy graphs.    Here's what I learned from the graphs:

• His eyes-closed alpha waves are similar to the others that I've measured
• Similar frequency (~10 Hz)
• Similar amplitude (~4 uV RMS)
• Similar spatial distribution across the head (strongest in the back)
• Similar coherence pattern (back-left and back-right, but not cross-hemisphere)
• We successfully induced visual entrainment with the blinking light (3.9 Hz)
• Similar amplitude as the eyes-closed Alpha waves (~4 uV RMS)
• Entrained brain waves appear all over the head
• Entrained brain waves are coherent everywhere except front-left and front-right
• The blinking light suppressed the PDR Alpha response

But what does it all mean?  Does it mean that the sleep-modification app on his smartphone will do anything?  No, this data and analysis does not speak to that question at all.  The goal here was just to help me (us?) learn about sensory EEG entrainment in general, and about my friend's individual EEG response in particular.  Now, that we've done the easy thing and gotten a bit smarter, we can maybe move on toward the harder thing (auditory entrainment) to try to answer the question as to whether the sleep-modification brainwave app is doing anything.  Now I have a better idea of what to look for.

So, thanks for reading.  This is so fun!  (for me at least...)

Next Steps:  In this follow-on post, I use a computer screen instead of a blinky light.  I show that I can entrain brain waves at a variety of speeds.  This is the first step in making an entrainment-based BCI!

Follow-Up:  I used visual entrainment to control a six-legged walker...with my brain waves!