Brain Got Beats -- Not Yet

I like controlling things with my mind.  That's why I do this brain-computer interface (BCI) thing.  The tough part of BCIs, though, is finding brain signals that are simple enough for the computer to detect, yet are also something that I can consciously control.  So far, I can do eyes-closed Alpha waves, concentration-controlled Beta/Gamma, and steady-state visual evoked potential (SSVEP).  I need more options.  Today, I'm going to try to do auditory steady state response (ASSR).  Or, more colloquially, does my brain got beats?

Can I use beating tones to entrain brainwaves?

Auditory Steady-State Response (ASSR)

The idea with ASSR is that we are looking for EEG signals from my brain that are driven by sounds presented to my ears.  When doing an ASSR, you use an audio tone whose amplitude is varied ("modulated") at a fixed rate such as 40 Hz.  Then, when you play that sound in your ears, you look in the EEG signals for a strong 40 Hz component.  Easy, eh?

Note that this is very similar to the steady-state visual evoked potential (SSVEP) that I used previously, where I'd make my computer screen blink at 8 Hz and 8 Hz signals would appear in my EEG.

Attention-Based ASSR?

If I want to use ASSR for a brain-computer interface (ie, for controlling robots!), there needs to be some way to consciously control my response to the sound.  For the SSVEP, where stimulation was my blinking computer screen, my response was much stronger if I consciously paid attention to the blinking screen.  This attention-based response was the key to being able to exploit it for a BCI.

Does ASSR have a similar attention-based component?  Until yesterday morning, I didn't know.  But then I came across this paper:  Do-Won Kim et al.  "Classification of selective attention to auditory stimuli: Toward vision-free brain–computer interfacing".  Journal of Neuroscience Methods 197 (2011) 180–185.  PDF here.

Kim's ASSR Setup

In the paper by Kim, they used two loudspeakers to present tones to the test subject.  The setup is shown below.  The subjects were sitting down in a comfy chair listening to the tones while wearing a small montage of EEG electrodes (Cz, Oz, T7, T8, ref at left mastoid, ground at right mastoid).

Test Setup as used by Kim (2011) for Evoking Auditory Steady-State Response (ASSR)

For the audio tones, they used a 2500 Hz tone from one speaker and a 1000 Hz tone from another speaker.  The key feature of ASSR, though, is the modulation of these tones.  For one of the tones, they varied the amplitude of the tone (ie, they alternately made it quiet and loud) at a rate of 37 Hz, while the other tone they modulated at a rate of 43 Hz.  These frequencies are the "beat rates" for the audio.  It is the 37 Hz or 43 Hz beat rate that they are looking for in the EEG (hence, "brain got beats?").

Below is what they saw in the EEG signals (Cz) for one of their subjects when the subject gave their attention to the 37 Hz modulated signal (red) or the 43 Hz modulated signal (blue).  There is clearly a difference.  This makes me happy.  This is what I want to recreate with my own testing.

Spectral Results for One Subject from Kim (2011) In Response to Steady-Pitch
Tones that were Amplitude Modulated at 37 Hz or 43 Hz.

My Test Setup

I want to recreate their results.  I'm going to create some audio files with the amplitude modulated signals, I'm going to play them into my ears via headphones, and I'm going to record my EEG signals (OpenBCI!) to look for my ASSR.

EEG Setup:  Reading more details from the paper, they said that they got the strongest response from the electrode at Cz, so I decided to start there.  I put one electrode at the top of my head (Cz) with the reference on my left ear lobe and the OpenBCI "bias" on my right ear lobe.  I used the gold electrodes and the Ten20 EEG paste that came with the OpenBCI kit.  Without really trying, I happened to get an electrode impedance of 20-30 kOhm at both Cz and at the reference, which are probably good enough.

My EEG Setup, Cz Only.  Also, unlike Kim, I used ear buds (headphones)
stead of loudspeakers to present my tones.

OpenBCI EEG System:  For this test, I happened to use my 16-channel OpenBCI system.  I'm only using one channel of EEG data, though, so I could have used the 8-channel systems (or even other systems, like OpenEEG) just as well.  I wired up my OpenBCI unit as shown below.  Starting from the left, the white wire is the "bias" (aka, driven ground) going to my right ear lobe, the brown wire is the electrode at the top of my head, and the black wire is the reference electrode on my left ear lobe.  Note that they are all plugged into the lower row of pins (the "N" inputs) on the lower board.  The system is being powered by four AA batteries and is sending its data wirelessly back to the PC.  I'm using the OpenBCI GUI in Processing.

Here's How I Plugged into the OpenBCI Board.

Audio Files:  I created my audio files in Audacity.  I created two sets of files, based on the frequencies used in the Kim paper: one set of files using a 1000 Hz tone and the other set using a 2500 Hz tone.  The Kim paper said that the strongest ASSR generally occurs for a beat frequency of 40 Hz.  I wanted to see my response at different beat frequencies, so for each tone I created three versions: one beating at 38 Hz, one at 40 Hz, and one at 42 Hz.  I made each version 20 seconds long.  I used a square wave (ie, on/off) amplitude modulation, though next time I might try sine wave modulation instead.

I Created My Amplitude-Modulated (AM) Test Tones in Audacity.  First, "generate" the
tone.  Then, to do the AM, go under "Effect" and select "Tremolo". 

Data and Analysis Files:  My audio files, my data files, and my analysis files are all on my GitHub here.  Note that I did my analysis using an IPython Notebook (see it here).  My specific Python installation is described here.

My ASSR Response

My goal is to see if I exhibit the ASSR response with this test setup.  To do the test, I wired myself up as discussed above, I queued up all six audio files (the three at 1000 Hz followed by the three at 2500 Hz), put in my ear buds, and started recording.

Eyes Closed:  The spectrogram below shows my Cz EEG signal when I did this test with my eyes closed.  That strong red stripe at 10 Hz is my Alpha response simply due to having my eyes closed.  What I do not see here are horizontal stripes of energy at 38, 40, or 42 Hz.  In other words, I do not see any brain waves entraining with the audio stimulation.  This is disappointing.

Spectrogram of EEG Signal from Cz with AM Auditory Stimulation Near 40 Hz.

My eyes were closed, hence the strong response at 10 Hz.

There is no signature of the 38-42 Hz AM Audio Stimulation.

Eyes Open:  I also performed this test with my eyes open.  A spectrogram of my EEG signal at Cz is shown below.  I started and ended the test with my eyes closed for 10 seconds, which you can see as 10 Hz Alpha waves at the start and end.  What I really want to see, though, is something corresponding to the audio stimulation at 38 Hz, 40 Hz, or 42 Hz.  Again, I see nothing.

Spectrogram of EEG Signal from Cz with AM Auditory Stimulation Near 40 Hz.

My eyes were open, except at the beginning at end.

There is no signature of the 38-42 Hz AM Audio Stimulation.

Average Spectrum:  To most closely mimic the plot from the Kim paper (ie, the graph that I copied earlier), I plotted the average spectrum.  In the Kim plot, there were clear peaks at his two beat frequencies (37 and 43 Hz).  In my equivalent plot below, there are no peaks at the three beat frequencies that I studied (38, 40, and 42 Hz).

Mean Spectrum During the Test Period.  There is no evidence of my brain waves entraining

with the 38, 40, and 42 Hz AM auditory signals.  Bummer.

Conclusion:  So, it is clear that i did not see any ASSR in my EEG recordings.  This is very disappointing to me.

Comparison to Kim

Why did Kim see ASSR and I did not?  I'm not sure.  Maybe my test setup or my audio files were sufficient different to prevent the response.  Or, maybe I'm reading too much into his results...

In looking back at his plot with the spectrum from one of his subjects (copied earlier in this post), I see that the y-axis is a linear axis, whereas I always do dB.  What might his values look like when converted to dB?

As an example, I see that his first peak is 0.40 uV^2, relative to a baseline of about 0.30 uV^2.  Converted to dB (re: 1 uV^2), this would be -4.0 dB and -5.2 dB.  Comparing to my own spectrum plot above, where my baseline is about -10 dB, any peak at -4.0 dB should be easily seen.  Therefore, if my own response were as strong has Kim's subject's response, I would think that I would see the response in my plots.  I don't see the peak, so I guess that I didn't have the response as strongly as Kim's subject.

Perhaps the "gotcha" here is that the difference in Kim's data between the peak (-4.0 dB) and the baseline (-5.2 dB) is only 1.2 dB.  That is a really small difference.  For reliable detection, I generally like to see 6-10 dB of difference.  It might be too much to hope to reliably see only a 1.2 dB difference.

Next Steps

I'm not going to give up yet.  I'm going to try again.  I'm going to try using the additional EEG electrodes as used by Kim and I'm going to try to use sine-wave modulation instead of square-wave modulation.  I want to see this response!

Detecting Alpha Waves - Guard Bands

In my previous post, I discussed a simple algorithm for detecting Alpha rhythms: (1) use an FFT spectrum to measure the EEG amplitude in the Alpha band and (2) compare this value to a fixed detection threshold to decide if Alpha are present.  As shown in the figure below, this approach yields good detection sensitivity (it correctly flags 66% of the eyes-closed data blocks) and a reasonably low number false alarms (it incorrectly flags 15 data blocks).  While this is good, I think that I can do better.  Let's talk about how...

Example EEG data (top) showing Alpha rhythms when my eyes are closed.  At each time slice, I measure the peak of the spectrum in the Alpha band (7.5-11.5 Hz), which yields the blue trace on the bottom.  By looking for any value above 3.5 uVrms, we are able to detect the presence of Alpha waves (as indicated by the red circles)

Alpha Band Detection is Not Specific Enough:  In the simple Alpha band detection algorithm discussed above, we are sensitive to any signal with lots of energy in the Alpha (7.5-11.5 Hz) band.  The problem is that there are signals besides Alpha rhythms that have energy in the Alpha band.  For example, the bottom plot below shows the spectrum (black line) for a segment of eyes-closed Alpha waves.  The plot  also shows the spectrum (red line) for a segment of "other" activity that is no an Alpha rhythm (it is probably motion artifact from the EEG lead wires).  As can be seen, both spectra show substantial energy in the Alpha band, and so they would both be flagged as "Alpha!" using my simple threshold detection approach.  For the segment of "other" activity, this would be a false alarm.  I don't want that.  I want to improve my algorithm to reject this kind of false alarm.

Two spectra: (1) a segment of eyes-closed Alpha rhythm that I do want to detect and (2) a segment of "other" EEG activity that I do not want to detect.  Both, however, show a high amplitude in the Alpha band.  So, my original detection rule that is simply based on the Alpha amplitude would not reject the "other" activity.

Introduce "Guard" Bands:  One way of distinguishing between the two example spectra above is to introduce "guard" bands on either side of the Alpha band.  The idea is that we measure the signal amplitude both in the Alpha band and in the guard bands.  Based on the plots above, we know that true eyes-closed Alpha activity will not show much energy in the guard bands whereas the confusing "other" activity can be rejected because it does show energy in the guard bands.

If we measure the mean EEG amplitude in the guard bands, as well as in the Alpha band, we can distinguish between the two signals.  True Alpha rhythms will not have much energy in the guard bands whereas most of our confusing "other" activity will show substantial energy in the guard bands.

Evaluating the Guard Amplitude:  To quantify the amplitude in the guard bands, I simply take the average of all the spectrum values that fall within our two guard bands (3-6.5 Hz and 13-18 Hz).  When I apply do this for our EEG data, I get the green trace shown in the middle plot below.  As you can see, it stays low during all of the legitimate eyes-closed Alpha activity and it jumps high only during the confusing other activity.  This looks promising!

Combined Detection Rules:  Based on this graphs above of the guard amplitude (green line) and of the Alpha amplitude (blue line), it looks like a good combination of rules would be to look for points where the Alpha amplitude is greater than 3.5 uVrms and, simultaneously, where the guard amplitude is less than 2.5 uVrms.  When I apply these detection rules, I get the red circles shown in the figure above.  Looks pretty good!  You'll note that the addition of the guard band has successfully rejected the false alarms that we had been getting at t=58, t=77, and t=123.  This is exactly what I was hoping for.

Quantifying the Improvement:  Compared to yesterday's results (tabulated below), this new detection algorithm obtains nearly the same sensitivity (65% vs 66%) with a greatly reduced number of false alarms (2 vs 15).  This is definitely an improvement in my Alpha detecting algorithm.
      
Guard       N_TRUE      N_FALSE
None        101 (66%)   15
2.5 uVrms   100 (65%)   2


Moving Forward:  For this example EEG recording, I am satisfied with the performance of this algorithm.  It would give me quite reliable performance while still being nicely sensitive.  This EEG recording was pretty "clean", however -- its Alpha was pretty strong and there was not too much confusing "other" activity.  I have other EEG recordings that are more difficult.  Next time, we'll look at those harder recordings, you'll see that even the combined Alpha+Guard algorithm is insufficient, and I'll discuss yet another extension (hopefully an improvement!) on this detection approach.

Follow-Up:  I further optimize this algorithm by using ROC curves to attack, head-on, the trade-off between sensitivity and false alarms.  Check it out here.

Detecting Alpha Waves - Threshold Detection

In my previous post, I showed some EEG data that I recorded from the brand-new OpenBCI V3 board.  The data that I showed included some Alpha waves that my brain generated (like most people's brains) simply by closing my eyes.  I've copied a spectrogram of that EEG data below.  You can see the Alpha waves as the horizontal stripe of energy near 10 Hz.  While it is pretty easy to see (to "detect") this signal by eye, it might be fun to get the computer to automatically detect these Alpha waves, so that you can use Alpha waves to make a brain-controlled light, or a brain-controlled robot, or to do some other cool hacking shenanigan.  How do we get the computer to detect the Alpha waves?  In this post, and in some follow-up posts, I'm going to discuss a few ways...

Spectrogram of the EEG Signals from the Back of my Head ("O1").  Note the horizontal stripe of energy near 10 Hz when my eyes are closed.  These are the Alpha waves being generated in my occipital lobe.

Simple Approach First:  There is a huge body of literature out there on the various signal processing techniques to address the "detection" problem.  Most approaches (or, at least, the language used to describe the approaches) get very technical very fast, even in introductory material, so I'm going to take a simple approach first, and only add additional complication as needed to solve particular problems.

Average EEG Spectrum when My Eyes are Closed.  The Alpha rhythm clearly shows up around 10 Hz.

Frequency View:  As a human being looking at the spectrogram at the top of this post, I easily see the horizontal stripe of energy that represents my Alpha waves.  Since the frequency is nearly constant, the Alpha waves should show up in a simple spectrum view of signal.  The spectrum view plotted above shows the EEG spectrum averaged entire period when my eyes are closed.  Clearly, there is a strong peak in the Alpha Band (7.5-12 Hz).  This is the tool that we will use to measure the Alpha waves.

Alpha Through Time:  Since the Alpha waves are clearly identifiable in the spectrum, and since the OpenBCI GUI already computes the spectrum as the EEG data arrives from the OpenBCI board, let's use the spectrum as our tool for focusing on just the Alpha waves.  To quantify the amplitude of the Alpha waves, I find the maximum value of the spectrum within the 7.5-12 Hz band.  Since the OpenBCI GUI computes a new spectrum every 200 msec, I get a new estimate of the Alpha amplitude five times a second.  The plot below shows the estimate of Alpha amplitude that results from this process.

Looking at the Alpha Band (7.5-11.5 Hz) through Time.  Notice that the amplitude is highly variable.  When my eyes are closed, the amplitude is generally much higher.

Choose a Detection Threshold:  As you can see in the plot above, the EEG amplitude in the Alpha band increases greatly when my eyes are closed (and, a bit surprisingly, we also see that my Alpha amplitude is not very steady...it varies a lot when my eyes are closed).  To have the computer decided when Alpha waves are present, the simplest approach is to pick a threshold value such that, when the signal amplitude is above the threshold, we declare that Alpha waves are present.  Looking at the plot, I picked a threshold value of 3.5 uVrms.

The red circles shows those data points where the Alpha amplitude is greater than my detection threshold of 3.5 uVrms.   It correctly captures most of the data when my eyes are closed, yet it also incorrectly captures a few moments of strong non-Alpha activity.

Detection Results:  The plot above shows the effect of setting the detection threshold at 3.5 uVrms.  The red circles shows those data points where amplitude in the Alpha band is above the threshold and we would declare that Alpha is present.  Based on the good coverage during the "eyes closed" portion of the data, I'd say that this detection threshold yields good sensitivity.

False Alarms.  To improve our sensitivity further, one could imagine lowering the detection threshold so that we capture more of the points within the "eyes closed" region.  Doing this, though, would also cause more points outside of the "eyes closed" region to be falsely detected as Alpha waves.  Even with our 3.5 uVrms threshold, there are several moments (t = 58, t = 77, t = 123) when strong broadband EEG activity happens to be strong enough to cross our detection threshold.  Since these detections are not due specifically to Alpha activity, we call these false alarms.

Balancing Sensitivity with False Alarms:  Selecting a good detection threshold requires one to balance the desire for high sensitivity with the requirement for a low false alarm rate.  After trying several different threshold values (see table below), 3.5 uVrms seems like it provides a decent balance for this EEG recording.  Other recordings might require a different threshold value.

Threshold    N_TRUE     N_FALSE
2.5 uVrms    126 (82%)  43
3.0 uVrms    112 (73%)  20      
3.5 uVrms    101 (66%)  15
4.0 uVrms    75 (49%)   12
4.5 uVrms    59 (39%)   11

Moving Forward:  With this simple method of quantifying the Alpha amplitude (ie, take the maximum value from the spectrum in the 7.5-12 Hz band) and with this simple method of deciding whether Alpha is present (ie, using a pre-defined detection threshold), we can easily have the computer detect our eyes-closed Alpha waves.  Sure, we might have a few false alarms but this is just our first try!  In the next post, I'll try adding a few techniques to be more selective to reduce our false alarms, without significantly degrading our sensitivity.

Follow-Up:  See how I reduce the false alarms by introducing Guard Bands!

First Alpha with OpenBCI V3

OK, I'm back to work now.  After my previous post, where I got my first data ever with the new OpenBCI board (aka, "V3"), I took a little hiatus while Joel worked through some issues with the Bluetooth link.  Everything appears to be working well now, so I'm back on the case.  Yesterday, I connected everything up and recorded my first real EEG data with the V3.  Exciting!

My Little OpenBCI V3 Board (8-Bit Version) with Homemade Electrode Adapter

EEG Setup:  To get started, I set an easy goal for myself -- just record some eyes-closed alpha waves.  So, I got out my trusty gold electrodes, my trusty Ten20 electrode paste, and put on a few electrodes.  I attached one electrode to the back-left of my head ("O1"), the reference electrode to my left earlobe, and the bias electrode to my right earlob.  So far, this is just like normal.

Software Setup:  For software, I used Arduino software for the OpenBCI Bluetooth dongle (aka, the RFduino "Host"), for the remote Bluetooth module on the OpenBCI V3 board itself (aka, the RFduino "Device"), and for the Atmel microcontroller that is the core of the OpenBCI V3 board (and which is programmed like an Arduino Uno).  The software is surely going to change with time, but right now I'm working with this code here.  On the PC side, I used a version of our Processing GUI that we modified to accept the new binary data format being generated by the V3 board.

Data and Analysis:  I did a couple of recordings of my eyes-closed alpha waves.  My data and analysis files are here.  Some example plots of the data that I recorded are shown below.  This is my first time trying to analyze the data using Python instead of Matlab.  Because I'm so new with Python, I was a lot slower in doing the analysis, but now that I've completed this one little task, I'm feeling pretty OK about the switch.  Maybe, just maybe, it is possible to learn new tricks!

EEG Data Recorded from the Back-Left of my Head ("O1") After Closing my Eyes
around t = 88 seconds.  By closing my eyes, I get alpha waves appear near 10 Hz.

Time-Domain Plot:  The top plot is a simple plot of the recorded EEG signal as a function of time.  Actually, it's not a totally "simple" plot because I have done some processing of the data.  I highpass filtered it to remove the DC component and I notch filtered it at 60 Hz and 120 Hz to get rid of power line interference.  In my opinion, though, time-domain plots are not very useful when zoomed out to a wide range of time (like we're doing here).  So, there's not much to say.

Spectrogram:  The middle plot is a spectrogram of the same data.  I love spectrograms.  Here, time is again on the horizontal axis, but now frequency is on the vertical axis.  The intensity of the color of each pixel shows how much signal energy is at the pixel's time and frequency.  Here, by the red horizontal line that appears ~10 Hz, you can clearly see the alpha waves.  Cool!

Frequency-Domain Plot:  Unfortunately, it is difficult to be quantitative about the amplitude of signals that are seen in the spectrogram.  So, once I located my alpha waves (t = 90 sec to t = 118 seconds), I plotted the mean spectrum for the data just in that time period.  The bottom plot shows this spectrum -- it shows the spectrum of my brain waves during t = 90 sec to t = 118 sec  You can see the prominent bump around 10 Hz.  These are my alpha waves.  As can be seen. the amplitude is approximately 4.1 uVrms and the peak is focused at 9.38 Hz.  That's my brain!  Specifically, that's my visual cortex when it's bored because my eyes are closed!

So, that's the quick fun that I had using the new OpenBCI V3 hardware and the fun that I had using Python for the first time to make decent graphs.  Learning new things makes me feel pretty empowered.  To celebrate, I'm going to go eat some breakfast now.  Mmm...Wheat Chex...I really know how to party.  ;)

Follow-Up:  Here's some additional discussion on how to detect these Alpha waves

EEG as WAV Files, Go Spectrograms!

OK, let's say that I just finished some cool new EEG experiment where I recorded my EEG response to watching cat videos while listening to the Pink Panther at half speed.  My next step would be to take a quick look at the data to get the overall big picture.  My favorite way of getting that overall view is to make a spectrogram (see example below).  My love for these oh-so-colorful plots runs deep.  The question is, how does one make spectrograms?  Well, in my opinion, if you don't have Matlab (and are afraid of Python), the next best way to make spectrograms is to use one of the multitude of audio editing software packages out there.  Many audio edit programs provide a spectrogram view.  This post is about getting EEG data into an audio program so that you can see your data.

A Spectrogram of EEG data that was Made in Matlab.  This shows data from my previous post,
where I was watching a movie with two different blink rates.  You can see how my brainwaves
entrained with the changing blink rate in the movie.

Converting to a WAV File:  The first step in using an audio program for EEG analysis is to convert one's EEG data into an audio file.  Since I usually work in Windows, I tend to convert all of my EEG data into WAV files.  I choose WAV because it is uncompressed.  I never choose MP3 because it is very unclear what its "perceptual coding" would do to my precious brainwave data.  So, a WAV file is what I would recommend.  But how do you get EEG data into a WAV format?  If your EEG data is in text format (such as is logged by the OpenBCI GUI), you could use my Processing sketch "ConvertToWAV".  This sketch will read in an OpenBCI log file and write each EEG channel out as its own WAV file. You can get the sketch on my GitHub.

Audacity:  Once the data is in WAV format, you can open it in any audio program.  A popular (and free!) audio editing program is Audacity.  While it is not my favorite audio editing program, it is perfectly sufficient for working with EEG data.  After opening your EEG data, the trick is to figure out how to switch the display from waveform to spectrogram.  The screen shot below shows how to do it.

Changing to Spectrogram View in Audacity

Once Audacity is in spectrogram mode, you need to zoom in on the vertical axis in order to see the interesting EEG features, which are usually focused in the lower frequencies.  In Audacity, you zoom simply with a click-and-drag on the vertical axis.  Then, after manipulating the spectrogram settings under the "Preferences" menu, you can get a spectrogram like the one shown below. While the color scheme hurts the eyes a bit, this spectrogram is good enough to see the same kind of EEG entrainment as seen in my original Matlab plot. Furthermore, the tools in Audacity let you further analyze the EEG data through zooming, filtering, amplifying, and (if you change the file's sample rate to increase the playback speed) you can use Audacity to listen to your own brain waves!  Audacity is definitely a useful tool for working with EEG data.

In Audacity, a Spectrogram of my EEG Data

The spectrogram settings that I used are shown in the screen shot below.

My Display Settings for Making EEG Spectrograms in Audacity.  I changed
the Window Size, the Gain, and the Range.

Cool Edit Pro:  I first started getting into spectrograms in the late 90's because this is when I started working with audio and music on the computer. What got me hooked on spectrograms was a piece of shareware called Cool Edit.  It was a stupid name for an otherwise outstanding program.  It was so useful that I spent the extra dollars and bought its upgrad -- Cool Edit Pro. Cool Edit Pro has a *great* spectrogram display, as shown below.  Unlike Audacity, which requires lots of manipulation of the spectrogram settings to get a useful view, the Cool Edit Pro display always seems just right.  Unfortunately, Cool Edit Pro isn't available anymore -- it was bought by Adobe in the early 2000s and became Adobe Audition.  Audition is also fine for making spectrograms (I have only used up to Audition 3.0), but it is expensive.

An EEG Spectrogram in Cool Edit Pro V1.2a.  It's an old school program that totally rocks.

In Cool Edit Pro, the only display parameter that you need to change is the "Resolution" (ie, FFT size).  You do that under the "Settings" menu.

My settings for viewing EEG spectrograms in Cool Edit Pro.
I changed the Resolution value.

Raven Lite:  A third option for making spectrograms is a bit more obscure.  A bunch of years ago, I came across a program called "Raven Lite", which is produced by the Ornithology Lab (ie, bird science) at Cornell University.  The "Lite" version is free.  You can download it and immediately use it for spectrograms, though it is crippled in other ways until you email them for a free (non-commercial) key. What I really like about Raven is that, as shown in the screen shot below, its spectrogram controls are right on the main window for easy manipulation.  Also, I like its color map options way better than what is available in Audacity.  Finally, Raven is one of the few programs that let you see both the spectrogram view and the waveform view at the same time (not shown).  It is really nice to have that capability.

Raven Lite 1.0 from the Cornell Laboratory of Ornithology. It's a pretty good viewer.  The settings for
the display or right here in the main window.

Other Options:  Because I have Matlab and Cool Edit Pro (and Audacity and Raven) I haven't spent a lot of time looking at other options.  Does Garage Band offer a spectrogram view?  Is there a plug-in for iTunes or Windows Media Player that gives spectrograms?  I'm curious to hear what you folks use.  Drop a comment and let me know!

Controlling Entrainment Through Attention

In a previous post, I showed that I could induce (entrain) brain waves at different frequencies simply by staring at blinking movies playing on my computer.  Having demonstrated this basic feasibility, my goal now is to exploit this phenomenon to make a brain-computer interface (BCI) to control future hacks.  My idea is to play two blinking movies simultaneously -- one at a slow speed and one at a fast speed.  I'm hoping that my brainwaves will only entrain with the blinking from the one movie that I choose to focus on.  Does my brain work this way?  Will my brain successfully reject the blinking from the movie that I'm ignoring?  Let's find out!

Simultaneous Blinking at Two Speeds:  Previously, I made some blinking movies where the whole screen would blink black or white at a given speed.  To make this idea work for a BCI, I want my screen to blink at two different rates at the same time.  So, as you can see in the video above, I made the left side of my screen blink at one rate while the right side of my screen blinks at a different rate.  I'm hoping that, if I focus my attention on the left side of my screen, by brainwaves will only become entrained at the left-side blink frequency, whereas if I were to focus on the right side of the screen, my brainwaves would follow the right-side blink frequency.

Swapping Sides:  To help with this test, I wanted to remove any effect of turning my head to change my gaze between the two sides of my screen.  So, in creating my dual-rate blinking movie, I had the movie automatically swap sides every 20 seconds.  As a result, it starts with fast blinking on the left and slow blinking on the right.  After 20 seconds, it swaps so that slow is on the left and fast is on the right.  It does this swap a few times.  The Matlab code that I used to make these movies is here.

I created a movie where the left and right sides blink independently -- left is fast and right is slow.
For this test, the two blink rates swapped sides every 20 seconds.

Choosing my Blink Rates:  Based on my previous results, it looks like my brain (coupled with my computer's limited ability to blink steadily) is most easily entrained in the 6-10 Hz frequency range. So, for this dual-rate movie, I chose "slow" to toggle between black and white at 10 Hz (ie, a 5 Hz white-white rate) and "fast" to toggle at 15 Hz (ie, a 7.5 Hz white-white rate).  In truth, I made a bunch of movies at different rates, but the the 10/15Hz movie worked the best, so I'll only show its results.

EEG Setup:  With my movies prepared, I gathered up my EEG stuff.  Like usual, I used my OpenBCI board and a few cup electrodes with Ten20 paste. I put one electrode on the left side of my forehead (Fp1), on one the left side of the back of my head (O1) and one on the right side of the back of my head (O2).  Using the impedance measuring feature, my impedances were 11 kOhm, 67 kOhm, and 28 kOhm (I seem to have an on-going problem getting a low impedance at O1).  My reference electrode was on my left ear lobe and my bias electrode was on my right ear lobe.  My OpenBCI board was connected to the PC via USB and I was logging data using my OpenBCI GUI in Processing.  For this test, I also used my photocell to confirm that my computer's blinking was sufficiently steady.

I used my OpenBCI V1 board to record my EEG into the computer.  I also attached a
photocell to confirm that the screen was blinking at the right rate.

Results:  After setting everything up, I started recording my EEG data and then I started playing the dual-rate blinking movie.  It was night time, so my room was pretty dark.  I focused my attention at the center of the left-hand movie. As described above, the left movie toggled fast-slow-fast-slow every 20 seconds, while the right movie played the opposite -- slow-fast-slow-fast.  Spectrograms of the EEG signals from my head are shown in the figure below.  As you can see, there was no entrainment seen in the signals from my forehead (as expected) but there was entrainment in the back of my head (also as expected).  The best entrainment was seen on the left side of my head.

Spectrograms of my EEG signals recorded while watching my dual-rate blinking movie.  The left-back
of my head exhibited the strongest entrainment to the blinking of my movie.  

Only Seeing the Left Blink Rate:  Because the left-back of my head (O1) gave the best entrainment, let's just focus on its results. The figure below shows just the results for the left-back of my head.  Note how,once the movie starts playing, my EEG signals seem to toggle between a fast blink rate (~7.5 Hz) and a slow link rate (~5 Hz).  This exactly follows the white-white blink rate of the left movie. So, my brainwaves successfully entrained to the movie that I was watching. Most importantly, there seems to be no signature in my EEG data from the blinking of the right movie.  This is success!

Spectrogram of my EEG data from the left-back of my head (O1) while watching my dual-rate
blinking movie.  I was focused just on the left movie.  Because of this focus, my brainwaves
appear to have entrained only with the left movie's blink rate.

Purposely Shifting My Attention:  OK,so I've demonstrated that my mind can successfully ignore one of the movies.  That's really good.  But, maybe I'm just biased to looking left. To really make this work for a BCI, I need to be able to shift my attention to either movie and have my brainwaves follow.  So, for my 2nd test, I started the same movie playing back.  But, this time, when the movies swapped sides every 20 seconds, I switched my attention to follow the movie that blinked faster. This means that I started by watching the left movie, then I watched the right, then left, then right.  My EEG response is shown below. Note that I showed strong entrainment and,most importantly, that my brainwaves only show the fast blink rate (7.5 Hz). So, by shifting my attention to follow the faster movie, I successfully rejected the effect of the slower blinking movie. Success again!

Spectrogram of my EEG data from the left-back of my head (O1) while watching my dual-rate blinking
movie.  While watching the movie, I switched my attention between left and right to follow the movie
 that blinked faster. Because of this focus, my brainwaves remained entrained only at the faster rate.

All the Elements are In Place:  It looks like I now have the elements in place for a 3-state BCI.  If I don't look at the movie at all, I get State 1: "Nothing". If I watch the blinking of the left movie, I get State 2: "Left".  If I watch the blinking of the right movie, I get State 3: "Right".  It may be possible to further divide my screen to get more blinking regions to add more BCI states. Maybe that's a good experiment for the future. Right now, though, I think that I'm going to turn my attention to a little robot that I got (thanks for the pointer Conor!) to see if I can control it with visual entrainment.  This is gonna be fun!

Follow-Up:  Interested in getting the EEG data from this post?  Try downloading it from my github!

Follow-Up:  I successfully used visual entrainment to control a six-legged robot!

Inducing Brain Waves with Visual Entrainment

A while back, I had a friend come over and I measured his EEG in response to staring at a blinking light.  We saw (as we hoped) that his brainwaves oscillated in sync with the blinking of the light.  I thought that this visual entrainment (aka "steady-stead visual evoked potential") was pretty cool.  Since then, I've learned that it can be used as the basis for a brain-computer interface (BCI).  Because I'm still searching for a good BCI paradigm, I decided to return to my exploration of visual entrainment.  Today, I'm going to show how I successfully used visual stimuli to induce brainwaves at different frequencies.  As a result, I can now see an good avenue for an EEG-based BCI.  Yes!  Let's go!

Inducing SSVEP Using a Toggling Checkerboard Pattern on my Computer Screen

Goal:  My goal today is to use visual stimuli to induce brainwaves across a range of frequencies.  Because I want to use this for a BCI, I'm trying to determine what kind of visual stimuli I should use and what EEG frequencies I can induce.  What does it take to make this work reliably?

Visual Setup:  In my previous post, my visual stimulation was simply a blinking head-lamp.  It was effective (and really bright!), but I had no control over its blink frequency.  As a result, I also had no control over the frequency of the brainwaves that it induced.  So, for today's test, I needed to get fancier.  I ditched the head-lamp and, instead, created a series of blinking movies that I could playback on my computer.  I controlled the "blinking rate" by saving my movies at different frame rates.  What exactly did the movie look like?  Well, at first, because of a paper that I read in the VEP literature, my movie used the checkerboard pattern shown in the picture above.  The movie toggled back-and-forth between this image and the inverse image (swap blacks and whites).  While this worked OK, I later switched to a simpler movie (code here) where the screen was simply all-white or all-black.  That seemed to work better.

EEG Setup:  Once I made my movies, I set myself up with my EEG system (OpenBCI ).  I my usual gold cup electrodes with Ten20 EEG paste.  I put one electrode on the back of my head (near O1) and I put another electrode on my forehead.  My reference electrode was on my left ear lobe and my bias electrode was on my right ear lobe.  Using the impedance-measuring feature of OpenBCI, the electrode on my forehead had an impedance of 24 kOhm and the one on the back of my head was about 65 kOhm.  I couldn't seem to get the back electrode to a lower value.

Channel 1 was on my Forehead, Channel 2 was on the back of my head.
My left ear lobe was my reference.  My right ear lobe was the bias.

I used my OpenBCI V1 board with Ten20 Paste.
I think that those owl napkins are fun!

Test Method:  Once I got everything setup, I launched the OpenBCI GUI in Processing and started an EEG recording.  To playback my homemade blinking movies, I opened up Windows Media Player and set it to full-screen mode.  I had ten movies, with each movie blinking at a different rate.  I had WMP play all 10 movies continuously in sequence.  Each movie was 20 seconds long, so the whole test took about 200 seconds.  It was nighttime when I did this test and my room was dimly lit.  I tried to stare at the screen and I tried to only blink my eyes at the transition between the different blinking rates.

Results, Checkerboard:  As usual, my preferred way to view the data is to make spectrograms.  In the figure below, the top plot is the data from my forehead and the bottom plot is the data from the back of my head.  From my forehead, there is nothing interesting except my eye blinks.  From the back of my head, we see several interesting features, which I've marked with blue and white arrows.  Note that these interesting features change every 20 seconds, which is the same as my 20-second movies.  It seems clear to me that these features are my brainwaves responding to changing of the blink rate in my movies.  Excellent!

Spectrogram of EEG Signal Recorded While Watching the Blinking Checkerboard Pattern.
The top plot is the signal from my forehead.  The bottom plot is from the back of my head.
"W-B" is the rate at which the movie switched from either white-to-black or black-to-white.
"W-W" is the rate if you measure just from white-to-white.

Entrained with the Blink Rate?  Looking at the three blue arrows, it appears that I have entrained brainwaves at 2 Hz, 4 Hz, and 6 Hz.  At these times, any given square in my movie was blinking at 1 Hz, 2 Hz, and then 3 Hz, if you count from white period to white period ("W-W").  Because I have entrained brainwaves as 2x the white-to-white frequency, it suggests that it is NOT white-to-white that matters, but that it is the transition between white/black or black/white that matters.  At least, that is what is implied for these three (out of 10) cases for the checkerboard stimuli.

Complications:  While that would be a fine conclusion, why does this rule not continue through the other 7 cases in this checkerboard test?  Why does it only work for the three cases with the blue arrows?  The cases with the white arrows do show some sort of EEG response, but not at any frequency that makes sense given the speed of my movies.  What is going on?  I've got two possible explanations: (1) either my movies are not playing back reliably during these other cases, or (2) the checkerboard pattern is too complicated to be a good starting point for learning about my brainwaves.

Modifying the Test:  Of these two possible explanations, it's easier for me to simplify the checkerboard than it is for me to fix the reliability of my movie playback.  So, I changed my movies so that the whole screen is either all all black or all white.  Hopefully, this simpler visual stimuli will make my EEG response easier to understand.

Results, Whole-Screen Blinking:  After recording my EEG while staring at the new movies, the spectrograms of my data are shown below.  Again, all of the interesting action is in the back of my head.  The bottom plot shows that I got good entrainment of my brainwaves for nearly *all* of the new movies.  I'm very pleased.  I'm also very curious about the jump between the cases marked with blue arrows versus the cases marked with red arrows.  What is happening here?

Spectrogram of EEG Signal Recorded While Watching the Whole Screen Toggle White or Black.
The top plot is the signal from my forehead.  The bottom plot is from the back of my head.

What Blink Rate Matters?  Looking at the first half of this plot, the blue arrows indicate cases that have results similar to the checkerboard data shown earlier.  Here, my brain seems to respond to every transition from white to black and from black to white (aka, the "W-B-W" speed).  But, for the second half of this plot, when the blinking is faster, it looks like my brainwaves follow the slower rate resulting from just the white-to-white frequency ("W-W").  Based on this weird result, I'm thinking that my brain doesn't actually care so much about whether the stimuli is W-B-W or W-W...it is simply sensitive to rhythmic visual stimuli in a certain frequency range.  I'm thinking that, whatever rhythmic stimuli falls in this frequency range, my brain will become entrained with it.

Quantifying Entrainment vs Frequency:  If it's simply the frequency that matters, it would be good to see which frequencies yield the strongest entrainment.  Sure, the spectrograms above suggest which frequencies are best, but I took the next step and actually measured the EEG response at each of the stimulation frequencies.  The plot below shows the EEG amplitude that I measured for each of the visual blinking frequencies.  Note that there are two lines, one for if you count based on the white-to-white frequency (blue line) or whether you're counting based on all the white/black and black/white transitions (red line).  This graph suggests that I seem to yield decent responses in the 6-10 Hz frequency range.  So, if I'm looking to use visual entrainment for a BCI, I should focus on the 6-10 Hz band.

Amplitude of EEG Signals Induced by Visual Entrainment.
My best responses seem to be in the 6-10 Hz band.
That could be a good target frequency range for use in a BCI.

Computer Could be Limiting my Performance:  As mentioned earlier, all of these results could be confounded by the possibility that my computer cannot reliably and steadily refresh my screen.  Perhaps it can reliably handle the frequencies at 10 Hz and below, but is not steady above 10 Hz.  Perhaps that's why my apparent response above 10 Hz falls off.  Sure, my computer claims that the screen has a 60 Hz refresh rate, but that doesn't mean that Windows or that Windows Media Player can keep up.  So, any next steps should include some method of assessing whether the computer is actually displaying my movies smoothly at the rate that I expect.

Entrainment for BCI:  My overall goal is to make a cool brain-computer interface (BCI).  Because I am showing that I can successfully measure visual entrainment, I would like to further explore how visual entrainment could be exploited for a BCI.  One idea is that I could simultaneously show two movies side-by-side, each blinking at its own rate.  Perhaps, if I'm lucky, my brainwaves will only respond to the one movie that I'm actually watching.  If that's the case, then I would have conscious control over my brainwaves (and, therefore, the BCI) simply by selecting which of the two movies that I watch.  That could be very cool.


Follow-Up:  I setup a photocell and my Arduino to measure the actual blink rate of the movies on my computer.  In my results, I found that I can't generate steady blinking faster than 10 Hz.  This is probably a strong reason why my EEG recordings exhibited no entrainment above 10 Hz...how can I entrain to signals that aren't there?!?

Follow-Up: I extended this work by having one movie blink at two different rates.  I found that I could control my entrained brainwaves by choosing which of the blink rates I focused on.  Pretty cool!  If you're interested, you can see the results in this post.

Follow-Up:  Interested in getting the EEG data from this post?  Try downloading it from my github!

Detecting Concentration

A couple of weeks ago, Sparkfun's new product post was all about the Neurosky Mindwave.  What really grabbed my attention was Nick Poole's video of his hack of using the Mindwave to bend a spoon.  That was a really fun and creative way to use EEG to interact with the physical world.  What also grabbed my attention was that it was yet another example of consumer EEG system saying that it detects "concentration", as if it were a well-known and well-defined EEG signature.  Along with terms like "focus" and "relaxation", I always felt that a word like "concentration" was too amorphous for serious consideration.  I mean, what exactly do "concentration" brain waves look like?  What is the signature?  I don't know.  But, given the coolness of Nick's demo, I decided to do some EEG Hacking to find out!

Neurosky Mindwave Electrode Setup:  I don't own a Neurosky Mindwave so I can't use that hardware to explore these "concentration" brain waves.  But, I do have an OpenBCI system, and it's pretty flexible, so I'll try that instead.  The main question is how to setup the electrodes.  Looking at the videos for the Mindwave, and looking at the Sparkfun hack pages, the Mindwave appears to use an electrode on the forehead and then another on an ear clip.  I'm assuming that the one on the ear clip is the reference electrode.  It does not appear to use a bias electrode, probably because they found that it was not needed for this body-mounted, battery-powered system.

OpenBCI Electrode Setup:  To mimic the Mindwave setup, I put a gold cup electrode on my forehead  and another on my left ear lobe.  The one on my forehead was plugged into Channel 1 of my OpenBCI board and the one on my ear lobe was used as the reference.  Because my system is not battery powered, I did use a bias electrode, which was an ear clip electrode placed on my right ear lobe (this is the first time I've tried the ear clip electrodes).  I also chose to stick another gold cup electrode on the back of my head, just to see what happened back there during this experiment.  Oh, and to attach my electrodes, I used standard Ten20 conductive paste.  My impedance check showed about 30 kOhm for each electrode, so not too bad.

Using OpenBCI (V2), 3 gold cup electrodes, and one ear clip.
Oh, and some guy's brain, too.

Procedure:  Watching Nick's video, he says that he is able to trigger the Mindwave's concentration detector by mentally counting backwards by 3, starting from 100.  This sounds pretty straight-forward and he clearly had good success with it.  Frankly, I was a little more skeptical about my own ability to make it happen.  So, in my data, to make it clear to me where I was trying to concentrate, I closed my eyes for a short period before and after my mental counting.  I did this because, by closing my eyes, I would generate strong alpha waves (10 Hz) that would clearly show up in the data.  As a result, after the test, I could look for the data between the two alpha wave recordings...this would be the period when I was concentrating.  Let's see what I got.

My First Look at the Data:  The spectrogram below shows how I typically look at an EEG recording for the first time.  Note that frequency is on the vertical axis and time is on the horizontal axis.  You can definitely see the signature of the alpha waves (that horizontal stripe around 10 Hz) at the beginning and at the end of my recording.  In the middle is the period of time when I was concentrating.  In this plot, I don't see anything interesting during the concentration portion of the test.  I just see some "noise" that looks little different from everything around it.  Bummer.

You can definitely see the alpha waves from my eyes being closed.  Good.
But, is anything happening during concentration?

Higher Frequencies?  But then I remembered reading somewhere (like in one of my own early posts?) that "concentration" was usually seen as increased activity in the higher EEG frequencies -- the so-called Beta waves (13-30 Hz).  So, I replotted the data where, this time, I zoomed way out on the frequency axis.  As you can see below, I'm now showing zero to 100 Hz.  In this new plot, you can clearly see that there is more EEG activity when I was concentrating compared to when I was not.  Now we're getting somewhere!

By zooming out to see the higher frequencies, it does look like there are more activity
in the high frequencies (20-100 Hz) when I am concentrating.  Cool!  (Note: the dark
horizontal stripe in the middle is the effect of my 60 Hz notch filter.)

Comparing the Spectra:  While spectrograms like the one above are helpful for quick qualitative views of both time and frequency, it is difficult to be quantitative with a spectrogram.  So, in the plot below, I show the average spectrum for a period of strong concentration (t = 90-130 sec) and I show the average spectrum for a period where my eyes were closed and my mind was especially quiet (t = 155-178 sec).  As can be seen below, the two spectra are definitely different, especially for frequencies above 22 Hz.

Comparing the average frequency spectrum with my eyes closed (t=90-130 sec)
to the average spectrum while concentrating (t=155-178 sec).  Note that
above 22 Hz, concentration exhibits more signal energy.

Detecting Concentration:  With the knowledge that, in my brain, "concentration" starts to show itself as increased EEG energy above 22 Hz, I can now contemplate building a concentration detector.  The key is to filter my EEG data so that I can assess the intensity of EEG activity in frequencies above 22 Hz.  Then, I'd pick a threshold to which I can compare the EEG intensity level.  If my EEG signals are stronger than my threshold, my detector would say that I am concentrating.  If I'm weaker than the threshold, my detector would say that I am not concentrating.  Sounds pretty easy, right?

Applying to My Recorded Data:  In the figure below, I apply this idea to the data that we've been discussing.  The top plot is the same spectrogram that I showed below.  The bottom plot is what happens when I filter the EEG data to show the intensity just for frequencies between 22 and 100 Hz.  You can see, the trace does indeed move up and down to reflect whether I'm concentrating or not.  Specifically, for the sustained concentration (t = 90-130 sec), my filtered EEG signal is running about 3.4 uVrms.  Then, when I close my eyes and relax (after t = 140 sec), my EEG signals drop down to about 2.0 uVrms.  So, if I were to define a threshold for detecting concentration, I might put it somewhere in the middle...say, around 2,7 uVrms.

Measuring the EEG amplitude in the 22-100 Hz frequency band.  Note how it is low while
my eyes are closed and that it goes higher while concentrating.

Feeling Some Success:  The plot above is making me pretty excited.  It suggests that I have conscious control over my EEG signals.  To date, I've only had strong success with controlling my Alpha waves (by opening and closing my eyes).  I've also had some small success with Mu waves, but they're really hard for me to get.  So, seeing this concentration-induced Beta (13-30 Hz) and Gamma (30-100 Hz) is pretty darned exciting.

Criticism: A critic reading this post might argue that I have not proven any link to concentration.  A critic might say that the increased high frequency EEG energy could just be a natural result of opening my eyes.  Based on the data shown so far, that would be a fair criticism.

Gathering More Eyes-Open Data:  To counter this criticism, the data below is from another test that I performed using the same setup.  In this test, I performed a similar procedure where I started with my eyes closed, had a period with my eyes open, and then finished with my eyes closed.  Unlike the previous test, though, I did not do my concentration exercise during the eyes-open period.  As a result, we should be able to see whether the increased high-frequency EEG activity is due to concentration or due to simply having my eyes open.

A second EEG test where I was NOT purposely concentrating during the eyes-open portion of the test.  Note that the EEG intensity is much less intense than seen during my previous test where I was purposely concentrating.

Not Concentrating:  In the plot above, you can see that there is a trend in my EEG signal strength, but that it is not related to the opening of my eyes.  At the beginning, when my eyes were closed, my high-frequency EEG signals were pretty low at 1.9 uVrms.  Then, when I opened my eyes (t = 210 sec), my EEG intensity increase only slightly to 2.0 uVrms and stayed that way for quite a while.  I think that this is strong evidence that simply opening your eyes does not specifically trigger increased Beta and Gamma activity.

Wandering Mind:  In the second half of my eyes-open period, we do see that my EEG intensity drifts upward.  Eventually, it averages about 2.5 uVrms.  Perhaps this increase reflects that I got bored and started thinking about my next EEG test.  Regardless of the reason, you'll note that even the increase to 2.5 uVrms still does not exceed the 2.7 uVrms threshold that we set a couple of paragraphs ago.  So, this small increase does not meet our criteria for "concentration".

Conclusion:  I think that this second data set is good evidence to declare that intense (>2.7 uVrms) Beta and Gamma activity is not due simply to opening my eyes.  I am feeling pretty confident that the intense high frequency EEG activity seen in the first data set is due to my concentration.  This means that Beta and Gamma activity is under my conscious control, which is the most exciting EEG result that I've had in a long time.

Next Steps:  Being under conscious control means that I could potentially use "concentration" as part of a brain-computer interface for future hacks.  I'm always looking for ways that I can try to control things in the physical world using just my brain waves.  Perhaps with some practice, I could use this technique to compete with Nick Poole in a spoon-bending competition!

Follow-Up:  I recorded my concentration level while eating breakfast, and found some really cool changes!
Follow-Up:  Interested in getting the data from this post?  Try downloading it from my github!