Brain Got Beats -- Not Yet

I like controlling things with my mind.  That's why I do this brain-computer interface (BCI) thing.  The tough part of BCIs, though, is finding brain signals that are simple enough for the computer to detect, yet are also something that I can consciously control.  So far, I can do eyes-closed Alpha waves, concentration-controlled Beta/Gamma, and steady-state visual evoked potential (SSVEP).  I need more options.  Today, I'm going to try to do auditory steady state response (ASSR).  Or, more colloquially, does my brain got beats?

Can I use beating tones to entrain brainwaves?

Auditory Steady-State Response (ASSR)

The idea with ASSR is that we are looking for EEG signals from my brain that are driven by sounds presented to my ears.  When doing an ASSR, you use an audio tone whose amplitude is varied ("modulated") at a fixed rate such as 40 Hz.  Then, when you play that sound in your ears, you look in the EEG signals for a strong 40 Hz component.  Easy, eh?

Note that this is very similar to the steady-state visual evoked potential (SSVEP) that I used previously, where I'd make my computer screen blink at 8 Hz and 8 Hz signals would appear in my EEG.

Attention-Based ASSR?

If I want to use ASSR for a brain-computer interface (ie, for controlling robots!), there needs to be some way to consciously control my response to the sound.  For the SSVEP, where stimulation was my blinking computer screen, my response was much stronger if I consciously paid attention to the blinking screen.  This attention-based response was the key to being able to exploit it for a BCI.

Does ASSR have a similar attention-based component?  Until yesterday morning, I didn't know.  But then I came across this paper:  Do-Won Kim et al.  "Classification of selective attention to auditory stimuli: Toward vision-free brain–computer interfacing".  Journal of Neuroscience Methods 197 (2011) 180–185.  PDF here.

Kim's ASSR Setup

In the paper by Kim, they used two loudspeakers to present tones to the test subject.  The setup is shown below.  The subjects were sitting down in a comfy chair listening to the tones while wearing a small montage of EEG electrodes (Cz, Oz, T7, T8, ref at left mastoid, ground at right mastoid).

Test Setup as used by Kim (2011) for Evoking Auditory Steady-State Response (ASSR)

For the audio tones, they used a 2500 Hz tone from one speaker and a 1000 Hz tone from another speaker.  The key feature of ASSR, though, is the modulation of these tones.  For one of the tones, they varied the amplitude of the tone (ie, they alternately made it quiet and loud) at a rate of 37 Hz, while the other tone they modulated at a rate of 43 Hz.  These frequencies are the "beat rates" for the audio.  It is the 37 Hz or 43 Hz beat rate that they are looking for in the EEG (hence, "brain got beats?").

Below is what they saw in the EEG signals (Cz) for one of their subjects when the subject gave their attention to the 37 Hz modulated signal (red) or the 43 Hz modulated signal (blue).  There is clearly a difference.  This makes me happy.  This is what I want to recreate with my own testing.

Spectral Results for One Subject from Kim (2011) In Response to Steady-Pitch
Tones that were Amplitude Modulated at 37 Hz or 43 Hz.

My Test Setup

I want to recreate their results.  I'm going to create some audio files with the amplitude modulated signals, I'm going to play them into my ears via headphones, and I'm going to record my EEG signals (OpenBCI!) to look for my ASSR.

EEG Setup:  Reading more details from the paper, they said that they got the strongest response from the electrode at Cz, so I decided to start there.  I put one electrode at the top of my head (Cz) with the reference on my left ear lobe and the OpenBCI "bias" on my right ear lobe.  I used the gold electrodes and the Ten20 EEG paste that came with the OpenBCI kit.  Without really trying, I happened to get an electrode impedance of 20-30 kOhm at both Cz and at the reference, which are probably good enough.

My EEG Setup, Cz Only.  Also, unlike Kim, I used ear buds (headphones)
stead of loudspeakers to present my tones.

OpenBCI EEG System:  For this test, I happened to use my 16-channel OpenBCI system.  I'm only using one channel of EEG data, though, so I could have used the 8-channel systems (or even other systems, like OpenEEG) just as well.  I wired up my OpenBCI unit as shown below.  Starting from the left, the white wire is the "bias" (aka, driven ground) going to my right ear lobe, the brown wire is the electrode at the top of my head, and the black wire is the reference electrode on my left ear lobe.  Note that they are all plugged into the lower row of pins (the "N" inputs) on the lower board.  The system is being powered by four AA batteries and is sending its data wirelessly back to the PC.  I'm using the OpenBCI GUI in Processing.

Here's How I Plugged into the OpenBCI Board.

Audio Files:  I created my audio files in Audacity.  I created two sets of files, based on the frequencies used in the Kim paper: one set of files using a 1000 Hz tone and the other set using a 2500 Hz tone.  The Kim paper said that the strongest ASSR generally occurs for a beat frequency of 40 Hz.  I wanted to see my response at different beat frequencies, so for each tone I created three versions: one beating at 38 Hz, one at 40 Hz, and one at 42 Hz.  I made each version 20 seconds long.  I used a square wave (ie, on/off) amplitude modulation, though next time I might try sine wave modulation instead.

I Created My Amplitude-Modulated (AM) Test Tones in Audacity.  First, "generate" the
tone.  Then, to do the AM, go under "Effect" and select "Tremolo". 

Data and Analysis Files:  My audio files, my data files, and my analysis files are all on my GitHub here.  Note that I did my analysis using an IPython Notebook (see it here).  My specific Python installation is described here.

My ASSR Response

My goal is to see if I exhibit the ASSR response with this test setup.  To do the test, I wired myself up as discussed above, I queued up all six audio files (the three at 1000 Hz followed by the three at 2500 Hz), put in my ear buds, and started recording.

Eyes Closed:  The spectrogram below shows my Cz EEG signal when I did this test with my eyes closed.  That strong red stripe at 10 Hz is my Alpha response simply due to having my eyes closed.  What I do not see here are horizontal stripes of energy at 38, 40, or 42 Hz.  In other words, I do not see any brain waves entraining with the audio stimulation.  This is disappointing.

Spectrogram of EEG Signal from Cz with AM Auditory Stimulation Near 40 Hz.

My eyes were closed, hence the strong response at 10 Hz.

There is no signature of the 38-42 Hz AM Audio Stimulation.

Eyes Open:  I also performed this test with my eyes open.  A spectrogram of my EEG signal at Cz is shown below.  I started and ended the test with my eyes closed for 10 seconds, which you can see as 10 Hz Alpha waves at the start and end.  What I really want to see, though, is something corresponding to the audio stimulation at 38 Hz, 40 Hz, or 42 Hz.  Again, I see nothing.

Spectrogram of EEG Signal from Cz with AM Auditory Stimulation Near 40 Hz.

My eyes were open, except at the beginning at end.

There is no signature of the 38-42 Hz AM Audio Stimulation.

Average Spectrum:  To most closely mimic the plot from the Kim paper (ie, the graph that I copied earlier), I plotted the average spectrum.  In the Kim plot, there were clear peaks at his two beat frequencies (37 and 43 Hz).  In my equivalent plot below, there are no peaks at the three beat frequencies that I studied (38, 40, and 42 Hz).

Mean Spectrum During the Test Period.  There is no evidence of my brain waves entraining

with the 38, 40, and 42 Hz AM auditory signals.  Bummer.

Conclusion:  So, it is clear that i did not see any ASSR in my EEG recordings.  This is very disappointing to me.

Comparison to Kim

Why did Kim see ASSR and I did not?  I'm not sure.  Maybe my test setup or my audio files were sufficient different to prevent the response.  Or, maybe I'm reading too much into his results...

In looking back at his plot with the spectrum from one of his subjects (copied earlier in this post), I see that the y-axis is a linear axis, whereas I always do dB.  What might his values look like when converted to dB?

As an example, I see that his first peak is 0.40 uV^2, relative to a baseline of about 0.30 uV^2.  Converted to dB (re: 1 uV^2), this would be -4.0 dB and -5.2 dB.  Comparing to my own spectrum plot above, where my baseline is about -10 dB, any peak at -4.0 dB should be easily seen.  Therefore, if my own response were as strong has Kim's subject's response, I would think that I would see the response in my plots.  I don't see the peak, so I guess that I didn't have the response as strongly as Kim's subject.

Perhaps the "gotcha" here is that the difference in Kim's data between the peak (-4.0 dB) and the baseline (-5.2 dB) is only 1.2 dB.  That is a really small difference.  For reliable detection, I generally like to see 6-10 dB of difference.  It might be too much to hope to reliably see only a 1.2 dB difference.

Next Steps

I'm not going to give up yet.  I'm going to try again.  I'm going to try using the additional EEG electrodes as used by Kim and I'm going to try to use sine-wave modulation instead of square-wave modulation.  I want to see this response!

EEG as WAV Files, Go Spectrograms!

OK, let's say that I just finished some cool new EEG experiment where I recorded my EEG response to watching cat videos while listening to the Pink Panther at half speed.  My next step would be to take a quick look at the data to get the overall big picture.  My favorite way of getting that overall view is to make a spectrogram (see example below).  My love for these oh-so-colorful plots runs deep.  The question is, how does one make spectrograms?  Well, in my opinion, if you don't have Matlab (and are afraid of Python), the next best way to make spectrograms is to use one of the multitude of audio editing software packages out there.  Many audio edit programs provide a spectrogram view.  This post is about getting EEG data into an audio program so that you can see your data.

A Spectrogram of EEG data that was Made in Matlab.  This shows data from my previous post,
where I was watching a movie with two different blink rates.  You can see how my brainwaves
entrained with the changing blink rate in the movie.

Converting to a WAV File:  The first step in using an audio program for EEG analysis is to convert one's EEG data into an audio file.  Since I usually work in Windows, I tend to convert all of my EEG data into WAV files.  I choose WAV because it is uncompressed.  I never choose MP3 because it is very unclear what its "perceptual coding" would do to my precious brainwave data.  So, a WAV file is what I would recommend.  But how do you get EEG data into a WAV format?  If your EEG data is in text format (such as is logged by the OpenBCI GUI), you could use my Processing sketch "ConvertToWAV".  This sketch will read in an OpenBCI log file and write each EEG channel out as its own WAV file. You can get the sketch on my GitHub.

Audacity:  Once the data is in WAV format, you can open it in any audio program.  A popular (and free!) audio editing program is Audacity.  While it is not my favorite audio editing program, it is perfectly sufficient for working with EEG data.  After opening your EEG data, the trick is to figure out how to switch the display from waveform to spectrogram.  The screen shot below shows how to do it.

Changing to Spectrogram View in Audacity

Once Audacity is in spectrogram mode, you need to zoom in on the vertical axis in order to see the interesting EEG features, which are usually focused in the lower frequencies.  In Audacity, you zoom simply with a click-and-drag on the vertical axis.  Then, after manipulating the spectrogram settings under the "Preferences" menu, you can get a spectrogram like the one shown below. While the color scheme hurts the eyes a bit, this spectrogram is good enough to see the same kind of EEG entrainment as seen in my original Matlab plot. Furthermore, the tools in Audacity let you further analyze the EEG data through zooming, filtering, amplifying, and (if you change the file's sample rate to increase the playback speed) you can use Audacity to listen to your own brain waves!  Audacity is definitely a useful tool for working with EEG data.

In Audacity, a Spectrogram of my EEG Data

The spectrogram settings that I used are shown in the screen shot below.

My Display Settings for Making EEG Spectrograms in Audacity.  I changed
the Window Size, the Gain, and the Range.

Cool Edit Pro:  I first started getting into spectrograms in the late 90's because this is when I started working with audio and music on the computer. What got me hooked on spectrograms was a piece of shareware called Cool Edit.  It was a stupid name for an otherwise outstanding program.  It was so useful that I spent the extra dollars and bought its upgrad -- Cool Edit Pro. Cool Edit Pro has a *great* spectrogram display, as shown below.  Unlike Audacity, which requires lots of manipulation of the spectrogram settings to get a useful view, the Cool Edit Pro display always seems just right.  Unfortunately, Cool Edit Pro isn't available anymore -- it was bought by Adobe in the early 2000s and became Adobe Audition.  Audition is also fine for making spectrograms (I have only used up to Audition 3.0), but it is expensive.

An EEG Spectrogram in Cool Edit Pro V1.2a.  It's an old school program that totally rocks.

In Cool Edit Pro, the only display parameter that you need to change is the "Resolution" (ie, FFT size).  You do that under the "Settings" menu.

My settings for viewing EEG spectrograms in Cool Edit Pro.
I changed the Resolution value.

Raven Lite:  A third option for making spectrograms is a bit more obscure.  A bunch of years ago, I came across a program called "Raven Lite", which is produced by the Ornithology Lab (ie, bird science) at Cornell University.  The "Lite" version is free.  You can download it and immediately use it for spectrograms, though it is crippled in other ways until you email them for a free (non-commercial) key. What I really like about Raven is that, as shown in the screen shot below, its spectrogram controls are right on the main window for easy manipulation.  Also, I like its color map options way better than what is available in Audacity.  Finally, Raven is one of the few programs that let you see both the spectrogram view and the waveform view at the same time (not shown).  It is really nice to have that capability.

Raven Lite 1.0 from the Cornell Laboratory of Ornithology. It's a pretty good viewer.  The settings for
the display or right here in the main window.

Other Options:  Because I have Matlab and Cool Edit Pro (and Audacity and Raven) I haven't spent a lot of time looking at other options.  Does Garage Band offer a spectrogram view?  Is there a plug-in for iTunes or Windows Media Player that gives spectrograms?  I'm curious to hear what you folks use.  Drop a comment and let me know!