Brain Got Beats -- Not Yet

I like controlling things with my mind.  That's why I do this brain-computer interface (BCI) thing.  The tough part of BCIs, though, is finding brain signals that are simple enough for the computer to detect, yet are also something that I can consciously control.  So far, I can do eyes-closed Alpha waves, concentration-controlled Beta/Gamma, and steady-state visual evoked potential (SSVEP).  I need more options.  Today, I'm going to try to do auditory steady state response (ASSR).  Or, more colloquially, does my brain got beats?

Can I use beating tones to entrain brainwaves?

Auditory Steady-State Response (ASSR)

The idea with ASSR is that we are looking for EEG signals from my brain that are driven by sounds presented to my ears.  When doing an ASSR, you use an audio tone whose amplitude is varied ("modulated") at a fixed rate such as 40 Hz.  Then, when you play that sound in your ears, you look in the EEG signals for a strong 40 Hz component.  Easy, eh?

Note that this is very similar to the steady-state visual evoked potential (SSVEP) that I used previously, where I'd make my computer screen blink at 8 Hz and 8 Hz signals would appear in my EEG.

Attention-Based ASSR?

If I want to use ASSR for a brain-computer interface (ie, for controlling robots!), there needs to be some way to consciously control my response to the sound.  For the SSVEP, where stimulation was my blinking computer screen, my response was much stronger if I consciously paid attention to the blinking screen.  This attention-based response was the key to being able to exploit it for a BCI.

Does ASSR have a similar attention-based component?  Until yesterday morning, I didn't know.  But then I came across this paper:  Do-Won Kim et al.  "Classification of selective attention to auditory stimuli: Toward vision-free brain–computer interfacing".  Journal of Neuroscience Methods 197 (2011) 180–185.  PDF here.

Kim's ASSR Setup

In the paper by Kim, they used two loudspeakers to present tones to the test subject.  The setup is shown below.  The subjects were sitting down in a comfy chair listening to the tones while wearing a small montage of EEG electrodes (Cz, Oz, T7, T8, ref at left mastoid, ground at right mastoid).

Test Setup as used by Kim (2011) for Evoking Auditory Steady-State Response (ASSR)

For the audio tones, they used a 2500 Hz tone from one speaker and a 1000 Hz tone from another speaker.  The key feature of ASSR, though, is the modulation of these tones.  For one of the tones, they varied the amplitude of the tone (ie, they alternately made it quiet and loud) at a rate of 37 Hz, while the other tone they modulated at a rate of 43 Hz.  These frequencies are the "beat rates" for the audio.  It is the 37 Hz or 43 Hz beat rate that they are looking for in the EEG (hence, "brain got beats?").

Below is what they saw in the EEG signals (Cz) for one of their subjects when the subject gave their attention to the 37 Hz modulated signal (red) or the 43 Hz modulated signal (blue).  There is clearly a difference.  This makes me happy.  This is what I want to recreate with my own testing.

Spectral Results for One Subject from Kim (2011) In Response to Steady-Pitch
Tones that were Amplitude Modulated at 37 Hz or 43 Hz.

My Test Setup

I want to recreate their results.  I'm going to create some audio files with the amplitude modulated signals, I'm going to play them into my ears via headphones, and I'm going to record my EEG signals (OpenBCI!) to look for my ASSR.

EEG Setup:  Reading more details from the paper, they said that they got the strongest response from the electrode at Cz, so I decided to start there.  I put one electrode at the top of my head (Cz) with the reference on my left ear lobe and the OpenBCI "bias" on my right ear lobe.  I used the gold electrodes and the Ten20 EEG paste that came with the OpenBCI kit.  Without really trying, I happened to get an electrode impedance of 20-30 kOhm at both Cz and at the reference, which are probably good enough.

My EEG Setup, Cz Only.  Also, unlike Kim, I used ear buds (headphones)
stead of loudspeakers to present my tones.

OpenBCI EEG System:  For this test, I happened to use my 16-channel OpenBCI system.  I'm only using one channel of EEG data, though, so I could have used the 8-channel systems (or even other systems, like OpenEEG) just as well.  I wired up my OpenBCI unit as shown below.  Starting from the left, the white wire is the "bias" (aka, driven ground) going to my right ear lobe, the brown wire is the electrode at the top of my head, and the black wire is the reference electrode on my left ear lobe.  Note that they are all plugged into the lower row of pins (the "N" inputs) on the lower board.  The system is being powered by four AA batteries and is sending its data wirelessly back to the PC.  I'm using the OpenBCI GUI in Processing.

Here's How I Plugged into the OpenBCI Board.

Audio Files:  I created my audio files in Audacity.  I created two sets of files, based on the frequencies used in the Kim paper: one set of files using a 1000 Hz tone and the other set using a 2500 Hz tone.  The Kim paper said that the strongest ASSR generally occurs for a beat frequency of 40 Hz.  I wanted to see my response at different beat frequencies, so for each tone I created three versions: one beating at 38 Hz, one at 40 Hz, and one at 42 Hz.  I made each version 20 seconds long.  I used a square wave (ie, on/off) amplitude modulation, though next time I might try sine wave modulation instead.

I Created My Amplitude-Modulated (AM) Test Tones in Audacity.  First, "generate" the
tone.  Then, to do the AM, go under "Effect" and select "Tremolo". 

Data and Analysis Files:  My audio files, my data files, and my analysis files are all on my GitHub here.  Note that I did my analysis using an IPython Notebook (see it here).  My specific Python installation is described here.

My ASSR Response

My goal is to see if I exhibit the ASSR response with this test setup.  To do the test, I wired myself up as discussed above, I queued up all six audio files (the three at 1000 Hz followed by the three at 2500 Hz), put in my ear buds, and started recording.

Eyes Closed:  The spectrogram below shows my Cz EEG signal when I did this test with my eyes closed.  That strong red stripe at 10 Hz is my Alpha response simply due to having my eyes closed.  What I do not see here are horizontal stripes of energy at 38, 40, or 42 Hz.  In other words, I do not see any brain waves entraining with the audio stimulation.  This is disappointing.

Spectrogram of EEG Signal from Cz with AM Auditory Stimulation Near 40 Hz.

My eyes were closed, hence the strong response at 10 Hz.

There is no signature of the 38-42 Hz AM Audio Stimulation.

Eyes Open:  I also performed this test with my eyes open.  A spectrogram of my EEG signal at Cz is shown below.  I started and ended the test with my eyes closed for 10 seconds, which you can see as 10 Hz Alpha waves at the start and end.  What I really want to see, though, is something corresponding to the audio stimulation at 38 Hz, 40 Hz, or 42 Hz.  Again, I see nothing.

Spectrogram of EEG Signal from Cz with AM Auditory Stimulation Near 40 Hz.

My eyes were open, except at the beginning at end.

There is no signature of the 38-42 Hz AM Audio Stimulation.

Average Spectrum:  To most closely mimic the plot from the Kim paper (ie, the graph that I copied earlier), I plotted the average spectrum.  In the Kim plot, there were clear peaks at his two beat frequencies (37 and 43 Hz).  In my equivalent plot below, there are no peaks at the three beat frequencies that I studied (38, 40, and 42 Hz).

Mean Spectrum During the Test Period.  There is no evidence of my brain waves entraining

with the 38, 40, and 42 Hz AM auditory signals.  Bummer.

Conclusion:  So, it is clear that i did not see any ASSR in my EEG recordings.  This is very disappointing to me.

Comparison to Kim

Why did Kim see ASSR and I did not?  I'm not sure.  Maybe my test setup or my audio files were sufficient different to prevent the response.  Or, maybe I'm reading too much into his results...

In looking back at his plot with the spectrum from one of his subjects (copied earlier in this post), I see that the y-axis is a linear axis, whereas I always do dB.  What might his values look like when converted to dB?

As an example, I see that his first peak is 0.40 uV^2, relative to a baseline of about 0.30 uV^2.  Converted to dB (re: 1 uV^2), this would be -4.0 dB and -5.2 dB.  Comparing to my own spectrum plot above, where my baseline is about -10 dB, any peak at -4.0 dB should be easily seen.  Therefore, if my own response were as strong has Kim's subject's response, I would think that I would see the response in my plots.  I don't see the peak, so I guess that I didn't have the response as strongly as Kim's subject.

Perhaps the "gotcha" here is that the difference in Kim's data between the peak (-4.0 dB) and the baseline (-5.2 dB) is only 1.2 dB.  That is a really small difference.  For reliable detection, I generally like to see 6-10 dB of difference.  It might be too much to hope to reliably see only a 1.2 dB difference.

Next Steps

I'm not going to give up yet.  I'm going to try again.  I'm going to try using the additional EEG electrodes as used by Kim and I'm going to try to use sine-wave modulation instead of square-wave modulation.  I want to see this response!

OpenBCI Accelerometer Data

The OpenBCI V3 board does more than just EEG.  Yes, I've already shown examples of doing ECG and EOG with my old V1 and V2 boards, but the new V3 board includes an accelerometer, which the old boards did not have.  How could an accelerometer be useful?  Well, you could use it to sense orientation (or change in orientation) of the head as part your BCI.  Or, you could use it to sense rough motion, which might suggest that you'll have motion artifacts in your EEG data.  Or, you could sense yourself tapping on the board as a way to introduce markers during your data collection.  There are many possibilities!  Today, I'm going to look at the accelerometer data for the first time.

OpenBCI V3 Board with Batteries

Goal: My goal is to record accelerometer data during known motions so that I can confirm that the data matches the motions.

Setup:  I used my OpenBCI V3 Board (see picture above) as delivered from OpenBCI.  On the OpenBCI board, I was running the same software as was shipped by OpenBCI in November, 2014.  The OpenBCI board used its wireless link to the PC.  On the PC, I ran the OpenBCI GUI in Processing.  The GUI logged the data to a file.

Procedure:  I inserted the batteries to my OpenBCI board to give it power. I started the OpenBCI GUI to begin recording data. Holding the board in my hand, I completed the following maneuvers:

1. Start with board flat and level (z-axis points up, like in the picture at the top)
2. Roll it 90 deg to the right (x-axis points down) and 90 deg left (x-axis points up)
3. Tip it nose down (y-axis points down) and nose up (y-axis points up)
4. Flip it upside down (z-axis points down)

Notice the markings on the OpenBCI board (zoomed picture below) that indicate the direction of the accelerometer's axes.

The accelerometer is the small black square towards the bottom.
Note that "X" points right, "Y" points forward, and "Z" comes up out of the board.

Data Files:  The 3-axis accelerometer data was saved to a text file by the Processing GUI.  I analyzed the data using Python. The data and analysis files are available on my EEGHacker repo on GitHub.  If you use this data, be sure to unzip the ZIP file in the SavedData directory!

Analysis:  The specific goals of this analysis are to confirm that the data is well behaved, that the correct axes are responding to the known motions, that the units are correct, and that the scale factors are correct.  I used an IPython Notebook to step through each one of these analyses.  You can see the IPython Notebook here.

Results, Data Continuity: The first thing I did was to look at the data to make sure that it was well behaved.  The most important part of being well behaved is that the data is continuous.  Looking at the packet counter in the data file (a counter which is transmitted by the OpenBCI board), there were no missing data packets.  Excellent.  I did see however that accelerometer data is only included in every 10th or 11th data packet.  Why?  Well, looking at the code on the OpenBCI board, it has configured the accelerometer to only produce data at 25 Hz.  So, compared to the 250 Hz sample rate for the EEG data (which then drives a 250 Hz rate for data packets), we see why we only get acceleration values every 10th or 11th packet.  It makes sense.  Good.

Results, Individual Axes: After ensuring that the data was continuous, I looked at the data values themselves.  I plotted the acceleration values as a function of time.  The plots below show the values recorded from each of the accelerometer's three axes.  As can be seen, the signals clearly reflect the maneuvers defined in my procedure.  Additionally, from these plots, we learn that negative acceleration values result when the accelerometer's axis is pointing down (relative to gravity) and positive values result when the axis is pointing up.  This polarity information is important if you wish to use the accelerometer data to estimate the orientation of the OpenBCI board.

Acceleration Values for the Accelerometer's Three Axes.
The three channels correspond to the X, Y, and Z axes.

Results, Scale Factor:  With the behavior of the 3 axes shown to be reasonable, I then wanted to confirm that the magnitude of the values were correct.  I wanted to make sure that the scale factors used for interpreting the raw values was correct.  The quickest way for me to confirm the scale factor was to compute the magnitude of the 3-axis acceleration vector.  When the device is at rest, the magnitude of the measured acceleration should equal gravity, which is 1.0 G.  As you can see below, the magnitude of our acceleration was generally close to 1.0 G (though often a little high), except when it was moving during its transitions between positions.  This is good.

The magnitude of the 3-axis acceleration vector should equal 1.0 G when at rest.
Ours equals about 1.044 G, which is within the known offset error bounds of the device

When I look very closely at the values, it appears that the typical reading is actually 1.044 G instead of 1.000 G.  There is a 44 mG difference.  Is this unexpected?  Well, yes, it was unexpected at first.  And then I read the datasheet.  Always look at the datasheet.  In this case, it reports that the accelerometer should have a typical offset error of 40 mG per axis.  For a 3-axis device, this could result in sqrt(40² + 40² + 40²) = 69 mG of error on my magnitude value.  As a result, my 44 mG value appears to be in-line with the device's advertised performance.  That's satisfying.

Conclusion:  With this test, I confirmed that my accelerometer is sending well-behaved data, with all three axes responding appropriately to known motions, with all axes having the correct scale factor.  These are good results and I'm pleased. Now it's time to figure out something fun to do with the accelerometer!

First Alpha with OpenBCI V3

OK, I'm back to work now.  After my previous post, where I got my first data ever with the new OpenBCI board (aka, "V3"), I took a little hiatus while Joel worked through some issues with the Bluetooth link.  Everything appears to be working well now, so I'm back on the case.  Yesterday, I connected everything up and recorded my first real EEG data with the V3.  Exciting!

My Little OpenBCI V3 Board (8-Bit Version) with Homemade Electrode Adapter

EEG Setup:  To get started, I set an easy goal for myself -- just record some eyes-closed alpha waves.  So, I got out my trusty gold electrodes, my trusty Ten20 electrode paste, and put on a few electrodes.  I attached one electrode to the back-left of my head ("O1"), the reference electrode to my left earlobe, and the bias electrode to my right earlob.  So far, this is just like normal.

Software Setup:  For software, I used Arduino software for the OpenBCI Bluetooth dongle (aka, the RFduino "Host"), for the remote Bluetooth module on the OpenBCI V3 board itself (aka, the RFduino "Device"), and for the Atmel microcontroller that is the core of the OpenBCI V3 board (and which is programmed like an Arduino Uno).  The software is surely going to change with time, but right now I'm working with this code here.  On the PC side, I used a version of our Processing GUI that we modified to accept the new binary data format being generated by the V3 board.

Data and Analysis:  I did a couple of recordings of my eyes-closed alpha waves.  My data and analysis files are here.  Some example plots of the data that I recorded are shown below.  This is my first time trying to analyze the data using Python instead of Matlab.  Because I'm so new with Python, I was a lot slower in doing the analysis, but now that I've completed this one little task, I'm feeling pretty OK about the switch.  Maybe, just maybe, it is possible to learn new tricks!

EEG Data Recorded from the Back-Left of my Head ("O1") After Closing my Eyes
around t = 88 seconds.  By closing my eyes, I get alpha waves appear near 10 Hz.

Time-Domain Plot:  The top plot is a simple plot of the recorded EEG signal as a function of time.  Actually, it's not a totally "simple" plot because I have done some processing of the data.  I highpass filtered it to remove the DC component and I notch filtered it at 60 Hz and 120 Hz to get rid of power line interference.  In my opinion, though, time-domain plots are not very useful when zoomed out to a wide range of time (like we're doing here).  So, there's not much to say.

Spectrogram:  The middle plot is a spectrogram of the same data.  I love spectrograms.  Here, time is again on the horizontal axis, but now frequency is on the vertical axis.  The intensity of the color of each pixel shows how much signal energy is at the pixel's time and frequency.  Here, by the red horizontal line that appears ~10 Hz, you can clearly see the alpha waves.  Cool!

Frequency-Domain Plot:  Unfortunately, it is difficult to be quantitative about the amplitude of signals that are seen in the spectrogram.  So, once I located my alpha waves (t = 90 sec to t = 118 seconds), I plotted the mean spectrum for the data just in that time period.  The bottom plot shows this spectrum -- it shows the spectrum of my brain waves during t = 90 sec to t = 118 sec  You can see the prominent bump around 10 Hz.  These are my alpha waves.  As can be seen. the amplitude is approximately 4.1 uVrms and the peak is focused at 9.38 Hz.  That's my brain!  Specifically, that's my visual cortex when it's bored because my eyes are closed!

So, that's the quick fun that I had using the new OpenBCI V3 hardware and the fun that I had using Python for the first time to make decent graphs.  Learning new things makes me feel pretty empowered.  To celebrate, I'm going to go eat some breakfast now.  Mmm...Wheat Chex...I really know how to party.  ;)

Follow-Up:  Here's some additional discussion on how to detect these Alpha waves

Moving from Matlab to Python

For any real data-hounds out there, you've probably noticed that I do most of my data analysis and plotting in Matlab.  It's a computing and plotting package that I've been using for a long, long time (almost 20 years...yikes!).  As a result, I'm very comfortable with it, which makes it nearly effortless for me to use it to explore new data.  Unfortunately, Matlab is very expensive (thousands of dollars), so it's unlikely that there are very many other hobbyists that are likely to have this tool.  As a result, while I have been sharing all of the Matlab EEG analysis code on my GitHub, it is a bit pointless since Matlab itself is so unavailable.

To make my EEG analysis code more usable for other folks, I've decided to put on my big boy pants and to try to learn something new.  I'm making the jump to Python.  Look out!

Developing Python Code in the Spyder IDE

The primary benefits of Python are that it's free and that it has a huge community of developers.  As an added benefit to me, there are also a lot of former Matlab programmers who have made the jump to Python, which means that there is a lot of Matlab-reminiscent Python code out there for Matlab junkies like myself to use as a gateway drug to Python.

When getting started in Python, it's helpful to use a pre-packaged Python distribution that includes a good selection of helpful packages along with the Python core.  Based on some guidance from my buddy Rob, I decided to use the "Anaconda" distribution because it includes tried-and-true packages numpy (numerical math routines and data structures), scipy (scientific computing routines, such as filtering), and matplotlib (a collection of Matlab-style plotting routines).  It's a pretty sweet distribution.  You can download it for free from the Anaconda website.

As with any programming language, one's first experience is highly dependent upon the quality of the development environment.  I've chosen to start with the Spyder IDE, which is shown in the screenshot above.  I don't know if it is considered to be an especially good or especially bad IDE, but it seems to do the job.  If you're a Matlab ninja turned Python super-hero, I'd love to hear what IDE you use.

So, moving forward, you should expect to see me share my Python code for my upcoming EEG Hacker posts.  While my Python style will be ugly (being a Python newbie, how can it be anything but ugly?), hopefully you can read it well enough to learn a few ideas (or to teach me a few ideas) on how to do some EEG processing.

Wish me luck...'cause I'm jumping in!