Tuesday, December 2, 2014

OpenBCI Accelerometer Data

The OpenBCI V3 board does more than just EEG.  Yes, I've already shown examples of doing ECG and EOG with my old V1 and V2 boards, but the new V3 board includes an accelerometer, which the old boards did not have.  How could an accelerometer be useful?  Well, you could use it to sense orientation (or change in orientation) of the head as part your BCI.  Or, you could use it to sense rough motion, which might suggest that you'll have motion artifacts in your EEG data.  Or, you could sense yourself tapping on the board as a way to introduce markers during your data collection.  There are many possibilities!  Today, I'm going to look at the accelerometer data for the first time.

OpenBCI V3 Board with Batteries
Goal: My goal is to record accelerometer data during known motions so that I can confirm that the data matches the motions.

Setup:  I used my OpenBCI V3 Board (see picture above) as delivered from OpenBCI.  On the OpenBCI board, I was running the same software as was shipped by OpenBCI in November, 2014.  The OpenBCI board used its wireless link to the PC.  On the PC, I ran the OpenBCI GUI in Processing.  The GUI logged the data to a file.

Procedure:  I inserted the batteries to my OpenBCI board to give it power. I started the OpenBCI GUI to begin recording data. Holding the board in my hand, I completed the following maneuvers:
  1. Start with board flat and level (z-axis points up, like in the picture at the top)
  2. Roll it 90 deg to the right (x-axis points down) and 90 deg left (x-axis points up)
  3. Tip it nose down (y-axis points down) and nose up (y-axis points up)
  4. Flip it upside down (z-axis points down)

Notice the markings on the OpenBCI board (zoomed picture below) that indicate the direction of the accelerometer's axes.

The accelerometer is the small black square towards the bottom.
Note that "X" points right, "Y" points forward, and "Z" comes up out of the board.

Data Files:  The 3-axis accelerometer data was saved to a text file by the Processing GUI.  I analyzed the data using Python. The data and analysis files are available on my EEGHacker repo on GitHub.  If you use this data, be sure to unzip the ZIP file in the SavedData directory!

Analysis:  The specific goals of this analysis are to confirm that the data is well behaved, that the correct axes are responding to the known motions, that the units are correct, and that the scale factors are correct.  I used an IPython Notebook to step through each one of these analyses.  You can see the IPython Notebook here.

Results, Data Continuity: The first thing I did was to look at the data to make sure that it was well behaved.  The most important part of being well behaved is that the data is continuous.  Looking at the packet counter in the data file (a counter which is transmitted by the OpenBCI board), there were no missing data packets.  Excellent.  I did see however that accelerometer data is only included in every 10th or 11th data packet.  Why?  Well, looking at the code on the OpenBCI board, it has configured the accelerometer to only produce data at 25 Hz.  So, compared to the 250 Hz sample rate for the EEG data (which then drives a 250 Hz rate for data packets), we see why we only get acceleration values every 10th or 11th packet.  It makes sense.  Good.

Results, Individual Axes: After ensuring that the data was continuous, I looked at the data values themselves.  I plotted the acceleration values as a function of time.  The plots below show the values recorded from each of the accelerometer's three axes.  As can be seen, the signals clearly reflect the maneuvers defined in my procedure.  Additionally, from these plots, we learn that negative acceleration values result when the accelerometer's axis is pointing down (relative to gravity) and positive values result when the axis is pointing up.  This polarity information is important if you wish to use the accelerometer data to estimate the orientation of the OpenBCI board.

Acceleration Values for the Accelerometer's Three Axes.
The three channels correspond to the X, Y, and Z axes.

Results, Scale Factor:  With the behavior of the 3 axes shown to be reasonable, I then wanted to confirm that the magnitude of the values were correct.  I wanted to make sure that the scale factors used for interpreting the raw values was correct.  The quickest way for me to confirm the scale factor was to compute the magnitude of the 3-axis acceleration vector.  When the device is at rest, the magnitude of the measured acceleration should equal gravity, which is 1.0 G.  As you can see below, the magnitude of our acceleration was generally close to 1.0 G (though often a little high), except when it was moving during its transitions between positions.  This is good.

The magnitude of the 3-axis acceleration vector should equal 1.0 G when at rest.
Ours equals about 1.044 G, which is within the known offset error bounds of the device

When I look very closely at the values, it appears that the typical reading is actually 1.044 G instead of 1.000 G.  There is a 44 mG difference.  Is this unexpected?  Well, yes, it was unexpected at first.  And then I read the datasheet.  Always look at the datasheet.  In this case, it reports that the accelerometer should have a typical offset error of 40 mG per axis.  For a 3-axis device, this could result in sqrt(402 + 402 + 402) = 69 mG of error on my magnitude value.  As a result, my 44 mG value appears to be in-line with the device's advertised performance.  That's satisfying.

Conclusion:  With this test, I confirmed that my accelerometer is sending well-behaved data, with all three axes responding appropriately, with the correct scale factor.  These are good results and I'm pleased. Now it's time to figure out something fun to do with the accelerometer!

Sunday, November 16, 2014

My Kickstarter OpenBCI Arrived!

It's arrived!  It's arrived!  My OpenBCI Kickstarter award has arrived!  And now, the guilty pleasure of unpacking a new piece of tech...


Sure, Joel and Conor did send me an early unit for me to test for them, but yesterday I received my actual purchased unit.  For their Kickstarter back in January, I choose the "OpenBCI Board -- Early Bird Special".  Based on their description, I thought that I'd get just the OpenBCI board.  It turns out that I got quite a bit more!


As you can see in center of the picture above, I got the OpenBCI board (8-bit version) as well as the USB Bluetooth dongle.  That was expected.  Looping around the outside are all the extra pieces that I didn't expect.  Starting from the left side of the picture, I got: a 4xAA battery holder, an OpenBCI sticker and sew-on patch, an OpenBCI T-Shirt, a set of electrode adapters, and two little bags of solderable female headers (for expanding the functionality of the OpenBCI board and of the dongle).  That's some good stuff!

Thanks, OpenBCI!

Sunday, November 2, 2014

Two Brains - One Robot

After my success with sharing the brain-controlled hex bug with Conor and Joel, we brainstormed on how we could make this hack even more fun.  We decided that the main problem with this hack is that only one person gets to participate -- the person driving the robot.  The solution?  Let's hook up multiple people at the same time to control the one robot.  It'll be like that 3-legged race, where you tie your leg to the leg of another person, and then you stumble together in slapstick hilarity until you both get to the finish line.  We are going to do the same thing, but with brain-controlled robots.  Here's how far we've gotten so far...


The Plan:  Our goal is to have multiple people control one robot via their brain waves.  To do this, we aimed to connect multiple people to a single OpenBCI board.  I have never connected multiple people to one EEG system before, so this was pretty exciting for me.  As shown in the figure below, the idea is that each player is responsible for just one of the robot's actions -- one player is responsible for "Turn Left", another for "Turn Right", etc.  Since the robot has four actions (Left, Right, Forward, Fire), we can have up to four players.

The Hexbug robot has four commands (Left, Right, Forward, Fire), so for multi-player fun,
connect four people to one OpenBCI board and work cooperatively!

Commanding the Robot:  In setting up this hack, I wanted to make it as easy as possible for the players to command the robot with their brain waves.  The easiest brain waves to generate and the easiest brain waves to detect are Alpha rhythms (ie, 10 Hz oscillations), specifically the Alpha rhythm that naturally occurs when you close your eyes.  So, with the setup above, we have the computer looking for Alpha waves in each person's EEG signal.  If the computer sees Alpha waves from Player 1, the computer issues a "Turn Left" command to the robot.  If the computer sees Alpha waves from Player 2, it issues a "Forward" command.  And so on...

EEG Setup:  To detect these eyes-closed Alpha waves, we put one electrode on the back of a player's head over the visual cortex (position "O1" in the 10-20 system).  We put another electrode on one ear lobe to act as the EEG reference.  Finally, we put a third electrode on the other ear lobe to act as the EEG Bias.

Individual Reference:  To allow each person to use their own reference electrode, we configured the software on the OpenBCI board to put the ADS1299 EEG chip into per-channel differential mode.  Unlike our normal mode of operation, which uses a common reference electrode via SRB1 or SRB2, this differential mode allows each channel (ie, each player) to have its own reference.  This is what we want!  We simply plug the O1 electrode into the channel's "P" input and the ear lobe reference into the channel's "N" electrode.

Common Bias:  The only tricky part is that we want all four players to be connected to the OpenBCI Bias.  This is tricky because the OpenBCI board does not have four Bias pins.  Well, as you can see below, all it takes is a soldering iron and you can connect a piece of pin header to turn the single Bias pin into four Bias pins.  Now we're hacking!

OpenBCI V3 Board With Extra Pins Soldered to the Bias Output

Connecting the Pieces:   The picture below shows all the connections to the OpenBCI board assuming three players.  On the lower left, we've got three pairs of wires (one pair for each player) plugged into the "P" and "N" inputs of three different channels.  Then, in the upper-left, you see three wires plugged into three of the four new Bias pins.  Finally, in the upper-right, you see five wires that go off to command the hacked Hexbug remote control.

OpenBCI Board with Connections Ready for Three Players

Making It Happen:  Being a rare thing that me and Joel and Conor are all together, it was really fun that we could work together to make this hack happen.  Joel worked the soldering iron to attach the pins and he modified the Arduino code running on the OpenBCI board to enable the per-channel differential mode.  Conor further modified the Arduino code as well as the Processing GUI to enable slower turning of the robot (originally, it was turning WAY too fast).  Then, I modified the Processing GUI to enable Alpha detection on the four individual players.  We did all this in parallel.  I'd never really done group-hacking before.  It was definitely fun.

Conor and Joel working through the details of the connecting the Hexbug remote control.

Testing It:  Once we pulled together all of the pieces, Conor and I began to test the complete setup (see pic below).  After a little tweaking, we got the whole system working, as shown in the video at the top of this post.  It was a group effort that worked out.  Pretty sweet.

Conor and Chip -- Two Brains, One Robot.

Breaking Robots:  So our original vision was to get this hack working so that we could have *two* 4-person teams, with each team controlling their own robot.  Luckily, we had multiple robots and multiple OpenBCI boards, so we thought that we could make it happen.  Unfortunately, as soon as Conor and I made our video, the robots started to break.  They don't like being stuffed in suitcases, I guess.  So, we were left with just one working robot.  Bummer.

Recruiting a Team:  At the AF LabHack, there were lots of folks doing their own hacking.  By the time we got our system working (with the one healthy robot), the other teams were scrambling to get there last results prior to presenting to the group...so we had a tough time recruiting volunteers for being part of a robot-control team.  In the short time we had left, we did get three enthusiastic folks step up.  We got them all equipped with EEG electrodes, tuned the system a bit and let them play!

Our Fine Volunteers.  Three Brains, One Robot.  

No Video:  At this point, we should be presenting a triumphant video.  Unfortunately, we don't have one.  If we did, what you'd see is that two of the three players could easily and repeatably use their eyes-closed Alpha waves to command the robot.  It was cool to see.

No Alpha:  The third player, though, did not have much luck controlling his part of the robot.  At first, I assumed that it was a problem with our system, but after a little debugging, I came to the conclusion that his brain simply wasn't generating eyes-closed Alpha.  He could have been trying too hard (you must be relaxed, without concentrating or being overly focused), or he could have been part of the 11% of the normal, healthy population that simply does not generate Alpha upon closing their eyes [Ref 1].  For these folks, I've got to come up with an alternate robot-control methodology...perhaps by the concentration signature of counting-backwards-by-three.

Next Steps:  The next steps are clear -- I have to get a bunch of people together, hook them up, and enjoy the shenanigans of many brains trying to control a single robot.  Should be fun!

Ref [1]: Gibbs FA, Gibbs EL, Lennox WG. Electroencephalographic classification of epileptic patients and control subjects. Arch Neurol Psychiatry. 1943;50:111–28, as referenced by http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3927247/

Monday, October 27, 2014

Sharing the Brain-Controlled Hex Bug

I got to meet up with Joel and Conor (of OpenBCI) for some hacking over the weekend.  We were at a hackathon sponsored by the Air Force Research Laboratory.  During some of the down-time between hackathon events, we got to do some hacking of our own.  Since Joel and Conor had never seen my brain-controlled hex bug up close, I brought out my stuff.  And nerdy-fun shenanigans ensued.

Brain-Controlled Robots Rule!

The EEG and computer setup was exactly the same as when I did it earlier -- one electrode on the back of the head near O1, the reference electrode on one ear lobe, and the bias electrode on the other electrode.  We used my same blinking movies to induce brain waves at 5 Hz and 7.5 Hz, and we the normal eyes-closed response to induce Alpha waves (which are near 10 Hz).  After playing around with the detection thresholds, we were able to get the system to work for both Joel and Conor.

As a spectator, I really enjoyed the tension and drama provided by Joel's showmanship:


And I also enjoyed the authority of Conor's brain-control skills:


...until his skills failed...


It was really fun to share this hack with Joel and Conor.  Personally, I find that making something move out here in the real world (like this toy robot) is way more fun than simply making traces move on a computer screen.  Sharing hardware hacks is where it's at.  Hardware hacking, FTW!

Saturday, October 25, 2014

Alpha Detection - Comparison Across EEG Recordings

In the previous analysis, I used ROC curves to determine that the a good trade-off between good detection sensitivity and low false alarms occurs when my alpha detection threshold is at 3.55 uVrms and my guard rejection threshold is at 2.55 uVrms.  These threshold values allowed me to successfully detection 56% of the eyes closed data blocks, while having zero false detections.  As follow-up, I promised that I'd look at more additional recordings of my Alpha waves to see if these threshold values were also appropriate for other EEG recordings.  Well, below are the results on 6 recordings.  As you can see, the results are mixed.

Sensitivity and False Alarm Rate for an Alpha Detection Threshold of 3.75 uV and a Guard Rejection Threshold of 2.5.  As you can see, there is a large variation in detection sensitivity and false alarm rate across the six different EEG recordings.
Multiple Files:  The plot above includes data from six EEG recordings.  The data files are specified in my Python analysis code here.  File 1-3 were from Oct 4, File 4 was from Oct 5 and was the subject of all my recent posts, File 5 was from May 31 (from my robot control) and File 6 was from May 8.  As you can see in the plots above, File 4 (the one discussed in my recent posts) was a very high quality recording because it was easy to detect the Alpha waves with high sensitivity (left plot) and with few false alarms (right plot).  By comparison, File 3 and 5 were much more challenging becuase, for the same detection thresholds, we get far lower sensitivity (left) plot) and far higher false alarms (right plot).  How do we decide the best thresholds to use for all files?

Lumped ROC Curve:  If we lump together all 6 sets of data, we can compute the receiver operating characteristic (ROC) curve, as we discussed in my previous post.  The plot below shows the ROC curve for all of the EEG data lumped together.  It shows that, if we were to target a false alarm rate of 1 false alarm per minute, we could achieve an Alpha detection sensitivity where we detection ~40% of all the data blocks where my eyes are closed.  It's not great, but it's not bad, either.

Lumping Together All 6 EEG Recordings, We Can Evaluate the Relationship Between Sensitivity and False Alarms When using Our Alpha Band vs Guard Band Detection Approach.

Detection Thresholds to Use:  If we wish to achieve a target false alarm rate of 1.0 per minute, our ROC analysis also yields the results below, which shows which threshold values to use for our Alpha detection criteria (3.8 uVrms) and for our guard rejection criteria (1.6 uVrms).

Thresholds to Use for Achieving 1.0 False Alarms Per Minute for All Six EEG Files Lumped Together Using the Alpha
+ Guard Discrimination Approach.

Better Detection Performance:  Using these two detection thresholds determined through the lumped data analysis above, we can evaluate the sensitivity and false alarm rate for each individual EEG recording.  These results are shown below.  As expected, File 3 and File 5 still have the lowest sensitivity.  File 2, however, now has the highest false alarm rate.

Detection Sensitivity (Left) and False Alarm Rate (Right) When Using the Detection and Rejection Thresholds That Should Yield an Overall False Alarm Rate of 1.0 Incorrect Detections Per Minute.

Is it Good Enough:  While these plots seem OK enough, is it good enough?  Well, that can only be answered by looking at the detection plots for the individual recordings.  Plots of the six recordings are presented blow, if you really want to see the details...click on any one of them to see a bigger version.  When I look at these figures, I'm feeling pretty good about these detection thresholds.


Files 1,2: 
Files 3,4: 


Files 5,6:


Next Steps:  Having established a good set of detection thresholds, we have two paths forward.  One path would be to continue this analysis to find other types of detection algorithms that, via the ROC curve, might show better detection performance.  Another approach would be to implement these detection threshold values in the real-time GUI to see if it give good performance.  I'm not sure which direction I'll take, but given that the Air Force Hackathon is starting shortly (which I and OpenBCI will be attending), I bet my next step will indeed involve actual hacking.  Let's do it!

Sunday, October 19, 2014

Detected Alpha Waves - ROC Curves

In my previous post, I improved my Alpha detection processing by introducing "guard bands", which allowed me to reject most of the non-Alpha activity that looked like Alpha waves.  In finding the best thresholds to use for detecting and rejection, I had to find the right balance between keeping good sensitivity (ie, detecting most of the true Alpha activity) while minimizing false alarms (ie, avoiding detecting non-Alpha activity as if it were Alpha).  Finding the right balance can be tough and can involve lots of guesswork.  In this post, I'll show how that process can be formalized...by making a ROC curve!

Example EEG data (top) and some example processing (middle and botom).  True Alpha activity is limited to the "Eyes Closed" period.  I used the amplitude in the Alpha band (blue trace) to detect the Alpha activity.  I used the amplitude in the Guard Band (green trace) to reject activity that is not specific to the Alpha band.  For my given detection threshold (Alpha > 3.5 uVrms) and rejection threshold (Guard > 2.5 uV), the red circles show when my algorithm detects Alpha activity.

Background:  The plot above shows the final result from the last post.  It shows the raw EEG data in the spectrogram at the top, as recorded from the back of my head (O1 with earlobe reference).  As you can see, when my eyes are closed, the horizontal stripe of color shows my Alpha rhythm.  The middle plot shows the amplitude of the EEG signal in the Guard band (I picked 3-6.5 Hz and 13-18 Hz) and the bottom plot shows the amplitude of the EEG signal in the Alpha band (7.5-11.5 Hz).  You can also see that, by eye, I picked a Guard rejection threshold of 2.5 uVrms and an Alpha detection threshold of 3.5 uVrms.  The red circles show the "detections" when the EEG signal satisfies the Alpha detection threshold (ie, is > 3.5 uVrms) and avoids the Guard rejection threshold (ie, is < 2.5 uVrms).  This works algorithm works way better than the Alpha-only detection approach that I used in my first attempt.  Theis Alpha+Guard algorithm isn't perfect, however.  You see that I still have a pair of false detections around 84 seconds.

Choosing the Thresholds, By Eye:  One of the key decisions in this algorithm is what value to use for the Alpha detection threshold and what value to use for the Guard rejection threshold.  Initially, I choose them by eye from the plots above and then used trial-and-error to tune them further.

Alpha-Guard Scatter Plot:  Another approach for choosing these thresholds is to use a dedicated plot such as the one below.  Here, I take the exact same Alpha and Guard values as plotted above and I re-plot the data.  But, instead of plotting the data as a smooth trace as a function of time, I plot the the data as a scatter plot of the Alpha and Guard value for each moment of time.  For each moment in time when my eyes are closed (which mean that true Alpha waves are likely present), I plotted the data point in blue.  For each data point outside of this time period (so Alpha are likely not present), I plotted the values in red.

For each moment in time, I measure the amplitude of the EEG signal in both the Alpha band and the Guard band.  Each dot in this plot represents the Alpha and Guard values for one moment in time.  The blue dots are from the time when my eyes were closed, so they should contain Alpha activity.  You can see how the Alpha threshold (black line) and Guard threshold (red line) can be used to try to discriminate between the two types of activity.

Choosing the Thresholds, Graphically:  With the Alpha-Guard scatter plot shown above, it is much easier to visually move the Alpha threshold (the horizontal black line) up-and-down, or to move the Guard threshold (the vertical red line) left-and-right.  Any points in the upper-left quadrant satisfy our detection rules (measured Alpha > Alpha Threshold and measured Guard < Guard Threshold).  The thresholds are currently shown at the values from my previous post -- Alpha Threshold at 3.5 uVrms and Guard Threshold at 2.5 uVrms.  We see that the vast majority of the detections are blue dots, which is good because that is the eyes-closed activity.  We also see the two false detections as the two red circles.  With this plot, we clearly see that if we raise our Alpha band a little bit, we could eliminate those two false detections...at the expense of losing more of the desired (blue dot) detections.  Again, we see that it's a balance between high sensitivity and a low rate of false alarms.

Try All Alpha and Guard Threshold Values:  To get a better picture of the effect that these two threshold values have on the sensitivity and false alarm rate, we can have the computer brute-force reprocess the data trying a wide range of threshold values.  By stepping through all the combinations of Alpha and Guard thresholds, the computer forms a dense table of the number of correct detections and the number of incorrect detections for our given EEG recording.  We can then use a contour plot to visualize this dense table of values.  These are shown below.

Effect on Sensitivity: The first plot below shows how the Alpha and Guard thresholds affect our detection sensitivity (the fraction of eyes-closed EEG activity was correctly detected).  The black "x" shows our current threshold values (3.5 uV for Alpha, and 2.5 uV for the Guard).  At the end of my previous post, I said that this achieved a sensitivity of 65%, which agrees with the plot below.  If I want higher sensitivity, this plot clearly shows that the ideal is toward the upper-left, which means lowering my Alpha detection threshold and raising my Guard rejection threshold.

After trying all combinations of Alpha and Guard thresholds (as shown by the black dots), we can see how the threshold values affect our sensitivity in detecting the true Alpha activity.  The highest sensitivity is in the top-left of this plot, meaning we want the lowest threshold for Alpha detection and the highest threshold for Guard rejection.

Effect on False Alarms:  The desire for high sensitivity, however, must be balanced by the need for a low false alarm rate.  The plot below shows how the threshold values affect the number of false alarms for this data set.  As before, the black "x" shows our current threshold values: 3.5 uV for Alpha, and 2.5 uV for the Guard.  In my previous post, I said that this point resulted in 2 false alarms, and the plot below agrees with that finding.  This plot says that if we want to further lower the false alarm rate, we need to move down and to the right, which is to lower our Guard rejection threshold and to raise the Alpha detection threshold.  This is exactly the opposite direction that was needed to increase our sensitivity.  Again, we're seeing that it is a struggle to optimize sensitivity versus false alarms

After trying all combinations of Alpha and Guard thresholds (as shown by the black dots), we can see how the threshold values affect the number of  false alarms (incorrect detection of non-Alpha activity).  The fewest false alarms are seen in the bottom-right of this plot, meaning we want the highest threshold for Alpha detection and the lowest threshold for Guard rejection.

Best Sensitivity for a Given False Alarm Rate:  Given that it is a trade between sensitivity and false alarm rate, how does one choose where to set the detection and rejection thresholds?  One approach would be to simply start by picking a number of false alarms that would seem to be be acceptable.  Then, one can search through the threshold values to find the combination that achieves the best sensitivity at the given false alarm rate.  That's what I did to make the plot below.  I used the dense grid of values before and directly plotted the sensitivity and false alarm count for every combination of threshold values.  Clearly, there is an upper limit running across this plot.  So, I added the green line to define the maximum sensitivity (for this detection algorithm) that can be achieved for any given false alarm rate.

A plot of the sensitivity versus number of false alarms for every combination of threshold values analyzed earlier.  There is clearly a maximum sensitivity value that occurs for each given false alarm rate.  This is Receiver Operating Characteristic (ROC) curve for this detection algorithm.
ROC Curve:  The green line in the plot above is a very useful tool for understanding the performance of the detection algorithm.  It is so useful that it has a name -- it is the "receiver operating characteristic" (ROC) curve for this detection algorithm.  The name is kinda funny because it originated in World War II during the development of radar.  In radar, one is trying to detect pulses of radio energy that have been reflected off distant aircraft.  When comparing different radar receivers, one can always appear to be more sensitive, though at the cost of increasing the rate of false alarms.  So, they developed the ROC curve as a way to make it easier to understand, compare, and optimize detection systems.  It was such an effective tool, that it has became a core technique in the general field signal detection and classification.

Comparing Alpha Detection Systems:  As an example of how useful it is in comparing detection "systems", the ROC curves below show the performance of "Alpha+Guard" algorithm versus my original "Alpha Only" detection algorithm.  As can be seen, the ROC curve for the Alpha+Guard algorithm is higher at point.  It is superior at any allowed false alarm rate.  Alpha+Guard is clearly superior.  Interestingly, this plot also exposes how the "Alpha Only" algorithm was never able to have less than 10 false alarms in this EEG recording.  If one cares about a low false alarm rate (which I certainly do), the ROC curve exposes this critical information.

ROC curves for two different detection algorithms: "Alpha+Guard" detects in the Alpha band and rejects based on the Guard bands, while "Alpha Only" does not include the rejection based on the Guard band.  The ROC curve for "Alpha+Guard" is higher at all false alarm rates, thereby showing its performance is superior.

Selecting the Operating Point:  The ROC curve is an excellent visualization for deciding which detection algorithm to use.  It is also helpful for understanding where the steepest trade-off exists between sensitivity and false alarms.  For the Alpha+Guard algorithm (unlike the Alpha Only algorithm), we see that the curve is generally quite flat, so there is not a severe penalty in targeting a very low false alarm rate.  As an experiment, let's target zero false alarms.  The plot above tells me that, at zero false alarms, it is possible to achieve a sensitivity of 56%.  Fine.  But what threshold values should I use to get this performance?  The ROC curve itself does not tell me that information.

Finding the Threshold Values:  When I made the ROC curve, I built it up by plotting the data generated through a brute-force assessment of all pairs of Alpha and Guard threshold values. These were the green circles shown two plots ago.  The ROC curve came from a subset of those green circles.  So, I can go back and find the threshold values that correspond to the green circles that just fall on the ROC curve.  These threshold values would be the optimal threshold values for a given false alarm rate.  The figure below plots these optimal threshold values as a function of allowed false alarms (ie, allowed number of incorrect detections).  If we are targeting zero false alarms, this plot says that we should choose an Alpha detection threshold of about 3.75 uV and a Guard rejection threshold of about 2.55 uV.

For a given number of allowed incorrect detections (ie, false alarms), this plot shows the Alpha and Guard threshold values that result in the highest sensitivity (ie, most number of correctly identified Alpha activity).  As with all figures in this analysis, the curves above are derived from a single EEG recording, which is why the curves are noisy.

Re-Process the EEG Data:  If we use these two threshold values and reprocess our EEG data, we get the detection performance seen below.  Again, the top plot is the raw EEG recording.  The middle plot shows the EEG amplitude in the Guard band along with (in red) the Guard rejection threshold.  The bottom plot shows the EEG amplitude in the Alpha band along with (in black) the Alpha detection threshold.  The red circles show those points that satisfy both thresholds -- the are the "detections" resulting from our algorithm.  You'll note that there are only detections when my eyes are closed -- there are no false detections.  This looks really good!  I think that I'm getting close to being able to deploy this into my OpenBCI GUI for real-time operation in detecting Alpha waves.

Detection Performance Using the Detection Thresholds for Zero False Alarms.  The detections (rec circles) resulting from this approach look pretty good!

Summary:  In this post I introduced a number of new ways of analyzing and plotting EEG data so that we can optimize the performance of our Alpha detection algorithm.  Specifically, I showed how the ROC curve is a great way of visualizing system performance in a way that smoothly handles the inherent trade-off between sensitivity and false alarms.  I showed how the ROC curve is a simple way to quickly see how one detection algorithm is better than another ("Alpha+Guard" is clearly superior to "Alpha Only") and to quickly see that there is little penalty for driving to an even lower false alarm rate.

Next Steps:  Up until now, we've just been looking at one EEG recording.  Therefore, all of our conclusions on which algorithm to use and which detection thresholds to use have been based upon a very small amount of data.  The results seen here might not generalize well, which means that my algorithm might not perform well when faced with future data.  To address this possibility, my next step is to bring in some of my other EEG recordings, all of which are noisier and more challenging.  With the additional data, we can better optimize our algorithm and have greater confidence that it'll work well on future data that has not been part of its training.

Follow Up:  I examine the performance on six different EEG recordings here.

Wednesday, October 15, 2014

Detecting Alpha Waves - Guard Bands

In my previous post, I discussed a simple algorithm for detecting Alpha rhythms: (1) use an FFT spectrum to measure the EEG amplitude in the Alpha band and (2) compare this value to a fixed detection threshold to decide if Alpha are present.  As shown in the figure below, this approach yields good detection sensitivity (it correctly flags 66% of the eyes-closed data blocks) and a reasonably low number false alarms (it incorrectly flags 15 data blocks).  While this is good, I think that I can do better.  Let's talk about how...

Example EEG data (top) showing Alpha rhythms when my eyes are closed.  At each time slice, I measure the peak of the spectrum in the Alpha band (7.5-11.5 Hz), which yields the blue trace on the bottom.  By looking for any value above 3.5 uVrms, we are able to detect the presence of Alpha waves (as indicated by the red circles)

Alpha Band Detection is Not Specific Enough:  In the simple Alpha band detection algorithm discussed above, we are sensitive to any signal with lots of energy in the Alpha (7.5-11.5 Hz) band.  The problem is that there are signals besides Alpha rhythms that have energy in the Alpha band.  For example, the bottom plot below shows the spectrum (black line) for a segment of eyes-closed Alpha waves.  The plot  also shows the spectrum (red line) for a segment of "other" activity that is no an Alpha rhythm (it is probably motion artifact from the EEG lead wires).  As can be seen, both spectra show substantial energy in the Alpha band, and so they would both be flagged as "Alpha!" using my simple threshold detection approach.  For the segment of "other" activity, this would be a false alarm.  I don't want that.  I want to improve my algorithm to reject this kind of false alarm.

Two spectra: (1) a segment of eyes-closed Alpha rhythm that I do want to detect and (2) a segment of "other" EEG activity that I do not want to detect.  Both, however, show a high amplitude in the Alpha band.  So, my original detection rule that is simply based on the Alpha amplitude would not reject the "other" activity.

Introduce "Guard" Bands:  One way of distinguishing between the two example spectra above is to introduce "guard" bands on either side of the Alpha band.  The idea is that we measure the signal amplitude both in the Alpha band and in the guard bands.  Based on the plots above, we know that true eyes-closed Alpha activity will not show much energy in the guard bands whereas the confusing "other" activity can be rejected because it does show energy in the guard bands.

If we measure the mean EEG amplitude in the guard bands, as well as in the Alpha band, we can distinguish between the two signals.  True Alpha rhythms will not have much energy in the guard bands whereas most of our confusing "other" activity will show substantial energy in the guard bands.

Evaluating the Guard Amplitude:  To quantify the amplitude in the guard bands, I simply take the average of all the spectrum values that fall within our two guard bands (3-6.5 Hz and 13-18 Hz).  When I apply do this for our EEG data, I get the green trace shown in the middle plot below.  As you can see, it stays low during all of the legitimate eyes-closed Alpha activity and it jumps high only during the confusing other activity.  This looks promising!


Combined Detection Rules:  Based on this graphs above of the guard amplitude (green line) and of the Alpha amplitude (blue line), it looks like a good combination of rules would be to look for points where the Alpha amplitude is greater than 3.5 uVrms and, simultaneously, where the guard amplitude is less than 2.5 uVrms.  When I apply these detection rules, I get the red circles shown in the figure above.  Looks pretty good!  You'll note that the addition of the guard band has successfully rejected the false alarms that we had been getting at t=58, t=77, and t=123.  This is exactly what I was hoping for.

Quantifying the Improvement:  Compared to yesterday's results (tabulated below), this new detection algorithm obtains nearly the same sensitivity (65% vs 66%) with a greatly reduced number of false alarms (2 vs 15).  This is definitely an improvement in my Alpha detecting algorithm.
      
Guard       N_TRUE      N_FALSE
None        101 (66%)   15
2.5 uVrms   100 (65%)   2


Moving Forward:  For this example EEG recording, I am satisfied with the performance of this algorithm.  It would give me quite reliable performance while still being nicely sensitive.  This EEG recording was pretty "clean", however -- its Alpha was pretty strong and there was not too much confusing "other" activity.  I have other EEG recordings that are more difficult.  Next time, we'll look at those harder recordings, you'll see that even the combined Alpha+Guard algorithm is insufficient, and I'll discuss yet another extension (hopefully an improvement!) on this detection approach.

Follow-Up:  I further optimize this algorithm by using ROC curves to attack, head-on, the trade-off between sensitivity and false alarms.  Check it out here.