Alpha Detection - Comparison Across EEG Recordings

In the previous analysis, I used ROC curves to determine that the a good trade-off between good detection sensitivity and low false alarms occurs when my alpha detection threshold is at 3.55 uVrms and my guard rejection threshold is at 2.55 uVrms.  These threshold values allowed me to successfully detection 56% of the eyes closed data blocks, while having zero false detections.  As follow-up, I promised that I'd look at more additional recordings of my Alpha waves to see if these threshold values were also appropriate for other EEG recordings.  Well, below are the results on 6 recordings.  As you can see, the results are mixed.

Sensitivity and False Alarm Rate for an Alpha Detection Threshold of 3.75 uV and a Guard Rejection Threshold of 2.5.  As you can see, there is a large variation in detection sensitivity and false alarm rate across the six different EEG recordings.

Multiple Files:  The plot above includes data from six EEG recordings.  The data files are specified in my Python analysis code here.  File 1-3 were from Oct 4, File 4 was from Oct 5 and was the subject of all my recent posts, File 5 was from May 31 (from my robot control) and File 6 was from May 8.  As you can see in the plots above, File 4 (the one discussed in my recent posts) was a very high quality recording because it was easy to detect the Alpha waves with high sensitivity (left plot) and with few false alarms (right plot).  By comparison, File 3 and 5 were much more challenging becuase, for the same detection thresholds, we get far lower sensitivity (left) plot) and far higher false alarms (right plot).  How do we decide the best thresholds to use for all files?

Lumped ROC Curve:  If we lump together all 6 sets of data, we can compute the receiver operating characteristic (ROC) curve, as we discussed in my previous post.  The plot below shows the ROC curve for all of the EEG data lumped together.  It shows that, if we were to target a false alarm rate of 1 false alarm per minute, we could achieve an Alpha detection sensitivity where we detection ~40% of all the data blocks where my eyes are closed.  It's not great, but it's not bad, either.

Lumping Together All 6 EEG Recordings, We Can Evaluate the Relationship Between Sensitivity and False Alarms When using Our Alpha Band vs Guard Band Detection Approach.

Detection Thresholds to Use:  If we wish to achieve a target false alarm rate of 1.0 per minute, our ROC analysis also yields the results below, which shows which threshold values to use for our Alpha detection criteria (3.8 uVrms) and for our guard rejection criteria (1.6 uVrms).

Thresholds to Use for Achieving 1.0 False Alarms Per Minute for All Six EEG Files Lumped Together Using the Alpha
+ Guard Discrimination Approach.

Better Detection Performance:  Using these two detection thresholds determined through the lumped data analysis above, we can evaluate the sensitivity and false alarm rate for each individual EEG recording.  These results are shown below.  As expected, File 3 and File 5 still have the lowest sensitivity.  File 2, however, now has the highest false alarm rate.

Detection Sensitivity (Left) and False Alarm Rate (Right) When Using the Detection and Rejection Thresholds That Should Yield an Overall False Alarm Rate of 1.0 Incorrect Detections Per Minute.

Is it Good Enough:  While these plots seem OK enough, is it good enough?  Well, that can only be answered by looking at the detection plots for the individual recordings.  Plots of the six recordings are presented blow, if you really want to see the details...click on any one of them to see a bigger version.  When I look at these figures, I'm feeling pretty good about these detection thresholds.

Files 1,2: 

Files 3,4: 

Files 5,6:

Next Steps:  Having established a good set of detection thresholds, we have two paths forward.  One path would be to continue this analysis to find other types of detection algorithms that, via the ROC curve, might show better detection performance.  Another approach would be to implement these detection threshold values in the real-time GUI to see if it give good performance.  I'm not sure which direction I'll take, but given that the Air Force Hackathon is starting shortly (which I and OpenBCI will be attending), I bet my next step will indeed involve actual hacking.  Let's do it!