|Sensitivity and False Alarm Rate for an Alpha Detection Threshold of 3.75 uV and a Guard Rejection Threshold of 2.5. As you can see, there is a large variation in detection sensitivity and false alarm rate across the six different EEG recordings.|
Lumped ROC Curve: If we lump together all 6 sets of data, we can compute the receiver operating characteristic (ROC) curve, as we discussed in my previous post. The plot below shows the ROC curve for all of the EEG data lumped together. It shows that, if we were to target a false alarm rate of 1 false alarm per minute, we could achieve an Alpha detection sensitivity where we detection ~40% of all the data blocks where my eyes are closed. It's not great, but it's not bad, either.
|Lumping Together All 6 EEG Recordings, We Can Evaluate the Relationship Between Sensitivity and False Alarms When using Our Alpha Band vs Guard Band Detection Approach.|
Detection Thresholds to Use: If we wish to achieve a target false alarm rate of 1.0 per minute, our ROC analysis also yields the results below, which shows which threshold values to use for our Alpha detection criteria (3.8 uVrms) and for our guard rejection criteria (1.6 uVrms).
|Thresholds to Use for Achieving 1.0 False Alarms Per Minute for All Six EEG Files Lumped Together Using the Alpha|
+ Guard Discrimination Approach.
Better Detection Performance: Using these two detection thresholds determined through the lumped data analysis above, we can evaluate the sensitivity and false alarm rate for each individual EEG recording. These results are shown below. As expected, File 3 and File 5 still have the lowest sensitivity. File 2, however, now has the highest false alarm rate.
|Detection Sensitivity (Left) and False Alarm Rate (Right) When Using the Detection and Rejection Thresholds That Should Yield an Overall False Alarm Rate of 1.0 Incorrect Detections Per Minute.|
Is it Good Enough: While these plots seem OK enough, is it good enough? Well, that can only be answered by looking at the detection plots for the individual recordings. Plots of the six recordings are presented blow, if you really want to see the details...click on any one of them to see a bigger version. When I look at these figures, I'm feeling pretty good about these detection thresholds.
Next Steps: Having established a good set of detection thresholds, we have two paths forward. One path would be to continue this analysis to find other types of detection algorithms that, via the ROC curve, might show better detection performance. Another approach would be to implement these detection threshold values in the real-time GUI to see if it give good performance. I'm not sure which direction I'll take, but given that the Air Force Hackathon is starting shortly (which I and OpenBCI will be attending), I bet my next step will indeed involve actual hacking. Let's do it!