Thread: Electrooculograh (EOG) from webcam - research assistant needed

Hello there- I'd like to help, but I have a set of questions, bare with me.

1) How close does the camera have to be to your face so as to recognize REM?

2) Will light be necessary? It's unlikely I could sleep with significant light

3) Where do I get these & on awakening, how do I get/ what do I do with the data from the night, & get it to you?

Originally Posted by IAmCoder

Lucid Scribe along with the Halovision plugin

All the best.

I'll be checking this thread!

Elite!

1) No idea, that is one of the things we will find with our research. You can experiment while awake by moving your eyes and seeing if it still has an effect. It will also depend on the settings.

2) At least a night light will be necessary, unless you have an infrared camera.

3) You can download them from lucidcode.com or here: Lucid Scribe 1.2.5 and Halovision 0.9.0. Once you have it running, you should see a folder structure with the log files - you can pm or email them to me.

Thanks!

Having trouble. I've downloaded both. I am getting an error "Invalid Drive: E" for the Lucid Scribe when I try to finish the installation and am getting prompted for what program to open Halovision with.

I'll be trying to get it to run!

Hey IamCoder,

Just posting a picture here that the program works just fine (I think?)
Anyway, seems it's identifying my right eye more than my left and more on my chest. I'll fiddle around with the settings, and it's probably because of the lighting, but just wanted to make sure that I'm at least in the right direction.




I'll go through finding my account on the LSD site, it's been a while since the FIELD test.

Very cool. It is picking up the movement around your chest because your shirt is too baggy - you need to suit up!

You are on the right track - can you get it to play the audio track by blinking every two seconds five times in a row? You might have more luck if you lean forward so that your face is the main thing on the screen and maybe at night when the lighting from the window is less bright.

bro, are you installing it to the E: drive? Can you try installing it to the C: drive. It shouldn't take up more than a dozen MB. The data files / logs are stored in the My Documents folder.

Things went well for a bit, but then it seemed nothing picked up from 10:59 AM to 12 something PM in the afternoon. I wanted to see if taking a nap would be better because I usually recall more intense dreams during the late morning and early afternoon, but there didn't seem to be much results other than it recording CPU.

I had the Laptop very close to my face and made sure I laid on one side, but it didn't seem to catch more activity in my eyes. I don't think my Laptop webcam is that bad to not let it pick up on the eyes, or maybe it could just be the settings I used overall that messed things up. Or again, probably due to the lighting in the afternoon, but I did my best to darken the area as much as I can, seems I'll have to find a nightlight tonight and prepare for a WBTB after a few months of not doing so.

I figured I post the images instead of e-mailing or pming the results in a file to you. I'll be more than happy to send the files to you, but no significant results were apparent here:

This first image involved testing things around a bit before actually taking a nap, everything else...dormant.

Thanks for the test run! The audio tracks at 09:50 - were they intentional from blinking or false positives from moving around? Are you able to make the Halovision channel move while lying down with your eyes closed, by moving your eyes in an exaggerated fashion behind closed lids?

You might need to increase the amplification and decrease the pixel threshold and pixels in a row to get it to do that. It is a balancing act to configure it so that it isn't too sensitive and picks up background noise, but still picks up the eye movements. Every room and lighting condition will need custom configuration until I add some more intelligence to the algorithm.

The camera feed stops if you minimize the webcam screen. Maybe that is what happened?

The movement on the Halovision channel might also be so much less than the CPU channel that it dwarfs it to the point where we can't see it. If you are still up for another test during your next nap, can you run it with the CPU channel disabled? Otherwise you will have to browse through the logs minute by minute to see if the Halovision channel picked up any movements.

Originally Posted by IAmCoder

Thanks for the test run! The audio tracks at 09:50 - were they intentional from blinking or false positives from moving around? Are you able to make the Halovision channel move while lying down with your eyes closed, by moving your eyes in an exaggerated fashion behind closed lids?

Yeah, those were intentional for the sake of testing to see if it can also pick up the eyes after I laid down.

You might need to increase the amplification and decrease the pixel threshold and pixels in a row to get it to do that. It is a balancing act to configure it so that it isn't too sensitive and picks up background noise, but still picks up the eye movements. Every room and lighting condition will need custom configuration until I add some more intelligence to the algorithm.

Definitely will keep in that in mind, I had all three elements as high as I could get it, major facepalm on my end.

The camera feed stops if you minimize the webcam screen. Maybe that is what happened?

Oh no, the images were what happened after, not screencaps during or anything of that sort. I had the webcam screen on the whole time during the nap.

The movement on the Halovision channel might also be so much less than the CPU channel that it dwarfs it to the point where we can't see it. If you are still up for another test during your next nap, can you run it with the CPU channel disabled? Otherwise you will have to browse through the logs minute by minute to see if the Halovision channel picked up any movements.

Yeah, figured the CPU would be a detriment to the overall results, will keep that in mind, and thank you so much for the advice and suggestions. I probably won't be taking a nap today since I'll have to take care of some things for the rest of the day. However, I do intend to do the classical 4 to 4.5 hour prior sleep =>>WBTB ==> ??? ==> Whatever happens, happens.

For the other forum I mentioned to you about, I'll be getting to that as quick as I can, but I figured I would just be the guinea pig first so that I won't leave those other members confused and such if any would want to try it out. Again, thank you for the time you're putting into this, and will post back with more results hopefully tomorrow!

If these can be interfaced with the software (which I would be able to with my basic knowledge of electronics and professional knowledge of programming): h t t p : / / w w w . dfrobot.com/image/cache/data/SEN0018/62-125-600x600.jpg
would that be of help? If motion by this device is detected, its means a large shift in infrared radiation is detected. Only moving your eyes will not trigger the signal on such a motion sensor. When you enter a room, it will signal you, but if you keep very still, it won't sense you until you move again.
So if the application detects certain signals, but the motion sensor detects them as well, they can probably be discarded as it probably isn't REM.

Let me know if your interested, and maybe I could add some code to the project and construct a simple PCB with USB interface.

Originally Posted by Linkzelda

Again, thank you for the time you're putting into this, and will post back with more results hopefully tomorrow!

Sure, and thank you for testing and spreading the word. I am glad to hear that you are able to consciously trigger the anthems by blinking - that is much harder with some of the other plugins.

Originally Posted by Devoney

Let me know if your interested, and maybe I could add some code to the project and construct a simple PCB with USB interface.

The halovision plugin has a setting for motion detection, where it will trigger the audio track to play whenever any motion is detected. The scale of motion that isn't from REM and the pattern should be easy enough to distinguish. But thanks for the offer! Do you suppose the infrared sensors would detect eye movement when wrapped in a headband and placed right by the temple? That might be worth investigating. What other sensors can you think of that might be interesting to record during sleep? I have written plugins for a handful: https://github.com/lucidcode - it is real easy to send the values to Lucid Scribe through the interface: Lucid Scribe – Develop Plugins.

Originally Posted by IAmCoder

The halovision plugin has a setting for motion detection, where it will trigger the audio track to play whenever any motion is detected

To much enthusiasm I suppose, I should have done my research.

You have listed quiet a few on Github already. Are you a professional programmer as well ? I can't think of any sensors now, but I'll keep it in mind maybe something pops up.

About that particular infrared sensor, I doubt it, I think it needs a shift in intensity if infrared radion, I think the movement of the eye doesn't cause that effect. But I've got a basic idea to test it. Isolate the sensor in a tube and aim the tube at my eye and see whether it responds. I'll give it a shot next time I went to the hardware store. I think I don't have that kind of tubing in stock.

Took a nap that was a little bit more than an hour, things are a lot better, thanks for the suggestion on reducing the pixel threshold and all that.:




More questions for you IamCoder:

1. Because I know you can only get so much from empirical data, would you like someone to give a recording of the program at work so that you'll be able to have a control variable where you can pick out what could potentially be a REM phenomenon? I can PM you the video files of the recording for future sessions and give you certain time points of where there were higher activities with eye movement. I wondered that you didn't want to have people expose themselves that much, but for me, there's not really much for me to worry about, it'll just be a recording of just my eyes and face with the interface you made that tracks it.

2. Please bear with me on this question here, but am I able to still upload the LSD (Lucid Scribe Data) on the LSD website of yours? When I try to do so, it's incompatible. I can e-mail it to you, but I don't see your contact information on both sites you own, can you PM me it if possible? Thanks in advance.

Anyway, will do a recording after a WBTB tonight (hopefully I can persevere and stay up a bit longer compared to my last attempt).

Devoney: Yip. I write info sec code that connects to FBI databases and DHS and DOD smart cards when I'm not hacking consciousness.

Linkzelda: 1. A recording sure would help, thanks. 2. I have hopefully fixed the export function - when I added the Halovision checkboxes, I made the hardware tag list too big so the preview button disappeared. Let me know if you still can't upload from the app. I have sent you my email address. Good luck with the WBTB!

Sent an e-mail with the recording and the logs. I'm not sure if I can do another attempt tonight because of an event I may or may not have to go to. Never mind, change of plans.

Noticed that I was able to get into a dream-like state, but wasn't able to recall the dream very well, but the others before the attempt I seemed to have done decently enough on. Hopefully I can get a recording where there's some signs that I'm going through the REM phenomenon.

Thanks. The phenomenon at 1:19:14 is pretty interesting - it picks up the movement a few seconds before you turn.

Hey IAmCoder, I apologize for not being active recently. I have another recording that I'll e-mail to you later on with the logs. I managed to get a lucid dream during that recording, but from what I recalled, I just remember a few eye movements.

Other than that, I've been trying to make sure I can increase the chances of getting a lucid dream instead of giving you logs that may be just background noises. Lately I've been getting all sorts of lucid dreams, and they occurred when I assumed I shouldn't do a recording for those days. Dang it.

I've been having a consistent sleeping schedule lately, and wanted to give a few days to wake up naturally in the middle of the night or early in the morning where REM periods would be more probable for me. As for the laptop issue, as long as I have the cooling fan, things should be alright, but the most I can get it is maybe 90 minutes before the computer starts hibernating. I don't think I need to record 4 hours of me going to sleep, I could, but I'll just have to make use of what I have now.

Now worries. Whew, I was starting to worry that the recording fried your machine.

Congrats on the lucid dream. Can't believe you caught one on record so quick! Look forward to the logs.

Sent you recording #3, I think it was about 90 minutes in general. About 43:39 I believe is when I saw my eyebrows moving for a bit. Other than that, getting a little better at staying on one side for a decent amount of time.

I saw your eyebrows moving there. Very cool. The lighting conditions were perfect. Do you remember having a dream during the recording? Other than the two eyebrow twitches that you pointed out, I couldn't find any other patterns indicative of REM sleep.

I couldn't recall any dreams at the time for that recording, I believe this was done early afternoon. Still doing my best to do a WBTB and actually staying up for a bit. Glad that the lighting conditions were ideal for you, I just hope I can make things ideal for a darker environment during 2-4 AM.

Check it, I have the first draft of the Android version on Google Play, for when you are on the go and need a mobile sleep research lab with an electrooculogram:
lucidcode.LucidScribe.HaloVision!

The time has come to wake this thread from cryogenic sleep with an update on the tech...

Back in 2014 Highlander strapped a webcam with the algorithm to his head:








Throughout 2015, many headbands and cameras later, I miniaturized the setup and made it work in the dark with night-vision:







And now... drumroll... it works with a match-box sized camera that still picks up eye movements when placed on the bedside table:



Lucid Scribe now has a checkbox to record video data when eye movements are detected. Here is a close-up of my eyes wandering around in dreamland:



And my wife sneaking in a dream during her beauty sleep:



After having slept with hundreds of electrodes and sensors, this method is by far the most comfortable, accurate and verifiable way of detecting dreams at home. Of course only half the time: when facing the camera... but no issues with signal degradation from dry skin or sensors slipping out of place.

A couple of followers have reproduced the sleep lab built from a Rapsberry Pi and NoIR camera and I have shipped at least one! So if you have a pi idling away at night: join the exploration!

I am currently making it talk to Android and iOS phones via Bluetooth and exploring a smart mirror version that has a touch screen with the video feed and perhaps an alarm clock interface.