Analyzing IR readings to recognize REM sleep

For some time, I've been trying to build my own REM-detecting dream mask using the traditional "IR LED + IR sensor pointed at eyelid" technique that most such masks use. After many failed attempts, I've finally got something that's fairly comfortable and seems to collect fairly good data. My problem now is trying to use the data to figure out when I'm in REM.

I've tried a variety of methods - calculating standard deviations, counting peaks, and high-pass filters are my most recent attempts - but I can't find anything that reliably says "this part is a dream." Some of my attempts have been good enough for a human to identify the REM stages, but I'm no closer to figuring out how to train a computer to identify REM. I've been Googling with every search parameter I can think of, but I've found only a single detailed description of a REM-detecting algorithm, and while it's an improvement over my existing attempts it still isn't enough.

I'd be grateful if anyone could provide some pointers, guidance, or assistance in figuring this stuff out. I'll be happy to provide any details or data that I can.

(Mods, if you think this is Research or Lucid Aid material, I won't object; but I put it in Science and Math since I think I'm looking for help with statistical analysis)

Well I have never done this before so I have no idea what sort of data you might get. If you compare data from a person sleeping but not in REM and one of someone in REM are there major differences? Seems like the key to getting a computer to identify REM is just comparing the data and finding some variable that is vastly different in the two states.

Unfortunately, I don't have any way to say "this data is from REM" and "this data is from nREM." All I can do is attempt different manipulations of the data and try to find one that produces data that "looks right" - i.e. that flags REM segments roughly 90 minutes apart. I've tried to look over the data visually, but there's so much of it I get lost in it :(

Here's the whole night from a couple of nights ago. The red line labelled "Buttons" counts how many times the button on the mask has been pressed. I press it whenever I wake up and occasionally if I can't get to sleep, so I generally figure that I need to ignore a few minutes of data any time the Button line jumps.
Attachment 7849

Just looking at it, I can't make heads or tails out of it. The only obvious thing from the graph is when I roll over in my sleep.

I know it's picking me up well enough. The first thing I always do when I turn it on is a sort of "calibration" - it's meaningless to the software, but I like to see it on the graph. Here's a magnified view of the very start of the night:
Attachment 7850

You can see me looking straight ahead, then left, then straight, then right, then straight. The green line is the raw sensor data (this happens to be from my left eye, but I log data from both eyes), and the violet line is a moving average. I take 8 readings per second, and this moving average has a window of 9 (4 readings on either side of the data point in question). When I'm testing it while awake, the readings hold steady as long as I hold my eyes steady. However, while asleep, something odd happens:
Attachment 7851

Those peaks are about 3.5 seconds apart. I can pull up almost any part of the graph on almost any night and find something similar. It's not always as "clean" as this section (sometimes the highs and lows vary considerably from peak to peak), but this wave is always there (and is usually in the neighbourhood of 4 seconds). I can only assume it has to do with my breathing, but I can't figure out how - it's not like I hold my breath for 20 seconds while doing my "calibration," and I do it with my head on my pillow.

The technique I alluded to in my initial post comes from this link and involves "calculating the mean value, then summing the absolute values of the differences of the individual readings and the mean," and then scanning those results for groups of values higher than a given threshold. Applying that algorithm to my data yields the following graph (sorry, I have no idea why DV is shrinking this image):
Attachment 7854

Obviously, that algorithm is nowhere near perfect, and it's possible my guesses are completely wrong; but I don't think I'm imagining that some of those groupings end about 90 minutes apart. Many other nights of data show similar patterns, so I think I'm getting legit data.

Here's the raw data for this night. I need to head to bed now, but on Monday I'll post some more data files.

Just a couple of ideas you could try. Maybe make a graph with just one eye, and see it if looks more readable if you are only checking one at a time. You can also try increasing or decreasing the number of readings per second, and see if either gives more readable data.

No idea - didn't read, but what I do know, is that IAMCoder is a good person to take contact up with for you. I believe, he's been developing REM detection algorithms from minimal EEG, which work very well for him.
What else I remember:
It was hard work and there's maybe nothing/not much else in that department freely available, or available at all.
So maybe he could help you, try pm him, I don't think, he'd mind.

Sorry for not searching deeper, but check this thread of his out: http://www.dreamviews.com/research/1...-40-hertz.html

Good luck! (good memory test for me as well...:roll:)

I've got readings from both eyes since, as I roll around during the night, it's common for one or the other sensor to stop producing useful data.

I've fiddled both with decreasing the number of readings (or more accurately just telling my analyzer to skip some of the rows), but I've had better luck using more samples and then smoothing them out.

Steph, it's funny you should mention IAMCoder - NyxCC also mentioned him, and I've been emailing back and forth with him for the last couple of days :)

I was able to roll a converter to my data format of choice: LSD, and start on an eye-movement-detection algorithm.

This is what Nazrax' eyes are up to during a 60-second printout of deep sleep:

http://i.imgur.com/ibG8OVY.png

And here are some eye-movements with the "stairway to heaven" visualization showing where the algorithm counted eye movements:

http://i.imgur.com/cXcQ7Zz.png

P.s.: thanks for the referral, StephL!

Hi. I hope someone can help. I have bought a R.E.M. dreamer mask which has 2 led lights and an infrared detector which detects eye movement during REM. I’m a little concerned that the infrared detector could damage the eye in some way as I’m sleeping with this detector thing bouncing off my eye all night.

are these infrared detectors safe or could they cause long term damage. I’m not too clued up about radiation or infrared so please excuse my lack of knowledge

the mask is called remdreamer and an be found at REM-Dreamer - Lucid Dream Induction DeviceÂ*

many thanks everyone

Ezzolucid, there is something I really don't understand.
you keep on asking this same question again and again and again, year after year, thread after thread, even though several members, including myself and the RemDreamer creator himself, replied you that there is no danger.
Even yourself have been using the RD without issue, right?

Like you said yourself :

Quote:

---

The Novadreamer detected REM in the same way, i dont think ive ever heard of any problems from anybody that wore one :-/

Ezzo

---

Are you gonna keep on asking this question until someone writes the answer you want to read, which is, let me guess, "yes it's dangerous, stay away from it or you're gonna be in troubles" ?
The only negative opinion you have is from a electronics forum. it is not from a physiologist and is only an opinion that I personally find not very convincing .

You will never have any proof that it's 100% safe because this is probably the only electronic system that puts a IR led emitter directly close to your (closed) eyelid, so there is probably no study that can bring a definitive answer.
But by keeping asking the same question forever, pretty much like if you were asking it for the first time, people will finish by thinking that you are trying to spread the imaginary idea that the RD is dangerous.
Like someone else already pointed it out.