The only way to be sure if you are in REM or nREM is to get hooked up to an EEG machine.
You can have episodes of REM at any stage, even during naps.
In most people there is a particular requirement for the brain chemistry to be just right in order to get into the REM state.

I've done some experimenting with the Zeo sleep mask. Now they claim about 70% comparison to a full EEG machine, and judging by what it says, I believe it is recording my dream status reasonably accurately.
I know this because it very often says I am in a period of REM just before I wake up after a dream.
I'm not saying that it's impossible to dream in the nREM states, in fact scientific research has shown that it is possible, it's just that it is much rarer than dreaming in the REM state.

The nREM states are states of brain activity that are much more subdued, whereas REM is almost the same as the awake state, in terms of the EEG trace.

It is most likely that if you wake just after a dream, even in the middle of the night (most people do, even if they don't realise), then you were in REM.