There are many ways to reality check, but my personal favorite is counting fingers. It has gotten to be such a subtle habit while awake that I constantly get lucids by doing it when I'm dreaming.

As far as it not working, I suspect that your problem is that you haven't gotten your reality check down as a physical habit yet. Another thing that you have to remember is that even if you don't have conscious control over the things you do in dreams while not lucid, you will still perform tasks that you have down to a habit without consciously making the decision to do them. Hence the beauty behind reality checks.