I did a WBTB at around 2:30 to attempt a VILD (How to VILD effectively) and my groggy self said screw it after about five minutes, turned over and just fell asleep.
Oh, that happens to me as well. How many hours of sleep did you get before that? If you already slept like 5 hours and still feel groggy, it might help to get out of bed for a while until you freshen up a bit.

EDIT: I also forgot to mention, I think I actually did a VILD correctly but I got so excited that I fell out of it. I started to enter a dream landscape but my heart started pounding so hard when I realized what was happening.. meh
That's cool, you were so close! It's pretty understandable to get excited while doing this. A lot of people are like "wow, it's working" and then snap out of it. It's something that one mostly overcomes with time. I think it helps to remember that one needs to fall asleep after all. Just be as relaxed as possible and keep on visualizing + the occasional mantra. The body will do the rest.