Hey uplate,
Could the intensity of the nightmares shock me into lucidity if I'd been programing myself before sleep?
Yeah, this is how most people have their first lucid dream, it was for me.
I would in real life and my dreams are becoming longer and more vivid, does that mean I'm getting closer?
You are having lucid dreams, if you realise you are dreaming, then this is a lucid. It sounds like you are making great progress with your control.
As for the rest, sounds just like you need to improve your dream control, there is no such thing as being 'too lucid', there are many techniques you could try, here are some of my favourite.
Shouting out what you want e.g. "clarify!"
Touching objects in the dream, stimulating senses.
Calm down, relax and take in your surroundings.

Good Luck!