I think that one day it's just going to work out - since it's very hard to find the border between aware enough and aware too much... After that one time you'll have a better sense of how it feels and will be able to have more control next time. Even though it's likely that some of the next tries (after the first success) will probably fail - on average you'll see an improvement.

That's how it's for me - after I restarted my LDing practice 4 months ago, I also started doing WILDs again (something that was fairly easy for me in the past). But just like it took me quite a few months to get better at it in the past, now after 4 months I'm still with a relatively low success rate (where it comes to directly maintaining awareness). But on average (monthly) I see improvement.

The yanking back out feeling, at least for me, is associated with maintaining "too much" awareness - so it's like my mind has to wake me up from that (and it also makes it harder to sleep afterwards). Maybe you should start experimenting with the level of focus you maintain, and focus slightly less. At least in my experience, even if I just lose awareness altogether it still raises the DILD chances in the following dream by a lot. So I lean towards falling asleep rather than over-focusing and not being able to sleep at all.

About going lucid by WILD - you shouldn't be frightened by it especially if you had DILDs before. The only difference is how you enter the dream... Have you experienced HH and HI? These are the hardest parts to me to pass, but after that one of the HI can develop into a full dream or a full dream can just appear. It's like some figures suddenly appear out of the blackness. For example, in a recent experience of mine a crack of light appeared in this blackness. Then the crack changed shape and looked as if it's a crack between a door and a wall that lead to a very bright space (relatively to the lighting inside - which is still blackness). So I just pulled the "door" away and entered the well lit hallway (and the dream started there). On a different recent occasion, the blackness changed into fog that started going away revealing a city. The way the scene starts varies a lot, but it's not frightening. Might feel like floating or falling or something similar, but it's not uncomfortable, just weird. Plus, FAs are very common so sometimes there won't even be a transition...