I've experienced the very same thing, and I usually keep the headphones in; because if it bothers your sleep that much, I've noticed you'll just take them out after your REM cycle (but I normally don't remember when I wake up between REM cycles ).
I say that you should try keeping them in, and see what happens like whether you find it easier to fall asleep or if you don't care for it at all; and then try taking them out. Just try experimenting to see what works best for you.
Personally, it works better to keep the headphones in for me