I think not. I listen to music to fall asleep all the time and still get lucid. Have you got a playlist? Maybe try not to make it the entire length of your nap, just enough to get you to fall asleep. If you listen to the same music long enough, you get sleepy as soon as it starts playing in the proper order so it doesn't have to be very long anyway, or at least that's how it is for me.