With practice your flying ability will increase, but the "roof" thing wont be so easy to get rid of
I have it for myself - If I raise above the roofline, there's no real town below, more like a cloned buildings and stuff, really weird.

My last experience with flying above rooftops ended in exteremely bizzare psychedelic-like scenery