I'm surprised nobody suggested this yet, but you could just ignore him? Go straight by him. He might follow you, but just don't let him bother you. There's every chance he could even give you advice or help you along in your dreams! Just let him be your friend!