How come the lenses in glasses don't flip the image?

A ray diagram of any lens would suggest that it should. I just don't get it.