I found it!
For anyone else curious, and I don’t know why in the world this isn’t on by default, you click on this Map Options icon above your mini-map to find this. I’d have never known it existed if @Spock hadn’t mentioned it was possible.
It really does make the gameplay map looks just a little more elegant.
Also, is it me or does this game just look a lot prettier than I remember? Everything from the map to the units themselves seem to carry more weight, somehow. And the colors! Oh, my. Anyway, here is a mountain range and a river with labels on, I can’t recommend the feature enough.