You can, but you need to put Steam in offline mode beforehand, as I understand it. This is only really a problem on an airplane or cruise ship or something, somewhere with no cellular internet access. On a train or a long car ride you could just tether to your phone to start games.
I agree this is something Valve really needs to fix, I’ve seen a bunch of people complaining about it.