Older CDs don’t sound crappy because old techniques were somehow inferior. In fact, recording techniques in general have changed very little over the last 20 years.
You also (mostly) can’t blame it on the weaknesses of old digital gear. CDs sounded bad back in the early days of the medium primarily because the D/A converters in consumer CD players were so primitive. The high end digital gear used to master the recordings was inferior compared to what’s available today, but the stuff used in the big studios for major label recordings was still pretty decent.
Here’s what you’re probably noticing:
First, while today’s audio recordings are most often native digital–recorded through a digital console to a disk-based DAW like Pro Tools, edited and mastered in software–the recordings of 20 years ago were most often analog up until mastering. Analog tape sounds great, but there are a few inherent weaknesses, one of which is a little unavoidable background noise, or tape hiss. Some CDs that are digital conversions of analog recordings will sometimes have a little noticeable hiss if you are listening at high volumes.
Second, mastering techniques have changed a lot since the 80’s and early 90’s. The most significant change is the tendency of mastering engineers to apply a lot of compression or hard limiting to final mix, which greatly decreases the dynamic range of a recording but makes it sound really loud and punchy. Recordings from even the early 90’s sound much quieter than modern recordings because of this practice. The advantage to that kind of aggressive compression is that our ears initially percieve loud recordings as sounding generally better, bassier, punchier, etc. Also, a loud recording will reveal fewer of the weaknesses of a cheap cd player/receiver/etc, because you don’t have to turn it up until you start to hear the background noise from your system. The disadvantage to that sort of mastering is that listening to a recording with very little dynamic range is fatiguing, but at first blush, that is probably the #1 reason that a new CD would sound better than an old one: at the same volume level, a new one will sound much louder and punchier.