If you don't follow a majority of the current trend of music like myself, I'm sure you'll find yourself looking around for older (and much better) stuff at the used music CD stores.
Unfortunately one of the best chains for new and used music closed down over the past few years, and one time, a friend and I were lucky enough to walk into one of their "closeout for good" sales, in which most of their merchandise including CD's were 80% or more off. For a little over 20 bucks each, we walked out like bandits. Prior to then, they had great sales on used discs and older slimline CD singles, such as 3 for $5.
Another good place to find used music at a steal is Half Price Books, and I know several people who shops there for music. Although their more recent used CD's are priced between $5-10, many of the local HPB locations have awesome clearance sections, priced at .50 cents to $3 each. Some of the stores just price them all at $1. And at each location, you're sure to find that damned used copy of the second Wilson Phillips album nobody wanted (Shadows And Light from 1992) in the clearance section. I would say close to half of my CD collection came from HPB, with all the good stuff I've found there for virtually a buck each, old and recent. And yes, I bought that not-as-good-as-the-first-one Wilson Phillips album LOL
Being a music collector, you'll find out in many cases the entire catalog of the artist or group is no longer in print, and sometimes you'll be paying outrageous prices for them on eBay, Amazon, etc. Again, used CD stores and places like Half Price Books comes in handy, because I can't tell you how many out-of-print CD's I've found at regular used CD price, and avoided paying $40 and up for it online. I generally keep a list of what to look out for, that way I can save $, plus 9 times out of 10, I soon find whatever CD I was looking for used at one of these local places. When push comes to shove, I've been lucky enough to find a copy of the hard-to-find disc's at reasonable prices, although still paying a pretty penny more than the original retail price.
Some dance music geeks collects the remix maxi CD singles, also known as EP's. Usually they feature remixes of the single, and in some cases, exclusive live tracks, or unreleased songs. Half Price Books usually has several of these, and I've stocked up on some of the best older singles, and have uncovered some really good remix versions of those classics.
An interesting note about CD singles, is sometimes you'll find a different version of a standard jewel case CD single, with the same artwork as the commercial version, but with a different (and usually better) track listing. I assume these unusual releases are promotional-only, and are rare finds that you'll only find among used CD's. Some of the promo singles I've found only have 1 track, and some only have artwork on the back of the jewel case (track information), with no cover artwork as the way it was distributed.
I have to say that the import versions of singles are some of my favorites, that I've gotten my hands on thus far. Japan and Germany especially catered to the dance music enthusiast needs at the time techno was the "in" thing. A great place I personally enjoy shopping at is Total Recall, located in Germany. Their prices are reasonable, but the shipping will kill ya when you order several items. It's still worth it, though. Check out Total Recall's website, and be sure to click the English version, and convert the $ to USD (the default is German and priced in EUR).
And luckily, I have some great friends across the country, and sometimes I'll get a surprise in the mail, such as an out-of-print 1990 CD Miss Mary Brewster found for my last birthday, that generally would have cost around $30. If I remember correct, he paid less than a couple dollars for it at his local used music store.
More on music collecting for another day!