Don't forget places like
Magnatunes. Magnatunes represents mostly indie bands. The music selection varies in quality but you get to preview everything you buy. I've had good luck with the jazz, classical and "world" music genres. The rock and pop are uneven.
The prices at magnatunes are right -- you pay what you think it's worth with a suggested price of US$8.00 of which the artist gets a full 50%. Even cooler, you pick the format(s) you want -- you can download just an mp3 copy or you can get wavs and say flac or ogg. There doesn't seem to be a limit to the number of formats. I tend to pull down the flac and re-encode to save them bandwidth but the option is there. Also, you can go back and re-download. It's not like the iTunes Music Store where you get one shot at the download. They also let you listen to almost the whole catalog in streaming format as mp3 radio. They've also just announced that you can give any album you buy to 3 of your friends. John Buckman, who started magnatunes, has a really refreshing philosophy to music -- very compatible with the Free Software movement.
There are also other places. Michael Robertson who started mp3.com started a new one called
mp3tunes.com. Seems to be pretty much the same artists from mp3.com. Again, the quality of the music varies. The prices are pretty much in line with the "majors" like iTMS but unlike magnatunes you only get one format -- mp3. Also, the previews are snippets whereas those on magnatunes are full length. On the whole, I have a hard time getting excited about mp3tunes because you pay $8.88 for an album in a lossy format.
I haven't tried allofmp3.com but have walked their catalog. It's very nice and complete. I think I'd use it as suggested -- for marginal interest items. Anything I like I buy as a CD and just rip/encode to disk.