I purchase from Amazon or iTunes..The iPod plays MP3's besides AAC/m4a..
If you purchase from iTunes, you can automatically sync to the ipod if you so choose..If you purchase complete Albums from Amazon it'll automagically store the tracks on your iTunes/music folder..You could also use WinAmp to sync the iPod, as I stated above..There are other types of Mp3 players...Others will recommend some soon enough..
If you want to store tracks from your CD collection, you first have to rip the CD tracks to HDD and convert to MP3 or AAC/m4a for the iPod..You can import WMA, WAV etc, and it'll convert to mp3 or AAC, your choice..
I normally use EAC or dBpoweramp to rip my CD's, but you can also do it via iTunes to rip and convert..
Some around here recommend CDex..There's also foobar2k, which is a very good media player/converting/tagging/ripping app..