From my experience, the headphones tend to affect the sound quality much more than the playback equipment playing CDs. For example, a good pair of headphones worth $200 connected to a basic $30 CD player will usually give better sound quality than a basic $30 pair of headphones connected to a professional $200 CD player. Think of it like building a high end car audio system; most end up buying $1,000’s in audio gear (amplifiers, speakers, etc.) and often end up using their existing car stereo or getting one costing less than 1/10th the price of the audio gear.
For the CD player, I would suggest checking user reviews first, such as on Amazon, NewEgg, etc. or getting a used player. The main issue I find with entry level equipment, is their reliability more than their usability/playback quality. To give an idea, one of the local supermarkets here was selling a â‚¬20 DVD player. A work colleague bought one and it refused to play any disc he loaded, which all showed “No disc”. So he brought it back and got it exchanged. The replacement player had the same problem. He brought it back, got a refund and bought a second hand DVD player, which played his discs fine.