Here's another opinion.
Beyond preset medium, most people have difficulty telling the difference on most encodes. There are a few "killer" clips that even preset extreme will show detectable problems (e.g. Castanets.flac from ff123.net).
However, as a general rule:
preset standard is transparent to 90% of the people
preset extreme is transparent to 99% of the people
preset insane is CBR
The fast options do NOT produce as high a quality as the non-fast presets. If you are going for maximum quality I would use extreme (or maybe standard). However, the 50% speedup on fast is not really worth it to me.
The bitrate on some stuff can be quite low even with extreme. Some movie soundtracks I have compressed end up with 160 kbps. This is because this is all it takes to preserve good quality.