Yep, it all depends .... you have to test, test, test. Use tools like the quality test in cdspeed to find out where the sweet spot for your media is.
You will probably find that most 8x disks burn the best at 8x. Sometimes 4x will be the same, sometimes it might be slightly better, other times it can be much worse. I also think burning 8x media well at 4x has a lot to do with the burner as well, as my benq ussually does a terrible job of burning 8x media at 4x (while doing an excellent job of burning 4x media at 4x, and 8x media at 8x)
For 16x speed disks, you will probably find you can get better results from a lower speed then 16, say 12 or 8. I havn't played with 16 speed media too much (too expensive, no avaliablity), so I haven't got a lot of insight into it yet