My Apple TV is running ATVFlash which opens it right up. It installs a whole bunch of stuff including nitoTV which allows you to define network shares. I have a 2TB NAS drive providing my content so there is no change in format, the VOB's are being accessed through the network share and not through iTunes.
Tried to disable upscaling on my DVD Player but couldn't see a way of doing it. There were 3 output resolutions for HDMI (576p, 720p and 1080i) and I changed the HDMI output to 576p and played the same episode and the quality was still better than the VOB. Going to try and get hold of a standard DVD player with no upscaling and plug that in to see the results...
If it is the upscaling that is the difference here, surely there must be some software out there which does exactly that which I could run my VOB's through??
Anyway, there must be something amiss here which I have not understood as I agree regarding the comments about passthrough... as it was my initial understanding that the streams remained untouched and there was no loss in quality.
By the way, my TV is a Samsung 46' 1080p LCD