It is not possible to convert from dvd video to xvid or H264 without some quality loss in the process. That is just the nature of video conversion. Before you give up though, I suggest doing a test and see if you can actually see the difference between them. There are many good tools available for this type of conversion, and some of the best ones are free.
Mount the ISO in VirtualClone Drive (or another virtual drive that you prefer) and convert to xvid using the standard settings in AutoGK.
Then try a conversion to H264 in Handbrake, or Ripbot264
Use your eyes for judgment of the quality, not any preconceived ideas about the purity of the original ISO. Movies vary tremendously in quality even from the studios, so you aren't dealing with a cut and dried standard from them.