You'll end up reencoding in the end as I haven't found a good way to get DVD rips or vobs to play reliably on the Xbox or PS3 for that matter.
What you need to do is take a step back and look at what the Xbox your using supports for formats/containers/codecs. I have a 360 and used this page to determine what the 360 supports:
Then I looked at what the MSS supports with PP2:
MP4 audio and video file support has been added and content that is stored on your home server and that is shared via Windows Media Connect, including files with extensions .mp4, .m4v, .m4b and .m4a
Then I use Handbrake to convert any media I have that I will be playing on the 360 to mp4 (via the 360 preset in Handbrake) as the other network streamers I have support it also.
"everything will be ok in the end. if it's not ok, it's not the end" - unknown