Same thing happens with Windows Media Player when you don't have a codec (ie. DivX .avi's). Are you sure you burnt a video dvd and not just put .avi's (or another format) onto a data dvd?
Reread your post- Video DVDs do not play audio or video in Windows without a DVD decoder installed (does not come with windows). So, you either burned audio only, or you did not burn a video DVD.