The dvd players I've seen with official divx support play avi files burned as data. The video codecs used are divx or xvid, and the audio is usually mp3. You should look in the manual of your dvd player and see if there are specific settings necessary for these files.
I'd examine the files you have now and see what is in them. Use either Gspot or MediaInfo.
It is quite easy to convert .mp4 files over to .avi files. There is a free tool called AviDemux which can do this. Import the file into AviDemux, set Video to Mpeg4-ASP (Xvid), set Audio to MP3 Lame. Set Format to AVI. Make a new file name for the converted video and make certain to put an .avi extension on the name. Click on File-->Save to start the conversion.
You can adjust the xvid and mp3 encoding settings by hitting Configure once you select them. If the videos are already compliant to what you need, and you just want to put them in an avi container, you can leave the Video and Audio settings on Copy.