Welcome to the forums dixiegirl.
Converting your mp4 files...that depends on what codecs are used in them now. Mp4 is a type of container, and the video and audio codecs used in that container may vary. If you want to know for sure what is in the mp4 files, examine them with a program called MediaInfo. It is free, and I find the Tree view within MediaInfo to be the most informative.
Your dvd player is divx compatible, which means it can almost certainly play avi files with divx or xvid video, and mp3 audio. There are many free programs that can convert mp4 to avi files with xvid and mp3 audio. AviDemux is one, Xvid4PSP is another. Video-to-Video is one that is gaining some popularity: http://www.videohelp.com/tools/Video-to-Video-Converter
If you would rather have dvd-video, I would suggest using AVStoDVD. There is a link to a guide for it in my signature, just below this message.