I haven't use this software much but you could demux it with YAMB.
Then work on the audio with a software like Audacity.
I think YAMB can mux the audio & video back together.
AviDeMux can also do the muxing but it will change the audio to AC3.
If the .mp4 audio is aac lc AviDeMux won't demux it with copy.
I guess it depends on how much the slight break bothers you.