Someone else pointed out (and no one ever responded) that when using the set filesize option, it doesn't take into account the correct audio bitrate when deciding the video bitrate.
Set your filesize, notice the preselected fixed bitrate in the box above. Now, adjust the filesize (I just delete the last digit) in any direction and then set it back to the filesize you want. Notice the fixed bitrate in the box above has changed downward by a factor of whatever audio bitrate you selected. The filesizes should become more accurate now.