the fact that your stand alone has issues with a dvdd ripped title makes me suspect that somehow your file size problem is related or an extension of it. you shouldnt have that problem at all with any ripping program. btw claddvd is another ripper you can try.
forgive me if this seems obvious but i am running out of ideas. have you changed the settings in either or both dvdd or smart ripper? for instance how the angles are treated? do you have stream processing enabled??? i suggest using the default settings in both rippers. maybe uninstall them and download the latest versions and try ripping without changeing he default settings.
when you say you are also haveing the problem with un encrypted dvds,what kind of dvds are you talking about? a vhs home movie transfered to dvdr? a store bought dvd thats un encrypted? or a backup of a store bought dvd that you want to make more copies from? personally i have had no luck making copies of copies. all of the store bought un encrypted dvds i have seen,and thats only a few,are all below the 4.3 dvd-r size. so if you used dvd2one on that it may act wierd.
or are you considering a previously ripped movie ,thats been sitting on your very spacious hard drive,an un encrypted title? just because you got it on your hard drive,and you can play it, doesnt mean that all the encryption has been successfuly removed. or even that the files are processable.
i do still think you should try one of the problematic titles with dvdshrink. if you process with dvdshrink one of the same titles you ripped to your hard drive that failed with dvd2one,and it still fails with dvdshrink then the problem almost has to be the rip. use the same ripped files that failed with dvd2one to be sure.
if you do find that dvdshrink fails processing the files from your hard drive,try and let dvdshrink read from the source disc. yes dvdshrink rips and process's in one step.
I also strongly advise you to rip all files at least until this issue is resolved,just to be sure that is ruled out as a contributing factor.
i would still like to know what specific titles you are having problems with. and if you have changed your rippers settings,what are they? i would like to try and reproduce the failure.
i dont know if any of this helps,but its all i can think of.