Are you trying to use an external hard drive? The slower input/output causes problems.
As I've heard it explained, Shrink works in memory and puts its output in a buffer to be sent to the hard drive. Modern cpu's work much faster than when Shrink was written, so it is possible to overload the buffer. Pausing Shrink for a few seconds should allow the information to be sent to the hard drive without exhausting available memory, but you will probably have to do this more than once.