Well the benefit of a 64bit CPU because of its ability to do 64bit apps is, when using Windows, zero to none. In about a month, XP64 will be released (at least, that's what some MS marketing guy said). Applications that are made for 64 bit can benefit from it.
As Linux has been ready for 64 bit for quite a long time now, several tests have been done on the Athlon 64 CPUs. The biggest measured difference in speed was a 54% increase (when using the same system, same installation and the 32 and 64bit version of an MP3 encoder called lame). This doesn't mean that all applications will run more than 54% faster when using 64 bit optimized software, but the differences can be big. My guess that, generally speaking, the performance increase will be about 10~20% on an average.
There are good reasons to buy a 64 bit chip (actually I'm talking about AMD chips right now, as their are no interesting Intel alternatives at this time) though. The K8 architecture is quite optimized, also for 32 bit. It has a native built-in memory controller (huge performance boost), dual channel abilities (on the s939 platform) and more interesting techniques. Also Cool'nquiet is an interesting technique; this lowers the clockspeed of the CPU when not that much power is needed, resulting in lower power consumpion, less heat dissipation and thus less noise production.
About the overclocking issue: if you get a mainboard that has working AGP/PCI locks and you have good memory, the Winchester based A64 (s939) can be quite an overclocking monster. I run my 3200+ Winchester on 2167 Mhz (I can't go any further because my mainboard isn't fitted well for overclocking so I can't really set the AGP and PCI locks), but I know more than enough ppl that have NF3/NF4 based systems that do more than 2.5Ghz with their 3000/3200+ Winchester CPU's. That's a lot of performance for very little money.