So you need a few free utilities to help you out. Memtest86+ is for testing the ram, you make a bootable CD/USB and you check the stability of the memory BEFORE booting into the OS. Then to monitor temperatures you get 'Coretemp'. To stress test the PC while monitoring the temperature you get 'Prime95' and choose the torture test.
Make sure you have the latest BIOS installed before doing any OC'ing.
First thing you should do is lower the multiplier that sets your RAM speed, effectively changing your ram from 800Mhz to 667Mhz. You have to do that to ensure the ram will be stable, when you OC the CPU the ram speed will increase and come back up to 800 (this is not important though). RAM speed is far less important than CPU speed.
If your CPU voltage is set to AUTO you are going to want to manually set that so that the BIOS doesn't automatically increase it too much. Then all you have to do is start to gradually increase the FSB (steps of 20 should be good) speed and then boot into Memtest86+ to ensure the RAM is stable (for maybe 10 minutes) and then go into your OS (turn off the internet) and don't access anything important to you because you have to make sure the OS is stable. Then run one of Prime95's torture tests and monitor the temperatures in coretemp you probably don't want to go over 70C.
You might want to back up any important data before starting because there is the chance that the OS will become corrupt. If you understand what your doing then it shouldn't happen.