First of all, you need CPU power for encoding.
So I would recommend a Quad-Core.
When encoding with free x264, AviSynth and so on, you need lots of ram.
So I think 8GB ram are good (and x64 OS)
To have several passes of video material, you need a large hdd.
I would suggest several 1GB hdds (maybe as Raid0)
Video can be onboard, but can be a cheap card also (maybe Nvidia 9400 or Ati 4350)
I prefer for Quadcores Intel CPUs, they have more horsepower at lower GHz, but are somewhat higher in price.
Don’t go for new Core i7, it’s good, but too high in price.
Maybe a Q9550 for overclocking? I have a Q9650 I OC to 4.2 GHz
Mainboard is your choice, I would use NVidia 9300 or Intel P45 (with ICH10R).
An “old” LG GGW-H20L would be “enough” as BluRay-Writer.