that's not what i'm talkin' about.
i also don't use an anti-virus-software on my pc here, and i really don't have any problems, 'coz i always know what i'm klickin' on...
BUT when in comes down to your browser, it's a completely different thing:
it's not like a firewall which shall prevent someone from hacking especially 'you', it's just the possibility, that maybe ANY website might use a know browser-bug to install you a keylogger, a small smtp-server to send spam to people, or whatever on your machine.
this is not a "specially-against-airhead-designed-attack" but a general thing which might hit anybody who uses an 'open, unsecure' browser.
of course, the risk of this is low, at least if you don't surf on 'underground' or whatever sites, but how much work do you have with d-loading one file (firefox) and double click it just to install over your existing installation?