PNG was created as an alternative to the patent-poisoned GIF format, and took on additional tasks that GIF never thought of.
GIF has a capability of up to 256 colour palettized, and also has the capability of interlace (for fast initial display), animation and transparancy (transparent colour). It uses a compression with UNISYS patents now expiring in most places in the world.
PNG can do palettized, and also truecolour, plus it supports alpha transparancy (though many browsers don’t) as well as more basic transparancy.
On the same material as GIF, it can often compress better, especially when fed through an optimizer such as PNGCRUSH, plus it can avoid reducing colour depth.
Even at maximum compression, PNG is still lossless, as it implies the same as maximum zip compression - just use more exhaustive searches for compressible areas of data.
For animation, there is the poorly supported MNG format.
JPG is a VERY poor choice for anything that demands pixel precision, since to preserve any degree of accuracy and clean-ness on text and line art, you end up setting low compression, and getting a large file.
JPG makes use of optical redundancy in photographic material, and will compress a photo far more than PNG would, but then PNG is lossles, unless you force a bit depth reduction.
PS. A tip for PNGCRUSH - given the way compression works, the best possible result will be found in full testing mode, with the compression locked to 9 - lower compressions might be faster, but if aiming for a “final”, applying max compression will never lose anything or be worse, and it will not slow down decompression.