Linux and Mac

You need optipng and jpegoptim for this.
In Debian/Ubuntu just run this command as superuser:

apt-get install optipng jpegoptim

For batch optimization of png and jpeg images, use these commands:

find . -iname '*.png' -print0 -exec optipng -o7 -strip all  "{}" \;
find . -iregex '.*\.\(jpg\|jpeg\|jpe\|jif|jfif|jfi\)$' -exec jpegoptim --all-progressive --strip-all --strip-com --strip-exif --strip-iptc --strip-icc "{}" \;

Multi-threaded variant (example for 4 cores):

find . -iname '*.png' -print0 | xargs -0 -n 1 -P 4 optipng -o7 -strip all
find . -iregex '.*\.\(jpg\|jpeg\|jpe\|jif|jfif|jfi\)$' -print0 | xargs -0 -n 1 -P 4 jpegoptim --all-progressive --strip-all --strip-com --strip-exif --strip-iptc --strip-icc

Notes

• optipng version older than 7 doesn’t work with -strip all, remove it from command in this case.
• For fast png optimization change -o7 to -o2, compress ratio will be just a bit worse, but it will work more faster
• Jpeg optimization is lossless, but you can use –max=90 to change compress ratio (100 for lossless compress, less is better compress but worse quality. My opinion – 90 is the best for web, difference with original image is hard to find.)

Microsoft Windows

You can use same optipng.exe and jpegoptim.exe commands, but foreach file iteration from Windows must be used.
Use this code, paste it to text file with name optimize.bat:

@ECHO OFF
:Loop
IF "%1"=="" GOTO Continue
jpegoptim.exe --strip-all %1
optipng.exe -o7 -strip all %1
SHIFT
GOTO Loop
:Continue

, then just drap images on this .bat file to optimize them.

You can download ready to use archive with script and programs: http://gmelikov.com/files/articles/3/win_imgoptimize.zip.