Wikipedia:

A simple and inexpensive way to remove rust from steel surfaces by hand is to rub the steel with aluminium foil dipped in water. Aluminium has a higher reduction potential than the iron in steel, which may help transfer oxygen atoms from the iron to the aluminium. The aluminium foil is softer than steel and will not scratch it, as steel wool will, but as the aluminium oxidizes, the aluminium oxide produced becomes a fine metal polishing compound.