soaking in vinegar will create a patina and help PREVENT rust... not get rid of it.
I would just use some wet/dry paper or a scotchbrite pad (I've also heard it works well) but I would be careful not to dull up the edge to much.
where is this rust?
maybe this would be a good time to post some pics or some beat up scrappers! <img src="/ubbthreads/images/graemlins/thumbup.gif" alt="" />