Rubber belt cleaner: http://popsupply.bizhosting.com/rubber_belt_cleaners.html

Or use the sole of an old tennis shoe.