Tom did ruin it for all following doctors.
I had read an interview with him, in which he mentioned that he would consider coming back to the series, but only as 'The Master'.
Apparently, someone else got the gig.