Like most of these things, how "hard" it is depends mostly on your adviser and the project you're working on - the main difference is the amount of time you put into it. My PhD was a new time record for our department - I just squeaked in a few months under five years. Typically 6-7 years is more typical for an experimental physicist.