The Gortex pants are louder than cotton pants. Not twice as loud, but a bit louder. They don't make the high pitch "zip...zip...zip..." that some nylon products make. It's more of a raspy sound. For the type of hunting I do, I don't think they hinder my stealth.
I had a pair of GAP wool trousers that I wore for years. I loved them because they were light, quiet, warm, yet breathed well and SUPER comfortable. I did get them wet a couple of times, but they dried quickly and did not lose their insulating properties while damp.
They finally bit the dust on a pretty gnarly hunt. I had to fix them with duct tape. I tried to salvage them, but they were too worn. They served me well.
I am considering getting a pair of wool from a local military surplus store.