Great question!

If you had not said the damper seemed to be helping with the creosote buildup I would have suspected the opposite. With the damper slowing down the draft and keeping smoke in the stove longer I would expect there to be an increase in the smelly creosote. But since that is not the case here are a couple of things to look at;

1. Do you have a chimney cap on that flue? Creosote isn't all that smelly unless it gets wet, then whooo-wee it smells harsh. What's worse is that when water mixes with creosote it makes a strong corrosive that eats away at your masonry chimney rather quickly. A chimney cap is an important contributor to the life of a masonry chimney. If your stove is an older stove that boils a lot of smoke, you may need to remove the spark arrestor if you have a clogging issue with the cap.
2. It could also be that the damper being closed when not in use is holding the smell in. The pipe damper has a small opening in the center to allow a little draft when in use so it does not choke out the fire or make the smoke come out into the room. That can let the smells come down with that wet, heavy air when it rains.
3. Have a look at the inside of your stove. It could be the damper is low in the pipe and the creosote is now coating the inside of the stove instead of the pipe. That would only be the case if the stove is an old one. Newer stoves which are EPA certified burn up all the smoke and creosote before they ever leave the stove.

When you build a fire the moisture is being dried and of course the smell is going out and not falling down the chimney.

I hope this helps you track down the problem.