Compared to many, I had easy pregnancies (except for the whole 2 miscarriages thing). I still say being pregnant sucks. Having two does start the no sleep thing all over again. That really sucks (I like my sleep). I don't really like the infant stage (my second was a collicy ball of puke so that doesn't help my opinion).
With that said, I am so grateful to have two. My two are 2-3/4 years apart. I had the second before I left the diaper stage with the first. You say you enjoy the increasing freedom. Oh I hear you. Having two sets you back on that freedom, but having two increases your freedom once you get it back.
Just about when your second starts walking they just go and play together and you just have to listen for bad noises. When you travel, you don't need to worry about playmates, they have each other. Going to a playground, you don't need to be the playmate, they have each other. Even having your kids watch a little tv while you take a moment to make dinner or just gather your thoughts is better with two. You are not putting them in front of the TV by themselves. They have each other. Truly, nothing is so beautiful as watching my boys being brothers. I know there are many many only children out there who are happy and well adjusted. I just see so much good they get out of having each other. The hard part lasts but such a small fraction of your life and you are giving your children such a blessing to have each other for a lifetime.
I think it gets harder to go back to pregnancy and infancy the farther you get from it. I thought about three but decided I just couldn't put myself through that. I thought three when we were done with diapers and cribs and all that.
Don't pressure yourself so much on a magic age number, but remember it does take about a year to get that second kid. You have a hard time picturing a second now, but you won't have a second now. You will have a second child a year from now.
Best wishes for this tough choice you face.