We've done two major *MAJOR* remodels, so although it wasn't a new, custom build from the ground up, it WAS a very similar experience. Personally, I loved it. And my husband and I make an excellent team in this particular area. Both houses ended up perfectly and very close to budget and timing. Even with that, however, it was extremely stressful.
We like to buy crappy houses in fantastic areas, live in them for a couple of years to see how the house operates, then remodel it to fit our needs perfectly. It's worked for us. We told our architect and builder up front that we're extremely picky and detail oriented, and that we'd be keeping a super close eye on everything. That we'd pick all our finishes, that I'd design the kitchens and bathrooms myself, that I'd use grid paper to do a design and the architect simply needed to make sure it could work and incorporate it into the plans. We chose people who were able to work with us on these issues, that understood that we were a pain in the a$$ to work with, that we had high expectations. Contracts were negotiated by the pros and my husband so that everyone was happy up front. EVERYTHING was in writing. Almost EVERY decision (and there are hundreds of decisions) was made before the project started (I had my kitchen faucets picked out before the architect was even hired, for example). The preparation stage took almost a year. Once we broke ground, though, it was smooth sailing (except for a few minor bumps, totally expected with a remodel of this magnitude). But please understand, we put in hours and hours and hours into this. I researched stuff for months, it was practically a full time job for a while there. It was stressful (we had to move out for months) and disruptive and, certainly, we all got snippy with each other at times. It is NOT something to undertake lightly. For most people, finding a home that's already done that comes close to what they want and need is a better way to go.
Regarding new construction, I have never, ever seen a new construction plan that I would consider buying (and we've looked at a lot over the years). They seem to be designed by people who would never live there themselves…or don't have kids…or never cook…or something. Some plans come close, but then I'm like "Well, I'd have to change that…that wall would have to come down…WHY is there a bathroom THERE?…that mudroom is TINY!" And all of a sudden, I'm doing a major remodel in my head. Plus I've heard there can be issues with quality, etc. My best friend lives in a new build - she likes (not loves) her house, but it's not without its issues. I hate her house - it's large and impressive, but it's so impractical! And new construction neighborhoods seem kinda…cold…is that the word I'm looking for? Without character? Everything all laid out perfectly, all the houses looking similar. You know what I mean? It's just not to my personal taste.
Anyway, building a house is a pretty difficult undertaking. It's expensive, time consuming and very, very stressful. And you might STILL not even end up with the perfect house. You have to really consider if you and your husband have the time and temperaments to go through something like this.