I switched at about 14 weeks with my 2nd...I am currently prego with #3 and think it will be around the same time with #3. It was 5 months with me for #1. All singletons. BUT people recommended to me that I buy a pair or two of pants and a skirt 2 sizes bigger than my normal size with my second because HONESTLY you stretch your maternity stuff out the first time and then they don't stay up at the beginning when you need them the next time around. I found that VERY helpful- just bought some inexpensive dress pants and a skirt for work. It also helped when I had to go back to work and my tummy was still puffy after giving birth :) Didn't have to wear maternity clothes but not in my skinny dress clothes yet with no give!
I recommend buying the few pairs of 2 sizes bigger pants and a few Bella Bands and wearing a twisty around your buttons and buttonholes and covering with the Bella Bands- they are a life saver!
I will tell you back in the late 70s and early 80s when my mom had me and my sister, for her second pregnancy, she was in maternity clothes by the 2nd month :) Of course back then clothes had ZERO give (no lycra, no stretch- ha!).
Best of luck!