I don't think you are unusual. I also didn't like traveling when my kids were as young as your youngest is. I found it physically (when pumping breastmilk, especially) and emotionally uncomfortable to be away from them on the few occasions I had to take short work trips.
But I also enjoyed the freedom from daily responsibility, and you'll enjoy it even more if you have time with your husband, too. San Francisco is lovely. I went there for 2 days when my younger son was a toddler, and I have great memories of the stuff I did in the evenings that trip. I missed my babies, but I sent them photos on my phone of the cool things I was seeing and it all worked out OK in the end.
Anyway, I guess what I'm saying is, don't feel guilty either way. Don't feel guilty if you aren't completely excited, and also don't feel guilty about leaving them. It doesn't mean you are strange, or not a good wife or whatever, I don't think - in a few years, you'll be eager to get a few days away but it's understandable with a just-weaning baby that it's a little harder to tear yourself away. Once you get there, though, I'm sure you'll have a lot of fun.