J.,
I took my 11-year-old son to Disneyland last summer and we both had a blast. We went to Disney World when he was eight years old and loved that also, although other than the absence of Epcot (much of which is not that well suited for a little one anyway), I actually prefer Disneyland. California Adventure is just a walk across the parking lot, whereas at Disney World going between the four different parks is a much bigger deal.
At four years old, if your son likes Legos he would love Legoland. We drove there from Anaheim one day of our trip last summer, and most of the rides are definitely geared to younger children and would be age appropriate for him. My son loves Legos so enjoyed the park but only found a couple of rides that weren't too young (boring) for him.
Although we didn't go on to San Diego to see Sea World (he's been to the one in San Antonio), I think that would be a great idea as well, if you can afford to hit several parks in one trip.
We stayed at the Park Vue Inn in Anaheim, right across the street from the main entrance of Disneyland. It is not a fancy hotel by any means, but very reasonably priced, suited our needs and made going back and forth to the park incredibly convenient. We rented a car and the Park Vue does not charge guests for parking, so we just left it in the lot while there and walked back and forth to Disney.
When we got tired, we walked back to our room, had lunch at IHOP next door (pricier than most IHOPs but less than Disney food), took a dip in the pool, a nap, and then back to Disney! You can request a room with a queen and a bunk bed which even at 11 my son thought was very cool.
If you are not paying for more than one or two days at Disney and are afraid you can't fit in everything you want to do, save Downtown Disney for a different day or evening as you do not have to have a ticket to the park to go there.
Also, if you do Legoland and want to buy Legos, don't wait until park closing time to go to The Big Store as things are picked over by then. Either plan to leave before the park closes and hit the store on your way out, or buy your sets earlier in the day and take them to your car or rent a locker for them. If the store does run out of something he really wants, keep in mind that Downtown Disney has a Lego store that appears to be as big or bigger than The Big Store at Legoland, and my kiddo found everything at the Downtown Disney Store that had run out at Legoland.
If you plan to fly, check the prices flying into LAX as opposed to San Diego -- often significantly less expensive.
Okay, that's probably more info than you really needed, but we had a great vacation and I hope your family will, too!
L.