Before every birthday and Christmas, we do a toy purge. The toys they rarely play with upstairs get moved down to the play area in the basement. Basement toys get donated. We try to throw away as many of the little happy-meal type toys that accumulate as possible. I have simply told my girls that we only have so much room, and we have to get rid of old toys before we can bring anything new in.
As far as gifts, we love consumable gifts. Passes to the local zoo, petting farm, kid's museum, pool, inflatable bounce place - any of these are a lot of fun. Their is the double advantage that A) you don't add to your clutter and B) you are creating a memory with your child instead of just giving them another thing. Your little one probably won't get the idea of a gift card, so you might want to include something small to go with it. We gave one little boy a straw farmer's hat and a punch card to the local petting zoo. One girl got a stuffed giraffe and a zoo pass. You could get a pool toy or a new towel to go with a swimming pass. That way they have something concrete to unwrap as well as a fun experience.
I also have enjoyed putting together themed gift boxes. We have done a princess box, with bracelets, a tiara, play makeup, etc. We also have done an art box, which was a big hit - it included a new coloring book, crayons, washable markers, scissors, etc. It wasn't too much of a clutter contributor, because everything stored back in the plastic box and went on a shelf.