S.T.
My husband is a sergeant in the NYPD - you have nothing to worry about - they are way, way, way too busy with real crime to come and aggravate you because your downstairs neighbor is hyper sensitive to noise. I've lived in all kinds of apratment situations over the years - and unless the noise was unrelenting or usually took place at 2 AM I wouldn't dream of coming upstairs to ask someone to keep their baby quiet. Unless there's loud screaming or ongoing banging through the night you're fine. Occasion toy banging is not a big deal.
One suggestion - change up the part of the living room where your baby plays if that's possible. If the corner he usually plays in is right over your neighbors recliner it could get tiring to be sitting below it. ALso, consider a peace offering - homemade cookies, bottle of wine, etc - depending on the neighbor. If it's an elderly lady maybe invite her up for tea or lunch one day and she's take a shine to your baby and you'll never get another complaint - if it's a younger person a bottle of wine may be the thing with a note that says something like "So sorry about the noise our baby makes - this works for me" - and add a smiley face.
Good luck - neighbors are always tricky - whether you're in the burbs and their kids is throwing rocks at your or if a downstairs person...