A.H.
I am a mother of four kids under 12 and an ex- property manager. We ran into this problem all the time. Leagally, unless you are blasting music or have a barking dog, present your self as a trashy loiterer or leave trash in the stairwell...You cannot be persued. It is against fair housing laws.
After all, it's not your fault that your landlord didn't take this into consideration when renting to you, assuming you were honest about having kids. If you want the comfort of 1st floor living then you must take pros with cons. Your neighbours can't do anything about it, but live with it. I'm sure they have enough common scense to not notify you with a broom. If the walls are thin, banging a broom will put a hole thru the popcorn ceilings and drywall.
Diplomacy.
You do want to be diplomatic with your neighbours. NO MATTER WHAT. There is nothing more miserable than living in a place where you get stared down everytime you pull in. Landlords, can't kick you out, but can pester you with noise courtesy notices.
I would recommend baking cookies (make sure they aren't diabetic!) or having your kids draw pictures of them holding hands with your neighbours. They will feel attached and wont become so angry if their adopted "grandkids" make a thud and thump every now and then. Gathering little details and giving them kindness allows them to get to know you and your family. Especially if they see you are a good mom that is doing everything you humanly can to keep the monkeys at bay.
Also, the kid you babysit, needs to be taken outside and given a high energy activity the moment he gets to your house. 30 mins outside and no sugar snacks should help. Also, free play time allows his wild side to come out. Try to give them a new "toy" or structured activity every 30 minutes. Painting on the patio, chaulk, playdoh, dollar store stuff. Trust me...It works! :)