Yes, 3 year olds can accuse. However, take it with a grain of salt. My 2 year old does tell me and/or our daycare provider if someone hits him or takes something from him...but he also says things that didn't happen. Not that they are trying to lie, but sometimes they will start talkign about something that happened in the past or they will be being creative and telling a "story".
I would either just watch very closely when he's over playing and tell him when he's being inappropriate. If his parent's don't like it, they can restrict him from playing with your daughter....
Or just cut it off now, start making yourself and your daughter unavailable. Does she have other friends she can play with in that area? Maybe start making more playdates with those children.
Since your newer, maybe you can find a play group or a mommy and me class at the local YMCA to meet other children and parents.