Hi,
I have to agree with Pam. As a former foster mom, I've transitioned many infants and toddlers. You really need to step back and let this little man bond with his mom, and learn to trust her to meet his needs. What's happening right now is that you are sabotaging his mom's ability to parent her child. I understand how hard it is to see him struggle with leaving you, but he will recover much more quickly if you step back just a bit - and keep in mind that children adjust much more quickly than adults! Perhaps you could be more helpful by supporting his mom by phone, without visiting as frequently, spreading visits out further over time for a while. That way you would know he's okay, you would be assured that mom is doing okay with him, and they can become a strong, healthy family unit. Obviously, you don't want him to feel that you've abandoned him or fallen off the face of the earth. And once they've established their parent/child relationship, it is absolutely appropriate for you to share in any milestone celebrations (like birthdays, school events, etc) and holidays if that works for you and the mom.
I hope you don't find my response harsh, but this child sounds confused. Love him enough to let go and let his mom take your place. There's plenty of time in his future to form a new kind of loving relationship with him as a favorite family friend. Trust me, it works. We just attended the High School graduation of one child who left us at age 2, and the 17th birthday party of another who left us at about age 18 months. We are 'grandparents' to the child of one of our first foster children. Many different children, many different relationships. It's all good! Best wishes -