There's an old kid's classroom poster I see around from time to time: "All Feelings Are Okay, It's What You Do With Them That Counts".
You do have the *right* to your feelings. That said, it sounds like you understand the precariousness of acting rashly or doing too much to rock the boat.
It's a long road for families who deal with adoption. I know this because I was legally adopted by my mom's second husband (whom I call my 'growing up dad'). Those relationships with both my bio and adoptive fathers have been a long road and didn't really resolve until I was well into adulthood.
What I am saying is that you have a lot of time ahead of you to make connections with your son. What I would encourage you to do for now is to find a counselor who helps families dealing with adoption and adoption issues, and to find some support in this way. Your parents are likely far more emotionally 'safe' for your son's adoptive family to keep in contact with than perhaps they feel you are. Not because of anything you've done, but there may be some questions coming up for their family or the boy which are already hard for them. ( I know that I was about this age when I started pressing to meet my bio dad. This was very difficult for my parents.)
For now, the best way to handle your feelings is to find a counselor or empathetic listener, and to process all of this in a safe, supportive environment. You don't have to 'do' anything beyond what you are doing now, which is just staying available. At twelve and early adolescence, this would be a time where I would just be 'open' and not press too much. It's likely that you would be able to develop more of a relationship as he becomes older and is emotionally ready to know more, to ask questions... and when he is old enough that his adoptive family will know that they do not have to 'protect' him so much as to let him suss things out on his own.
A couple of years ago, a woman I knew socially decided to find her birth mother-- it turned out to be my stepmom's sister. Even in her thirties, they have made great strides in developing a relationship with each other. Even in her thirties, it was also still very painful for her adoptive parents, too.
All this to say, please try to take care of yourself first, and then let your son reach out to you when *he's* ready. Be patient, and know that by giving your son to live with a family who wanted him was the kindest, best thing you could do for *him*, given your situation at the time.