All kids cycle sleeping well for a while and then not. So expect regression from time to time.
My first observation is that you moved him into a "big boy bed" too soon. My daughter is almost 3 and just got it last month. Every kid is different, but I wouldn't do that to a kid who has sleep transition issues.
Second, follow a routine. Before bed we 1) pick up toys, 2) brush teeth, 3) read EXACTLY 2 books, 4) sit on bed for nighttime drink (she has hydration issues, not recommended for all kids), then that's it. EVERY night is eactly the same. She went through a month and a half of challenging it, but you have to stick to it.
Thirdly, I know you don't want to hear it, but you are setting your kid up for failure if you cannot keep you responses consistent and stick to your guns. That teaches him that anything can happen, and he can rely on nothing, which can make him more insecure. You do him no favors by giving in.
Good luck. (recommend the love and logic media)