It can be astounding what deep thoughts three-year-olds have. Those little brains and hearts are working hard trying to figure out everything they see and hear.
So your son's concern is quite normal.
When my older son was that age or a tad younger, he saw an abandoned commercial building near our home being demolished, and he was shocked - wondered how the building felt about being taken apart, whether it hurt, whether the building was sick and dying. And he wasn't being imaginative or fanciful! Children really want to understand life around them, but since they don't have a lot of experience, they don't think the way grownups do. And that's fine.
You may never find out why this question has popped up right now, but it's evidently giving him great concern at the moment. I commend you for not brushing it off.
A lot of what you tell your bright boy about death depends on your beliefs. As a Christian, I started with what God says in the Bible about heaven and about His neverending love for His children. I was able to answer my children's concerns from that standpoint. (After that, questions still popped up from time to time, of course.) You need to know what you believe and start from there.
But please make it SIMPLE - your son has to process everything that you tell him, too.
Be sure to let him know that yes, things change and sometimes there are big surprises. Yes, sometimes people do die (pets die, too). And it's OK to be sad when we miss those we love. But there will always, always be someone to love him and take care of him. Emphasize that his mommy and daddy are planning to be around a long, long time, and if that changes, there will STILL be people around to love him.