This is one of those topics that is great to have as an on-going discussion. Don't feel like you have to tell them everything they need to know in one conversation! As with other things, you will want to provide the basic truths for them in a manner that fits your child and your family.
Here is what I've learned with my daughters thus far:
I had an uncle die (we were very close) from pancreatic cancer when my older daughter was 3-1/2 and the younger was only 5 months old. We all went to his funeral, and talked about how him, sharing stories and time together. A year later a very close friend of mine also died of pancreatic cancer, and we had many more talks about death and dying. (My children did not go with me to the funeral because I needed time and space to grieve without worrying about what they were doing and might need, and I wanted to be fully available to my friend's children in any way they needed.)
One thing that surprised me, was to hear my daughter say one day (seemingly "out-of-the-blue" when we were talking about Jesus): "Mommy, we don't like Jesus, right?!" Wondering where and how I had gone wrong, I asked, "Why do you say that, Darlin'?" She said, "Because He takes away the people we love." Ack. I took a deep breath, and told her that I was very glad that she told me, because that is not how it is at all! I went on to explain that God created us and our real home is with Him, and not in these bodies. We talked about how we have fun when we go out, but it is always nice to come home, and that it is even more true of Heaven. As much as Mommy and Daddy love you, God loves you EVEN MORE, so that when our bodies die, Jesus is waiting to welcome us home!
We don't like watching our loved ones hurt, and in Heaven there is no more hurt or sadness. So death is a very good thing for the person who died, but sometimes those of us who are still on earth miss getting to see and touch and talk with the person who died, and it is okay to be sad and miss him/her, but it is also important to remember that we will see all our loved ones again one day, and what a happy day that will be.
My girls are now 6 and 9, and recently my almost-95-y-o grandmother died. Sometimes we'll look at her artwork and talk about things we remember about her, and that we miss having her around. Then we talk about how much fun she is having in Heaven now with her Lord, whom she loves so much, and getting to see her brothers and sisters, her husband, her parents, and everyone she loves, and surely she's dancing and so happy not to be confined in a body that doesn't work well anymore.
All the best to you as you move through your grief. I pray that it is a time of growth and healing for your family.