First, ((HUG)). I'm sorry for your loss.
In my mind, you did the important thing - you went and had a lovely visit with him while he was still alive. To me, that matters much more than a funeral.
However, funerals are not really for the person who died. They are for those left behind. Will your family understand if you are not there? I'd say call your mom and dad, and grandma if she's still alive, share sympathy, talk about the visit you had with your grandpa at Christmas, and then explain the impossible situation you are in financially and that because of it you cannot come. Also call any of your siblings and have the same conversation.
Ideally, your family will accept that you cannot come right now, though you would like to be there with them. They will explain on your behalf to anyone rude enough to question why you are not there. While the funeral is important, you should not go into debt to get there.
As for whether you will regret not being there, that very much depends on how your family behaves when you tell them you are not coming and on your own feelings about funerals. I think that by being there with your grandpa at Christmas, you gave him a much greater gift than showing up to mourn him after he's gone. But if you're the type of person with a strong need for closure, consider carefully how you might feel in a year if you're not there.