V.B.
Yes. Those 3-4 kids are worth it.
If you do not want to repeat the program/club next year, then that would be fine. But I would not drop it NOW, when you still have that small handful of students who are prepared and anticipating it. Why would you do that to them? If the other kids are unprepared or don't come, so what? They have squandered an opportunity. That's on them. Don't force that on the other kids who want to participate and who ARE participating.
--
By the way... how old are the kids? Elementary? Middle school? High school?
Perhaps every week is too often. Maybe scale it down to every other week? And when you finish the book, make a decision about doing another at that point. The ones who are coming regularly may want to take a break when they finish the book, before jumping into another one. I would suggest that when you are one or two meetings from finishing the book, that you take a poll regarding interest in forging ahead or taking a breather.