You sound like you are doing fine, you just need some new ideas. This is the perfect time of year for it too.
Make a schedule.. this is really just an outline, so that you an have some guidance.
mark it for the week. then divide it into parts so that it can be interchangeable if the weather changes or your needs change.
Wake up, breakfast, get dressed.
Go outside and play.. Running, jumping, climbing. Throw the ball kick the ball. Maybe some sand time in a bucket or tray, with trucks and cars or buckets and shovels.
Snack and run errands or snack and then play inside. Hide and seek, play house with cooking utensels. Play restaurant. He can be the cook. you can be the customer. Build with blocks, legos.
Maybe another day at this time there is a story time at the library or the book store. Many times they also have an art project. Remember to have a snack before you go. Swimming is also a wonderful after nap activity. We would go every other day and as she got older we went almost every day.
Come home and have a quiet lunch. Have your son help carry his plate, carry his drink to the table, ask him to carry moms plate. Ask him to find an apple that you 2 can share. Or ask him if h would like to peel the tangerines for the 2 of you.. Have clean up time. "Please carry moms plate to the kitchen"..During all of this use the "thank you" and "your welcomes" with each task.
Maybe one day have a picnic in a park or at a downtown location. In Austin during friday's at noon, there are free concerts outside City hall.. Dance with your son. Spin with your son.. Take a camera.
Visit a petting zoo. Go to a school campus once school is out and let him ride his trike all around the school. We loved doing this at The University of Texas or the State capitol grounds. Museums just to see the bones or fish.. They do not care about the other stuff, so do not expect him to want to look at Texas history if it is mostly text and mannequins.. Also do not be surprised if all he wants to do is go up and down the stairs, just remind him no running inside buildings and to always use his inside voice.
Nap time or quiet time. Quiet music or book on tape. May need to darken his room.
Once or twice a week is a morning of "running errands". Plan it out and give you son a list of what he is supposed to help you remember. At Target we need, Toilet paper and soap.. At the grocery store we need milk, eggs and ice cream for dad. Help me remember to by gas for the car. Remind me to check the oil. Have him be responsible for always having his diaper bag or travel bag. Let him decide what snack and drink will go with you all.
Have him remember he needs you to put his sunscreen on and he needs to always wear his hat when you are going outside. Bug spray too if needed.
In the afternoon maybe an art project or playdo time.. Then outside again lots of running, jumping climbing. Snack time can be during this time.
You can get an idea.. I would always write down dates of free shows and activities and they would be tentative in case we had time and emotionally a good day for an adventure.. If not, it was no big deal.