we never did a reward chart - we did a chores list. my son is now 4 1/2, we did them at almost 3, and around 4. our problem was that mornings were a nightmare. so i made a 'list' (in pictures) of all the things we had to do each morning before we left for work/daycare. if he did all his "chores", he was allowed to watch tv, have a small snack (daycare provided breakfast), or play with toys.
The first time i drew, and the S. time i used clip art lol. i put pictures of a potty, clothes, and a toothbrush. i also used a fish the first time, because his job was to feed the fish, and this last time i did the dog because now that's his job. then i put check boxes out to the side of each picture. as he completed each task he got to mark the box. when they were all filled, the bottom had pictures of a TV, (toy) car, and some grapes (snack). he got to pick one.
when we did it at 3, i had to really get firm with him and many mornings he fought me tooth and nail (multiple timeouts, over and over). eventually he "got it" - i wasn't kidding, he DOES have to do these things every morning. i was surpised at how forcefully the kid could fight these simple tasks that i had always taken for granted "had" to be done. it was a hard battle.
around 4, he decided to act like a butt and challenge me on things again, so i started over with the chore list, and it didn't take but a couple of days that time before he was back with the program. it was really more a matter of enforcing the "no fun things until chores are done" rule, which i had started to slack a bit with. if i was more laid back and he got to watching tv or playing, before his "chores" were done, i found it was MUCH harder to get him to do them. much easier to hit the floor running when he gets out of bed, then when the "work" is done, reward him with his fun thing. that was his reward, when the work is done is time to play or whatever, i was never comfortable with buying him toys or giving him candy as a reward for things he should be doing anyway.
both times, it was really more of a tool to help him remember the things he needed to get done, rather than a "reward" chart. eventually he knew the list by heart and quit needing to mark it off, which he got bored with, anyway.
i think at 3 1/2 it's too early to expect them to go an entire day keeping a rewards chart in mind. "oh, better not have a tantrum, mom won't give me a star on my rewards chart!" is just too much to ask. kids are so minute-by-minute and here-and-now. long term things like that, to me don't seem to work so great. i really feel like the best way to teach good behavior is to A. model it, and B. correct correct correct - teaching them every moment, which is our job. yes, we have to repeat things 100 times. (or more!) kind of part of the parenting gig....but i hope you find something that works for you.