sounds like you're trying to teach him how to reason and make good decisions. He's too young for this. His brain development is not to the point that he can reason. Consistent and immediate consequences teach him how to behave. It's if you do this, this will happen. At this age he isn't able to figure out that if he does this, this will happen but if he does this, that will happen, or etc. He acts impulsively, without thinking. Don't ask him for his choices. Tell him the choice he made was wrong. He'll understand this.
I agree that you're setting him up for believing he can manipulate you. You want to know his reasons. So if he can figure out the "right" reason he'll be free. It's too confusing when you want to know why he did something. It doesn't matter why. It only matters that he did or did not do something and that he has a consequence consistently, unemotionally, and immediately given.
Along this same line, don't ask him what he did when you know what he did. Say, "you didn't put your toys away. Do it now or go to your room." Going to his room is the consequence for not doing it. Then when he's able to say he's sorry and put away the toys he can come out of his room.
Don't argue with him trying to convince him that you know he's lying. It only causes him to keep repeating his lie which reinforces lying. If he doesn't already believe it's the truth he will after repeating it several times. When he doesn't tell the truth tell him you know that's not true and go from there. No discussion. At this age deal with what he did or didn't do that was wrong and ignore the lie.
He's at the age when he's learning about the difference between the truth and a lie. He doesn't know the difference between fantasy and reality. When you tell him that what he's said is not the truth you are teaching him the difference. Not believing him is the consequence.
An example of how preschoolers, even into the 2nd and 3rd grade don't understand about fantasy and truth is how scared they get by their dreams, by even mildly scary movies and stories. They feel real to them.
When he lies he may even think he's telling the truth because what he says is the way he wants it to be. For this reason be careful that his consequences for misbehavior is appropriate for his age and given in love. Fear of mom's, dad's anger can create the need to lie.
I respond when this happens with my granddaughter with "that's a great story but it's not the truth." I don't argue with her. We don't have a discussion. I tell her this is the way it is and this is what is going to happen now.
I never ask her if she did something when I already know she did, thus eliminating her need to lie.
And I never make her misbehavior a "big deal." It was something she should or should not have done and there is a consequence for it. I try to make the consequence fit the behavior as much as possible. They call it natural consequences. I've found sending a child to their room until they can behave to fit the natural consequence criteria frequently. I don't want to be in the same room with someone misbehaving. And...the child needs time to regain their good behavior.
When she was 5 she said that her aunt lived next to the school. I told her that wasn't true and she told a creative story about how it was true. I finally thought to say, "that is a very nice story and I wish it were true." She agreed that she wished it was true. Thirty minutes later she said the same thing. I could tell her she was lying or I could validate her wish to have her aunt live next to the school and let her know it was a good fantasy and not real. She didn't need to be disciplined for her creative thinking. She did need to know that her story is a fantasy and not reality.