As a retired police officer, I suggest that he's just a grouch; neither mental or criminal. I would be cautious around him, ie. not try to talk with him or get close to him. But I wouldn't worry about his being mental or criminal. He's probably just a grouch.
I urge you to talk with your son about this. He may have more insight that will calm your fears. I also think having your son talk with him, out of uniform and as a civilian, when he can do so casually, is a good idea. By doing so he'll get a feel for what kind of person he is. Have your son be neighborly.
I always suggest getting to know your neighbors. If this were happening to me, even before I was an officer, I would've stopped and asked him point blank what the matter was. I would be polite but direct. I'd say, you sound angry and I want to know if I should avoid you or are you just having a bad day? Did I misunderstand you? Maybe you weren't talking to me. I don't want to have ill feelings with my neighbors. That sort of thing.
You can choose to live in fear or to face the situation and get it resolved. If he replies with more hostility you'll know to avoid him on your walks.
I suggest that the tennis shoes and the hammock may just be coincidences. It sounded like you were saying the shoes may have been there for some time.
I might also talk with him about the hammock. I'd say someone cut our hammock. Have you seen anyone lurking around our house? It's a neighborly thing to let your neighbors know when things like this happen so that they can protect their property. This would be a good opening for your son to begin his casual conversation with the man.
You'll know more about who he is and whether or not you have something to fear by trying to communicate with him. Let your son make an evaluation based on his training and experience. Retaliation is not likely to happen but if it does, then you have something with which to deal.
It's highly unlikely that he'll attempt to injure you. He didn't threaten you, after all. He could've said, talk to me again and you'll be sorry. Then you'd know to not talk with him and be fairly certain there won't be a problem because you're not going to talk to him again.
I also suggest you were, unintentionally egging him on. I probably wouldn't have said anything more after his "you don't have to talk with me if you don't want to." I'd say "sorry to have interrupted you" and walked off. I might say something like, I'd just say I thought you were talking to me and I'm sorry to intrude. "What cha want" said in a nice tone is still getting into his business especially if he weren't talking to you and he is a grouch.
Again, the choice is yours. Face this and find out who he is about or live in fear.
BTW even if he does have a mental illness he is probably not dangerous.