How do you know this? Did the neighbors explicitly tell you that they no longer care for him?
About 25 years ago, we lived in a neighborhood and had an inside/outside cat. She was outside mostly during the day, but would come and go, and normally showed up to come in for the night. Occasionally she might be gone for a day or two. She was chipped, but we quit putting a collar on her because it would get caught on fences, low tree limbs, etc.
One day we were in our front yard, my son was playing with our cat, as one of the neighbors (mom, dad, and daughter) were out for a walk.
Their daughter comes running into our yard, screaming “Fluffy, fluffy, fluffy!!”
The dad says, “hey that’s our cat that we adopted about 2 months ago.”
I said, “From the pound?”
He says, “no, she’s a stray that just came up to our house, and we let her in and fed her, and she just kept coming back.”
I told him that we had moved here about two years previously with the cat and that she was definitely ours and that she was chipped. And that she had apparently scammed them into feeding her, and that she was being well taken care of at our house.
So maybe, your cat is scamming you into letting her in and feeding her, while she’s getting fed at her original home as well.