I think there’s another aspect of the legal system that people don’t think about; segregation. We define crimes, enact laws, and imprison people for the purpose of separating people who commit those crimes from the rest of society.
Some crimes are just unique acts. But most crimes are essentially a lifestyle choice; most crimes are committed by people who commit those crimes on a regular ongoing basis. If you take the people who commit crimes on a regular ongoing basis and separate them from society by imprisoning them, you are going to reduce the amount of crimes that occur. This is true even if there is no deterrent or rehabilitative effect.