Without this my hand would indeed go through my desk. But it would also go through everything else. And everything else would go through my hand. And so on. So, what would we end up with? A big ball of mass?
I used to wonder about this back in catholic school. The nuns said that in heaven you could pass through walls. If that were true, we’d (us angels :)) would all end up in the center of the earth.
Do not ask a nun questions like this.