The effect on our world is observable; the cause, not necessarily.
For example, consider the common Christian belief that Jesus was raised from the dead through the power of God.
As stated, hypothetically we could empirically observe and verify that Jesus had been raised from the dead. And, hypothetically for the purposes of this discussion, lets assume we observe this with absolute certainty; he was definitely dead, and is definitely later alive. There is no possibility that we are in error on either point.
But we still could not empirically observe and verify that this was worked through the power of God. We might conclude that we could not explain the resurrection of Jesus from our own understanding of the empirically observable world, and from there we might be tempted to conclude that it had indeed been worked by the power of God, or some other supernatural cause. But that, strictly speaking, would be a leap of faith which would be unsupported by any empirical evidence. Possibly our understanding of what it is to be dead, or what it is to be alive, is inaccurate. Our belief that the dead can never come to life in any natural way could be mistaken. We might be making exactly the same mistake as people who could not account for the phenomenon of lightning from their understanding of the empirically observable world, and therefore concluded that it had a supernatural cause.
In short, we might be confusing our understanding of the empirically-observable world with the reality of the empirically observable world. Which, of course, the scientific method requires us not to do. It’s inherent in every scientific proposition that it’s falsifiable; it could be wrong. In principle, however good our understanding of the natural world, it may still be wrong in some respect, and and if we correct that our good understanding would become better.
So, the resurrection of Jesus had happened and you and I had observed it, even if we believed or accepted it to have a supernatural cause we wouldn’t have observed the supernatural cause; we would be taking it on faith. If we could observe the cause of the resurrection, it wouldn’t be a supernatural cause.
Which means that, whatever criteria we have for believing or disbelieving that Jesus was raised through the power of God, the absence of any empirical evidence is a complete red herring. If this was not worked through the power of God, we would not expect to find empirical evidence that it was. If it was worked through the power of God, we would still not expect to find any empirical evidence that it was. The absence of empirical evidence tells us absolutely nothing at all, one way or the other, about the metaphysical claim, and those who mention it in this context are dragging in a complete red herring.
I think we need to be clear about the argument that Russell illustrated with his teapot. Russell wrote that, if it was claimed that a teapot orbited the sun between Earth and Mars, it would be nonsensical to ask him to accept the claim merely because there was no proof that there was no such teapot.
That argument is valid, and it is fully applicable to metaphysical claims. I can assert, say, that it is objectively wrong for Nazis to murder Jews, but I cannot demand that you should accept my claim merely because there is no proof to the contrary.
But you can’t validly extend this to make a stronger statement; that we must reject the proposition unless there is empirical evidence for it. A teapot would be empirically observable, and it is reasonable to ask what empirical evidence there is for a teapot in the orbit mentioned. But an ethical norm is not empirically observable, and it is nonsense to ask what empirical evidence there is for the postulated ethical norm. There might be good reasons for rejecting the postulated norm, but this certainly isn’t one.
The strong version of Russell’s teapot - that every claim must be rejected if not supported by empirical evidence - is not, SFAIK, a version that Russell himself ever advanced. He was, after all, a philosopher; he perfectly understood that empirical evidence has no relevance to metaphysical claims. Russell’s point was simply that the evidence against any claim is not a reason to accept it. But with respect to metaphysical claims the converse is also true; the absence of evidence is not a reason to reject it. Evidence, in the empirically-observable scientific sense, is just not a concept that has any application to metaphysical claims.