If you are given a falsehood, and then you can also prove that it is in fact false, you have a contradiction, from which, in conventional formal logic, anything can be proven (this fact is referred to as ex contradictione quodlibet, among a million other names). Nothing more really needs to be said about this other than that this is the rule in conventional formal logic: “A implies B” is considered, by conventional definition*, to hold in every case except where A is true and B is false; therefore, if A is actually false, then it implies everything. So if A is provably false, then everything can be proven from it. That’s just the conventional way one sets up a formal logic.
But one needn’t use such a convention in one’s formalization. There are systems called paraconsistent logics, where not everything is provable from a contradiction. Of particular importance are the relevance logics, which try to only ascribe truth to “A implies B” when there is some sort of relevant connection between the statements A and B to carry the force of the implication, so to speak. These logics aren’t the standard one (classical logic), but they definitely have their uses.
*: As for this convention regarding “A implies B” (the convention of material implication), why would one want to adopt it? The best motivation is probably out of a desire for truth-functional semantics. Suppose you want the truth value of “A implies B” to be solely determined by the truth values of A and of B, with all truth values as either True or False. Well, the statements “Bush is a husband implies Bush is a male”, “Bush is a Parisian implies Bush is a Frenchman”, and “Bush is a Californian implies Bush is an American” are all pretty clearly true, while the statement “Bush is an American implies Bush is a Frenchman” is pretty clearly false. Thus, we see that “True implies True”, “False implies False”, and “False implies True” all come out to True, while “True implies False” comes out to False, fixing for us the definition of the material conditional. [Note that, as a result, one can’t reasonably give a truth-functional semantics to paraconsistent/relevance logics].