The toxicity of cyanide, like any other poisonous gas, depends on concentration and the time of exposure. 150 ppm is potentially fatal within an hour; 300 ppm is fatal within a few minutes. (At room temperature, cyanide has a vapor pressure of 987,000 ppm, which means that it is capable of reaching this concentration if there is enough solid HCN present; in other words, the solid compound turns rapidly into vapor.) It is not clear what concentration was used, though witnesses report that the people inside died within 5 to 20 minutes – which doesn’t require a very high concentration.
According to this page (from which I also obtained the concentration data above), the two underground gas chambers had ventilation systems, and the two that were built above ground used natural ventilation. Even with natural ventilation, the concentration of HCN in the atmosphere immediately outside the gas chamber would be much, much smaller than within, because the gas diffuses quickly in the atmosphere. Once ventilation began (whether mechanical or natural), the concentration inside the chamber would drop very quickly. The actual time that it would take would depend on the amount of HCN introduced, the rate of ventilation, and the temperature. It is certainly plausible that the concentration of HCN in the gas chambers was below fatal levels within the times described by witnesses, particularly if the Sonderkommando used gas masks.
See here for more information on the columns, including some surviving photographs. Since the cyanide was introduced from outside, any HCN that escaped from the top of the column would quickly dissipate into the atmosphere. Most accounts of the homicidal process mention that those who handled the Zyklon crystals wore gas masks. A gas mask to counter HCN is relatively simple – all that is necessary is a compound, such as a nitrite, that oxidizes CN[sup]-[/sup] to cyanate, OCN[sup]-[/sup]. Additionally, cyanide poisoning can be reversed if treatment is given within a few minutes.
Finally, the Zyklon-B compound itself has some features that make it less likely that there were many accidental deaths resulting from its handling. (See here again.) There were two irritants in the compound – one that dissipated quickly, before the HCN concentration was life-threatening, and one that lingered. Those handing the gas, as well as the Sonderkommando who entered the gas chambers after the homicidal process, would have known from the presence of the warning compounds that there was gas present, and could have either worn a gas mask or left the area.
It seems unfortunate, in a way, that the gas used to murder perhaps a million people was not extremely dangerous to those who handled it. But these issues – the risk to those outside the chambers, and the time it took for the chambers to be cleared of gas – are important in countering Holocaust denial. Some deniers have claimed that the gas was so dangerous that it would have killed those who handled it, or that it lingered for so long (or that the bodies were so toxic afterwards) that it would have been impossible to kill as many people as historians claim were killed. The URL I linked to deals with these claims, and there are quite a few others.