Heat resistance is not the same as insulating. Things like silicone or leather are resistant to the heat, but they provide very little insulation. When protecting against small very hot items (like welding splatter, or holding a lightweight oven tray) they are fine, as they have enough mass to adsorb the heat without their temperature rising too much.
As noted above, cast iron has a lot of energy in it. Big heavy things. So the protection afforded by a silicone or leather glove isn’t going to work. It will simply be heated up by the iron to the point where it starts to burn you.
You need insulation. Padded oven mitts (but they are always too thin I find), dry towel (fluffier the better), or both. You don’t really need the high temperature resistance of leather welding gloves, you are not being spattered with molten metal. The skillet won’t set the towel on fire.