There are three basic properties to consider for toughness. Strength, modulus, and anistropy.
Strength is the easiest to understand. Pull on a bar of “stuff” that is one inch square, and note how much force it takes until it breaks. For common steel (nails), this would be about 50,000 lbs. For high grade hardened steel it’d be around 250,000 lbs. Plastic in kid’s toys, about 5,000 lbs.
Modulus (rigidity) is pretty easy too. Do the pull-test again, but this time measure how much stretch per inch per 1000 lbs applied, and periodically release the load to see how much of the stretch is permanent.
A low modulus material will stretch farther than a high modulus one. Most hard materials do not permanently deform (yield), they just break - this is important! Glass and ceramics are in that category. Most soft materials behave almost like taffy after yielding (lengthening by tens or hundreds percent for steels and plastics).
Bear with me here, I’m just getting to the punchline now…
Energy is force x distance, so with the above two tests you can figure the strain energy or “modulus of toughness” for a material. This is energy per unit volume to break something. As you have observed, a hard material is strong - but not usually tough. When the math is done, a softer material can often absorb more energy before failing because it can stretch - even though it is “weaker” since it breaks under smaller loads.
Anisotropy makes things interesting, a lot of materials have “grain” to them or behave differently under different loadings. Concrete is pretty strong in compression, not so much at all in tension (soil is a more extreme example of the same thing - it’ll hold up a building but you can pull weeds by hand).
Manufacturing process often sows the seeds of failure - rolled or drawn metal stuff splits along the direction of rolling more easily, bent stuff tends to open cracks on the tension side (the “outside” of the bend), cast stuff usually cools unevenly and sets up funny stresses internally, scratches tend to become cracks after lots of load cycles…
Hope this wasn’t too redundant or boring.