A deeply embedded foreign body will typically fester if it’s wood. The inflammatory reaction will soften the wood, and a small pocket of pus will typically eventually work its way to the surface. Wood is better than glass or metal at protecting bacteria from the body defenses.
Sometimes the whole thing will become walled off and remain as a small, firm, scar-like nodule.
On rare occasions, infection can find its way into adjacent bone, which simmers along indefinitely (osteomyelitis) and is very hard to cure non-surgically. Also rare, but possible, is extension of infection into a relatively avascular compartment such as a tendon sheath. Live germs in that space can create a signficant problem, especially in the hand.
Remote seeding from any active nidus of infection is possible, although also unlikely. For example, organisms seeded from an abscess in one area such as a hand might gain a foothold in another such as heart valve (usually a surface with some prior damage). However we get bacteremias all the time and the body defenses are very good at clearing them, so it’s highly unlikely a splinter to a finger will become systemically problematic without announcing itself locally first.