Nuke 'em from orbit. It’s the only way to be sure.
But seriously, maybe call an exterminator? It’s a one time cost and you can get their expertise about how to do it yourself in the future if (more likely when) they show up again.
As an amateur, dousing the gaps between slats with wasp/bee spray either during the day or after sundown (sunrise/sunset are bad times) has generally worked, though they do eventually come back. Uncomfortable or not, long sleeves/pants (no real need for bulky clothing) and keep as much distance as possible. Have that evacuation plan in place in case they get riled.