I’m currently teaching a class that deals with transportation security issues. One issue I haven’t quite been able to adequately grapple with is the following:
Security of the United States is an important priority of the federal government. Homeland Security has been tasked with making elements of the US transportation infrastructure more secure. In selecting, implementing, and evaluating mutiple policies (or plans) addressing security concerns, how does one determine the success (or failure) of a given policy or plan?
Does some aspect of risk assessment/management come into play when determing the success of failure of a given security policy/plan? Does modeling or simulation constitute an important element in deriving some metric with which to gauge the success/failure of a given policy/plan? Does public perception play any role in the evaluation process?
I am familiar with how one might go about implementing a particular security system and the steps one might undertake to implement such a system (involves risk assessment/management to some degree). But I’d like to hear from people who have some experience with security issues (in general) as to how one determines the success criteria for a good security system (and therefore, a potential way of evaluaitng a successful transportation policy or plan).
Since the purpose of security is primarily to make the secured feel safe, the answer to your question is to increase the effort until the secured start to complain about the cost in either money or convenience, or liberty or whatever is considered valuable at the moment. Trying to come up with some kind of objective measure is not going to work since the major variable is how safe the people feel-which changes independently of the security measures.
Thanks for the info - looks as if IOSS is a good place to start.
rbroome, you may be right to some extent. One of the questions I posed to my students was - should psychological considerations be an important criteria in evaluating the success (or failure) of a given security sytem? In other words, decisions relating to security success/failure may be, in part, based on (subjective) perception.
While evaluation may not be entirely objective, there should be some formal or systematic way with which to make an evaluation. After all, one can design a given security system (say, for a nuclear power research facility) that can identify threats, vulnerabilities, and negative consequences to such a facility. These can be put in the context of risk assessment/risk management. One way to evaulate the success of a given security system is how well is helps reduce the overall risk to the facility.
Have you read any of Bruce Schneier’s work on the subject? He has a lot to say about what doesn’t work and not much about what does, but he makes some good points about evaluating good security. One rather trivial point I like was his point that DHS would like to count the number of pocketknives and corkscrews confiscated from trustworthy Americans as an indicator of the success of their airport security measures. Schneier points out that almost all of these are in fact a failure (false positive) of the security policy.
I think rbroome has a good point, but I think it only applies to some security systems, not all. For example, much of airport security is what Schneier calls “security theater”, pointless in terms of real security but good at making people feel secure. This is important for the economic health of the airlines, but not for their actual security. On the other hand, a lot of security systems should be as transparent as possible. Many aspects of computer security, for example, have no bearing on whether the users feel secure. Many users don’t know enough to consider the issue of whether they feel secure or not (hence the problem in many self-administered situations) and security needs to make them secure without them noticing or getting in the way of usability too much.
I think proving security success objectively is not possible. It’s much like proving a negative. You can, however, prove security failures. This is why vulnerability testing is such an important part of many security policies. In the end, your test program depends on the competency of the testers, but at least looking for failures is a positive condition which can be demonstrated. In regards to transportation security, it should be pointed out that most airport security policies, especially bag screening, fail dismally when confronted with testing because the policies have been designed to make travellers feel secure but have not taken into account how threats actually happen.
He’s written many, but “Beyond Fear” is probably the one that deals most specifically with your interests. He also has a blog. Here’s one of his recent entries on airport screening.
But airport security is doing what we (at least the traveling public) wants. It makes them feel safe. It is equivalent to hiring nurses as cabin attendants back in the very early days.
Aircraft hijacking became obsolete about noon on Sept 11th-when pilots realized they could no longer afford to give up their planes to hijackers. All that theater in the terminals isn’t reducing the already zero chances of a hijacking. But it makes the traveling public feel better.
It is expensive though. In many ways besides the $ cost.