In the movie The Last Starfighter, the protagonist is selected as a starfighter pilot not by a general review of his educational and professional work experience combined with a behavioral interview and reference checks, but on the basis of his extraordinary performance on a skills test which had been disguised as a video game. To some extent, this is a nerd fantasy - to be selected for something prestigious based on the one (and possibly only) thing you do well.
Has such a strategy ever been used in real life to make real hiring decisions? Specifically, I’m looking for cases where experience, education, references, and personality take a back seat to raw performance on some sort of skills or knowledge evaluation. Bonus points if the candidates don’t even know they are taking a test, like in the movie.
For example, has a manager ever decided that, “I have three openings and five thousand applicants. It’ll take us months to review everyone’s resume, conduct behavioral interviews, and check all references, and we don’t have that kind of time. I’ll just book a convention center and call everyone in next Monday for a skills and knowledge test. I’ll take everyone who scored above 80% and rank them and make offers to the three that scored the highest.”
Yes, I’m well aware that such a strategy has serious downsides, but I can see how it might be a help in certain circumstances and might also have been tried by a manager as a creative solution to try to help a company fix obviously broken hiring procedures that were bringing in large numbers of candidates who couldn’t hack it in the end.
Picking a silly example, to be an MI5 surveillance officer one part of the test is a test on surveillance skills (map reading etc, I presume). I know this cause it says on the website, not because I have or ever would apply to be a surveillance officer.
Picking an example from my real life experience, when I wasn’t sure what I wanted to do for a job, one computer related one I applied to do asked me to program something (I forget what) in the first round in a language they made up.
p.s. I have a serious objection to behavioural interviews. They assume that your memory works in a certain way. Being something of an autodidact, I have an astonishing memory in certain ways, but an awful memory in others, and I just cannot answer the questions unless I have anticipated them in advance, because my memories of events in my life are not “indexed” to the things they ask about. In other words, to answer a question like “talk about a time you displayed leadership” or something like that, all I can do is go through things I have done in my head, basically randomly, and wonder about whether each one refers to a case of leadership or not. And of course even if it does there’s no reason to think it’s a good example.
Companies that “objectively” recruit using behavioural interviews and other quanititative approaches may indeed do better than those who interview using gut feeling alone, and I have no problem with that in principle if it can be demonstrated, but I do worry that those who need innovative talent rather than drones need to realise they will be missing out on a whole type of *useful *diversity.
It sounds like these were work opportunities where the test was a single prong in a multi-pronged evaluation process. Can you get hired as an MI5 surveillance officer solely by blowing the map reading/surveillance skills test out of the water, past experience or education be damned? I’ve been hired for a job where one stage of the process is an exam, but other things were apparently considered too, such as degree and work experience.
Before The Last Starfighter came out, there was an Urban Legend that the military was using video games (in video game emporia, not at home video devices, which were clunky back then) to screen for potential fighters. I doubbt if anyone believed it, but it was well-known back then, and the appearance of the trope in the film wasn’t a Great Revelation or a Clever Idea, but simply everyone recognizing that they changed the “they” who were spying on us from our own human military to Space Aliens.
Although i wouldn’t be surprised if a lot of training and perhaps some screening was done on simulators, I can’t beluieve that anyone was picking prospective pilots or gunners by watching how they performed on arcade video games.
I got my first job as a graphic designer due to a take home test which was basically an example assignment. I must have done particularly well because I was straight out of school with no real work experience.
I hire my receptionists that way. Yes, we take resumes and do an interview too, but ultimately you get the job or not based on test calls. We have two scenarios drawn from actual client calls and we provide a section of our procedures manual and redacted sections of the real client information sheets. If you can’t follow the instructions, apply the client information and respond to the client in a friendly and helpful manner, I don’t care what your qualifications are or how you did in the interview.
This year, we had some particularly interesting results.
One woman, with extensive experience and a stellar resume, decided to do the test calls her way, and even argued with us about our calling script. She did not get the job - I like constructive advice, but if you can’t even follow the directions long enough to get through the interview…
Another applicant has some experience, but it’s limited, and her educational background is lousy (even by the minimal standards of a receptionist position). But she handled the test calls with grace and efficiency. She’ll be starting next week.
Tangentially related: People are often called in to brainstorm for an organization (or military) in order to get people who think differently or outside of the box. As an example after (before?) 9-11, some govt agency assembled a bunch of people from Hollywood to brainstorm on ways that terrorists may attack the U.S. in incredibly destructive ways using very little materials or planning. They used this focus group due to thinking about action, explosions, and fear as they would know more. Now they weren’t permanently employed, but they would have been compensated for their time.
It’s really the best way to hire someone. It’s just not always that easy. There are a lot of skills which are too complicated to reduce to a simple test. Background and education are substitutes for a demonstration of a person’s actual abilities. A person’s work experience can do the same thing. Sometimes you can’t tell what a person actually did at another job, but if you can find out the details you have a measure of their ability. Even if there are required certifications or educational requirements for a job, when you have to choose among candidates, you can go with the one who can best demonstrate their skills.
Very true, but in the reality of hiring, sometimes you have to make do with what you can afford, what is pleasing to your managers, and what seems to work. A lot of software development jobs seem to take a hybrid approach with a skills and/or knowledge test as well as a review of the candidate’s background with neither one overriding the other, and, in theory, a candidate who did poorly on the skills test might get the job by having a stellar educational and work history, but there’s only so much that can substitute - if you do flat out terrible on the test, no number of years of experience will make up for it. I thought the Google skills challenges were just to get you into the door as a candidate - you weren’t guaranteed a job based on passing the test. Are there any companies that hire based on raw score on a skills or knowledge test and basically ignore whether or not you have a MS with 10 years of experience or three quarters of a BS with no experience? Yes, it excludes part of the person from evaluation but perhaps in the hands of a skilled manager who can design the right test, it might improve the quality of new hires.
I wasn’t hired by Microsoft, but I got myself an all expenses paid trip to Seattle my senior year of college probably because I nailed a particularly obscure SQL question during an on-campus interview with them, which by coincidence, had just been covered that day (or a recent day) in my Databases class.
My attempts at solving some algorithmic questions on a whiteboard at the Microsoft campus didn’t quite go as well. One involved using bitwise C operators (which I was only vaguely aware of) and the other was writing out a sorting algorithm.
I was hired by AT&T out of college basically (I assume) on my score on a programming aptitude test.
They did interview me, but the job was as a COBOL programmer in an IBM 360 environment. I knew neither COBOL (we programmed in PASCAL at school (1981)) or MVS (the interviewer mentioned JCL, I asked her what that was).
When I got to the first day of training class, everyone was talking about how hard the test was - it had been a story about an assistant who could only follow simple instructions, which was (I thought) pretty obviously a stand in for a programming language.