Validity of Graphology

Cecil's Columns/Staff Reports
41

Bill H. wants a page of text. If the first paragraph of Cecil’s response isn’t enough, how about the first 2 paragraphs? There really is a fair amount there. That might be about a page when written out.

The use of a non-trained control is a “placebo effect control”. You claim to have significant experience and some training. This person would have no clue of what to look for to qualify traits. What is “angry writing”? You know what that means, but I don’t, and I imagine most people don’t.

I figure we have two controls. We have the pure chance by numbers, and we have the untrained person duplicating the effort.

If you would prefer Chronos be a participant, then let’s select someone else as the intermediary. I picked him because he suggested we run the test, and because of his previous experience as a staff member.

Quantities of participants, 10 to 20 was my thought as well. This isn’t the end all/be all of graphology testing, just evaluating the claim Bill H. made about being able to match handwriting to the person who wrote it. Ramifications of what that means to reading personality can be determined later, if he actually passes the test.

The suggestion for extra names in the list doesn’t mean fake samples. If I have a fill in the blank situation and a list of terms, if you first fill in all you know you will sometimes be left with only one or two to guess, which is easier when all the words have to be used. With a dummy name or two in the list, you’re assured that the last matchup is not a default “that’s who was left”, but an actual effort to match.

This is the one that gets me. Isn’t the similarity issue what we’re trying to test?

My proposal is we come up with a possible list of participants, say 30 names. From that list, Bill H. can cull the list to 20, before the samples are mailed. Once the list is solidified, the participants mail the samples (copied first 2 paragraphs of Cecil’s response in this column) to our intermediary. The intermediary takes the samples, applies some code number to each of the samples, and sends them to Bill H. A second copy is sent to C K Dexter Haven. Bill takes as long as he needs and then between reading the samples and communicating via email with posters, he performs the matching, and posts his results in the thread. Dex does his matching without emailing the posters. After Bill posts, Dex posts his matches. Then the intermediary follows up with the correct decoding.

Note in the above I didn’t include dummy names for the list. We could cull the list of names to 22, then have the intermediary pick two arbitrarily to remove and not send their samples. That’s an alternate option.

With the Dex control, should he be allowed to communicate with the posters, too? He’s not looking for personality, but other traits to observe. Like if Ducky dot’s her I’s with little hearts, it’s pretty obvious the sample is from a girl. :wink:

Achernar, what are IPIP-NEO scores?

42

Well, I’m a little skeptical of the utility of a “non-expert control,” too. If Bill H. is correct and you can match handwriting to personality traits, then you would certainly expect an expert to outperform a novice. But by how much? I would maintain that most people can differentiate between masculine and feminine writing. Most people know the tale about John Hancock’s signature being large because of his ego. Bill H. himself just told you that passionate people press hard when writing. So even a novice should be able to outperform random chance.

Problem here, of course, is that we don’t know what a reasonable score would be for a novice. Having a (relatively) high-scoring novice could mean that he or she is applying, counsciously or uncounsciously, simple elements of graphology. Or it could mean that he or she is matching, counsciously or uncounsciously, handwriting to physical traits. Or it could mean that there was actually something about the paper used that indicates who wrote what. Or whatever. And having a (relatively) low-scoring novice would show nothing more than a statistical calculation would. So having an additional person attempt to match samples adds nothing to the experiment.

I don’t think so. Bill H. is saying, I think, that he’s afraid of getting a sample of people who are too similar in personality (i.e., people that tend to read CCC). Similar people would then send in similar writing samples.

43

I understand what you’re saying. And if you’re right, that an additional person adds nothing to the experiment, then all that means is that the experiment is still useless.

The IPIP-NEO is a quantitative personality test that’s free. Any other such test would be just as good, I think.

44

For paper, I would recommend that we use blank 8.5" by 11" printer paper, since that’s the sort most likely to be available to the largest portion of our sample group. Unfortunately, this probably rules out international samples, but then, I suspect that a sample from someone whose first language isn’t English would already contain some obvious cues, so this probably only loses us the UK and Ireland.

Extra names (without samples to match) seem methodologically plausible, but if we do that, someone other than me is going to have to do the analysis. I know how to check a permutation, but I don’t know how to check a choice + permutation.

We might also allow Bill H. to list a confidence rating for each of his samples, and possibly a second choice as well. Again, if we go that route, I’m going to bow out of the statistical analysis.

As for an untrained control: If we go that route, we want the control to have access to exactly as much information as the test subject. Perhaps we might even have Bill H. and the control CC each other on all e-mails, as well as all correspondents. We want to minimize the number of variables, here: We don’t want to compare Bill H.'s method of determining personality to another method; we want to compare it to using no method at all. Should the initial test be successful, further tests might compare to other methods. But first things first. Note also, that we’re not just comparing the subject to the control; we’re also comparing both to a statistical expectation. If both do better than statistically expected, well then that tells us something too.

Incidentally, all of the correspondence should probably be made public afterwards.

And although I would approve, on general principles, having each participant only writing the sample once and then photocopying them, I know of no method of copying them which would preserve all of the information which Bill H. says is relevant. Pressure and choice of writing implement, in particular, would be lost. Again, we want to minimize variables, so subject and control should have access to the same amount of information. This would be more closely approximated by two original samples from each participant, than by one photocopied sample. This would introduce some variance, of course, but the length of the sample and the random assignation of the samples to subject and control should help alleiviate that.

I don’t know how many participants we would need to make a good test. With ten participants, there would be 10! possible permutations, or over 3 million, so all of them correctly matched would definitely be statistically significant. But we don’t know how strong the effect we’re looking for is. If Bill H. is any better than random chance, then a sufficiently large sample will reveal that. But if he’s only, say, one percent better, then it would take an awfully large sample.

45

Irishman, I understand your concern, but I need to mandate this part:

That means that we must have a pool of people write samples with the expectation that some of them will be discarded after I see their writing. Some I may discard before the experiment, but honestly, I’m only expecting to eliminate people like Dr CobWeb, who I don’t trust to be honest in the writing he gives me or the answers he gives me in emails.

In fact, not to be a wet blanket but I’ll go one step further. It’s entirely possible - based on the expected test population - that I’ll get 30 samples that are similar enough that I’m not comfortable narrowing down even to ten, and I may throw up my hands. I don’t expect this, but I’m being frank. The average reader of the CCC column is an extremely narrow population compared to the average english writing population.

As an example, I expect most readers of this to have great “attention to detail”, something that is very varied in an average population. There are other examples of course. And if I end up with a pool where all of the candidates share those features, I will fail.

46

(continued)
But… if in the off chance it happens that the samples all contain very similar traits, I can show after the fact what features were similar, and give examples of others in the English writing population that differ.

Honestly, I don’t think there’s a way to completely alleviate my concern here (i.e. to get an “average population” and still get “members from the SDMB”), but I the best thing to do is just try and see how it goes. After I see the samples, I’ll have a better feel for how things will go.

Also… one of my joys in this hobby is not only analyzing but collecting past analyses I’ve done both for the sake of reference as well as a “souvenir.” I’m accepting copied text for the sake of this experiment, but that actually isn’t very interesting to me. Therefore, I’d like each participant to not only write the copied text for the experiment, but also write me a signed letter (about anything) of a couple paragraphs. This letter will not be used as part of the experiment, but can be sent to the designated tester to be sent to me after the experiment is over.

47

By the way, we’ve really gotten off the original question.

Come on Cecil. You made a pretty broad claim. That 200+ objective scientific studies have shown this is nonsense. You didn’t give an example of why it was nonsense, you didn’t even give empirical evidence. You handed it off to 200+ teams of objective, scietific experts. Now…

Where’s the beef?

48

I am new to this site and have been following the discussion here with interest. Anyway, I volunteer to submit a writing sample.
It occurs to me that people with an interest in graphology might write/answer in such a way as to make things easy for Bill H. I state for the record that I have little knowledge of graphology and am skeptical.

49

Okay, well, since I have a writing sample on my webpage, I’m disqualified anyway. If Chronos wants to play then I’ll volunteer to moderate. But I should point out that I’m very dissatisfied with the setup of this experiment, and if it goes as currently planned, I will find its results completely useless. But I can still help out, if it will help other people. :slight_smile:

Bill H., would you please look at the cites that Duck Duck Goose mentioned at the beginning of this thread? Don’t discount them just because someone with a bias cited them; even wholly respected and wholly correct sources get referenced by people with agendas. Seriously, you claim to be scientific, so you must realize how silly your dismissal was.

50

I’ve been thinking, and I think it might help if we have a clear statement of what we are actually testing, and what we are not testing, and the ramifications of the test. This is so everyone agrees the point of the test, and that we assure each other that the test procedure does, in fact, accomplish what we wish.

Bill H. claims to be able to match handwriting samples to their writers based upon the personality of the writers, and not the content of the writing samples.

He does not claim to be able to diagnose illness, or to predict whether the person will steal from an potential employer, or assess guilt or innocence regarding crimes.

The claim is that enough personality traits of the writer are evident in the writing style independent of content that the writer can be identified.

The ramifications for graphology as a whole: a successful test for Bill H. might indicate there are some personality traits that are evident in the handwriting independent of content. It does not say anything about the validity of handwriting analysis as a hiring tool for evaluating an employee’s fitness for a job, ability to do the work, or honesty and integrity.

This is my attempt. Comments?

51

I have a strong aversion to a test that attempts to discern “personality.” Look at a typical horoscope any day of the week, and you’ll probably find 3/4ths of them describe your personality perfectly.

Terms that Bill H has mentioned, like “sensitive to criticism” or “creative”… OK, is there anyone here who would call themselves NOT “sensitive to criticism”? Anyone who would call themselves NOt “creative”? In short, these “personality test” adjectives are complete bovine excrement.

Bill H begins this thread by commenting that he uses graphology as an “amusing party trick.” I was at a party that had a tarot reader to read me, and it was amusing, and about as accurate (and ambiguous) as I would expect.

If Bill thinks that he can identify WHO wrote which sample, that’d be verifiable. But I don’t know how we verify ambiguous and somewhat universal character traits. I mean, after all, every one of us has moments of being “aggressive” and moments of being “passive”, for instance, so a psalmist – er, sorry, graphologist who tells me that I am sometimes aggressive will have hit the mark, and another who tells me I am sometimes passive will ditto.

I’m not much interested in going to all this trouble for a bunch of meaningless “personality” adjectives. I am interested if Bill H thinks that he can actually identify people based on handwriting samples and on his interpretation of the personality from their posts.

52

oh.

because of the ink color you mean, or because i’m already prejudiced in favor of the efficacy of graphologists?

give me another chance!

53

Just curious, but does this include things like the NEO PI-R?

54

If you need more subjects, I’m in, provided the test is about matching up handwriting styles with names, not providing personality descriptions.

55

hmmmmm…

We started off where I asked for cites to a pretty strong claim. I didn’t say I had cites to back me; I’ve been clear I don’t. But cites were claimed, and I think it’s fair to see them.

Then we got into this discussion on a potential test ourselves. Others may not realize that what I’ve signed up to do here will require me to dedicate perhaps 40 hours to. Perhaps more. I’m not a man with a lot of spare time. Actually, very little. But it could be amusing, and if it fills some scientific void amongst friends, why not?

But then, there’s these:

Achernar wrote

C K Dexter Haven wrote

That’s from two of the key people who’ve defined the rules to this point.

I didn’t offer to do this test because I have something to prove or because I’ll benefit in any way.

I offered to do it in the hopes it’ll apply some science to an area in question. But if others don’t see any scientific validity to it, I’m sure not going to waste a working week of my life to it.

And most importantly, it’s pretty annoying that I’m backed in this corner when I made no claims of hordes of “objective scientific” experts backing me. Where’s the cites?

56

Cicada2003 wrote

Friend, you disqualified yourself because you told me a key attribute about your writing. So now, if you are a subject, and you’re the only one with custom ink and a fountain pen, I’ll say “that one is Cicada2003’s,” and I and everyone else will know I didn’t match those by matching your personality to your handwriting.

You can’t tell the magician what your card is. It wrecks the trick.

57

No, seriously, Bill. DDG provided 23 cites in the very first reply to you in this thread. Did you look at any of them?

58

I’m afraid we’re going to have to wait for Cecil to jump in here. But he’s failed to do so before.

59

And by the way, I’m a little tired of comments like this from people like C K Dexter Haven, who should really know better:

You seem very confident. Where’s that from? You’ve been clear you have no experience with this. So do you also have access to that magic briefcase full of names?

And if you have neither (experience nor access to reputable studies) then how can you be so certain?

But most importantly, even if you are certain, you degrade your argument enormously by stooping to name calling. You put yourself on the level of that Cobweb guy.

60

Archarner, actually I did completely read and address both of the cites that Duck Duck Goose was kind enough to look up and post. To reiterate:

  • Both cites are specifically about the applicability of handwriting analysis to potential quality of an employee. I’ve said that I don’t think it is applicable. That’s not under debate in this thread.

  • The main study cited by both sites (Rafaeli & Klimosky, 1983) was about how salespeople fared after being employed, and comparing them to a handwriting analysts prediction. Not only do I not think that handwriting is a good predictor of capability as an employee, but this is so specific even within that (whether the candidate will sell or not) that no reasonable person would claim it.

  • The first site has heavily biased (as I pointed out in some of their other positions) on the subject of privacy in the workplace. Which may be a valid social position, but it’s an indisputable strong bias. I wouldn’t expect anyone to accept a “scientific paper” I presented that was sponsored by a psychics group.

Also, though I very much appreciated DDG’s research, it was not 23 cites, it was through 2. I’m sorry, but I’m not going to the library to dig through old magazines, and I’m not going to dig through a list that hasn’t already been at least glanced at either.

And 2 is a very different number from 200+.