This is a follow up to my post on anecdotal information, and it’s a story of how close I came to slamming the door on a whole bunch of great students. A little erroneous idea of mine ran amok.
I was relatively new in my role as Registrar at Briercrest College and was generally frustrated at how late people were to submit necessary documents and information. I had a few colleagues who were very impressive researchers and academics – generally quite powerful people in the College. We were talking about the relationship between taking care of business and being successful, and as Registrar, I made the mental leap that students who took care of business by applying early must therefore be our most successful students. The corollary must also be true, I mused, late applicants must be our worst students. As further “proof”, almost all our students on Academic Probation were procrastinators and therefore they must have been late applicants.
Armed with my new anecdotal information, I got on my high horse and proposed that we set an earlier application deadline and have a hard-and-fast rule that no late applications would be considered because these students had no chance of being successful.
I took this proposal to our VP, Dr. Mark Lee (one of my heroes and still a good friend) for his input and approval. He took a look at it and asked me one question: “Do you have any data to support your conclusion that late applicants are our worst students? I’m somewhat skeptical of the idea.”
I smiled. This was going to be easy.
But you’ve heard me say that before…
I walked back down the wide hall to my office and one of my staff and I sat down to write the queries and reports for the database. But what I saw on the screen just couldn’t be correct. I said to my staff, “Mark Twain said, ‘There are lies, damn lies, and then there are statistics.’ Let’s rewrite this report to make it correct.”
After rewriting the report three times, we simply couldn’t make it speak the truth. Damn statistics! Mark Twain also said, “Figures often beguile me, particularly when I have the arranging of them myself…”
I must have arranged the figures all wrong because the reports were the exact opposite of what I knew to be the truth. Why were these late applicants in our top 25% of students when they should be in the bottom? It didn’t make sense.
So I thought I would track our Academic Probation students and see what their application dates were. Hmmm…. There must really be a problem with our report, or our database, or something, I thought, because these students were among our earliest applicants. Time to bring in the big guns.
I called Dr. Mark Lee and we went over the data, double-checking everything. Finally, he sat back and stroked his goatee, “I guess you have to accept it – our better students apply later than you think, and our worst students apply the earliest.” he said. “I think you’d better stop accepting early applications!” he smirked.
I let out a deflated sigh. Damn statistics!
I had been fooled by the power of assumption and anecdote. I had been schooled by the power of hard data and someone who knew how to ask the right questions.
It only took me another 10 years to figure out that I could fight fire with fire. I could use the power of assumptions, anecdotes, and powerful orators to fight lies.
But I’m a slow learner.