Friday, July 23, 2010

The question no one has asked yet about the language test for new immigrants

I'm not in a position to evaluate the test itself. It pings my bad idea radar, but I can't make any definitive statements without more sample questions. And, of course, the idea of not having an exemption for people whose first (and perhaps only) language is already English or French is just as silly as it sounds. But after looking at the one sample question provided by the Toronto Star, the first question that comes to mind is:

Who stands to profit from this test?

The question provided shows a pie chart of Tomoko's expenses, broken down as follows:

Rent and food = 45%
Study materials = 25%
Clothes = 15%
Entertainment = 15%

Then some sentences about the pie chart are provided, and the candidate is instructed to correct the sentences. The first sentence given is:

Tomoko spends an equal amount of money on rent, food, study materials and entertainment.


The correct answer is to change "equal" to "unequal". But I didn't get that right away, and I don't know if it ever would have occurred to me to answer that way. I was reading the sentence wondering "Is this sentence trying to say rent=food=study materials=entertainment? Because rent and food aren't broken down separately. In any case, study materials is more than entertainment. So what are they asking me?" I probably would have ended up writing something true but far more complicated, like "Tomoko spends more on rent and food combined than on study materials, and more on study materials than on entertainment."

Part of the reason I didn't see the desired answer is because food and rent are a single item on the pie chart but listed as two separate, comma-delineated items in the question, which made me think the question is asking me for something more complicated than it is, on the assumption that this change is meaningful. In most language tests I've taken, when something is different in two different places on the same page, that's meaningful. (And I'd love to know the story of how that happened! If I or anyone on my team had been translating it, we would never have let that through and would have pointed it out to the client as something that might prevent candidates from being evaluated perfectly fairly.)

Another part of the reason is that in the many language tests and other tests I've taken and real-life situations I've been in, equal vs. unequal is never really a factor. More vs. less is a factor and specific numbers are a factor, but I have never in my life needed to think about equal vs. unequal in this sort of way. It's just never been the sort of thing that is meaningful enough to be on a test because it's so excessively obvious.

I understand the words perfectly, of course. It's just not within the scope of my experience with Things That A Test Might Be Asking Me. Understand, this sample was the very first time I've ever seen the IELTS, and this equal vs. unequal question was the very first IELTS question I ever saw. I didn't know what to expect, I didn't know if it would be insultingly easily or humblingly difficult, I didn't know what kinds of things they were looking for or what kinds of skills they were looking to test. All I had going in was a lifetime as a native speaker of English and decades of experience as a student of languages and as a student in general. It was not a failure of my English skills, it was a result of my lack of familiarity with this particular test.

"But you're just a mildly interested passer-by clicking on an internet link," you might be thinking, "In real life people prepare for tests!"

Yeah, that's why I'm wondering who profits.

In googling about this test, I found a lot of things for sale. Sample tests, exercise booklets, preparation kits, tutoring services - often at prices that would put a significant dent in a newcomer's budget. Free sample tests (some of dubious quality) certainly do exist, but for-pay materials fall into one's lap far more readily.

People do tend to get significantly lower marks when taking a test sight unseen than when they know what to expect from the test. For example, the result of my first professionally-administered IQ test was 135. Subsequent tests (both professionally-administered and not) clocked in at 150. During the first test, I just stared at the memory test pictures for the designated amount of time and struggled through the questions. In subsequent tests I knew they'd be asking things like "How many bluebirds are there?" or "What time does the store open?", so I was able to focus on those things and make good use of my memorizing time. The first time around, I guessed on every single memory question. Now, I systematically memorize the exact things they're looking for and get every question right with certainty. Familiarity with the test itself makes a massive difference, even if the candidate's skills level is the same.

And there seems to be a huge for-profit industry out there charging money to make people familiar with the IELTS test. If I were an investigative reporter looking for a juicy story, or a political partisan intent on bringing the current government down, I'd be digging into seeing exactly who stands to pocket these profits. If I were a member of the current government trying to make this policy look credible, I'd be working on making test preparation materials readily available at no cost, and/or work on making sure every single question in the version of the test administered to our immigrants is so clear and unambiguous in its expectations that there's no penalty for never having seen the test before.

Update: This is an interesting development. Language Log has determined that the sample test provided by the Star is not, in fact, a typical IELTS question. It seems it's an excerpt from unofficial training materials.

Several questions remain: So what does an actual IELTS question look like? Given that native speakers and second-language speakers make different kinds of mistakes, can the IELTS fairly and usefully assess native speakers? And what is the motivation behind suddenly testing native speakers? Is there a particular existing problem that this is meant to address? What is gained?

9 comments:

Christopher said...

I'm of the opinion that generally immigration restrictions should be lowered not increased. I think that people should be able to move freely about so tests like this just don't work for me. Besides, what if someone just can't read? Hell, I'm sure there are plenty of english speakers who couldn't figure that out.

impudent strumpet said...

One thing that occurred to me but wasn't relevant enough to put in the original post is: could Eddie Izzard pass the test?

He's an intelligent, articulate, successful native speaker of English, and he also happens to be dyslexic. I don't know anything about how well he tests, but he has mentioned in his act that he reads slowly and spells poorly. (He mentioned in one of the shows I saw that he always gets GLBTQ wrong - misses letters, adds extra letters - specifically because of dyslexia.)

I don't know if they have any special accommodations for learning disabilities (or, for that matter, blindness? deafness?). But as a Canadian I desperately hope they haven't created a situation where a person is disqualified from immigrating because they have exactly the same language skills as Eddie Izzard.

laura k said...

I agree with Christopher, of course. But even beyond that, there must be something I'm missing. Because immigrants to Canada, with the exception of refugees and family reunification folks, must be competent in one of the official languages. That has been the case for many many years. People seem to be approaching this question as if it's something new.

I think I'm missing something, perhaps you will fill me in.

impudent strumpet said...

It's very new, started after you completed your citizenship process. I <a href='http://www.thestar.com/news/canada/article/838085--all-immigrants-face-mandatory-language-test">just heard about it</a> in the past few days.

laura k said...

So the difference is that native speakers of English and French cannot opt out of the test. The language compentency requirement itself is not new.

impudent strumpet said...

So they've always had this for-profit test? And no one has, like, noticed?

laura k said...

I don't know if it's the same test. But prospective immigrants who are not native speakers of English or French have had to take a proficiency test. That's been the case for years. Native speakers of one language - they use the expression "mother tongue" - get a pass.

Refugees and family unification - like an elderly parent coming to live with their child's family - also get a pass.

But I totally don't know if it's been the same test.

impudent strumpet said...

Even if it is the same test, expanding it to native speakers makes me even more suspicious of the profit motive, because what argument could there possibly be for creating a situation where people who are already linguistically integrated might be rejected on a technicality?

If I were government communications, I'd be putting out specific real-life problems that this is meant to address. If I were opposition, I'd get expert opinions on whether this test is suitable for native speakers, because we have completely different sets of problems in our mother tongue.

laura k said...

Yeah. I agree. I wish I could write about this now. Don't know if I'll have time before I go away, probably not.