On Percentile Ranking

Some folks who have not been following the earlier threads of discussion have misinterpreted my last post. That post was not a defence of the percentile ranking system of comparing Board marks. It was merely to refute the assertion of Dheeraj (and others, by the way) that “They gave a report which said that more studies needed to be done with data from more boards for more years.
This had two problems. One, MHRD would have taken a long time to get all this data. ….”

I had argued that the ISI report had not stated that more studies would need to be done before one could say that percentile ranking can be used for comparison.

Now,  the claim is that if the two assumptions are made, then, percentile ranking can be used for comparison. Actually there is only one assumption. The first assumption,  to quote the ISI report (not my words!), “Aggregate scores are expected to increase from less meritorious to more meritorious students in any particular subject”, is a mere technical one, which is saying that marks are not given randomly. The student who does better, gets more marks.   Clearly, if this is questioned, Board marks cannot be used for anything. It is the second assumption that is the basis for asserting that we can use percentile ranks for comparison. And that is, “Merit distribution is the same in all boards.” .  Some are asking, what is merit? How do you define it?  I cannot do much more but to suggest synonyms for merit in this context: innate ability, intelligence.

Some have questioned the above assumption. Fine. After all, it is an assumption. It is an assumption that cannot be “proved”. One argument against it has been that some of the Boards are so small, that this assumption will surely not hold. I have tried to argue that it is the size of the base population that the Board represents, that should be looked at. The numbers will not be so small then. There may be an exception or two (a Sanskrit Board exists, I am told), but this will introduce very few errors, if any.

The point I wish to make in this post is the following: if Board results across the 42 Boards of the country are to be compared in any form, THE ONLY REASONABLE way is to use percentile ranks. Nobody has suggested , to the best of my knowledge, any other method in any of the comments, posts, articles, etc. that have come out in the last three months.

I have also tried to argue that it is reasonable to do so because the errors introduced by this method, if any, are no different than the errors introduced in other  stages of any admission process: question paper setting, evaluation (even machine evaluation), tie-breaking rules, state of mind of the candidate on the exam day, the health of the candidate on the exam day, and so on.

I have also argued that, because of the bunching effect, at least for the “good” students, the difference in marks due to Board results will be very small. The candidate at the bottom of the top 10% will be 5 marks (with 50% weightage) away from the top candidate, and probably 3-4 marks away from the last candidate who qualifies on the basis of Board marks alone. So he has a very good chance to make this up in the exams. In fact, as opposed to using any cut-off, EVERYONE has a chance., no matter what his Board marks are.

There has been some objections of combining percentile ranks with marks of the exams, stating that such combinations are not valid. Why not? It is only for the purpose of ranking. THE IIT SYSTEM HAS BEEN COMBINING PERCENTILE RANKS and MARKS FOR YEARS. M. Tech admission rules (framed by MHRD!) requires that 70% weightage be given to GATE scores and 30% to tests / interviews conducted. Till a few years back, GATE scores were available only in percentile form. We used to combine the percentile with the marks obtained in a  local test to rank the students. Almost always, the local test decided the ranking, as the students had percentiles in the range of 95-99. So, a candidate had to do well in GATE (to get within the 95-99 range ) but he had to be “intelligent” to get into the system! This is exactly what is being proposed in here, except that we are not restricting candidates to the 95-99 range, but we are allowing everyone to compete.