Rankings matter. After all this is possibly the only systematically aggregated data point that students have, to match their requirements and the college offerings. It is Rankings which help students understand the relative pecking order of the b-schools on various parameters and also the importance of the parameters themselves. It is these ranking surveys which help bring to public light what cannot be gathered either individually or through the mass social media; and they are efficient for the reader gets vast, consolidated and analysed information at a very low cost. Just that there are a few concerns:
1.Are all the aggregate measures broken up into relevant dimensions?
2.Are the dimensions measurable?
3.Do these measures capture what they intend to measure?
4.Is the captured data accurate?
5.Is the captured data comparable?
6.Is the analysis transparent and fair?
Let’s use, what I call, the thermometer principle to understand the above concerns. The temperature indicated by the thermometer is taken as gospel and rarely disputed. What we need to be cautious is the fact that there are a number of risks underlying this analysis. The thermometer could be wrongly calibrated; Unless I place the thermometer in my mouth, the benchmark 98.4C may be irrelevant; Numbers without units (C,F,K )are meaningless ; If the mercury loses its physical property we are in for trouble; and hopefully we do not commit parallax error as we read the mercury level.
Now all these and more are the risks underlying the ratings survey. Often sociological phenomena are too complex (compared to physical sciences) to be analytically broken. They suffer from multiple data biases – the one capturing data is unable to accurately capture it and at the same time the response provider may have reason to exaggerate or hide or may even be ignorant. Worse still if the right questions aren’t raised with the right stakeholder we may end up creating a lot of bunkum.
If the above two paragraphs aren’t reason enough to scare us, there is an even bigger scare. Over the years b-schools have increased intake in each location and have also expanded geographically. Foreign b-schools have begun either tying up with Indian b-schools or are aggressively directly marketing their campuses to the affluent Indian middle class. All this calls for aggressive marketing effort from the local Indian b-school; forcing colleges to possibly manipulate or even indulge in falsehoods while participating in these surveys.
Despite all these, one cannot deny the relevance of ratings. Hence the reader has to become increasingly skeptical, without becoming agnostic, for that would be futile.
So what does the prospective b-school graduate do? The caveat emptor principle holds. A few things are possible:
1.First and foremost look out for the credibility of the organization doing this survey. While reputation is difficult to measure, some of these can help – Is the survey done regularly and punctually? Is the survey methodology transparently communicated? While the questions used to measure might have to incrementally change over the years, rapid & radical changes to the questionnaire should be viewed suspiciously. Are most of the prominent b-schools surveyed or just one or two to lend credibility?
2.Has the data been standardized? A simple way of standardizing is using percentages. A college (A) with 100 faculty for 1000 students might still be worse than a college (B) with 10 faculty for 60 students. College B despite 100% placements might have lesser number of students placed than A and so is the problem on the quantum of academic research. Unless we adjust for size, numbers can be misleading.
3.Don’t look at the final ratings of the survey. Instead prioritize what you think are more important parameters and decide your own priority list of colleges.
4.Understand how the parameters are measured – Are they just opinions? Are any tangible measures used? Are multiple stakeholders covered? Use the data to draw your own conclusions.
5.Based on the survey Results, talk to the alumni or faculty or administrators of the college you’ve shortlisted and get direct insights. Nothing can substitute for self-analysis; and yes also search through social media to corroborate all these findings.
There are many more questions you would want to ask, and my advice is you should. A little bit of common sense, some attention to detail and a lot of perseverance can help you get much more out of the Rating survey than you might imagine. A few truths I have discovered over the years about Rating Surveys are:
1.Screaming headlines lie
2.Devil lies in the detail
3.Complex statistical analysis is often intended to obfuscate
4.Number of data points is less relevant than the richness of data exhibited through its attributes
5.The number of Ratings surveys keeps going up as the Industry becomes competitive
6.The top few and the bottom few in most Ratings surveys are unlikely to be wrong
7.Ratings which are more transparent on methodology are likely to be more accurate than the rest
These, along with your own thumb rules, can be used to remove the smoke screen around the rating rantings.
Prof. C Srinivasa Rao is a Faculty at Accend School of Management and Entrepreneurship (ASoME) and is an alumnus of XLRI Jamshedpur and IIM Bangalore. He can be contacted at firstname.lastname@example.org.
For more information on the college and courses, please click on ASoME