How (not) to spin data

The latest press-release from CSO’s QNHS-based analysis of Q2 2002 – Q2 2008 data on educational attainment sounds self-congratulatory:

“In the second quarter of 2008, 29% of all persons aged 15-64 had attained a third level qualification. The proportion of people with a third level qualification increased steadily over the years since Q2 2002 when the comparable level was just over one fifth (22%). Excluding 15-24 year olds (the age group most likely to be still in education), just over one third (34%) of 25-64 year olds had a third level qualification, and this had also increased gradually annually since 2002 when the level was 25%. … The latest available figures for all EU member states showed that, in the second quarter of 2007, 30% of all 25-34 year olds had a third level qualification. The equivalent figure for Ireland was 44%, ranking the country second in the EU only to Cyprus (47%). The lowest levels of third level attainment were reported in Czech Republic (16%), Slovakia (18%), Italy (19%) and Austria (19%).”

There is more spin in these remarks than truth about the quality of our labour force. Here is why.

Based on the set of three main international rankings, Ireland’s third level education system ranks 22nd out of 38 countries, or 15th within EU27. Less than 20% of our entire crop of third level attendees and graduates come from the universities and institutes that make it into top 500 in the world rankings. Furthermore, CSO own data shows that only 18% of our 15-64 year olds have managed to actually complete a third year degree - a real measure of the ‘third-level qualification’ attainment.

This suggests that only ca 3.6% of our labour force had a real, internationally competitive educational qualification - i.e a Bachelor's degree or above. For comparison, the figure is closer to 9% in the US and 10% in the UK.

Figure 3 in the CSO report shows that Cyprus leads EU27 in the proportion of the labour force with third level qualification. Cyprus actually fails to rank amongst top 50 countries around the world for quality of its 3rd level education. Apart from Cyprus hardly constituting a worthy competitor for the 'knowledge' economy-bound Ireland Inc, virtually all of the countries named by CSO as being laggards to Ireland score better than Ireland in the university league tables. May it be the case that the CSO's (and EU's) methodology for measuring success in education is confusing quantity with quality?

A regression of the proportion of the 3rd level education attainment for 25-34 year olds on the proportion of early school dropouts shows a strongly negative relation between two variables:
For the entire set of countries (EU27): y = -0.2051x + 33.649 R2 = 0.0079
For small EU countries sub-sample: y = -0.2966x + 35.017 R2 = 0.0648
This is logical, as more drop outs should, in theory, imply fewer potential graduates. In both cases, Ireland and Cyprus come up as strong and influential outliers. Exactly the same happens when we look at the positive relationship between the proportion of the graduates with completed secondary education and those with third level qualification.

Now, considering the two equations above, two additional facts, not considered by the CSO, emerge as being of interest to the case of Ireland's education success. First note the difference in the intercepts between the two groups of countries. This suggests that an average small EU state has a higher share of the labour force with thrid level education. Second, the adverse impact of early school drop outs on educational attainment is stronger for smaller countries than for the larger states. In some ways, this again points to some problems with the CSO data, because, as it turns out, Ireland scores almost exactly the average for the smaller EU15 states in terms of the share of the population who failed to complete secondary education. So by both of these facts, Ireland is an outlier - an exception to the rule.

Hmmm… something is not quite right.

One possible explanation of this puzzle is that somehow Irish society is so polarized across the social demarcation lines that a substantial share of population has extremely poor graduation rates for secondary education, while another substantial sub-group has extremely high graduation rates for third level education. But this would not explain the case of Cyprus where income inequality is different in composition and magnitude from Ireland.

Another explanation is that Ireland, and possibly Cyprus, suffer from quality dillution in education at the third level, also known as 'dumbing down' of our universities. Much has been written and said in public about this matter, but the latest CSO numbers may be providing an indirect hint at the extent of the problem. If the vast majority of Irish Universities and third level institutions cannot reach top 500 in the world league tables, an army of the graduates (and, given the CSO methodology, an even bigger army of the 'near' graduates) they produce might just deliver the equivalent of the Chariman Mao's Great Leap Forward in education - massive boost in quantity, at a cost to quality.