Are the numbers what they seem?
Among culinary arts in Central Asia, the cooking of government statistics is one of the most entertaining. Here are a few favorite recipes:
The student-faculty ratio. The Ministry of Education and Science wants colleges to reduce the number of students taught by a typical instructor because it regards smaller classes as better. Recently, a Ministry official threatened to suspend KIMEP’s license, largely for a student-faculty ratio of allegedly 20 (that is, 20 students per professor). KIMEP estimates its ratio as below 2 for graduate students and as 15 or 16 for undergraduates. Probably the official was confused by a rule-of-thumb here that an undergraduate class with 20 students will earn enough tuition to pay for itself. In principle, classes with fewer than 20 registrants would lose money, so KIMEP would cancel them. That rule is honored in the breach. KIMEP often approves a small class that students need in order to graduate or that they express a strong interest in taking. The modus operandi here is the same as at most universities – large introductory classes subsidize small, advanced classes and classes (such as in foreign languages) that require a lot of attention to each student. (A large class makes money because the teacher’s pay is spread over more students, each of whom pays the same tuition for this class as for a small one.)
In his newspaper interview, the official’s tone of shocked outrage suggested that scheduling a college class of 20 students was a venal sin. In reality, scholars have debated the impact of class size on student performance for more than a half century. Twenty years ago, the consensus of economists, from statistical studies, was that class size did not affect learning systematically.
Since then, as often happens in empirical research, studies taking new approaches have qualified the original blunt conclusion. The impact of class size in colleges may differ from that in primary and secondary schools -– the settings for most studies -- since college class sizes vary a lot. Third-graders rarely sit in 300-student lecture halls. Standardized test scores -– the usual proxy for student achievement -- may not measure learning well, since instructors often teach the test and nothing else; perhaps we should use the graduate’s wages. Class size may matter more in poorer countries, where students cannot afford such aids as Cliff’s Notes to cope with monster classes.
Warm bodies, cold statistics
Despite such caveats, one cannot ignore the sheer weight of scores of estimates that have failed to find that class size matters much. Of 112 estimates surveyed by Hanushek in 1986, 89 found no statistically significant effect of class size on student achievement. Only nine were statistically significant and carried the negative sign -- that is, students learned less in larger classes. (In a general comment on Hanushek’s work, econometrician Alan Krueger asserts that drawing several estimates from the same flawed study may give it undue weight in the survey of all studies. This particular survey was of 33 publications.) The idea that large classes stunt learning may merit the Scots’ Verdict – “Not proven.” I am not a specialist, but my impression is that the one durable conclusion from econometric studies of student achievement is that students with richer and more educated parents do better.
In any event, the Ministry wanted KIMEP to cut its student-to-faculty ratio to 8. Unfortunately, the Ministry apparently calculates this ratio in terms of warm bodies. A student who takes one course every year counts just as much as a student who takes 15. Consequently, the ratio may mean virtually anything. Suppose that a school has one professor and eight students. Then the student-faculty ratio is eight if every student takes one course per year. It is also eight if every student takes 15 courses per year. Obviously, the professor can devote more time to a student in a given class in the former scenario. In the latter, he would be overwhelmed by preparations of 15 times as many classes.
The Ministry could get a more accurate picture of the quality of learning at, say, Java College by calculating its ratio in terms of full-time equivalents, which is what experts do. At KIMEP, a fulltime professor teaches eight courses per year, and a fulltime student takes 10 courses. Two students, each taking five courses per year, would count as one fulltime-equivalent student. In our two earlier scenarios, the first has .8 of a fulltime-equivalent student, and the second has 12. If the professor teaches fulltime, then the student-faculty ratio is .8 in the first scenario and 12 in the second. Now it’s clear which school is truly crowded.
As is its wont, the Ministry has neglected the likely consequences of its policy. If only warm bodies matter, then schools will satisfy the student-faculty maximum by hiring the cheapest warm bodies around. Those would be adjuncts, who receive a fraction of the salaries, and none of the benefits, of regular faculty. Since adjuncts often have only a fraction of the learning of regular professors, the Ministry’s policy may torpedo its stated intent of improving education.
Consider an English class of 24 students, taught by a native speaker for $900. To satisfy the Ministry’s mandate, Java College replaces the teacher with three adjuncts, each willing to teach eight students for $300. Now each student can receive thrice as much time from a teacher as before. Unfortunately, his new teacher knows only a few words of English.
Shrinking classes isn’t cheap. The college must either cut back on something else, like teacher quality, or pass on the cost to someone else, namely, the student. The cost may be justified, but in any event it must be considered.
One reason for the continuing controversy over class sizes is that they affect much of what a school does. Reducing class size changes other school traits, which themselves affect learning. When we ask whether class size matters, do we mean just its direct effect on learning? Or also its indirect effects?
“…While class size always affords opportunities for increased investment in each child’s learning, it is not obvious that every school takes up such opportunities,” writes economist Caroline Hoxby. “The actual effect of reducing class size will depend on the incentives a school faces.”
Understating unemployment
The unemployment rate. Most people think that this is the share of potential workers who don’t have jobs. In that light, the rate may approximate one’s chances of going jobless. In reality, the unemployment rate is the share of the labor force that seeks work but can’t find it. (The labor force is the sum of the employed and unemployed.) What if you’re so discouraged that you quit even looking for a job? Does the government count you as unemployed? Nope: It counts you out of the labor force. Consequently, the unemployment rate understates joblessness. This is useful to keep in mind, because Kazakhstan’s government often attributes falls in the rate of unemployment entirely to its own programs to create jobs.
Some governments try to estimate the number of discouraged workers. In the United States, the Bureau of Labor Statistics telephones or personally interviews 60,000 households each month. In August 2010, 1.1 million Americans were too discouraged to look for work -– about one-sixth of the number who still sought jobs although they had been unemployed for at least half a year (“chronic” unemployment). About as many Americans drop out of the labor force in order to go to school, recover from illness, or to look after their families, or for similar reasons, as because they give up on finding jobs.
Employment rates. It’s logical to define the employment rate as 1 minus the unemployment rate. Kazakhstan, with an unemployment rate of 6%, would have an employment rate of 94%; i.e., 94% of the labor force has jobs. However, economists rarely use this statistic, because it adds nothing to what we already know from the unemployment rate.
More interesting is the share of those able to work who have jobs. The employment-to-population ratio is defined in terms of those aged 15 or older. This statistic gives us a sense of whether the economy is operating at full capacity. When the ratio is high, then employers may have trouble finding workers and thus will raise their wage offers. A general increase in wages will force up prices as employers try to pass on their costs to consumers. Hello, inflation.
In Kazakhstan, the employment-to-population ratio is just under two-thirds, according to the World Bank. It is significantly higher for men (roughly 70%) than for women (below 60%).
A way to gauge a nation’s willingness to work is to look at the share of the labor force in the working-age population, called the “labor force participation ratio.” In 2008, this ratio in Kazakhstan was 75% for men and 63.5% for women, according to the United Nations. In the United States, the long-run rate has been about two-thirds and declining (to 65% in August 2010). In August 2010, it was 71.7% for men and 58.6% for women, according to the U. S. Bureau of Labor Statistics. Severe recession has undoubtedly lowered the American rates for August, and they are not seasonally adjusted. Still, one wonders whether Kazakhstanis work harder than Americans.
People sometimes argue that a fall in the unemployment rate -– or, if you will, a rise in the employment rate -– must boost production. That argument confuses the rate of employment with the number employed. An example may clear this up. Suppose that 100 adults comprise the labor force; of them, 50 work and 50 look for work. The unemployment rate is 50%, and so is the employment rate (when defined as 1 minus the unemployment rate). Now, 25 of the unemployed stop searching for jobs; so they drop out of the labor force. Seventy-five remain in the labor force. The new rate of unemployment is 33% (25/75), and the rate of employment is 67% (50/75). The employment rate has risen sharply -- but output remains unchanged, because we still have just 50 on the job.
Labor statistics. Employment and unemployment rates are usually calculated in terms of workers. The government counts you as employed even if you work only one hour per week. You will count as much as does someone who works 60 hours a week.
But the 60-hour worker provides 60 times as much employment as the one-hour worker. Thus employment ratios distort our view of the economy’s performance. In a recession, when firms put workers on a part-time basis, the ratios overestimate the economy’s use of labor. In a strong recovery, when firms pay overtime because they have trouble finding new workers, employment ratios underestimate labor use. Policymakers who rely on these ratios to signal impending inflation may be in for a nasty surprise. Measuring employment in terms of hours worked would correct these faults; the underemployed worker would be recognized as such.
Maybe someday Astana’s chinovikii chefs will consider serving up statistics that are less piquant and more nourishing. -- Leon Taylor, tayloralmaty@gmail.com
Disclaimer: I don’t speak for KIMEP.
Good reading
Angrist, Joshua D. and Victor Lavy. 1999. Using Maimonides’ rule to estimate the effect of class size on scholastic achievement. Quarterly Journal of Economics 114: 2, May, pages 533-575. In the 1100s, the Talmudic scholar Maimonides wrote: “Twenty-five children may be put in charge of one teacher. If the number in the class exceeds 25 but is not more than 40, he should have an assistant to help with the instruction. If there are more than 40, two teachers must be appointed” (page 534). Public schools in Israel still use the rule. This gives researchers an unusually clear chance to study the impact of class size on student performance, because Maimonides’ rule can affect performance only through class size -- not through other factors that are hard to control for. The study finds that making classes smaller will increase test scores for fourth and fifth graders but not for third graders.
Card, David, and Alan Krueger. 1992. School quality and black-white relative earnings: A direct assessment. Quarterly Journal of Economics 107, February, pages 151-200.
Card, David and Alan B. Krueger. 1996. School resources and student outcomes: An overview of the literature and new evidence from North and South Carolina. The Journal of Economic Perspectives 10:4, autumn, pages 31-50. This innovative analysis links smaller classes to greater adult earnings.
Hanushek, Eric A. 1997. Assessing the effects of school resources on student performance: An update. Educational Evaluation and Policy Analysis 19: 2, summer, pages 141-164. “The close to 400 studies of student achievement [in the literature on educational production] demonstrate that there is not a strong or consistent relationship between student performance and school resources, at least after variations in family inputs are taken into account” (page 141).
Hanushek, Eric A. 1996. Measuring investment in education. The Journal of Economic Perspectives 10:4, autumn, pages 9-30. Spending may fail to boost student achievement because of diminishing returns to spending and because of a lack of incentives for teachers like merit pay. (By “diminishing returns,” I mean that spending another dollar does less and less good.)
Hanushek, Eric A. 1986. The economics of schooling: Production and efficiency in public schools. Journal of Economic Literature 24: 3, September, pages 1141-1177. This influential survey of 147 econometric estimates concludes that “differences in [school] quality do not seem to reflect variations in expenditures, class sizes, or other commonly measured attributes of schools and teachers. Instead, they appear to result from differences in teacher ‘skills’ that defy detailed description but that possibly can be observed directly” (page 1142).
Hoxby, Caroline M. 2000. The effects of class size on student achievement: New evidence from population variation. Quarterly Journal of Economics 115: 4, November, pages 1239-1285. This study of 649 elementary schools controls for some factors that determine class size. It finds no statistically significant effect of class size on student achievement. The quote in the text above is from page 1240.
Kazakhstan Today. 2009. Unemployment rate in Kazakhstan to decline. June 8. The government credited its own anti-recession policies with halving the rate of unemployment.
Krueger, Alan B. 2002. Economic considerations and class size. National Bureau of Economic Research Working Paper 8875, April. “…Hanushek’s pessimistic conclusion [in his 1997 survey] about the effectiveness of schooling inputs results from the fact that he inadvertently places a disproportionate share of weight on a small number of studies that frequently used small samples and estimated misspecified models” (page 2). When one gives all studies the same weight, or a weight that increases with the quality of the journal, then class size affects student achievement significantly.
United Nations Statistics Division. Online at data.un.org . International statistics, including economic data for Kazakhstan.
United States Bureau of Labor Statistics. Online at www.bls.gov . Labor data and explanations.
Revised on October 9, 2010.
