While Most Students Don’t Learn, Some Do

Academically AdriftTo say the least, Richard Arum and Josipa Roksa have made waves in higher education with their book Academically Adrift: Limited Learning on College Campuses (Chicago: U of Chicago P, 2010), a national, longitudinal, quantitative study of student learning in the first-two years of college in the U.S. The question driving the study is: “How much are students actually learning in contemporary higher education?” The overarching finding is unsettling. “The answer for many undergraduates,” Arum and Roksa conclude, “is not much” (p. 34).

Bad News #1: Most Students Are Not Learning

In what is certainly the most provocative snippet of the book, Arum and Roksa write:

With a large sample of more than 2,300 students, we observe no statistically significant gains in critical thinking, complex reasoning, and writing skills for at least 45 percent of the students in our study. . . . While they may be acquiring subject-specific knowledge or greater self-awareness on their journeys through college, many students are not improving their skills in critical thinking, complex reasoning, and writing. (p. 36)

These findings are made all the more “disconcerting” when Arum and Roksa go on to point out that “at least one study has indicated that most of the gains in general skills occur in the first two years of college” and that another study “reported that students’ academic motivation and interest in academic subject matter declined during their first year in college, leaving little hope that they would notably improve their academic skills in subsequent years” (p. 36).

Bad News #2: Most Students Are Not Trying

Not only are students not learning, they don’t even seem to be trying to learn. An analysis of student time-use surveys over the past century found that “average time studying fell from twenty-five hours per week in 1961 to twenty hours per week in 1981 and thirteen hours per week in 2003” (3). Even worse, in the present study, “37 percent of students reported spending less than five hours per week preparing for their courses” (69). So what do students do their time? On average, as the following chart indicates, they sleep a little, work a little, study a little, and socialize a lot (p. 97).

Chart on Student Time Use reproduced from Arum and Roksa

What makes these numbers so important is that, when it comes to student learning, “how much time and effort students invest in their classes is paramount: Studying is crucial for strong academic performance as ‘nothing substitutes for time on task'” (131).

Bad News #3: Many Students Are Not Challenged

If students aren’t trying to learn, that may not be entirely their fault. Arum and Roksa present another finding that suggests that many students are not challenged to work on those very areas in which they are not improving. Specifically, they found that: “Fifty percent of students in our sample had not taken a single course during the prior semester that required more than twenty pages of writing, and one-third had not taken one that required even forty pages of reading per week” (p. 71). While a much more precise inquiry into student reading and writing would certainly be beneficial, these findings seem to indicate that students are not being asked to do enough reading and writing. On that point, Arum and Roksa comment:

If students are not being asked by their professors to read and write on a regular basis in their coursework, it is hard to imagine how they will improve their capacity to master performance tasks . . . that involve critical thinking, complex reasoning, and writing. (p. 71)

Bad News #4: Nonwhite Students Are Especially Not Learning

Sadly but predictably, socioeconomic variables, especially race and ethnicity, played a dramatic role in which groups of students, on average, learned the least. Arum and Roksa report that African-American, Asian, and Hispanic students “not only entered higher education with lower [skills] than their white peers, they also gained less” over time, on average (39). In the typical understatement of social scientists, Arum and Roksa surmise that “[t]his pattern suggests that higher education in general reproduces social inequality” (40).

Good News #1: Some Students Are Learning

While “growth is quite limited” overall, Arum and Roksa write that “some students” do in fact “demonstrate notable gains” (p. 56). More specifically, while almost half of students seemed to learn nothing and while most students seemed to learn very little, one out of ten students seem to learn quite a bit, when it comes to writing, reasoning, and critical thinking. Arum and Roksa consider this “the most important finding missing from the popular discussion of the book” (“Questions” [PDF]).

To put it in numbers: 45 percent of students did not improve in a “statistically significant” way on the instrument Arum and Roksa used to measure learning (p. 36). The average improvement, only slightly better, was 7 percentile points—which means, for example, that those entering college at “the 50th percentile of an incoming class would reach a level equivalent to the 57th percentile of an incoming class” by the end of the second year (p. 35). In stark contrast, the top ten percent of students improved by 43 percentile points—which means, again, that those entering college at “the 50th percentile of an incoming class . . . would reach a level equivalent to the 93rd percentile” of an incoming class by the end of the second year (p. 56).

Good News #2: Good Teaching Makes a Difference

Why do some students learn when so many others don’t? As noted, socioeconomics factor in hugely. Much more positively, Arum and Roksa found that good teaching can also make a difference. They write that “when faculty have high expectations and expect students to read and write reasonable amounts,” students not only spend “more time studying” but also “learn more” (p. 119). More specifically, Arum and Roksa found evidence that more learning happens:

  • when students perceived that their teachers had high expectations (p. 93);
  • when teachers assigned at least 40 pages of reading per week and at least 20 pages of writing over the course of the semester—presumably communicating and enacting those high expectations (p. 94); and
  • when students spend at least 12 hours per week studying alone—presumably responding to those expectations and working on that all that reading and writing (p. 97).

These findings provide additional empirical support to the idea that better teaching in general can lead to more in-depth learning. They also underscore the importance of the specific teaching practices of communicating high expectations and assigning substantial reading and writing.

Study Not Perfect But Still Important

It is important to note, as Doug Lederman and David Bills aptly explain,  that Arum and Roksa’s findings have faced substantial controversy. To begin with, some critics argue that the instrument on which Arum and Roksa base their most important findings, the Collegiate Learning Assessment (CLA), does not meaningfully measure students’ writing and critical thinking skills. This is certainly a fair criticism, given that standardized tests, virtually by definition, define and measure writing and critical thinking reductively, reducing the qualitative to the quantitative.

Also, others question certain points of Arum and Roksa’s statistical analysis, particularly regarding their striking “45-percent conclusion,” a figure that, Alexander W. Astin notes, “is well on its way to becoming part of the folklore about American higher education.” The claim that 45 percent of students did not improve in a “statistically significant” way, Astin explains, depends entirely on an “utterly arbitrary” decision as to what counts as “statistically significant.” With a more or less stringent cut off, the figure of 45 percent could have been much higher or lower, with the very same data.

These and other criticisms qualify the study in important ways, and put it in context. But they do not invalidate it. At the very least, the book presents some substantial evidence about the state of learning. If anything, because of how much discussion and debate has surrounded the book, these detractions make it all the more important as a reference point in talking about learning and the lack of learning in higher education. Even if the findings of the study are far from perfect, Arum and Roksa warrant serious consideration.

This post, the first of two on the book Academically Adrift, deals with Arum and Roksa’s findings on how little students are learning and why. The second post looks at their proposals for reform.

6 responses to “While Most Students Don’t Learn, Some Do

  1. After having been in the profession of higher education teaching for over two decades, I find it hard to contest the general validity of the study. Since good teaching is the uncontested variable in the whole question of why students do not learn, we cannot just prescribe to just do all those things indicated by the study and assume that it will happen. It can be assumed that the majority of those in the teaching profession would like to do a good job in teaching, doing all those things that are prescribed by the authors such as requiring readings, giving students written assignments, and more. The study also shows how students most likely prefer to spend their time (51 % in leisure activities).

    That said, given the fact that tertiary education is going to be a crucial factor not only to them but to the nation and the world as a whole, the next question should be what is to be done by those whose life is dedicated to teaching. Giving reading and writing assignments to hone students’ critical thinking is not going to be enough. It is good to note that some schools use student evaluations in rewarding (more in punishing) those who are involved in teaching. I believe student feedback on one’s teaching is very important. However, it should be used only as such, a feedback to the person who is teaching so that she or he can improve her or his performance. If and when it goes beyond that, let us not forget that it could instigate fear in the minds of many that some students could take is as getting back at the instructor, particularly when it is known that she or he needs their students’ positive evaluations for tenure.

    The biggest problem could probably be traced to academic administration processes and mechanisms that do not put in place the proper means to help teachers from being intimidated by student retaliatory actions. This condition may not exist in research institutions and I am not sure if this is rampant in most of the teaching institutions either. If the educational work particularly through the fostering of critical thinking has to compete with the 51% of socializing and recreating activities that students do as shown in the study, it is going to be a very hard battle to win. We need all the ammunitions to muster. Therefore, the next big research question appears to be finding the answer as to what can be done to compete with socializing and recreating.

    Personally, I would like to thank you for posting this particular issue. Identifying the problem is the first step in finding a solution.

    • You’ve made multiple important points, here, A.

      In particular, your point is well taken that simply assigning more reading and writing is not enough.

      I think that if Arum and Roksa were able to control more precisely for a wider range of effective teaching practices, they would have been able to show even greater average learning gains correlated to a smaller group of students who experienced the most effective teaching. As it stands now, the group of courses that assigned so many pages of reading and writing must surely include instances that did so effectively and instances that did not do so effectively. Similarly, among the group that assigned fewer pages, there must have been some that assigned less but did it effectively and others who just didn’t. So more precision would have been able to tell us more clearly what teaching practices work.

      But since we actually already know what sorts of teaching practices work (as documented in other studies), I think one way of looking at the ultimate usefulness of this study is that it confirms and underscores the importance of those other studies by highlighting the scope of the problem.

      An upcoming post on the blog (about another study) will deal more directly with the question of socializing vs. studying in student culture and what, if anything, we might be able to do about it. One thing, though, that this study does highlight (that I didn’t mention in the post) is that on average students did study more for those professors who assigned the recommended amounts of reading and writing, by an average of 2 hours per week, if I recall correctly. Probably not enough. But it goes to show that we can have some impact on whether students will spend their time productively.

      I do wonder what variation there is in the amount of socializing from school to school, since the 51% is an overall average. I imagine that at many schools, students (particularly non-traditional students, etc.) work much more and socialize much less, which requires a different sort of solution. Either way, though, without students spending a lot more time trying to learn, we cannot expect that they will.

      Thanks for commenting!

  2. Pingback: To Reform College, Prioritize Learning | Teaching & Learning in Higher Ed.·

  3. Pingback: Students Are Horrible in Every Way – The Conversation - Blogs - The Chronicle of Higher Education·

  4. Pingback: Students are horrible in every way |·

  5. Pingback: Online Education Can Solve a Real Problem (not the one you think) - University 5.0 University 5.0 Online Education Can Solve a Real Problem (not the one you think) - University 5.0 | Fresh Thinking about Higher Ed in an Online World·

Thoughts? Please Comment . . .

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s