When A MOOC Exploits Its Learners: A Coursera Case Study
By now, if you read this blog, I assume you at least have a passing familiarity with massively online open courses (MOOCs), the “disruptive” technology set to flip higher education on its head (or not). Such courses typically involve tens of thousands of learners, and they are associated with companies like Coursera, Udacity, and EdX. To facilitate a 50,000:1 teacher-student ratio, they rely on an instructional model requiring minimal instructor involvement, potentially to the detriment of learners. As the primary product of these companies, MOOCs demand monetization – a fancy term for “who exactly will make money off of this thing?”. None of these companies have a plan for getting self-sufficient let alone profitable, and even non-profits need sufficient revenue to stay solvent.
Despite a couple of years of discussion, the question of monetization remains largely unresolved. MOOCs are about as popular as they were, they still drain resources from the companies hosting them, and they still don’t provide much to those hosts in return. The only real change in the year following “the year of the MOOC” is that these companies have now begun to strike deals with private organizations to funnel in high performing students. To me, this seems like a terrifically clever way to circumvent labor laws. Instead of paying new employees during an onboarding and training period, business can now require employees to take a “free course” before paying them a dime.
Given these issues, you might wonder why faculty would be willing to develop (note that I don’t use the word “teach”) a MOOC. I’ve identified two major motivations.
Some faculty motivate their involvement in MOOCs as a public service – in other words, how else are you going to reach tens of thousands of interested students? The certifications given by MOOCs are essentially worthless in the general job marketplace anyway (aside from those exploitative MOOC-industry partnerships described above). So why not reach out to an audience ready and eager to learn just because they are intrinsically motivated to develop their skills? This is what has motivated me to look into producing an I/O Psychology MOOC. But I am a bit uncomfortable with the idea of a for-profit company earning revenue via my students (probably, eventually, maybe), which is why I’ve been reluctant to push too fast in that direction. Non-profit MOOCs are probably a better option for faculty like me, but even that has unseen costs.
The other major reason faculty might develop a MOOC is to gain access to a creative source of research data. With 10000 students, you also have access to a research sample of 10000 people, which is much larger than typical samples in the social sciences (although likely to be biased and range restricted). But this is where I think MOOCs can become exploitative in a different way than we usually talk about. I concluded this from my participation as a student in a Coursera MOOC.
I’m not going to criticize this MOOC’s pedagogy, because frankly, that’s too easy. It suffers from all the same shortcomings of other MOOCs, the standard trade-offs made when scaling education to the thousands-of-students level, and there’s no reason to go into that here. What I’m instead concerned about is how the faculty in charge of the MOOC gathered research data. In this course, research data was collected as a direct consequence of completing assignments that appeared to have relatively little to do with actual course content. For example, in Week 4, the assignment was to complete this research study, which was not linked with any learning objectives in that week (at least in any way indicated to students). If you didn’t complete the research study, you earned a zero for the assignment. There was no apparent way around it.
In my experience on one of the human subjects review boards at my university, I can tell you emphatically that this would not be considered an ethical course design choice in a real college classroom. Research participation must be voluntary and non-mandatory. If an instructor does require research participation (common in Psychology to build a subject pool), there must always be an alternative non-data-collection-oriented assignment in order to obtain the same credit. Anyone that doesn’t want to be the subject of research must always have a way to do exactly that – skip research and still get course credit.
Skipping research is not possible in this MOOC. If you want the certificate at the end of the course, you must complete the research. The only exception is if you are under 18 years old, in which case you still must complete the research, but the researchers promise that they won’t use your data. We’ll ignore for now that the only way to indicate you are under 18 is to opt-in – i.e. it is assumed you’re over 18 until you say otherwise (which would also be an ethics breach, at least at our IRB).
Students never see a consent form informing them of their rights, and it seems to me that students in fact have no such rights according to the designers of this MOOC. The message is clear: give us research data, or you can’t pass the course.
One potential difference between a MOOC and a college classroom is that there are no “real” credentials on the line, no earned degree that must be sacrificed if you drop. The reasoning goes: if you don’t want to participate in research, stop taking the MOOC. But this ignores the psychological contract between MOOC designer and student. Once a student is invested in a course, they are less likely to leave it. They want to continue. They want to succeed. And leveraging their desire to learn in order to extract data from them unwillingly is not ethical.
The students see right through this apparent scam. I imagine some continue on and others do not. I will copy below a few anonymous quotes from the forums of students who apparently did persist. One student picked up on this theme quite early in a thread titled, “Am I here to learn or to serve as reseach [sic] fodder?”:
I wasn’t sure what to think about this thread or how to feel about the topic in general, but now that I completed the week 4 assignment I feel pretty confident saying that the purpose of this course is exclusively to collect research data… I don’t know that it’s a bad thing, but it means I feel a bit deceived as to the purpose of this course. Not that the research motives weren’t stated, but it seems to me that the course content, utility, and pedagogy is totally subordinate to the research agenda.
The next concerns the Week 6 assignment, which require students to add entries to an online research project database:
So this time I really have the impression I’m doing the researchers’ work for them. Indeed, why not using the thousands of people in this MOOC to fill a database for free, with absolutely no effort? What do we, students, learn from this? We have to find a game which helps learning, which is a good assignment, but then it would be sufficient to enter the name of the game in the forum and that’s it. I am very disappointed by the assignments for the lasts weeks. In weeks 1 and 2 we had to reflect and really think about games for learning. Then there was the ‘research fodder’ assignment. And this week we have to do the researchers’ job.
In response to such criticisms, one of the course instructors posted this in response:
Wow! Thanks for bringing this up! In short, these assignments are designed to be learning experiences. They mirror assignments that we make in the offline courses each semester. I’m really sorry if you, or anyone else felt any other way. It was meant to try and make the assignments relevant, and to create “course content on the fly” so that the results of the studies would be content themselves.
That strikes me as a strange statement. Apparently these assignments have been used in past in-person courses as important “learning experiences” but are also experimental and new. It is even stranger in light of the final week of the class, which asks students to build a worldwide database of learning games for zero compensation without any followup assignments (to be technical: adding to the database is mandatory whereas discussing the experience on the forums – the only part that might conceivably be related to learning – is optional). So I am not sure what to believe. Perhaps the instructors truly believe these assignments are legitimate pedagogy, but we’ll never really know.
As for me, although the lecture videos have been pretty good, I have hit my breaking point. In the first five weeks of the course, I completed the assignments. I saw what they were doing for research purposes before now but decided I didn’t care. Week 6 was too much – when asked to contribute data to a public archive, I decided that getting to experience this material was not worth the price. Perhaps I could watch the videos this week anyway, but I don’t feel right doing it, since this is the price demanded by the instructors. Even right at the finish line, I will not be completing this MOOC, and I can only wonder how many others dropped because they, too, felt exploited by their instructors.
| Previous Post: | Does Gamifying Survey Progress Improve Completion Rate? |
| Next Post: | Can You Trust Self-Help Mental Health Information from the Internet? |
Given (of all of these articles): All on-line courses are identical.
Unwise assumption: There are MOOC providers which try to do land grabs, and try to produce hundreds of courses at minimal cost. There are MOOC providers which make a small number of courses, spending millions per course. Hint: Think about why some MOOC providers would spend millions per course, when it’s possible to create ones for $50k. Different MOOC providers have very different sets of values as well.
Given (of all of these articles): MOOCs are mostly a subset of in-person courses, with key elements missing.
Unwise assumption: Both residential and on-line have elements the other cannot replicate. Which elements are more important remains to be seen.
Well… In my case, I recognize that the certifications are mostly worthless. So I don’t do the assignments at all. I thought that this represented how the majority consumed them. No?
For me, Coursera is a replacement for watching TV, rounding things I wanted to know more about, but could not be bothered to make a schedule for a formal course. I certainly get less out of the course than those who do the assignments. But I am satisfied for the value I get out of it. I get to drop parts of courses that I don’t feel are interesting or useful for me, rather than what the instructor decided is useful for me… something I cannot do in a real course, where I make full commitments. Overall, I think Coursera is great… and fulfills what is the mission of universities… to enhance public learning.
Your point stands though. It just looks like the instructor simply did not think it through. I am sure they will provide an alternative, now that the issue has been raised. I would not judge MOOCs as a whole though, because of the issue. The world is better with them, than without. I would have preferred that they simply uploaded them to Youtube though.
Just wanted to note the irony.
You are complaining about the productive exploitation of the student base by the instructor by providing a service of progressive advancement… in a course that teaches productively exploiting the client base by a service provider via introducing a fun element of progressive advancement. In other words, the gamification professor is gamifying you – which should be fine ethically, at least for this particular course.
It might be surprising if he didn’t do it :-). I wonder what % of the other 537 courses Coursera provides do this. If not much, then we should not be critiquing Coursera or MOOCs as a whole.
In my academic work on gamification, I actually promote the idea that gamification should be used only for optional activities that benefit the student but that students might not otherwise want to engage in (e.g. extra studying). If games and gamified processes are mandatory, they can be terribly manipulative. The same is true here – I don’t care if MOOCs collect data or not, but participants have a right to know precisely what data is being collected and how it will be used, so that they can make an informed decision not to participate if they so choose.
@Peter
I agree with your first sentence only — assuming all online courses are the same. However, the blog does NOT saying that ALL OF THEM are this way, but it is pointing out at least one obvious exploitation of MOOC that is already in place.
Unwise assumption: “Think about why some MOOC providers would spend millions per course, when it’s possible to create ones for $50k.” This does not seem to be the major reason of what MOOC producer nowadays. Yes, there are some cases, but they are very few. Also, this case seems to be for non-profit MOOC more than for-profit MOOC which is the main concern of this blog.
Unwise assumption: “Both residential and on-line have elements the other cannot replicate. Which elements are more important remains to be seen.” I agree that both have their own unique elements (may be benefit), but assume that one is more important that others is too simple. Certain elements may need to be presented in both cases if certain elements exist. Your assumption is to think that each element has no relation but more or less importance than one another.
I’m glad you’ve highlighted this – some folks are getting the impression that I am somehow anti-MOOC. That is not true; I think MOOCs are full of possibility, however badly they tend to operate right now. I’d really like to teach a MOOC (or something MOOC-like) myself one day. What I am concerned with is the lack of transparency and a false bill of sale. MOOC learners should know exactly what they are getting themselves into, and this is often unclear. For-profit MOOCs are specifically driven (by definition) to make money off of students, and students have a right to know how that’s happening.
I’ve taken about 6 free MOOC’s and have to say none of the courses “required” research surveys. Besides, what do you expect for a free course? Some courses are “pay to play” courses, but either way most of those are considerably cheaper than tuition costs at a university. I have noticed that some instructors pitch their own products (books, research papers, etc.) but I don’t have a problem with that; it’s still a source of learning and none of the instructors have required purchasing their books. And as far as “research” goes I have no problem with that either so long as the data collected is in the aggregate and anonymous. As far as the certificates go, I understood from the getgo that they were not creditable. I take MOOC’s for mere exposure to the course content. I think it’s a great community service. The more complicated the world becomes the more we need to “be aware”.
While this may be obvious to you, I am more concerned with people for whom it is not obvious. MOOC providers are promoting the idea that MOOC courses are “as good as” college credit (whatever that might mean to prospective students) and valuable on their resume. Combined with these dubious data collection practices, the situation becomes quite manipulative.
You also assume that data collection is aggregated and anonymous. Do you have any assurance that this is true? What is to stop them from taking things you’ve shared in your homework or in the forums and posting them publicly, or writing them up in an academic journal? Without an informed consent process, you are not told what will happen to your data. That is not right.
Perhaps things have changed, but when I took Psych 101 at Rice University in 1979, I was required to participate in research studies to get a decent grade. No matter what you made on tests and homework, there was no way to get better than a C (D perhaps?) without participating. I made a B when I had an A average simply because I participated in one too few research projects. While it was possible to pass (with a D) without participating in research, there were no alternate ways of getting out of it. And this was not some fly-by-night little school but rather a prestigious well-recognized university.
Things have definitely changed. The alternatives vary by institution, but there must always be an alternative. For example, at my institution, students can complete library reports (i.e. independently read and critique an academic journal article). Although it probably takes about the same amount of time (1 report replaces 1 hour of research participation credit), that option is generally more boring than participating in a research study, so many students don’t go for it. It is critical though that the option is there if students don’t wish their data to be used in research.
I find the word “exploit” in this context offensive. Suppose the MOOC leaders say upfront: “We will be asking you some questions and hope to use the data in our research studying XYZ. We hope you will also learn from these exercises. Mooc participants will also collaborate on creating an online archive of games.” And then ask you to check a box before participating in the MOOC. I think this is sufficient informed consent. The providers are offering something and I see no reason why they can’t ask for something. No one (at least for now) has to take any MOOC. The university also offers something, and asks anywhere from $500 to $2500 per CREDIT. Most employers now mandate possession of a college degree even for jobs that do not require the capabilities acquired through college (and which can be gained in other ways). And in the U.S. the total student college debt exceeds $1 TRILLION. If you are looking aro0und for something explotive, why not start with university education.
I agree completely that MOOCs aren’t by definition exploitative. But that is not what I am arguing. We can “suppose” how this MOOC would not have been exploitative if things had been different. But that is very much the point – they didn’t do these different things you are suggesting.
There is nothing to stop you from providing misleading information in the survey. This is the correct response and is something which everybody should be encouraging. Unfortunately some people might equate such a thing to lying. It’s very important to teach that, in cases such as marketing surveys, where people unethically ask for needless information, people have an ethical duty to provide incorrect information. Without people doing this we will gradually build up to a total surveillence society. If people do do this then the information gathered becomes polluted and useless.
All the best;
Anna.
Let me preface my post by saying that I’m a staff member for this course, though my comments and views here are my own and not those of the institution or faculty.
First, I’d like to say that I don’t take issue with most of your points. You claim to be an authority in your field, one directly related to the issues MOOCs are working to solve, and I do not have counter points to your claims about pedagogy. But, I do have problems with your inferences that the course is manipulative, or more specifically (and negatively), “manipulative”.
The faculty faced many issues in producing and running this course, but one significant challenge with this MOOC should not be disregarded: they are making claims against methods of learning that MOOCs themselves traditionally employ, but the platform and learning framework available with Coursera, as well as the course’s massive reach and need for ubiquitous, common technology negated the use of more novel approaches to teaching and learning—approaches more in line with the content being covered. Their response was to do away with traditional quizzes entirely, and let activities and discussion take their place, with live research as a significant component. You and your colleagues comments are representative of this, and show that some students still dislike participating as research participants in an academic course. Thousands of others had no issue, and completed every assignment in the course. For every comment condemning the course, there were several praising and thanking the faculty for providing it.
Some people do not like being used as research participants. That’s perfectly fine, and the course made it apparent from Day 1 that research was a core outcome of the course for the faculty and that continued participation implied the student’s acceptance of this. Please understand that research is a driving force behind much of a modern academic institution’s focus, and serves to support cutting-edge teaching and learning. FWIW, UW is a globally top-ranked research institution (http://www.wisc.edu/research/), and as a former student I know I’ve personally benefited from that research activity.
One final point. This course was provided free (in economic cost terms) to all students, though you can imaging the cost to produce it (economic, opportunity, etc.) was enormous. MOOCs are evolving at a very rapid pace, not only in their design and method of delivery, but also in their value to the institutions (for profit and not) that provide and benefit from them. I tend to believe that they are moving to encourage learning without promoting silos of domain experience, and to encourage learners of all backgrounds to broaden horizons and look for interesting applications of this knowledge across domains. I expect that, as they improve there will be better distinction between courses that have a partial research intent and those that do not, and students can self-select accordingly as they see fit.
I do hope you conduct your own MOOC. Perhaps then you’ll see what a challenge it can be to meet everyone’s desires for something provided free. Or, maybe you’ll produce the “one true way” that we can all learn and benefit from. One can only hope…
This is a very interesting comment, and I’m a little surprised. Let me respond to your comment piece by piece.
You seem to be agreeing with me as far as pedagogy is concerned. Obviously MOOCs do not enable an approach to learning that most education researchers currently recommend, and your MOOC is no different. I don’t think that’s good or bad – it is simply the “cost of doing business” in MOOCs. I just think students have a right to know what they’re getting themselves into. I hope you would not disagree with that?
It is fine if you don’t want to use multiple choice quizzes as assignments. But the assignments provided often did not relate well to course content. If you can explain to me how the act of filling out a self-report survey itself helps a person learn specific course content, please go ahead.
Your use of “colleague” as rhetoric to condemn my position as insular is an odd one. Others commenting here are not my colleagues in the traditional sense; this post was picked up by a few news aggregators and has been read by about 10000 people – I don’t actually know who they are. So perhaps this viewpoint is broader than you believe.
“Some people don’t like being used as research participants” troubles me in a lot of ways. First, you imply that you are “using” people, which implies they have no choice in the matter. The Belmont Report, which I hope you read as part of your human subjects training, is very clear on the principles of respect for persons, beneficence, and justice. Human subjects research must be voluntary. They are doing YOU a favor. You do not “use” them. If you feel you are “using” them, you should not be conducting research – ever.
You also imply with this statement that some people being fine with it and others not makes it ethical. This is not a valid argument. Even one student whose rights have been infringed is too many. Perhaps “the course made it apparent” to you, but current guidelines on the protection of human subjects indicate that “apparent” means “provided informed consent.” There was no consent, you coerced students already in the course to participate in your research in order to be recognized as completing the course, and thus the situation was exploitative.
I’m VERY intrigued that you chose to call out the course and university that I’m referencing. Are you implying that Carnegie Foundation university classifications impact what defines ethical behavior and that UW is somehow exempt from this because it is “globally top-ranked”?
The course was very much NOT free. You admit this by stating “(in economic cost terms)”. The course cost student time, privacy, and control of data. If a student knowingly and willingly gives up those things to participate in a MOOC, no harm done – but that is not what happened. “As they improve there will be better distinction” implies that regulation over human subjects research is not yet sufficiently complete to guide us in this situation. That is incorrect. It is. Use an informed consent process.
To use your phrase, I don’t pretend to have the “one true way” to design a MOOC. I do claim an understanding of ethical behavior when conducting human subjects research, and that is what I had a problem with in this course.
I won’t argue with you on your own turf, and you do make many valid, eloquent points that the MOOC didn’t meet your (or your “colleagues”) expectations. That’s too bad. But I still thank you for posting my ineffective rebuttal.
I want to finish by stating that you’re being very selective with your references, and a majority of the students who actively participated in the class found it to be beneficial, including the activities, of which some may have had an IRB-approved research intent. If you’re willing, a quick perusal of the assignment forums and weekly overview videos will show that the faculty did indeed connect the learning objectives of the assignments to the course material. Though questions about the research intent are plenty, they were responded to and if any student would have asked for an alternative assignment option it would have been granted (or waived, though forum activity was still expected). The message was never “give us research data, or you can’t pass the course”—that was an incorrect inference based on our mandated IRB statement that data “may” be used for research purposes.
Thank you for posting my response. Again, it is my own, and not a reflection of my institution or the faculty.
My point entirely! Students should not be forced to make inferences about their rights on their own – their rights should be clearly stated up front in an informed consent document.
And I never said that the course content was poor – there was certainly a lot of great lecture content, so as a MOOC, it was one of the better ones I’ve sat in on. Maybe a little Gee-heavy, but otherwise good. 😉
But even so, that doesn’t change whether students were treated ethically, and that’s what I took issue with – hopefully that’s a bit clearer now.
Of course research obtained this way should be rather worthless as people might not care answering or, worse, provide deliberately wrong answers because they are annoyed
There will certainly be data quality problems, but many of those are (mostly) solvable through dishonesty/lazy response detection, and there’s a family of techniques to deal with that. The bigger issue is range restriction – whoever you’re trying to make conclusions about, it’s likely that a MOOC population is only a subset of that group. The major problem is that we don’t really know WHICH subset – are they especially high motivation? Low motivation? Both? Different in some other regard? And that is not a solvable problem without a lot more research on who exactly goes into a MOOC and who exactly drops out. I don’t think any data collected within a MOOC about anything other than MOOC-related behaviors would be publishable right now, but that’s assuming researchers admit to using MOOC datasets in their research reports (dishonesty – or at least incomplete/lazy reporting – will likely be common).
@ Richard Landers: You may find Professor Paul Oliver Dehaye’s work on Moocs related http://paulolivier.dehaye.org/
Complete waste of $50 to earn a course certificate. I waited over 2 weeks for them to finish ‘grading’ and then they deny my certificate due identity verification. But they had no problem verifying my credit.
First, thank you for registering to earn a Course Certificate in Programming Mobile Applications for Android Handheld Systems: Part 1 on Coursera.
Unfortunately, our review shows your identity wasn’t verified for enough assignments in this course to earn a Course Certificate. This can be caused by skipping webcam photos or typing pattern samples, or submitting blank webcam photos.
In order to receive your Course Certificate, you can try to resubmit the assignments that were initially unverified with verification. If you’re unable to submit assignments again or otherwise need assistance, please contact our support team.
We hope to see you in another class soon.