Advancing MOOCs for Development – The need for unbiased research

The “Advancing MOOCs for Development Initiative” is a collaborative effort by a number of academic and development institutions to push MOOCs for Professional Development in the developing world. In the first stage of this project, a report has been released on the current state of MOOC usage in Colombia, the Philippines, and South Africa. This blogpost is a brief review of the report, and some reflections on my own study. For an in-depth breakdown of the results of this study versus actual platform data, I highly recommend reading Willem van Valkenburg from TU Delft’s critique of this study.

From its initial inception through a $1 million grant from USAID, this initiative has been of interest to me, as the parallels with my own research are obvious. “MOOCs for Development” features in the title of my PhD, and both are interested in considering the usage of MOOCs in the developing world, albeit in different contexts. However, this is where the similarity ends.

Fundamentally, this initiative begins with the position that MOOCs are a good thing. The title “Advancing MOOCs” makes this clear – and any study that comes out of this initiative is inherently biased in this regard. On the other hand, my research title is, more accurately, “MOOCs for Development?” The question mark is subtle, but key, because it suggests a questioning of the narrative of MOOCs as the solution to the challenges facing the developing world – and begins with a neutral, if not sceptical view of MOOCs. When the overwhelming literature suggests MOOCs are successful largely for experienced educated lifelong learners, there is a need to be cautious when making bold claims, particularly with respect to the developing world.

Given this inherent bias of this initiative, it was no surprise that the press release for this report emphasized “High MOOC completion rates in the developing world”, claiming almost 80% completion rates. For anyone already engaged in MOOC research, it’s clear that this sort of reporting is incredibly careless and misleading. What these headlines fail to mention is that these figures are reporting the results of their survey (incredibly biased sample) of MOOC students, rather than actual platform data. Having worked with FutureLearn data on Indian learners, their completion rate is in fact far lower than the average completion rate, and similar findings are reported in the post from TU Delft. In fact, even the report mentions the limitations of these findings, yet this does not figure anywhere in the press releases, leading to a perception that MOOCs are incredibly successful in the developing world.

PR spin aside, the research design is fairly sound. The survey is robust (and incredibly similar to my survey design), and the method of comparison between the three contexts, as well as interviews with policy makers, makes this quite a comprehensive study. As the focus was for professional development, they only surveyed 18-35 year olds to answer their specific research questions, which does miss out on some of the key demographics of MOOC learners, but they mention this in their discussion of limitations, which addresses most of the broader concerns with this study as well. It is a shame though, as a decent research study has been discredited through poor PR management, and a lack of objectivity in general.

More broadly, given my (and I’m sure the authors of this reports’) desire to improve the state of higher education in the developing world, it can be tempting to leave aside objectivity in favour of a positive, pro-technology narrative. The reality, however, is that there are systemic challenges to broader adoption of MOOCs in the West, and these challenges are further amplified in a developing world context.

As I begin to analyse my own survey data, this report allows me to reflect on some of the limitations of my own study, and the need to be cautious when reporting findings. Given the ways in which I have recruited participants, I know for a fact that my sample is biased. Almost a quarter of my respondents claim to have taken 9+ MOOCs. This is not the norm. Further, as a large proportion of respondents were contacted through university and academic intermediaries, I know that the demographics are going to skew in favour of younger, college going students. I am also aware that my respondents are not just educated, but also possess a high speed internet connection and the digital literacies needed to find, enrol, and likely complete MOOCs, making them almost certainly (economically) part of the top 10% of the Indian population. I have to be careful when making generalizations from my study into the broader population, and put the sample in context when considering the benefits (if any) for society. But of course, “Advancing MOOCs for the top 10% of the developing world” doesn’t make for as fundable a project.

Study on MOOC Learners from India – A Call for Participation

Since the mainstream success of Massive Open Online Courses (MOOCs) in 2012, millions of learners around the world have used these free online learning resources. Learners from India represent one of the largest non-US number of participants, with 10-15% of all enrolments in Coursera and edX coming from India. Similarly, we have seen numerous anecdotal accounts[1,2,3,4] of Indians who have taken MOOCs and then landed successful jobs, entrance into prestigious universities, etc.

Anecdotal accounts aside, there hasn’t been an empirical study of MOOC learners from India – who they are, where do they come from, and most importantly, what are their experiences and knowledge gained from these resources. With the Indian Government investing significantly in bringing MOOCs within the formal higher educational space in India, it is imperative that a cautioned, evidence-based approach is used to understand what role, if any, MOOCs can have in the Indian context. For this reason, I am conducting a pan-India cross-disciplinary study of MOOC learners from India, as part of my PhD at The Open University, UK.

I have designed a brief survey, that should take no longer than 5-7 minutes to complete. This survey will give us a baseline understanding of the types of learners that use MOOCs in India, and will be useful in our understanding of the rich and varied experiences of Indian learners, that can help course providers get a better understanding of their audience

If you have enrolled in a MOOC and are from India, please consider taking and sharing the following survey:

https://www.surveymonkey.com/r/Indianlearners

Further, if you are an academic in India, please consider sharing the survey with your students. This survey has already been distributed at a number of IITs and IIMs, and if you would be interested in learning more about MOOC usage at your institution or workplace, please contact me (details below), I can create a unique survey link for your institution, and share the findings of the same.

Lastly, if you have any questions about this study, or would just like to share your thoughts or discuss potential collaborations, please get in touch!

My Twitter handle is @janeshsanzgiri , and my email address is <janesh.sanzgiri@open.ac.uk>

 

Thank you for your interest in this study!

Recruiting Participants through Social Networks and the Challenge of Data

I write this post today on the 1st of February as my survey finally goes live – and hope to reflect through this entire chaotic process.

Since the start of my PhD, the one issue that has plagued my nightmares almost on a weekly basis has been my lack of data. The frustration of not having access to data has undoubtedly changed the direction of my research. While I never had a clear idea of what I wanted to research at the start of my PhD, a lot of my hesitation stemmed from uncertainty around data.

My data drought seemed all the more frustrating, when everyone around me seemed to be overflowing in data – almost not knowing what to do with it. I had times when I questioned whether PhD researchers can ever look at MOOC data, given how closely guarded it is. I remember meeting a prominent academic in the field at a conference and complaining about my lack of data – His response was probably the best advice you could give any starting PhD student – “Find an existing Project – Piggy Back on it – Get Data – Win”. I had tried that and failed, due to office politics. And now I, as well as my supervisors, were incredibly apprehensive about leaving my project in the hands of any other group, who at any time might pull out because they don’t owe me shit.

And so here I am – with almost a dozen different templates for spamming my survey through social networks and email. I know it’s going to be an uphill battle. Peoples’ attention spans on social media, especially with posts that might look like advertising, is incredibly low. There are a number of design decisions that have been made within the survey, its length, type of questioning, order of questioning, etc. that might address some of this – but first they need to click on the bloody link. Coupled with the fact that my target population could be literally anywhere – means I need to cast a net as wide as possible – literally.

I’ve provided incentive, much to the disagreement of my supervisors, in the form of a sweepstakes for 5 x £20 Amazon Online Vouchers. This may not seem like much to us, but to an Indian respondent, it might just make that clickbaity Facebook post seem more worthwhile.

I’ve created a list of literally every university in India’s Facebook group, and will be posting in all of them. LinkedIn group related to Indians? Sure I’ve joined them all. I’ve created a WhatsApp group message with overly gratuitous usage of the Indian Flag Emoji. There’s been a lot of effort put into perfectly phrasing a Tweet that triggers just the right emotional/patriotic response in less than 130 characters (Leave 10 to @ someone). As my survey is going to run for almost six months, there are multiple phases of desperation I will encounter in recruiting more participants. At a later stage, depending on responses, I even plan on @ Tweeting Bollywood celebrities with millions of followers, hoping they Retweet me to seem more socially conscious (I mention “help us improve #Indian higher #education” rather disingenuously in all my postings – Hey, don’t judge.).

At the end of the six months, I’d either have completely failed, or I might have invented a new research method of recruiting through social networks. Either outcome is scary to me. I hope to keep this blog as an account of this journey.