A comparison of self-paced and instructor-paced online
courses: The interactive effects of course delivery mode and
student characteristics
Sara Top olovec, Delft University of Technology
Keyw ords: MOOCs, online learning, self-p aced learning, stu d ent su ccess
1. IN TROD UCTION
Massive op en online cou rses (MOOCs) ap p eared in 2013. Since then, the nu m ber of cou rses has increased rap id ly, and p arallel to this, the cou rse form ats and m od es of d elivery keep d evelop ing as w ell. Lately, a grow ing nu m ber of MOOCs is offered in a self-p aced (or a self-p aced -ap p roaching) form at.
Self-p aced form ats can d iffer from one another, e.g. d ep end ing on t he p latform , institu tion, or even the cou rse itself, w hich also p resents a challenge to research this top ic, as there is no one d esign of a self-p aced cou rse. H ow ever, u su ally su ch cou rses are offered (op en) for a longer p eriod , all m aterials and activities are available from the beginning, and there is only one d u e d ate at the end of the cou rse. This m eans stu d ents can choose m ore flexibly w hen they w ant to stu d y. For exam p le, a certain stu d ent m ight com p lete the w hole cou rse in a cou p le of d ays of intensive stu d ying, w hile another stu d ent m ight w ork throu gh the cou rse over several m onths. As su ch, MOOCs are ap proaching the characteristics of offerings of op en ed u cational
resou rces (OERs), w here one can find variou s m aterials, from vid eo lectu res, to read ings, and exercises and exam s available at all tim es. In contrasts to OERs,
how ever, self-p aced MOOCs are p ackaged w ithin an id ea of a classroom , albeit a less stru ctu red one, w here stu d ents d rop in and d rop ou t at variou s tim es, bu t m ight still have a p ossibility of interaction w ith others, or even receive su p p ort from teachers or other staff m em bers.
The self-p aced m od e of d elivery can be attractive from several p erspectives, e.g. from a stu d ent, teacher, or organizational p ersp ective.
From a student perspective, the self-p aced form at offers increased flexibility, since stu d ents are only bou nd by one d u e d ate. This “p rom ise of tim e” cou ld be esp ecially beneficial for stu d ents w ho have d ifficu lty find ing tim e for stu d ying, for exam p le bu sy p rofessionals, or stu d ents w ithou t a com p u ter at hom e. In a self-p aced cou rse they can take ad d itional tim e to finish the cou rse. Stu d ies ind eed show that stu d ents
from less d evelop ed cou ntries are u su ally less su ccessfu l in MOOCs (e.g. H ennis, Top olovec, Poqu et, & Vries, 2016; Kizilcec & H alaw a, 2015; Kizilcec,
Perez-Sanagu stín, & Mald onad o, 2017), and one p ossible exp lanation cou ld be that they have few er p ossibilities d u e to a p oorer internet connection, or lack o f resou rces at their d isp osal at all tim es. Ad d itionally, the self-p aced form at can seem beneficial for stu d ents in general, becau se the m ain obstacle to com p leting cou rses is lack of tim e, or tim e-m anagem ent d ifficu lties, as rep orted in several stu d ies (Bonk & Lee, 2017; Kizilcec & H alaw a, 2015; N aw rot & Dou cet, 2014; Yeom ans & Reich, 2017).
From a teacher perspective, it m ay be easier to ru n a cou rse once over a longer p eriod in a self-p aced form at than m u ltip le tim es in an instru ctor -p aced form at, esp ecially if low er teacher involvem ent w ou ld be exp ected in su ch cou rses by d esign. Rhod e (2009) ind icated that stu d ents ind eed u nd erstand that interactions in a self-p aced cou rse are challenging, even thou gh they consid er interactions one of the m ost im p ortant p arts of their learning exp erience. This su ggests that having less teacher su p p ort and involvem ent available in su ch cou rses m ight not be incom p atible w ith stu d ents’ exp ectations.
From an organizational perspective, self-p aced cou rses m ight be beneficial in tw o w ays. Firstly, they can lead to better financial ou tcom es, for exam p le cou rses that are
offered over a longer period can attract m ore stu d ents, and m ore of them m ight bu y a certificate. Costs connected w ith every ru n can also be low er, d ep end ing on how the organization op erates and finances activities related to a given cou rse ru n . Ad d itionally, they can su p p ort organizational efforts and am bitions to contribu te to social good . Self-p aced cou rses can be easily available constantly, rather than once or tw ice a year in a fixed interval. This m eans the availability of op en ed u cation for stu d ents arou nd the w orld can be increased , and can this can contribu te to the am bition of op ening u p ed u cation even fu rther.
H ow ever, self-p aced cou rses m ay also com e at a p rice. Online cou rses are alread y challenging for stu d ents, the m ajority of them not com p leting the cou rses (e.g. H end erikx, Kreijns, & Kalz, 2017; H ennis et al., 2016; Reich, 2014), even those w ho intend to (Blackm ore, 2014; H end erikx et al., 2017; Kizilcec et al., 2017; Reich, 2014; Yeom ans & Reich, 2017; Wilkow ski, Deu tsch, and Ru ssell, 2014). Good
self-regu latory skills are im p ortant for stu d ent su ccess (H ood , Littlejohn, & Milligan, 2015; Kizilcec et al., 2017; N aw rot & Dou cet, 2014), w hich cou ld exp lain w hy stu d ents w ith higher ed u cational backgrou nd , and thu s m ore p reviou s learning exp erience, as w ell as old er stu d ents in online cou rses generally p erform better (e.g. H ennis et al., 2016; Kizilcec & H alaw a, 2015; Kizilcec et al., 2017). Self-regu latory skills cou ld be even m ore im p ortant in self-p aced cou rses, since the learning exp erience is less d irected from ou tsid e. Fu rtherm ore, low or alm ost nonexistent interaction that is often associated w ith self-p aced cou rses, cou ld be an im p ortant d ow nsid e since stu d ies show that stu d ents find interaction im p ortant (e.g. Kizilcec & H alaw a, 2015;
Rhod e, 2009). It is p ossible teachers cou ld foster an interactive self-p aced cou rse d esp ite asynchronou s learning p aths of stu d ents, how ever, it w ou ld likely requ ire a lot of effort and staff com m itm ent that the self-p aced m od e often tries to m inim ize. Desp ite its grow ing p rom inence as the choice for d elivery m od e, su rp risingly little research is d evoted to exp loring the self-p aced form at, its effectiveness, and its im p act on learning ou tcom es.
Sou thard , Med d au gh, and France-H arris (2015) com p ared the self-p aced and instru ctor-p aced d elivery m od e of a sp ecific cou rse, and conclu d ed that the self-p aced cou rse can be equ ally, or even m ore effective than the instru ctor -self-p aced cou rse. H ow ever, their d esign of the self-p aced cou rse d iffered from the instru ctor -p aced cou rse in m ore than ju st the m od e of d elivery; in their stu d y they alread y tried to offset the p otential shortcom ings of a self-p aced cou rse by ad ap ting the cou rse d esign, and by setting GPA requ irem ents need ed to enroll in the self-p aced cou rse. With that, rather than d em onstrating w hether the d elivery m od e can affect the
ou tcom es, they show ed that self-p aced cou rses can be as effective as instru ctor-p aced cou rses if certain d esign choices are m ad e. H ow ever, step s su ch as setting a higher GPA requ irem ent for stu d ents to enter the cou rse is not the best ap p roach to solving p ossible shortcom ings of a self-p aced m od e if the aim is to serve everyone arou nd the w orld , and even m ore so if the aim is to offer free ed u cation to those w ho have less access to it otherw ise. In another stu d y, Carey, Kleim an, Ru ssell, Venable, and Lou ie (2008) fou nd that both versions of the cou rse (self-p aced and facilitated cohort grou p ) w ere rated highly, and both w ere effective in altering teacher's p ed agogical beliefs and increasing their know led ge, w hich, together w ith their follow -u p stu d y (Ru ssel, Kleim an, Carey, & Dou glas, 2009), also ind icates that a w ell-d esigned self-p aced cou rse can be effective, d esself-p ite having m inim al interactions.
H ow ever, no research has focu sed on com p aring these tw o cou rse form ats m ore d irectly. A w ell-d esigner cou rse, w hether a self-p aced or an instru ctor-p aced w ill likely have a p ositive im p act on learning ou tcom es, how ever, d o self-p aced cou rses need d ifferent ap p roaches to their d esign to facilitate the best exp erience and
learning? Cou ld characteristics of a self-p aced cou rse affect stu d ent m otivation, enjoym ent, satisfaction, or learning? Does flexibility and m ore tim e actu ally lead to better ou tcom es?
The p resent stu d y aim s to exp lore the effects of the d elivery m od e of online cou rses (self-p aced com p ared to instru ctor -p aced m od e) on p erform ance and com p letion. While there have been m any stu d ies focu sing on p red iction of stu d ent su ccess, p erform ance, or retention in the p ast, both u sing behavioral characteristics and
stu d ent characteristics (e.g. Engle, Mankoff, & Cabrey, 2015; Gerlich, Mills, & Sollosy, 2009; Kenned y, Coffrin, & Barba, 2015; Lim , 2016), no su ch stu d y yet has focu sed on the com p arison of instru ctor-p aced and self-p aced d elivery m od es. This stu d y
focu ses on this com p arison sp ecifically, as w ell as on the interaction of the d elivery m od e w ith other characteristics that can influ ence stu d ent p erform ance in a cou rse, su ch as gend er and age, p reviou s exp erience, or learning preferences. With this, it aim s to exp lore w hether self-p aced form at ind eed ham p ers stu d ent p erform ance, as w ell as if the d elivery m od e has a d ifferent effect on d ifferent stu d ents. Su ch insights can su p p ort fu tu re d ecisions, for exam p le w hich step s need to be taken, or w hich interventions shou ld be d ep loyed in self-p aced cou rses, to su p p ort stu d ents m ost efficiently.
2. METHOD
2.1. Courses
This stu d y u ses d ata from 35 d ifferent cou rses (8 cou rses ran tw ice, for a total of 43 ru ns) offered by Delft University of Technology that ran in 2016 and 2017 on the ed X p latform . These cou rses w ere chosen becau se they had the sam e su rvey d ep loyed , they had no other exp erim ents ru nning in them , and they d id not have a p ossibility to receive an honor certificate (a certificate in the free track).
Cou rses d iffered in su bject, cou rse d esign, length, and d elivery m od e, for exam p le. Altogether there w ere 14 ru ns in p aced m od e. Of these, one cou rse ran as self-p aced tw ice, and 5 of cou rses also ran in the instru ctor -self-p aced m od e once d u ring the inclu d ed p eriod . There w ere 29 in stru ctor-p aced ru ns of cou rses, of these, one cou rse ran in the instru ctor-paced m od e tw ice.
2.2. Participants
In total, 12 739 p articip ants w ere inclu d ed in the analysis (7 923 from instru ctor-p aced cou rses, 4 751 from self-ctor-p aced cou rses). Stu d ents w ere inclu d ed based on the follow ing criteria: (i) they ind icated they intend ed to com p lete the cou rse in the p re-su rvey; and (ii) they enrolled before the start d ate in instru ctor -p aced cou rses, or 90 d ays or m ore before the end d ate in self-p aced cou rses). While stu d ents m ight still be able to p ass the cou rse even if they enroll late, the extent of this p ossibility greatly d ep end s on cou rse d esign itself, like the length of the cou rse, the nu m ber and w eight of grad ed assignm ents, and how d u e d ates are set u p . Therefore, a u niform cu t-off of enrolm ent before the start w as u sed in instru ctor -p aced cou rses. Self-p aced cou rses also d iffer greatly in term s of their op en p eriod , and the nu m ber of m od u les they contain, bu t stu d ents can still enroll after the cou rse op ens and have all the chance to p ass the cou rse, and enjoy the benefits of the self-p aced form at. Since one of the p rom ises of the self-p aced m od e is flexibility, w hich allow s stu d ents to w ork arou nd their sched u les and other resp onsibilities better, the cu t -off of three m onths before the cou rse end s w as u sed . This ensu res that stu d ents have m ore tim e available to
com p lete the cou rse than they w ou ld otherw ise have in a regu lar, instru ctor -p aced cou rse of certain length.
A d etailed list of all cou rses and related p articip ant nu m bers inclu d ed in the stu d y is available in Ap p end ix A.
2.3. Data
Pre-su rvey w as d ep loyed at the beginning (i.e. In the first/ introd u ctory m od u le) of all cou rses. The su rvey is stand ard and contains qu estions abou t stu d ents m otivation, exp erience, and d em ograp hics. For the p u rpose of this stu d y, the follow ing variables w ere u sed :
- gender: m ale or fem ale; d ue to low num bers, the “other” category w as exclu d ed from the analysis;
- age: self-reported age 16 years or more, and up to an d includ ing 80 years; - experience: w hether stud ents ind icated they had com pleted at least one online
cou rse before or not;
- education: cod ed as “higher” if the level w as bachelor level or higher, and coded as “lower” otherwise;
- English: level of English proficiency, self-rated on a 5-point scale from poor to very good ;
- relevance: to w hat extent stud ents agreed they enrolled in the course for their w ork/ career, self-rated on a 5-p oint scale from strongly d isagree to strongly agree;
- pace preference: the self-rated preference for follow ing the course at set pace as op p osed to their ow n p ace, rated on a 5-p oint bip olar scale;
- available hours: how m any hours stud ents rated they had available on average p er w eek, from 0 to 20 hou rs (integer);
- Human development index (HDI): H DI w as d eterm ined based on the country that w as recognized au tom atically by the su rvey tool. The H DI valu es are based on d ata obtained from “H u m an Develop m ent Rep orts” (n.d .).
Ad d itionally, the follow ing d ata w ere u sed in the analysis from the d ata file available from ed X:
- pass: ind icates w hether stud ents passed the course or not, and w ere therefore eligible for a certificate;
- track: w hether stud ents w ere enrolled in the free track (aud it) or paid for a certificate (verified ). This inform ation d oes not d ifferentiate betw een stu d ents w ho alread y p aid at the beginning of the cou rse, or later d uring the cou rse. Every stu d ents w ho w as enrolled in a verified track at the end of the cou rse is considered “verified”.
2.4. A nalysis
Du ring the p rocess of d ata p rep aration, the d u p licated resp onses for stu d ents w ere d ealt w ith in the follow ing m anner, and in this ord er:
(i) w ithin the sam e cou rse: All com p lete resp onses on aforem entioned su rvey variables w ere sorted by d ate, and the first resp onse w as kep t in the
analysis;
(ii) betw een d ifferent cou rses: one rand om resp onse (from a rand om cou rse) for each p articip ant w as kep t in the analysis.
For the aim of this stu d y, a logistic regression w as p erform ed w ith “p ass” as the ou tcom e, and other su rvey and p latform variables as p red ictors, inclu d ing the cou rse d elivery m od e (self-p aced or instru ctor-p aced ). Interaction term s of all these
p red ictors w ith the d elivery w ere inclu d ed in the m od el as w ell, and cou rses w ere inclu d ed as fixed effects. Other exp loration of d ata w as cond u cted to shed ad d itional light on resu lts, and su p p ort interp retation.
3. RESULTS & D ISCUSSION
This stu d y aim ed to research the effects of the cou rse d elivery m od e and its interactive effects w ith d ifferent stu d ent characteristics and ratings. Exp erience show s that self-p aced cou rses record low er com p letion nu m bers, and this can be observed in this sam p le as w ell: 11.60% stu d ents p assed the cou rse in the self-p aced cou rses, and 20.02% in instru ctor -p aced cou rses. The average p assin g rates p er
cou rse w ere sim ilar, 12.46% and 20.02% resp ectively. In fact, a logistic regression that fits the d elivery m od e as the only p red ictor ind icates the m od e is a statistically
significant p red ictor of w hether a stu d ent w ill p ass a cou rse or not (p<0.001). To exp lore the effect of the self-p aced m od e on cou rse com p letion, p articu larly in interaction w ith d ifferent characteristics, a logistic regression m od el that w as p erform ed inclu d ed several other variables as m ain effects, as w ell as their
interaction w ith the d elivery m od e. Since cou rses can d iffer very m uch in term s of length, d esign, d ifficu lty levels, etc., cou rses w ere inclu d ed in the m od el as fixed effects. The resu lts are p resented in Table 1, and are in general sim ilar to previou sly id entified factors related to stu d ent su ccess in MOOCs, su ch as a higher H DI, age, p reviou s exp erience, or higher ed u cation (e.g. H ennis et al., 2016; Kizilcec & H alaw a, 2015; Kizilcec et al., 2017).
Fu rtherm ore, the self-p aced m od e has an interactive effect on stu d ent su ccess in com bination w ith certain characteristics. The interactive effects betw een the tw o variables of interest are p resented in Figure 1.
Table 1
Results of the logistic regression with the outcome whether students passed the course or not.
estim ate SE z-valu e p
track (verified ) 3.51 0.10 35.56 < 0.001
exp erience (yes) 0.50 0.09 5.38 < 0.001
age_centered 0.02 0.003 4.66 < 0.001
p ace p reference 0.11 0.03 3.58 < 0.001
H DI_centered 1.22 0.35 3.46 < 0.001
available hou rs_centered 0.03 0.01 2.94 0.003
ed u cation (higher) 0.29 0.10 2.81 0.005 m od e (SP) : p ace p reference -0.13 0.05 -2.66 0.008 m od e (SP) : track (verified ) -0.37 0.15 -2.49 0.013 m od e (SP) : age_centered -0.02 0.01 -2.42 0.015 m od e (SP) : hou rs_centered 0.03 0.02 1.90 0.057 m od e (SP) : English -0.08 0.08 -1.01 0.313 m od e (SP) : H DI_centered 0.56 0.57 0.98 0.326 relevance 0.03 0.04 0.97 0.332
gend er (fem ale) -0.08 0.08 -0.92 0.358
m od e (SP) : relevance -0.05 0.06 -0.75 0.451
m od e (SP) : gend er (fem ale) 0.09 0.14 0.64 0.520
m od e (SP) : ed u cation (higher) -0.03 0.16 -0.21 0.831
m od e (SP) : exp erience (yes) 0.01 0.14 0.05 0.959
English 0.002 0.04 0.04 0.972
m od e (SP) 0.005 0.25 0.02 0.985
Intercep t -3.754 0.26 -14.64 < 0.001
Fixed cou rse effectsa
N otes. Centered variables w ere centered arou nd the m ean. Other continu ou s variables (5-p oint scales) w ere
centered arou nd the m id d le op tion. a
Cou rse effects are available in Ap p end ix B.
The relationship betw een cou rse su ccess and p reference for the cou rse m od e is reversed for self-p aced and instru ctor-p aced cou rses (Figure 1a): w hile stu d ents w ith a higher p reference for self-d irected p ace p erform slightly better in self-p aced
cou rses, the op p osite is tru e for instru ctor-p aced cou rses. This find ing is not u nexp ected , bu t it not necessarily relevant as som ething that can su p p ort cou rse d esign: cou rses are u su ally offered either in one m od e or the other at one tim e, w hich m eans stu d ents d o not have the op tion to choose a cou rse based on their p reference. H ow ever, if the cou rse w as offered in both m od es at the sam e tim e, and stu d ents cou ld choose, this cou ld lead to better learning and com p letion ou tcom es. Another p ossibility w ou ld be to m ake it p ossible to bring the self-p aced m od e of a cou rse closer to stu d ent p references for a set p ace w ith technical solu tions, e.g. if the p latform w ou ld allow stu d ents to set their ow n d u e d ates, w hich they w ou ld then need to resp ect.
Figure 1. (a) The relationship betw een self-rated p acing p reference and p assing rates
betw een tw o cou rse p acing m od es. (b) Differences in su ccess rates of au d it and verified stu d ents in tw o cou rse p acing m od es. (c) The relationship betw een age and p assing rates betw een tw o cou rse p acing m od es.
When it com es to the enrolm ent track of stu d ents (Figure 1b), a larger d ifference in p erform ance is observed for verified stu d ents in the self-p aced com p ared to
instru ctor-p aced cou rses, than for au d it stu d ents. To p u t it another w ay: even thou gh stu d ents in th e verified track m ore likely com p lete the cou rse in general, stu d ents benefit slightly less from a p aid track in self-p aced cou rses. This stu d y d id not take into accou nt at w hat p oint in tim e stu d ents u p grad ed to the verified track. Stu d ents w ho d id so alread y in the beginning m ight have been m ore m otivated than stu d ents w ho only p aid after they p assed the cou rse. H ow ever, it is p ossible that even the latter stu d ents cou ld see bu ying a certificate as a p ossibility if they p ass the cou rse even before they actu ally u p grad e, w hich cou ld also m otivate them to continu e. It m u st be noted , how ever, that the au d it stu d ents in the instru ctor -p aced cou rses alread y have very low p assing rates, w hich m eans that the p assing rates can d ecrease m u ch less than in the verified track.
Fu rtherm ore, old er stud ents have less of an ad vantage over you nger stu d ents in self-p aced cou rses (Figure 1c), w hich m eans they are m ore affected by the self-self-p aced m od e. This is an interesting find ing, becau se w e cou ld exp ect you nger stu d ents to be less exp erienced in self-regu lation and self-d irected learning, w hich are im p ortant asp ects for su ccess in online cou rses (H ood et al., 2015; Kizilcec et al., 2017; N aw rot & Dou cet, 2014), and p ossibly even m ore in self-p aced cou rses. H ow ever, sim ilarly as w ith the enrolm ent track, the you ngest stu d ents alread y have very low p assing rates in instru ctor-p aced cou rses, w hich m eans that their p erform ance in self-p aced
These resu lts su ggest that the self-p aced m od e cou ld have a m od erating effect on stu d ent su ccess throu gh other characteristics and asp ects, how ever, it m ight not affect all stu d ents equ ally. What d oes it m ean in relation to general low er p assing rates of stu d ents in self-p aced cou rses? There are tw o p ossible exp lanations, w hich can also be intertw ined .
Firstly, the course d esign m ight not su p p ort stu d ents in the best w ay in self-p aced cou rses, and one grou p cou ld be m ore affected . H ow ever, the effects of the m ajority of factors d o not interact w ith the d elivery m od e of the cou rse, w hich m eans that they are sim ilar in both m od es. Fu rtherm ore, age interacts w ith the m od e in an u nexp ected d irection, for exam p le, w hich m akes the interp retation based on self-regu latory skills, and its translation into strategies for cou rse d esign, m ore d ifficu lt. As ind icated p reviou sly, stu d ents w ith a higher p reference for a set p ace cou ld be su p p orted , how ever not as m u ch w ithin the self-p aced form at itself, since the natu re of the self-p aced cou rse is p recisely that – that it is self-p aced . Therefore, based on these resu lts alone, it is d ifficu lt to u nd erstand w hat, if anything, cou ld be d one to ad d ress this w ith the cou rse d esign and stu d ent su p p ort activities.
Second ly, it is likely that the au d ience in self-p aced cou rses is d ifferent than in instru ctor-p aced cou rses, w hich contribu tes to overall low er p assing rates in self-p aced cou rses. Ind eed , a look at the variables inclu d ed in the m od el show s several d ifferences betw een the self-p aced and the in stru ctor-p aced m od e. In self-p aced cou rses there are less stu d ents in a verified track (9.34% vs. 13.36%), w ith p reviou s exp erience w ith com p leting an online cou rse (64.88% vs. 69.16%), stu d ents are you nger (30.94 vs. 33.56% years old ), and slightly less ed u cated (74.07% vs. 80.42% with “higher” education), which are all factors connected with lower success in MOOCs in general. Therefore, it seem s that there can be slightly m ore stu d ents in the self-p aced cou rses w ith less favorable characteristics for cou rse su ccess, w hich can contribu te to overall low er com p letion rates.
H ow ever, it m u st be noted that the general d ifferences in characteristics cou ld also be accentu ated by the cou rses inclu d ed in this stu d y, esp ecially in the self-p aced track, w here only 13 d ifferent cou rses w ere inclu d ed . For exam p le, in the self-p aced grou p of cou rses there is a cou rse w ith m any p articip ants, and a very you ng au d ience on average. On the other hand , there is a cou rse in the instru ctor -p aced grou p w ith a m u ch old er au d ience on average than other cou rses, thou gh the effect in this case is sm aller. Ad d itionally, even if w e exp ect that the d ifference in au d ience d oes exist becau se of observed d ifferences in p assing rates, it is d ifficu lt to interp ret. For
exam p le, it is not easy to u nd erstand w hy self-p aced cou rses w ou ld attract a slightly you nger au d ience. Ad d itional research shou ld be cond u cted , for exam p le exp loring w hy stu d ents of d ifferent ages find the self-p aced m od e attractive.
This stu d y also has som e lim itations. First of all, w hile a relatively high nu m ber of cou rses w as inclu d ed in the analysis, the nu m bers are still rather lim ited and can bias the resu lts if w e consid er vast d ifferences betw een cou rses in term s of their su bject, d ifficu lty, d esign, etc., esp ecially w ithin the sm aller p ool of self-p aced cou rses. Fu rtherm ore, a cou p le of variables on a 5-p oint Likert scale w ere inclu d ed , and consid ered as continu ou s variables, w orking u nd er the assu m p tion that d ist ances betw een op tions are equ al, w hich m ight not hold tru e. Ad d itionally, this stu d y aim ed to exp lore factors of stu d ents su ccess w ithin a grou p of stu d ents w ho
ind icated they w anted to com p lete the cou rse. It can be argu ed that “com p letion” can be interp reted d ifferently by d ifferent stu d ents. For exam p le, som eone m ight
interp ret this as p assing the cou rse, som eone else as p articip ating in the cou rse u ntil the end , and the third stu d ent as p articip ating in all grad ed assignm ents. This m eans that the p assing rate m ight not necessary be the best m easu re for all stu d ents w ho indicate they want to “complete” the course.
4. CON CLUSION S
The p resent stu d y su ggests that the relationship betw een the cou rse m od e (self-p aced or instru ctor-(self-p aced ) and the su ccess in cou rse is m ore com (self-p lex than sim (self-p ly saying the self-p aced m od e is related to low er com p letion rates. While factors of su ccess are sim ilar in both m od es, p articu lar grou p s of stu d ents m ight be (m ore) affected by the self-p aced m od e. Fu rtherm ore, self-p aced cou rses m ight be m ore p op u lated by stu d ents w ith characteristics that are in general connected to low er su ccess rates, althou gh it is not necessarily easy to u nd erstand w hy, and esp ecially, how this can be translated to ad ap tations in cou rse d esign or p rep aration of
interventions that cou ld ad d ress this sp ecifically. More research is need ed into self-p aced d elivery m od e of cou rses, how it interacts w ith other asself-p ects, and how cou rse d esign and p latform solu tions can best su p p ort stu d ents in self-p aced cou rses.
REFEREN CES
Blackm ore, K. (2014). Measu res of su ccess: varying intention and p articip ation in MOOCs. In B. H egarty, J. McDonald , & S.-K. Loke (Ed s.), Rhetoric and Reality:
Critical perspectives on educational technology. Proceedings ascilite Dunedin 2014
(p p . 549-553).
Bonk, C. J., & Lee, M. M. (2017). Motivations, achievem ents, and challenges of self-d irecteself-d inform al learners in op en eself-d u cational environm ents anself-d MOOCs.
Journal of Learning for Development-JL4D, 4(1). Retrieved from
http :/ / w w w .jl4d .com
Carey, R., Kleim an , G., Ru ssell, M., Venable, J. D., & Lou ie, J. (2008). Online Cou rses for Math Teachers: Com p aring Self-Paced and Facilitated Cohort Ap p roaches.
The Journal of Technology, Learning, and A ssessment, 7(3). Retrieved from
http :/ / w w w .jtla.org
Engle, D., Mankoff, C., & Cabrey, J. (2015). Coursera’s Introductory Human Physiology Cou rse: Factors that Characterize Su ccessfu l Com p letion of a MOOC. The International Review of Research in Open and Distributed Learning,
16(2). Retrieved from http :/ / w w w .irrod l.org
Gerlich, R. N ., Mills, L. H ., & Sollosy, M. (2009). An Evalu ation of Pred ictors of Achievem ent on Selected Ou tcom es in a Self-Paced Online Cou rse. Research In
Higher Education Journal, 4, 1-14.
H end erikx, M. A., Kreijns, K., & Kalz, M. (2017). Refining su ccess and d rop ou t in m assive op en online cou rses based on the intention –behavior gap . Distance
Education, 38(3), 353-368. d oi:http s:/ / d oi.org/ 10.1080/ 01587919.2017.1369006
H ennis, T. A., Top olovec, S., Poqu et, O., & d e Vries, P. (2016). Who is the Learner: Profiling the Engineering MOOC Stu d ent. In SEFI 44th A nnual Conference,
Tampere, Finland.
H ood , N ., Littlejohn, A., & Milligan, C. (2015). Context cou nts: H ow learners' contexts influ ence learning in a MOOC. Computers & Education, 91, 83-91. d oi:http :/ / d x.d oi.org/ 10.1016/ j.com p ed u .2015.10.019
H u m an Develop m ent Rep orts (n.d .). Retrieved from http :/ / hd r.u nd p .org/ en/ com p osite/ IH DI
Kennedy, G., Coffrin, C., & Barba, P. de (2015). Predicting success: How learners’ p rior know led ge, skills and activities p red ict MOOC p erform ance. In
Proceedings of the Fifth International Conference on Learning A nalytics A nd Knowledge (p p . 136-140). Pou ghkeep sie, N ew York.
d oi:10.1145/ 2723576.2723593
Kizilcec, R. F., & H alaw a, S. (2015, March). Attrition and achievem ent gap s in online learning. In Proceedings of the Second (2015) A CM Conference on Learning@ Scale (p p . 57-66). ACM. d oi:http :/ / d x.d oi.org/ 10.1145/ 2724660.2724680
Kizilcec, R. F., Pérez-Sanagu stín, M., & Mald onad o, J. J. (2017). Self-regu lated learning strategies p red ict learner behavior and goal attainm ent in Massive Op en Online Cou rses. Computers & Education, 104, 18-33.
d oi:http :/ / d x.d oi.org/ 10.1016/ j.com p ed u .2016.10.001
Lim , J. M. (2016). Pred icting su ccessfu l com p letion u sing stu d ent d elay ind icators in u nd ergrad u ate self-p aced online cou rses. Distance Education, 37(3), 317-332. d oi:http :/ / d oi.org/ 10.1080/ 01587919.2016.1233050
N aw rot, I., & Dou cet, A. (2014, Ap ril). Bu ild ing engagem ent for MOOC stu d ents: introd u cing su p p ort for tim e m anagem ent on online learnin g p latform s. In
Proceedings of the 23rd International Conference on W orld W ide W eb (p p .
1077-1082). ACM. d oi:http :/ / d x.d oi.org/ 10.1145/ 2567948.2580054
Reich, J. (2014). MOOC com p letion and reten tion in the context of stu d ent intent.
EDUCA USE Review Online. Retrieved from http s:/ / er.ed u cau se.ed u /
Rhod e, J. F. (2009). Interaction Equ ivalency in Self-Paced Online Learning
Environm ents: An Exploration of Learner Preferences. The International Review
of Research in Open and Distributed Learning, 10(1), 1-23. Retrieved from
http :/ / w w w .irrod l.org
Ru ssel, M., Kleim an, G., Carey, R., & Douglas, J. (2009). Com p aring Self-Paced and Cohort-Based Online Cou rses for Teachers. Journal of Research on Technology in
Education, 41(4), 443–466.
d oi:http s:/ / d oi.org/ 10.1080/ 15391523.2009.10782538
Sou thard , S., Med d au gh, J., & France-H arris, A. (2015). Can SPOC (Self- Paced Online Cou rse) Live Long and Prosp er? A Com p arison Stu d y of a N ew Sp ecies of Online Cou rse Delivery. Online Journal of Distance Learning A dministration,
18(2). Retrieved 30 Oct 2017, from
http s:/ / w w w .w estga.ed u / ~d istance/ ojd la/ su m m er182/ sou thard _m ed d au g h_harris182.htm l
Wilkow ski, J., Deu tsch, A., & Ru ssell, D. M. (2014, March). Stu d ent skill and goal achievem ent in the m ap p ing w ith google MOOC. In Proceedings of the first
A CM conference on Learning@ scale conference (p p . 3-10). ACM. d oi:
http :/ / d x.d oi.org/ 10.1145/ 2556325.256624
Yeom ans, M., & Reich, J. (2017, March). Planning p rom p ts increase and forecast cou rse com p letion in m assive op en online cou rses. In Proceedings of the Seventh
International Learning A nalytics & Knowledge Conference (p p . 464-473). ACM.
Appendix A: Courses, their delivery mode, and numbers of participants
Table 2
Courses, their delivery mode, and numbers of participants
m od e N
AE1110x-3T2016 self-p aced 657
BMI.1x-3T2016 instru ctor-p aced 128 BMI.2x-1T2017 instru ctor-p aced 105
BMI.3x-1T2017 instru ctor-p aced 33
BMI.4x-2T2017 instru ctor-p aced 19
Bw N 101x-1T2017 instru ctor-p aced 212
Circu larX-1T2016 self-p aced 491
Circu larX-3T2016 instru ctor-p aced 77 CTB3300WCx-3T2016 instru ctor-p aced 268 CTB3365DWx-1T2017 instru ctor-p aced 89
CTB3365DWx-3T2016 self-p aced 151
CTB3365STx-1T2017 instru ctor-p aced 116
CTB3365STx-3T2016 self-p aced 167
DDA691x-3T2016 self-p aced 478
DDA691x-4T2016 instru ctor-p aced 102
DPB001x-2T2017 self-p aced 49
DPB001x-3T2016 instru ctor-p aced 259 EIT001x-3T2016 instru ctor-p aced 93 EnergyX-2T2016 instru ctor-p aced 602
EX102-1T2016 instru ctor-p aced 829
EX102-2T2016 self-p aced 332
EX103x-2T2016 instru ctor-p aced 385
Fram e101x-2T2017 self-p aced 99
GEO101x-1T2016 instru ctor-p aced 570
LfE101x-3T2016 self-p aced 1189
MathMod 1x-2T2017 instru ctor-p aced 193 MED01x-1T2017 instru ctor-p aced 211 MEP101x-3T2016 instru ctor-p aced 534
N GIx-3T2016 self-p aced 189
N UCLEAR01x-3T2016 instru ctor-p aced 237
OG101x-2T2017 instru ctor-p aced 84
OT.1x-3T2016 instru ctor-p aced 99
RCUC101x-1T2017 instru ctor-p aced 709 Sp atial101x-2T2017 instru ctor-p aced 150 TBP01x-3T2016 instru ctor-p aced 310
TP101x-1T2017 self-p aced 193
TP102x-3T2016 instru ctor-p aced 25
TPM1x-2T2016 self-p aced 1045
TW3421x-1T2016 self-p aced 367
TW3421x-3T2016 self-p aced 319
Urbanism X-1T2017 instru ctor-p aced 256 Visu al101x-1T2016 instru ctor-p aced 253 Visu al101x-2T2016 instru ctor-p aced 65
Total 12739
instructor-paced 7923
Appendix B: Course effects
Table 3
Course effects from the logistic regression presented in Table 1
estim ate SE z-valu e p
BMI.1x 1.18 0.34 3.46 0.001 BMI.2x 0.71 0.40 1.80 0.073 BMI.3x 1.31 0.53 2.49 0.013 BMI.4x 1.89 0.63 3.02 0.003 Bw N 101x 1.48 0.29 5.03 < 0.001 Circu larX 1.00 0.22 4.53 < 0.001 CTB3300WCx 0.24 0.33 0.72 0.471 CTB3365DWx -0.12 0.31 -0.39 0.700 CTB3365STx 0.55 0.27 2.01 0.045 DDA691x -0.26 0.26 -1.00 0.318 DPB001x 0.72 0.28 2.55 0.011 EIT001x 3.32 0.32 10.38 < 0.001 EnergyX 0.75 0.26 2.89 0.004 EX102 1.73 0.22 7.94 < 0.001 EX103x 1.00 0.27 3.71 < 0.001 Fram e101x 0.63 0.42 1.49 0.136 GEO101x 1,67 0.25 6.72 < 0.001 LfE101x 0.78 0.20 3.81 < 0.001 MathMod 1x -1.96 0.76 -2.58 0.010 MED01x 0.36 0.32 1.14 0.256 MEP101x -0.15 0.28 -0.53 0.597 N GIx 1.44 0.27 5.39 < 0.001 N UCLEAR01x 0.95 0.31 3.10 0.002 OG101x 0.63 0.42 1.49 0.137 OT.1x 1.16 0.37 3.14 0.002 RCUC101x 1.29 0.25 5.17 < 0.001 Sp atial101x -0.30 0.43 -0.70 0.487 TBP01x 0.50 0.30 1.67 0.094 TP101x -0.28 0.40 -0.71 0.479 TP102x 0.36 0.76 0.48 0.632 TPM1x 0.16 0.22 0.73 0.464 TW3421x 0.61 0.22 2.71 0.007 Urbanism X 0.60 0.30 1.97 0.049 Visu al101x 0.29 0.31 0.92 0.358
