learning management system research paper

Exploring user experience of learning management system

International Journal of Information and Learning Technology

ISSN : 2056-4880

Article publication date: 15 July 2021

Issue publication date: 12 August 2021

This paper aims to explore the perspectives of university students on the learning management system (LMS) and determine factors that influence user experience and the outcomes of e-learning.

Design/methodology/approach

This paper employs a mixed-method approach. For qualitative data, 20 semi-structure interviews were conducted. Moreover, for quantitative data, a short survey was developed and distributed among the potential respondents.

The results showed that students, particularly in programs where courses are mainly offered online, are dependent on such learning platforms. Moreover, the use of modular object-oriented dynamic learning environment (Moodle) as an application of LMS was rated positively, and e-learning was considered as an effective sustainable learning solution in current conditions.

Originality/value

The authors have illustrated empirically how the notion of UX of the LMS provides a means of exploring both students' participation in e-learning and their intention towards using such learning platforms.

• Computer-assisted learning
• Learning management systems
• User experience

Maslov, I. , Nikou, S. and Hansen, P. (2021), "Exploring user experience of learning management system", International Journal of Information and Learning Technology , Vol. 38 No. 4, pp. 344-363. https://doi.org/10.1108/IJILT-03-2021-0046

Emerald Publishing Limited

Copyright © 2021, Ilia Maslov, Shahrokh Nikou and Preben Hansen

Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode

1. Introduction

The development of the information and communication technologies (ICTs) and Internet have changed the way educational services are delivered in higher education ( Shaltoni et al. , 2015 ). In addition, the widespread use of learning management system (LMS) stands for a significant technological development in higher education and due to the challenging situation, many educational institutions have been forced to move to distant- and online-only education. In addition, the sudden and unexpected shift in teaching and learning modes has given rise to the use of information and communication technology to support learning. For example, modular object-oriented dynamic learning environment (Moodle) platform, an application of an LMS is being increasingly used to facilitate e-learning ( Dogoriti et al. , 2014 ; Lisnani and Putri, 2020 ). E-learning is an effective sustainable learning solution and offers tremendous opportunities for learning beyond the traditional boundaries. For example, increased reach to thousands of learners, facilitating the interaction between learners and educators, collaborative learning, and facilitating the teaching process planning ( Bansode and Kumbhar, 2012 , p. 415). However, despite the massive use of such learning platforms, there are multiple factors, which could potentially impact the outcomes and the results of e-learning. The two of which are (1) learner's perceptions about e-learning and (2) the information technology (including the quality, reliability, ease of use and usefulness) used to facilitate e-learning ( Benigno and Trentin, 2000 ; Sun et al. , 2008 ). While, the purpose of such systems is to provide quality education and training, students' intention to adopt and use LMSs is instrumental to the outcome of e-learning. In addition, the success of e-learning is highly dependent on the users' experience and perceptions towards such systems. The user experience (hereinafter UX) is a broad phenomenon describing how the LMS is perceived and used in e-learning processes. UX of both learners' and teachers' of LMS platforms, while considered to be crucial, influences the process of teaching and learning ( De Carvalho and Silva, 2008 ; Jeong, 2016 ; Nakamura et al. , 2017a ; Zaharias and Pappas, 2016 ; Zanjani, 2017 ).

Other interpretations of the UX add that UX explores how a person feels about using a product-e.g. the experiential, affective, meaningful and valuable aspects of product use ( Vermeeren et al. , 2010 ). Moreover, UX models are often regarded as objective part (e.g. functionality, reliability, usefulness and efficiency of the system) and subjective parts (e.g. attractiveness, appeal, pleasure, satisfaction of the system) ( Laguna Flores, 2019 , p. 23). Therefore, we use this distinction when interpreting the respondents' perceptions of UX. During the software development of learning systems, usability and user acceptance are considered highly significant as these systems are used by people with varied skill sets (administrators, students, teachers) ( Krishnamurthy and O'Connor, 2013 ). Simultaneously, in many software developments companies, usability and UX are either neglected or not properly considered. To resolve this, Ardito et al. (2014) suggested that public organisations should explicitly mention the usability and UX requirements in the calls for tenders for ICT products. In a recent study, Butt et al. (2020) argue that ICTs are the powerful and important tools for advancement, growth, reform, alteration, development and transformation in education (p. 350).

From a theoretical standpoint, current literature on e-learning and LMS are predominantly quantitative and focuses on technology adoption through the use of classical theoretical frameworks such as Technology Acceptance Model (TAM: Davis, 1989 ) and usability testing ( Nakamura et al. , 2017b , p. 1015). However, technology adoption and UX models are rarely compared with each other, despite that both allow for exploring the experiential component in human-computer interactions, providing rich insights about the factors that influence the adoption and the use of technology and how they are related ( Hornbæk and Hertzum, 2017 ). Studies (i.e. technology adoption and UX models) have significant overlaps in the phenomenon that they seek to explore: the experience of use and how it affects the actual and intended use ( Hornbæk and Hertzum, 2017 ). In the e-learning process, learners' feedback could be used at the evaluation stage for the consequent improvement of the process by the design team ( Khan, 2004 ). All organisations developing LMSs seek to improve their products, of which UX is a major part ( Cavus and Zabadi, 2014 , p. 525). The identification and evaluation of the UX elements addressed during the design of the product or service are crucial for innovation ( Krawczyk et al. , 2017 ). However, to date, none of the research in usability and UX of LMSs proposed solutions to the identified issues in usability and UX of studied LMS ( Nakamura et al. , 2017a ). As such, for a better evaluation of LMS in a given context, we argue that a qualitative-based research could be an alternative approach allowing users to express their perceptions and to make questions specific to the UX and the features of LMSs. With this approach, we respond to earlier Nakamura et al.’ s (2019) call who outlined the need for methods where it is possible to investigate UX of LMS, allowing users to supply a detailed overview of their experiences.

This paper aims to address this limitation by performing research in a pragmatic stance, which includes users' feedback on the UX of an LMS to propose potential solutions to identify issues of the UX and usability of an LMS. To do so, we use Moodle as an application of LMS and use university students as the potential users of LMS to perform our research. Moodle-based e-learning platform enables teachers to use multiple teaching tools like question banks, assignments, feedback, forums, and quizzes, enabling students to enrich their learning experience ( Bansode and Kumbhar, 2012 , p. 415). We explicitly use Moodle, as it is among the most widely used LMS platforms in the world, having up to 60% of the market share ( Kuran et al. , 2017 ; Machado and Tao, 2007 ; Teo et al. , 2019 ). While there are no significant differences in terms of features between different LMSs ( Al-Ajlan, 2012 ; Poulova et al. , 2015 ), differences might exist in terms of UX ( Sahid et al., 2016 ; Nichols, 2016 ) and different contexts of the LMS use, such as different cultures ( Wang et al. , 2013 , p. 76).

The research questions guiding this research are “ What factors impact the university students' perceptions of the UX of LMS?” and “what the potential solutions to the challenges of the UX identified by the university students”? To answer the research questions and to address the previously mentioned research gap, this qualitative research employs a holistic UX model developed by Topolewski et al. (2019) and conducted several semi-structured interviews with the informants.

The remainder of this paper proceeds as follows. First, we offer the literature review of the core concepts such as UX, e-learning and LMS, followed by a discussion of the theoretical framework. Second, the methodology is described. Third, then results are provided, and finally, we present discussion and conclusions.

2. Literature review

2.1 user experience (ux) and usability.

UX is associated with a broad range of fuzzy and dynamic concepts, including emotional, affective, experiential, hedonic and aesthetic variables ( Hassenzahl and Tractinsky, 2006 ).

The unit of analysis for UX is too malleable, ranging from a single aspect of an individual user's interaction with a standalone application to all aspects of multiple users' interactions with the company and its merging of services from multiple disciplines ( Sward, 2006 ).

The landscape of UX research is fragmented and complicated by diverse theoretical models with different foci such as pragmatism, emotion, affect, experience, value, pleasure, beauty, hedonic quality, etc. ( Law et al. , 2008 ).

There are also two opposing views on how UX should be studied and evaluated (i.e. quantitative and qualitative)–an argument rooted in the classical philosophical debate on reductionism versus holism ( Law et al. , 2014 ), despite that UX itself may change over time ( Fenko et al. , 2010 , p. 34). While, both UX and usability play important roles in measuring the quality of the use of LMSs and the e-learning process, these two may be considered as somewhat overlapping concepts ( Nakamura et al. , 2017b ). Usability is generally regarded as ensuring that interactive products are easy to learn, effective to use and enjoyable from the user's perspective. Usability has also several goals for interactive products. For instance, they must be effective to use, efficient to use, safe to use, having good utility, easy to learn, and easy to remember how to use ( Rogers et al. , 2007 , p. 20). Then, we may indeed see certain overlaps with the UX definition provided above in terms of how users evaluate their use of the product (where the key element in usability definition is use, so as in UX). Bevan (2008) argued that the UX and usability might share the goals of being efficient, effective and satisfying to use, but usability is more quantitative-oriented and more objective in nature (e.g. website speed and efficiency of work, frequency of the appearance of specific errors when evaluating the use of the system). While, on the one hand, usability can be considered as a part of UX, or as a separate concept measuring the use of the product objectively and pragmatically, very often with the quantitative-based techniques. The UX is then entirely subjective and hedonic, hence it is harder to evaluate it with quantitative methods. Thus, there are two distinct goals regardless of terminology: optimising human performance and optimising user satisfaction with achieving both pragmatic and hedonic goals ( Bevan, 2009 ). In our research, through the perspective of the UX model that is employed, we are addressing the usability as a phenomenon that is encapsulated in the broader UX.

In addition to the above discussion on UX and usability, it is important to address the user-centred design (UCD) as an important element of the UX model. UCD is an approach to design processes whereby a trained researcher observes and/or interviews largely passive or reactive users, whose contribution is to perform instructed tasks and/or give their opinions about the product concepts that were not generated by the users ( Sanders and Stappers, 2008 , p. 5). Furthermore, UCD can be characterised as a broad term to describe a broad philosophy, a variety of methods and approaches to design processes, whereby there is a spectrum of ways how users are involved in UCD ( Abras et al. , 2004 , p. 445). Detweiler (2007) considered UCD to be an iterative process of three phases: (1) understanding users (observing and interviewing end-users and other stakeholders to gather requirements), (2) defining interaction (creating use cases based on the output from phase one) and (3) user interface (UI) design (iterative creations and evaluations of prototypes). UCD then helps to design the product in a way that users need, and organisational goals are considered simultaneously, bringing value to both sides.

To this end, UX is very often about the value of the product and how this value is experienced by the users, so that organisational goals are met. Simultaneously, usability is very often dealing with the UI of an interactive product and how it is designed so that the tasks can be executed. This can be measured using efficiency, effectiveness and satisfaction. Thus, while employing UCD, both UX and usability are considered. In this research, we employ UCD to focus on users' needs, how the value of the product (LMS) is experienced by users, identifying potential challenges and then proposing the solutions, so that the organisation (i.e. university in our research) could meet better its goals (i.e. better educational outcome) in a pragmatic research approach.

2.2 E-learning

E-learning as a paradigm of modern education is the use of telecommunication technology to deliver information for education and training ( Sun et al. , 2008 ). E-learning participation refers to the teaching and learning facilitated and supported by Internet technologies ( Garrison and Anderson, 2003 ). E-learning is an iterative process that goes from the planning stage through design, production and evaluation to delivery and maintenance stages ( Khan, 2004 ). In this research, e-learning refers to the overall technological system for delivering teaching and learning, whereas participation in e-learning is the act of use of telecommunication to deliver teaching and learning within such a system. Fleming et al. (2017) identified (1) low perceived complexity of the e-learning system, (2) perception of the taught knowledge as useful and (3) availability of technical support, as some of the potential predictors of future use and overall satisfaction of using e-learning. In addition, personal perceptions about e-learning could influence attitudes and impact whether a user would intend to participate in e-learning in the future ( Sun et al. , 2008 ). Service quality (supportiveness of the service), information quality (learning content and interactivity) and system quality (attractive and ease of use of the website interface) are also different aspects of e-learning quality ( Uppal et al. , 2018 ). These might affect the aspects and perceptions of the UX of LMS. Hence, in this research, while e-learning is about the use of IT for providing teaching and learning, UX is about studying the use of those IT systems, and hence we draw connections between these two phenomena. Other research in the area is in line with this assumption as it has been argued that UX of LMS platforms may influence the process of online teaching and learning ( De Carvalho and Silva, 2008 ; Jeong, 2016 ; Nakamura et al. , 2017a ; Zaharias and Pappas, 2016 ).

Moreover, there are both advantages and disadvantages to e-learning. On a more positive side, e-learning allows for a learner-centred, self-paced, cost-effective way of learning. However, there is a lack of social interactions, potentially uncomfortable for some people, and higher degrees of frustration and confusion, with higher preparation time for instructors ( Zhang et al. , 2004 ). It should be noted that this research focuses mainly of e-learning in the university settings and we do not intend to focus on the pedagogical elements of e-learning or discuss the different actors involved in e-learning. We also do not discuss the rise of additional costs and investments in developing a mixed (hybrid) teaching model.

2.3 Learning management system (LMS) moodle

Web-based information systems are systems based on web technology and provide new approaches to design and development compared to the traditional computer software. LMS is a powerful software system enhancing learning ( Brusilovsky, 2003 ). Onofrei and Ferry (2020 , p. 1568) argued that such tools can be used to supplement traditional classroom teaching, and to enhance students' learning in a more efficient manner than students taught in a face-to-face learning environment. The LMS, as a type of e-learning tools, provides an automated mechanism to deliver course content and track learning progress ( Dalsgaard, 2006 ). There are two types of LMS: open-source and closed-source. Open-source LMSs are generally free of charge and customisable based on the user preferences at a low cost ( Bansode and Kumbhar, 2012 , p. 415). Al-Ajlan (2012 , p. 193) outlined a list of features of an LMS, which may be considered as components of an LMS, as shown in Table 1 . We would expect different features (components) of an LMS which impacts students' perceptions when evaluating their UX of an LMS. We also expect students to focus more on learning tools, which are visible to them, rather than on supporting and technical tools (see Table 1 ).

Moodle (Modular Object-Oriented Dynamic Learning Environment) is an open-source LMS ( Poulova et al. , 2015 , p. 1303). Moodle-based e-learning programs can be used to enable teachers to enrich students' learning experiences ( Bansode and Kumbhar, 2012 ). Moodle’s first prototypes were created by Martin Dougimas in 1999 and Moodle 1.0 was released in August 2002. While there are many LMS solutions of e-learning in the global e-learning market ( Pappas, 2013 ), Moodle is one of the most widely used e-learning platform in higher education ( Machado and Tao, 2007 ; Teo et al. , 2019 ). Other popular LMSs are Blackboard Learn and Canvas, in addition to custom-made LMSs ( Kuran et al. , 2017 ). Sheshasaayee and Bee (2017) stated that Moodle helps to find optimal ways of learning and optimal learning results and plays a vital role in terms of measuring student's knowledge, skills, and disciplinary practices (p. 738). Moodle offers a relatively acceptable user-friendly interface and whiteboard feature allowing to present the learning content clearly. Moodle provides communication and collaboration features (including real-time chat, discussion and sharing of files) for students and teachers ( Cavus and Zabadi, 2014 ). Therefore, it deems appropriate to use Moodle to assess and evaluate students' perceptions of UX of LMS. However, while we acknowledge Moodle has communicative features facilitating distance learning, there are certain limitations in doing everything completely online, such as in teaching philosophy, which demands a more personal and dialogical communication and pedagogical issues ( Vrasidas, 2004 ).

Poulova et al. (2015) in a comparative analysis of Moodle with three other LMSs (Blackboard, Claroline, EKP) asserted that Moodle's features do not basically differ from Blackboard and EKP, other than being free of charge. However, there might be greater differences between LMSs in terms of their UX. Moodle's UX is significantly better than Schoology's UX in terms of attractiveness, dependability and novelty ( Sahid et al. , 2016 ). Moodle's UX is found significantly worse than UX of university's custom-made LMS iQualify in terms of usability, navigational features, content and overall, as iQualify was specifically designed to the needs of the institution where it was used ( Nichols, 2016 ). Teachers' perceived usefulness have affected the university students' usage frequency of Moodle ( Wang et al. , 2013 , p. 76). The authors also found a difference in the communication mechanism in teaching, potentially due to the differences in culture and students' background, affecting the frequency of the use of different features. We would consider Moodle's technological functionality not differing significantly from other popular LMSs. However, overall UX is very likely to differ across different LMSs and universities and/or cultures. We assume this effect is due to the inherently subjective nature of the UX, which is dependent not only on the more objective concepts of the technological functionality but also on the user's perceptions, cultural background and the context of usage. Literature has identified some other potential factors proven to impact the use of LMS, e-learning and the UX. Potential factors affecting e-learning are gender, age, the experience of use, culture, race, family income, religion, political activities and cognitive aspects ( Maldonado et al. , 2011 ). Moreover, e-learning acceptance may be influenced by the course major and study level ( Al-Gahtani, 2016 ). The language of the website interface could potentially affect the UX, according to how translatable the original text is into another language and preferred by the user ( Bowker, 2015 ). Exchange student status is related to the individuals' learning style preferences ( Holtbrügge and Mohr, 2010 ).

3. Theoretical framework

The holistic UX model is developed by Pallot et al. (2014) who view UX as a multidimensional and multi-faceted construct due to the many different types of use experiences, including social and emphatical. Each type of experience is then decomposed into elements and properties that allow evaluation of its perceived quality. In this research, we employed the UX model from Topolewski et al. (2019) , which is an adaptation of the Pallot et al. (2014) original UX model. Some researchers have quantitatively verified the reliability and validity of the model (e.g. Krawczyk et al. , 2017 ; Topolewski et al. , 2019 ). In the adapted model Topolewski et al. (2019) , the societal dimension is excluded from the model, hence UX properties were considered to stand for human, social and business dimensions. The human dimension presents emotional and cognitive factors, social dimension presents emphatical and interpersonal factors, and business dimension presents the economic and technological factors. The definitions of the UX properties used to develop the questionnaire can be found in Appendix 1 .

4. Methodology

4.1 research design.

The following decisions were made regarding the data collection. For qualitative data, several semi-structured interviews were planned and conducted. Nakamura et al. (2017a) stated that for usability and UX evaluation of LMSs, interviews are widely used research techniques. Moreover, for quantitative data, a short survey was developed and distributed among the potential respondents. Levin (2006) stated that cross-sectional studies are often used because they are relatively inexpensive, but still allow to estimate the prevalence of outcome of interest (pp. 24–25).

4.2 Data collection

As mentioned, we are interested in evaluating the university students' UX, given that this demographic group is highly relevant end-users of LMS to determine the process of e-learning in higher education institutions. Hence, we focus on attaining a sample of university students at any level. We employed a convenience sampling strategy as it allows to choose participants who are available and easily accessible. Al-Gahtani (2016) , in the study of e-learning acceptance and assimilation, considered the convenience sampling technique as appropriate for researching the topic. Hwang and Salvendy (2010) stated that for usability evaluation of software products, there should be 8 to 12 respondents at the minimum. In total, 28 students were invited to participate, and the final dataset comprises of 10 male and 10 female students, hence the response rate was 71.14%. The students ( N  = 8) who rejected the invitation stated that they had busy schedule. However, the sample consisting of 20 participants is enough to perform qualitative analysis when testing users about the use of products ( Hwang and Salvendy, 2010 ).

We adopted the 24 questions developed by Topolewski et al. (2019) to collect both qualitative and quantitative data. The questions covered three main dimensions (business, human and social) of UX model with each dimension having two factors. The business dimension includes economic and technological factors, the human dimension includes emotional and cognitive factors, and the social dimension includes the emphatical and interpersonal factors. Of those 24 questions, 3 were used to evaluate students' intention to use LMS, particularly Moodle (see Appendix 1 ). In addition, we asked respondents to provide background information about their age, gender, length of using Moodle and/or other LMSs, and chosen Moodle's UI language. The data were collected between February and March 2020. The interview materials were then transcribed and imported to a Word document and were later analysed with qualitative analysis tool (NVivo). After data preparation, the researchers imported the data to the Nvivo, this allowed us to have a more nuanced data analysis (partially via the quantification of qualitative data). However, coding is not a substitute for deep and repeated immersion in the transcript data ( Campbell et al. , 2013 , p. 308), meaning researchers continually viewed original data transcripts. The recorded semi-structured interviews had an average length of 25 min; interviews lasted between 14 and 53 min.

For quantitative data, we asked the same respondents ( N  = 20) to answer the same 24 questions using a Likert scale from 1 to 7, where 1 being “very unfulfilling with UX property” and 7 being “very fulfilling with UX property” (see Appendix 2 ). The data were transferred into Excel spreadsheet and were consequently descriptively analysed. We performed simple descriptive statistical analysis on the answers given by the respondents on the 24 questions. The descriptive analysis helped us to evaluate the variables that may affect the UX and to evaluate the reliability and validity of the study results.

5.1 Descriptive analysis

As shown in Table 2 , the average age of the participants was 23.4, ranging from 20 to 31 years old. One female student chose not to reveal her age. There were 17 students from Finland, 1 from Russia, 1 from Kazakhstan and 1 from Italy. Finnish students mentioned that they used Moodle also in Swedish (with the exception of one male). A Russian female stated she used Moodle in English or in Finnish, whereas Kazakh and Italian students stated they only used Moodle in English. With the exception of an Italian student, all the other 19 students were from one Finnish university. There were 11 bachelor's level students (six females and five males), and 9 master's level students (four females and five males). Academic majors of the students vary significantly. All students reported that they were somewhat experienced with the use of the LMS. The average use of Moodle was 3.5 years, ranging between three months and seven years. Some students ( N  = 12) also had experience using other LMSs similar to Moodle with the average use of 1.35 years (six months to six years). In total, the combined average use of LMSs was 4.83 years. More descriptive information is provided in Appendix 2 .

Descriptive analysis of UX scores given by the respondents allowed us to find attributes that were considered as relatively important based on the means values. The most important attributes were (scores over 5): usefulness, pleasantness, productivity, reliability, efficiency, fulfilness, confidence, engagement, meaningfulness, respectfulness, as well as highly reported intentions to use. The highest value was given to question in relation to students' intention to use Moodle, “to which degree are you convinced of using Moodle in the near future” (Mean = 6.5). Some attributes of LMS (Moodle) such as user-friendliness, enjoyment, collaborativeness, comprehensiveness, communicativeness, attentiveness, helpfulness and responsiveness were evaluated as being good (scores between 4 and 5). The highest value was given to question in relation to the responsiveness of LMS (Moodle) system with the Mean = 4.91. At the same time, the descriptive results showed that there were some mildly low-rated elements (scores less than 4): entertaining, novelty, attractiveness. The highest value was given to question in relation to the attractiveness of LMS (Moodle) system with the Mean = 3.55. Standard deviation, a measure of how much variability there are in quantitative scores showed that standard deviation for most of the UX properties and intention to use was between 0.75 and 1.8–some elements having higher variability (e.g. Novelty) than others (e.g. Productivity).

5.2 Qualitative analysis

The qualitative data were analysed based on the holistic UX model developed by Topolewski et al. (2019) . Below, we provide detailed explanations of the interview data and elaborate on the findings according to the respondents' intention to use LMS (Moodle) system, and the three dimensions of UX model (the human dimension presenting [emotional and cognitive factors], social dimension presenting [emphatical and interpersonal factors] and business dimension presenting [economic and technological factors].

5.2.1 Economical

Regarding the effect of economic factors, the results showed that Moodle was perceived as useful and productive platform for learning, based on multiple perspectives. Moodle is useful because it provides an overview of the course together with all relevant course information provided by the teacher. Moodle helps with doing the assignments and with sending files to the teacher. Moodle is either informative or provides means for easy finding the information, the information was provided by ( N  = 11) interviewees. One interviewee mentioned that “ Moodle could be slow, such as loading documents in a browser window, which could be improved.” Another interviewee mentioned that “ group work could be improved in Moodle by implementing some sort of a feature for group work.” As much as ten interviewees mentioned that Moodle is easy for navigating and using it, but for some, it is a hard platform to navigate and use, or that it gets easier to use over time of experience. For some, Moodle may be a pleasant platform ( N  = 5), neutral ( N  = 2), or unpleasant ( N  = 1). One student cited very high productivity of Moodle “in the sense that it [Moodle] helps to monitor all tasks and assignments that I have to accomplish.” For many, Moodle is not entertaining. Moodle is perceived as a school app, and hence it does not have to be entertaining ( N  = 8). This might be improved by implementing brighter and more entertaining colours in the user interface (UI), like the university's colours.

5.2.2 Technological

Regarding the effect of technological factors, the results showed Moodle is generally not novel for multiple reasons, although partial novelty may remain. One is that Moodle is used for a long time. Moodle could be novel at first ( N  = 5) and it may not be novel if other LMSs were used before. There are either no problems in terms of reliability ( N  = 9), or only minor problems, mostly related to technical issues: service breaks, such as server downtime or website's inaccessibility, login issues. For some, reliability depends on the teacher. Moodle is intuitive, easy to use and generally efficient for studies with structured content. Efficiency depends on how the teacher uses Moodle. Moodle helped in several ways with the school tasks and assignments: helping to submit the files to the teacher, to manage the time, to access the information needed for completing the tasks, to manage the percentage of the course completion and the calendar. The interviewees mentioned some issues concerning the information in Moodle: there are occasional problems with the courses in Moodle, like having trouble locating how to add new courses or to categorise and to find course content ( N  = 7). This finding is consistent with ( Khan et al. , 2017 ) who also indicated that the informativeness of the course content in LMS is important factor to students experience of LMSs. Moodle's user-friendliness was mentioned with respect to the UX or the length of using Moodle, with the use of Moodle has become easier over time ( N  = 6). User-friendliness can be improved by categorising information in Moodle, making the enrolment to courses feature easier, providing a feature to filter or categorise the courses according to the user's criteria, making a tutorial “how to use Moodle,” and making more interactive links.

5.2.3 Emotional

Regarding the effect of emotional factors, the results showed the attractiveness of Moodle can be evaluated as mostly neutral, boring and dull, but it does not have to be attractive. Simultaneously, attractiveness can be judged as clear, simple, minimalistic. It is enjoyable or neutral to use Moodle, but it is a study tool; hence, enjoyment is not essential. Some interviewees found Moodle as easy and simple to use. Having everything in one place improved enjoyment. Enjoyment can be improved by implementing a chat function or adding mediums of communication in the courses (e.g. providing Q&A). Enjoyment depends on the course (e.g. interactive content in the course). One of the interviewees mentioned that it is joyful to use Moodle because of its design. Another interviewee indicated that using a mobile version of Moodle meant going to YouTube to view study content videos, thus having to view advertisements, which is unpleasant. Moodle is quite good in terms of its fulfilment, but for some, it may be neutral or bad. Moodle's fulfilment is affected by the teacher and/or by the student's efforts. As much as ten interviewees mentioned that Moodle can help to improve grades. The reasons provided include, (1) information provided regarding the criteria for how the assignments will be evaluated and (2) having an easy place to access useful information or reading extra material.

5.2.4 Cognitive

Regarding the effect of cognitive factors, the results showed that the opinions can be divided into comprehensiveness and helpfulness of the Moodle, as half of the interviewees evaluated as good, and the other half evaluated as neutral or bad. According to one interviewee, roughly, 30% of courses are badly structured, 50% are fine and 20% are good. Many commented communication methods are discussion forums, in-person communication, emails, personal messages, Wiki, Q&A section. Discussion forums are helpful and can have exciting discussions that help with an understanding of content. However, discussion forums can have a feeling of a “fake” or “unreal” discussion and difficulties of communicating. Personal communication or emails were frequently preferred to Moodle, especially with teachers. Comprehensiveness provided by Moodle differs for communication with students and with teachers. Teacher's encouragement can improve comprehensiveness. Moodle is meaningful and engaging, although depending on the teacher, the course content and how simply and it is structured. The content (e.g. articles) is the main factor affecting the meaningfulness of Moodle. Viewing different types of visual content (images, videos) to reading text material was preferred by some interviewees. The interactivity of the content and having everything in one place was engaging and helping to plan studies and can have a forced engagement: being told by the teacher, or because a student “has to.” Also, some mentioned that deadlines for assignments can increase the engagement. The following features in Moodle were also mentioned to be engaging quizzes, easiness of downloading documents, final course evaluation survey and percentage bar of completing tasks.

5.2.5 Emphatical

Regarding the effect of emphatical factors, the results showed that the attentiveness and responsiveness were mostly evaluated as good or relatively high, but with certain issues. Email notifications about new content posts were commented by one as: “Sometimes you get emails that are relevant, sometimes you do not, and sometimes you get emails every time, and that is very annoying.” Some responded to the notifications because they had to, some said that they always respond and some who mentioned that they never respond. Some preferred not to respond to others altogether. Responsiveness features included discussion forums, grades and Wikis. Some interviewees mentioned the direct messaging or chat functions were not presented to them, for others, while these functions were present, but did not work. A mobile app can improve responsiveness. Moodle is mostly helpful, which differed when communicating with students or teachers, and it depends on the course and/or teacher. Help can be received through forums, but some preferred other platforms, like email. Overall, communication is respectful but formal and official, with occasional issues when arguing, understanding each other's point of view and timing discussions to complete assignments.

5.2.6 Interpersonal

Regarding the effect of interpersonal factors, the results showed that the communicativeness is quite neutral, and collaboration is mostly unfulfilling. For some, it is possible to communicate or collaborate, often noting discussion forums, but prefer to use other mediums. Communication with students and with teachers is different: communicating with teachers is more widespread, but even then, many were found to prefer communicating with teachers by email. Personal communication is occasionally preferred, often at the premises of the university, especially for group work, unless there is a lack of time, and then other platforms are used. Discussion forums were evaluated as “old-school,” too formal, and not authentic enough; stating facts rather than communicating in mandatory discussions, which is a worse UX than instant messaging. Four interviewees mentioned that Moodle is used to find contact information to contact in other platforms: WhatsApp, Google Drive or email. Confidence in communication on Moodle is very high, because of formal communication with a free expression of opinions. Confidence was higher if a course required a password to enrol. Some lack of confidence was mentioned too, e.g. when students were unsure when others would reply, hence causing issues with last-minute mandatory discussions.

5.2.7 Intention to use

Regarding the students' intention to use LMS and in particular Moodle, the results showed that Moodle will be necessarily used without choice throughout the studies, and not afterwards unless it is a part of work, due to the lack of motivation and ability to log in after graduation. Willingness to use Moodle differed across interviewees widely. For example, some mentioned that Moodle is fine to use as a utilitarian tool, but not as an entertaining platform, like YouTube, as the former requires active, instead of the passive information consumption. Moodle is often recommended as an LMS, although opinions differ, depending on the experience of usage of other LMSs. Moodle can be recommended because it is easy to use, all relevant information is placed conveniently in the same place, easy to send tasks to the teacher for evaluation, easy to receive information from the teacher. Less frequently was mentioned that Moodle has a clear structure, offers basic functionality, it is efficient to use, easy for teachers to administer Moodle. Some mentioned that in Moodle it is possible to monitor the studying process, Moodle facilitates studying and that the design is liked. However, some mentioned that they would not recommend Moodle because of weak elements like communication, including group communication, user-friendliness of Moodle, navigation and use hardness, course enrolment, layout and/or UI, discussion forums, usability issues. Only one interviewee mentioned lack of customisability of the front page, lack of mobile app and navigation of Moodle. Moodle can be improved by making the course list clearer, optimising “technical stuff,” providing access for non-enrolled students to audit courses and improving relevance of content post notifications.

6. Discussions

In this research, a mixed-method approach was used to evaluate the university students' perception of UX of LMS particularly their perceptions towards Moodle as a learning platform. To do so, we interviewed 20 university students to find answers for our research questions “ what factors impact the university students' perceptions of the UX of LMS ”? and “ what are the potential solutions to the challenges of the UX identified by the university students ”? The results showed that students considered Moodle to be an easy and intuitive study-related tool that facilitates the learning. For several students, Moodle was found to be helpful in relation to the engagement of students in their studies through several features, such as deadlines and a completion percentage of the course. The results showed that many features of Moodle were rarely used, e.g. Wikis, whereas some were more frequently used, e.g. discussion forums. Moodle was also found to be generally quite reliable, with only minor technical issues that almost did not cause any problems. Many students stated they had to use Moodle, although having no problem with that, some even underlined that they are very dependent on Moodle in their learning. Most students stated that Moodle has an easy to use and navigating user interface (UI), although some did not. Regarding the challenges of the UX (RQ 2), the UI of Moodle was characterised as neutral, pastel, somewhat dull and not attractive, although did not concern students much.

The analysis of the qualitative data (answers to RQ 1) showed that Moodle was mostly used by the students in order to retrieve the contact details or information about the course. However, many also outlined the usefulness of the feature to send tasks to the teachers. Some students stated that looking up information on Moodle was better than looking for the information in the libraries. There were a few students who have characterised their UX as very limited, using Moodle just to upload/send the tasks or download documents and lecture slides. Although Moodle was not considered novel, for most, it was not a problem, some even suggesting that novelty may have a negative correlation with the ease of use due to the lack of experience and skills of using Moodle. Moreover, Moodle was frequently discussed in the perspective of using it together with other people in the social context or group dynamics. The UX of Moodle depends on how teachers structure the course in many of the UX aspects. Furthermore, many found the communication over Moodle to be formal and goal-oriented, yet dry and hypocritical. Certain features helped with communicating over Moodle, although these were limited, and may have to be improved. Discussion forums feature was the most widely mentioned feature, which was characterised by many of the students as old-school, although with some potential use if it were improved to be more modern with the chat function and group communication. Furthermore, the use of communicating features of Moodle was found to be different if the communication was with teachers or with other students. To compensate for the lack of communicating and collaborative functionalities of Moodle, thus impacting the students' UX, many students referred to using other platforms: YouTube, email, Google Drive, WhatsApp and Facebook. Additionally, other platforms were mentioned as affecting the UX of Moodle; for example, notifications sent to emails help students to be more attentive. Some found the UX to be improved by other platforms, whereas some stated that the UX of Moodle to be negatively affected.

6.1 Theoretical contributions

The theory of a holistic UX is currently in the early stages of validation within different research contexts. So, it is important to underline that the definition and understanding of the holistic UX (that is, the UX is viewed with a full and comprehensive set of factors that affect UX) is still limited ( Tokkonen and Saariluoma, 2013 ; Van Schaik and Ling, 2008 ). By using a UX framework of Topolewski et al. (2019) , this paper theoretically contributes to the holistic UX of an LMS research by providing manifold insights about such learning platforms.

First, we theoretically contribute to holistic UX theory by evaluating the applicability of the holistic UX model. We found that the questions about the factors of the holistic UX model tend to elicit respondents' qualitative answers that are quite broad, which also encompass perceptions and experiences simultaneously about multiple features (or components) of an LMS as well as contextual factors. For example, emotional and cognitive factors tend to include students' comments that teacher's proficiency in organising course content may promote students' engagement in using LMS. Other web-systems (like YouTube or Outlook Email) were also frequently mentioned when evaluating UX. Furthermore, this calls for a question of trying to view the UX as not being strictly bounded to the UX product itself, but rather to the product, other products that are used in conjunction and the context simultaneously. The following factors were mentioned and seem to have affected the UX of an LMS: web-platforms (including UX of previously used LMSs), teachers' use of an LMS, contextual factors, organisational context and the use of an LMS by others.

Second, a holistic view of UX of the LMS in our study seems to go quite broad, encompassing not only the UX of the LMS (Moodle) itself, but also the whole context, hence UX of an LMS is reported to be dependent on a system consisting of other phenomena: Internet infrastructure, communal, cultural and organisational norms and boundaries, among other potential factors that may exist.

Third, this research by employing the holistic UX in the context of LMS found that the UX of Moodle as an LMS and its perception vary across students. Many students stated that their learning in the university is very dependent on the LMS, particularly in the programs where courses are predominantly online. However, the use of LMS was limited for some students, who claimed that they preferred traditional learning to e-learning over LMS. Generally, Moodle as an LMS is not viewed as an entertaining or joyful platform and should be considered different from a platform like YouTube. Social context and use by others are essential factors in the UX of LMS. However, communication features were found to be weakly satisfying. Our results show that UX of LMS was found to be clear and good enough as a studying tool, but not very attractive. Other platforms were found to impact the UX of LMS: YouTube, email, Google Drive, WhatsApp and Facebook. These platforms were either replacing, complimenting or compensating certain UX elements and features (components) of an LMS. Additionally, how teachers used LMS was considered to be among the most influential factors in the students' UX perceptions of LMS. Although we would expect these findings to be highly case- and context-dependent (due to the nature of studied phenomena), we also could make an argument that a similar study about students' holistic UX of LMS in other contexts may have similar findings. From a more theoretical standpoint, these findings could be used to guide further research into exploring specific parts of the UX of an LMS.

6.2 Practical implications

The second research question was, “ what are the potential solutions to the identified issues of the UX of the students? ” Many students were not aware of the existence of all Moodle's features. Sometimes even not aware of the features, which were proposed to be implemented in Moodle, like group chat function. While it can be argued that it is actually impossible to expect students to know all features, they could be informed by training programs or by popup windows in the UI, as proposed by one student. Several students suggested improving the content presented on Moodle by making it more interactive, engaging, and visually appealing. Some students outlined issues with navigation, mainly in the area of enrolling and navigating between courses. Students suggested improving the visual appeal of UI of Moodle by making UI more colourful while maintaining the already-present clearness and simplicity. However, UI was not found to be an essential priority, and thus it was suggested to spend resources to focus on improving other issues first. Many students suggested improving communication on Moodle by modernising discussion forum mechanisms, implementing private messages, chat function, group communication. Interestingly, many students stated that in principle, there are possibilities for proper communication over Moodle, but are not widely used. Therefore, it could be argued that the students' UX of LMS could potentially be enhanced, if the missing features and functionalities indicated are closely considered and added to the Moodle.

The students also pointed some interesting issues about the teachers' role, especially their role about communication over Moodle and the informativeness of the course content similar to Khan et al. (2017) findings. For example, it was suggested that teachers engage students in compelling, and engaging discussions. The students mentioned that teachers should provide alternatives to “state a fact” type of required discussion in course assignments, which were found to be boring and shallow. Some students stated that they liked using discussion forums if the discussion is personally relevant, for example, in their careers or personal interests. We also found that students appreciate the informativeness of content in LMS as this impact their experience with LMSs significantly. To overcome this issue, we advise teachers to facilitate a discussion over LMS that promotes students' career perspectives or answers to their interests. However, each situation is unique, and thus teachers are encouraged to look into methods to create more personally engaging discussions. It is strongly advised that universities consider issues over how to manage LMS in terms of published content, given that content is a significant element in the UX of LMS, and as such significantly impact students' e-learning.

Many students found that Moodle is just a “blank slate” tool from many of the UX properties' perspectives, with UX depending on how that tool is used. Additionally, many students stated that some teachers create course content that is more interesting than other teachers. Thus, it is highly proposed that special initiatives are taken to improve teachers' abilities to use LMS and to structure the content in an engaging way. In addition, Abusalim et al. (2020) argued that management and policymakers at the universities should focus on helping teachers shift to student-centred styles of pedagogies prior to making investments in IT infrastructure, indicating that investment in learning tools and IT alone does not necessarily improve the productivity. Based on the students' perception of LMS, it is advised to inform the teachers on their role as facilitators of the discussion over LMS, with many stating that teachers have the power to increase students' engagement in the discussions over the Moodle. An example of such an initiative could be organising training programs for the teachers, where they are taught how to structure the course content in a more engaging way. Another example could be to suggest the university administration to promote communities of practice of the teachers around using Moodle. Finally, we suggest universities to appoint dedicated personnel to maintain and improve the quality of content over Moodle through helping teachers structuring the content of their courses.

7. Conclusions

In this paper, we took the perspective of the UX of LMSs, analysing the changes in students' perceptions towards e-learning solutions. We observe that many features of LMSs (in particular, Moodle) are important determinants and positively influence students' perceived learning outcome and the UX of LMSs.

We contribute to the literature by providing insights on how the UX of the LMS can be improved. The areas for improvement concern advanced feature of Moodle, UI of Moodle, communication over Moodle and the use of Moodle by the teachers. The latter two are considered to be more critical issues than the other. Designers of LMSs may pay more attention to the UI, communication features and compatibility of Moodle with other web-platforms and the learnability of Moodle by students. Teachers and administration of universities, in general, may be advised to take active, somewhat more centralised roles to manage the UX of LMS, given that this may have a detrimental impact on the e-learning of students. Even though many students commented that Moodle is user-friendly for the most part, navigating in the content between different courses were found challenging by some. Furthermore, the mobile version was found to be not very comfortable and compatible to work with, in addition to complaints regarding the mobile app of Moodle.

Moreover, we found that Moodle is not suitable for teaching philosophy completely online due to the lack of personal communication. Similar to Cavus and Zabadi (2014) who raised the importance of real-time synchronous discussion and chat function of Moodle, our findings also show that these features are very important determinants for the students. The findings show that the UX depends on how universities design and maintain Moodle. If Moodle is designed by experts and professionals, then the UX might be evaluated positively. If Moodle is designed by amateurs and maintained improperly (e.g. hosted on bad servers), then UX would suffer. Finally, this research contributes to the literature by proposing potential solutions to the problems identified in the UX of LMS (Moodle).

7.1 Limitations and future research

There are some limitations in this research. First, we have done our best to collect, document and analyse the data as carefully as possible. However, it is possible that not all issues of UX of LMSs were found. Second, the research context should be considered as a potential source of bias for the collected data and the findings. The data findings may not be directly extrapolated to other Moodle versions, LMSs or universities. Finally, the sampling technique could be an issue too. There are some recommendations and suggestions for future research. Future research may want to utilise a different methodology to explore the same issue. Future studies may attempt at refining the methodology that aims to elicit improvement of the UX of an LMS. A research dedicated to analysing the potential gender difference in evaluating the UX may be proposed. Some UX properties were more important (like helpfulness) than others (attractiveness), suggesting a need to evaluate in future studies, the priority of UX elements to focus on in future LMS development.

Features (components) of LMS

Descriptive statistics of the quantitative responses

Entertaining - Degree to which Moodle entertains users.

Pleasantness - Degree to which Moodle is pleasant to use.

Productivity - Degree to which Moodle helps users to be more productive.

Usefulness - Degree to which Moodle allows users to carry out tasks.

Novelty - Degree to which Moodle is new to the user.

Efficiency - Degree to which Moodle allows users to be efficient.

Reliability - Degree to which Moodle is reliable.

User - Friendliness - Degree to which Moodle is easy-to-use and intuitive enough.

Attractiveness - Degree to which Moodle is visually attractive.

Enjoyment - Degree to which Moodle is enjoyable.

Fulfilment - Degree to which Moodle allows users to achieve properly a task.

Comprehensiveness - Degree to which Moodle allows users to understand others.

Engagement - Degree to which Moodle allows users to engage in their tasks.

Meaningfulness - Degree to which Moodle allows users to provide meaningful results.

Attentiveness - Degree to which Moodle allows users to be attentive to others.

Helpfulness , - Degree to which Moodle allows users to help others.

Respectful - Degree to which Moodle allows users to be respectful of others.

Responsiveness - Degree to which Moodle allows users to be responsive to others.

Collaborativeness - Degree to which Moodle allows users to collaborate with others.

Communicative - Degree to which Moodle allows users to communicate to others.

Confidence - Degree to which Moodle allows users to trust others.

Convincingness - Degree to which users are convinced of using Moodle in the near future.

Willingness - Degree to which users are willing to re-use Moodle.

Recommend - Degree to which users are willing to recommend using Moodle in other universities.

Abras , C. , Maloney-Krichmar , D. and Preece , J. ( 2004 ), User-centred Design. Bainbridge, W. Encyclopedia of Human-Computer Interaction , Sage Publications , Thousand Oaks .

Abusalim , N. , Rayyan , M. , Jarrah , M. and Sharab , M. ( 2020 ), “ Institutional adoption of blended learning on a budget ”, International Journal of Educational Management , Vol. 34 No. 7 , pp. 1203 - 1220 , doi: 10.1108/IJEM-08-2019-0326 .

Al-Ajlan , A.S. ( 2012 ), “ A comparative study between e-learning features ”, Methodologies, Tools and New Developments for E-Learning , InTech Open Access , New York , pp. 191 - 214 , available at: http://cdn.intechweb.org/pdfs/27926.pdf .

Al-Gahtani , S.S. ( 2016 ), “ Empirical investigation of e-learning acceptance and assimilation: a structural equation model ”, Applied Computing and Informatics , Vol. 12 No. 1 , pp. 27 - 50 .

Ardito , C. , Buono , P. , Caivano , D. , Costabile , M.F. and Lanzilotti , R. ( 2014 ), “ Investigating and promoting UX practice in industry: an experimental study ”, International Journal of Human-Computer Studies , Vol. 72 No. 6 , pp. 542 - 551 .

Bansode , S.Y. and Kumbhar , R. ( 2012 ), “ E-learning experience using open-source software: Moodle ”, DESIDOC Journal of Library and Information Technology , Vol. 32 No. 5 , pp. 409 - 416 .

Benigno , V. and Trentin , G. ( 2000 ), “ The evaluation of online courses ”, Journal of Computer Assisted Learning , Vol. 16 No. 3 , pp. 259 - 270 .

Bevan , N. ( 2008 ), “ Classifying and selecting UX and usability measures ”, International Workshop on Meaningful Measures: Valid Useful UX Measurement , Vol. 11 , pp. 13 - 18 .

Bevan , N. ( 2009 ), “ What is the difference between the purpose of usability and UX evaluation methods? ”, Proceedings of the Workshop UXEM , Vol. 9 , pp. 1 - 4 .

Bowker , L. ( 2015 ), “ Translatability and UX: compatible or in conflict? Localisation focus ”, The International Journal of Localisation , Vol. 14 No. 2 , pp. 15 - 27 .

Brusilovsky , P. ( 2003 ), “ A distributed architecture for adaptive and intelligent LMSs ”, Workshop ‘Towards Intelligent LMSs,’ 11th International Conference on Artificial Intelligence in Education .

Butt , R. , Siddiqui , H. , Soomro , R.A. and Asad , M.M. ( 2020 ), “ Integration of industrial revolution 4.0 and IOTs in academia: a state-of-the-art review on the concept of Education 4.0 in Pakistan ”, Interactive Technology and Smart Education , Vol. 17 No. 4 , pp. 337 - 354 .

Campbell , J.L. , Quincy , C. , Osserman , J. and Pedersen , O.K. ( 2013 ), “ Coding in-depth semi-structured interviews: problems of unitization and intercoder reliability and agreement ”, Sociological Methods and Research , Vol. 42 No. 3 , pp. 294 - 320 .

Cavus , N. and Zabadi , T. ( 2014 ), “ A comparison of open-source LMSs ”, Procedia-Social and Behavioural Sciences , Vol. 143 , pp. 521 - 526 .

Dalsgaard , C. ( 2006 ), “ Social software: E-learning beyond LMSs ”, European Journal of Open, Distance and E-Learning , Vol. 9 No. 2 , pp. 1 - 7 .

Davis , F.D. ( 1989 ), “ Perceived usefulness, perceived ease of use, and user acceptance of information technology ”, MIS Quarterly , Vol. 13 No. 3 , pp. 319 - 340 .

De Carvalho , A.F.P. and Silva , J.C.A. ( 2008 ), “ The importance of usability criteria on LMSs: lessons learned ”, ICEIS No. 5 , pp. 154 - 159 .

Detweiler , M. ( 2007 ), “ Managing UCD within agile projects ”, Interactions , Vol. 14 No. 3 , pp. 40 - 42 .

Dogoriti , E. , Pange , J. , Anderson , S. and G. ( 2014 ), “ The use of social networking and LMSs in English language teaching in higher education ”, Campus-Wide Information Systems , Vol. 31 No. 4 , pp. 254 - 263 , doi: 10.1108/CWIS-11-2013-0062 .

Fenko , A. , Schifferstein , H.N. and Hekkert , P. ( 2010 ), “ Shifts in sensory dominance between various stages of user–product interactions ”, Applied Ergonomics , Vol. 41 No. 1 , pp. 34 - 40 .

Fleming , J. , Becker , K. and Newton , C. ( 2017 ), “ Factors for successful e-learning: does age matter? ”, Education+ Training , Vol. 59 No. 1 , pp. 76 - 89 .

Garrison , D.R. and Anderson , T. ( 2003 ), E-learning in the 21st Century: A Framework for Research and Practice , Routledge Falmer , London .

Hassenzahl , M. and Tractinsky , N. ( 2006 ), “ UX-a research agenda ”, Behaviour and Information Technology , Vol. 25 No. 2 , pp. 91 - 97 .

Holtbrügge , D. and Mohr , A.T. ( 2010 ), “ Cultural determinants of learning style preferences ”, Academy of Management Learning and Education , Vol. 9 No. 4 , pp. 622 - 637 .

Hornbæk , K. and Hertzum , M. ( 2017 ), “ Technology acceptance and UX: a review of the experiential component in HCI ”, ACM Transactions on Computer-Human Interaction (TOCHI) , Vol. 24 No. 5 , pp. 1 - 30 .

Hwang , W. and Salvendy , G. ( 2010 ), “ Number of people required for usability evaluation: the 10±2 rule ”, Communications of the ACM , Vol. 53 No. 3 , pp. 130 - 133 .

Jeong , H.Y. ( 2016 ), “ UX based adaptive e-learning hypermedia system (U-AEHS): an integrative user model approach ”, Multimedia Tools and Applications , Vol. 75 No. 21 , pp. 13193 - 13209 .

Khan , B.H. ( 2004 ), “ The people—process—product continuum in E-learning: the E-learning P3 model ”, Educational Technology , Vol. 44 No. 5 , pp. 33 - 40 .

Khan , I.U. , Hameed , Z. , Yu , Y. and Khan , S.U. ( 2017 ), “ Assessing the determinants of flow experience in the adoption of LMSs: the moderating role of perceived institutional support ”, Behaviour and Information Technology , Vol. 36 No. 11 , pp. 1162 - 1176 .

Krawczyk , P. , Topolewski , M. and Pallot , M. ( 2017 ), “ Towards a reliable and valid mixed methods instrument in UX studies ”, 2017 International Conference on Engineering, Technology and Innovation (ICE/ITMC) , IEEE , pp. 1455 - 1464 .

Krishnamurthy , A. and O'Connor , R.V. ( 2013 ), “ An analysis of the software development processes of open-source e-learning systems ”, European Conference on Software Process Improvement , Springer , Berlin, Heidelberg , pp. 60 - 71 .

Kuran , M.Ş. , Pedersen , J.M. and Elsner , R. ( 2017 ), “ LMSs on blended learning courses: an experience-based observation ”, International Conference on Image Processing and Communications , Springer , Cham , pp. 141 - 148 .

Laguna Flores , J. ( 2019 ), Determining the UX Level of Operating Computer Systems in the Central Bank of Mexico , Doctoral Dissertation , Massachusetts Institute of Technology , Cambridge, MA .

Law , E. , Hvannberg , E. and Cockton , G. ( 2008 ), “ Maturing usability: quality in software, interaction and value ”, Human-Computer Interaction Series , Springer-Verlag , Berlin .

Law , E.L.C. , Roto , V. , Hassenzahl , M. , Vermeeren , A.P. and Kort , J. ( 2009 ), “ Understanding, scoping and defining UX: a survey approach ”, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems , ACM , pp. 719 - 728 .

Law , E.L.C. , Van Schaik , P. and Roto , V. ( 2014 ), “ Attitudes towards UX (UX) measurement ”, International Journal of Human-Computer Studies , Vol. 72 No. 2 , pp. 526 - 541 .

Levin , K.A. ( 2006 ), “ Study design III: cross-sectional studies ”, Evidence-Based Dentistry , Vol. 7 No. 1 , pp. 24 - 25 .

Lisnani , L. and Putri , R.I.I. ( 2020 ), “ Designing Moodle features as e-learning for learning mathematics in COVID-19 pandemic ”, Journal of Physics: Conference Series , IOP Publishing , Vol. 1657 No 1 , p. 012024 .

Machado , M. and Tao , E. ( 2007 ), “ Blackboard vs Moodle: comparing UX of LMSs ”, 37th Annual Frontiers in Education Conference-Global Engineering: Knowledge Without Borders, Opportunities Without Passports , IEEE , pp. S4J - 7 .

Maldonado , U.P.T. , Khan , G.F. , Moon , J. and Rho , J.J. ( 2011 ), “ “E‐learning motivation and educational portal acceptance in developing countries” ”, Online Information Review , Vol. 35 No. 1 , pp. 66 - 85 .

Nakamura , W.T. , de Oliveira , E.H.T. and Conte , T. ( 2017a ), “ Usability and UX evaluation of LMSs ”, Proceeding of the 19th International Conference on Enterprise Information Systems , Vol. 3 , pp. 97 - 108 .

Nakamura , W. , Marques , L. , Rivero , L. , Oliveira , E. and Conte , T. ( 2017b ), “ Are generic UX evaluation techniques enough? A study on the UX evaluation of the edmodo LMS ”, Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE) , Vol. 28 No. 1 , p. 1007 .

Nakamura , W.T. , Marques , L.C. , Rivero , L. , de Oliveira , E.H. and Conte , T. ( 2019 ), “ Are scale-based techniques enough for learners to convey their UX when using a LMS? ”, Revista Brasileira de Informática na Educação , Vol. 27 No. 01 , pp. 104 - 131 .

Nichols , M. ( 2016 ), “ A comparison of two online learning systems ”, Journal of Open, Flexible, and Distance Learning , Vol. 20 No. 1 , pp. 19 - 32 .

Norman , D. ( 1999 ), Invisible Computer: Why Good Products Can Fail, the Personal Computer Is So Complex and Information Appliances Are the Solution , MIT Press , Cambridge, MA .

Onofrei , G. and Ferry , P. ( 2020 ), “ Reusable learning objects: a blended learning tool in teaching computer-aided design to engineering undergraduates ”, International Journal of Educational Management , Vol. 34 No. 10 , pp. 1559 - 1575 , doi: 10.1108/IJEM-12-2019-0418 .

Pallot , M. , Kalverkamp , M. , Vicini , S. , Trousse , B. , Vilmos , A. , Furdik , K. and Nikolov , R. ( 2014 ), “ An experiential design process and holistic model of UX for supporting user Co-creation ”, Open Innovation Yearbook .

Pappas , C.H. ( 2013 ), “ What are the most important features in a LMS? eLearning Industry ”, available at: http://elearningindustry.com/learning-management-systems-comparison-checklist-of-features .

Poulova , P. , Simonova , I. and Manenova , M. ( 2015 ), “ Which one, or another? Comparative analysis of selected LMS ”, Procedia-Social and Behavioural Sciences , Vol. 186 , pp. 302 - 1308 .

Rogers , Y. , Sharp , H. , Preece , J. and Tepper , M. ( 2007 ), “ Interaction design: beyond human-computer interaction”, netWorker: craft Netw ”, Computer , Vol. 11 No. 4 , p. 34 .

Sahid , D.S.S. , Santosa , P.I. , Ferdiana , R. and Lukito , E.N. ( 2016 ), “ Evaluation and measurement of LMS based on UX ”, 2016 6th International Annual Engineering Seminar (InAES) , IEEE , pp. 72 - 77 .

Sanders , E.B.N. and Stappers , P.J. ( 2008 ), “ Co-creation and the new landscapes of design ”, Co-design , Vol. 41 No. 1 , pp. 5 - 18 .

Shaltoni , A.M. , Khraim , H. , Abuhamad , A. and Amer , M. ( 2015 ), “ Exploring students' satisfaction with universities' portals in developing countries: a cultural perspective ”, International Journal of Information and Learning Technology , Vol. 32 No. 2 , pp. 82 - 93 , doi: 10.1108/IJILT-12-2012-0042 .

Sheshasaayee , A. and Bee , M.N. ( 2017 ), “ Evaluating UX in Moodle LMSs ”, 2017 International Conference on Innovative Mechanisms for Industry Applications (ICIMIA) , IEEE , pp. 735 - 738 .

Sun , P.C. , Tsai , R.J. , Finger , G. , Chen , Y.Y. and Yeh , D. ( 2008 ), “ What drives a successful e-Learning? An empirical investigation of the critical factors influencing learner satisfaction ”, Computers and Education , Vol. 50 No. 4 , pp. 1183 - 1202 .

Sward , D. ( 2006 ), “ Gaining a competitive advantage through UX design ”, available at: http://www.intel.com/it/pdf/comp-adv-user-exp.pdf .

Teo , T. , Zhou , M. , Fan , A.C.W. and Huang , F. ( 2019 ), “ Factors that influence university students' intention to use Moodle: a study in Macau ”, Educational Technology Research and Development , Vol. 67 No. 3 , pp. 749 - 766 .

Tokkonen , H. and Saariluoma , P. ( 2013 ), “ How UX is understood? ”, 2013 Science and Information Conference , IEEE , pp. 791 - 795 .

Topolewski , M. , Lehtosaari , H. , Krawczyk , P. , Pallot , M. , Maslov , I. and Huotari , J. ( 2019 ), “ Validating a UX model through a formative approach: an empirical study ”, 2019 IEEE International Conference on Engineering, Technology and Innovation (ICE/ITMC) , IEEE , pp. 1 - 7 .

Uppal , M.A. , Ali , S. and Gulliver , S.R. ( 2018 ), “ Factors determining e‐learning service quality ”, British Journal of Educational Technology , Vol. 49 No. 3 , pp. 412 - 426 .

Van Schaik , P. and Ling , J. ( 2008 ), “ Modelling UX with web sites: usability, hedonic value, beauty and goodness ”, Interacting with Computers , Vol. 20 No. 3 , pp. 419 - 432 .

Vermeeren , A.P. , Law , E.L.C. , Roto , V. , Obrist , M. , Hoonhout , J. and Väänänen-Vainio-Mattila , K. ( 2010 ), “ UX evaluation methods: current state and development needs ”, Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries , pp. 521 - 530 .

Vrasidas , C. ( 2004 ), “ Issues of pedagogy and design in e-learning systems ”, Proceedings of the 2004 ACM Symposium on Applied Computing , pp. 911 - 915 .

Wang , Y.H. , Tseng , Y.H. and Chang , C.C. ( 2013 ), “ Comparison of students' perception of Moodle in a Taiwan university against students in a Portuguese university ”, International Conference on Web-Based Learning , Springer , Berlin, Heidelberg , pp. 71 - 78 .

Zaharias , P. and Pappas , C. ( 2016 ), “ Quality management of LMSs: a UX perspective ”, Current Issues in Emerging eLearning , Vol. 3 No. 1 , pp. 58 - 64 .

Zanjani , N. ( 2017 ), “ The important elements of LMS design that affect user engagement with e-learning tools within LMSs in the higher education sector ”, Australasian Journal of Educational Technology , Vol. 33 No. 1 , pp. 19 - 31 .

Zhang , D. , Zhao , J.L. , Zhou , L. and Nunamaker , J.F. Jr ( 2004 ), “ Can e-learning replace classroom learning? ”, Communications of the ACM , Vol. 47 No. 5 , pp. 75 - 79 .

Corresponding author

Related articles, all feedback is valuable.

Please share your general feedback

Report an issue or find answers to frequently asked questions

Contact Customer Support

An official website of the United States government

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( Lock Locked padlock icon ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

• Publications
• Account settings
• Advanced Search
• Journal List

An Overview of the Common Elements of Learning Management System Policies in Higher Education Institutions

Darren turnbull, ritesh chugh.

• Author information
• Article notes
• Copyright and License information

Corresponding author.

Accepted 2022 Jun 14; Issue date 2022.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Learning management systems form an integral part of the learning environments of most universities and support a wide range of diverse activities and operations. However, learning management systems are often regulated by institutional policies that address the general use of Information Technology and Communication services rather than specific learning management system policies. Hence, we propose that learning management system environments are complex techno-social systems that require dedicated standalone policies to regulate their operation. This preliminary study examined a selection of learning management system policies from twenty universities in four countries to identify some of the elements that are considered necessary for inclusion in policy documents. Seventeen individual elements of learning management system policy documents were identified from a synthesis of the policies. These were classified into six policy categories: Accounts, Courses, Ownership, Support, Usage, and Protection. The study also identified three additional qualities of learning management system policy documents: standalone comprehensibility, platform-neutral statements, and contemporary relevance. The findings of this study will serve as a useful template for developing dedicated standalone policies for the governance of university learning management systems.

Keywords: Learning management system, LMS policy, Virtual learning environment, Online learning policy, University digital policy

Introduction

Learning management systems (LMS) are online software systems used to support various instructional, learning and assessment activities, and are central elements of many university course delivery systems (Turnbull et al., 2021 ; Weaver et al., 2008 ; Yueh & Hsu, 2008 ). The management and administration of LMSs is usually a centralized function in universities and other higher education institutions. Like other critical aspects of university operations, the effective administration of institutional LMSs depends on creating and communicating effective policies governing their use (Naveh et al., 2010 ). However, many universities do not have explicit policies dedicated to defining the parameters of LMS operations, such as acceptable codes of conduct for users engaging with these systems (Mohammadi et al., 2021 ). Instead, many institutions rely on generic policies covering Information Technology and Communication infrastructure. These policies do not necessarily cater to the LMS environment's unique characteristics as a complex, interrelated social system that technology-focused, infrastructure-heavy regulations cannot efficiently govern. Hence, there is a need for the development of dedicated LMS policies that not only address the technological environment of LMS platforms but also consider the human factors associated with people-to-people exchanges within this environment. This paper examines a cross-section of twenty LMS policies in universities from four countries with the aim of discovering some of the contemporary elements of policy development that emphasize the unique nature of LMS environments. In essence, this preliminary study is a snapshot of a cross-section of LMS policies in some of the world’s prominent higher education institutions.

The function of policy in an organization is to formally promulgate standard approaches to managing essential issues for its diverse stakeholders. A policy document is the tangible manifestation of rules and protocols that convey specific messages to various parties (von Solms & von Solms, 2004 ). A policy can best be understood in the context of the institutions and social relationships that give them purpose (Mosse, 2004 ). However, policy formation processes in organizations are often criticized because they lack consultative approaches that adequately consider the interests of the intended recipients of policy documents. Moreover, effective policies should recognize the changing and uncertain characteristics of the phenomena they address and be sufficiently malleable to accommodate substantial change. For universities that often exist and thrive in a perpetual state of flux, particular care must be taken to ensure that policy documents can survive technological and pedagogical upheavals that may impact their communities. The advent of COVID-19 has laid bare the necessity for universities to develop contingency plans to deal with major operational disrupters such as global pandemics (Rodrigues et al., 2020 ). LMS technology, associated pedagogical practices, and instructional design are proving to be critical elements in adapting to the post-COVID learning and teaching landscape. A key question that merits attention is whether the policies that drive the administration and operation of LMSs are up to this challenge?

An LMS learning environment is a complex techno-social system that requires institutional guidance from many diverse perspectives. At the core of an LMS’s existence is the technology that facilitates the functions necessary to carry out educational activities. These functions include learning material dissemination, stakeholder communication, student grading, progress monitoring, and records maintenance (Fathema et al., 2015 ; Turnbull et al., 2019 ). In choosing an LMS, universities can outsource their requirements to an external provider (such as Canvas) or adapt an open-source solution (such as Moodle). Open source LMSs offer the advantage of freely available source code that universities can adapt to suit their specific circumstances, and are unencumbered by recurring license fee expenses (Dobre, 2015 ). Conversely, proprietary solutions come bundled with ready-made quality tested modules, are relatively easy to deploy, and are supplied with ongoing technical support (Breskich et al., 2021 ). In either case, universities would need to develop appropriate policies to govern critical aspects of LMS use, such as mandatory adoption of an LMS platform by lecturers, the dissemination of learning materials, online announcements within the LMS, the establishment of discussion forums, the conduct of quizzes and tests, and the provision of feedback to students (Rafi et al., 2015 ).

University administrators must also consider whether established policies on general university issues need to be adjusted to accommodate the operation of LMSs in their unique cyber environment. For example, LMSs are capable of generating, via automated processes, vast amounts of student data that can be stored, analyzed and repurposed. However, many universities lack privacy policies that specifically address how this data will be used to inform organizational processes at a broad level (Brown & Klein, 2020 ). Another area of concern is intellectual property and the rights afforded to individuals and institutions that host material online. For LMSs based on open-source solutions, determining ownership of learning materials could be a simple trade-off between institutional and personal interests. However, this could be problematic for proprietary systems because the provider may have hosting terms and conditions that require materials and courses to be created to meet their corporate objectives (Pierson et al., 2013 ). To overcome this, universities could negotiate specific clauses in hosting agreements that clearly define ownership rights of course materials and other considerations such as acceptable use of course content by end-users engaged with the system (Pierson et al., 2013 ). Other general policy issues that could impact LMS use include respectful communication, sexual harassment, discrimination, and plagiarism. Governing bodies at universities have a duty of care to ensure that a regulatory framework is appropriately represented in the policy documents created for LMS use.

This paper presents the preliminary findings of a cross-section of prominent universities in four countries. The following section outlines the methodology adopted to select a cross-section of university LMS policies and analyze their content. In the results section, each selected policy's major elements are presented and coded into seventeen policy elements, further consolidated into six distinct categories. Next, the analysis and discussion section considers these policy elements in the context of LMS policy practice. This is followed by an implication for future policy section that presents a strategy for universities to create adaptive policy documents governing the adoption, maintenance, and use of university LMSs. The final section encapsulates the main arguments for creating dedicated LMS policies, while recommending that future research could help overcome the limitations of this study and contribute to the identification of other important policy elements that could be included in LMS policy documents.

The main aim of this study is encapsulated in the following research question: What are the main elements of university policies on LMS deployment and use that regulate the management of LMSs and control user access? In answering this question, it was decided to select a sample of publicly available online LMS policy documents from universities. The approach taken was to examine five prominent universities in each of four English-speaking countries: the United States of America (USA), the United Kingdom (UK), Canada, and Australia. The 2020 university rankings from the Times Higher Education World University Rankings publication were used to locate institutions in these countries. This annual publication ranks the world’s universities based on a weighted score comprising industry income, international diversity, teaching, research, and citations (Times Higher Education World University Rankings, 2020 ). Apart from the time and resource constraints required to keep the review manageable, these countries were selected because of similarities in the administrative structures of their universities and the policy-making processes that support the public dissemination of university policies. Another reason for selecting these universities was that their policies were available online in English.

Working down the list of universities in each of the four countries in order of ranking, we searched for publicly available policy documents governing LMS use. If a policy document was located, it was further examined to determine if it had sufficient detail and relevance to be included in this study. To be included, the selected policy documents had to address issues relating to general LMS use or focus on particular LMS platforms such as Moodle or Canvas. Policy documents created by LMS vendors and adopted by universities without alteration were rejected, as were documents that relied on general IT usage and acceptance without specifically mentioning LMS issues. All policies had to be accessible from university public websites. The advice from the Australian Law Commission is that information available on public websites that is not encrypted is a “generally available publication” (Australian Law Reform Commission, 2010 ). Prior studies of university security policies have successfully used this approach (Brown & Klein, 2020 ; Doherty et al., 2009 ). After a university was rejected, the next highest-ranking university in that country was selected for examination. This process continued until policy documents from five universities in each country were identified. In total, twenty policy documents were selected for analysis after rejecting ten policies that did not meet the selection criteria. Document analysis was primarily used to examine the content of the university policies. Document analysis involves the selection and appraisal of information from documents for synthesis into discrete categories (Bowen, 2009 ). In the context of this study, document analysis involved the systematic analysis of web-based policy documents and subsequent synthesis of policy elements into common categories. The results of this process are outlined in the following section.

Table 1 outlines the significant components of each university’s LMS policy. A commonly used abbreviation identifies each university. The last column of the table, LMS, indicates the name of any LMS platform referred to in the policy. If there was no mention of a particular platform, ‘None’ is indicated. A graphic representation of the distribution of LMS systems by country is displayed in Fig.  1 .

University LMS policies and their major elements

1 This abbreviation was modified from UC which denotes the University of Cambridge in this table.

LMS distribution by country

Canvas, an LMS designed and maintained by Instructure, was the most mentioned LMS platform in the examined policies. Instructure is a USA-based company founded in 2008 whose principal business model is vested in providing LMS solutions via Canvas (Instructure, 2021 ). Instructure maintains Canvas systems in all four countries to a varying degree but is most prominently represented in the policies of the US institutions: Caltech, HU and SUSM. The next most featured LMS was Moodle, an open-source system adopted by the LSE and UCL in the UK, and in Canada by Concordia. Blackboard, a proprietary system, was referenced in the policy documents in Monash in Australia and UB in the UK. Duke University’s Sakai-based policy was the only open-source system referenced in the USA documents.

The major policy elements identified in Table 1 were further examined in each document to identify a set of elements that could describe the entire range of issues collectively presented by the entire document sample. Using an iterative process, seventeen elements were eventually identified, and these were further segmented into six broad policy categories, as shown in Fig.  2 .

LMS policy elements by category

The relative merits of these elements in contributing to effective LMS policy formation form the basis of the discussion presented in the following section.

Analysis and Discussion

Figure  3 displays a distribution of derived policy elements substantially identified in each university’s LMS-related policy publication.

Distribution of policy elements by country of origin

These elements and their relevance to LMS policy formation are presented in the context of the six policy categories in the following discussion. Institutions mentioned in the discussion are also referred to by their commonly used abbreviation or short form, as defined in Table 1 .

Accounts (Account Responsibility, Account Unacceptable Use, and Passwords)

The primary way that access is granted to users of online resources is to authenticate the unique user accounts and passwords assigned to each user. This name-password combination represents the identity of the individual to be granted access to available resources (Shim et al., 2005 ). For LMSs, the user account also defines the role or function that the user can perform within the system. For example, there are different classes of users, such as students, administrators and teachers—all of whom are granted different access rights based on an assessment of their needs (Luminita, 2011 ). Both authentication (establishing the identity of a user) and authorization (granting permission to access specific resources or functions) are key security features that regulate LMS operations. Policies from both the University of Toronto and Concordia University had explicit statements that dealt with the formation and/or maintenance of user account passwords for their LMSs. It is certainly possible that the other institutions had policies that dealt with passwords that were embedded in more general policies. This is because many university information systems are integrated with a single sign-on, so separate LMS policies on user account characteristics are not necessary.

Responsibility for account use was a rather broad area that was addressed in the policies of nine institutions from all four countries: USA (Caltech, HU), UK (UO, UC), Canada (UBC, UT), and Australia (UQ, Monash, UCan). Issues covered here included resource sharing conditions such as restrictions on certain content, including viruses, defamatory materials and documents that might breach copyright regulations, content ownership and implied permissions granted to hosting services once materials are uploaded, specific account owner responsibilities by role, and access conditions such as obtaining parental consent if the user is not an adult.

In the final element in this category, unacceptable use, four universities provided clear examples of inappropriate usage (HU, UT, Concordia, UT). These included the use of fraudulent identification to create user accounts, allowing other people to access user passwords, potential conflicts of interest arising from student users granted administrator rights, and leaving logged-on accounts unattended.

Courses (Content, Enrollment, Quality)

The production, deployment, and maintenance of course-related assets within an LMS is a complex activity and difficult to codify in a policy document in a manner that applies to all users and situations. Nevertheless, numerous examples of course-related policy statements were cited in the reviewed documents. Guidelines on course content requirements, how to upload materials, content ownership, and student use of course content were featured elements of nine policies in all four countries: USA (Caltech, Duke), UK (UC, UCL), Canada (UR, UVic), and Australia (UQ, UA, UCan). Enrollment guidelines were mentioned in two policy documents: the University of Bristol specified that most students were enrolled automatically in the LMS based on registration in other student information systems, and the University of Pennsylvania provided details of procedures for enrolling students, faculty, and teaching assistants as separate groups of users. The final element in this category, course quality, was covered by several policies and included end-of-course procedures and general course quality issues such as removing old course content (University of Bristol), preserving a copy of gradebook information at the end of the semester (Duke University), the provision of resources to maintain course development quality (Macquarie University), advice on navigation aids to include in course materials (University College of London), and advice on acceptable reasons for concluding courses (University of Pennsylvania). End-of-course procedures are included in the quality category because of their importance in preserving the integrity of the information gathered over the duration of a course.

Ownership (Copyright, Intellectual Property)

The terms ‘intellectual property’ and ‘copyright’ are related concepts. Copyright in a university context refers to a legal right to distribute, reproduce, or sell academic works such as books, syllabi, and scholarly publications. By contrast, intellectual property encompasses principles of copyright but is considered a more comprehensive term that is particularly suited to dealing with issues raised by online ownership (Masson, 2010 ). In a sense, copyright issues could be considered subordinate to the principle of intellectual property. Policies from three universities made explicit reference to intellectual property or materials ownership (UC, Duke, UVic). Copyright was included as a major section in the LMS policy documents of six universities (Caltech, HU, SUSM, UB, UR, UA). The choice of the label ‘copyright’ instead of ‘intellectual property’ to convey ownership information is interesting. Nadel ( 2004 ) contends in his article investigating the impact of copyright law on creative output that the existing legal framework governing copyright may be responsible for reducing the incentive to create and share intellectual property. The term ‘copyright’ tends to be associated with the legal consequences of non-compliance, whereas ‘intellectual property’ is generally interpreted as a non-threatening descriptor of creations of the mind . Avoiding the use of language with punitive overtones such as copyright restrictions to identify the ownership of intellectual property may assist in promoting an environment that encourages the open exchange of ideas within the structured environment of modern LMSs.

Usage (LMS Acceptable Use, LMS Access Conditions)

LMS usage issues are a core concern of universities and represent the largest category of elements identified in the policy documents. Conditions that define acceptable parameters of LMS use, such as restrictions on use, communication regulations, conditions of sale for e-commerce purchases made via institutional LMSs, and fair use provisions, featured prominently in ten policies from institutions in three countries: USA (Caltech, HU, SUSM, DU), UK (UO, UC, UCL, LSE, UB) and Canada (UR). Access conditions that define the circumstances under which users may attempt to locate resources within the LMS environment are closely related to acceptable use. Examples of such conditions featured in the policies include user rights of access, access rights according to user role, and access duration limitations. These were prevalent in the policies in 11 institutions from all four countries: USA (HU, SUSM, Duke), UK (UO, UC, UB), Canada (UT, UBC, Concordia) and Australia (UC, UCan).

Support (Infrastructure, Help Services, Service Providers)

Support in this context refers to the resources and effort required to maintain a satisfactory level of LMS service to university communities. This could be provided externally by the LMS vendor or the university itself. LMSs are complex learning environments that require ongoing support to ensure their continued effectiveness. For end-users in particular, one of the essential ingredients to maintaining a positive engagement with LMS services is the knowledge and expectation that technical support is available when needed (Alshammari et al., 2016 ). The examined papers demonstrated support for LMS operations in three areas: infrastructure, help services, and service providers. Infrastructure support was an element applicable to the policies of two universities and included guidance on system maintenance and upgrade processes and other support issues (Penn, Monash). The University of Pennsylvania’s Canvas usage policy outlined conditions for providing help services to users of their Canvas system, while the University of Bristol provided a response-time policy for help requests from users of their LMS. Three universities (Caltech, DU, Monash) included information on service provider responsibilities in their policies. The support required to service LMS operations that ensure the user community is fully engaged with online learning environments is complex and subject to the unique operating conditions of each university. The cross-section of support-related statements identified in the policy documents indicates that institutional support is essential regardless of whether the adopted LMS platform is an open-source solution maintained by the university or an external proprietary system.

Protection (Privacy, Backups, Compliance, Reporting)

The protection category of elements comprises policy statements on privacy, information backups, compliance initiatives, and reporting mechanisms for instances of policy breaches. Data protection in the form of user data privacy is essential information that should be included in LMS policy statements. Universities have a duty of care to ensure that student interests are central to the formation of data privacy policies and practices that ensure LMS users are able to control how they engage with the technology, are non-discriminatory, and make data extraction and use transparent to the user community (Brown & Klein, 2020 ). One UK university, UC, and one Canadian university, UBC, explicitly dealt with privacy issues in their policies. Backups, the second element in this category, included statements from the policies of five universities on a variety of dimensions, including backup and recovery, course archiving, student record retention, and data storage (DU, LSE, UQ, UB, UCan). The compliance element that deals with breaches of acceptable usage rules was a prominent feature of eight university policies (Caltech, UO, UC, LSE, Concordia, UQ, UA, Macquarie). Policy statements in these documents included compliance guidelines, disciplinary actions, relevant applicable laws, and compliance exemptions. Reporting mechanisms for unauthorized usage of LMSs were covered in four policy documents (HU, UO, Concordia, UQ). Issues raised in these documents included reporting protocols to be followed in response to breaches of acceptable content rules, the need to report instances of copyright infringements when discovered, misuse of information by system users, and the circumstances under which reports on policy non-compliance are systematically provided to responsible committees. In summary, it was apparent that protection issues were a prominent concern of policymakers at the universities investigated, with multiple distinct inclusions of protection-related policy elements in universities from the four countries considered in this study.

Implications for Future Policy

This study has sought to explore self-contained university policy documents dedicated to LMS use that contain standalone statements regulating LMS administration and user engagement. In the course of our analysis of the 20 selected universities in Table 1 , it was discovered that some universities simply published material (or links to materials) that point to vendor documentation relevant to their product, such as Instructure’s Connect. All of the policy statements on LMSs listed in Table 1 are directed towards LMS administration and use and reinforce our argument that complex techno-social systems like LMSs require policy frameworks in their own right. By identifying common threads throughout all of the documents considered in this study, a list of possible components for an LMS policy template was developed, as indicated in Fig.  2 . This study has also highlighted additional non-content-related qualities of good LMS policy design worthy of inclusion in policy-making procedures. First, policies should be self-contained with sufficient detail in plain English to convey key information, so that reference to other documents is not necessary to understand particular policy elements. For example, the University of Regina’s web-based policy statement relating to the use of copyright material only contains a link to a general university policy on copyright (University of Regina, 2021 ) rather than explaining what copyright means for users of an LMS. By contrast, Caltech’s LMS copyright statements clearly define how these apply to the Canvas LMS in an organizational context (California Institute of Technology, 2020 ). Second, it is preferable to ensure that policy statements are platform-neutral. In other words, they do not embed the names or titles of specific LMSs within the document text. The University of Bristol’s LMS policy, while including many of the elements identified in this study, is structured around the Blackboard proprietary system and refers to ‘Blackboard’ as a synonym for all LMSs. This may serve to entrench Blackboard as the only term relevant to LMS deployment for stakeholders at the university, limiting or voiding the applicability of the Blackboard-based policy to other platforms that may be adopted in the future. A more technology-agnostic approach, such as Cambridge’s CLMS policy, maintains the focus of policy elements on the institution and its functions rather than the tool and the vendor. Finally, policy documents should be constructed in such a way that they can be easily modified and adapted to abrupt changes in circumstances that are difficult to anticipate (Walker et al., 2001 ). For example, the COVID-19 pandemic has forced many educational institutions to adopt digital learning as a necessary emergency measure to compensate for the lack of face-to-face instruction which has, in turn, led to the rapid development of policies and functional plans that could address the potential victimization of disadvantaged students coping with the new requirements of online learning (Karakose, 2021 ). Given the ongoing impact of COVID-19 on university operations worldwide, a thorough revision or replacement of LMS policies (should they exist) could improve their relevance to current operations.

Contemporary LMSs are complex techno-social systems that support a broad range of educational activities in modern universities. Policy documents are the principal means for university administrations to convey important information to stakeholders on how the adoption, maintenance, and use of these systems is to be regulated. Unfortunately, many universities neglect to develop standalone policy documents that specifically regulate the administration and use of LMSs. Rather, they rely on more generic policies such as Information Technology and Communication usage agreements, privacy regulations, and intellectual property rights to address specific issues arising from the use of complex LMSs. To help gain an understanding of the essential elements to include in an LMS policy, comparative exploration of the contents of existing LMS policy documents from a cross-section of prominent universities is a good starting point.

This preliminary study examined a snapshot of written LMS policies from twenty universities in four countries and identified seventeen elements that could be included in a policy design template. These elements were classified into six policy categories: Accounts, Courses, Ownership, Usage, Support, and Protection. In addition, three other qualities of LMS policy statements were also established: standalone comprehensibility, platform-neutral statements, and contemporary relevance. Together these categories and qualities provide a practical starting point for universities to design or enhance their LMS policy statements to better align LMS administration and use with stakeholder interests and broader community responsibilities.

This study is limited by the inclusion of English-only institutions from four countries based on a single ranking scheme. The use of additional rating metrics and expansion of the selection criteria to include prominent non-English language documents would significantly enhance the generalizability of the study. The authors also acknowledge that institutions not featuring prominently in university rankings may provide insights into LMS policy formation that could contribute to this study. Future studies could, for example, expand on our findings in this paper by exploring LMS policy formation in a cross-section of universities that specialize in providing educational services to disadvantaged communities or remote regional areas. The inclusion of internal documents, if available, could also provide valuable insights into LMS policy issues. Despite these limitations, significant insights into existing LMS policy elements were gained from this study. As Walker et al. ( 2001 ) contend, we can learn from the policy choices of others:

“Policy analysis should take into account the fact that the effects of policy choices depend on information about events that have happened and events that are yet to happen, including choices made by others.” (p. 283).

We hope this paper inspires future research into LMS policy formation to further help educational institutions design more cohesive LMS policies that address the concerns and interests of LMS stakeholders.

Author Contribution

All authors made an equal contribution to the development of this manuscript.

This submission has not been previously published and is not under consideration for publication elsewhere.

Open Access funding enabled and organized by CAUL and its Member Institutions

Declarations

Conflict of interest.

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Author Agreement

All authors have seen and approved this manuscript for submission to TechTrends.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Contributor Information

Darren Turnbull, Email: [email protected].

Ritesh Chugh, Email: [email protected].

Jo Luck, Email: [email protected].

• Alshammari SH, Ali MB, Rosli MS. The Influences of Technical Support, Self Efficacy and Instructional Design on the Usage and Acceptance of LMS: A Comprehensive Review. Turkish Online Journal of Educational Technology-TOJET. 2016;15(2):116–125. doi: 10.17718/tojde.762034. [ DOI ] [ Google Scholar ]
• Australian Law Reform Commission. (2010). Generally available publications . Australian Government. Retrieved 05/05/2021 from https://www.alrc.gov.au/publication/for-your-information-australian-privacy-law-and-practice-alrc-report-108/11-individuals-the-internet-and-generally-available-publications/generally-available-publications-2/
• Bowen GA. Document analysis as a qualitative research method. Qualitative Research Journal. 2009;9(2):27–40. doi: 10.3316/qrj0902027. [ DOI ] [ Google Scholar ]
• Breskich, V., Mishchenko, E., & Uvarova, S. (2021). LMS Technical Analysis: problems of choice. E3S Web of Conferences , 244 . 10.1051/e3sconf/202124407005
• Brown M, Klein C. Whose data? Which rights? Whose power? A policy discourse analysis of student privacy policy documents. The Journal of Higher Education. 2020;91(7):1149–1178. doi: 10.1080/00221546.2020.1770045. [ DOI ] [ Google Scholar ]
• California Institute of Technology. (2020). Canvas Acceptable Use Policy - Caltech . California Institute of Technology. Retrieved 13/4/2021 from http://teach.caltech.edu/faqs/canvas-acceptable-use-policy-caltech
• Cambridge University Press. (2019). Terms of Use: Cambridge LMS . University Printing House. Retrieved 12/4/2021 from https://www.cambridgelms.org/main/p/en/ouragreement/termsofuse/
• Concordia University. (2021). Security guidelines for on-line access to the Moodle course management system . IT Services. Retrieved 16/4/2021 from https://fcms.concordia.ca/moodle/terms_of_use.html
• Dobre I. Learning management systems for higher education - An overview of available options for higher education organizations. Procedia - Social and Behavioral Sciences. 2015;180:313–320. doi: 10.1016/j.sbspro.2015.02.122. [ DOI ] [ Google Scholar ]
• Doherty NF, Anastasakis L, Fulford H. The information security policy unpacked: A critical study of the content of university policies. International Journal of Information Management. 2009;29(6):449–457. doi: 10.1016/j.ijinfomgt.2009.05.003. [ DOI ] [ Google Scholar ]
• Duke University. (2020). Duke Sakai Appropriate Use Policy . Duke University. Retrieved 12/4/2021 from https://sakai-duke.screenstepslive.com/s/sakai_support/m/93106/l/1178494-policies-terms-of-service
• Fathema N, Shannon D, Ross M. Expanding the Technology Acceptance Model (TAM) to examine faculty use of Learning Management Systems (LMSs) in higher education institutions. Journal of Online Learning & Teaching. 2015;11(2):210–232. [ Google Scholar ]
• Harvard University. (2020). Harvard acceptable use policy for canvas . Harvard University Information Technology. Retrieved 12/4/2021 from https://harvard.service-now.com/ithelp/[email protected]?id=kb_article&sys_id=9d718485db0d57cc83a2f3f7bf961902
• Instructure. (2021). Our Company Story . Retrieved 16/4/2021 from https://www.instructure.com/about/our-story
• Karakose, T. (2021). The impact of the COVID-19 epidemic on higher education: Opportunities and implications for policy and practice. Educational Process International Journal, 10 (1), 7–12. 10.22521/edupij.2021.101.1
• London School of Economics & Political Science. (2020). LSE moodle terms of use . London School of Economics & Political Science. Retrieved 12/4/2021 from https://elearning.lse.ac.uk/moodle-terms-of-use.html
• Luminita DC. Information security in E-learning Platforms. Procedia - Social and Behavioral Sciences. 2011;15:2689–2693. doi: 10.1016/j.sbspro.2011.04.171. [ DOI ] [ Google Scholar ]
• Masson SR. Online highway robbery: Is your intellectual property up for grabs in the online classroom? Journal of Online Learning and Teaching. 2010;6(1):256–263. [ Google Scholar ]
• Mohammadi, M. K., Mohibbi, A. A., & Hedayati, M. H. (2021). Investigating the challenges and factors influencing the use of the learning management system during the Covid-19 pandemic in Afghanistan. Education and Information Technologies, 26 , 5156–5198. 10.1007/s10639-021-10517-z [ DOI ] [ PMC free article ] [ PubMed ]
• Mosse D. Is good policy unimplementable? Reflections on the ethnography of aid policy and practice. Development and Change. 2004;35(4):639–671. doi: 10.1111/j.0012-155x.2004.00374.x. [ DOI ] [ Google Scholar ]
• Macquarie University. (2021). Learning technology policy . Macquarie University. Retrieved 16/4/2021 from https://policies.mq.edu.au/document/view.php?id=188
• Monash University Studies Online. (2019). Guidelines for Blended and Online Learning . Monash University. Retrieved 18/4/2021 from https://www.monash.edu/__data/assets/pdf_file/0008/1051487/monash-university-studies-online-2013.pdf
• Nadel MS. How current copyright law discourages creative output: The overlooked impact of marketing. Berkeley Tech. LJ. 2004;19:785. [ Google Scholar ]
• Naveh G, Tubin D, Pliskin N. Student LMS use and satisfaction in academic institutions: The organizational perspective. The Internet and Higher Education. 2010;13(3):127–133. doi: 10.1016/j.iheduc.2010.02.004. [ DOI ] [ Google Scholar ]
• Pierson, M. W., Terrell, R. R., & Wessel, M. F. (2013). Massive Open Online Courses (MOOCS): intellectual property and related issues . National Association of College and University Attorneys Virtual Seminar.
• Rafi, A., Samsudin, K., & Hanafi, H. F. (2015). Differences in perceived benefit, use, and learner satisfaction between open source LMS and proprietary LMS. In E-Learning-Instructional Design, Organizational Strategy and Management. IntechOpen , 364-378. 10.5772/60670
• Rodrigues M, Franco M, Silva R. COVID-19 and disruption in management and education academics: Bibliometric mapping and analysis. Sustainability. 2020;12(18):7362. doi: 10.3390/su12187362. [ DOI ] [ Google Scholar ]
• Shim SSY, Geetanjali B, Vishnu P. Federated identity management. Computer. 2005;38(12):120–122. doi: 10.1109/MC.2005.408. [ DOI ] [ Google Scholar ]
• Stanford Medicine. (2021). Course Content Access and Appropriate Use Policy . Technology and digital solutions,. Retrieved 12/4/2021 from https://med.stanford.edu/edtech/services/LearningTechnologies/course-content-access.html
• Times Higher Education World University Rankings. (2020). The Times Higher Education World University Rankings 2020 . Retrieved 10/4/2021 from https://www.timeshighereducation.com/world-university-rankings/2020/world-ranking
• The University of Adelaide. (2018). SCHEDULE B: Minimum Use of MyUni, . The University of Adelaide. Retrieved 12/4/2021 from https://www.adelaide.edu.au/policies/669/?dsn=policy.document;field=data;id=5643;m=view
• The University of Queensland. (2021). eLearning - Procedure . The University of Queensland. Retrieved 13/4/2021 from https://ppl.app.uq.edu.au/content/3.80.08-elearning
• Turnbull, D., Chugh, R., & Luck, J. (2019). ‘Learning management systems: An overview’, In Tatnall A. (Ed.), Encyclopedia of Education and Information Technologies , Springer Nature, Cham, Switzerland. 10.1007/978-3-319-60013-0_248-1
• Turnbull D, Chugh R, Luck J. Issues in learning management systems implementation: A comparison of research perspectives between Australia and China. Education and Information Technologies. 2021;26:3789–3810. doi: 10.1007/s10639-021-10431-4. [ DOI ] [ Google Scholar ]
• University of Bristol. (2021). Blackboard policies . Digital Education Office. Retrieved from  http://www.bristol.ac.uk/digital-education/tools/blackboard/policies.html . Accessed 13 Apr 2021.
• University of British Columbia. (2021). Canvas & Privacy | Canvas at UBC . Learning Technology Support Hub. Retrieved 13/4/2021 from https://faculty.canvas.ubc.ca/canvas-privacy/
• University of Canberra. (2019). UCLearn (Canvas) teaching site publishing procedures . University Education Committee. Retrieved 17/4/2021 from https://www.canberra.edu.au/Policies/PolicyProcedure/Index/480
• University College of London. (2020, 2020–05–06). UCL Connected Learning Baseline . Retrieved 12/4/2021 from https://www.ucl.ac.uk/teaching-learning/publications/2020/may/ucl-connected-learning-baseline
• University of Oxford. (2019). Canvas VLE terms of use for academic and administrative staff . Retrieved 12/4/2021 from https://www.ctl.ox.ac.uk/files/canvastermsofusejune2019pdf
• University of Pennsylvania. (2017, 2017–10–24). Policies . University of Pennsylvania. Retrieved from  https://infocanvas.upenn.edu/about/policies/ . Accessed 12 Apr 2021.
• University of Regina. (2021). UR Courses Terms of Use | UR Courses, University of Regina . Retrieved 13/4/2021 from https://www.uregina.ca/urcourses/about/terms-of-use/index.html
• University of Toronto. (2018). Administrative Access and Confidentiality Agreement . Information technology Services. Retrieved 15/4/2021 from https://act.utoronto.ca/wp-content/uploads/2018/03/Quercus-Administrative-Access-and-Confidentiality-Agreement.pdf
• University of Victoria. (2021). LMS Policy - University of Victoria . University of Victoria. Retrieved 16/4/2021 from https://www.uvic.ca/til/services/coursespaces/policy/index.php
• von Solms R, von Solms B. From policies to culture. Computers & Security. 2004;23(4):275–279. doi: 10.1016/j.cose.2004.01.013. [ DOI ] [ Google Scholar ]
• Walker WE, Rahman SA, Cave J. Adaptive policies, policy analysis, and policy-making. European Journal of Operational Research. 2001;128(2):282–289. doi: 10.1016/S0377-2217(00)00071-0. [ DOI ] [ Google Scholar ]
• Weaver D, Spratt C, Nair CS. Academic and student use of a learning management system: Implications for quality. Australasian Journal of Educational Technology. 2008;24(1):30–41. doi: 10.14742/ajet.1228. [ DOI ] [ Google Scholar ]
• Yueh H, Hsu S. Designing a learning management system to support instruction. Communication of the ACM. 2008;51(4):59–63. doi: 10.1145/1330311.1330324. [ DOI ] [ Google Scholar ]
• View on publisher site
• PDF (1.3 MB)
• Collections

Similar articles

Cited by other articles, links to ncbi databases.

• Download .nbib .nbib
• Format: AMA APA MLA NLM

Add to Collections

• Open access
• Published: 14 October 2020

An analysis of users’ preferences on learning management systems: a case on German versus Spanish students

• Hasan Tinmaz 1 &
• Jin Hwa Lee   ORCID: orcid.org/0000-0001-6205-0634 2  

Smart Learning Environments volume  7 , Article number:  30 ( 2020 ) Cite this article

14k Accesses

11 Citations

1 Altmetric

Metrics details

The recent advancements in information and communication technologies have altered instructional contexts and re-shaped them into smart learning environments. One of the most common practices of these environments are learning management systems (LMS) where the learners and instructors utilize a software platform to fulfill, support and manage instructional activities around predefined objectives. Successful implementations of LMS have brought a variety on its usage from different cultures, genders, age groups or schooling levels. Hence, this study focuses on understanding the role of culture on LMS design, in along with the effects of gender, age and school year variables. The study participants were German ( n  = 83) and Spanish (n = 83) university students attending a fully online course offered by a South Korean university. At the end of the course, the students were asked to fulfill a survey on effective LMS design by pointing which features of LMS were more important for them. The survey included twenty questions on four major design factors; content management (six items), ease of use (five items), communication within LMS (four item) and screen design (five items). The dataset was analyzed by non-parametric statistical techniques around four variables on four dimensions (and their related survey questions). The most important result was insufficiency of one unique LMS design for all students which demonstrates the necessity of student demographics tailored smart systems. Additionally, age and gender variables were not making significant differences on LMS design as much as culture and school year variables. The study also revealed that while German students would appreciate goal-oriented individual learning, Spanish students would value process-oriented group learning with active communication. Furthermore, many features of LMS were highly valued by the freshman students more than other levels. The paper discusses these variables with possible explanations from the literature and depicts implementations for future design practices.

Introduction

Teaching and learning processes have continuously evolved with technological advances. Correspondingly, the rapid development of information and communication technologies has shaped traditional classrooms into smart learning environments. For instance, sharing learning materials online has enabled the learners to study whenever and wherever they want. Online attendance marking systems have also dramatically reduced the amount of time spent by the instructors to check their students’ attendance and thus increased the actual teaching and learning time. Moreover, the assignments or examinations can also be delivered online with appropriate instructions and feedback to support learning outside of scheduled classes. Likewise, there are many other ubiquitous combinations of pedagogical practices supported or facilitated with recent technologies.

Due to increasing number of available smart learning features, it has become indispensable to manage these features for effective and organized instructional processes. Currently, it is commonly seen that educational institutes operate their own Learning Management Systems (LMS) and provide various online smart learning features for a diverse group of students. An LMS is known as a web-based system that possesses an extensive range of pedagogical and course administration tools (Yakubu, 2019 ). Through these educational tools, LMS can facilitate group chats, discussions, document sharing, assignment submission, quizzes, grading and course evaluations (Bove, & A.,, & Conklin, S., 2020 ). Moreover, LMS has a potential to serve students with diverse backgrounds including culture, age or gender.

Previous studies have focused on identifying various learning features of LMS that can influence students’ learning outcomes. However, it seems that the results of previous studies were controversial with inconsistent learning outcomes of the students. One possible reason can be due to the lack of thorough understanding on students’ learning preferences, needs and diverse backgrounds. As the essence of an LMS is to facilitate self-regulated learning (Douglas & Alemanne, 2007 ), there is a need for analyzing and understanding users’ preferences when applying LMS in educational contexts which will serve various learning needs of the students.

As such, this study aims to analyze key factors that can influence users’ preferences on LMS use and gain a deeper understanding of how to maximize the learning outcomes through LMS by considering four essential independent variables; culture, gender, age and school years. The results of this study will contribute to a successful implementation of smart learning in class.

Literature review

The existing learning management systems literature has identified four major factors that need to be considered for successful implementation of LMS in order to fulfill students’ learning needs and expectations. In this part, four major factors which were yielded from the conclusions of the existing literature review were elaborated. These four factors which are relevant with LMS user experiences are culture, gender, age, and school year.

Cultural factor

When the researchers conducted literature review and analysis, it was unfolded that previous studies highlighted the importance of cultural factor in online learning. According to Hunt and Tickner ( 2015 ), culture is defined as “…a complex and multi-dimensional construct that represents the shared values, beliefs, and basic assumptions of groups of people. It includes elements such as language, customs, social behavior, and religion, and it influences how individuals relate to the world…” (p. 27). One of the most widely used models for understanding characteristics of cultural behaviors was developed by Hofstede ( 1986 ). In this model, he highlighted four dimensions of cultural behaviors or characteristics; ‘small versus large power distance’, ‘individualism versus collectivism’, ‘masculinity versus femininity’, and ‘tolerance of uncertainty and ambiguity versus uncertainty avoidance’.

First of all, power distance is defined as “the extent to which the less powerful members of institutions and organizations within a country expect and accept that power is distributed unequally” (p. 61) (Hofstede, Hofstede, & Minkov, 2010 ). For example, learning characteristics in high power distance cultures are more oriented towards one-way, directive, and instructor-based learning (Swierczek & Bechter, 2010 ). The second dimension, individualism versus collectivism, as mentioned by Mercado, Parboteeah, and Zhao ( 2004 ), means “the tendency of members of a society to act as individuals or members of groups, and to which a culture values individual versus collective achievement or well-being” (p. 185). It can be seen that individualistic culture is more results oriented whereas collective culture is more consensus and discussion oriented (Swierczek & Bechter, 2010 ).

In terms of the third dimension, masculinity versus femininity, a masculine culture shows clearly distinct emotional gender roles for men and women whereas feminine culture has overlapped emotional gender roles (Hofstede et al., 2010 ). Hence it seems that the learning characteristics in masculine society are achievement and competition focused rather than being affiliation oriented (Swierczek & Bechter, 2010 ). Lastly, uncertainty avoidance is defined as “the extent to which the members of a culture feel threatened by ambiguous or unknown situations” (p. 191) (Hofstede et al., 2010 ). With high uncertainty avoidance, guided and structured learning is preferred over independent and open-ended learning (Swierczek & Bechter, 2010 ).

Since our study involves German and Spanish students, their cultural differences were reviewed by using the four-dimensional model of Hofstede (Hofstede et al., 2010 ). It was noticed that Germany is a less power distant country than Spain and their relative power distance scores were 35 and 57 respectively. Thus, it is expected that Spanish students are more familiar with a hierarchical learning environment than German students. In terms of individualism versus collectivism, Germany and Spain obtained individualism scores of 67 and 51 respectively showing their different cultural views on individual versus group. It is expected that German students are more oriented towards individual learning and achievement. On the other hand, Spain’s score was the second lowest among the European countries reflecting higher cultural tendency towards collectivism. When masculinity scores are compared, Germany appears to be a masculine society while Spain is a feminine society with relative scores of 66 and 42 respectively. It implies that German students value high performance and competition-based learning whereas Spanish students prefer harmony and non-competitive learning. Finally, uncertainty avoidance scores for both countries seem generally higher than the average. The relative scores for Germany and Spain were 65 and 86 respectively. As Spain scored higher than Germany, it is predicted that Spanish students will be more reluctant to experience changes, ambiguities, and undefined situations in learning.

In addition to identifying learners’ characteristics, the four-dimensional model of Hofstede can be also used to analyze LMS acceptance levels in educational institutes. For example, Asunka ( 2016 ) identified cultural factors responsible for low LMS acceptance levels of university faculty members. Through a participatory action research approach, the study engaged 10 faculty members for one semester. Among the four cultural dimensions of Hofstede’s model, ‘power distance’ was identified as the most influencing factor followed by ‘individualism versus collectivism’ and ‘uncertainty avoidance’. Although this study applied the Hofstede’s model on instructors rather than students, it highlighted significance of cultural factors in LMS implementation. In addition, Tarhini, Hone, Liu, and Tarhini ( 2017 ) revealed that these four cultural dimensions play an important role in students’ technology acceptance level by influencing subjective norms. The study collected data from 569 undergraduate and postgraduate students in Lebanon using e-learning tools.

Although the Hofstede’s model was not directly applied, previous studies have also reported the impact of cultural factors on students’ learning performance and behaviors. For example, Liu, Liu, Lee, and Magjuka ( 2010 ) investigated the impact of cultural differences on international students’ learning performance. This study involved international students from India, China, and Russia who attended an online Master of Business Administration (MBA) program. Through this study, cultural factors including language, communication tool use, plagiarism, time zone differences, and a lack of multicultural contents were suggested as potential barriers for online learning that can affect students’ learning performance. Similarly, cultural influences on learning behaviors were reported by Swierczek and Bechter ( 2010 ). Their study performed qualitative and quantitative analyses of e-learning behaviors of participants from South Asia, East Asia, and Europe. The study results indicated that European students tend to be individualistic and prefer learning by induction whereas South and East Asian students value affiliation and avoid high uncertainty in learning. In addition, East Asian students appeared to be more active and involved in e-learning. The study suggested several cultural factors responsible for different learning behaviors which included language, technology, the role of instructor, and the level of interaction required. In other words, LMS designing and implementation should cater for various learning needs of students that can arise due to different cultural backgrounds.

The existence of cultural factors often creates cultural barriers to limit the potentials of online learning facilitated by LMS. Thus, it is inevitable to identify possible solutions to overcome these barriers. As mentioned above, Asunka ( 2016 ) reported existence of cultural barriers among the instructors that can trigger their anxiety, uncertainty, and indifference towards LMS usage. However, identification of responsible cultural dimensions, regular discussions, and monitoring the outcomes of LMS implementation throughout the semester made positive changes in the instructors’ views on LMS. It can be noted that identification of cultural variables, along with other possible variables, plays a critical role in the outcomes of LMS usage. To overcome cultural barriers, Parrish and Linder-VanBerschot ( 2010 ) investigated the influence of cultural dimensions in online learning, which involved social relationships, epistemological beliefs, and temporal perceptions. Using the cultural dimensions of learning framework as a diagnostic tool, possible solutions to overcome the challenges of multicultural learning were suggested as increased awareness, culturally sensitive communication, modified instructional design processes, and efforts to accommodate critical cultural differences.

Gender factor

Apart from cultural differences, another important factor drawn from the literature review conclusions is the gender difference. Previous studies have reported different characteristics of male and female students involved in online learning. According to a study conducted by Cuadrado-García, Ruiz-Molina, and Montoro-Pons ( 2010 ), male and female students showed significant differences in the assessment and use of e-learning activities. This study involved a bilingual e-learning project between two European universities. The study revealed that female students achieved better final grades than male students with significantly higher resource views on LMS. Furthermore, male students needed more assistance with the online software. These results also support the argument from Bruestle et al. ( 2009 ) who stated that e-learning favors female students due to its flexible and interactive learning approach. Gender difference is also evident in general internet usage patterns (Lim & Meier, 2011 ). Out of four general internet use reasons; ‘social networking’, ‘personal knowledge’, ‘formal learning’, and ‘entertainment’, the males focused more on entertainment whereas the females were engaged with social networking. This corresponds to the study results obtained by Adamus, Kerres, Getto, and Engelhardt ( 2009 ). Their study indicated that female university students focused more on communication and cooperation with openness to other’s proposals. After all, such characteristics of female students highly influenced their learning outcomes in an online training program.

On the other hand, the impact of gender difference was questioned by several studies. Astleitner and Steinberg ( 2005 ) reported that gender differences in web-based learning were insignificant. This study conducted a meta-analysis of fourteen empirical studies related to web-based learning. One of the possible explanations for such results can be that certain features of web-based learning might decrease the gender gaps in cognitive process of information. Another explanation provided in their study was that gender differences are only observed when strong accumulating effects are given during the learning process. In addition, Al-Azawei ( 2019 ) only discovered slight gender differences in LMS acceptance levels. This study involved 302 undergraduate students in Iraq and utilized the extended Technology Acceptance Model (TAM) to predict learners’ perceptions towards LMS adoption. It was found that female students were more concerned with ease of use, whereas male students were more concerned with technology usefulness. However, the differences were not significant. Thus, continuous research on gender effects should be carried out for clarification.

Age and school year factors

Although cultural factors and gender differences were suggested to play an important role in online learning and LMS implementation, previous studies have also shown that students’ age can influence the learning outcomes. According to a study conducted by McSporran and Young ( 2001 ) in a first-year introductory course for the computing systems bachelor degree, older students are more motivated to learn, better at communicating online, and at organizing their learning schedules. In the same study, female students showed better performance than male students did. Hence, both gender and age factors were related to students’ learning outcomes.

As drawn from the literature review, four major factors could affect the dynamic process of smart learning through the operation of LMS. In addition, LMS requires students’ active participation and engagement with learning because they often need to access online course materials without simultaneous prompting or instructions (Beer, Clark, & Jones, 2010 ). This implies that students who largely depend on substantial instructor direction may struggle with LMS, as it demands a certain level of self-discipline (Douglas & Alemanne, 2007 ). You ( 2016 ) also verified the importance of self-regulated learning in online courses based on LMS data measures from 530 college students. Thus, careful examinations of each influencing factor on LMS should be carried out in order to induce self-regulated learning and maximize the learning outcomes. As such, our study focuses on four important variables; ‘cultural dimension’ along with ‘gender’, ‘age’, and ‘school year’. By analyzing students’ preferred LMS functions or design, in relation to these four variables, the study results will extend current understanding of online learner preferences and provide useful guidance for smart learning environments, facilitated by LMS.

Study sample and context

Fraenkel, Wallen, and Hyun ( 2012 ) have categorized case studies as intrinsic (detailed description of one context), instrumental (focusing on a case for comprehending a more detailed phenomenon) and collective (several different or similar cases scrutinized simultaneously), which may include “…one individual, classroom, school, or program…” (p. 435). Based on that description, this study falls under the definition of instrumental case study where the researchers have had an interest in understanding more than how some students value the importance of certain tools/elements for learning management system design. The researchers have been interested in a larger goal of understanding the role of the cultural dimension for learning management system design. Hence, the researchers who conducted this case study; have been more interested in revealing conclusions that could be implemented beyond a particular case than it is. Thus, this study aims to check the following hypotheses which were derived from the conclusions of the profound literature review:

There is a statistically significant difference on each item of learning management design survey with respect to students’ cultural background on being German versus Spanish.

There is a statistically significant difference on each item of learning management design survey with respect to students’ gender.

There is a statistically significant difference on each item of learning management design survey with respect to students’ age.

There is a statistically significant difference on each item of learning management design survey with respect to students’ school year.

The participants of this case study were comprised of German ( n  = 83) and Spanish (n = 83) university students who were attending a fully online ‘Management of Information Systems (MIS)’ course provided by a South Korean university in Fall semester of 2018. Both German and Spanish students utilized the same LMS which was delivered via the university in South Korea. The course took fifteen weeks and one of the researchers of this study was the course instructor of the online course. This course introduces students the basics of modern management information systems and how they have become an integral part of the global operations of the digital companies. The course begins with discussions on the potential of information systems and technologies in improving operational efficiency of common business processes and how they could be managed effectively. Information technology infrastructure, databases and telecommunications are covered earlier than digging deeper into enterprise, supplier and customer applications. Having gained some knowledge of a variety of MIS applications, the students are equipped with practical skills for selection, acquisition and deployment of different information systems, for which the course uses several case studies and exercises.

At the time of data collection, the students were already using the same Korean university LMS together for a month and these students used different LMS (similar features) in their educational lives previously. Additionally, Learning Management Systems (LMS) was one of the management information system applications within the course topics. Hence, the researchers safely assumed that the participants of these study have had enough knowledge on making personal judgement on the survey items.

At the end of week four, the students were given the prepared survey as a voluntary activity to be filled before week four. As Table  1 shows, majority of the participants (66%) are in 18–25 age group and there is nearly an equal gender representation (53% male and 47% female) for the participants.

Although there was an equal number of students for each country ( n  = 83 for both Germany and Spain), there was diversity in school year (since the course was open for the registration to all school levels/years). Table  2 demonstrates that dominant groups were either freshman students (27%) or master students (28%).

Study instrument

The study instrument consists of four demographics related questions which were drawn from the literature review conclusions; gender (male or female), age (under 18, 18–25 or above 25), country (Germany or Spain) and school year (freshman, sophomore, junior, senior or master student). Afterwards, the researchers utilized the learning management design criteria of course textbook written by Laudon and Laudon ( 2018 ) to their survey (Table  3 ). Each category was marked on a five-point Likert scale from ‘not important at all’ to ‘very important’ where the students were grading the importance of each element for a learning management system. When the survey had been finalized, it was uploaded to a survey webpage and kept online for a week. The online survey was five webpage long where the first page was about students’ four demographics and the rest was dedicated to each dimension in Table 3 separately.

Data analysis

The researchers conducted two crosstab analyses on the final dataset; ‘age versus gender’ and ‘school year versus country’. Afterwards, the researchers applied normality test to the dataset on SPSS. Twenty items were checked against four demographic variables whether they show normal distribution on their levels. Both the Kolmogorov–Smirnov and Shapiro–Wilk tests rejected the null hypothesis of a normal population distribution for all four demographics ( p  = 0.05) (Denis, 2019 ). Therefore, the researchers decided to continue with non-parametric statistical techniques.

Additionally, the dataset was checked for its reliability. Twenty items were checked with one hundred sixty six participants and Cronbach alpha was calculated as 0.84, which shows a good score for reliability. Since Cronbach’s alpha does not assume normality (Sheng & Sheng, 2012 ), there was no issue of use for that not-normally distributing dataset.

The researchers calculated the mean scores and standard deviations for twenty items in total, for German students only and for Spanish students only. The results were tabulated and commented accordingly. After these fundamental statistics, the comparison tests were conducted. First of all, three variables (age, gender and country) were checked for each of these twenty items by using Mann Whitney U tests. The significant items were reported with their comments. Lastly, Kruskal Wallis H tests were conducted on twenty items for school year variable with its five levels; freshman, sophomore, junior, senior and master students. The significant items were reported and mentioned accordingly.

The total mean scores and standard deviations of each of the twenty design items were presented in Table  4 ( n  = 166). Additionally, the mean scores and standard deviations were calculated separately for German students ( n  = 83) and Spanish students (n = 83) and tabulated in Table  4 . Within the ‘content management’ dimension, the highest means were observed for private storage (M = 3.82) and online whiteboard (M = 3.59). Integrated offline mode (M = 3.66) and calendar integration (M = 3.59) were revealed as the most valued items of ‘ease of use’ dimension. The dimension ‘communication within the LMS’ was calculated over 3.00 for each of its items; respectively chat system (M = 3.54), notifications (M = 3.42), discussion forum (M = 3.33) and survey feature (M = 3.24). For the last dimension of screen ‘design’, the participants mostly valued marking files/courses as their favorites (M = 3.82) and choosing a personal layout (M = 3.78).

When the mean scores of each country were considered, it is easy to see that many design items were valued differently. Thirteen items were more valued by Spanish students and seven items were more valued by German students. The higher mean score for each item was highlighted grey in Table  4 . The simple mean score differences may not show the real statistically differentiating items. Therefore, the comparison tests of Mann-Whitney U tests and Kruskal Wallis H tests were run for better understanding.

First Mann-Whitney U tests were conducted for gender variable (male versus female) on twenty items of learning management system design. The only significantly differentiating item was appeared on the first item of ‘ease of use’ dimension; ‘allowing downloading multiple files’ ( U  = 2811.500, p  = 0.037). The mean rank demonstrated that male students (mean rank = 90.55, n  = 88) valued the downloading multiple files feature more than female students do (mean rank = 75.54, n  = 78).

Other Mann-Whitney U tests were implemented for age variable (18–25 versus above 25) on twenty learning management system design items. The results yielded only one single significantly differentiating item which belongs to ‘communication within the LMS’ dimension; ‘survey feature’ ( U  = 2450.000, p  = 0.023). The mean rank for ‘above 25’ group ( n  = 56) is higher than ‘18–25’ age group ( n  = 110); 94.75 and 77.77 respectively.

The last Mann-Whitney U tests were run for the country variable (Germany versus Spanish). As Table  5 demonstrates, sixteen items were significantly differentiated around country variable. Among these sixteen significantly differentiating items, only four design items’ mean ranks were higher for German students; ‘uploading assignments’, ‘accessing learning materials’, ‘learning materials are available before lectures’ and ‘simple navigation structure’. The Spanish students’ mean ranks were higher than German students for the other twelve design items; ‘comment feature’, ‘online whiteboard’, ‘private storage’, ‘easy enrollment of subject’, ‘integrated offline mod’, ‘calendar integration’, ‘chat system’, ‘discussion forum’, ‘survey feature’, ‘notifications’, ‘choose a personal design/layout’, and ‘mark files/courses as favorite’.

The last comparison tests were conducted on the school year variable for twenty design items separately. The Kruskal Wallis H tests results unfolded eleven significantly differentiating design items around the school year variable; ‘comment feature’, ‘online whiteboard’, ‘private storage’, ‘allowing downloading multiple files’, ‘learning materials are available before lectures’, ‘calendar integration’, ‘discussion forum’, ‘survey feature’, ‘notifications’, ‘choose a personal design/layout’ and ‘mark files/courses as favorite’. Although the number of students in each school level differs from each other, the mean ranks could still be used to get a deeper understanding for school years on each design item. Table  6 shows that except ‘learning materials are available before lectures’ design items where the master students had the highest mean rank, freshman students’ mean ranks were the highest for the other ten significantly differentiating design items.

Discussion and conclusion

As LMS has become a crucial element of different instructional contexts, the efforts trying to unfold its successful design factors have been studied more than ever before. The previous studies enlisted four essential success factors for LMS implementations. Therefore, this study aims to understand LMS design from a cultural point of view in additional variables of gender, age and school year. The general results clearly demonstrated that one unique LMS design will not be useful and appreciated by the students all the time. In that sense, other than setting up a commonly designed LMS on their school smart systems, the managers/instructors should prefer a more user centered approach where the LMS will be tailored according to their students’ demographics (especially the variables discussed in this study).

When German and Spanish students were compared with non-parametric statistical tests, it seemed that Spanish students generally more valued various features of LMS. In particular, Spanish students claimed that ease of use and communication within the LMS are important features for their learning. In the content management section, Spanish students also valued comment feature and online whiteboard as evident in the mean scores and the Mann-Whitney U test results. This implies that Spanish students would prefer learning through communication. Hence, the instructional designers or practitioners should offer more interactive and communicative opportunities to Spanish students on their LMS.

On the other hand, German students have put a strong emphasis on LMS features such as ‘uploading assignments’, ‘accessing learning materials’, ‘learning materials are available before lectures’ and ‘simple navigation structure’. Most of these features are directly related to the final grade and individual learning. Such different characteristics of German and Spanish students could affect their learning behaviors in a way that German students would value goal-oriented individual learning and Spanish students would value process-oriented group learning with active communication. This gives clues to the instructors while designing their instructional activities on LMS. For instance, Spanish students should be directed toward more group assignments whereas German students would appreciate more individual self-studies and exercises.

In fact, this study results are relevant to a four-dimensional model of cultural differences proposed by Hofstede ( 1986 ). Based on Hofstede’s model, the individualism-collectivism dimension provides a possible explanation for different characteristics of German and Spanish students. As described by Mercado et al. ( 2004 ), individualism values personal achievement or well-being of an individual, which suits with the characteristics observed from German students. On the other hand, collectivism highlights group achievement or group actions, which can match with the characteristics shown by Spanish students. These findings can also explain both countries’ different cultural dimension scores on individualism versus collectivism as mentioned in the literature review (Hofstede et al., 2010 ). As German students highly valued two LMS features, ‘accessing learning materials’ and ‘learning materials are available before lectures’, it implies that German students want to be prepared for their classes. Such preparation might be related to uncertainty avoidance for what they will learn in class. If so, it will contradict the results obtained by Hofstede et al. ( 2010 ) as Germany’s uncertainty avoidance score was lower than Spain. Therefore, the fundamental reason for accessing learning materials should be clarified to further explain such contradictory results. The remaining two cultural dimensions, power distance and masculinity, could not be related to our study results as students’ perspectives on hierarchical learning and competition-based learning were not assessed. Despite the cultural differences, it should be also noted that both groups of students similarly valued certain features of LMS such as reviewing grade, downloading multiple files, language selection, and access through mobile application. In both cultures, the LMS features related to users’ convenience seem equally important.

In this study, significant gender differences were not observed. The only difference observed was that male students valued downloading multiple files feature of LMS more than female students, which could mean male students favor efficiency when using LMS. As suggested by Astleitner and Steinberg ( 2005 ), LMS features might actually reduce gender differences compared to the offline class environments or there were not enough accumulating effects to induce gender differences in our study. Another possible explanation could be that gender differences are created due to the learning materials or course contents rather than LMS itself. Further elucidation on gender effect is required in future studies.

In terms of the age variable, this study results indicated that higher age group students more valued communication within the LMS, in particular, survey feature. This result is also supported by McSporran and Young ( 2001 ) as their study showed better communication skills from older students. As a learner’s age increases, it might also develop online/offline communication skills and thus learning through communication becomes a preferable option. However, it should be noted that communication features of LMS were not necessarily valued by students with the higher school year. In other words, the school year variable does not induce the same effect on the learning process or learning preference as the age variable does. Therefore, instructors should not assume similar learning behaviors between the higher age group and higher school year group when designing and implementing an LMS.

When the school year variable was examined, numerous features of LMS were highly valued by the freshman students. One of the possible reasons would be an exposure to new smart learning environments. As freshman students need to adapt in the university education system, they need to pay a particular attention to each element of an LMS. Once the adaptation period is over, the significance of LMS features might be reduced and students will gradually utilize specific LMS functions that are directly relevant to their learning process. Indeed, each feature of LMS was valued differently in each school year apart from the freshman period. It is interesting to note that the master students highly valued availability of the learning materials before lectures. This possibly indicates that postgraduate programs emphasize more on pre-class learning, which is often observed in learner-centered environments.

Our study explored various features of LMS valued by different groups of students based on their cultural background, gender, age, and school year. Out of the four hypotheses tested in our study, the first (cultural background) and the fourth (school year) hypotheses were validated whereas the second (gender) and the third (age) hypotheses were partially validated. Although not every hypothesis was fully validated, there are several important recommendations for instructors or education providers based on our study results. First of all, it is advised that future LMS design should consider the four-dimensional model of Hofstede ( 1986 ), especially the individualism-collectivism dimension to cater for various learning needs of international students. Understanding the effect of culture on LMS design, delivery and implementation will provide more user satisfaction leading toward more success stories in education. Secondly, learning materials on LMS should be checked for possible inducing factors of gender differences. In that sense, the instructors should be informed about gender bias issues. Thirdly, more communication features of LMS will be effective in the courses with higher age groups. Lastly, LMS can provide more guidelines or assistance for freshman students and create a wide range of learner-centered environments for postgraduate programs.

Since this study was delimited to two specific cultures, prospective studies must focus on adding more variety to similar culture based design studies. In order to gain a further understanding of LMS and smart learning process, future studies should investigate more various cultural groups and their learning characteristics. Moreover, due to the sample limitation of this study, the researchers highly recommend to conduct prospective studies with larger sample size to analyze group with parametric techniques. If possible and available, students’ LMS logs (including the most commonly used tools) should be analyzed for a better understanding of LMS tools and their usage by different cultures. Similarly, different variables examined in this study could be compared with assessment or examination grades to identify which LMS features can maximize the learning outcomes for a particular group of students. Additionally, qualitative interview schedules should be integrated into culture based studies to understand its effects in depth.

Since this study implemented the convenience sampling which might have the disadvantage of bias, the similar studies should be replicated in different courses or universities to check if the observed results are due to onetime occurrence. Moreover, LMS has also been utilized in business world where different companies' training activities are supported by these smart systems. In that sense, the research in business world could assist us to understand the deeper influence of culture.

The instructional stakeholders must always remember that future studies on culturally sensitive LMS design will contribute to the achievement of better learning in the waves of upcoming digital revolution era.

Availability of data and materials

The corresponding author declared here all types of data used in this study available for any clarification. The author of this manuscript ready for any justification regarding the data set. To make publically available of the data used in this study, the seeker must send an email to the mentioned email address. The profile of the respondents was completely confidential.

Abbreviations

• Learning management systems

Master of business administration

Management of information systems

Adamus, T., Kerres, M., Getto, B., & Engelhardt, N. (2009). Gender and e-tutoring - A concept for gender sensitive e-tutor training programs. In Fifth European symposium on gender & ICT digital cultures: Participation - empowerment - diversity . University of Bremen March 5-7, 2009 - Retrieved from http://www.informatik.uni-bremen.de/soteg/gict2009/proceedings/GICT2009_Adamus.pdf .

Al-Azawei, A. (2019). The moderating effect of gender differences on learning management system acceptance: A multi-group analysis. Italian Journal of Educational Technology , 27 (3), 257–278. https://doi.org/10.17471/2499-4324/1088 .

Article   Google Scholar  

Astleitner, H., & Steinberg, R. (2005). Are there gender differences in web-based learning? An integrated model and related effect sizes. Association for the Advancement of Computing in Education , 13 (1), 47–63.

Google Scholar  

Asunka, S. (2016). Helping faculty overcome cultural barriers to adoption and use of web-based learning technologies: A participatory action research approach. In Revolutionizing education through web-based instruction , (pp. 300–316).

Chapter   Google Scholar  

Beer, C., Clark, K., & Jones, D. (2010). Indicators of engagement . In Proceedings Ascilite Sydney 2010 , (pp. 75–86) Retrieved from http://ascilite.org.au/conferences/sydney10/procs/Beer-full.pdf .

Bove, L., & A., & Conklin, S. (2020). Learning strategies for faculty during a learning management system migration. Online Journal of Distance Learning Administration , 23 (1), 1–10.

Bruestle, P., Haubner, D., Schinzel, B., Holthaus, M., Remmele, B., Schirmer, D., & Reips, U. D. (2009). Doing e-learning/doing gender? Examining the relationship between students' gender concepts and e-learning technology. In Fifth European symposium on gender & ICT digital cultures: Participation - empowerment - diversity . University of Bremen March 5–7, 2009. Retrieved from http://www.informatik.uni-bremen.de/soteg/gict2009/proceedings/GICT2009_Adamus.pdf .

Cuadrado-García, M., Ruiz-Molina, M. E., & Montoro-Pons, J. D. (2010). Are there gender differences in e-learning use and assessment? Evidence from an interuniversity online project in Europe. Procedia - Social and Behavioral Sciences , 2 (2), 367–371. https://doi.org/10.1016/j.sbspro.2010.03.027 .

Denis, D. J. (2019). SPSS data analysis for univariate, bivariate, and multivariate statistics . Hoboken: Wiley.

Douglas, I., & Alemanne, N. D. (2007). Measuring student participation and effort . Algarve: Paper presented at the international conference on cognition and exploratory learning in digital age Retrieved from https://www.researchgate.net/publication/241134504_Measuring_student_participation_and_effort .

Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate research in education , (8th ed., ). New York: McGraw-Hill Humanities/Social Sciences/Languages.

Hofstede, G. (1986). Cultural differences in teaching and learning. International Journal of Intercultural Relations , 10 (3), 301–320. https://doi.org/10.1016/0147-1767(86)90015-5 .

Hofstede, G., Hofstede, G. J., & Minkov, M. (2010). Cultures and organizations: Software of the mind. Revised and expanded , (3rd ed., ). New York: McGraw-Hill.

Hunt, A., & Tickner, S. (2015). Cultural dimensions of learning in online teacher education courses. Journal of Open, Flexible, and Distance Learning , 19 (2), 25–47.

Laudon, K. C., & Laudon, J. P. (2018). Management information systems: Managing the digital firm , (15th ed., ). New York: Pearson.

Lim, K., & Meier, E. B. (2011). Different but similar: Computer use patterns between young Korean males and females. Educational Technology Research and Development , 59 (4), 575–592. https://doi.org/10.1007/s11423-011-9206-5 .

Liu, X., Liu, S., Lee, S. H., & Magjuka, R. J. (2010). Cultural differences in online learning: International student perceptions. Journal of Educational Technology & Society , 13 (3), 177–188.

McSporran, M., & Young, S. (2001). Does gender matter in online learning? Unitec New Zealand working paper. Retrieved from https://repository.alt.ac.uk/348/1/ALT_J_Vol9_No2_2001_Does%20gender%20matter%20in%20online%20l.pdf .

Mercado, S., Parboteeah, K. P., & Zhao, Y. (2004). On-line course design and delivery: Cross-national considerations. Strategic Change , 13 (4), 183–192. https://doi.org/10.1002/jsc.677 .

Parrish, P., & Linder-VanBerschot, J. A. (2010). Cultural dimensions of learning: Addressing the challenges of multicultural instruction. International Review of Research in Open and Distance Learning , 11 (2), 1–19. https://doi.org/10.19173/irrodl.v11i2.809 .

Sheng, Y., & Sheng, Z. (2012). Is coefficient alpha robust to non-normal data? Frontiers in Psychology , 3 (34). https://doi.org/10.3389/fpsyg.2012.00034 .

Swierczek, F. W., & Bechter, C. (2010). Cultural features of e-learning: A euro Asian case study. In J. M. Spector et al. (Eds.), Learning and instruction in the digital age , (pp. 291–308). New York: Springer.

Tarhini, A., Hone, K., Liu, X., & Tarhini, T. (2017). Examining the moderating effect of individual-level cultural values on users’ acceptance of e-learning in developing countries: A structural equation modeling of an extended technology acceptance model. Interactive Learning Environments , 25 (3), 306–328. https://doi.org/10.1080/10494820.2015.1122635 .

Yakubu, M. N. (2019). The effect of quality antecedents on the acceptance of learning management systems: A case of two private universities in Nigeria. International Journal of Education and Development using Information and Communication Technology , 15 (4), 101–115.

You, J. W. (2016). Identifying significant indicators using LMS data to predict course achievement in online learning. The Internet and Higher Education , 29 , 23–30.

Download references

Acknowledgements

Not applicable.

Authors declare that no funding for the research was provided from any side.

Author information

Authors and affiliations.

Technology Studies, Endicott College of International Studies, Woosong University, Daejeon, South Korea

Hasan Tinmaz

Global Healthcare Management, Sol International School, Woosong University, 171 Dongdaejeon-ro, Dong-gu, Daejeon, 34606, South Korea

Jin Hwa Lee

You can also search for this author in PubMed   Google Scholar

Contributions

Hasan Tinmaz was responsible from method and results sections. Jin Hwa Lee was in charge of literature review and discussion sections. The authors worked together on the rest of the manuscript. The authors read and approved the final manuscript.

Corresponding author

Correspondence to Jin Hwa Lee .

Ethics declarations

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Tinmaz, H., Lee, J.H. An analysis of users’ preferences on learning management systems: a case on German versus Spanish students. Smart Learn. Environ. 7 , 30 (2020). https://doi.org/10.1186/s40561-020-00141-8

Download citation

Received : 13 April 2020

Accepted : 30 September 2020

Published : 14 October 2020

DOI : https://doi.org/10.1186/s40561-020-00141-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Smart learning

AI-Based Personalized E-Learning Systems: Issues, Challenges, and Solutions

Ieee account.

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

learning management system Recently Published Documents

Total documents.

• Latest Documents
• Most Cited Documents
• Contributed Authors
• Related Sources
• Related Keywords

Academia : Web Platform for E-learning.

Abstract: This paper illustrates how we can improve the existing manual system with the help of E-learning management system. The method aims to build an E-learning web application having better and safer user experience and provides an interactive teaching-learning platform for students and teachers. E-learning Management System is way of solving the educational problems using the modern technologies. It gives an error free, secure, reliable and fast management system. It can assist the user to concentrate on learning rather to concentrate on the record keeping and other stuff. It will help organization in better utilization of resources. Keywords: Web application, Database, backend, frontend, platform, E-learning, Frameworks

Perceived convenience, usefulness, effectiveness and user acceptance of information technology: evaluating students’ experiences of a Learning Management System

Disciplinary differences in the uptake of a learning management system: a case of a south african historically disadvantaged institution.

The provision of a Learning Management System (LMS) for use in distributed, blended or open distance e-learning as a management tool has become a basic standard requirement in higher learning institutions globally. Many students and lecturers use an LMS in support of innovative and engaged teaching and learning, both inside and outside the classroom—whether blended or open leaning. However, many academics choose not to make use of the institutional LMS. This is the specific issue that this study addresses, with a particular focus on the role played by disciplinary differences in the uptake of an LMS. The research question guiding the study is thus: To what extent do disciplinary differences affect the uptake of an LMS? The research study drew on Legitimation Code Theory, a sociological theory that explains the knowledge principles underpinning practices, in this case, the practice of the uptake (or non-uptake) of an institutional LMS. The study made use of quantitative data collection and data analysis methods, drawing on the institutional LMS activity data. The study found that there was a significant relationship between the disciplines and LMS uptake. However, the study also found a number of unexpected exceptions, where the nature of the discipline did not seem to impact uptake or non-uptake. The contribution that the study makes is to show the significant role that the academics’ home discipline plays in LMS uptake.

PENERAPAN GOOGLE CLASSROOM BERBANTUKAN GOOGLE MEET DALAM PEMBELAJARAN EKONOMI DAN PENGARUHNYA TERHADAP HASIL BELAJAR

Pemberlakuan Surat Edaran Kemendikbud Nomor 15 Tahun 2020 tentang Pedoman Penyelenggaraan Belajar Dari Rumah Dalam Masa Darurat Penyebaran Corona Virus Disease (Covid-19) berimplikasi pada pilihan Pembelajaran Jarak Jauh (PJJ) dalam jaringan (daring) atau pembelajaran jarak jauh luar jaringan (luring). PJJ moda daring memunculkan permasalahan kesiapan satuan pendidikan dalam membuat Learning Management System (LMS) dengan kemampuan finansial terbatas. Umumnya pimpinan satuan pendidikan menetapkan kebijakan menggunakan aplikasi pendidikan umum. Salah satu aplikasi pendidikan umum yang banyak digunakan pada satuan pendidikan menengah adalah Google Classroom berbantukan Google Meet. Penerapan aplikasi tersebut menuntut penjelasan ilmiah, yakni seberapa besar pengaruh penerapan Google Classroom berbantukan Google Meet pada pembelajaran ekonomi terhadap hasil belajar peserta didik. Penelitian menggunakan pendekatan kuantitatif dengan metode penelitian survai. Subjek penelitian terdiri dari peserta didik pembelajaran ekonomi yang dilakukan tim peneliti. Hasil uji instrumen menunjukkan instrumen memenuhi kriteria validitas, reliabilitas, dan normalitas. Hasil uji hipotesis menunjukkan penerapan Google Classroom berbantukan Google Meet pada pembelajaran kompetensi dasar konsep ekonomi memberikan pengaruh sebesar 82,1% terhadap hasil belajar. Pengaruh tersebut berasal dari dimensi tujuan pembelajaran daring dan dimensi kemudahan pembelajaran daring.

Usage of Learning Management System during the COVID-19 Pandemic

Implementation of microservices architecture in learning management system e-course using web service method.

Building a learning system using the Learning Management System is already too much but there are many problems that make people think about using it because it is difficult to maintain the system, so to build a system takes a lot of time because the basis for developing the system is monolith-based and difficult to develop if the system development team grows or change. Learning Management System needs to be implemented for easy access to learning because online learning is very important to be a good alternative. From these problems led to the idea to create a Learning Management System in which it can solve services and communicate between services to facilitate efficient development. This study aims to apply the Web Service Implementation Methodology used in building a Learning Management System for an E-Course that facilitates online learning by implementing web services. To perform inter-service solving the methodology used in making this application is the web service method which is implemented on the microservices architecture. Programming languages for differentiating services are PHP and JavaScript. For the user interface, ReactJS is used as the API data manager for each service and the database uses MySQL. This application will later be used as a medium to facilitate online learning, there are members (users) as student actors and admins to manage mentor and course data that will be displayed.

Integrating Library Resources in a Learning Management System: Exploring Instructor Obstacles and Motivations

Developing engagement in the learning management system supported by learning analytics, using moodle quiz to create language consolidating and testing activities.

The recent pandemic outbreak (2019-2021) shed light on the practical difficulties of technology use in education and forced many instructors to seek training or train themselves in using technologies, including learning management system (LMS). During this time, we noticed that, although many tutorials are available, for example, to help instructors in the use of the Moodle LMS, these are mostly generic and do not focus on languages. Furthermore, there is a scarcity of papers demonstrating how to pair pedagogy with technology. In fact, only 1.1% of the papers published between 2006-2016 in ReCALL, CALICO, and CALL Journals deal with virtual learning environments (VLE). This chapter shows the case of two instructors who originally worked individually, and then collaborated in using Moodle Quiz for languages. Their quest in creating activities with constructivist characteristics is described; the knowledge, skills, and experiences gained are shared; the end products are displayed; and some conclusions are drawn.

Higher Education in India in the Time of Pandemic, Sans a Learning Management System

Higher education in India was caught completely unawares by the COVID-19 pandemic and the necessitated closure of educational institutions. Despite almost a decade of experience with online and distance learning at some top-tier and private institutions, the vast majority were unprepared and looked for quick solutions for different components of teaching–learning depending on the need of the hour. The immediate tool sought was a videoconferencing platform to substitute in-class lectures. With no access to a learning management system, faculty chose one platform for videoconferencing, one for interaction with students, and another for uploading class notes. Disparity in students’ access to devices and the internet presented challenges. Assessment of learning, which hitherto was largely pen and paper based, was delayed for lack of a viable solution. Experiences documented in this study demonstrate faculty resilience, but lack of institutional leadership and preparedness is starkly evident.

Export Citation Format

Share document.