data analysis in research lesson plan

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Can J Hosp Pharm
• v.68(4); Jul-Aug 2015

Creating a Data Analysis Plan: What to Consider When Choosing Statistics for a Study

There are three kinds of lies: lies, damned lies, and statistics. – Mark Twain 1

INTRODUCTION

Statistics represent an essential part of a study because, regardless of the study design, investigators need to summarize the collected information for interpretation and presentation to others. It is therefore important for us to heed Mr Twain’s concern when creating the data analysis plan. In fact, even before data collection begins, we need to have a clear analysis plan that will guide us from the initial stages of summarizing and describing the data through to testing our hypotheses.

The purpose of this article is to help you create a data analysis plan for a quantitative study. For those interested in conducting qualitative research, previous articles in this Research Primer series have provided information on the design and analysis of such studies. 2 , 3 Information in the current article is divided into 3 main sections: an overview of terms and concepts used in data analysis, a review of common methods used to summarize study data, and a process to help identify relevant statistical tests. My intention here is to introduce the main elements of data analysis and provide a place for you to start when planning this part of your study. Biostatistical experts, textbooks, statistical software packages, and other resources can certainly add more breadth and depth to this topic when you need additional information and advice.

TERMS AND CONCEPTS USED IN DATA ANALYSIS

When analyzing information from a quantitative study, we are often dealing with numbers; therefore, it is important to begin with an understanding of the source of the numbers. Let us start with the term variable , which defines a specific item of information collected in a study. Examples of variables include age, sex or gender, ethnicity, exercise frequency, weight, treatment group, and blood glucose. Each variable will have a group of categories, which are referred to as values , to help describe the characteristic of an individual study participant. For example, the variable “sex” would have values of “male” and “female”.

Although variables can be defined or grouped in various ways, I will focus on 2 methods at this introductory stage. First, variables can be defined according to the level of measurement. The categories in a nominal variable are names, for example, male and female for the variable “sex”; white, Aboriginal, black, Latin American, South Asian, and East Asian for the variable “ethnicity”; and intervention and control for the variable “treatment group”. Nominal variables with only 2 categories are also referred to as dichotomous variables because the study group can be divided into 2 subgroups based on information in the variable. For example, a study sample can be split into 2 groups (patients receiving the intervention and controls) using the dichotomous variable “treatment group”. An ordinal variable implies that the categories can be placed in a meaningful order, as would be the case for exercise frequency (never, sometimes, often, or always). Nominal-level and ordinal-level variables are also referred to as categorical variables, because each category in the variable can be completely separated from the others. The categories for an interval variable can be placed in a meaningful order, with the interval between consecutive categories also having meaning. Age, weight, and blood glucose can be considered as interval variables, but also as ratio variables, because the ratio between values has meaning (e.g., a 15-year-old is half the age of a 30-year-old). Interval-level and ratio-level variables are also referred to as continuous variables because of the underlying continuity among categories.

As we progress through the levels of measurement from nominal to ratio variables, we gather more information about the study participant. The amount of information that a variable provides will become important in the analysis stage, because we lose information when variables are reduced or aggregated—a common practice that is not recommended. 4 For example, if age is reduced from a ratio-level variable (measured in years) to an ordinal variable (categories of < 65 and ≥ 65 years) we lose the ability to make comparisons across the entire age range and introduce error into the data analysis. 4

A second method of defining variables is to consider them as either dependent or independent. As the terms imply, the value of a dependent variable depends on the value of other variables, whereas the value of an independent variable does not rely on other variables. In addition, an investigator can influence the value of an independent variable, such as treatment-group assignment. Independent variables are also referred to as predictors because we can use information from these variables to predict the value of a dependent variable. Building on the group of variables listed in the first paragraph of this section, blood glucose could be considered a dependent variable, because its value may depend on values of the independent variables age, sex, ethnicity, exercise frequency, weight, and treatment group.

Statistics are mathematical formulae that are used to organize and interpret the information that is collected through variables. There are 2 general categories of statistics, descriptive and inferential. Descriptive statistics are used to describe the collected information, such as the range of values, their average, and the most common category. Knowledge gained from descriptive statistics helps investigators learn more about the study sample. Inferential statistics are used to make comparisons and draw conclusions from the study data. Knowledge gained from inferential statistics allows investigators to make inferences and generalize beyond their study sample to other groups.

Before we move on to specific descriptive and inferential statistics, there are 2 more definitions to review. Parametric statistics are generally used when values in an interval-level or ratio-level variable are normally distributed (i.e., the entire group of values has a bell-shaped curve when plotted by frequency). These statistics are used because we can define parameters of the data, such as the centre and width of the normally distributed curve. In contrast, interval-level and ratio-level variables with values that are not normally distributed, as well as nominal-level and ordinal-level variables, are generally analyzed using nonparametric statistics.

METHODS FOR SUMMARIZING STUDY DATA: DESCRIPTIVE STATISTICS

The first step in a data analysis plan is to describe the data collected in the study. This can be done using figures to give a visual presentation of the data and statistics to generate numeric descriptions of the data.

Selection of an appropriate figure to represent a particular set of data depends on the measurement level of the variable. Data for nominal-level and ordinal-level variables may be interpreted using a pie graph or bar graph . Both options allow us to examine the relative number of participants within each category (by reporting the percentages within each category), whereas a bar graph can also be used to examine absolute numbers. For example, we could create a pie graph to illustrate the proportions of men and women in a study sample and a bar graph to illustrate the number of people who report exercising at each level of frequency (never, sometimes, often, or always).

Interval-level and ratio-level variables may also be interpreted using a pie graph or bar graph; however, these types of variables often have too many categories for such graphs to provide meaningful information. Instead, these variables may be better interpreted using a histogram . Unlike a bar graph, which displays the frequency for each distinct category, a histogram displays the frequency within a range of continuous categories. Information from this type of figure allows us to determine whether the data are normally distributed. In addition to pie graphs, bar graphs, and histograms, many other types of figures are available for the visual representation of data. Interested readers can find additional types of figures in the books recommended in the “Further Readings” section.

Figures are also useful for visualizing comparisons between variables or between subgroups within a variable (for example, the distribution of blood glucose according to sex). Box plots are useful for summarizing information for a variable that does not follow a normal distribution. The lower and upper limits of the box identify the interquartile range (or 25th and 75th percentiles), while the midline indicates the median value (or 50th percentile). Scatter plots provide information on how the categories for one continuous variable relate to categories in a second variable; they are often helpful in the analysis of correlations.

In addition to using figures to present a visual description of the data, investigators can use statistics to provide a numeric description. Regardless of the measurement level, we can find the mode by identifying the most frequent category within a variable. When summarizing nominal-level and ordinal-level variables, the simplest method is to report the proportion of participants within each category.

The choice of the most appropriate descriptive statistic for interval-level and ratio-level variables will depend on how the values are distributed. If the values are normally distributed, we can summarize the information using the parametric statistics of mean and standard deviation. The mean is the arithmetic average of all values within the variable, and the standard deviation tells us how widely the values are dispersed around the mean. When values of interval-level and ratio-level variables are not normally distributed, or we are summarizing information from an ordinal-level variable, it may be more appropriate to use the nonparametric statistics of median and range. The first step in identifying these descriptive statistics is to arrange study participants according to the variable categories from lowest value to highest value. The range is used to report the lowest and highest values. The median or 50th percentile is located by dividing the number of participants into 2 groups, such that half (50%) of the participants have values above the median and the other half (50%) have values below the median. Similarly, the 25th percentile is the value with 25% of the participants having values below and 75% of the participants having values above, and the 75th percentile is the value with 75% of participants having values below and 25% of participants having values above. Together, the 25th and 75th percentiles define the interquartile range .

PROCESS TO IDENTIFY RELEVANT STATISTICAL TESTS: INFERENTIAL STATISTICS

One caveat about the information provided in this section: selecting the most appropriate inferential statistic for a specific study should be a combination of following these suggestions, seeking advice from experts, and discussing with your co-investigators. My intention here is to give you a place to start a conversation with your colleagues about the options available as you develop your data analysis plan.

There are 3 key questions to consider when selecting an appropriate inferential statistic for a study: What is the research question? What is the study design? and What is the level of measurement? It is important for investigators to carefully consider these questions when developing the study protocol and creating the analysis plan. The figures that accompany these questions show decision trees that will help you to narrow down the list of inferential statistics that would be relevant to a particular study. Appendix 1 provides brief definitions of the inferential statistics named in these figures. Additional information, such as the formulae for various inferential statistics, can be obtained from textbooks, statistical software packages, and biostatisticians.

What Is the Research Question?

The first step in identifying relevant inferential statistics for a study is to consider the type of research question being asked. You can find more details about the different types of research questions in a previous article in this Research Primer series that covered questions and hypotheses. 5 A relational question seeks information about the relationship among variables; in this situation, investigators will be interested in determining whether there is an association ( Figure 1 ). A causal question seeks information about the effect of an intervention on an outcome; in this situation, the investigator will be interested in determining whether there is a difference ( Figure 2 ).

Decision tree to identify inferential statistics for an association.

Decision tree to identify inferential statistics for measuring a difference.

What Is the Study Design?

When considering a question of association, investigators will be interested in measuring the relationship between variables ( Figure 1 ). A study designed to determine whether there is consensus among different raters will be measuring agreement. For example, an investigator may be interested in determining whether 2 raters, using the same assessment tool, arrive at the same score. Correlation analyses examine the strength of a relationship or connection between 2 variables, like age and blood glucose. Regression analyses also examine the strength of a relationship or connection; however, in this type of analysis, one variable is considered an outcome (or dependent variable) and the other variable is considered a predictor (or independent variable). Regression analyses often consider the influence of multiple predictors on an outcome at the same time. For example, an investigator may be interested in examining the association between a treatment and blood glucose, while also considering other factors, like age, sex, ethnicity, exercise frequency, and weight.

When considering a question of difference, investigators must first determine how many groups they will be comparing. In some cases, investigators may be interested in comparing the characteristic of one group with that of an external reference group. For example, is the mean age of study participants similar to the mean age of all people in the target group? If more than one group is involved, then investigators must also determine whether there is an underlying connection between the sets of values (or samples ) to be compared. Samples are considered independent or unpaired when the information is taken from different groups. For example, we could use an unpaired t test to compare the mean age between 2 independent samples, such as the intervention and control groups in a study. Samples are considered related or paired if the information is taken from the same group of people, for example, measurement of blood glucose at the beginning and end of a study. Because blood glucose is measured in the same people at both time points, we could use a paired t test to determine whether there has been a significant change in blood glucose.

What Is the Level of Measurement?

As described in the first section of this article, variables can be grouped according to the level of measurement (nominal, ordinal, or interval). In most cases, the independent variable in an inferential statistic will be nominal; therefore, investigators need to know the level of measurement for the dependent variable before they can select the relevant inferential statistic. Two exceptions to this consideration are correlation analyses and regression analyses ( Figure 1 ). Because a correlation analysis measures the strength of association between 2 variables, we need to consider the level of measurement for both variables. Regression analyses can consider multiple independent variables, often with a variety of measurement levels. However, for these analyses, investigators still need to consider the level of measurement for the dependent variable.

Selection of inferential statistics to test interval-level variables must include consideration of how the data are distributed. An underlying assumption for parametric tests is that the data approximate a normal distribution. When the data are not normally distributed, information derived from a parametric test may be wrong. 6 When the assumption of normality is violated (for example, when the data are skewed), then investigators should use a nonparametric test. If the data are normally distributed, then investigators can use a parametric test.

ADDITIONAL CONSIDERATIONS

What is the level of significance.

An inferential statistic is used to calculate a p value, the probability of obtaining the observed data by chance. Investigators can then compare this p value against a prespecified level of significance, which is often chosen to be 0.05. This level of significance represents a 1 in 20 chance that the observation is wrong, which is considered an acceptable level of error.

What Are the Most Commonly Used Statistics?

In 1983, Emerson and Colditz 7 reported the first review of statistics used in original research articles published in the New England Journal of Medicine . This review of statistics used in the journal was updated in 1989 and 2005, 8 and this type of analysis has been replicated in many other journals. 9 – 13 Collectively, these reviews have identified 2 important observations. First, the overall sophistication of statistical methodology used and reported in studies has grown over time, with survival analyses and multivariable regression analyses becoming much more common. The second observation is that, despite this trend, 1 in 4 articles describe no statistical methods or report only simple descriptive statistics. When inferential statistics are used, the most common are t tests, contingency table tests (for example, χ 2 test and Fisher exact test), and simple correlation and regression analyses. This information is important for educators, investigators, reviewers, and readers because it suggests that a good foundational knowledge of descriptive statistics and common inferential statistics will enable us to correctly evaluate the majority of research articles. 11 – 13 However, to fully take advantage of all research published in high-impact journals, we need to become acquainted with some of the more complex methods, such as multivariable regression analyses. 8 , 13

What Are Some Additional Resources?

As an investigator and Associate Editor with CJHP , I have often relied on the advice of colleagues to help create my own analysis plans and review the plans of others. Biostatisticians have a wealth of knowledge in the field of statistical analysis and can provide advice on the correct selection, application, and interpretation of these methods. Colleagues who have “been there and done that” with their own data analysis plans are also valuable sources of information. Identify these individuals and consult with them early and often as you develop your analysis plan.

Another important resource to consider when creating your analysis plan is textbooks. Numerous statistical textbooks are available, differing in levels of complexity and scope. The titles listed in the “Further Reading” section are just a few suggestions. I encourage interested readers to look through these and other books to find resources that best fit their needs. However, one crucial book that I highly recommend to anyone wanting to be an investigator or peer reviewer is Lang and Secic’s How to Report Statistics in Medicine (see “Further Reading”). As the title implies, this book covers a wide range of statistics used in medical research and provides numerous examples of how to correctly report the results.

CONCLUSIONS

When it comes to creating an analysis plan for your project, I recommend following the sage advice of Douglas Adams in The Hitchhiker’s Guide to the Galaxy : Don’t panic! 14 Begin with simple methods to summarize and visualize your data, then use the key questions and decision trees provided in this article to identify relevant statistical tests. Information in this article will give you and your co-investigators a place to start discussing the elements necessary for developing an analysis plan. But do not stop there! Use advice from biostatisticians and more experienced colleagues, as well as information in textbooks, to help create your analysis plan and choose the most appropriate statistics for your study. Making careful, informed decisions about the statistics to use in your study should reduce the risk of confirming Mr Twain’s concern.

Appendix 1. Glossary of statistical terms * (part 1 of 2)

• 1-way ANOVA: Uses 1 variable to define the groups for comparing means. This is similar to the Student t test when comparing the means of 2 groups.
• Kruskall–Wallis 1-way ANOVA: Nonparametric alternative for the 1-way ANOVA. Used to determine the difference in medians between 3 or more groups.
• n -way ANOVA: Uses 2 or more variables to define groups when comparing means. Also called a “between-subjects factorial ANOVA”.
• Repeated-measures ANOVA: A method for analyzing whether the means of 3 or more measures from the same group of participants are different.
• Freidman ANOVA: Nonparametric alternative for the repeated-measures ANOVA. It is often used to compare rankings and preferences that are measured 3 or more times.
• Fisher exact: Variation of chi-square that accounts for cell counts < 5.
• McNemar: Variation of chi-square that tests statistical significance of changes in 2 paired measurements of dichotomous variables.
• Cochran Q: An extension of the McNemar test that provides a method for testing for differences between 3 or more matched sets of frequencies or proportions. Often used as a measure of heterogeneity in meta-analyses.
• 1-sample: Used to determine whether the mean of a sample is significantly different from a known or hypothesized value.
• Independent-samples t test (also referred to as the Student t test): Used when the independent variable is a nominal-level variable that identifies 2 groups and the dependent variable is an interval-level variable.
• Paired: Used to compare 2 pairs of scores between 2 groups (e.g., baseline and follow-up blood pressure in the intervention and control groups).

Lang TA, Secic M. How to report statistics in medicine: annotated guidelines for authors, editors, and reviewers. 2nd ed. Philadelphia (PA): American College of Physicians; 2006.

Norman GR, Streiner DL. PDQ statistics. 3rd ed. Hamilton (ON): B.C. Decker; 2003.

Plichta SB, Kelvin E. Munro’s statistical methods for health care research . 6th ed. Philadelphia (PA): Wolters Kluwer Health/ Lippincott, Williams & Wilkins; 2013.

This article is the 12th in the CJHP Research Primer Series, an initiative of the CJHP Editorial Board and the CSHP Research Committee. The planned 2-year series is intended to appeal to relatively inexperienced researchers, with the goal of building research capacity among practising pharmacists. The articles, presenting simple but rigorous guidance to encourage and support novice researchers, are being solicited from authors with appropriate expertise.

Previous articles in this series:

• Bond CM. The research jigsaw: how to get started. Can J Hosp Pharm . 2014;67(1):28–30.
• Tully MP. Research: articulating questions, generating hypotheses, and choosing study designs. Can J Hosp Pharm . 2014;67(1):31–4.
• Loewen P. Ethical issues in pharmacy practice research: an introductory guide. Can J Hosp Pharm. 2014;67(2):133–7.
• Tsuyuki RT. Designing pharmacy practice research trials. Can J Hosp Pharm . 2014;67(3):226–9.
• Bresee LC. An introduction to developing surveys for pharmacy practice research. Can J Hosp Pharm . 2014;67(4):286–91.
• Gamble JM. An introduction to the fundamentals of cohort and case–control studies. Can J Hosp Pharm . 2014;67(5):366–72.
• Austin Z, Sutton J. Qualitative research: getting started. C an J Hosp Pharm . 2014;67(6):436–40.
• Houle S. An introduction to the fundamentals of randomized controlled trials in pharmacy research. Can J Hosp Pharm . 2014; 68(1):28–32.
• Charrois TL. Systematic reviews: What do you need to know to get started? Can J Hosp Pharm . 2014;68(2):144–8.
• Sutton J, Austin Z. Qualitative research: data collection, analysis, and management. Can J Hosp Pharm . 2014;68(3):226–31.
• Cadarette SM, Wong L. An introduction to health care administrative data. Can J Hosp Pharm. 2014;68(3):232–7.

Competing interests: None declared.

Further Reading

• Devor J, Peck R. Statistics: the exploration and analysis of data. 7th ed. Boston (MA): Brooks/Cole Cengage Learning; 2012. [ Google Scholar ]
• Lang TA, Secic M. How to report statistics in medicine: annotated guidelines for authors, editors, and reviewers. 2nd ed. Philadelphia (PA): American College of Physicians; 2006. [ Google Scholar ]
• Mendenhall W, Beaver RJ, Beaver BM. Introduction to probability and statistics. 13th ed. Belmont (CA): Brooks/Cole Cengage Learning; 2009. [ Google Scholar ]
• Norman GR, Streiner DL. PDQ statistics. 3rd ed. Hamilton (ON): B.C. Decker; 2003. [ Google Scholar ]
• Plichta SB, Kelvin E. Munro’s statistical methods for health care research. 6th ed. Philadelphia (PA): Wolters Kluwer Health/Lippincott, Williams & Wilkins; 2013. [ Google Scholar ]

How to Create a Data Analysis Plan: A Detailed Guide

by Barche Blaise | Aug 12, 2020 | Writing

If a good research question equates to a story then, a roadmap will be very vita l for good storytelling. We advise every student/researcher to personally write his/her data analysis plan before seeking any advice. In this blog article, we will explore how to create a data analysis plan: the content and structure.

This data analysis plan serves as a roadmap to how data collected will be organised and analysed. It includes the following aspects:

• Clearly states the research objectives and hypothesis
• Identifies the dataset to be used
• Inclusion and exclusion criteria
• Clearly states the research variables
• States statistical test hypotheses and the software for statistical analysis
• Creating shell tables

1. Stating research question(s), objectives and hypotheses:

All research objectives or goals must be clearly stated. They must be Specific, Measurable, Attainable, Realistic and Time-bound (SMART). Hypotheses are theories obtained from personal experience or previous literature and they lay a foundation for the statistical methods that will be applied to extrapolate results to the entire population.

2. The dataset:

The dataset that will be used for statistical analysis must be described and important aspects of the dataset outlined. These include; owner of the dataset, how to get access to the dataset, how the dataset was checked for quality control and in what program is the dataset stored (Excel, Epi Info, SQL, Microsoft access etc.).

3. The inclusion and exclusion criteria :

They guide the aspects of the dataset that will be used for data analysis. These criteria will also guide the choice of variables included in the main analysis.

4. Variables:

Every variable collected in the study should be clearly stated. They should be presented based on the level of measurement (ordinal/nominal or ratio/interval levels), or the role the variable plays in the study (independent/predictors or dependent/outcome variables). The variable types should also be outlined.  The variable type in conjunction with the research hypothesis forms the basis for selecting the appropriate statistical tests for inferential statistics. A good data analysis plan should summarize the variables as demonstrated in Figure 1 below.

5. Statistical software

There are tons of software packages for data analysis, some common examples are SPSS, Epi Info, SAS, STATA, Microsoft Excel. Include the version number,  year of release and author/manufacturer. Beginners have the tendency to try different software and finally not master any. It is rather good to select one and master it because almost all statistical software have the same performance for basic and the majority of advance analysis needed for a student thesis. This is what we recommend to all our students at CRENC before they begin writing their results section .

6. Selecting the appropriate statistical method to test hypotheses

Depending on the research question, hypothesis and type of variable, several statistical methods can be used to answer the research question appropriately. This aspect of the data analysis plan outlines clearly why each statistical method will be used to test hypotheses. The level of statistical significance (p-value) which is often but not always <0.05 should also be written.  Presented in figures 2a and 2b are decision trees for some common statistical tests based on the variable type and research question

A good analysis plan should clearly describe how missing data will be analysed.

7. Creating shell tables

Data analysis involves three levels of analysis; univariable, bivariable and multivariable analysis with increasing order of complexity. Shell tables should be created in anticipation for the results that will be obtained from these different levels of analysis. Read our blog article on how to present tables and figures for more details. Suppose you carry out a study to investigate the prevalence and associated factors of a certain disease “X” in a population, then the shell tables can be represented as in Tables 1, Table 2 and Table 3 below.

Table 1: Example of a shell table from univariate analysis

Table 2: Example of a shell table from bivariate analysis

Table 3: Example of a shell table from multivariate analysis

aOR = adjusted odds ratio

Now that you have learned how to create a data analysis plan, these are the takeaway points. It should clearly state the:

• Research question, objectives, and hypotheses
• Dataset to be used
• Variable types and their role
• Statistical software and statistical methods
• Shell tables for univariate, bivariate and multivariate analysis

Further readings

Creating a Data Analysis Plan: What to Consider When Choosing Statistics for a Study https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4552232/pdf/cjhp-68-311.pdf

Creating an Analysis Plan: https://www.cdc.gov/globalhealth/healthprotection/fetp/training_modules/9/creating-analysis-plan_pw_final_09242013.pdf

Data Analysis Plan: https://www.statisticssolutions.com/dissertation-consulting-services/data-analysis-plan-2/

Photo created by freepik – www.freepik.com

Dr Barche is a physician and holds a Masters in Public Health. He is a senior fellow at CRENC with interests in Data Science and Data Analysis.

Post Navigation

16 comments.

Thanks. Quite informative.

Educative write-up. Thanks.

Easy to understand. Thanks Dr

Very explicit Dr. Thanks

I will always remember how you help me conceptualize and understand data science in a simple way. I can only hope that someday I’ll be in a position to repay you, my dear friend.

Plan d’analyse

This is interesting, Thanks

Very understandable and informative. Thank you..

love the figures.

Nice, and informative

This is so much educative and good for beginners, I would love to recommend that you create and share a video because some people are able to grasp when there is an instructor. Lots of love

Thank you Doctor very helpful.

Educative and clearly written. Thanks

Well said doctor,thank you.But when do you present in tables ,bars,pie chart etc?

Very informative guide!

Submit a Comment Cancel Reply

Your email address will not be published. Required fields are marked *

Notify me of follow-up comments by email.

Notify me of new posts by email.

Submit Comment

  Receive updates on new courses and blog posts

Never Miss a Thing!

Subscribe to our mailing list to receive the latest news and updates on our webinars, articles and courses.

You have Successfully Subscribed!

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

18.3 Preparations: Creating a plan for qualitative data analysis Learning Objectives

Learning objectives.

Learners will be able to…

• Identify how your research question, research aim, sample selection, and type of data may influence your choice of analytic methods
• Outline the steps you will take in preparation for conducting qualitative data analysis in your proposal

Now we can turn our attention to planning your analysis. The analysis should be anchored in the purpose of your study. Qualitative research can serve a range of purposes. Below is a brief list of general purposes we might consider when using a qualitative approach.

• Are you trying to understand how a particular group is affected by an issue?
• Are you trying to uncover how people arrive at a decision in a given situation?
• Are you trying to examine different points of view on the impact of a recent event?
• Are you trying to summarize how people understand or make sense of a condition?
• Are you trying to describe the needs of your target population?

If you don’t see the general aim of your research question reflected in one of these areas, don’t fret! This is only a small sampling of what you might be trying to accomplish with your qualitative study. Whatever your aim, you need to have a plan for what you will do once you have collected your data.

Decision Point: What are you trying to accomplish with your data?

• Consider your research question. What do you need to do with the qualitative data you are gathering to help answer that question?

To help answer this question, consider:

• What action verb(s) can be associated with your project and the qualitative data you are collecting? Does your research aim to summarize, compare, describe, examine, outline, identify, review, compose, develop, illustrate, etc.?
• Then, consider noun(s) you need to pair with your verb(s)—perceptions, experiences, thoughts, reactions, descriptions, understanding, processes, feelings, actions responses, etc.

Iterative or linear

We touched on this briefly in Chapter 17 about qualitative sampling, but this is an important distinction to consider. Some qualitative research is linear , meaning it follows more of a traditionally quantitative process: create a plan, gather data, and analyze data; each step is completed before we proceed to the next. You can think of this like how information is presented in this book. We discuss each topic, one after another.

However, many times qualitative research is iterative , or evolving in cycles. An iterative approach means that once we begin collecting data, we also begin analyzing data as it is coming in. This early and ongoing analysis of our (incomplete) data then impacts our continued planning, data gathering and future analysis. Again, coming back to this book, while it may be written linear, we hope that you engage with it iteratively as you are building your proposal. By this we mean that you will revisit previous sections so you can understand how they fit together and you are in continuous process of building and revising how you think about the concepts you are learning about.

As you may have guessed, there are benefits and challenges to both linear and iterative approaches. A linear approach is much more straightforward, each step being fairly defined. However, linear research being more defined and rigid also presents certain challenges. A linear approach assumes that we know what we need to ask or look for at the very beginning of data collection, which often is not the case.

With iterative research, we have more flexibility to adapt our approach as we learn new things. We still need to keep our approach systematic and organized, however, so that our work doesn’t become a free-for-all. As we adapt, we do not want to stray too far from the original premise of our study. It’s also important to remember with an iterative approach that we may risk ethical concerns if our work extends beyond the original boundaries of our informed consent and IRB agreement. If you feel that you do need to modify your original research plan in a significant way as you learn more about the topic, you can submit an addendum to modify your original application that was submitted. Make sure to keep detailed notes of the decisions that you are making and what is informing these choices. This helps to support transparency and your credibility throughout the research process.

Decision Point: Will your analysis reflect more of a linear or an iterative approach?

• What justifies or supports this decision?

Think about:

• Fit with your research question
• Available time and resources
• Your knowledge and understanding of the research process

Reflexive Journal Entry Prompt

• What evidence are you basing this on?
• How might this help or hinder your qualitative research process?
• How might this help or hinder you in a practice setting as you work with clients?

Acquainting yourself with your data

As you begin your analysis, you need to get to know your data. This usually means reading through your data prior to any attempt at breaking it apart and labeling it. You might read through a couple of times, in fact. This helps give you a more comprehensive feel for each piece of data and the data as a whole, again, before you start to break it down into smaller units or deconstruct it. This is especially important if others assisted us in the data collection process. We often gather data as part of team and everyone involved in the analysis needs to be very familiar with all of the data.

Capturing your reaction to the data

During the review process, our understanding of the data often evolves as we observe patterns and trends. It is a good practice to document your reaction and evolving understanding. Your reaction can include noting phrases or ideas that surprise you, similarities or distinct differences in responses, additional questions that the data brings to mind, among other things. We often record these reactions directly in the text or artifact if we have the ability to do so, such as making a comment in a word document associated with a highlighted phrase. If this isn’t possible, you will want to have a way to track what specific spot(s) in your data your reactions are referring to. In qualitative research we refer to this process as memoing . Memoing is a strategy that helps us to link our findings to our raw data, demonstrating transparency. If you are using a Computre-Assisted Qualitative Data Analysis Software ( CAQDAS) software package, memoing functions are generally built into the technology.

Capturing your emerging understanding of the data

During your reviewing and memoing you will start to develop and evolve your understanding of what the data means. This understanding should be dynamic and flexible, but you want to have a way to capture this understanding as it evolves. You may include this as part of your memoing or as part of your codebook where you are tracking the main ideas that are emerging and what they mean. Figure 18.3 is an example of how your thinking might change about a code and how you can go about capturing it. Coding is a part of the qualitative data analysis process where we begin to interpret and assign meaning to the data. It represents one of the first steps as we begin to filter the data through our own subjective lens as the researcher. We will discuss coding in much more detail in the sections below covering various different approaches to analysis.

Decision Point: How to capture your thoughts?

• What will this look like?
• How often will you do it?
• How will you keep it organized and consistent over time?

In addition, you will want to be actively using your reflexive journal during this time. Document your thoughts and feelings throughout the research process. This will promote transparency and help account for your role in the analysis.

For entries during your analysis, respond to questions such as these in your journal:

• What surprises you about what participants are sharing?
• How has this information challenged you to look at this topic differently?
• Where might these have come from?
• How might these be influencing your study?
• How will you proceed differently based on what you are learning?

By including community members as active co-researchers, they can be invaluable in reviewing, reacting to and leading the interpretation of data during your analysis. While it can certainly be challenging to converge on an agreed-upon version of the results; their insider knowledge and lived experience can provide very important insights into the data analysis process.

Determining when you are finished

When conducting quantitative research, it is perhaps easier to decide when we are finished with our analysis. We determine the tests we need to run, we perform them, we interpret them, and for the most part, we call it a day. It’s a bit more nebulous for qualitative research. There is no hard and fast rule for when we have completed our qualitative analysis. Rather, our decision to end the analysis should be guided by reflection and consideration of a number of important questions. These questions are presented below to help ensure that your analysis results in a finished product that is comprehensive, systematic, and coherent.

Have I answered my research question?

Your analysis should be clearly connected to and in service of answering your research question. Your examination of the data should help you arrive at findings that sufficiently address the question that you set out to answer. You might find that it is surprisingly easy to get distracted while reviewing all your data. Make sure as you conducted the analysis you keep coming back to your research question.

Have I utilized all my data?

Unless you have intentionally made the decision that certain portions of your data are not relevant for your study, make sure that you don’t have sources or segments of data that aren’t incorporated into your analysis. Just because some data doesn’t “fit” the general trends you are uncovering, find a way to acknowledge this in your findings as well so that these voices don’t get lost in your data.

Have I fulfilled my obligation to my participants?

As a qualitative researcher, you are a craftsperson. You are taking raw materials (e.g. people’s words, observations, photos) and bringing them together to form a new creation, your findings. These findings need to both honor the original integrity of the data that is shared with you, but also help tell a broader story that answers your research question(s).

Have I fulfilled my obligation to my audience?

Not only do your findings need to help answer your research question, but they need to do so in a way that is consumable for your audience. From an analysis standpoint, this means that we need to make sufficient efforts to condense our data. For example, if you are conducting a thematic analysis, you don’t want to wind up with 20 themes. Having this many themes suggests that you aren’t finished looking at how these ideas relate to each other and might be combined into broader themes. Having these sufficiently reduced to a handful of themes will help tell a more complete story, one that is also much more approachable and meaningful for your reader.

In the following subsections, there is information regarding a variety of different approaches to qualitative analysis. In designing your qualitative study, you would identify an analytical approach as you plan out your project. The one you select would depend on the type of data you have and what you want to accomplish with it.

Key Takeaways

• Qualitative research analysis requires preparation and careful planning. You will need to take time to familiarize yourself with the data in general sense before you begin analyzing.
• Once you begin your analysis, make sure that you have strategies for capture and recording both your reaction to the data and your corresponding developing understanding of what the collective meaning of the data is (your results). Qualitative research is not only invested in the end results but also the process at which you arrive at them.

Decision Point: When will you stop?

• How will you know when you are finished? What will determine your endpoint?
• How will you monitor your work so you know when it’s over?

A research process where you create a plan, you gather your data, you analyze your data and each step is completed before you proceed to the next.

An iterative approach means that after planning and once we begin collecting data, we begin analyzing as data as it is coming in.  This early analysis of our (incomplete) data, then impacts our planning, ongoing data gathering and future analysis as it progresses.

The point where gathering more data doesn't offer any new ideas or perspectives on the issue you are studying.  Reaching saturation is an indication that we can stop qualitative data collection.

Memoing is the act of recording your thoughts, reactions, quandaries as you are reviewing the data you are gathering.

These are software tools that can aid qualitative researchers in managing, organizing and manipulating/analyzing their data.

A document that we use to keep track of and define the codes that we have identified (or are using) in our qualitative data analysis.

Part of the qualitative data analysis process where we begin to interpret and assign meaning to the data.

A research journal that helps the researcher to reflect on and consider their thoughts and reactions to the research process and how it may be shaping the study

Doctoral Research Methods in Social Work Copyright © by Mavs Open Press. All Rights Reserved.

Share This Book

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

6 Analyzing Data from Your Classroom

ESSENTIAL QUESTIONS

• What are the best ways to organize and analyze your data?
• What methods of data analysis will be most effective for your study?
• What claims can you make after analyzing your data?
• How do your claims contribute to the knowledge base?

You’ve determined the methods for data collection and then collected that data for your action research project. It is now time to conduct the analysis of your data, which precedes drawing conclusions and sharing your findings. During your action research project, you have been informally analyzing your data and now you can formally analyze to develop findings and reflect on their implications for practice. This will also provide an opportunity to identify unanswered questions and possible new directions. As an action researcher, you will create a coherent and reliable story from all the data collected. This is a key part of the professional development or professional learning aspect of action research. As an action researcher, you are looking to create meaning from your practice by utilizing rich descriptions and narratives, and you are developing expertise by examining situations closely and analyzing them.

Beginning the Meaning Making Process

Before you begin your data analysis, you should revisit the intended goals of the project. Equally, you should think about your research question and reacquaint yourself with your literature review to clearly envision what you have been investigating and why. The goal of data analysis is to identify themes and patterns to provide robust evidence for any claims you are able to make from your findings. You will need to look at the data you have collected from several sources and relate these to your original, expected, outcomes. Of course, you will also be mindful of unexpected outcomes which may be of significance to your study too. Your conclusions should relate to the original intended objectives of the study. Again, your literature review will also help with the analysis, and it will provide distinctions in terms of what we know and don’t know as a field of study. Your findings should either confirm previous literature or provide new knowledge in relation to previous literature.

During this stage of the project, it is also important for you to reflect on the research process itself. Did your project go as planned? What would you do differently? What were the biggest challenges? For those interested in your study, they will be interested in knowing about your challenges, as well as your successes.

Organizing your Data

You will want to make a note of everything you possibly can when you collect data. As you organize your data, take a look at the notes or personal journal you have kept during your data collection period. Your notes may reveal that you have initiated important data analysis in the time and space of collecting the data. This sort of analysis could be personal notes on the themes which related to your original research aims and questions. You may have even made determinations about whether to gather additional data. More importantly, you may have noted some unanticipated themes or ideas that emerged during data collection. I think it is valuable to put these things to the front of your mind as you organize your data and begin the analysis stage.

Analysis and Presentation of your Data

I would like to address some general issues related to analyzing and presenting your data. Here are links to specific examples of data analysis in four action research articles, in the journal Networks: An Online Journal for Teacher Research.  These are not necessarily ideal examples, but they provide a variety to spark thinking about your own study and discussion among your classmates.

Using quantitative data

You may have collected some quantitative data to provide demographic, contextual, or academic background for your study. Quantitative data helps support, supplement, and complement the qualitative data you have collected. While it is likely that you have not collected a massive amount of quantitative data, any amount will support a stronger argument. You should be able to analyze and represent the quantitative data using tables or charts. Computer software, such as Microsoft Excel, is suitable for this purpose, and can even handle basic correlations between multiple sets of data (e.g., gender and test scores). If an action research project involves several sites and the data are extensive, you may consider using a statistical package, such as IBM’s SPSS. Your quantitative data can be presented in charts and graphs developed by these programs. Including charts and graphs is worthwhile for two reasons. First, a visual representation is often easier for many readers to understand when digesting data-based information. Second, visual representations break up continuous narratives which can useful when conveying a considerable amount of numerical data and the subsequent correlations. Visual representations can also be an effective way to present qualitative data, or to at least give the reader a glimpse or preview of the data as a whole representation before reading the narrative.

Using qualitative data

Since your action research project was probably located within your professional context and focused on your practice, you likely explored attitudes, behaviors, and feelings that required collecting qualitative data. Most of this data will be in the form of descriptive text or short answer text, which you will need to analyze and interpret. For qualitative data, analysis of the text will require you to develop an analytical framework to use as the basis of analysis. This framework can also be subjective, so being clear and upfront about your framework is important for validity and reliability. If your data collection resulted in a large amount of descriptive text, it may seem overwhelming to analyze. This is quite normal for a qualitative study and having a lot of data means that you will have plenty to work from. If you have a considerable amount of descriptive data, you could use computer software (as outlined later in this chapter) which is relatively simple to use.

Transcription of your data is something that is often overlooked, or avoided, for several reasons. For some types of data collection (e.g., interviews, focus groups, discussions, etc.) it is useful to have the recording transcribed so that you can analyze the text more easily than listening to the recording. Transcribing is often avoided because of time constraints (or because the researcher cannot afford to have someone transcribe for them). However, if you are going to analyze the data, you should not think of it as purely transcription – it is your first opportunity to engage with the data. This will facilitate a more efficient process of analysis as it will more than likely be the second time you have engaged with the data. If you have previewed or already experienced the data, to save time you might transcribe only the parts that are pertinent to the study or your interest.

Analyzing qualitative data

For better, or worse, there is no universally correct way to analyze qualitative data; however, it is important to be systematic in your method of analysis. As I mentioned earlier, your data analysis probably started initially during your data collection. The questions you asked, the frameworks that you used, and the types of documents you collected would have provided some themes and categories that naturally developed as part of this process. I have suggested to new researchers a step-by-step approach to help them get started:

• Organize your data. Begin by listing the different sets of data you have collected, show how they are related, and how they will support each other (triangulation).
• Read the content. You need to read the data, probably several times, to develop a sense of what the data are indicating. All your data – observation notes, field diaries, policy documents and so on – need to be looked at. Common words and themes should start to emerge.
• Highlight relevant sections and aspects of the data.
• Develop categories to sort evidence. As you examine the data you will need to use actual evidence (numbers, actual quotes, artifacts, etc.) from your data to support your claims. You want these pieces of evidence to be the most vivid or clear representation for the categories you develop. For example, if you interviewed fifteen students and twelve of the interview transcripts provided evidence that the students’ understanding had grown due to your instructional intervention, you would want to note that twelve of fifteen students interviewed demonstrated growth in understanding, and possibly provide a quote or sample of how this was demonstrated from one of those twelve students. This sort of evidence enhances the trustworthiness of your findings.
• Code your data. Codes will develop from the categories you use to sort the evidence you find in the range of data. Codes also help you when you do a second or third analysis of the data as it guides your examination of the data. (Coding is discussed later in this chapter.)
• Review and narrow the codes. You may begin with a lot of initial codes, but you will want to narrow these to the most significant, well-evidenced, or best triangulated data. Most likely, these narrow codes will become the significant themes to report on your study.
• Interpret your findings. Once you have narrowed your codes, and have evidence in place to support those codes, it is time to interpret the data and develop meaning within the context of your study, and field. This is where your literature review will be useful again.
• Validate the findings. Validation, in addition to this process (see figure 6.1 below), can take many forms. In previous chapters, I had discussed using critical friends to confirm the validity of your interpretations.
• Create report and plan dissemination .

A framework for qualitative data analysis and interpretation

If you are feeling a bit overwhelmed by the amount of qualitative data you collected, you may find Creswell’s (2009) framework to analyze and interpret qualitative data useful (See figure 6.1).

Figure 6.1 Qualitative Data Analysis, interpreted from Creswell (Creswell, 2009, p. 185)

Similarly to above, Creswell also proposes a step-by-step approach to provide practitioners with a guide to undertake action research. I have summarized this in the following section.

Step 1. Organize the data for analysis. You will need to transcribe interviews, scan material, type up your notes, and sort or arrange the different types of data.

Step 2. Read all the data thoroughly. Get a general sense of the data and reflect on their overall meaning. You may have received an initial impression from the data collection, but make notes in the margins or spaces and record any other initial thoughts at this stage.

Step 3. Begin detailed coding and analysis. Coding organizes the material into meaningful chunks of text. When coding, think about:

• code based on previous literature and common sense;
• code what is surprising and unanticipated;
• code for the unusual which may be of conceptual interest to readers.

You may want to hand-code the data, use highlighting colors, or cut and paste text segments onto cards. You may also use a computer software package to help to code, organize and sort the information (e.g., NVivo)

Step 4. Codes should be representative of the categories, topic, setting, or people that are part of the analysis. Creswell suggests generating 5-7 categories. These will be supported with quotations and specific evidence form the data and may represent headings in your report.

Step 5. Decide how you will represent the codes, themes, and descriptions in the narrative. The narrative will summarize the findings from the analysis. This could be a discussion that outlines the project chronologically, a detailed discussion of several themes (including sub-themes, specific illustrations, multiple perspectives from individuals, and quotations), or a discussion with interconnecting themes. Visuals, graphs, figures, or tables are also useful to support the discussion.

Step 6. This final step involves making an interpretation or deriving meaning from the data. Meaning might come from, but is not limited to, lessons learned from the data. Meaning can also be derived when comparing findings to the literature or theories from the literature review.

Positionality and qualitative data

When analyzing qualitative data, the issue of your own positionality will need to be addressed. Positionality was mentioned in a previous chapter; however, addressing your positionality involves how your own social identity and experiences may impact your interpretation of the data. For example, an educator-researcher may have complex identities that they need to be aware of when they are analyzing the data. As a privileged white male with a terminal degree of education, I have to realize I may not fully relate to the experiences of many of my students, and this is important if I am analyzing the attitudes and beliefs of my students. I need to keep that under consideration throughout the research process, but especially as a I deriving meaning from the perspectives of my students. Therefore, positionality is very important for an educator-researcher who is planning and implementing action in a classroom, while they are also a teacher. It is also important to consider the possible impact of being an educator-researcher and acknowledge the possible influence this may have on the interpretations they make and any bias which may influence the research process. Qualitative research is interpretive research with the researcher typically involved in a sustained and intensive experience with the participants, which opens up a range of potential ethical and personal issues into the qualitative research process.

Many action research reports include a section on positionality, in which the researchers write a narrative describing their positionality and keep that visible as they analyze data.  Below are some questions regarding what would constitute a positionality statement:

Positionality Statements

• Who am I? (Including demographics, epistemologies and philosophies, journey in education, etc.)
• What do I believe about teaching/learning?
• What do I believe about this topic?
• What are my expectations of this study?
• What are my connections or dis-connections with the participants?
• What are my experiences with the context of this study?

Analysis with computer software

It is now common for data to be analyzed using computer software. However, as Mertler (2008) notes, it is a misconception to think that the software will do the analysis, as data analysis still requires the use of inductive logic, and therefore, advanced technologies cannot take the place of the human brain. Computer software primarily helps researchers organize and store data. Software such as NVivo can also provide very efficient systems for coding a lot data, as well as many different types of data, including social media and video. Software like NVivo can be expensive for an educator-researcher. There are several free and/or cheaper applications and software that provide many of the same features.

Coding your data is such an important part of the analysis process, I want to devote a bit more discussion to the process. Simply put, coding entails identifying the main themes and patterns within your data. Coding is meant to help you conceptualize and condense your data into meaningful and manageable chunks from which to make conclusions. Coding data can take many shapes and forms. Regardless of how you choose to code your data, it is important to keep your research goals and research questions in the forefront of your mind. After immersing yourself in your data sources, it is possible to feel somewhat overcome by thinking and possibilities sparked by the data. This feeling may be caused by one of two issues you will have to deal with in your analysis. First, it is possible you may think that nearly everything you have collected is relevant and significant to the study, which could lead to some stress in how to determine what to focus on, or what is most significant. Coding should help reduce this stress by bringing patterns and themes to the forefront to help you prioritize some aspects of the data, and make it feel much more manageable. Once you begin to realize you are coding only those things which are relevant, you will ease the stress and begin to enjoy the analysis and coding process. Second, coding can be taxing work because of the constant processing, categorizing, and depth of thinking. Like it is suggested when revising writing, take regular breaks to maintain your full concentration, and in the case of research, to also review your coding criteria.

Using Evidence and Generating Knowledge

The main purpose of gathering data, through a research process, is to provide evidence. In order to provide evidence, you need to analyze the data you have collected. Again, it is important to remember the starting place of your inquiry, and what you are looking for in the study. You began with goals at the start of your research and mapped out your data collection strategically, now you have the data which will provide evidence for articulating your claims and developing pedagogical theories.

Regardless of the type of data you have collected, quantitative data or qualitative data or a combination of the two, ultimately the significance and impact of your research will depend on the quality of data you have collected, the interpretations you make, and your reflections and conclusions. Therefore, the significance of your study will depend upon the quality of the data you have collected and depth of your data analysis.

While you are engaged in data analysis, it might be useful to highlight the data that could be used as evidence to support your claims when you share your research. In the past, I have color coded different types of evidence.

So, what do we mean when we say provide evidence? When researchers provide evidence, they are providing pieces of data that support their claims about what their study did or did not demonstrate. In the next chapter, we will discuss how to share or report your findings. When you share or report you can think of it as an argument that you are making about your findings and subsequent claims. The data is used as evidence to support your claims and strengthen your argument. It is important to remember that to develop valid claims to knowledge, you will need to support your claims with evidence using relevant parts of your data. Therefore, evidence may take the form of survey results, quotes or extracts from interview transcripts, selections from your classroom observation notes, artifacts, photographs, and examples of students’ work.

Generating Knowledge

The purpose of research is to generate new knowledge. As an educator and researcher, the knowledge you produce will be based on your practice. Once you have findings and claims, this will most likely affect your practice. You will articulate knowledge that is generated from how your research has affected your practice and contemplate what significance it may have for other practitioners. This process amounts to you building personal theories about what you have done and demonstrated in your study. Therefore, your theories will emerge from your practice and this will contribute new knowledge to the existing knowledge base. Your data will provide illustrative examples of what happened in your classroom and you will cite relevant evidence. When you develop knowledge from your study, the claims you make and the theories you formulate are original, as you have employed your own critical thinking skills and informed judgement. Your critical thinking and informed judgement are demonstrated in the evidence you provide to validate your claims to knowledge.

Creating Trustworthy Claims to Knowledge

Your research findings and claims to knowledge are much more impactful if demonstrated as trustworthy. Action research is often conducted in collaborative teams, involving communities of educator researchers. Collaborative teams have built in opportunities to increase trustworthiness of studies. Having multiple people make interpretations of the same data creates trustworthiness through common understandings, making the findings more representative. The trustworthiness of research is also based on readers or consumers of your research accept your claims to knowledge. It is scary for many to think they need to validate claims to knowledge, and readers or consumers will critically evaluate their claims. As mentioned in previous chapters, trustworthiness can be accomplished methodologically, and when you report or share your findings you need to simply articulate your methods for trustworthiness. For example:

Achieving Trustworthiness

• Articulate your procedures clearly;
• Explain how you conducted your research thoroughly;
• Describe the robustness of your data collection methods;
• Make clear how triangulation was achieved.

Expect to be challenged on any aspect of your research claims. This is where validation meetings with different groups of people are useful, to have them consider their research processes and findings from different perspectives both during their research and at the conclusion of the project. When you report or share your research, you should include details of your validation meetings and any developments that resulted from these meetings.

Critical Friends can also be useful in regard to trustworthiness. When you establish the project recruit, Critical Friends. Explain that their role is to evaluate all aspects of the research by challenging your assumptions and considering ways to reduce subjectivity and ethical issues. Critical Friends can be helpful in thinking about all aspects of your research, even the implications, usefulness, and replicability of your research. You can utilize Critical Friends either individually or as a group to provide formative feedback at different points of the research.

Data Analysis Checklist

• What is your positionality in regard to the data? How will it affect your analysis?
• How will you organize and prepare your raw data for analysis?
• Read and engage with all of your data.
• Code. What themes or categories are emerging across the data?
• What descriptions will you use to define and characterize your codes?
• Are the codes and/or themes interrelated? Are there sub-codes?
• How will you represent codes in the final report?
• What theories can you use to interpret the codes?
• What do your themes, codes, and descriptions mean in relation to your research question(s)?
• Would a Critical Friend or colleague’s review of your analysis add to the trustworthiness of the study?

Action Research Copyright © by J. Spencer Clark; Suzanne Porath; Julie Thiele; and Morgan Jobe is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License , except where otherwise noted.

Share This Book

• Review Article
• Open access
• Published: 22 June 2020

Teaching analytics, value and tools for teacher data literacy: a systematic and tripartite approach

• Ifeanyi Glory Ndukwe 1 &
• Ben Kei Daniel 1  

International Journal of Educational Technology in Higher Education volume  17 , Article number:  22 ( 2020 ) Cite this article

26k Accesses

40 Citations

14 Altmetric

Metrics details

Teaching Analytics (TA) is a new theoretical approach, which combines teaching expertise, visual analytics and design-based research to support teacher’s diagnostic pedagogical ability to use data and evidence to improve the quality of teaching. TA is now gaining prominence because it offers enormous opportunities to the teachers. It also identifies optimal ways in which teaching performance can be enhanced. Further, TA provides a platform for teachers to use data to reflect on teaching outcome. The outcome of TA can be used to engage teachers in a meaningful dialogue to improve the quality of teaching. Arguably, teachers need to develop their teacher data literacy and data inquiry skills to learn about teaching challenges. These skills are dependent on understanding the connection between TA, LA and Learning Design (LD). Additionally, they need to understand how choices in particular pedagogues and the LD can enhance their teaching experience. In other words, teachers need to equip themselves with the knowledge necessary to understand the complexity of teaching and the learning environment. Providing teachers access to analytics associated with their teaching practice and learning outcome can improve the quality of teaching practice. This research aims to explore current TA related discussions in the literature, to provide a generic conception of the meaning and value of TA. The review was intended to inform the establishment of a framework describing the various aspects of TA and to develop a model that can enable us to gain more insights into how TA can help teachers improve teaching practices and learning outcome. The Tripartite model was adopted to carry out a comprehensive, systematic and critical analysis of the literature of TA. To understand the current state-of-the-art relating to TA, and the implications to the future, we reviewed published articles from the year 2012 to 2019. The results of this review have led to the development of a conceptual framework for TA and established the boundaries between TA and LA. From the analysis the literature, we proposed a Teaching Outcome Model (TOM) as a theoretical lens to guide teachers and researchers to engage with data relating to teaching activities, to improve the quality of teaching.

Introduction

Educational institutions today are operating in an information era, where machines automatically generate data rather than manually; hence, the emergence of big data in education ( Daniel 2015 ). The phenomenon of analytics seeks to acquire insightful information from data that ordinarily would not be visible by the ordinary eyes, except with the application of state-of-the-art models and methods to reveal hidden patterns and relationships in data. Analytics plays a vital role in reforming the educational sector to catch up with the fast pace at which data is generated, and the extent to which such data can be used to transform our institutions effectively. For example, with the extensive use of online and blended learning platforms, the application of analytics will enable educators at all levels to gain new insights into how people learn and how teachers can teach better. However, the current discourses on the use of analytics in Higher Education (HE) are focused on the enormous opportunities analytics offer to various stakeholders; including learners, teachers, researchers and administrators.

In the last decade, extensive literature has proposed two weaves of analytics to support learning and improve educational outcomes, operations and processes. The first form of Business Intelligence introduced in the educational industry is Academic Analytics (AA). AA describes data collected on the performance of academic programmes to inform policy. Then, Learning Analytics (LA), emerged as the second weave of analytics, and it is one of the fastest-growing areas of research within the broader use of analytics in the context of education. LA is defined as the "measurement, collection, analysis and reporting of data about the learner and their learning contexts for understanding and optimising learning and the environments in which it occurs" ( Elias 2011 ). LA was introduced to attend to teaching performance and learning outcome ( Anderson 2003 ; Macfadyen and Dawson 2012 ). Typical research areas in LA, include student retention, predicting students at-risk, personalised learning which in turn are highly student-driven ( Beer et al. 2009 ; Leitner et al. 2017 ; Pascual-Miguel et al. 2011 ; Ramos and Yudko 2008 ). For instance, Griffiths ( Griffiths 2017 ), employed LA to monitor students’ engagements and behavioural patterns on a computer-supported collaborative learning environment to predict at-risk students. Similarly, Rienties et al. ( Rienties et al. 2016 ) looked at LA approaches in their capacity to enhance the learner’s retention, engagement and satisfaction. However, in the last decade, LA research has focused mostly on the learner and data collections, based on digital data traces from Learning Management Systems (LMS) ( Ferguson 2012 ), not the physical classroom.

Teaching Analytics (TA) is a new theoretical approach that combines teaching expertise, visual analytics and design-based research, to support the teacher with diagnostic and analytic pedagogical ability to improve the quality of teaching. Though it is a new phenomenon, TA is now gaining prominence because it offers enormous opportunities to the teachers.

Research on TA pays special attention to teacher professional practice, offering data literacy and visual analytics tools and methods ( Sergis et al. 2017 ). Hence, TA is the collection and use of data related to teaching and learning activities and environments to inform teaching practice and to attain specific learning outcomes. Some authors have combined the LA, and TA approaches into Teaching and Learning Analytics (TLA) ( Sergis and Sampson 2017 ; Sergis and Sampson 2016 ). All these demonstrate the rising interest in collecting evidence from educational settings for awareness, reflection, or decision making, among other purposes. However, the most frequent data that have been collected and analysed about TA focus on the students (e.g., different discussion and learning activities and some sensor data such as eye-tracking, position or physical actions) ( Sergis and Sampson 2017 ), rather than monitoring teacher activities. Providing teachers access to analytics of their teaching, and how they can effectively use such analytics to improve their teaching process is a critical endeavour. Also, other human-mediated data gathering in the form of student feedback, self and peer observations or teacher diaries can be employed to enrich TA further. For instance, visual representations such as dashboards can be used to present teaching data to help teachers reflect and make appropriate decisions to inform the quality of teaching. In other words, TA can be regarded as a reconceptualisation of LA for teachers to improve teaching performance and learning outcome. The concept of TA is central to the growing data-rich technology-enhanced learning and teaching environment ( Flavin 2017 ; Saye and Brush 2007 ). Further, it provides teachers with the opportunity to engage in data-informed pedagogical improvement.

While LA is undeniably an essential area of research in educational technology and the learning sciences, automatically extracted data from an educational platform mainly provide an overview of student activities, and participation. Nevertheless, it hardly indicates the role of the teacher in these activities, or may not otherwise be relevant to teachers’ individual needs (for Teaching Professional Development (TPD) or improvement of their classroom practice). Many teachers generally lack adequate data literacy skills ( Sun et al. 2016 ). Teacher data literacy skill and teacher inquiry skill using data are the foundational concepts underpinning TA ( Kaser and Halbert 2014 ). The development of these two skills is dependent on understanding the connection between TA, LA and Learning Design (LD). In other words, teachers need to equip themselves with knowledge through interaction with sophisticated data structures and analytics. Hence, TA is critical to improving teachers’ low efficacy towards educational data.

Additionally, technology has expanded the horizon of analytics to various forms of educational settings. As such, the educational research landscape needs efficient tools for collecting data and analyzing data, which in turn requires explicit guidance on how to use the findings to inform teaching and learning ( McKenney and Mor 2015 ). Increasing the possibilities for teachers to engage with data to assess what works for the students and courses they teach is instrumental to quality ( Van Harmelen and Workman 2012 ). TA provides optimal ways of performing the analysis of data obtained from teaching activities and the environment in which instruction occurs. Hence, more research is required to explore how teachers can engage with data associated with teaching to encourage teacher reflection, improve the quality of teaching, and provide useful insights into ways teachers could be supported to interact with teaching data effectively. However, it is also essential to be aware that there are critical challenges associated with data collection. Moreover, designing the information flow that facilitates evidence-based decision-making requires addressing issues such as the potential risk of bias; ethical and privacy concerns; inadequate knowledge of how to engage with analytics effectively.

To ensure that instructional design and learning support is evidence-based, it is essential to empower teachers with the necessary knowledge of analytics and data literacy. The lack of such knowledge can lead to poor interpretation of analytics, which in turn can lead to ill-informed decisions that can significantly affect students; creating more inequalities in access to learning opportunities and support regimes. Teacher data literacy refers to a teachers’ ability to effectively engage with data and analytics to make better pedagogical decisions.

The primary outcome of TA is to guide educational researchers to develop better strategies to support the development of teachers’ data literacy skills and knowledge. However, for teachers to embrace data-driven approaches to learning design, there is a need to implement bottom-up approaches that include teachers as main stakeholders of a data literacy project, rather than end-users of data.

The purpose of this research is to explore the current discusses in the literature relating to TA. A vital goal of the review was to extend our understanding of conceptions and value of TA. Secondly, we want to contextualise the notion of TA and develop various concepts around TA to establish a framework that describes multiple aspects of TA. Thirdly, to examine different data collections/sources, machine learning algorithms, visualisations and actions associated with TA. The intended outcome is to develop a model that would provide a guide for the teacher to improve teaching practice and ultimately enhance learning outcomes.

The research employed a systematic and critical analysis of articles published from the year 2012 to 2019. A total of 58 publications were initially identified and compiled from the Scopus database. After analysing the search results, 31 papers were selected for review. This review examined research relating to the utilisation of analytics associated with teaching and teacher activities and provided conceptual clarity on TA. We found that the literature relating to conception, and optimisation of TA is sporadic and scare, as such the notion of TA is theoretically underdeveloped.

Methods and procedures

This research used the Tripartite model ( Daniel and Harland 2017 ), illustrated in Fig.  1 , to guide the systematic literature review. The Tripartite model draws from systematic review approaches such as the Cochrane, widely used in the analyses of rigorous studies, to provide the best evidence. Moreover, the Tripartite model offers a comprehensive view and presentation of the reports. The model composes of three fundamental components; descriptive (providing a summary of the literature), synthesis (logically categorising the research based on related ideas, connections and rationales), and critique (criticising the novel, providing evidence to support, discard or offer new ideas about the literature). Each of these phases is detailed fully in the following sections.

Tripartite Model. The Tripartite Model: A Systematic Literature Review Process ( Daniel and Harland 2017 )

To provide clarity; the review first focused on describing how TA is conceptualised and utilised. Followed by the synthesis of the literature on the various tools used to harvest, analyse and present teaching-related data to the teachers. Then the critique of the research which led to the development of a conceptual framework describing various aspects of TA. Finally, this paper proposes a Teaching Outcome Model (TOM). TOM is intended to offer teachers help on how to engage and reflect on teaching data.

TOM is a TA life cycle which starts with the data collection stage; where the focus is on teaching data. Then the data analysis stage; the application of different Machine Learning (ML) techniques to the data to discover hidden patterns. Subsequently, the data visualisation stage, where data presentation is carried out in the form of a Teaching Analytics Dashboard (TAD) for the teacher. This phase is where the insight generation, critical thinking and teacher reflection are carried out. Finally, the action phase, this is where actions are implemented by teachers to improve teaching practice. Some of these actions include improving the LD, changing teaching method, providing appropriate feedback and assessment or even carrying out more research. This research aims to inform the future work in the advancement of TA research field.

Framing research area for review

As stated in the introduction, understanding current research on TA can be used to provide teachers with strategies that can help them utilise various forms of data to optimise teaching performance and outcome. Framing the review was guided by some questions and proposed answers to address those questions (see Table  1 )

Inclusion and exclusion criteria

The current review started with searching through the Scopus database using the SciVal visualisation and analytical tool. The rationale for choosing the Scopus database is that it contains the largest abstract and citation database of peer-reviewed research literature with diverse titles from publishers worldwide. Hence, it is only conceivable to search for and find a meaningful balance of the published content in the area of TA. Also, the review included peer-reviewed journals and conference proceedings. We excluded other documents and source types, such as book series, books, editorials, trade publications on the understanding that such sources might lack research on TA. Also, this review excluded articles published in other languages other than English.

Search strategy

This review used several keywords and combinations to search on terms related to TA. For instance: ’Teaching Analytics’ AND ’Learning Analytics’ OR ’Teacher Inquiry’ OR ’Data Literacy’ OR ’Learning Design’ OR ’Computer-Supported Collaborative Learning’ OR ’Open Learner Model’ OR ’Visualisation’ OR ’Learning Management System’ OR ’Intelligent Tutoring System’ OR ’Student Evaluation on Teaching’ OR ’Student Ratings’.

This review searched articles published between 2012 to 2019. The initial stage of the literature search yielded 58 papers. After the subsequent screening of previous works and removing duplicates and titles that did not relate to the area of research, 47 articles remained. As such, a total of 36 studies continued for full-text review. Figure  2 , shows the process of finalising the previous studies of this review.

Inclusion Exclusion Criteria Flowchart. The selection of previous studies

Compiling the abstracts and the full articles

The review ensured that the articles identified for review were both empirical and conceptual papers. The relevance of each article was affirmed by requiring that chosen papers contained various vital phrases all through the paper, as well as, title, abstract, keywords and, afterwards, the entire essay. In essence, were reviewed giving particular cognisance and specific consideration to those section(s) that expressly related to the field of TA. In doing as such, to extract essential points of view on definitions, data sources, tools and technologies associated with analytics for the teachers. Also, this review disregarded papers that did not, in any way, relate to analytics in the context of the teachers. Finally, 31 articles sufficed for this review.

Systematic review: descriptive

Several studies have demonstrated that TA is an important area of inquiry ( Flanders 1970 ; Gorham 1988 ; Pennings et al. 2014 ; Schempp et al. 2004 ), that enables researchers to explore analytics associated with teaching process systematically. Such analytics focus on data related to the teachers, students, subjects taught and teaching outcomes. The ultimate goal of TA is to improve professional teaching practice ( Huang 2001 ; Sergis et al. 2017 ). However, there is no consensus on what constitutes TA. Several studies suggest that TA is an approach used to analyse teaching activities ( Barmaki and Hughes 2015 ; Gauthier 2013 ; KU et al. 2018 ; Saar et al. 2017 ), including how teachers deliver lectures to students, tools usage pattern, or dialogue. While various other studies recognise TA as the ability to applying analytical methods to improve teacher awareness of student activities for appropriate intervention ( Ginon et al. 2016 ; Michos and Hernández Leo 2016 ; Pantazos et al. 2013 ; Taniguchi et al. 2017 ; Vatrapu et al. 2013 ). A hand full of others indicate TA as analytics that combines both teachers and students activities ( Chounta et al. 2016 ; Pantazos and Vatrapu 2016 ; Prieto et al. 2016 ; Suehiro et al. 2017 ). Hence, it is particularly problematic and challenging to carry out a systematic study in the area of analytics for the teachers to improve teaching practice, since there is no shared understanding of what constitutes analytics and how best to approach TA.

Researchers have used various tools to automatically harvest important episodes of interactive teacher and student behaviour during teaching, for teacher reflection. For instance, KU et al. ( 2018 ), utilised instruments such as; Interactive Whiteboard (IWB), Document Camera (DC), and Interactive Response System (IRS) to collect classroom instructional data during instruction. Similarly, Vatrapu et al. ( 2013 ) employed eye-tracking tools to capture eye-gaze data on various visual representations. Thomas ( 2018 ) also extracted multimodal features from both the speaker and the students’ audio-video data, using digital devices such as cameras and high-definition cameras. Data collected from some of these tools not only provide academics with real-time data but also attract more details about teaching and learning than the teacher may realise. However, the cost of using such digital tools for large-scale verification is high, and cheaper alternatives are sort after. For instance, Suehiro et al. ( 2017 ) proposed a novel approach of using e-books to extract teaching activity logs in a face-to-face class efficiently.

Vatrapu ( 2012 ) considers TA as a subset of LA dedicated to supporting teachers to understand the learning and teaching process. However, this definition does not recognise that both the learning and teaching processes are intertwined. Also, most of the research in LA collects data about the student learning or behaviour, to provide feedback to the teacher ( Vatrapu et al. 2013 ; Ginon et al. 2016 ; Goggins et al. 2016 ; Shen et al. 2018 ; Suehiro et al. 2017 ), see, for example, the iKlassroom conceptual proposal by Vatrapu et al. ( 2013 ), which highlights a map of the classroom to help contextualise real-time data about the learners in a lecture. Although, a few research draw attention to the analysis of teacher-gathering and teaching practice artefacts, such as lesson plans. Xu and Recker ( 2012 ) examined teachers tool usage patterns. Similarly, Gauthier ( 2013 ) extracted the analysis of the reasoning behind the expert teacher and used such data to improve the quality of teaching.

Multimodal analytics is an emergent trend used to complement available digital trace with data captured from the physical world ( Prieto et al. 2017 ). Isolated examples include the smart school multimodal dataset conceptual future proposal by Prieto et al. ( 2017 ), which features a plan of implementing a smart classroom to help contextualise real-time data about both the teachers and learners in a lecture. Another example, Prieto et al. ( 2016 ), explored the automatic extraction of orchestration graphs from a multimodal dataset gathered from only one teacher, classroom space, and a single instructional design. Results showed that ML techniques could achieve reasonable accuracy towards automated characterisation in teaching activities. Furthermore, Prieto et al. ( 2018 ) applied more advanced ML techniques to an extended version of the previous dataset to explore the different relationships that exist between datasets captured by multiple sources.

Previous studies have shown that teachers want to address common issues such as improving their TPD and making students learn effectively ( Charleer et al. 2013 ; Dana and Yendol-Hoppey 2019 ; Pennings et al. 2014 ). Reflection on teaching practice plays an essential role in helping teachers address these issues during the process of TPD ( Saric and Steh 2017 ; Verbert et al. 2013 ). More specifically, reflecting on personal teaching practice provides opportunities for teachers to re-examine what they have performed in their classes ( Loughran 2002 ; Mansfield 2019 ; Osterman and Kottkamp 1993 ). Which, in turn, helps them gain an in-depth understanding of their teaching practice, and thus improve their TPD. For instance, Gauthier ( 2013 ), used a visual teach-aloud method to help teaching practitioners reflect and gain insight into their teaching practices. Similarly, Saar et al. ( 2017 ) talked about a self-reflection as a way to improve teaching practice. Lecturers can record and observe their classroom activities, analyse their teaching and make informed decisions about any necessary changes in their teaching method.

The network analysis approach is another promising field of teacher inquiry, especially if combined with systematic, effective qualitative research methods ( Goggins et al. 2016 ). However, researchers and teacher who wish to utilise social network analysis must be specific about what inquiry they want to achieve. Such queries must then be checked and validated against a particular ontology for analytics ( Goggins 2012 ). Goggins et al. ( 2016 ), for example, aimed at developing an awareness of the types of analytics that could help teachers in Massive Open Online Courses (MOOCs) participate and collaborate with student groups, through making more informed decisions about which groups need help, and which do not. Network theory offers a particularly useful framework for understanding how individuals and groups respond to each other as they evolve. Study of the Social Network (SNA) is the approach used by researchers to direct analytical studies informed by network theory. SNA has many specific forms, each told by graph theory, probability theory, and algebraic modelling to various degrees. There are gaps in our understanding of the link between analytics and pedagogy. For example, which unique approaches to incorporating research methods for qualitative and network analysis would produce useful information for teachers in MOOCs? A host of previous work suggests a reasonable path to scaling analytics for MOOCs will involve providing helpful TA perspectives ( Goggins 2012 ; Goggins et al. 2016 ; Vatrapu et al. 2012 ).

Teacher facilitation is considered a challenging and critical aspect of active learning ( Fischer et al. 2014 ). Both educational researchers and practitioners have paid particular attention to this process, using different data gathering and visualisation methods, such as classroom observation, student feedback, audio and video recordings, or teacher self-reflection. TA enables teachers to perform analytics through visual representations to enhance teachers’ experience ( Vatrapu et al. 2011 ). As in a pedagogical environment, professionals have to monitor several data such as questions, mood, ratings, or progress. Hence, dashboards have become an essential factor in improving and conducting successful teaching. Dashboards are visualisation tools enable teachers to monitor and observe teaching practice to enhance teacher self-reflection ( Yigitbasioglu and Velcu 2012 ). While a TAD is a category of dashboard meant for teachers and holds a unique role and value [62]. First, TAD could allow teachers to access students learning in an almost real-time and scalable manner ( Mor et al. 2015 ), consequently, enabling teachers to improve their self-knowledge by monitoring and observing students activities. TAD assists the teachers in obtaining an overview of the whole classroom as well as drill down into details about individual and groups of students to identify student competencies, strengths and weaknesses. For instance, Pantazos and Vatrapu ( 2016 ) described TAD for repertory grid data to enable teachers to conduct systematic visual analytics of classroom learning data for formative assessment purposes. Second, TAD also allows for tracking on teacher self-activities ( van Leeuwen et al. 2019 ), as well as students feedback about their teaching practice. For example,Barmaki and Hughes ( 2015 ) explored a TAD that provides automated real-time feedback based on speakers posture, to support teachers practice classroom management and content delivery skills. It is a pedagogical point that dashboards can motivate teachers to reflect on teaching activities, help them improve teaching practice and learning outcome ( 2016 ). The literature has extensively described extensively, different teaching dashboards. For instance, Dix and Leavesley ( 2015 ), broadly discussed the idea of TAD and how they can represent visual tools for academics to interface with learning analytics and other academic life. Some of these academic lives may include schedules such as when preparing for class or updating materials, or meeting times such as meeting appointments with individual or collective group of students. Similarly, Vatrapu et al. ( 2013 ) explored TAD using visual analytics techniques to allow teachers to conduct a joint analysis of students personal constructs and ratings of domain concepts from the repertory grids for formative assessment application.

Systematic review: synthesis

In this second part of the review process, we extracted selected ideas from previous studies. Then group them based on data sources, analytical methods used, types of visualisations performed and actions.

Data sources and tools

Several studies have used custom software and online applications such as employing LMS and MOOCs to collect online classroom activities ( Goggins et al. 2016 ; KU et al. 2018 ; Libbrecht et al. 2013 ; Müller et al. 2016 ; Shen et al. 2018 ; Suehiro et al. 2017 ; Vatrapu et al. 2013 ; Xu and Recker 2012 ). Others have used modern devices including eye-tracker, portable electroencephalogram (EEG), gyroscope, accelerometer and smartphones ( Prieto et al. 2016 ; Prieto et al. 2018 ; Saar et al. 2017 ; Saar et al. 2018 ; Vatrapu et al. 2013 ), and conventional instruments such as video and voice recorders ( Barmaki and Hughes 2015 ; Gauthier 2013 ; Thomas 2018 ), to record classroom activities. However, some authors have pointed out several issues with modern devices such as expensive equipment, high human resource and ethical concerns ( KU et al. 2018 ; Prieto et al. 2017 ; Prieto et al. 2016 ; Suehiro et al. 2017 ).

In particular, one study by Chounta et al. ( 2016 ) recorded classroom activities using humans to code tutor-student dialogue manually. However, they acknowledged that manual coding of lecture activities is complicated and cumbersome. Some authors also subscribe to this school of thought and have attempted to address this issue by applying Artificial Intelligence (AI) techniques to automate and scale the coding process to ensure quality in all platforms ( Prieto et al. 2018 ; Saar et al. 2017 ; Thomas 2018 ). Others have proposed re-designing TA process to automate the process of data collection as well as making the teacher autonomous in collecting data about their teaching ( Saar et al. 2018 ; Shen et al. 2018 ). Including using technology that is easy to set up, effortless to use, does not require much preparation and at the same time, not interrupting the flow of the class. In this way, they would not require researcher assistance or outside human observers. Table  2 , summarises the various data sources as well as tools that are used to harvest teaching data with regards to TA.

The collection of evidence from both online and real classroom practice is significant both for educational research and TPD. LA deals mostly with data captured from online and blended learning platforms (e.g., log data, social network and text data). Hence, LA provides teachers with data to monitor and observe students online class activities (e.g., discussion boards, assignment submission, email communications, wiki activities and progress). However, LA neglects to capture physical occurrences of the classroom and do not always address individual teachers’ needs. TA requires more adaptable forms of classroom data collection (e.g., through video- recordings, sensor recording or by human observers) which are tedious, human capital intensive and costly. Other methods have been explored to balance the trade-off between data collected online, and data gathered from physical classroom settings by implementing alternative designs approach ( Saar et al. 2018 ; Suehiro et al. 2017 ).

Analysis methods

Multimodal analytics is the emergent trend that will complement readily available digital traces, with data captured from the physical world. Several articles in the literature have used multimodal approaches to analyse teaching processes in the physical world ( Prieto et al. 2016 ; Prieto et al. 2017 ; Prieto et al. 2018 ; Saar et al. 2017 ; Thomas 2018 ). In university settings, unobtrusive computer vision approaches to assess student attention from their facial features, and other behavioural signs have been applied ( Thomas 2018 ). Most of the studies that have ventured into multimodal analytics applied ML algorithms to their captured datasets to build models of the phenomena under investigation ( Prieto et al. 2016 ; Prieto et al. 2018 ). Apart from research areas that involve multimodal analytics, other areas of TA research have also applied in ML techniques such as teachers tool usage patterns ( Xu and Recker 2012 ), online e-books ( Suehiro et al. 2017 ), students written-notes ( Taniguchi et al. 2017 ). Table  3 outlines some of the ML techniques applied from previous literature in TA.

Visualisation methods

TA allows teachers to apply visual analytics and visualisation techniques to improve TPD. The most commonly used visualisation techniques in TA are statistical graphs such as line charts, bar charts, box plots, or scatter plots. Other visualisation techniques include SNA, spatial, timeline, static and real-time visualisations. An essential visualisation factor for TA is the number of users represented in a visualisation technique. Serving single or individual users allows the analyst to inspect the viewing behaviour of one participant. Visualising multiple or group users at the same time can allow one to find strategies of groups. However, these representations might suffer from visual clutter if too much data displays at the same time. Here, optimisation strategies, such as averaging or bundling of lines might be used, to achieve better results. Table  4 represents the visualisation techniques mostly used in TA.

Systematic review: critique

Student evaluation on teaching (set) data.

Although the literature has extensively reported various data sources used for TA, this study also draws attention to student feedback on teaching, as another form of data that originates from the classroom. The analytics of student feedback on teaching could support teacher reflection on teaching practice and add value to TA. Student feedback on teaching is also known as student ratings, or SET is a form of textual data. It can be described as a combination of both quantitative and qualitative data that express students opinions about particular areas of teaching performance. It has existed since the 1920s ( Marsh 1987 ; Remmers and Brandenburg 1927 ), and used as a form of teacher feedback. In addition to serving as a source of input for academic improvement ( Linse 2017 ), many universities also rely profoundly on SET for hiring, promoting and firing instructors ( Boring et al. 2016 ; Harland and Wald 2018 ).

Technological advancement has enabled institutions of Higher Education (HE) to administer course evaluations online, forgoing the traditional paper-and-pencil ( Adams and Umbach 2012 ). There has been much research around online teaching evaluations. Asare and Daniel ( 2017 ) investigated the factors influencing the rate at which students respond to online SET. While there is a verity of opinions as to the validity of SET as a measure of teaching performance, many teaching academics and administrators perceive that SET is still the primary measure that fills this gap ( Ducheva et al. 2013 ; Marlin Jr and Niss 1980 ). After all, who experiences teaching more directly than students? These evaluations generally consist of questions addressing the instructor’s teaching, the content and activities of the paper, and the students’ own learning experience, including assessment. However, it appears these schemes gather evaluation data and pass on the raw data to the instructors and administrators, stopping short of deriving value from the data to facilitate improvements in the instruction and the learning experiences. This measure is especially critical as some teachers might have the appropriate data literacy skills to interpret and use such data.

Further, there are countless debates over the validity of SET data ( Benton and Cashin 2014 ; MacNell et al. 2015 ). These debates have highlighted some shortcomings of student ratings of teaching in light of the quality of instruction rated ( Boring 2015 ; Braga et al. 2014 ). For Edström, what matters is how the individual teacher perceives an evaluation. It could be sufficient to undermine TPD, especially if the teachers think they are the subjects of audit ( Edström 2008 ). However, SET is today an integral part of the universities evaluation process ( Ducheva et al. 2013 ). Research has also shown that there is substantial room for utilising student ratings for improving teaching practice, including, improving the quality of instruction, learning outcomes, and teaching and learning experience ( Linse 2017 ; Subramanya 2014 ). This research aligns to the side of the argument that supports using SET for instructional improvements, to the enhancement of teaching experience.

Systematically, analytics of SET could provide valuable insights, which can lead to improving teaching performance. For instance, visualising SET can provide some way, a teacher can benchmark his performance over a while. Also, SET could provide evidence to claim for some level of data fusion in TA, as argued in the conceptualisation subsection of TA.

Transformational TA

The growing research into big data in education has led to renewed interests in the use of various forms of analytics ( Borgman et al. 2008 ; Butson and Daniel 2017 ; Choudhury et al. 2002 ). Analytics seeks to acquire insightful information from hidden patterns and relationships in data that ordinarily would not be visible by the natural eyes, except with the application of state-of-the-art models and methods. Big data analytics in HE provides lenses on students, teachers, administrators, programs, curriculum, procedures, and budgets ( Daniel 2015 ). Figure  3 illustrates the types of analytics that applies to TA to transform HE.

Types of analytics in higher education ( Daniel 2019 )

Descriptive Analytics Descriptive analytics aims to interpret historical data to understand better organisational changes that have occurred. They are used to answer the "What happened?" information regarding a regulatory process such as what are the failure rates in a particular program ( Olson and Lauhoff 2019 ). It applies simple statistical techniques such as mean, median, mode, standard deviation, variance, and frequency to model past behaviour ( Assunção et al. 2015 ; ur Rehman et al. 2016 ). Barmaki and Hughes ( 2015 ) carried out some descriptive analytics to know the mean view time, mean emotional activation, and area of interest analysis on the data generated from 27 stimulus images to investigate the notational, informational and emotional aspect of TA. Similarly, Michos and Hernández-Leo ( 2016 ) demonstrated how descriptive analytics could support teachers’ reflection and re-design their learning scenarios.

Diagnostic Analytics Diagnostic analytics is higher-level analytics that further diagnoses descriptive analytics ( Olson and Lauhoff 2019 ). They are used to answer the "Why it happened?". For example, a teacher may need to carry out diagnostic analytics to know why there is a high failure rate in a particular programme or why students rated a course so low for a specific year compared to the previous year. Diagnostic analytics uses some data mining techniques such as; data discovery, drill-down and correlations to further explore trends, patterns and behaviours ( Banerjee et al. 2013 ). Previous research has applied the repertory grid technique as a pedagogical method to support the teachers perform knowledge diagnostics of students about a specific topic of study ( Pantazos and Vatrapu 2016 ; Vatrapu et al. 2013 ).

Relational Analytics Relational analytics is the measure of relationships that exists between two or more variables. Correlation analysis is a typical example of relational analytics that measures the linear relationship between two variables ( Rayward-Smith 2007 ). For instance, Thomas ( 2018 ) applied correlation analysis to select the best features from the speaker and audience measurements. Some researchers have also referred to other forms of relational analytics, such as co-occurrence analysis to reveal students hidden abstract impressions from students written notes ( Taniguchi et al. 2017 ). Others have used relational analytics to differentiate critical formative assessment futures of an individual student to assist teachers in the understanding of the primary components that affect student performance ( Pantazos et al. 2013 ; Michos and Hernández Leo 2016 ). A few others have applied it to distinguish elements or term used to express similarities or differences as they relate to their contexts ( Vatrapu et al. 2013 ). Insights generated from this kind of analysis can be considered to help improve teaching in future lectures and also compare different teaching styles. Sequential pattern mining is also another type of relational analytics used to determine the relationship that exists between subsequent events ( Romero and Ventura 2010 ). It can be applied in multimodal analytics to cite the relationship between the physical aspect of the learning and teaching process such as the relationship between ambient factors and learning; or the investigation of robust multimodal indicators of learning, to help in teacher decision-making ( Prieto et al. 2017 ).

Predictive Analytics Predictive analytics aims to predict future outcomes based on historical and current data ( Gandomi and Haider 2015 ). Just as the name infers, predictive analytics attempts to predict future occurrences, patterns and trends under varying conditions ( Joseph and Johnson 2013 ). It makes use of different techniques such as regression analysis, forecasting, pattern matching, predictive modelling and multi-variant statistics ( Gandomi and Haider 2015 ; Waller and Fawcett 2013 ). In prediction, the goal is to predict students and teachers activities to generate information that can support decision-making by the teacher ( Chatti et al. 2013 ). Predictive analytics is used to answer the "What will happen". For instance, what are the interventions and preventive measures a teacher can take to minimise the failure rate? Herodotou et al. ( Herodotou et al. 2019 ) provided evidence on how predictive analytics can be used by teachers to support active learning. An extensive body of literature suggests that predictive analytics can help teachers improve teaching practice ( Barmaki and Hughes 2015 ; Prieto et al. 2016 ; Prieto et al. 2018 ; Suehiro et al. 2017 ) and also to identify group of students that might need extra support to reach desired learning outcomes ( Goggins et al. 2016 ; Thomas 2018 ).

Prescriptive Analytics Prescriptive analytics provides recommendations or can automate actions in a feedback loop that might modify, optimise or pre-empt outcomes ( Williamson 2016 ). It is used to answer the "How will it best happen?". For instance, how will teachers make the right interventions for students that have been perceived to be at risk to minimise the student dropout rate or what kinds of resources are needed to support students who might need them to succeed? It determines the optimal action that enhances the business processes by providing the cause-effect relationship and applying techniques such as; graph analysis, recommendation engine, heuristics, neural networks, machine learning and Markov process ( Bihani and Patil 2014 ; ur Rehman et al. 2016 ). For example, applying curriculum Knowledge graph and learning Path recommendation to support teaching and learners learning process ( Shen et al. 2018 ).

Actionable Analytics Actionable analytics refers to analytics that prompt action ( Gudivada et al. 2016 ; Gudivada et al. 2018 ; Winkler and Söllner 2018 ). Norris et al. ( 2008 ) used the term action analytics to describe "the emergence of a new generation of tools, solutions, and behaviours that are giving rise to more powerful and effective utilities through which colleges and universities can measure performance and provoke pervasive actions to improve it". The educational sector can leverage some of these innovative, new and cutting edge technologies and techniques such as Natural Language Processing (NLP) ( Sergis and Sampson 2016 ; Taniguchi et al. 2017 ), big data analytics ( Goggins et al. 2016 ) and deep learning ( Prieto et al. 2018 ) to support teacher in both the teaching and learning processes.

Institutional Transformation Data in themselves are not useful; they only become valuable if they can be used to generate insight. In other words, analytics can be applied to institutional data to optimise productivity and performance of the institutional operations, thereby providing value that can transform the institutional practices. In education, there are various purposes of analytics, ranging from those that provide institutions with an overview or deep-down microscopic view of individual students, faculty, curriculum, programs, operations and budgets, to those capable of predicting future trends. Unveiling the value of TA empowers the teachers to identify issues and transform difficulties into opportunities. These opportunities can be employed to optimises the institutional processes, enhance learner experiences and improve teaching performance. TA and LA both play a vital role in effectively reforming and transforming the educational sector to catch up with the fast pace at which data generates. For example, with the extensive use of online and blended learning platforms, the application of analytics will enable institutional stakeholders at all levels to gain new insights into educational data. Today, the HE sector is at crossroads, where there is a need for synergies in learning research and data analytics to transform the way teaching and learning are fundamentally carried out.

The link between TA, LA and LD

Primarily, TA aims to link the centrepiece of LA and remodel them to address teaching challenges. More specifically, TA argues that connecting and analysing insights generated from LA methods and tools with those generated from in-class methods and tools, through TA tools could support teacher reflection and improve TPD based on evidence. Hence, this concept is presented further in the next subsection.

Conceptual framework of TA

Based on the different perceptions of TA described in previous reviews, this study proposes a conceptual framework for TA to model the complex interaction existing around TA. Three nodes (LA, TA and LD) are interconnected to each other forming a triadic network with the teacher at the centre, performing value-added interactions to make informed based decisions. Each part of this interconnection forms a triangle, totalling three triangles (A, B and C) (see Fig.  4 ).

Conceptualisation of TA. Triadic TA Conceptual Framework

The proposed framework is not bound to any particular implementation of learning or design technology. Instead, the point is to describe the elements of analytics and data sources that are key for each domain to guide the use of analytical methods, tools and technology to support the multiple dimensions of learning design successfully.

This triad illustrates the interaction occurring between the teacher, the LA and the LD, to inform TPD. Hernández-Leo et al. ( 2019 ) argued that LD could contribute to structuring and orchestrating the design intent with learners digital trace patterns, advancing the knowledge and interpretation of LA. LA tailored to fit the design intent could be considered by teachers as contributing to the enhancement of the LD in subsequent design interactions. For example, LA could be an information tool to inform the tutors or designers of pedagogical decision making ( Persico and Pozzi 2015 ). Hence, a teacher may want to utilise LA to make just-in-time pedagogical decisions, such as grouping students based on their performance.

Similarly, a teacher may want to investigate if the estimated time taken for students to carry out learning tasks is reasonable or whether adjustments need to be made to the course design ( Hernández-Leo et al. 2019 ; Pozzi and Persico 2013 ). This domain can also provide teachers with analytics regarding the challenges and difficulties students face in the problem-solving phase while performing a task. In return, they give the teacher information in the form of TAD summarising the various challenges students encountered with that activity. They may also provide solutions on how to address them. For example, an early alert system that instantiates a dashboard for instructors using some metrics calculations such as login counts and page views ( Thille and Zimmaro 2017 ). The data sources in the LA node can improve teachers’ awareness, which could also lead to the improvement of LD and help to distinguish design elements that could modify future designs. Data collection in this domain is mostly automatic through virtual learning environments (e.g., LMS, MOOCs). Other forms of data collection may include social media platforms (e.g., Facebook, Tweeter), wearable sensors (e.g., eye-trackers, EEG), software tools that support and collect data related to specific student activities and attendance ( Bakharia et al. 2016 ; Bos and Brand-Gruwel 2016 ).

This triangle represents the relationship between the teacher, the LD and TA. While experiencing LD, TA endeavours to handle continues teachers’ engagement, progression, achievement and learners satisfaction ( Bakharia et al. 2016 ; Sergis and Sampson 2017 ). For example, exploring the impact of video shot on instructor performance and student learning. Using MOOC AB testing, teachers could experiment whether a difference in video production setting would have any impact on the instructors acting performance, or whether any changes in format and instructors performance will result in detectable differences in student viewing behaviour ( Chen et al. 2016 ).

Further, data sources in TA could assist teacher reflection on the impacts of their LD. Data collection could also be automatic by the use of wearable sensors on the teachers while performing teaching activities, also known as in-class analytics. Several institutions now record video contents of their face-to-face classes. Some others even go a step further by collecting their physiological data. These datasets, as mentioned earlier, have a way of exemplifying and illustrating things that ordinarily, a book of pedagogy cannot convey, in providing systematic feedback for the teachers. It involves capturing data during a traditional in-class, face-to-face teacher-centric instruction or teacher-student interaction (where students learn by directly or indirectly interacting with instructors in a lab or lecture hall) and analysing data to identify areas of possible improvements. The kind of data usually captured in this setting are audio, video, body movement, brain activity, cortex activity, to mention just a few. For example, a teacher can perform diagnostic analysis on class recorded videos to expose what is intrinsic during his lecture. This kind of diagnostic analysis could help teachers understand more about their teaching and discover areas of further improvement. SET is another form of data about the teachers; they are collected via the institutional application platforms ( Hernández-Leo et al. 2019 ) and can be visualised to improve teaching performance..

Analytics that happens in the LD involves the visualisation of teaching design to facilitate teacher reflection on the lesson plan, visualisation of the extent to which the lesson plan aligns with the educational objectives, and finally, validation of the lesson plan to highlight potential inconsistencies in the teaching design. For example, a teacher can visualise the number of assessment activities of the lesson plan or the various types of educational resources used in the lesson plan, to know if they are still valid or obsolete. Similarly, a teacher could analyse the time allocated for each lesson activity, to find out if the time allocated for each activity is good enough, or visualise the level of inconsistencies of time misappropriations and imbalances between the overall lesson plan and the individual lesson activities.

This area presents the communication between the teacher, the LA and the TA. Chinchu Thomas ( 2018 ) explored the correlation between student ratings on teaching and student physiological data. Similarly, Schmidlin ( 2015 ) established how to analyse and cross-reference data without decrypting the data sources. Hence, we argue that SET could be linked with LA such as student digital traces from LMS ( Stier et al. 2019 ) and other forms of data (such as attendance data), without compromising privacy. This claim for data fusion could support the teachers to make informed-decisions in new ways. For example, analytics performed on linked datasets could quickly reveal those student opinions that may not count at the end of the semester courses.

Visualisations that could quickly realise students with low participation rates and link it to their opinions, without revealing any identity. Additionally, teachers may be interested in comparing the view of students with low participation rate with those of high participation rate. This kind of information may lead teachers towards making explicit judgements with evidence. A tutor may choose to disregard the opinions of those students that participated less than 20 per cent in-class activities and assignments, as well as had a low attendance rate. Hence, narrowing concentration more on the opinions of students that participated in improving teaching practice.

However, considering ethical concerns, data fusion at the individual level still requires explicit and informed consent from the students whose data are collected ( Menchen-Trevino 2016 ). Other issues such as privacy concerns, data fusion can be problematic as this usually requires that the teachers know student identities. However, from a programmatic perspective, extra measures can be put in place to address this concern. Algorithms can be interfaced to mask student identities to some other unique identities to make them anonymous but linked ( Schmidlin et al. 2015 ) to provide a richer set of data for the teacher to make informed decisions.

Teachers can get a better picture towards improving the context in which learning happens, only if they can be informed about both how they teach and how students learn. Hence, this framework aims to continually provide teachers with interesting information from intelligent feedback based on data generated from users and learning context to improve their learning design and teaching outcome continuously.

Teaching Outcome Model (TOM)

Design-based research advances instructional design work, theory, and implementation as iterative, participatory, and located rather than processes "owned and operated" by designers of instructions ( Wang and Hannafin 2005 ). TOM is an iterative process that follows a design-based research approach to guide teachers, researchers, faculty and administrators on how to utilise data to improve the quality of teaching and learning outcome. This model enables teachers to investigate and evaluate their work using data. Consequently, improving the teacher use of data to inform teaching practice. To build more awareness with regards to teaching data, TOM models TA through iterative cycles of data collection, data analysis, data visualisation and action stages which are interdependent of each other (see Fig.  5 ). Design-based research, as a pragmatic methodology, can guide TOM while generating insights that can support teacher reflections on teaching and student learning. Conversely, TOM ensures that design-based research methodologies can be operational and systemised. Following the various stages outlined in the model, teachers can regularly identify, match and adjust teaching practice, and learning design to all the learners need.

Teaching Outcome Model. TA Life cycle

In the data collection stage, a constant stream of data accumulates from the digital traces relating to teaching daily activities and engagements, including structured and unstructured data, visual and non-visual data, historical and real-time data. It is also important to note that the rate at which diverse data accumulates in our educational system will keep growing. According to Voithofer and Golan ( 2018 ), there are several ways to mine teaching and learning data without professional knowledge that is beyond the necessary teacher training experience in data literacy, administering learning design and class orchestration. Subscribing to this school of thought, adopting Big data infrastructure in our institutions will guarantee easy access to data by the various stakeholders, this will also mitigate the bottleneck of disparate data points existing in our educational sector. Therefore, enabling educators to focus more attention on instruction, setting up interactive class activities, and participating more on discussions that will create more data for evidence-based decision making. Also, the misuse of data is a broad primary concern ( Roberts et al. 2017 ). One critical matter is identifying the types of data that can be collected, analysed and visualized; to ensure that the right people have access to the data for the right purpose. As such, implementing data governance policies around institutional data such as; ’open definition of purpose, scope and boundaries, even if that is broad and in some respects, open-ended’ is critical ( Kay et al. 2012, p 6 ). This sort of measure will introduce clarity and address issues around who controls what data as well as security and privacy issues around data.

Analysis stage

This step involves the different ways of working with data to ensure data quality. Professionals such as data scientists, programmers, engineers and researchers need to work together with the teachers at this level. They can apply data mining techniques, statistical methods, complex algorithms, and AI techniques (such as NLP, AI, ML, deep learning) to adequately transform data into the useful analytical process. Analytics in the education space presents in diverse forms including, descriptive, diagnostic, predictive and prescriptive. These different forms of analytics can be utilised to offer a high-level view or fine-grained view of individual learners, teacher, faculty and their various activities, engagements and behaviours. Unravelling the value of data analytics empowers teachers and researchers to identify problems and transform challenges into opportunities that can be utilised to support teacher reflection and enrich teacher data-literacy experiences. For example, teachers can apply NLP on text data to gather topics from discussion posts, contributions participants have made within collaborative projects and their sentiments.

Furthermore, ML techniques could be combined with TA to enhance teaching outcome. For instance, chatbots could support the teacher by acting as a teacher assistant in large classes. An essential consideration in analytics, however, is that data can be easily de-identified ( Roberts et al. 2017 ; Cumbley and Church 2013 ), especially when data sets increase in size and scope and are combined to generate big data. To resolve these concerns, a particular university introduced a two-stage method of data de-identification coupled with data governance to restrict data access ( De Freitas et al. 2015 ).

Visualisation stage

This stage ensures data presentation in useful and meaningful ways to teachers. Empowering teachers with interactive visual interfaces and dashboards that facilitate teacher cognition and promote reflection about pre-processed and fine-grained teaching and learning activities. Through TAD, can project real-time and historical information from different data sources that might not be necessarily interoperable, and results summarised ( Moore 2018 ). However, visualisation is "what you see is what you get"; meaning that information presentation method may affect its interpretation, and consequently, may influence decision-making. Hence, it is necessary to address issues around visualisations in diverse forms such as; visual analytics and exploratory data analysis to create room for visual interactivity, exploratory visualisation to discover trends, patterns, relationships and behaviours. For example, a teacher can use a TAD to monitor student engagement. When the student engagement is poor, it may prompt the teacher to take necessary actions such as; changing teaching material and making it more interactive. Additionally, there are also questions around privacy, such as who has access to visualisations relevant to an instructor, such as other faculty members participating in the course, directly or indirectly, administrators, researchers, potential employees of other institutions.

Action stage

At this stage, informed-decision leads to action and actions unavoidably reshape our environment; subsequently, regenerate new data. Additionally, there is a to create tools that will be useful to the teacher to understand and make meaning of data quickly. Actions taken by teachers can be used to improve the course design and assessment (value-added formative assessment). In any case, predictive analytics prompts an epistemological question; how should we ensure effective action by the teacher based on flawed predictions such that the system does not collapse?

Discussion and conclusion

This article presents the result of a systematic literature review aimed at describing the conception, and synthesis of the current research on the notion of TA, to provide insight into how TA can be used to improve the quality of teaching. The first part of the article described what is meant by TA to consolidate the divergent discourse on TA. The review showed that TA applies to analytics on teaching activities as well as methods of improving teachers’ awareness on students’ activities, including supporting the teachers to understand student learning behaviours to provide adequate feedback to teachers. In essence, the primary goal of TA is to improve teaching performance. The literature also revealed the several tools and methods are available for extracting digital traces associated with teaching in addition to traditional student evaluation tools. However, one of the main challenges recognised was the cost associated with some devices used to capture in-class activities, and ML techniques have been proposed to minimise this challenge.

The literature has also recognised teacher inquiry as a promising area of research in TA and came to a consensus that methods, like multimodal analytics and SNA, could help promote teacher inquiry and teacher reflection. Visualisations and visual analytics techniques are very significant in TA and also encourage teacher inquiry. The use of visualisation dashboards and TAD are essential tools that the modern-day teachers require to carry out a continuous and efficient reflection on teaching practice.

The emphasis of the synthesis of TA was clearly on data collection, analysis and visualisation, as illustrated in Fig.  6 . In the literature, the various kinds of data collected and used to improve teaching practice, include:

Digital trace data; "records of activity (trace data) undertaken through an online information system (thus, digital)" [119]. They incorporate various activities generated from custom applications and learning environments that leave digital footprints.

Image data are photographic or trace objects that represent the underlying pixel data of an area of an image element.

Physiological data are body measurement based on body-mounted sensors ( Lazar et al. 2017 ), used to extract data from teachers while performing classroom teaching activities.

Audio-video stream data or recorded lecturer data with captured physical teaching activities and students learning activities. Hence, attainable with mounted cameras, computer or mobile cameras connected to applications like Zoom and Skype, eye tracks with recording capabilities and digital cameras connected to learning environments such as Eco365.

Social data are data with online social activities, including utilising the repertory grid technique to collect students’ assessment data from social media sites.

Text data, including quantitative and qualitative data, data generated from text documents such as discussion forums, students essay or articles, emails and chat messages.

Dimensions of TA. Illustration of TA based on the literature

Analysis in this context refers to the application of Educational Data Mining (EDM) and deep learning techniques mostly used to process data. EDM approaches is a complicated process that requires an interweaving of various specialised knowledge and ML algorithms, especially to improve teaching and learning ( Chen 2019 ). NLP and classification are the two main EDM techniques applied in TA. However, the review also recognised the use of other methods such as clustering and deep learning techniques, to support teachers.

As commonly said, a picture is worth more than a thousand words; visualisation can effectively communicate and reveal structures, patterns and trends in variables and their interconnections. Research in TA has applied several visualisation techniques including Network, Timeline, Spatial, Table and Statistical Graphs. For instance, SNA is a form of visual analytics that is used to support teachers to determine how different groups interact and engage with course resources. Identifying differences in interaction patterns for different groups of students may result in different learning outcomes, such as, how access patterns of successful groups of students differ from that of unsuccessful students. Applying visualisation techniques can support teachers in areas such as advising underperforming students about effective ways to approach study. Visualisation can enable teachers to identify groups of students that might need assistance and discover new and efficient means of using collaborative systems to achieve group work that can be taught explicitly to students.

However, while acknowledging the incomplete nature of data and complexities associated with data collection, analysis and use, teachers should take caution to avoid bais. Data collected in one context may not be directly applicable to another or have both benefits and cost for individuals or groups from which data was harvested. Therefore, key stakeholders, including teachers, course directors, unit coordinators and researchers must pay proper attention to predictive models and algorithms and take extra care to ensure that the contexts of data analysed are carefully considered. There are also privacy concerns, such as who has access to view analytics relating to a teacher, including other faculty members both directly or indirectly involved in the course, administrators, researchers, future employees of other institutions. It will be useful for institutions to have clear guidelines as to who has access to what and who views what. Other issues around data include how long should data remain accessible ( Siemens 2013 ), with big data technology and infrastructure, data should be kept for as long as it can exist. Pardo and Siemens ( 2014 ) acknowledged that the use of analytics in higher education research has no clear interpretation of the right to privacy. They seem opposed to the need for absolute privacy, on the basis that the use of historical data enhances research with potential rewards for the future of teaching professional development and student outcome.

The review provided in the current article highlighted the significant limitations in the existing literature on teaching analytics. The TAD is proposed to guide teachers, developers, and researchers to understand and optimise teaching and the learning environments. The critical aspect of this review is establishing the link between LA, TA and LD and its value in informing teachers’ inquiry process. Also, the review describes the relationship between LA, TA and LD. Finally, the article proposes TOM, which draws from a research-based approach to guide teachers on how to utilise data to improve teaching. The outcome of this model is a TAD that provides actionable insights for teacher reflection and informed decision-making. Therefore, showing the value that TA brings to pedagogic interventions and teacher reflection.

Theoretical implications

The analysis of data collected from the interaction of teachers with technology and students is a promising approach for advancing our understanding of the teaching process and how it can be supported. Teachers can use data obtained from their teaching to reflect on their pedagogical design and optimise the learning environment to meet students’ diverse needs and expectations.

Teacher-centric learning design can improve the utility of new technologies and subsequent acceptance of the use of these technologies to improve the quality of teaching and enhance students learning experience. TAD is one class of tools that can be designed in such a way that will improve teaching practice.

Research on learning analytics has revealed useful insights about students’ learning and the context in which they learn. While the ability to track, harvest and analyse various forms of learning analytics can reveal useful insights about learners’ engagement with learning environments, our review suggests that there is limited focus on analytics relating to the teacher, their teaching approaches and activities. Also, there has been increasing advances in the design of learner and teaching dashboards. However, many teachers still struggle with understanding and interpreting dashboards partly because they lack data literacy skills, and mostly because most the design of many of the tools does not include teachers as partners.

Although, TAD enable teachers to inspect, and understand the processes and progress relating to their teaching, the current implementations of TAD in general, does not adequately provide teachers with the details they need or want in a readily usable format. Educational technology developers can utilise our proposed model to design better tools for improving teaching practice. For example, a TAD can be designed to perform text analytics on students qualitative comments about a course taught, and results presented to the teacher in the form of themes, sentiments and classification; such that it will support the instructor’s needs and preferences for insight generation and reflection.

Teachers monitor, observe and track both teaching and learning activities to make appropriate decisions. Moreover, it is also important to note that visualisations can be misrepresented, misinterpreted or misused by the viewer [122]. Hence, perception and cognition remain a significant challenge in TAD. Consequently, it becomes necessary to design and write algorithms that extract information visualisation, in such a way that allows adequate understanding by teachers. It is also crucial for dashboards to integrate multiple sources such as combining both the learning and teaching activities into a TAD, to create room for teachers to comprehend, reflect on and act upon the presented information quickly.

Also, the current state of technology shows little progress in taking TA, raising concerns about the accurate validity and scalability of innovations such as predictive analytics and TAD. Furthermore, the ethical issues of data use are not considered sufficient to establish institutional policies which incorporate TA as part of quality education models.

Finally, consideration of the framework’s three layers as a whole raises new questions and opportunities. For example, linking educational performance and satisfaction to specific learning design involves consideration of elements of all three layers. This review has shown that TA is a new and essential area of analytics in education. The study also suggests that the conceptualisation of teaching analytics is still at its infancy. However, the practical and successful use of teaching analytics is highly dependent on the development of conceptual and theoretical foundations into consideration.

Implications for practice

This review has uncovered the value of TA and its role in fostering data literacy skills in teachers to support evidence-based teaching. The purpose of TOM is to guide the development of teaching dashboard, and for researchers to develop strategies that help meaningful ways in which data can be presented to teachers. Teacher dashboards can empower the teachers with tools that create new opportunities to make data-informed strategic decisions, utilising the power of analytics and visualisation techniques. Consequently, increasing the efficiency and effectiveness of the institution, including, improving teaching practice, curriculum development and improvement, active learning engagement and improved students’ success. TOM also presents a platform for teaching academics who may have the best understanding of their course contexts, to provide a significant contribution to a culture of data-informed teaching practice within an institution.

The responsibility for managing the systems that provide the analytics usually falls within the control and supervision of the institution’s information technology (IT) department, and often, they have little to no knowledge of their pedagogical applications to teaching and learning. Likewise, academics and their fields of learning support are often deprived of IT skills and have little to no professional understanding of how software systems work. TOM provides opportunities for the teachers to be involved in the design of TA by providing significant interaction and collaboration between the IT and the other sectors that interpret and act upon the information flow.

Additionally, institutions need to provide teaching staff with the necessary training that fosters the development of data literacy skills, and in the use of data and analytical or visualisation dashboards to monitor their teaching practice. Based on some of the challenges identified in the present review, it is imperative institutions ensure that data is collected transparently, with the awareness of all the stakeholders involved, and informed consent of individuals where appropriate. With the advancements in computing technology, data collection, analysis and use have significantly increased, large amounts of data can be continually pulled from different sources and processed at fast speeds. Big data offers institutions the opportunity to implement big data infrastructures and utilise the full potential of data analytics and visualisation. However, institutions also need to consider implementing a data governance framework to guide the implementation and practice of analytics.

The conceptual framework of TA was established to demonstrate the relationship between LA, TA and LD, which can be useful knowledge to various institutional stakeholders, including the learners, teachers, researchers and administrators. However, there are also issues around data ownership, intellectual property rights, and licensing for data re-use (the students, the instructor, the researcher or the institution). For instance, the same data sources can be shared amongst the various stakeholders, but with different level of access, as such data sharing agreement would be needed to guide sharability without infringing on rights, violating privacy or disadvantaging individuals. The implementation of data sharing agreement would require the building of institutional, group as well as individual trust, which would include guidelines on sharing data within the institution and between third parties, such as external organisations and other institutions. In general, stricter data management policies that guide data collection, analysis and use is essential for every institution.

Limitations and future research

Teaching analytics is an emergent phenomenon in the learning analytics and data science literature, with a limited body of published work in the area, as such conclusions drawn from the review are limited to the databases interrogated and articles reviewed. Further, findings in the review are likely to be influenced by our interpretation of the literature and untestable assumptions. For example, linking LA, TA and LD and their underlying assumptions is not grounded in empirical work. The review serves as an advocacy for teacher data literacy and the ability to work with various forms of data. However, working with a single data point may not be publicly accessible to teachers.

Moreover, the combination of analytics on the several data points may lead to some level of identification, and this would require navigating issues around access, protecting privacy, and obtaining appropriate consents. Therefore, it is almost impossible for individual teachers to comprehend not only the scope of data collected, analysed and used but also the consequences of the different layers of collection, analysis and use. Consequently, making it challenging for teachers to make use of the full potentials of data to make informed choices in learning design. No matter how straightforward or transparent institutional policies around data are, the sheer complexity of the collection, analysis and use has made it impossible, posing a fundamental issue for the stakeholders trying to use analytics to enhance teaching practice and learning outcome across an institution.

In future research, we hope to carry out more extensive empirical research on how TOM could be applied to address issues with regards to ethical and privacy concerns about the utilization of TA. We are currently exploring how teaching analytics dashboards can be used to support teacher data literacy and use analytics to improve teaching practice and learning outcome.

Availability of data and materials

Not applicable.

Abbreviations

Academic analytics

Artificial intelligence

Educational data mining

Higher education

Interactive whiteboard

• Learning analytics

Learning design

Learning management system

Machine learning

Massive open online courses

Natural language processing

Open learners model

Student evaluation of teaching

Social network analysis

• Teaching analytics

Teaching analytics dashboard

Term frequency inverse document frequency

• Teaching and learning analytics
• Teaching outcome model

Technology, pedagogy, and content knowledge

Teacher professional development

Adams, M.J., & Umbach, P.D. (2012). Nonresponse and online student evaluations of teaching: Understanding the influence of salience, fatigue, and academic environments. Research in Higher Education , 53 (5), 576–591.

Article   Google Scholar  

Anderson, T. (2003). Getting the mix right again: An updated and theoretical rationale for interaction. The International Review of Research in Open Distributed Learning , 4 (2).

Asare, S., & Daniel, B.K. (2017). Factors influencing response rates in online student evaluation systems: A systematic review approach. In E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education . Association for the Advancement of Computing in Education (AACE), (pp. 537–541).

Assunção, M.D., Calheiros, R.N., Bianchi, S., Netto, M.A., Buyya, R. (2015). Big data computing and clouds: Trends and future directions. Journal of Parallel and Distributed Computing , 79 , 3–15.

Bakharia, A., Corrin, L., De Barba, P., Kennedy, G., Gašević, D., Mulder, R., Williams, D., Dawson, S., Lockyer, L. (2016). A conceptual framework linking learning design with learning analytics. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge . ACM, (pp. 329–338).

Banerjee, A., Bandyopadhyay, T., Acharya, P. (2013). Data analytics: Hyped up aspirations or true potential? Vikalpa , 38 (4), 1–12.

Barmaki, R., & Hughes, C.E. (2015). Providing real-time feedback for student teachers in a virtual rehearsal environment. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction . ACM, (pp. 531–537).

Beer, C., Jones, D., Clark, K. (2009). The indicators project identifying effective learning: Adoption, activity, grades and external factors. In Ascilite . Citeseer.

Benton, S.L., & Cashin, W.E. (2014). Student ratings of instruction in college and university courses , (pp. 279–326): Springer.

Bihani, P., & Patil, S. (2014). A comparative study of data analysis techniques. International Journal of Emerging Trends & Technology in Computer Science , 3 (2), 95–101.

Google Scholar  

Borgman, C.L., Abelson, H., Dirks, L., Johnson, R., Koedinger, K.R., Linn, M.C., Lynch, C.A., Oblinger, D.G., Pea, R.D., Salen, K. (2008). Fostering learning in the networked world: The cyberlearning opportunity and challenge. a 21st century agenda for the national science foundation. https://doi.org/10.1037/e532532011-001 .

Boring, A. (2015). Gender biases in student evaluation of teachers . Paris. https://doi.org/10.1016/j.jpubeco.2016.11.006 .

Boring, A., Ottoboni, K., Stark, P.B. (2016). Student evaluations of teaching are not only unreliable, they are significantly biased against female instructors. Impact of Social Sciences Blog . The London School of Economics and Political Science.

Bos, N., & Brand-Gruwel, S. (2016). Student differences in regulation strategies and their use of learning resources: implications for educational design. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge . ACM, (pp. 344–353).

Braga, M., Paccagnella, M., Pellizzari, M. (2014). Evaluating students’ evaluations of professors. Economics of Education Review , 41 , 71–88.

Butson, R., & Daniel, B. (2017). The Rise of Big Data and Analytics in Higher Education , (pp. 127–140): Auerbach Publications.

Charleer, S., Klerkx, J., Odriozola, S., Luis, J., Duval, E. (2013). Improving awareness and reflection through collaborative, interctive visualizations of badges. In ARTEL13: Proceedings of the 3rd Workshop on Awareness and Reflection in Technology-enhanced Learning, vol. 1103 . CEUR Workshop Proceedings, (pp. 69–81).

Chatti, M.A., Dyckhoff, A.L., Schroeder, U., Thüs, H. (2013). A reference model for learning analytics. 5-6 , 4 , 318–331.

Chen, L.-L. (2019). Enhancing teaching with effective data mining protocols. Journal of Educational Technology Systems , 47 (4), 500–512.

Chen, Z., Chudzicki, C., Palumbo, D., Alexandron, G., Choi, Y.-J., Zhou, Q., Pritchard, D.E. (2016). Researching for better instructional methods using ab experiments in moocs: results and challenges. Research and Practice in Technology Enhanced Learning , 11 (1), 9.

Choudhury, S., Hobbs, B., Lorie, M., Flores, N. (2002). A framework for evaluating digital library services. D-Lib magazine , 8 (7/8), 1082–9873.

Chounta, I.-A., McLaren, B.M., Albacete, P.L., Jordan, P.W., Katz, S. (2016). Analysis of human-to-human tutorial dialogues: Insights for teaching analytics. In IWTA@ EC-TEL , (pp. 9–17).

Cumbley, R., & Church, P. (2013). Is “big data” creepy? Computer Law & Security Review , 29 (5), 601–609.

Dana, N.F., & Yendol-Hoppey, D. (2019). The Reflective Educator’s Guide to Classroom Research: Learning to Teach and Teaching to Learn Through Practitioner Inquiry : Corwin.

Daniel, B. (2015). Big data and analytics in higher education: Opportunities and challenges. British Journal of Educational Technology , 46 (5), 904–920. https://doi.org/10.1111/bjet.12230 .

Daniel, B., & Harland, T. (2017). Higher Education Research Methodology: A Step-by-Step Guide to the Research Process . Routledge London. https://doi.org/10.4324/9781315149783 .

Daniel, B.K. (2019). Artificial reality: The practice of analytics and big data in educational research. In: Pedersen, J.S., & Wilkinson, A. (Eds.) In Big data: Promise, application and pitfalls . https://doi.org/10.4337/9781788112352.00018 . Edward Elgar, Cheltenham, (pp. 287–300).

Chapter   Google Scholar  

De Freitas, S., Gibson, D., Du Plessis, C., Halloran, P., Williams, E., Ambrose, M., Dunwell, I., Arnab, S. (2015). Foundations of dynamic learning analytics: Using university student data to increase retention. British Journal of Educational Technology , 46 (6), 1175–1188.

Dix, A.J., & Leavesley, J. (2015). Learning analytics for the academic: An action perspective. J. UCS , 21 (1), 48–65.

Ducheva, Z., Pehlivanova, M., Dineva, S. (2013). Possibilities for students to evaluate and improve electronic courses. In The 8th International Conferemnce on Virtual Learning ICVL .

Edström, K. (2008). Doing course evaluation as if learning matters most. Higher Education Research & Development , 27 (2), 95–106.

Elias, T. (2011). Learning analytics. Learning , 1–22.

Ferguson, R. (2012). Learning analytics: drivers, developments and challenges. International Journal of Technology Enhanced Learning , 4 (5/6), 304–317.

Fischer, F., Wild, F., Sutherland, R., Zirn, L. (2014). Grand Challenge Problems from the Alpine Rendez-Vous , (pp. 3–71): Springer.

Flanders, N.A. (1970). Analyzing Teacher Behavior . Addison-Wesley P.C.

Flavin, M. (2017). Disruptive Technology Enhanced Learning: The Use and Misuse of Digital Technologies in Higher Education : Springer. https://doi.org/10.1057/978-1-137-57284-4 .

Gandomi, A., & Haider, M. (2015). Beyond the hype: Big data concepts, methods, and analytics. International Journal of Information Management , 35 (2), 137–144.

Gauthier, G. (2013). Using teaching analytics to inform assessment practices in technology mediated problem solving tasks. In IWTA@ LAK .

Ginon, B., Johnson, M.D., Turker, A., Kickmeier-Rust, M. (2016). Helping Teachers to Help Students by Using an Open Learner Model. https://doi.org/10.1007/978-3-319-45153-4_69 .

Goggins, S.P. (2012). Group informatics: A multi-domain perspective on the development of teaching analytics. In Proceedings of the TaPTA Workshop at EC-TEL .

Goggins, S.P., Galyen, K., Petakovic, E., Laffey, J.M. (2016). Connecting performance to social structure and pedagogy as a pathway to scaling learning analytics in moocs: an exploratory study. Journal of Computer Assisted Learning , 32 (3), 244–266.

Gorham, J. (1988). The relationship between verbal teacher immediacy behaviors and student learning. Communication Education , 37 (1), 40–53.

Griffiths, D. (2017). The use of models in learning design and learning analytics. Interaction Design and Architecture(s) Journal , 33 , 113–133.

Gudivada, V.N., Irfan, M., Fathi, E., Rao, D. (2016). Cognitive analytics: Going beyond big data analytics and machine learning (Vol. 35, pp. 169–205).

Gudivada, V.N., Rao, D.L., Ding, J. (2018). 2. Evolution and Facets of Data Analytics for Educational Data Mining and Learning Analytics , (pp. 16–42). New York. https://doi.org/10.4324/9780203728703-3 .

Harland, T., & Wald, N. (2018). Vanilla teaching as a rational choice: the impact of research and compliance on teacher development. Teaching in Higher Education , 23 (4), 419–434.

Hernández-Leo, D., Martinez-Maldonado, R., Pardo, A., Muñoz-Cristóbal, J.A., Rodríguez-Triana, M.J. (2019). Analytics for learning design: A layered framework and tools. British Journal of Educational Technology , 50 (1), 139–152.

Herodotou, C., Hlosta, M., Boroowa, A., Rienties, B., Zdrahal, Z., Mangafa, C. (2019). Empowering online teachers through predictive learning analytics. British Journal of Educational Technology . https://doi.org/10.1111/bjet.12853 .

Huang, C.-W. (2001). Educlick: A computer-supported formative evaluation system with wireless devices in ordinary classroom. In Proceedings of Int. Conference on Computers in Education, 2010 , (pp. 1462–1469).

Joseph, R.C., & Johnson, N.A. (2013). Big data and transformational government. IT Professional , 15 (6), 43–48.

Kaser, L., & Halbert, J. (2014). Creating and sustaining inquiry spaces for teacher learning and system transformation. European Journal of Education , 49 (2), 206–217.

Kay, D., Korn, N., Oppenheim, C. (2012). Legal, risk and ethical aspects of analytics in higher education. Analytics Series . JISC Cetis (Centre for educational technology and interoperability standards).

KU, O., LIANG, J.-K., CHANG, S.-B., WU, M. (2018). Sokrates teaching analytics system (stas): An automatic teaching behavior analysis system for facilitating teacher professional development. In Proceedings of the 26th International Conference on Computers in Education. Philippines: Asia-Pacific Society for Computers in Education .

Laney, D. (2001). 3d data management: Controlling data volume, velocity and variety. META Group Research Note. META group research note , 6 (70), 1.

Lazar, J., Feng, J.H., Hochheiser, H. (2017). Research Methods in Human-computer Interaction : Morgan Kaufmann.

Leitner, P., Khalil, M., Ebner, M. (2017). Learning analytics in higher education—a literature review , (pp. 1–23): Springer.

Libbrecht, P., Kortenkamp, U., Rebholz, S., Müller, W. (2013). Tales of a companion teacher analytics. In IWTA@ LAK .

Linse, A.R. (2017). Interpreting and using student ratings data: Guidance for faculty serving as administrators and on evaluation committees. Studies in Educational Evaluation , 54 , 94–106.

Loughran, J.J. (2002). Effective reflective practice: In search of meaning in learning about teaching. Journal of teacher education , 53 (1), 33–43.

Macfadyen, L.P., & Dawson, S. (2012). Numbers are not enough. why e-learning analytics failed to inform an institutional strategic plan. Journal of Educational Technology Society , 15 (3).

MacNell, L., Driscoll, A., Hunt, A.N. (2015). What’s in a name: Exposing gender bias in student ratings of teaching. Innovative Higher Education , 40 (4), 291–303.

Mansfield, J. (2019). Pedagogical Equilibrium: The Development of Teachers’ Professional Knowledge : Routledge.

Marlin Jr, J.W., & Niss, J.F. (1980). End-of-course evaluations as indicators of student learning and instructor effectiveness. The Journal of Economic Education , 11 (2), 16–27.

Marsh, H.W. (1987). Students’ evaluations of university teaching: Research findings, methodological issues, and directions for future research. International Journal of Educational Research , 11 (3), 253–388.

McKenney, S., & Mor, Y. (2015). Supporting teachers in data–informed educational design. British journal of educational technology , 46 (2), 265–279.

Menchen-Trevino, E. (2016). Web historian: Enabling multi-method and independent research with real-world web browsing history data. In IConference 2016 Proceedings . https://doi.org/10.9776/16611 .

Michos, K., & Hernández Leo, D. (2016). Towards understanding the potential of teaching analytics within educational communities. In: Vatrapu, R.G.B.B.S., & Kickmeier-Rust, M. (Eds.) In Vatrapu R, Kickmeier-Rust M, Ginon B, Bull S. IWTA 2016 International Workshop on Teaching Analytics. Proceedings of the Fourth International Workshop on Teaching Analytics, in Conjunction with EC-TEL 2016; 2016 Sept 16; Lyon, France.[place Unknown]: CEUR Workshop Proceedings , (pp. 1–8).

Moore, B.L. (2018). 6. The Role of Data Analytics in Education Possibilities and Limitations, 1st edn.https://doi.org/10.4324/9780203728703-8.

Mor, Y., Ferguson, R., Wasson, B. (2015). Learning design, teacher inquiry into student learning and learning analytics: A call for action. British Journal of Educational Technology , 46 (2), 221–229.

Müller, W., Rebholz, S., Libbrecht, P. (2016). Automatic inspection of e-portfolios for improving formative and summative assessment. In International Symposium on Emerging Technologies for Education . Springer, (pp. 480–489).

Norris, D., Baer, L., Leonard, J., Pugliese, L., Lefrere, P. (2008). Action analytics: Measuring and improving performance that matters in higher education. EDUCAUSE Review , 43 (1), 42.

Olson, D.L., & Lauhoff, G. (2019). Descriptive data mining , (pp. 129–130): Springer.

Osterman, K.F., & Kottkamp, R.B. (1993). Reflective Practice for Educators: Improving Schooling Through Professional Development : ERIC.

Pantazos, K., & Vatrapu, R. (2016). Enhancing the professional vision of teachers: A physiological study of teaching analytics dashboards of students’ repertory grid exercises in business education. In System Sciences (HICSS), 2016 49th Hawaii International Conference On . IEEE, (pp. 41–50).

Pantazos, K., Vatrapu, R.K., Hussain, A. (2013). Visualizing repertory grid data for formative assessment. In IWTA@ LAK .

Papamitsiou, Z., & Economides, A.A. (2016). Learning analytics for smart learning environments: A meta-analysis of empirical research results from 2009 to 2015. Learning, Design, and Technology: An International Compendium of Theory, Research, Practice, and Policy , 1–23. https://doi.org/10.1007/978-3-319-17727-4_15-1 .

Pardo, A., & Siemens, G. (2014). Ethical and privacy principles for learning analytics. British Journal of Educational Technology , 45 (3), 438–450.

Pascual-Miguel, F., Chaparro-Pelaez, J., Hernandez-Garcia, A., Iglesias-Pradas, S. (2011). A characterisation of passive and active interactions and their influence on students’ achievement using moodle lms logs. International Journal of Technology Enhanced Learning , 3 (4), 403–414.

Pennings, H.J., Brekelmans, M., Wubbels, T., van der Want, A.C., Claessens, L.C., van Tartwijk, J. (2014). A nonlinear dynamical systems approach to real-time teacher behavior: Differences between teachers. Nonlinear Dynamics, Psychology, and Life Sciences , 18 (1), 23–45.

Persico, D., & Pozzi, F. (2015). Informing learning design with learning analytics to improve teacher inquiry. British Journal of Educational Technology , 46 (2), 230–248.

Pozzi, F., & Persico, D. (2013). Sustaining learning design and pedagogical planning in cscl. Research in Learning Technology , 21 . https://doi.org/10.3402/rlt.v21i0.17585 .

Prieto, L.P., Magnuson, P., Dillenbourg, P., Saar, M. (2017). Reflection for action: Designing tools to support teacher reflection on everyday evidence. https://doi.org/10.31219/osf.io/bj2rp .

Prieto, L.P., Rodriguez-Triana, M.J., Kusmin, M., Laanpere, M. (2017). Smart school multimodal dataset and challenges. In Joint Proceedings of the Sixth Multimodal Learning Analytics (MMLA) Workshop and the Second Cross-LAK Workshop Co-located with 7th International Learning Analytics and Knowledge Conference, vol. 1828 . CEUR, (pp. 53–59).

Prieto, L.P., Sharma, K., Dillenbourg, P., Jesús, M. (2016). Teaching analytics: towards automatic extraction of orchestration graphs using wearable sensors. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge . ACM, (pp. 148–157).

Prieto, L.P., Sharma, K., Kidzinski, Ł., Rodríguez-Triana, M.J., Dillenbourg, P. (2018). Multimodal teaching analytics: Automated extraction of orchestration graphs from wearable sensor data. Journal of Computer Assisted Learning , 34 (2), 193–203.

Ramos, C., & Yudko, E. (2008). "hits"(not "discussion posts") predict student success in online courses: a double cross-validation study. Computers & Education , 50 (4), 1174–1182.

Rayward-Smith, V.J. (2007). Statistics to measure correlation for data mining applications. Computational Statistics & Data Analysis , 51 (8), 3968–3982.

Article   MathSciNet   MATH   Google Scholar  

Remmers, H.H., & Brandenburg, G. (1927). Experimental data on the purdue rating scale for instructors. Educational Administration and Supervision , 13 (6), 399–406.

Rienties, B., Boroowa, A., Cross, S., Farrington-Flint, L., Herodotou, C., Prescott, L., Mayles, K., Olney, T., Toetenel, L., Woodthorpe, J. (2016). Reviewing three case-studies of learning analytics interventions at the open university uk. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge . ACM, (pp. 534–535).

Roberts, L.D., Chang, V., Gibson, D. (2017). Ethical considerations in adopting a university-and system-wide approach to data and learning analytics , (pp. 89–108): Springer. https://doi.org/10.1007/978-3-319-06520-5_7 .

Romero, C., & Ventura, S. (2010). Educational data mining: a review of the state of the art. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) , 40 (6), 601–618.

Saar, M., Kusmin, M., Laanpere, M., Prieto, L.P., Rüütmann, T. (2017). Work in progress–semantic annotations and teaching analytics on lecture videos in engineering education. In Global Engineering Education Conference (EDUCON), 2017 IEEE . IEEE, (pp. 1548–1551).

Saar, M., Prieto, L.P., Rodríguez-Triana, M.J., Kusmin, M. (2018). Personalized, teacher-driven in-action data collection: technology design principles. In 2018 IEEE 18th International Conference on Advanced Learning Technologies (ICALT) . IEEE, (pp. 58–62).

Saric, M., & Steh, B. (2017). Critical reflection in the professional development of teachers: Challenges and possibilities. CEPS Journal , 7 (3), 67–85.

Saye, J.W., & Brush, T. (2007). Using technology-enhanced learning environments to support problem-based historical inquiry in secondary school classrooms. Theory Research in Social Education , 35 (2), 196–230.

Schempp, P., McCullick, B., Pierre, P.S., Woorons, S., You, J., Clark, B. (2004). Expert golf instructors’ student-teacher interaction patterns. Research Quarterly for Exercise and Sport , 75 (1), 60–70.

Schmidlin, K., Clough-Gorr, K.M., Spoerri, A. (2015). Privacy preserving probabilistic record linkage (p3rl): a novel method for linking existing health-related data and maintaining participant confidentiality. BMC Medical Research Methodology , 15 (1), 46.

Sergis, S., & Sampson, D.G. (2016). Towards a teaching analytics tool for supporting reflective educational (re) design in inquiry-based stem education. In Advanced Learning Technologies (ICALT), 2016 IEEE 16th International Conference On . https://doi.org/10.1109/icalt.2016.134 . IEEE, (pp. 314–318).

Sergis, S., & Sampson, D.G. (2017). Teaching and learning analytics to support teacher inquiry: A systematic literature review , (pp. 25–63): Springer. https://doi.org/10.1007/978-3-319-52977-6_2 .

Sergis, S., Sampson, D.G., Rodríguez-Triana, M.J., Gillet, D., Pelliccione, L., de Jong, T. (2017). Using educational data from teaching and learning to inform teachers’ reflective educational design in inquiry-based stem education. Computers in Human Behavior . https://doi.org/10.1016/j.chb.2017.12.014 .

Shen, J., Chen, H., Jiang, J. (2018). A research on techniques for data fusion and analysis of cross-platform mooc data. In 2018 17th International Conference on Information Technology Based Higher Education and Training (ITHET) . IEEE, (pp. 1–8).

Siemens, G. (2013). Learning analytics: The emergence of a discipline. American Behavioral Scientist , 57 (10), 1380–1400.

Stier, S., Breuer, J., Siegers, P., Thorson, K. (2019). Integrating Survey Data and Digital Trace Data: Key Issues in Developing an Emerging Field. https://doi.org/10.1177/0894439319843669 .

Subramanya, S. (2014). Toward a more effective and useful end-of-course evaluation scheme. Journal of Research in Innovative Teaching , 7 (1).

Suehiro, D., Taniguchi, Y., Shimada, A., Ogata, H. (2017). Face-to-face teaching analytics: Extracting teaching activities from e-book logs via time-series analysis. In Advanced Learning Technologies (ICALT), 2017 IEEE 17th International Conference On . IEEE, (pp. 267–268).

Sun, J., Przybylski, R., Johnson, B.J. (2016). A review of research on teachers’ use of student data: From the perspective of school leadership. Educational Assessment, Evaluation and Accountability , 28 (1), 5–33.

Taniguchi, Y., Suehiro, D., Shimada, A., Ogata, H. (2017). Revealing hidden impression topics in students’journals based on nonnegative matrix factorization. In Advanced Learning Technologies (ICALT), 2017 IEEE 17th International Conference On . IEEE, (pp. 298–300).

Thille, C., & Zimmaro, D. (2017). Incorporating learning analytics in the classroom. New Directions for Higher Education , 2017 (179), 19–31.

Thomas, C. (2018). Multimodal teaching and learning analytics for classroom and online educational settings. In Proceedings of the 2018 on International Conference on Multimodal Interaction . ACM, (pp. 542–545).

ur Rehman, M.H., Chang, V., Batool, A., Wah, T.Y. (2016). Big data reduction framework for value creation in sustainable enterprises. International Journal of Information Management , 36 (6), 917–928.

Van Harmelen, M., & Workman, D. (2012). Analytics for learning and teaching. CETIS Analytics Series , 1 (3), 1–40.

van Leeuwen, A., Rummel, N., van Gog, T. (2019). What information should cscl teacher dashboards provide to help teachers interpret cscl situations? International Journal of Computer-Supported Collaborative Learning , 1–29. https://doi.org/10.1007/s11412-019-09299-x .

Vatrapu, R.K. (2012). Towards semiology of teaching analytics. In Workshop Towards Theory and Practice of Teaching Analytics, at the European Conference on Technology Enhanced Learning, TAPTA, vol. 12 .

Vatrapu, R.K., Kocherla, K., Pantazos, K. (2013). iklassroom: Real-time, real-place teaching analytics. In IWTA@ LAK .

Vatrapu, R., Reimann, P., Bull, S., Johnson, M. (2013). An eye-tracking study of notational, informational, and emotional aspects of learning analytics representations. In ACM International Conference Proceeding Series . https://doi.org/10.1145/2460296.2460321 . https://www.scopus.com/inward/record.uri?eid=2-s2.0-84876499638&doi=10.1145%2f2460296.2460321&partnerID=40&md5=e7b4d83a3e33e7a1c3c5b5f56d5ebe7d , (pp. 125–134).

Vatrapu, R., Reimann, P., Hussain, A., Kocherla, K. (2013). Towards teaching analytics: Repertory grids for formative assessment (rgfa). In CSCL 2013 Conference Proceedings, vol 2 , (pp. 422–426).

Vatrapu, R., Tanveer, U., Hussain, A. (2012). Towards teaching analytics: communication and negotiation tool (coneto). In Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design . ACM, (pp. 775–776).

Vatrapu, R., Teplovs, C., Fujita, N., Bull, S. (2011). Towards visual analytics for teachers’ dynamic diagnostic pedagogical decision-making. In Proceedings of the 1st International Conference on Learning Analytics and Knowledge . ACM, (pp. 93–98).

Verbert, K., Duval, E., Klerkx, J., Govaerts, S., Santos, J.L. (2013). Learning analytics dashboard applications. American Behavioral Scientist , 57 (10), 1500–1509.

Voithofer, R., & Golan, A.M. (2018). 5. Data Sources for Educators, 1st edn.https://doi.org/10.4324/9780203728703-7, (p. 18).

Waller, M.A., & Fawcett, S.E. (2013). Data science, predictive analytics, and big data: a revolution that will transform supply chain design and management. Journal of Business Logistics , 34 (2), 77–84.

Wang, F., & Hannafin, M.J. (2005). Design-based research and technology-enhanced learning environments. Educational Technology Research and Development , 53 (4), 5–23.

Williamson, B. (2016). Digital education governance: data visualization, predictive analytics, and ’real-time’ policy instruments. Journal of Education Policy , 31 (2), 123–141.

Winkler, R., & Söllner, M. (2018). Unleashing the potential of chatbots in education: A state-of-the-art analysis. In Academy of Management Annual Meeting (AOM) . https://doi.org/10.5465/ambpp.2018.15903abstract .

Xu, B., & Recker, M. (2012). Teaching analytics: A clustering and triangulation study of digital library user data. Educational Technology & Society , 15 (3), 103–115.

Yigitbasioglu, O.M., & Velcu, O. (2012). A review of dashboards in performance management: Implications for design and research. International Journal of Accounting Information Systems , 13 (1), 41–59.

Download references

Acknowledgements

The research reported is part of an ongoing PhD research study in the area of Big Data Analytics in Higher Education. We also want to thank members of the Technology Enhanced Learning and Teaching (TELT) Committee of the University of Otago, New Zealand for support and for providing constructive feedback.

This research project was fully sponsored by Higher Education Development Centre, University of Otago, New Zealand.

Author information

Authors and affiliations.

Higher Education Development Centre, University of Otago, Dunedin, New Zealand

Ifeanyi Glory Ndukwe & Ben Kei Daniel

You can also search for this author in PubMed   Google Scholar

Contributions

IGN conceived and presented the Conceptualisation of Teaching Analytics and Teachingv Outcome Model. BKD developed the Tripartite Approach that was utilised in this research. BKD encouraged IGN to perform a systematic review of teaching analytics that was guided by the Tripartite Approach. BKD supervised the findings of this work. IGN took the lead in writing the manuscript. All authors discussed the results, provided critical feedback and contributed to the final manuscript.

Corresponding author

Correspondence to Ifeanyi Glory Ndukwe .

Ethics declarations

Competing interests.

The authors declare that they have no competing interests. All authors have approved the manuscript and agree with its submission to the International Journal of Education Technology in Higher Education. This manuscript has not been published and is not under consideration for publication elsewhere.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Ndukwe, I.G., Daniel, B.K. Teaching analytics, value and tools for teacher data literacy: a systematic and tripartite approach. Int J Educ Technol High Educ 17 , 22 (2020). https://doi.org/10.1186/s41239-020-00201-6

Download citation

Received : 29 October 2019

Accepted : 01 April 2020

Published : 22 June 2020

DOI : https://doi.org/10.1186/s41239-020-00201-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Join our mailing list

Search form

• All ODI Projects
• K-16 Projects
• Workforce Prep Projects
• All ODI Resources
• K-16 Resources
• Workforce Prep Resources
• Community of Practice
• Press and News

You are here

Resources for educators using data in the classroom, odi_souhegan_nh_20151109_781.jpg.

EDC's Oceans of Data Institute (ODI) has compiled a list of data activities, lessons, and resources for the classroom, sorted by grade level: PreK | K-16 | Elementary School | Middle School |  Middle and High School | High School | High School and Postsecondary | Postsecondary | Teachers' Reference .

This list includes materials developed by ODI and ODI Collaborators, as well as lessons and activities developed by others that look like promising tools.

PreSchool Data Toolbox ODI RESOURCE Grades: PreK Disciplines: Science, Math NSF-funded A teacher-facing app plus digital teacher guide (available in the Apple and Android stores or online) that provides lesson plans for a series of data-focused investigations for PreK learners.

10 Sources to Free Big Data Sources Grades: K-16 Disciplines: Science, Technology, Engineering, Math 10 options for free big data sources for use in your classrooms.

CitizenScience.gov Grades: K-16 Disciplines: Science A catolog of hundreds of citizen science projects .

Data Education in Schools' Data Education Resources Grades: K-12 Disciplines: Art, Language, Math, Science, Social Studies This collection of free and open source teaching resources and lesson plans were curated by a government-funded initiatve in Scotland. Grade levels are listed by the Scottish equivalent. 

Data Nuggets Grades: K-12 Discipline: Science NSF-funded Free classroom activities, co-designed by scientists and teachers. Students practice interpreting quantitative information and making claims based on evidence.

Data Science for Everyone Grades: K-12 Discipline: Data Science A collection of resources for educators, students, industry leaders, and anyone interested in expanding their knowledge in data science education.

Entry Points to NASA Data Grades: 3-12 Disciplines: Science NASA offers a collection of topical data archives in a variety of science disciplines including astrophysics, biological and physical science, earth science, and more. Introductory resources will be more suitable for younger grades, while Advanced resources may only be suitable for older students and those interested in digging deeper.

My NASA Data Grades: 3-12 Disciplines: Earth Science My NASA Data offers global Earth science data collected from satellites and provides mini lessons, interactives, and lesson plans to support students and teachers, grades 3-12, in analyzing and interpreting this data.

NASA/IPAC Teacher Archive Research Program: Other  Education & Public Outreach Programs Using Real Data Grades: K-16 Discipline: Astronomy The NASA/IPAC Teacher Archive Research Program has compiled a list of astronomy programs that use real data, in addition to listing public-web-access robotic telescope options, citizen science programs, research opportunities for students, and more.

National Geographic  Grades: K-16 Disciplines: Earth Science , Geography, Biology, Mathematics , Social Studies , Engineering , Physics , Anthropology National Geographic-Funded Explore three lessons and 131 activities that work with data available through the National Geographic Education resource library.

NOAA Data Resources for Educators Grades: K-16 Disciplines: Earth Science , Geography, Biology, Climate, Oceanography, Life Science NOAA offers classroom-ready data resources, climate data resources, historical data resources, ocean and freshwater data resources, real-time data resources, weather and atmosphere data resources, and tutorials.

Using Data in the Classroom Grades: K-16 Disciplines: Science, Math, Geography, Physics NSF-Funded Extensive set of resources for educators and resource developers interested in effective teaching methods and pedagogical approaches for using data in the classroom.

ELEMENTARY SCHOOL

Streams of Data Lesson Plans & Supporting Materials ODI RESOURCE Grade: 4 Discipline: Science NSF-funded Free lesson plans for a five-day sequence about rivers and flooding. The goal of these lessons is to support development of analytical thinking skills around authentic science data.

TERC Inquiry Project Grades: 3-5 Discipline: Science NSF funded Using this curriculum, students use measurement, mathematical and graphical representations, and discussion to build scientific explanations about objects and materials in the world around them.  In particular, the investigations on  Volume ,  Heavy for Size , and  Two Scales , have children working with data.

Real Word, Real Science Curriculum Modules ODI COLLABORATION Grades: 5-6 Discipine: Science NASA funded Use authentic NASA and NOAA data to study the effects of the Earth's chaning climate on the animals and plants of Maine's diverse habitats. 

MIDDLE SCHOOL 

DataClubs Grades: 6-8 Discipline:  Science, Other Three modules on  Teens and Time ,   Ticks and Lyme Disease , and  Injuries On and Off the Field  (coming soon). Each module includes approximately 10 hours of activities, PDFs of printable content, and a list of materials needed. There are also links to several curated datasets in  CODAP , a free, web-based data analysis tool.

MIDDLE SCHOOL AND HIGH SCHOOL

CODAP (The Common Online Data Analysis Platform) ODI COLLABORATION Grades: 6-12 Discipline: Multiple NSF-funded An online, open-source data analysis platform that can be used in conjunction with a variety of data types and curricula. CODAP is geared toward middle and high school students. It can help students visualize and interpret data, and make evidence-based claims from the data.

DataClassroom Grades: 6-12 Disciplines: Current Events, Math, Science DataClassroom offers access to 75+ curated datasets, animated hypothesis tests, plus graphs and visuals for a variety of topics (free plan). Other options are available through paid plans. 

Discover Data Grades: 6-12 Disciplines: ELA, Math, Science Discover Data offers a variety of resources for middle and high school students and teachers, including student interactive, classroom activities, video career profiles, career resource guides and educator webinars.

Gapminder Grades: 6-12 Disciplines: Geography, Social Science A UK data visualization tool for middle to high school geography and social science courses with lessons and resources available for teachers wanting to use Gapminder in their classrooms.

High-Adventure Science Grades: 6-12 Discipline: Science NSF-funded Six free Earth and space science modules for middle and high school students to explore and engage in real-world data on unanswered science problems.

Maine Data Literacy Project Grades: 6-12 Disciplines: Science, Math, Statistics This project provides a framework, teaching materials, and professional development for middle and high school teachers to help students acquire skills and language for making sense of data and graphs as evidence to support their reasoning.

YouCubed Data Science Lessons Grades: 6-10 Discipline: Data Science Lessons that introduce students (and teachers) to data science, as well as additional data literacy resources .

HIGH SCHOOL

Biointeractive Grades: 9-12 Discipline: Science Hundreds of free science education multimedia resources. Includes apps, animations, videos, interactives, and virtual lab.

EDC Earth Science ODI RESOURCE Grades: 9-12 Discipline: Science NSF-funded A full-year high school earth science course fully aligned to  A Framework for K-12 Science Education: Practices, Crosscutting Concepts and Core Ideas  (NRC, 2012), EDC Earth Science is designed around the belief that students are capable of rigorous and in-depth explorations in science when given adequate support, structure, and motivation for learning. Students perform data-intensive investigations set in real-world contexts and engage in a variety of activities that build critical data-using skills. Sample data activities include: Looking at Climate Data , Patterns in Surface Currents , The Peru Current , and Water Running Dry .

Fathom Grades: 9-12 Discipline(s): Math, Statistics Fathom is a software for teaching data analysis and statistics. Teacher resources for Statistics, Algebra 1, Algebra 2, Precalculus, and Calculus, as well as a collection of data-driven physics labs is provided.

Introduction to Data Science Curriculum Grades: 9-12 Discipline: Computer Science, Data Science, Math, Statistics NSF-funded This high school data science curriculum fuses computer science, math, and statistics. 

Model My Watershed Grades: 9-12 Discipline: Science NSF-funded This watershed-modeling app is a tool to analyze real land use and soil data in your own neighborhood and watershed, model runoff and water-quality impacts, and compare how different scenarios could modify the watershed.

Other Worlds/Other Earths Curriculum Activities ODI COLLABORATION Grades: 9-12 Disciplines: Science, Astronomy NSF-funded EDC worked with the Harvard Smithsonian Center for Astrophysics to create an engaging set of curriculum activities that involve students in using data encoded in the light from distant stars to search for exoplanets and possible life beyond Earth. In addition, EDC also helped the Center for Astrophysics with develop, pilot, and field test a Web interface that allows students to detect exoplanets using online telescopes and Web-based data analytical tools.  Learn more .

Science Education Solutions Grades: 9-12 Discipline: Science This is a high school Earth Science curriculum that integrates Geo Information Systems (GIS) software and data to engage students in authentic, real-world data.

Stats4Stem.org Grades: 9-12 Discipline: Statistics NSF-funded This is a collection of learning, assessment, tutoring, data, and computing resources for statistics educators and their students.

Zoom In! Science ODI RESOURCE Grades: 9-12 Discipline: Biology, Earth Science NSF-funded Students use data to investigate problems in biology and Earth Science in these 6 on-line lessons.

HIGH SCHOOL AND POST SECONDARY 

Artificial Intelligence Methods in Data Science EDC RESOURCE Grades: 9-12, Post-Secondary Discipline:  Computer Science, Data Science NSF-funded These open source AI modules were developed through the NSF funded Science+C project. These modules were developed for high school students but can be easily adapted for community college students and may offer a starting point for those of you searching for AI coursework.

DryadLab Activities Grades: 9-12, Post-Secondary Discipline:  Not specific These activities are a project of the Dryad Digital Repository, which makes a wide variety of research data underlying scientific and medical publications discoverable, freely reusable, and citable.

DataBasic.io Grades: 9-12, Post-Secondary Discipline:  Not specific Provides 3 tools— Word Counter ,  WTFcsv , and  SameDiff —as well as sample activities to introduce students to the skills needed to analyze large datasets.

Ocean Tracks ODI RESOURCE Grades: 9-16 Discipline: Science NSF-funded These modules were developed to engage undergraduate students with authentic scientific data through investigations that mirror those currently being conducted by scientists studying the broad-scale effects of climate and human activities on top predators in ocean ecosystems. Using the Ocean Tracks interactive map and data analysis tools, students will explore and quantify patterns in the migratory tracks of marine animals in the northern Pacific Ocean and relate these behaviors to fluctuations and trends in physical oceanographic variables. Modules include:

• Fact of Artifact?: Interpreting Patterns in Ocean Tracks Data
• What's Up in the Pacific Ocean?: Connecting Productivity and Tuna Migration
• Faster, Farther, Deeper: Exploring the Physiology of Highly Migratory Ocean Predators
• Do You Come Here Often?: The Making of Biological Hotspots
• He Fed, She Fed
• Saving Sharks: Proposing a New Marine Protected Area

POST SECONDARY 

StatPrep Grades: Post-Secondary Discipline: Statistics NSF-funded StatPrep offers resources for use in your classes to teach data from a data centric point of view.

Teaching with Data Grades: Post-Secondary Discipline: Science NSF-funded This is a collection of social science resources from both free and subscription-based sources.

TEACHERS' REFERENCE

Amplifying Statistics and Data Science Discipline: Data Science, Math, Statistics Amplifying Statistics and Data Science in Classrooms  is organized into two modules, each with 5 units. You can earn a 20 hour certificate of completion for each module (40 hrs for both). In both modules, there are opportunities to hear from experts, learn with colleagues to gain different perspectives on teaching and learning statistics and data science, and to build a library of resources to support your teaching.

Resource Collection: Data Science Textbooks, Tools, and Certifications ODI RESOURCE Discipline: Data Science This collection of resources was generated by  Data Pathways Community of Practice  members—faculty and administrators from 2-and 4-year institutions building data programs.

Resources to Teach and Learn Data in High School Discipline: Data Science This website offers tools and lesson plans to teach data science.

Thinking Big ODI RESOURCE Discipline: Science Thinking Big , featured in the Summer 2015 issue of NSTA's The Science Teacher, explores curricular strategies for transitioning students to working with large, complex data sets.

Visualizing Oceans of Data: Educational Interface Design ODI RESOURCE NSF-funded This reports presents more than 70 cross-cutting and specific guidelines for interface and data visualization tool development and discusses key considerations (principles, research, and theory) that inform these guidelines. Through specific examples, the report explains how to avoid visualization pitfalls and make scientific databases more broadly accessible to meet the needs of diverse learners. A follow up report,  Visualizing Oceans of Data: Ocean– A Case Study  discusses the design and development of the Ocean Tracks interface, and reflects on the design guidelines in light of students' and teachers' experiences with the interface. 

Know of other great data literacy resources for educators? Email us at [email protected] .

Probability and Data Analysis Lesson Plan: What Are the Chances?

Submitted by: angela watson.

In this probability and data analysis lesson plan, students review these skills throughout a unit of study. Students then create a paper-based or digital activity that challenges classmates to apply data analysis, graphing, and/or probability skills to fictitious or real-world situations.

Students will:

• Review probability skills learned throughout a unit of study.
• Select a real-world situation in which data analysis and/or probability skills can provide insight into a topic and help predict outcomes.
• Create a paper-based or digital activity that challenges classmates to apply data analysis, graphing, and/or probability skills to the selected real-world situation.
• Internet access for BrainPOP
• Materials for creating their activities (poster board, markers, graphing paper, or whatever other items you would like students to use)

Preparation:

Lesson procedure:.

• Show one of the movie topics in the Probability or Data Analysis unit that is most closely related to your current topic of study.
• Tell students that they will create a game for their classmates that encourages them to think deeply about probability and/or data analysis. They may want their classmates to research a topic and calculate the odds of a particular outcome. Ideas for topics include: election outcomes, hurricane or other natural disaster track prediction, the amount of money a movie/album/book will gross based on other sales in the genre, stock market predictions, and so on.
• Divide students into groups/pairs or allow them to self-select groups, and then use the BrainPOP topics in the Probability and Data Analysis Unit to research the skill or concept they chose. Encourage students to explore the Related Reading features to learn more and get ideas for their activity.
• Students should then create their activity proposal, explaining how the activity will work and the math skills it will incorporate.The activity should should challenge their classmates to apply data analysis, graphing, and/or probability skills to the fictitious or real-world situation they selected and solve an interesting problem. Encourage students to use graphing paper, calculators, websites, etc. to create their game.
• Have each group of students present their proposals to the class or to another group for feedback and assistance in refining their topic and activity.
• Ask students to turn in their final proposals to you for approval. Help students clarify their objectives and procedure as needed.
• Give students time in class to create their activity, test it out, and then turn their activity in for you to check.
• Allow students to try out one another's activities during upcoming class periods. Each group of students should complete an assessment form for the activity or write a comment summary giving constructive feedback on how well the activity worked, how enjoyable it was, and how much it helped them understand the topic and practice their math skills.
• After students have tried a variety of activities, allow each group of students to read the feedback they received about their own activity. Students should then self-reflect: What was successful about the activity? What would they improve?

• BrainPOP Jr. (K-3)
• BrainPOP ELL
• BrainPOP Science
• BrainPOP Español
• BrainPOP Français
• Set Up Accounts
• Single Sign-on
• Manage Subscription
• Quick Tours
• About BrainPOP

• Terms of Use
• Privacy Policy
• Trademarks & Copyrights

Use Promo Code: SAVE70 for 70% OFF! Join Now

• Lesson Plan Examples
• Data Analysis And Interpretation

Free 12th Grade Data Analysis And Interpretation Lesson Plan (Research)

Topic:data analysis and interpretation, objectives & outcomes.

• To understand the process of data analysis and interpretation in research projects, including identifying independent and dependent variables, data collection methods, and how to analyze and interpret data to draw conclusions and make predictions.
• Examples of research studies with data analysis and interpretation (can be real or simulated)
• Data analysis and interpretation worksheets with questions and prompts for each step of the process
• Have students complete a short writing exercise about a recent event or experience that involved collecting and analyzing data (e.g. a project for a class or a hobby). Ask them to describe the data they collected, how they analyzed it, and what they learned from the process.
• As a class, discuss the benefits of analyzing and interpreting data instead of guessing or making assumptions.

Direct Instruction

• Introduce the concept of research data and explain how it is collected and analyzed in scientific research.
• Discuss the importance of analyzing and interpreting data correctly, including the potential consequences of misinterpretation.
• Review the key steps in data analysis, including identifying the data set, defining the variables, and determining the appropriate statistical tests to use.
• Demonstrate how to interpret data using charts, graphs, and statistical tests.

Guided Practice

• Have students work in pairs to analyze and interpret a data set on a specific topic, using the key steps described in the direct instruction.
• Provide guidance and support as needed.
• Allow time for students to present their analyses to the class and discuss their findings.

Independent Practice

• Have students work individually or in pairs to analyze and interpret a data set on a different topic, using the key steps described in the direct instruction.
• Review the key steps for analyzing and interpreting data and link them to the concepts covered in the lesson.
• Ask students to share their favorite part of the lesson and why.
• End the lesson by asking students to share one thing they learned about data analysis and interpretation.
• Observe students as they work in pairs to analyze and interpret data.
• Ask students to independently complete a worksheet or written activity on analyzing and interpreting data.
• Have students present their group project and results orally to the class.

Create amazing lesson plans 10X faster with AI.

Use AI to instantly generate high-quality lesson plans in seconds

Use AI to instantly generate high-quality lesson plans in seconds.

Get your first year for ONLY $14.70 using code SAVE70 at checkout! 🚀

See it in action! Watch the video!

Interviews as Narrative, Data, and Sources Lesson Plan

Creator: Özge Savaş Duration: 1 – 2 class periods Published : Summer 2020

In this lesson, students will learn the heterogeneity and richness qualitative methods, specifically narrative research, offers. Students will be able to generate new research questions, apply coding and analysis for oral history and life history research, by using the interviews in the Global Feminisms Project Archive.

Keywords : Narrative Research, Oral History, Qualitative Research, Interview, Life History Research, Coding, Qualitative Data Analysis Country sites:  Nicaragua , China , India , Brazil , Poland , USA

Learning Objectives

• Students will identify and generate appropriate research questions for oral history research and for life history research.
• Students will code and analyze interview data in order to answer questions of oral history research and life history research.

Video Clips

Download all available materials.

< Back to Topical Lesson Plans

Office of the Vice President for Research

Four clas faculty researchers secure prestigious early career awards.

Continuing  an upward trend of University of Iowa faculty securing prestigious early-career grants, four investigators from the Departments of Physics and Astronomy and Computer Science have been awarded notable grant awards to advance their careers.

DeRoo, Hoadley advance space instrumentation with Nancy Grace Roman Technology Fellowships in Astrophysics for Early Career Researchers

Casey DeRoo and Keri Hoadley , both assistant professors in the Department of Physics and Astronomy, each received a Nancy Grace Roman Technology Fellowship in Astrophysics for Early Career Researchers. The NASA fellowship provides each researcher with $500,000 over two years to support their research in space-based instrumentation. 

Hoadley’s research is two-pronged. She will design and ultimately prototype a mirror-based vacuum ultraviolet polarizer, which will allow researchers to access polarized light from space below 120-nanometer wavelength. Polarizing light at such a low wavelength is crucial to building optics for NASA’s future Habitable World Observatory (HWO), the agency’s next flagship astrophysics mission after the Nancy Grace Roman Space Telescope. 

“Our vacuum ultraviolet polarizer project is meant to help set up our lab to propose to NASA for one or more follow-up technology programs, including adapting this polarizer for use in vacuum systems, duplicating it and measuring its efficiency to measure additional flavors of polarized UV light, quantifying the polarization effects introduced by UV optical components that may be used on HWO, and building an astronomical instrument to measure the polarization of UV from around massive stars and throughout star-forming regions,” said Hoadley.

In addition, Hoadley and her team will build a facility to align, calibrate, and integrate small space telescopes before flight, using a vacuum chamber and wavelengths of light typically only accessible in space, which could help the university win future small satellite and suborbital missions from NASA. 

DeRoo will work to advance diffraction gratings made with electron beams that pattern structures on a nanometer scale.   Like a prism, diffraction gratings spread out and direct light coming from stars and galaxies, allowing researchers to deduce things like the temperature, density, or composition of an astronomical object.

The fellowship will allow DeRoo to upgrade the university’s Raith

 Voyager tool, a specialized fabrication tool hosted by OVPR’s Materials Analysis, Testing and Fabrication (MATFab) facility.

“These upgrades will let us perform algorithmic patterning, which uses computer code to quickly generate the patterns to be manufactured,” DeRoo said. “This is a major innovation that should enable us to make more complex grating shapes as well as make gratings more quickly.” DeRoo added that the enhancements mean his team may be able to make diffraction gratings that allow space instrument designs that are distinctly different from those launched to date.

“For faculty who develop space-based instruments, the Nancy Grace Roman Technology Fellowship is on par with the prestige of an NSF CAREER or Department of Energy Early Career award,” said Mary Hall Reno, professor and department chair. “Our track record with the program elevates our status as a destination university for astrophysics and space physics missions.”

Uppu pursues building blocks quantum computing with NSF CAREER Award

Ravitej Uppu, assistant professor in the Department of Physics and Astronomy, received a 5-year NSF CAREER award of $550,000 to conduct research aimed at amplifying the power of quantum computing and making its application more practical. 

Uppu and his team will explore the properties of light-matter interactions at the level of a single photon interacting with a single molecule, enabling them to generate efficient and high-quality multiphoton entangled states of light. Multiphoton entangled states, in which photons become inextricably linked, are necessary for photons to serve as practical quantum interconnects, transmitting information between quantum computing units, akin to classical cluster computers. 

“ In our pursuit of secure communication, exploiting quantum properties of light is the final frontier,” said Uppu. “However, unavoidable losses that occur in optical fiber links between users can easily nullify the secure link. Our research on multiphoton entangled states is a key building block for implementing ‘quantum repeaters’ that can overcome this challenge.”

Jiang tackles real-world data issues with NSF CAREER Award

Peng Jiang, assistant professor in the Department of Computer Science, received an NSF CAREER Award that will provide $548,944 over five years to develop tools to support the use of sampling-based algorithms. 

Sampling-based algorithms reduce computing costs by processing only a random selection of a dataset, which has made them increasingly popular, but the method still faces limited efficiency. Jiang will develop a suite of tools that simplify the implementation of sampling-based algorithms and improve their efficacy across wide range of computing and big data applications.

“ A simple example of a real-world application is subgraph matching,” Jiang said. “For example, one might be interested in finding a group of people with certain connections in a social network. The use of sampling-based algorithms can significantly accelerate this process.”

In addition to providing undergraduate students the opportunity to engage with this research, Jiang also plans for the project to enhance projects in computer science courses.

• Global Mobile Trends 2024

Module: Spectrum , Digital Consumer , Fixed, TV and Convergence , IoT and Enterprise , Mobile Operators and Networks

Global mobile trends 2024 79791138.

• GSMA Intelligence

Enter AI and the 'open of everything'

GSMA Intelligence has released the latest edition in the Global Mobile Trends series. Much has changed since our first publication in 2015. Technology has advanced and companies have come and gone. As always, the purpose of Global Mobile Trends is simple: understand the biggest and most important things happening in telecoms and the broader TMT industries, and explain what they mean for people, companies and governments.

This Global Mobile Trends report examines the biggest developments expected in 2024. These include 5G’s next wave; the impact of generative AI; 5G-Advanced; new experiences from 5G-enabled services such as FWA; the future of entertainment; cloud and edge compute; private wireless; eSIM; the rise of satellites; and sustainability strategies.

This comprehensive report draws on GSMA Intelligence data, survey findings and industry conversations to focus on the trends that matter most and what they mean for those in telecoms and beyond.

Contact our research team

Get in touch with us to find out more about our research topics and analysis.

Related Data & Charts

• Lorem ipsum

To cite our research, please see our citation policy in our Terms of Use , or contact our Media team for more information.

Related Research