Step-by-Step Guide 14 (Jamovi):

Reliability Analysis

Overview

Here we will show you how you can reduce a series of variables into one using a Reliability Analysis.

Essentially, what this allows to you to do is add the scores of each of the variables together to make a new variable. The reliability analysis tells us whether there is sufficient internal consistency to do so.  

Note: A reliability analysis assumes that there is only one factor and that all variables you use are weighted the same. If this assumption does not hold, use a EFA. 

Dataset & Variables

Skoczylis, Joshua, 2021, "Extremism, Life Experiences and the Internet", https://doi.org/10.7910/DVN/ICTI8T, Harvard Dataverse, Version 3.

Variables required for test:

Variable Use in Example: n/a 

Or

Variable Use in Example: We are using all the variables that measure Social Media Use (Read, Write, Follow, Share, Spread)

Note: Many ordinal variables in the extremism dataset are classified as categorical. That said, Jamovi does recognise them as ordinal, as long as these are not text (e.g. the underlying level needs to be numbered). But still double check your variables before you use them. 

Reliability Analysis: Step-by-Step Guide

Purpose of Test:

The Extremism dataset has a lot of variables on Social Media use about e.g. Facebook and how it is used.  Below we will create a new variable that gives us a continues variable that measures an individuals Social Media use. 

Always make sure that all the variables you selected use the same measures e.g. Likert scale

1.

Select your Variables

Navigate to Analyses > Factor > Reliability Analysis

Now select your variables. For the example we are using all the variables that Measure. 

By default this test pre-selects the Cronbach's Alpha test. This result will appear in the results section on your right. 

2.

Check your result & Item Reliability 

There is no need to change the default scale test (although you can also add McDonald's Omega if you wish).

This is how you interpret the results:

Above 0.9 -  Excellent

0.8 to 0.9  - Good 

0.7 to 0.8 - Acceptable

0.6 to 0.7 - Questionable

0.5 to 0.6 - Poor

below 0.5 - unacceptable

Values below 0.7 suggest that the variables you have selected may not be closely enough correlated to make one new variable. 

If this is the case, select the Cronbach's Alpha (if item dropped) under the Item Statistics. This will produce a list that tells you how the Cronbach's Alpha would change if you drop an item from the list. 

Results:

The Cronbach's Alpha is 0.889 which suggest that the variables are reliably correlated. The list in the image below just tells you how the value would change if an item was dropped. 

3.

Reverse Scale where necessary

Sometimes, your scales may need to be reversed. For example

Variable 1: 1 (Strongly Agree), 2 (Agree), 3 (neither Agree/nor disagree), 4 (Disagree), 5 (Strongly Disagree)

Variable 2:  5(Strongly Agree), 4 (Agree), 3 (neither Agree/nor disagree), 2 (Disagree), 1 (Strongly Disagree)

As you can see the two variables measure the same things, but the scores are reversed. To fix this, we need to reverse score one off the variables. So what we end up with is this:

Variable 1: 1 (Strongly Agree), 2 (Agree), 3 (neither Agree/nor disagree), 4 (Disagree), 5 (Strongly Disagree)

Variable 2: 1 (Strongly Agree), 2 (Agree), 3 (neither Agree/nor disagree), 4 (Disagree), 5 (Strongly Disagree)

Jamovi will tell you when you and which variable to reverse code. 

Jamovi tells us when we need to do so, but do double check as sometimes they are already in the right oder. All you have to do is move the item in question over to the Reverse Scale Items side. The results in the table on the right will be automatically adjusted. 

3.

Save Results as new variable

Finally, you can save your new variable under the Save Tab. You have the following two options:

I prefer the Sum, but either works. 

To make the results easier to understand, you can standardise the variable using z-scores. This is easily done by using the computing function. A refresher of how to do this is available here. 

Voila, you have now got a new variable you can use in any further analysis.