Step-by-Step Guide 13 (Jamovi):

Factorial ANOVA

Overview

Here you will learn the following

Factorial ANOVAs (sometimes called two-way ANOVAs) are similar to a one-way ANOVA. The major difference is that you can use one or more categorical variables as your independent variable (one of which must have 3 or more outcomes). 

In many ways, once you are starting to use more than one categorical variable you should consider using Linear Regression, as this will provide you with much more information and is much more flexibe. If you want to know more details about the difference between a Factorial ANOVA and a one-way ANOVA read this post. 

You can also perform an ANOVA on repeated measures. This is not covered here. But a good introduction is available as is a step-by-step guide on how to run a Repeated Measures ANOVA in Jamovi 

Dataset used for Factorial ANOVA

Skoczylis, Joshua, 2021, "Extremism, Life Experiences and the Internet", https://doi.org/10.7910/DVN/ICTI8T, Harvard Dataverse, Version 3.

Factorical ANOVA Test

Factorial ANOVA Hypothesis

Because a Factorial ANOVA explores the effects of two categorical factors as well as the effect of the categorical factors on each other, there are three pairs of hypotheses. 

H01: An individual's socioeconomic status has no impact on their levels of self-confidence.

Ha1: An individual's socioeconomic status has an impact on their levels of self-confidence.

H02: An individual's Gender has no impact on their level of self-confidence

Ha2: An individual's Gender has an impact on their level of self-confidence

H03: An individual's Gender and socioeconomic status have no impact on their levels of self-confidence.

Ha3: An individual's Gender and socioeconomic status have an impact on their levels of self-confidence.

Variables required

Independent Variable(s): 

Confidence_Self: This variable measures the participants self confidence score. Negative scores represent low self confidence, high scores represent high self confidence

Dependent Variable(s): 

Factor 1

Socio-Economic_Status: This variable records the participants work status.

Factor 2

Gender: The participants gender

or 

No Ordinal variable was used in this example. 

But you can select ordinal variables instead of nominal ones 

Factorial ANOVA Assumptions

Test Assumptions Not Met: 

If your assumptions are not met, use the non-parametric Kruskal Wallis test. We will go through this test below.

Note:  There is some discussion amongst statisticians of how stringent one must be with these assumptions as an ANOVA can be relatively robust when the normality assumption is violated. Be aware that ignoring the violation of normality may increase your risk of Type 1 error.  

If your sample is small check your Kurtosis in the Descriptive Statistics section. If it is negative, you should not use a Standard ANOVA. 

Consider using linear regression as an alternative. This will also allow you to use more than two independent variables. 

Factorial ANOVA: Step-by-Step Guide 

Using a Factorial ANOVA

by Datalabcc

1.

Select your Variables Overall Model Effect & Effect Size

Navigate to Analyses > ANOVA > ANOVA

Select your relevant variables in the Dependent & Fiexed Factors boxes.

Now select your Model Fit.

The overal Model Fit provides you with p-value for all your models together.

Effect Size: Select Partial n2 (partial Eta Squared). This will give you the Effect Size for your models. 

2.

Check ANOVA Assumptions are met

Before we look at the results, we need to check that the assumptions are met. 

In the Assumptions Check section, select the Homogeneity test (equal variance) and the Normality test to check whether the assumptions are violated.

From the tables below we can see that the Homogeneity is not violated (p-values betwen 0.648 and 0.238).

The the normality assumption, however, is violated (p <.001)

In this case, we will continue, but be aware that the chance of a Type 1 error may increase.

2.

Select your Post-hoc test

In the Post-Hoc Tests section decide which variables you want a post-hoc test for and drag them over to the field on the right. 

Now select your test (Correction section) and add the effect size. 

As we are using more than one variable, there will be lots of tables.

2.

Generate a Marginal Means Plot

Plots to visualise the Factorial ANOVA can easily be generated in the Estimated Marginal Means Section.

Move the variable you want to include over to the Marginal Means (Term 1) box. If you had more than two variables, you might have more Terms - but this is something we will cover in Linear Regression.

Sometimes is can also be useful to get a Marginal Means table. The information in it can be used to provide context.

Results: Factorial ANOVA

1.

Outcome: Accept Null Hypotheses for H1-H3

Based on the results from the table above, we have to accept the Null Hypothes for H1-3.  

Model 1 (impact of socioeconomic status on self-confidence) returns a significant p-value (<.001), but based on the effect size (0.034) we can conclude that one's socioeconomic status has no effect on self-confidence levels, so we accept the null hypothesis despite the significant p-value.

The same results are visualised below. Unfortunatley, the lables overlap. This is hopefully, something that Jamovi will fix in the future. 

If the results were significant we could have also explored the post-hoc test tables. They have been included below, to show you what they might look like. 

Tukey Post-hoc test for Gender:

Tukey Post-hoc test output for Socioeconomic Status:

Tukey Post-hoc test outcompe for Gender*Socioeconomic Status. This is a very big table, so only part of it has been included below. But it should give you a good idea of what an interaction table might look like.