Tutorial 6: Basic Correlations

While we will explore how probabilities are calculated, you will not have to do this yourself. But you do need to understand the underlying concepts.  So with that said, let's break down the concept of probability into a very simple and understandable explanation. Probability is essentially a way to quantify how likely it is that something will happen. 

Calculate Probability

Imagine you have a regular six-sided die, and you want to calculate the probability of rolling a 5. Since there are six possible outcomes (1, 2, 3, 4, 5, 6) and only one of these outcomes is a 4, the probability of rolling a 5 is 1 out of 6 or 0.166 or 16.6% (0.166*100).

What happens if we have two independent events and we want to know the probability of both of let's say getting 2 sixes? Let's go back to the dice. When you roll a die twice, the outcome of each roll is independent of the other, meaning the result of the first roll does not affect the result of the second roll. This scenario introduces the concept of joint probability for independent events.

Joint Probability for Independent Events

The joint probability of two independent events, A and B, happening is the product of their individual probabilities. This is often expressed as:

Probabilities ( A and B) = Probability(A) * Probability (B)

Hypothesis testing is a fundamental aspect of statistical analysis, allowing us to make inferences about our data and, by extension, the world around us. While descriptive statistics provide insights into individual variables, hypothesis testing delves deeper, exploring the relationships and differences between variables to validate or refute assumptions about our data.

Basics of Hypothesis Testing

At its core, a hypothesis is a statement positing a particular relationship or condition, such as "All cars are red." For a hypothesis to be meaningful in a statistical context, it must be testable. This involves setting up two contrasting statements: the Null Hypothesis (H0) and the Alternative Hypothesis (Ha).

Null Hypothesis (H0): This posits the absence of a statistically significant relationship between the variables in question. It represents the default or status quo, serving as the claim to be tested. For instance, it might assert that there is no difference in a particular attribute across groups.

Alternative Hypothesis (Ha): This serves as the counterclaim and suggests that there is a statistically significant relationship between variables. It is what you might believe to be true or hope to prove.

Two other terms that you should learn are directional and non-directional hypotheses. Directional Hypotheses specify the direction of the expected relationship or difference between variables. For Example: 

• H0: There is no difference in income between men and women.

• Ha: Women have a lower income than men.

Non-directional Hypotheses do not specify the direction of the relationship or difference but simply state that a difference or relationship exists. 

• H0: There is no difference in income between men and women. 

• Ha: There is a difference in income between men and women.

Testing Hypotheses

To evaluate these hypotheses, you select an appropriate statistical test based on the nature of your data and hypotheses. The outcome of this test, expressed through p-values and confidence intervals, helps determine whether to reject or accept the Null Hypothesis. 

P-values quantify the probability of observing the collected data, or something more extreme, assuming the Null Hypothesis is true. A low p-value indicates a higher chance that the Null Hypothesis can be rejected in favour of the Alternative Hypothesis.

Confidence Intervals provide a range of values within which the true value of the parameter is expected to lie, with a certain degree of confidence. This can offer additional insight into the relationship between variables.

Important Considerations

It's crucial to remember that while the p-value and your decision on the hypotheses can indicate the presence or absence of a statistically significant relationship, they do not inform about the strength or practical significance of the relationship. For that, you would look into effect size measures and other statistical analyses.

Type 1 Error (False Positive)

A Type 1 error occurs when the Null Hypothesis (H0) is incorrectly rejected when it is actually true. This is akin to a false alarm or a false positive result. In the context of hypothesis testing, it means that you conclude there is a significant effect or difference when, in reality, there isn't one.

Example: 

If your null hypothesis states there is no difference in income between men and women and the alternative Hypothesis claims the opposite,  a Type 1 error would mean concluding there is a difference, when actually there is no difference.

Type 2 Error (False Negative)

A Type 2 error happens when the Null Hypothesis is incorrectly accepted when the Alternative Hypothesis (Ha) is actually true. This error is referred to as a false negative. It means missing out on identifying a genuine effect or difference.

Example: 

If your null hypothesis states there is no difference in income between men and women and the alternative Hypothesis claims the opposite,  a Type 2 error would mean concluding there is no difference, when actually there is.

Significance Levels also called P-Values

This is a threshold set by the researcher to decide how much risk of a Type 1 error is acceptable. In the Social Sciences, this is commonly set at 0.05 (often also referred to as alpha). This indicates that the research accepts that there is a 0.05 (or 5%) chance of falsely rejecting the Null Hypothesis is deemed acceptable. You can of course set lower levels such as 0.01, or even 0.001. 

Balancing Type 1 and Type 2 Errors

In statistical analysis, there's often a trade-off between minimizing Type 1 and Type 2 errors. Reducing the risk of one can increase the risk of the other. The challenge for researchers is to design their studies and choose significance levels that balance these risks appropriately, taking into account the context and consequences of potential errors.

Key Takeaways

Type 1 Error: Rejecting a true Null Hypothesis (false positive).

Type 2 Error: Failing to reject a false Null Hypothesis (false negative).

You must balance the risks of Type 1 and Type 2 errors, often by setting an appropriate significance level and designing studies with sufficient power to detect true effects.

Understanding probabilities is crucial to grasping the concept of p-values, which are a fundamental aspect of statistical hypothesis testing. A p-value helps you determine the significance of your results in a statistical hypothesis test.

What is a P-value?

Inferential statistics always involve calculating p-values, which are crucial in hypothesis testing. A p-value is a number between 0 and 1 that indicates the probability of observing the collected data (or something more extreme) under the assumption that the null hypothesis is true. It helps in deciding whether to reject or accept the null hypothesis. The smaller the p-value, the stronger the evidence against the null hypothesis, making us more inclined to reject it.

Zero (0) indicates an event is impossible under the null hypothesis.

One (1) indicates certainty that the event will occur under the null hypothesis.

The closer the p-value is to 0, the stronger the evidence against the null hypothesis. In the social sciences, a p-value of 0.05 or lower is typically sought. This threshold signifies a 5% probability that the observed results could occur by chance, meaning there is a 95% confidence level that the results are not due to random chance. This 0.05 threshold, however, is somewhat arbitrary and can be adjusted based on the acceptable risk level of the research. For instance, a threshold of 0.1 implies a 10% risk of a false positive is acceptable.

It's important to note that the choice of significance level (e.g., 0.05) depends on the context of the research and the researcher's willingness to accept the risk of a Type 1 error (falsely rejecting the null hypothesis). While 0.05 is standard, especially in social sciences, different fields may choose different thresholds based on the norms and consequences of errors in those fields.

Parametric data refers to datasets that assume the underlying data distribution is known and usually follows a normal distribution (bell curve). This assumption about the distribution allows for the application of parametric statistical tests, which can offer more powerful and precise insights when the assumptions are correctly met. The key characteristics of parametric data include:

• Distribution: The data should be approximately normally distributed.

• Interval or Ratio Scale: Parametric data is typically on an interval or ratio scale, allowing for meaningful comparisons using mathematical operations.

• Homogeneity of Variances: The variance among groups should be similar.

Because parametric tests rely on these assumptions, they are best used when your data is from a large sample size (which, according to the Central Limit Theorem, tends to be normally distributed as sample size increases) or when you have prior evidence to believe your data follows a normal distribution.

Types of Parametric Tests

Here are some of the most common parametric tests you might use:

• Pearson Correlation Coefficient: Measures the strength and direction of the linear relationship between two continuous variables. Use a correlation Matrix to display this data

• Linear Regression: Used to model the relationship between a dependent variable and one or more independent variables. It assesses how much the dependent variable changes when any one of the independent variables is varied, while the other independent variables are held fixed.

• t-test (Student's t-test): Used to compare the means between two groups. There are different versions of the t-test, including the independent samples t-test (for comparing two independent groups) and the paired sample t-test (for comparing two related groups or the same group at two different times).

• ANOVA (Analysis of Variance): Used when comparing the means across three or more groups. ANOVA tests whether there's a statistically significant difference between the group means. If the overall ANOVA is significant, post-hoc tests can determine which specific groups differ from each other.

T-test and ANOVAs have been placed last here, as they are special applications of Linear Regression. In this course,  you will learn about Linear Regression first, as this will give you a really good understanding of the other tests.  

Importance of Meeting Assumptions

Note: Most statistical programmes allow you to test for assumptions before you run the tests. Take the time to do so.  

The accuracy of parametric tests hinges on the validity of their underlying assumptions. If these assumptions are violated, the results may not be reliable. For instance, if your data significantly deviates from normality, using a parametric test could lead to incorrect conclusions. In such cases, transforming the data or opting for non-parametric alternatives might be necessary.

Parametric tests are powerful tools in statistical analysis, offering precise estimates and inferences about the population when their assumptions are met. They are particularly valuable in experimental designs and quantitative research where the assumptions about the population distribution can be justified.

Non-parametric data refers to data that does not assume an underlying distribution. This can be due to various reasons such as:

• The data does not follow a normal distribution.

• The sample size is too small to confidently assume a normal distribution.

• The data is ordinal (ranked) or nominal (categorical) and doesn’t have inherent numerical meaning that allows for mean or standard deviation calculations in a way that parametric tests require.

Non-parametric methods are more flexible and don’t require the data to meet strict assumptions about the population parameters. This makes them very useful in dealing with real-world data that doesn’t neatly fit the ideal conditions for parametric tests.

Let's assume you are interested in the average extremism score of the UK population. It is impractical to measure every single person's extremism score, so instead you measure the extremism score of a sample population. 

Based on this sample you can calculate the average extremism score. Let's assume you took a different sample, chances are you would get a different average score. So how can we be confident that our results are close to the true population average extremism score? 

This is where confidence intervals come into play!

Understanding Confidence Intervals

A confidence interval gives you a range of values within which the true average height is likely to fall. It's like saying, "We are 95% confident that the true average extremism score of the population is between x and y. 

Here's a simple breakdown of how they work (in reality your statistical software will calculate them for you):

Choose Your Confidence Level: This is usually 95%, but it can be 90%, 99%, or any other value depending on how confident you want to be. A 95% confidence level means that if you were to take 100 different samples and compute a confidence interval from each of them, about 95 of those intervals would contain the true average height.

Calculate the Sample Mean (Average): This is the average extremism score from your sample.

Calculate the Margin of Error: This involves some statistics, including the standard deviation (how spread out the extremism score is) and the sample size. The larger your sample, the smaller your margin of error will be.

Create the Interval: You take the sample mean and add and subtract the margin of error to it. This gives you the lower and upper bounds of your confidence interval.

So, if your sample extremism mean is 0.87 and you are using 95% confidence intervals you can be 95% confident that the mean extremism score of the population is somewhere between 0.80 to 0.93. You can, of course,  increase your confidence by selecting higher confidence intervals e.g. 99%. 

In essence, confidence intervals provide a way to quantify the uncertainty of your estimate. They don't guarantee that the true average falls within the interval, but they give you a range that is likely to contain the true average based on your sample data and the level of confidence you've chosen. To watch a really easy explanation of how, when and why we use them watch this video here. 

Effect size and correlation coefficients are both measures used in statistics to quantify relationships between variables, but they serve different purposes and convey different types of information. Note, these only become important if your p-value has reached your set threshold, e.g., 95%:

Effect Size

Effect size measures the magnitude of a relationship or the strength of an effect in a population, without being influenced by the sample size. It provides a quantitative description of the difference between groups or the size of a relationship within a study, making it easier to understand the practical significance of research findings. Common examples include Cohen's d, which quantifies the difference between two means relative to the standard deviation, and eta squared, which measures the proportion of variance explained in the dependent variable by the independent variable.

Test such as t-test and the non-parametric test will have effect sizes available to you. 

Correlation Coefficients:

Note: These are used to measure the strength of a linear relationship - so use them for continuous and ordinal data

Correlation coefficients quantify the strength and direction of a linear relationship between two continuous variables. They range from -1 to 1, where 1 indicates a perfect positive linear relationship, -1 indicates a perfect negative linear relationship, and 0 indicates no linear relationship. The most common correlation coefficient is Pearson's r, which measures the degree to which two variables linearly relate to each other (Continuous). For ordinal data use Spearman's or Kendall's Tau B. 

In summary, effect size is broader and can describe the magnitude of any type of effect, not just linear relationships, making it useful for understanding the practical importance of results. Correlation coefficients specifically measure the linear relationship between two variables, providing insight into how closely two variables change together.

If you want to compare nominal data, or ordinal data using various levels, you would use a contingency table. We covered how to create a contingency table and how to interpret the data in the previous tutorial. Here we will understand in more detail how we can use this test for hypothesis testing.

Let's say we want to explore the relationship between Extreme Yes/No and Share Nothing with Society. To figure this out, we use a contigency table where on one side we list whether someone is extreme or not and other side list whether they agreed with the statement Violence is effective in creating a new Society. This data can be explained in counts and/or percentages. This table helps you organize the counts of different combinations of categories (e.g., Extreme, Strongly Disagree, Extreme, Somewhat Agree, etc).

Looking at the counts and/or percentages can give you some idea about the relationship. Using a Chi-square test coupled with measures of association allows us to test our hypothesis and measure how strong the relationship is. This test looks at the numbers in your table and calculates whether the differences you see are just due to chance, or if they're likely reflecting a real pattern or relationship. In simple terms, the Chi-Square test helps you figure out if the observations you made are likely true for the broader population or if they just happened by accident in your survey. 

For nominal data, where categories do not have an inherent order, several measures of association are used to analyze relationships between variables. Here are some common ones:

Interpreting the output

The Chi-Square Test of Independence tests whether there is a significant association between two nominal/ordinal variables (if both are ordinal, consider using Spearman's test). You can also add levels - but avoid having too many levels. The p-value will tell you whether you should accept or reject your null Hypothesis. 

You can measure the strength of the association between the variables using the following tests (more are available).  

• Cramer's V: Derived from the Chi-Square statistic, Cramer's V is a measure of association between two nominal variables, providing a value between 0 and 1. This measure adjusts for the number of categories and the sample size, giving a standardized measure of association strength.

• Phi Coefficient: This is a measure of association specifically for 2x2 contingency tables (two variables that each have two levels). It's similar to Pearson's correlation coefficient but is used for binary/nominal variables.

• Contingency Coefficient: This is another measure based on the Chi-Square statistic used for nominal data. It indicates the strength of the association between two variables, but it is less commonly used than Cramer's V because it is affected by the size of the table (number of categories). 

These measures allow researchers to understand and quantify the strength and direction of associations between nominal variables, providing insights into the relationships in categorical data.

Let's use what we know and apply it to the above example using the Extreme Yes/No and Violence is effective to createing a new Society variables. Here is the Hypothesis we will test: 

H0: There is no difference between being extreme or not and their attitudes to whether Violence is an effective way to achieve a new society. 

Ha: There is a difference between being extreme or not and their attitudes to whether Violence is an effective way to achieve a new society.