Skip to content
R for the Rest of Us Logo
R for the Rest of Us Logo

Reliability

This lesson is locked

Get access to all lessons in this course.

Transcript

Click on the transcript to go to that point in the video. Please note that transcripts are auto generated and may contain minor inaccuracies.

Your Turn

Perform a test of internal consistency using both omega and alpha on the SWLS scale at time 2.

Does the scale have good internal consistency at time 2, as well?

Learn More

Read more about the omega() and alpha() functions in the psych package.

The Personality Project also has a great tutorial on Using R and the psych package to find omega.

Interested in learning more about why it may be preferential to use omega instead of alpha? Check out the article "Thanks coefficient alpha, we'll take it from here " by McNeish (2018). Use this link to download the unformatted open access version of the article.

Have any questions? Put them below and we will help you out!

You need to be signed-in to comment on this post. Login.

Maria Cristina Limlingan

Maria Cristina Limlingan

January 13, 2022

Thanks for sharing about omega - this is good information to know. I'm wondering if you suggestions about how to do inter rater reliability between two people in R? (e.g. 2 people doing an observation and want to find out if they are coding similarly)

Dana Wanzer

Dana Wanzer

January 14, 2022

There's a couple packages that can do it, including irr and tidycomm. For a previous project of mine, I went with tidycomm (using the test_icr function) because we had missing data and more than 2 coders, plus it can calculate a wide variety of IRR values. https://cran.r-project.org/web/packages/tidycomm/tidycomm.pdf

Maria Cristina Limlingan

Maria Cristina Limlingan

February 10, 2022

Thanks for the suggestion Dana! I used the tidycomm package and got it to run - but I wanted to make sure if the way I structured the file was right - do you have an example of how the data should look like in R? Also, just in case you had some thoughts - if the data we are trying to check reliability is nominal/categorical - I read the Hayes and Krippendorff paper below and based like that is Krippendorff the recommended alpha you would use? Thanks in advance for your help! Hayes, A. F., & Krippendorff, K. (2007). Answering the Call for a Standard Reliability Measure for Coding Data. Communication Methods and Measures, 1(1), 77–89. https://doi.org/10.1080/19312450709336664

Dana Wanzer

This vignette from the package does a good job of showing how the data needs to be structured: https://cran.r-project.org/web/packages/tidycomm/vignettes/v04_icr.html

And yes, my understanding is to use Krippendorff's alpha if you have missing data or ordinal/interval/ratio level of measurement in the coding.