Research Methods
Lesson 4
Introduction to Reliability & Validity Investigation & Design Produced by
Defining reliability: • One of the main goals in research is to design and carry out studies in such a way that the replication or the repetition of one’s findings is possible. • In order to achieve that goal, it is important that the measures we use should posses good reliability or consistency. • Reliability can be defined as a term that means dependability or consistency. If the findings are replicated consistently then the outcome can be said to be reliable.
So what does reliability mean ? • Firstly, in psychology for our experiment to be reliable it must be possible for another psychologists to replicate it. Upon replicating it they should find similar results. This is because they will use the same measures you used in your experiment. If they achieve the same results we can say that your experiment and measures are reliable. What are these measures ? • Let’s say you design a questionnaire for adults that can measure whether they had a secure or insecure attachment with their parents during childhood and you test it and believe it is reliable.
• By you claiming it is reliable, you mean you have tested the questionnaire on several people who have been in secure and insecure attachments and it does produce the appropriate results. Further, when you tested the same participants time and time again over different periods of time, it produced the same results. This is how you would go about claiming the measure for attachment is reliable. • In order for it to be reliable for academics in psychology to use as a test to measure secure and insecure attachments. Other psychologists would first have to replicate your test on several of their own participants in the same way as you have. This includes repeating the questionnaire with the same participants over a time. If they produce the same type of results you have, we then can say that your measure for attachment is reliable.
Validity: • In addition to our measurement being reliable or consistent we also want our measurement to be valid. Validity means that we are actually measuring what we say we are measuring. •
For example if our questionnaire on attachment comprised of the following questions: 1) How often do you eat green beans? 2)
Do you have vegetables for both lunch and dinner?
3)
What is your favourite vegetable?
4)
How often do you eat it?
5)
Is cabbage part of your diet?
• Without a doubt your questionnaire on attachment is not valid as the questions have no relevance to attachment in childhood. In fact it is a questionnaire asking you some details about your diet opposed to attachment. Internal and External Validity: • There are two broad categories of validity. Namely internal validity and external validity. Internal Validity: • Internal validity refers to whether the results or the outcome of a study were really due to the manipulation of the IV whilst all other variables were held constant and there are no extraneous variables.
External Validity: • External validity refers to whether the findings can be generalised to settings other than the research settings. • This includes population validity, which is the questions of whether the findings can be generalised to other people and ecological validity, which is the question of whether the results can be generalised to other settings other than the one the study was carried out in.
Validity
Internal Validity
Population Validity
External Validity
Ecological Validity
Validity for Psychometric Test: • Different methods are employed to measure the validity of tests, such as intelligence tests and personality tests. Test of these measures are referred to as psychometric tests. • The three measures commonly used face validity, concurrent validity and predictive validity. • Let us look at each of these in the slides that follow in this lesson.
Face Validity: • This is the most basic method of assessing validity and quite simply involves an ‘eyeball test’. That is does the test look as if it is measuring what is says it’s measuring at face value. Example: • Reflecting on our questionnaire on attachment, if a psychologists just had a look at it at face value or does an ‘eyeball test’ they would not need to go any further because at face value one can see the questions are about ones vegetable diet opposed to attachment and conclude the test is invalid for attachment.
Concurrent Validity: • This involves comparing the results yielded by a new test with those from an older test known to have good validity. For example, if a participant achieved an IQ score of 148 on an older IQ test, but scored 113 on the new IQ test, questions may be raised over the validity of this new test. Example: • Reflecting on our attachment questionnaire, we would give participants an existing valid attachment test and our new test to complete. If participants are found to achieve the same results in both test we can say our questionnaire has concurrent validity. If it does not achieve that then the chances are that our test could be invalid.
Predictive Validity: • This is the ability of the test to predict performance on future tests. If a test can do this it is said to have good predictive validity. Example: • There are tests currently available that crunch up or use your GCSE grades and predict whether you will pass you’re A Levels. Since these tests have been used on a wide sample and the prediction has been quite realistic. We can say that these test have good predictive validity.
Validity for Psychometric Tests
Face Validity
Concurrent Validity
Predictive Validity
Further Reading: • www.psychologyrevise.com Investigation and Design • Wider reading and more examples. • Mind Map
Research Methods Reliability & Validity