For example, one would expect new measures of test anxiety or physical risk taking to be positively correlated with existing measures of the same constructs. This is known as convergent validity. Assessing convergent validity requires collecting data using the measure. Discriminant validity , on the other hand, is the extent to which scores on a measure are not correlated with measures of variables that are conceptually distinct.
For example, self-esteem is a general attitude toward the self that is fairly stable over time. It is not the same as mood, which is how good or bad one happens to be feeling right now. If the new measure of self-esteem were highly correlated with a measure of mood, it could be argued that the new measure is not really measuring self-esteem; it is measuring mood instead.
All these low correlations provide evidence that the measure is reflecting a conceptually distinct construct. Method of assessing internal consistency through splitting the items into two sets and examining the relationship between them. In reference to criterion validity, variables that one would expect to be correlated with the measure.
The extent to which scores on a measure are not correlated with measures of variables that are conceptually distinct. Skip to content Chapter 5: Psychological Measurement. Define reliability, including the different types and how they are assessed. Define validity, including the different types and how they are assessed. Describe the kinds of evidence that would be relevant to assessing the reliability and validity of a particular measure.
Psychological researchers do not simply assume that their measures work. Instead, they conduct research to show that they work. If they cannot show that they work, they stop using them.
There are two distinct criteria by which researchers evaluate their measures: reliability and validity. Reliability is consistency across time test-retest reliability , across items internal consistency , and across researchers interrater reliability.
Validity is the extent to which the scores actually represent the variable they are intended to. Validity is a judgment based on various types of evidence. The reliability and validity of a measure is not established by any single study but by the pattern of results across multiple studies.
The assessment of reliability and validity is an ongoing process. Then assess its internal consistency by making a scatterplot to show the split-half correlation even- vs. Discussion: Think back to the last college exam you took and think of the exam as a psychological measure. Determining cause and effect is one of the most important parts of scientific research.
You want to find out how blood sugar levels are affected by drinking diet soda and regular soda, so you conduct an experiment. The value of a dependent variable depends on an independent variable, so a variable cannot be both independent and dependent at the same time.
It must be either the cause or the effect, not both! Yes, but including more than one of either type requires multiple research questions. For example, if you are interested in the effect of a diet on health, you can use multiple measures of health: blood sugar, blood pressure, weight, pulse, and many more. Each of these is its own dependent variable with its own research question.
You could also choose to look at the effect of exercise levels as well as diet, or even the additional effect of the two combined. Each of these is a separate independent variable. To ensure the internal validity of an experiment , you should only change one independent variable at a time. To ensure the internal validity of your research, you must consider the impact of confounding variables. If you fail to account for them, you might over- or underestimate the causal relationship between your independent and dependent variables , or even find a causal relationship where none exists.
A confounding variable is closely related to both the independent and dependent variables in a study. An independent variable represents the supposed cause , while the dependent variable is the supposed effect. A confounding variable is a third variable that influences both the independent and dependent variables. Failing to account for confounding variables can cause you to wrongly estimate the relationship between your independent and dependent variables.
There are several methods you can use to decrease the impact of confounding variables on your research: restriction, matching, statistical control and randomization. In restriction , you restrict your sample by only including certain subjects that have the same values of potential confounding variables. In matching , you match each of the subjects in your treatment group with a counterpart in the comparison group.
The matched subjects have the same values on any potential confounding variables, and only differ in the independent variable. In statistical control , you include potential confounders as variables in your regression. In randomization , you randomly assign the treatment or independent variable in your study to a sufficiently large number of subjects, which allows you to control for all potential confounding variables.
Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organizations.
However, there are also some drawbacks: data collection can be time-consuming, labor-intensive and expensive. Operationalization means turning abstract conceptual ideas into measurable observations.
Hypothesis testing is a formal procedure for investigating our ideas about the world using statistics. It is used by scientists to test specific predictions, called hypotheses , by calculating how likely it is that a pattern or relationship between variables could have arisen by chance. There are five common approaches to qualitative research :.
There are various approaches to qualitative data analysis , but they all share five steps in common:. The specifics of each step depend on the focus of the analysis. Some common approaches include textual analysis , thematic analysis , and discourse analysis. In scientific research, concepts are the abstract ideas or phenomena that are being studied e.
Variables are properties or characteristics of the concept e. The process of turning abstract concepts into measurable variables and indicators is called operationalization. A Likert scale is a rating scale that quantitatively assesses opinions, attitudes, or behaviors. It is made up of 4 or more questions that measure a single attitude or trait when response scores are combined. To use a Likert scale in a survey , you present participants with Likert-type questions or statements, and a continuum of items, usually with 5 or 7 possible responses, to capture their degree of agreement.
Overall Likert scale scores are sometimes treated as interval data. These scores are considered to have directionality and even spacing between them. The type of data determines what statistical tests you should use to analyze your data. An experimental group, also known as a treatment group, receives the treatment whose effect researchers wish to study, whereas a control group does not. They should be identical in all other ways. A true experiment a. However, some experiments use a within-subjects design to test treatments without a control group.
Blinding means hiding who is assigned to the treatment group and who is assigned to the control group in an experiment. If participants know whether they are in a control or treatment group , they may adjust their behavior in ways that affect the outcome that researchers are trying to measure. If the people administering the treatment are aware of group assignment, they may treat participants differently and thus directly or indirectly influence the final results.
A quasi-experiment is a type of research design that attempts to establish a cause-and-effect relationship. The main difference with a true experiment is that the groups are not randomly assigned. Quasi-experimental design is most useful in situations where it would be unethical or impractical to run a true experiment. Quasi-experiments have lower internal validity than true experiments, but they often have higher external validity as they can use real-world interventions instead of artificial laboratory settings.
Simple random sampling is a type of probability sampling in which the researcher randomly selects a subset of participants from a population. Each member of the population has an equal chance of being selected.
Data is then collected from as large a percentage as possible of this random subset. The American Community Survey is an example of simple random sampling. In order to collect detailed data on the population of the US, the Census Bureau officials randomly select 3. If properly implemented, simple random sampling is usually the best sampling method for ensuring both internal and external validity.
However, it can sometimes be impractical and expensive to implement, depending on the size of the population to be studied,. If you have a list of every member of the population and the ability to reach whichever members are selected, you can use simple random sampling. Cluster sampling is a probability sampling method in which you divide a population into clusters, such as districts or schools, and then randomly select some of these clusters as your sample.
There are three types of cluster sampling : single-stage, double-stage and multi-stage clustering. In all three types, you first divide the population into clusters, then randomly select clusters for use in your sample. Cluster sampling is more time- and cost-efficient than other probability sampling methods , particularly when it comes to large samples spread across a wide geographical area.
However, it provides less statistical certainty than other methods, such as simple random sampling , because it is difficult to ensure that your clusters properly represent the population as a whole.
In stratified sampling , researchers divide subjects into subgroups called strata based on characteristics that they share e. Once divided, each subgroup is randomly sampled using another probability sampling method. Using stratified sampling will allow you to obtain more precise with lower variance statistical estimates of whatever you are trying to measure.
For example, say you want to investigate how income differs based on educational attainment, but you know that this relationship can vary based on race. Using stratified sampling, you can ensure you obtain a large enough sample from each racial group, allowing you to draw more precise conclusions. Yes, you can create a stratified sample using multiple characteristics, but you must ensure that every participant in your study belongs to one and only one subgroup.
In this case, you multiply the numbers of subgroups for each characteristic to get the total number of groups. Systematic sampling is a probability sampling method where researchers select members of the population at a regular interval — for example, by selecting every 15th person on a list of the population.
If the population is in a random order, this can imitate the benefits of simple random sampling. There are three key steps in systematic sampling :. A mediator variable explains the process through which two variables are related, while a moderator variable affects the strength and direction of that relationship.
A confounder is a third variable that affects variables of interest and makes them seem related when they are not. In contrast, a mediator is the mechanism of a relationship between two variables: it explains the process by which they are related.
Including mediators and moderators in your research helps you go beyond studying a simple relationship between two variables for a fuller picture of the real world. They are important to consider when studying complex correlational or causal relationships.
Mediators are part of the causal pathway of an effect, and they tell you how or why an effect takes place. Moderators usually help you judge the external validity of your study by identifying the limitations of when the relationship between variables holds. Control variables help you establish a correlational or causal relationship between variables by enhancing internal validity. Researchers often model control variable data along with independent and dependent variable data in regression analyses and ANCOVAs.
In experimental research, random assignment is a way of placing participants from your sample into different groups using randomization.
With this method, every member of the sample has a known or equal chance of being placed in a control group or an experimental group. In contrast, random assignment is a way of sorting the sample into control and experimental groups. Random sampling enhances the external validity or generalizability of your results, while random assignment improves the internal validity of your study.
Then, you can use a random number generator or a lottery method to randomly assign each number to a control or experimental group. You can also do so manually, by flipping a coin or rolling a dice to randomly assign participants to groups. Random assignment is used in experiments with a between-groups or independent measures design. Random assignment helps ensure that the groups are comparable.
In general, you should always use random assignment in this type of experimental design when it is ethically possible and makes sense for your study topic. In a between-subjects design , every participant experiences only one condition, and researchers assess group differences between participants in various conditions.
In a within-subjects design , each participant experiences all conditions, and researchers test the same participants repeatedly for differences between conditions.
Between-subjects and within-subjects designs can be combined in a single study when you have two or more independent variables a factorial design. In a mixed factorial design, one variable is altered between subjects and another is altered within subjects. While a between-subjects design has fewer threats to internal validity , it also requires more participants for high statistical power than a within-subjects design. Within-subjects designs have many potential threats to internal validity , but they are also very statistically powerful.
In a factorial design, multiple independent variables are tested. If you test two variables, each level of one independent variable is combined with each level of the other independent variable to create different conditions. A confounding variable is a type of extraneous variable that not only affects the dependent variable, but is also related to the independent variable.
There are 4 main types of extraneous variables :. Controlled experiments require:. Usually, these two measurements are used in psychological tests and research materials. Outside the research field, however, these two words are used interchangeably. What is Validity — Definition, Features, Types 2. What is Reliability — Definition, Features, Types 3. Validity is the extent to which a test measures what it claims to measure. In other words, it means the accuracy of a test. Therefore, it is a scientific test or piece of research that actually measures what it sets out to measure, or how well it reflects the reality it claims to represent.
Kelly who stated that a test is valid if it measures what it claims to measure, formulated the concept of validity. Validity implies the extent to which the research instrument measures, what it is intended to measure. Reliability refers to the degree to which assessment tool produces consistent results, when repeated measurements are made.
It relates to the extent to which an experiment, test or any procedure gives the same result on repeated trials. Influencing factors for validity are: process, purpose, theory matters, logical implications, etc. Influencing factors for reliability are: test length, test score variability, heterogenicity, etc. Even if validity of an instrument is poor for certain test , it can have high reliability for other tests.
0コメント