{"id":4071,"date":"2024-05-01T07:00:42","date_gmt":"2024-05-01T11:00:42","guid":{"rendered":"https:\/\/elpsclone.flywheelsites.com\/?p=4071"},"modified":"2024-05-13T16:22:58","modified_gmt":"2024-05-13T20:22:58","slug":"how-do-i-critically-consume-quantitative-research","status":"publish","type":"post","link":"https:\/\/cehhs.utk.edu\/elps\/how-do-i-critically-consume-quantitative-research\/","title":{"rendered":"How Do I Critically Consume Quantitative Research?"},"content":{"rendered":"<p><strong>By Austin Boyd\u00a0<\/strong><\/p>\n<p><span data-contrast=\"auto\">Every measurement, evaluation, statistics, and assessment (MESA) professional, whether they <\/span><span aria-label=\"Rich text content control\"><span data-contrast=\"auto\">\u200b<\/span><span data-contrast=\"auto\">are<\/span><span data-contrast=\"auto\">\u200b<\/span><\/span> <span data-contrast=\"auto\">established educators and practitioners or aspiring students, engages with academic literature in some capacity. Sometimes for work, <\/span><span aria-label=\"Rich text content control\"><span data-contrast=\"auto\">\u200b<\/span><span data-contrast=\"auto\">\u200b<\/span><\/span><span data-contrast=\"auto\">other times for pleasure, but always in the pursuit of new knowledge. But how do we as consumers of research determine whether the quantitative research we engage with is high quality?<\/span><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559738&quot;:180,&quot;335559739&quot;:180,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p>\n<p><span data-contrast=\"auto\"><img loading=\"lazy\" decoding=\"async\" width=\"171\" height=\"228\" class=\"size-full wp-image-3905 alignright\" src=\"https:\/\/cehhs.utk.edu\/elps\/wp-content\/uploads\/sites\/9\/2023\/11\/Picture2.jpg\" alt=\"\" \/>My name is Austin Boyd, and I am a researcher, instructor, and ESM alumni. <\/span><span aria-label=\"Rich text content control\"><span data-contrast=\"auto\">\u200b<\/span><span data-contrast=\"auto\">\u200b<\/span><\/span><span data-contrast=\"auto\">I have read my fair share of articles over the past decade and was fortunate enough to publish a few of my own. I have read articles in the <\/span><span aria-label=\"Rich text content control\"><span data-contrast=\"auto\">\u200b<\/span><span data-contrast=\"auto\">\u200b<\/span><\/span><span data-contrast=\"auto\">natural, formal, applied, and social sciences, and while they all shared the title of peer-reviewed publication, there was definitely variability in the quality of quantitative research from one article to the next. Initially, it was difficult for me to even consider the idea that a peer-reviewed publication would be anything less than perfect. However, as I have grown as a critical consumer of research, I have devised six questions to keep in mind when reading articles with quantitative analyses that allow me to remain objective in the face of exciting results.<\/span><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559738&quot;:180,&quot;335559739&quot;:180,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p>\n<ol>\n<li data-leveltext=\"%1.\" data-font=\"Cambria\" data-listid=\"1\" data-list-defn-props=\"{&quot;335552541&quot;:0,&quot;335559685&quot;:1080,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Cambria&quot;,&quot;469769242&quot;:[65533,0],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;%1.&quot;,&quot;469777815&quot;:&quot;multilevel&quot;}\" data-aria-posinset=\"1\" data-aria-level=\"1\"><span data-contrast=\"auto\">\u200b<\/span><span aria-label=\"Rich text content control\"><span data-contrast=\"auto\">\u200b<\/span><b><span data-contrast=\"auto\">\u00a0\u00a0\u00a0\u00a0 <\/span><\/b><b><span data-contrast=\"auto\">\u200b<\/span><\/b><\/span><b><span data-contrast=\"none\">What is the purpose of the article?\u00a0<\/span><\/b><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559738&quot;:180,&quot;335559739&quot;:180,&quot;335559740&quot;:240,&quot;335572071&quot;:0,&quot;335572072&quot;:0,&quot;335572073&quot;:0,&quot;335572075&quot;:0,&quot;335572076&quot;:0,&quot;335572077&quot;:0,&quot;335572079&quot;:0,&quot;335572080&quot;:0,&quot;335572081&quot;:0,&quot;335572083&quot;:0,&quot;335572084&quot;:0,&quot;335572085&quot;:0,&quot;335572087&quot;:0,&quot;335572088&quot;:0,&quot;335572089&quot;:0,&quot;469789798&quot;:&quot;nil&quot;,&quot;469789802&quot;:&quot;nil&quot;,&quot;469789806&quot;:&quot;nil&quot;,&quot;469789810&quot;:&quot;nil&quot;,&quot;469789814&quot;:&quot;nil&quot;}\">\u00a0<\/span><\/li>\n<\/ol>\n<p><span data-contrast=\"auto\">The first question to keep in mind when reading an article is<\/span><span aria-label=\"Rich text content control\"><span data-contrast=\"auto\">\u200b<\/span><span data-contrast=\"auto\">,<\/span><span data-contrast=\"auto\">\u200b<\/span><\/span><span data-contrast=\"auto\"> \u201c<\/span><span aria-label=\"Rich text content control\"><span data-contrast=\"auto\">\u200b<\/span><span data-contrast=\"auto\">W<\/span><span data-contrast=\"auto\">\u200b<\/span><\/span><span data-contrast=\"auto\">hat is its purpose?\u201d Articles may state these in the form of research questions or even in the title by using words such as \u201cempirical\u201d, \u201cvalidation\u201d, and \u201cmeta-analysis\u201d. While the purpose of an article has no bearing on its quality, it does impact the type of information a reader should expect to obtain from it. Do the research questions indicate that the article will be presenting new exploratory research on a new phenomenon or attempting to validate previous research findings<\/span><span data-contrast=\"auto\">?<\/span><span data-contrast=\"auto\"> Remaining aware of the article\u2019s purpose allows you to determine if the information is relevant and in the scope of what it should be providing.<\/span><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559738&quot;:180,&quot;335559739&quot;:180,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p>\n<ol>\n<li data-leveltext=\"%1.\" data-font=\"Cambria\" data-listid=\"1\" data-list-defn-props=\"{&quot;335552541&quot;:0,&quot;335559685&quot;:1080,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Cambria&quot;,&quot;469769242&quot;:[65533,0],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;%1.&quot;,&quot;469777815&quot;:&quot;multilevel&quot;}\" data-aria-posinset=\"2\" data-aria-level=\"1\"><b><span data-contrast=\"none\">What information is provided about obtaining participants and about the participants themselves?<\/span><\/b><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559738&quot;:180,&quot;335559739&quot;:180,&quot;335559740&quot;:240,&quot;335572071&quot;:0,&quot;335572072&quot;:0,&quot;335572073&quot;:0,&quot;335572075&quot;:0,&quot;335572076&quot;:0,&quot;335572077&quot;:0,&quot;335572079&quot;:0,&quot;335572080&quot;:0,&quot;335572081&quot;:0,&quot;335572083&quot;:0,&quot;335572084&quot;:0,&quot;335572085&quot;:0,&quot;335572087&quot;:0,&quot;335572088&quot;:0,&quot;335572089&quot;:0,&quot;469789798&quot;:&quot;nil&quot;,&quot;469789802&quot;:&quot;nil&quot;,&quot;469789806&quot;:&quot;nil&quot;,&quot;469789810&quot;:&quot;nil&quot;,&quot;469789814&quot;:&quot;nil&quot;}\">\u00a0<\/span><\/li>\n<\/ol>\n<p data-ccp-border-between=\"0px none #000000\" data-ccp-padding-between=\"0px\"><span data-contrast=\"auto\">The backbone of quantitative research is data. In order to have any data, participants or cases must be found and measured for the phenomena of interest. These participants are all unique, and it is this uniqueness that needs to be disclosed to the reader. Information on the population of interest, how the selected participants were recruited, who they are, and why their results were or were not included in the analysis is essential for understanding the c<\/span><span aria-label=\"Rich text content control\"><span data-contrast=\"auto\">\u200b<\/span><span data-contrast=\"auto\">\u200b<\/span><\/span><span data-contrast=\"auto\">ontext of the research. Beyond the article itself, the demographics of the participants are also important for planning future research. While research participants are largely Western, educated, industrialized, rich, and democratic societies (<\/span><span aria-label=\"Rich text content control\"><span data-contrast=\"auto\">\u200b<\/span><span data-contrast=\"auto\">\u200b<\/span><\/span><span data-contrast=\"auto\">WEIRD; <\/span><span data-contrast=\"auto\">Henrich et al., 2010), it should not be assumed that this is the case for all research. The author(s) of an article should disclose demographic information of the participants<\/span><span data-contrast=\"auto\">,<\/span><span data-contrast=\"auto\"> so the readers understand the context of the data and the generalizability of the results, and so that researchers can accurately replicate or expand the research to ne<\/span><span aria-label=\"Rich text content control\"><span data-contrast=\"auto\">\u200b<\/span><span data-contrast=\"auto\">\u200b<\/span><\/span><span data-contrast=\"auto\">w contexts.<\/span><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559738&quot;:180,&quot;335559739&quot;:180,&quot;335559740&quot;:240,&quot;335572071&quot;:0,&quot;335572072&quot;:0,&quot;335572073&quot;:0,&quot;335572075&quot;:0,&quot;335572076&quot;:0,&quot;335572077&quot;:0,&quot;335572079&quot;:0,&quot;335572080&quot;:0,&quot;335572081&quot;:0,&quot;335572083&quot;:0,&quot;335572084&quot;:0,&quot;335572085&quot;:0,&quot;335572087&quot;:0,&quot;335572088&quot;:0,&quot;335572089&quot;:0,&quot;469789798&quot;:&quot;nil&quot;,&quot;469789802&quot;:&quot;nil&quot;,&quot;469789806&quot;:&quot;nil&quot;,&quot;469789810&quot;:&quot;nil&quot;,&quot;469789814&quot;:&quot;nil&quot;}\">\u00a0<\/span><\/p>\n<ol>\n<li data-leveltext=\"%1.\" data-font=\"Cambria\" data-listid=\"1\" data-list-defn-props=\"{&quot;335552541&quot;:0,&quot;335559685&quot;:1080,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Cambria&quot;,&quot;469769242&quot;:[65533,0],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;%1.&quot;,&quot;469777815&quot;:&quot;multilevel&quot;}\" data-aria-posinset=\"3\" data-aria-level=\"1\"><b><span data-contrast=\"none\">Do the analyses used make sense for the data and proposed research question(s)?<\/span><\/b><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559738&quot;:180,&quot;335559739&quot;:180,&quot;335559740&quot;:240,&quot;335572071&quot;:0,&quot;335572072&quot;:0,&quot;335572073&quot;:0,&quot;335572075&quot;:0,&quot;335572076&quot;:0,&quot;335572077&quot;:0,&quot;335572079&quot;:0,&quot;335572080&quot;:0,&quot;335572081&quot;:0,&quot;335572083&quot;:0,&quot;335572084&quot;:0,&quot;335572085&quot;:0,&quot;335572087&quot;:0,&quot;335572088&quot;:0,&quot;335572089&quot;:0,&quot;469789798&quot;:&quot;nil&quot;,&quot;469789802&quot;:&quot;nil&quot;,&quot;469789806&quot;:&quot;nil&quot;,&quot;469789810&quot;:&quot;nil&quot;,&quot;469789814&quot;:&quot;nil&quot;}\">\u00a0<\/span><\/li>\n<\/ol>\n<p><span data-contrast=\"auto\">In order to obtain results from the quantitative data collected, some form of analysis must be conducted. The most basic methods of exploring quantitative data are called statistics (<\/span><span data-contrast=\"auto\">Sheard, 2018<\/span><span data-contrast=\"auto\">). T<\/span><span aria-label=\"Rich text content control\"><span data-contrast=\"auto\">\u200b<\/span><span data-contrast=\"auto\">\u200b<\/span><\/span><span data-contrast=\"auto\">he selected statistical analysis should align with the variables presented in the article and answer the research question(s) guiding the project. There is a level of subjectivity as to which statistical analysis should be used to analyze data. Variables measured on a nominal scale should not be used as the outcome variable when conducting analyses that look at the differences between group means, such as <\/span><span aria-label=\"Rich text content control\"><span data-contrast=\"auto\">\u200b<\/span><span data-contrast=\"auto\">\u200b<\/span><\/span><span data-contrast=\"auto\">t-tests and ANOVAs, while ratio scale variables should not be used to conduct analyses dealing with frequency distributions, such as chi-square tests. However, there are analyses which require the same variable types, making them seemingly interchangeable. For example, t-tests, logistic regressions, and point biserial analyses all use two variables, one continuous and one binary. However, each of these analyses addresses different research questions such as \u201cIs there a difference between groups?\u201d, \u201cCan we predict an outcome?\u201d, and \u201cIs there a relationship between variables?\u201d. While there is a level of subjectivity as to which statistical analysis can be used to analyze data, there are objectively incorrect analyses based on both the overarching research questions and the scale of measurement of the available variables in the data.\u00a0<\/span><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:160,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p>\n<ol>\n<li data-leveltext=\"%1.\" data-font=\"Cambria\" data-listid=\"1\" data-list-defn-props=\"{&quot;335552541&quot;:0,&quot;335559685&quot;:1080,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Cambria&quot;,&quot;469769242&quot;:[65533,0],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;%1.&quot;,&quot;469777815&quot;:&quot;multilevel&quot;}\" data-aria-posinset=\"4\" data-aria-level=\"1\"><b><span data-contrast=\"none\">What results are provided?<\/span><\/b><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:160,&quot;335559740&quot;:240,&quot;335572071&quot;:0,&quot;335572072&quot;:0,&quot;335572073&quot;:0,&quot;335572075&quot;:0,&quot;335572076&quot;:0,&quot;335572077&quot;:0,&quot;335572079&quot;:0,&quot;335572080&quot;:0,&quot;335572081&quot;:0,&quot;335572083&quot;:0,&quot;335572084&quot;:0,&quot;335572085&quot;:0,&quot;335572087&quot;:0,&quot;335572088&quot;:0,&quot;335572089&quot;:0,&quot;469789798&quot;:&quot;nil&quot;,&quot;469789802&quot;:&quot;nil&quot;,&quot;469789806&quot;:&quot;nil&quot;,&quot;469789810&quot;:&quot;nil&quot;,&quot;469789814&quot;:&quot;nil&quot;}\">\u00a0<\/span><\/li>\n<\/ol>\n<p><span data-contrast=\"auto\">While a seemingly straightforward question, there is a lot of information that can be provided about a given analysis. The most basic, and least informative, is a blanket statement about the statistical<\/span> <span data-contrast=\"auto\">significance. Even if there is no statistically significant result to report, a blanket statement is not sufficient information about the analyses with all the different values that can be reported for each analysis. For example, a t-test has a t value, degrees of freedom, <\/span><i><span data-contrast=\"auto\">p<\/span><\/i> <span data-contrast=\"auto\">value, confidence interval, power level, and effect size, all of which provide valuable information about the results. While having some of these values does allow the reader to calculate the missing ones, the onus should not be put on the reader to do so (Cohen, 1990). Additionally, depending on the type of statistical analysis chosen, additional tests of the data must be conducted to determine if the data meets the assumptions necessary for the analysis. The results of these tests of assumptions and the decisions made based on them should be reported and supported by the existing literature.<\/span><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:160,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p>\n<ol>\n<li data-leveltext=\"%1.\" data-font=\"Cambria\" data-listid=\"1\" data-list-defn-props=\"{&quot;335552541&quot;:0,&quot;335559685&quot;:1080,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Cambria&quot;,&quot;469769242&quot;:[65533,0],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;%1.&quot;,&quot;469777815&quot;:&quot;multilevel&quot;}\" data-aria-posinset=\"5\" data-aria-level=\"1\"><b><span data-contrast=\"none\">Is there any discussion of limitations?<\/span><\/b><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:160,&quot;335559740&quot;:240,&quot;335572071&quot;:0,&quot;335572072&quot;:0,&quot;335572073&quot;:0,&quot;335572075&quot;:0,&quot;335572076&quot;:0,&quot;335572077&quot;:0,&quot;335572079&quot;:0,&quot;335572080&quot;:0,&quot;335572081&quot;:0,&quot;335572083&quot;:0,&quot;335572084&quot;:0,&quot;335572085&quot;:0,&quot;335572087&quot;:0,&quot;335572088&quot;:0,&quot;335572089&quot;:0,&quot;469789798&quot;:&quot;nil&quot;,&quot;469789802&quot;:&quot;nil&quot;,&quot;469789806&quot;:&quot;nil&quot;,&quot;469789810&quot;:&quot;nil&quot;,&quot;469789814&quot;:&quot;nil&quot;}\">\u00a0<\/span><\/li>\n<\/ol>\n<p><span data-contrast=\"auto\">Almost every article has limitations in some form or other, which should be made known to the reader. If an article didn\u2019t have any limitations, the author would make a point to state as much. Limitations include limits to the generalizability of the findings, confounding variables, or simply time constraints. While these might seem negative, they are not immediate reasons to discredit an article entirely. As was the case for the demographics, the limitations provide further context about the research. They can even be useful in providing direction for follow-up studies in the same way a future research section would.\u00a0<\/span><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:160,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p>\n<ol>\n<li data-leveltext=\"%1.\" data-font=\"Cambria\" data-listid=\"1\" data-list-defn-props=\"{&quot;335552541&quot;:0,&quot;335559685&quot;:1080,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Cambria&quot;,&quot;469769242&quot;:[65533,0],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;%1.&quot;,&quot;469777815&quot;:&quot;multilevel&quot;}\" data-aria-posinset=\"6\" data-aria-level=\"1\"><b><span data-contrast=\"none\">Do you find yourself still having questions after finishing the article?\u202f<\/span><\/b><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:160,&quot;335559740&quot;:240,&quot;335572071&quot;:0,&quot;335572072&quot;:0,&quot;335572073&quot;:0,&quot;335572075&quot;:0,&quot;335572076&quot;:0,&quot;335572077&quot;:0,&quot;335572079&quot;:0,&quot;335572080&quot;:0,&quot;335572081&quot;:0,&quot;335572083&quot;:0,&quot;335572084&quot;:0,&quot;335572085&quot;:0,&quot;335572087&quot;:0,&quot;335572088&quot;:0,&quot;335572089&quot;:0,&quot;469789798&quot;:&quot;nil&quot;,&quot;469789802&quot;:&quot;nil&quot;,&quot;469789806&quot;:&quot;nil&quot;,&quot;469789810&quot;:&quot;nil&quot;,&quot;469789814&quot;:&quot;nil&quot;}\">\u00a0<\/span><\/li>\n<\/ol>\n<p><span data-contrast=\"auto\">The final question to keep in mind once you have finished reading an article is \u201cDo you still have questions?\u201d At the end of an article, you shouldn\u2019t find yourself needing more information about the study. You might want to know more about the topic or similar research, but you shouldn\u2019t be left wondering about pieces of the research design or other methodological aspects of the study. High-quality research deserves an equally high-quality article, which includes ample information about every aspect of the study.<\/span><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559738&quot;:180,&quot;335559739&quot;:180,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p>\n<p><span data-contrast=\"auto\">While not an exhaustive list, these six questions are designed to provide a starting point for determining if research with quantitative data is of high quality. Not all research is peer-reviewed, including conference presentations, blog posts, and white papers, and simply being peer-reviewed does not make a publication infallible. It is important to understand how to critically consume research in order to successfully navigate the ever-expanding body of scientific research.<\/span><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559738&quot;:180,&quot;335559739&quot;:180,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p>\n<p><span data-contrast=\"auto\">\u200b<\/span><span aria-label=\"Rich text content control\"><span data-contrast=\"auto\">\u200b<\/span><span data-contrast=\"auto\">\u200b<\/span><\/span><span aria-label=\"Rich text content control\"><span data-contrast=\"auto\">\u200b<\/span><b><span data-contrast=\"auto\">\u00a0\u00a0\u00a0\u00a0 <\/span><\/b><b><span data-contrast=\"auto\">\u200b<\/span><\/b><\/span><b><span data-contrast=\"auto\">Additional Resources:<\/span><\/b><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:160,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p>\n<p><a href=\"https:\/\/blogs.lse.ac.uk\/impactofsocialsciences\/2016\/05\/09\/how-to-read-and-understand-a-scientific-paper-a-guide-for-non-scientists\/\"><span data-contrast=\"none\">https:\/\/blogs.lse.ac.uk\/impactofsocialsciences\/2016\/05\/09\/how-to-read-and-understand-a-scientific-paper-a-guide-for-non-scientists\/<\/span><\/a><span data-contrast=\"auto\">\u00a0<\/span><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:160,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p>\n<p><a href=\"https:\/\/statmodeling.stat.columbia.edu\/2021\/06\/16\/wow-just-wow-if-you-think-psychological-science-as-bad-in-the-2010-2015-era-you-cant-imagine-how-bad-it-was-back-in-1999\/\"><span data-contrast=\"none\">https:\/\/statmodeling.stat.columbia.edu\/2021\/06\/16\/wow-just-wow-if-you-think-psychological-science-as-bad-in-the-2010-2015-era-you-cant-imagine-how-bad-it-was-back-in-1999\/<\/span><\/a><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:160,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p>\n<p><a href=\"https:\/\/totalinternalreflectionblog.com\/2018\/05\/21\/check-the-technique-a-short-guide-to-critical-reading-of-scientific-papers\/\"><span data-contrast=\"none\">https:\/\/totalinternalreflectionblog.com\/2018\/05\/21\/check-the-technique-a-short-guide-to-critical-reading-of-scientific-papers\/<\/span><\/a><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:160,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p>\n<p><a href=\"https:\/\/undsci.berkeley.edu\/understanding-science-101\/how-science-works\/scrutinizing-science-peer-review\/\"><span data-contrast=\"none\">https:\/\/undsci.berkeley.edu\/understanding-science-101\/how-science-works\/scrutinizing-science-peer-review\/<\/span><\/a><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:160,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p>\n<p><a href=\"https:\/\/www.linkedin.com\/pulse\/critical-consumers-scientific-literature-researchers-patients-savitz\/\"><span data-contrast=\"none\">https:\/\/www.linkedin.com\/pulse\/critical-consumers-scientific-literature-researchers-patients-savitz\/<\/span><\/a><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:160,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p>\n<p><span data-contrast=\"auto\">\u200b<\/span><span aria-label=\"Rich text content control\"><span data-contrast=\"auto\">\u200b<\/span><b><span data-contrast=\"auto\">\u00a0\u00a0\u00a0\u00a0 <\/span><\/b><b><span data-contrast=\"auto\">\u200b<\/span><\/b><\/span><b><span data-contrast=\"auto\">References:<\/span><\/b><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:160,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p>\n<p><span data-contrast=\"auto\">Cohen, J. (1990). Things I have learned (So Far).\u202f<\/span><i><span data-contrast=\"auto\">The American Psychologist<\/span><\/i><span data-contrast=\"auto\">,\u202f<\/span><i><span data-contrast=\"auto\">45<\/span><\/i><span data-contrast=\"auto\">(12), 1304\u20131312. DOI: <\/span><a href=\"https:\/\/doi.org\/10.1037\/0003-066X.45.12.1304\"><span data-contrast=\"none\">10.1037\/0003-066X.45.12.1304<\/span><\/a><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559685&quot;:720,&quot;335559739&quot;:160,&quot;335559740&quot;:240,&quot;335559991&quot;:720}\">\u00a0<\/span><\/p>\n<p><span data-contrast=\"auto\">Henrich, J., Heine, S., &amp; Norenzayan, A. (2010). The weirdest people in the world?\u202f<\/span><i><span data-contrast=\"auto\">Behavioral and Brain Sciences, 33<\/span><\/i><span data-contrast=\"auto\">\u202f(2-3), 61-83 DOI:\u202f<\/span><a href=\"http:\/\/dx.doi.org\/10.1017\/S0140525X0999152X\"><span data-contrast=\"none\">10.1017\/S0140525X0999152X<\/span><\/a><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559685&quot;:720,&quot;335559739&quot;:160,&quot;335559740&quot;:240,&quot;335559991&quot;:720}\">\u00a0<\/span><\/p>\n<p><span data-contrast=\"auto\">Sheard, J. (2018). Chapter 18 \u2013 Quantitative data analysis. In K. Williamson &amp; G. Johanson (Eds.), <\/span><i><span data-contrast=\"auto\">Research Methods<\/span><\/i><span data-contrast=\"auto\">\u202f(2nd ed., pp. 429-452). Chandos Publishing.<\/span><span data-contrast=\"auto\"> DOI: <\/span><a href=\"https:\/\/doi.org\/10.1016\/B978-0-08-102220-7.00018-2\"><span data-contrast=\"none\">10.1016\/B978-0-08-102220-7.00018-2<\/span><\/a><span data-contrast=\"auto\">\u00a0\u00a0<\/span><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559685&quot;:720,&quot;335559739&quot;:160,&quot;335559740&quot;:240,&quot;335559991&quot;:720}\">\u00a0<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>By Austin Boyd\u00a0 Every measurement, evaluation, statistics, and assessment (MESA) professional, whether they \u200bare\u200b established educators and practitioners or aspiring students, engages with academic literature in some capacity. Sometimes for work, \u200b\u200bother times for pleasure, but always in the pursuit of new knowledge. But how do we as consumers of research determine whether the quantitative [&hellip;]<\/p>\n","protected":false},"author":86,"featured_media":3780,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_eb_attr":"","inline_featured_image":false,"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":"","_links_to":"","_links_to_target":""},"categories":[44],"tags":[],"class_list":{"0":"post-4071","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-utkmadblog","8":"entry"},"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.8 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>How Do I Critically Consume Quantitative Research? - Educational Leadership and Policy Studies<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/cehhs.utk.edu\/elps\/how-do-i-critically-consume-quantitative-research\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How Do I Critically Consume Quantitative Research? - Educational Leadership and Policy Studies\" \/>\n<meta property=\"og:description\" content=\"By Austin Boyd\u00a0 Every measurement, evaluation, statistics, and assessment (MESA) professional, whether they \u200bare\u200b established educators and practitioners or aspiring students, engages with academic literature in some capacity. Sometimes for work, \u200b\u200bother times for pleasure, but always in the pursuit of new knowledge. But how do we as consumers of research determine whether the quantitative [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/cehhs.utk.edu\/elps\/how-do-i-critically-consume-quantitative-research\/\" \/>\n<meta property=\"og:site_name\" content=\"Educational Leadership and Policy Studies\" \/>\n<meta property=\"article:published_time\" content=\"2024-05-01T11:00:42+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-05-13T20:22:58+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/cehhs.utk.edu\/elps\/wp-content\/uploads\/sites\/9\/2023\/10\/MADblog2-1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"560\" \/>\n\t<meta property=\"og:image:height\" content=\"315\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Jonah Hall\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Jonah Hall\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/how-do-i-critically-consume-quantitative-research\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/how-do-i-critically-consume-quantitative-research\\\/\"},\"author\":{\"name\":\"Jonah Hall\",\"@id\":\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/#\\\/schema\\\/person\\\/6bfd2c32cad7f6a411dfcf7e5dd65055\"},\"headline\":\"How Do I Critically Consume Quantitative Research?\",\"datePublished\":\"2024-05-01T11:00:42+00:00\",\"dateModified\":\"2024-05-13T20:22:58+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/how-do-i-critically-consume-quantitative-research\\\/\"},\"wordCount\":1297,\"image\":{\"@id\":\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/how-do-i-critically-consume-quantitative-research\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/wp-content\\\/uploads\\\/sites\\\/9\\\/2023\\\/10\\\/MADblog2-1.png\",\"articleSection\":[\"Evaluation Methodology Blog\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/how-do-i-critically-consume-quantitative-research\\\/\",\"url\":\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/how-do-i-critically-consume-quantitative-research\\\/\",\"name\":\"How Do I Critically Consume Quantitative Research? - Educational Leadership and Policy Studies\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/how-do-i-critically-consume-quantitative-research\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/how-do-i-critically-consume-quantitative-research\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/wp-content\\\/uploads\\\/sites\\\/9\\\/2023\\\/10\\\/MADblog2-1.png\",\"datePublished\":\"2024-05-01T11:00:42+00:00\",\"dateModified\":\"2024-05-13T20:22:58+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/#\\\/schema\\\/person\\\/6bfd2c32cad7f6a411dfcf7e5dd65055\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/how-do-i-critically-consume-quantitative-research\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/how-do-i-critically-consume-quantitative-research\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/how-do-i-critically-consume-quantitative-research\\\/#primaryimage\",\"url\":\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/wp-content\\\/uploads\\\/sites\\\/9\\\/2023\\\/10\\\/MADblog2-1.png\",\"contentUrl\":\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/wp-content\\\/uploads\\\/sites\\\/9\\\/2023\\\/10\\\/MADblog2-1.png\",\"width\":560,\"height\":315},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/how-do-i-critically-consume-quantitative-research\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"How Do I Critically Consume Quantitative Research?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/#website\",\"url\":\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/\",\"name\":\"Educational Leadership and Policy Studies\",\"description\":\"University of Tennessee, Knoxville\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/#\\\/schema\\\/person\\\/6bfd2c32cad7f6a411dfcf7e5dd65055\",\"name\":\"Jonah Hall\",\"url\":\"https:\\\/\\\/cehhs.utk.edu\\\/elps\\\/author\\\/jhall152\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"How Do I Critically Consume Quantitative Research? - Educational Leadership and Policy Studies","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/cehhs.utk.edu\/elps\/how-do-i-critically-consume-quantitative-research\/","og_locale":"en_US","og_type":"article","og_title":"How Do I Critically Consume Quantitative Research? - Educational Leadership and Policy Studies","og_description":"By Austin Boyd\u00a0 Every measurement, evaluation, statistics, and assessment (MESA) professional, whether they \u200bare\u200b established educators and practitioners or aspiring students, engages with academic literature in some capacity. Sometimes for work, \u200b\u200bother times for pleasure, but always in the pursuit of new knowledge. But how do we as consumers of research determine whether the quantitative [&hellip;]","og_url":"https:\/\/cehhs.utk.edu\/elps\/how-do-i-critically-consume-quantitative-research\/","og_site_name":"Educational Leadership and Policy Studies","article_published_time":"2024-05-01T11:00:42+00:00","article_modified_time":"2024-05-13T20:22:58+00:00","og_image":[{"width":560,"height":315,"url":"https:\/\/cehhs.utk.edu\/elps\/wp-content\/uploads\/sites\/9\/2023\/10\/MADblog2-1.png","type":"image\/png"}],"author":"Jonah Hall","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Jonah Hall","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/cehhs.utk.edu\/elps\/how-do-i-critically-consume-quantitative-research\/#article","isPartOf":{"@id":"https:\/\/cehhs.utk.edu\/elps\/how-do-i-critically-consume-quantitative-research\/"},"author":{"name":"Jonah Hall","@id":"https:\/\/cehhs.utk.edu\/elps\/#\/schema\/person\/6bfd2c32cad7f6a411dfcf7e5dd65055"},"headline":"How Do I Critically Consume Quantitative Research?","datePublished":"2024-05-01T11:00:42+00:00","dateModified":"2024-05-13T20:22:58+00:00","mainEntityOfPage":{"@id":"https:\/\/cehhs.utk.edu\/elps\/how-do-i-critically-consume-quantitative-research\/"},"wordCount":1297,"image":{"@id":"https:\/\/cehhs.utk.edu\/elps\/how-do-i-critically-consume-quantitative-research\/#primaryimage"},"thumbnailUrl":"https:\/\/cehhs.utk.edu\/elps\/wp-content\/uploads\/sites\/9\/2023\/10\/MADblog2-1.png","articleSection":["Evaluation Methodology Blog"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/cehhs.utk.edu\/elps\/how-do-i-critically-consume-quantitative-research\/","url":"https:\/\/cehhs.utk.edu\/elps\/how-do-i-critically-consume-quantitative-research\/","name":"How Do I Critically Consume Quantitative Research? - Educational Leadership and Policy Studies","isPartOf":{"@id":"https:\/\/cehhs.utk.edu\/elps\/#website"},"primaryImageOfPage":{"@id":"https:\/\/cehhs.utk.edu\/elps\/how-do-i-critically-consume-quantitative-research\/#primaryimage"},"image":{"@id":"https:\/\/cehhs.utk.edu\/elps\/how-do-i-critically-consume-quantitative-research\/#primaryimage"},"thumbnailUrl":"https:\/\/cehhs.utk.edu\/elps\/wp-content\/uploads\/sites\/9\/2023\/10\/MADblog2-1.png","datePublished":"2024-05-01T11:00:42+00:00","dateModified":"2024-05-13T20:22:58+00:00","author":{"@id":"https:\/\/cehhs.utk.edu\/elps\/#\/schema\/person\/6bfd2c32cad7f6a411dfcf7e5dd65055"},"breadcrumb":{"@id":"https:\/\/cehhs.utk.edu\/elps\/how-do-i-critically-consume-quantitative-research\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/cehhs.utk.edu\/elps\/how-do-i-critically-consume-quantitative-research\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/cehhs.utk.edu\/elps\/how-do-i-critically-consume-quantitative-research\/#primaryimage","url":"https:\/\/cehhs.utk.edu\/elps\/wp-content\/uploads\/sites\/9\/2023\/10\/MADblog2-1.png","contentUrl":"https:\/\/cehhs.utk.edu\/elps\/wp-content\/uploads\/sites\/9\/2023\/10\/MADblog2-1.png","width":560,"height":315},{"@type":"BreadcrumbList","@id":"https:\/\/cehhs.utk.edu\/elps\/how-do-i-critically-consume-quantitative-research\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/cehhs.utk.edu\/elps\/"},{"@type":"ListItem","position":2,"name":"How Do I Critically Consume Quantitative Research?"}]},{"@type":"WebSite","@id":"https:\/\/cehhs.utk.edu\/elps\/#website","url":"https:\/\/cehhs.utk.edu\/elps\/","name":"Educational Leadership and Policy Studies","description":"University of Tennessee, Knoxville","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/cehhs.utk.edu\/elps\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/cehhs.utk.edu\/elps\/#\/schema\/person\/6bfd2c32cad7f6a411dfcf7e5dd65055","name":"Jonah Hall","url":"https:\/\/cehhs.utk.edu\/elps\/author\/jhall152\/"}]}},"featured_image_src":"https:\/\/cehhs.utk.edu\/elps\/wp-content\/uploads\/sites\/9\/2023\/10\/MADblog2-1.png","featured_image_src_square":"https:\/\/cehhs.utk.edu\/elps\/wp-content\/uploads\/sites\/9\/2023\/10\/MADblog2-1.png","author_info":{"display_name":"Jonah Hall","author_link":"https:\/\/cehhs.utk.edu\/elps\/author\/jhall152\/"},"_links":{"self":[{"href":"https:\/\/cehhs.utk.edu\/elps\/wp-json\/wp\/v2\/posts\/4071","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cehhs.utk.edu\/elps\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cehhs.utk.edu\/elps\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cehhs.utk.edu\/elps\/wp-json\/wp\/v2\/users\/86"}],"replies":[{"embeddable":true,"href":"https:\/\/cehhs.utk.edu\/elps\/wp-json\/wp\/v2\/comments?post=4071"}],"version-history":[{"count":0,"href":"https:\/\/cehhs.utk.edu\/elps\/wp-json\/wp\/v2\/posts\/4071\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cehhs.utk.edu\/elps\/wp-json\/wp\/v2\/media\/3780"}],"wp:attachment":[{"href":"https:\/\/cehhs.utk.edu\/elps\/wp-json\/wp\/v2\/media?parent=4071"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cehhs.utk.edu\/elps\/wp-json\/wp\/v2\/categories?post=4071"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cehhs.utk.edu\/elps\/wp-json\/wp\/v2\/tags?post=4071"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}