Finding Fit: A Statistical Journey
By: Sara Hall
As a graduate student in Evaluation, Statistics, and Measurement, I’ve learned a thing or two about fit. Not just in terms of statistical models, but in my own academic journey and beyond. Life is kind of like running one big regression on your choices – sometimes, the model explains everything and other times is all error terms and cold coffee. Somewhere in between lies the essence of goodness of fit. In this blog post I will take you through my experience of finding the right graduate program, using some statistical concepts to illustrate my process.
The Initial Model: Leadership & Decision-Making
Two years ago, I began my graduate studies in a Leadership and Decision-making program. I had been out of the academy for 10 years. I had a successful career in sales, children old enough to reach the microwave, and a supportive group of friends that could help with childcare as well as navigating graduate school. A good friend and former colleague was teaching quantitative and qualitative analysis and methodology in a Leadership program. He encouraged me to apply with the promise that we would be working together again, and I could pursue my research interests with his support. For as long as I can remember, I have wanted to teach and do research. The timing was perfect, and this seemed like the best opportunity as a non-traditional student to at least get a PhD to teach and do research, even if not exactly in my field of interest. Two things are relevant to note:
- My research and career goals do not include a focus on leadership and decision-making.
- My friend accepted a position (a much better fitting one, see what I did there?) at a different university a week before classes started.
In statistics, we often talk about “goodness of fit” – how well a model describes a set of observations. My program choice was a model that looked great on the spreadsheet but failed to capture the nuances of my data, in this case, my interests and career goals. I was dealing with a poor model fit. The residuals – the differences between what I expected and what I experienced, only grew larger as the months turned into years. I was determined to see through and complete my degree, but my frustration was palpable. I was trying to fit a curvilinear model to a linear relationship. My R-squared value was disappointingly low.
Example of Poor Goodness of Fit
Notice in this exaggerated example, a curve is inappropriately used to fit a clearly linear data pattern. The strong positive linear pattern of the data points suggests that as program value increases goal opportunity also increases. The fitted curve completely misses the underlying pattern, indicating poor model fit. The R-squared value indicates the model explains none of the variance and performs about 33 times worse than if the prediction was simply based on the mean.
Reassessing the Model: Searching for a Better Fit
Just as we refine our statistical models when they fail to adequately explain our data, I concluded I needed to reassess my academic path. The final straw was being told that theory was less important than application while I was working feverishly to map a theory of identity deconstruction that could be generalized to various populations for use in clinical settings. As atheoretical methodologist who values the balance of theory and action, it was a kick in the gut. Turns out it was just what I needed. I began talking to friends whose interests aligned with mine, reaching out to professors and mentors for advice, and really challenging myself to think through what I wanted to do with my scholarly pursuits and the potential consequences of leaving my current program. From there I began looking at different programs and creating my own information criteria (I heart Bayes!). In the same way residuals reflect the gap between outcomes and predictions, or expectations and experiences in my case, I wanted to minimize the residuals in my decision-making by selecting a program the most closely aligned with my personal and professional aspirations. I developed a framework, inspired by statistical concepts like Bayesian information criteria, to create four dimensions of alignment that were of critical importance to my decision to change programs (research, faculty, career, academic). I then used this information set to evaluate and compare the different programs based on how well they matched my interests, goals, and priorities. In this context, I viewed each program as a distinct model where specification defined how the four dimensions of alignment (research, faculty, career, and academic priorities) interact and contribute to program fit.
Here is a link to a tutorial providing the steps necessary to run Bayesian goodness-of-fit testing for regression models using R developed by Andres F. Barrientos and Antonio Canale.
The New Model: Evaluation, Statistics, & Measurement (ESM)
The ESM program immediately stood out to me. The classes were intriguing, the faculty profiles contained research focuses I wanted to explore, and the career options were many I could see myself enjoying. Specifically, the focus on creating applied learning experiences grounded in atheoretical foundation aligned well with my personal approach to both teaching and learning. I met with faculty who echoed my values while also piquing my curiosity for subject matter I had not previously considered exploring. I wanted to learn from them and I felt I could contribute positively to the program. After careful consideration, I chose to make the switch to ESM. The difference was immediately apparent – it was like finding a model with an excellent fit! I had a well-specified model, capturing the complexity of my academic aspirations without over or under fitting. The residuals between my expectations and experiences shrunk. The Radar Chart to the right compares the two programs across five dimensions of important considerations when choosing a Graduate program. The Evaluation program consistently scores higher across all dimensions, indicating better alignment with the important considerations than the Leadership program.
The Importance of Fit
Good statistical models strike a balance between simplicity and explanatory power. ESM provided the right balance of theory and application for me. Finding the right graduate program is a lot like fitting a statistical model. Graduate school is a continuous process of adaptation requiring careful analysis and sometimes, a willingness to start over. Changing programs can be a hard decision but we shouldn’t force ourselves to fit into programs that don’t align with our goals and expectations for our educational experience. My journey to ESM is a reminder that it is okay to reassess, to look for a better fit, and to make changes. Both life and regression analysis are iterative processes in which goodness of fit can influence the predicted outcomes. It is important to reflect on our experiences and take action when adjustments need to be considered. I encourage you to reflection how you define success in your graduate journey and ask, does your current path align with that definition? To hold yourself accountable try setting specific goals at the start of each semester and revisit them mid-way through to make modifications if necessary. In both statistics and graduate school, the end game is not just to find any fit, but to find the best fit. When you do, the adjusted R-squared of your experience will be higher and so will your confidence in achieving your vision of your future.
Whether you are just starting to consider graduate school, evaluating your goodness of fit in a current program, or just wanting to reflect, this YouTube video, Picking the Graduate Program that is Perfect for You, by Dr. Sharon Milgram is full of helpful advice and considerations.
About the Author
I am a current graduate student in the ESM program. My research interests include identity deconstruction and evaluating the use of AI in higher education. I love all things methodology and have passion for factor analysis.