Using Student Application Essays to Understand Student Engagement in the Beedie Full-time MBA Program: Computerized Text Analysis

Grant program: Teaching and Learning Development Grant (TLDG)

Grant recipient: Leyland Pitt, Beedie School of Business

Project team: Sarah Lord Ferguson, research assistant

Timeframe: November 2018 to February 2019

Funding: $6000

Final report: View Leyland Pitt's final report (PDF)

Description: The majority of graduate business schools that offer full time MBA (FTMBA) programs require prospective students to complete applications that provide insight into their suitability or otherwise for the course of study. Included in these application packages would be selection variables such as scores on selection tests (e.g., GMAT, GRE), GPA, letters of reference, resumes, and an essay by the applicant. Selection tests and previous academic performance provide objective guides to admission decision makers, and letters of reference ideally should provide an independent assessment of candidates. Assessing essays will always be subjective, and while it might be possible to rank, rather than rate them, evaluating them is always a time-consuming, resource intensive task.

The overall objective of the research proposed in this project is to investigate whether the essays from all applicants to the Beedie FTMBA might be used to better inform admission decisions, and enhance the quality of the student intake and the student experience. We plan to use the text analysis tool LIWC (Linguistic Inquiry and Word Count). Originally designed to discover which features of writing about negative life experiences could predict subsequent health improvements (Pennebaker & Francis, 1996), the tool’s use has since been expanded to analyze text in sources as varied as classical literature, personal narratives, press conferences, and transcripts of everyday conversations (Pennebaker & Graybeal 2001) and also student application essays (Pennebaker, Chung, Frazee, Lavergne, & Beaver, 2018).

For the purposes of this project, we focus on four summary variables that LIWC calculates for a piece of text, and from an analytical perspective, we will compare to the following for statistically significant differences: Successful versus unsuccessful applicants; male versus female applicants; local versus foreign students. We will also examine correlations between LIWC data and GMAT scores for all applicants, and GPAs of accepted students.

In summary then, this project will analyze all of the essays submitted by applicants to the Beedie FTMBA over the past ten years. The scores on the LIWC summary variables for each applicant will then be used as either predictor or dependent variables in a series of comparisons. In this way, we hope to turn what is merely a simple application requirement (the essay) into a tool that can guide admission decisions far more effectively, and be used as information that can inform program managers identify and assist students who may need additional help.

References:
Pennebaker, J. W., Chung, C. K., Frazee, J., Lavergne, G. M., & Beaver, D. I. (2014). When small words foretell academic success: The case of college admissions essays. PloS one, 9(12), e115844.

Pennebaker, J. W., & Francis, M. E. (1996). Cognitive, emotional, and language processes in disclosure. Cognition & Emotion, 10(6), 601-626.

Pennebaker, J. W., & Graybeal, A. (2001). Patterns of natural language use: Disclosure, personality, and social integration. Current Directions in Psychological Science, 10(3), 90-93.

Questions addressed:

• Are there significant differences between the essays of successful and unsuccessful applicants to the Beedie FTMBA program on the four LIWC summary variables?
• Are there significant differences between the essays of male and female applicants to the Beedie FTMBA program on the four LIWC summary variables?
• Is there a significant relationship between GMAT/GRE scores of all applicants to the Beedie FTMBA program and their scores on the four LIWC summary variables?
• Is there a significant relationship between final FTMBA GPA scores of all successful applicants to the Beedie FTMBA program and their scores on the four LIWC summary variables?

Knowledge sharing: Findings will be shared with colleagues at a small informal seminar led by the research assistant, and PhD students will be encouraged to attend. One of the RAs will submit and present a paper at the Academy of Marketing Conference in London, UK in July 2019, which has a conference track specifically dedicated to management education. This will not preclude the subsequent preparation and submission of a journal article to a management education journal such as the Journal of Management Education.

Keywords: Admissions essays, content analysis