Skip to main page content

• HOME
• Current
• Past issues
• Contact us
• Subscribe
• Sign up for e-Alerts
• Help

PHYS THER
Vol. 87, No. 9, September 2007, pp. 1164-1180
DOI: 10.2522/ptj.20060221

This Article Abstract Full Text (PDF) Supplemental Figures and Tables All Versions of this Article: ptj.20060221v1 87/9/1164    most recent Submit a response Alert me when this article is cited Alert me when Rapid Responses are posted Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Utzman, R. R Articles by Jewell, D. V Search for Related Content PubMed PubMed Citation Articles by Utzman, R. R Articles by Jewell, D. V Related Collections Physical Therapist Education Social Bookmarking                      What's this?

Education Special Series

Use of Demographic and Quantitative Admissions Data to Predict Academic Difficulty Among Professional Physical Therapist Students

Ralph R Utzman, Daniel L Riddle and Dianne V Jewell

RR Utzman, PT, MPH, PhD, is Associate Professor and Academic Coordinator of Clinical Education, Division of Physical Therapy, West Virginia University School of Medicine, PO Box 9226, Morgantown, WV 26506-9226 (USA)
DL Riddle, PT, PhD, FAPTA, is Otto D Payton Professor, Department of Physical Therapy, Medical College of Virginia Campus, Virginia Commonwealth University, Richmond, Va
DV Jewell, PT, PhD, CCS, FAACVPR, is Assistant Professor, Department of Physical Therapy, Medical College of Virginia Campus, Virginia Commonwealth University

Address all correspondence to Dr Utzman at: rutzman{at}hsc.wvu.edu

Submitted August 8, 2006; Accepted April 18, 2007

 
Background and Purpose: The purpose of this study was to determine whether admissions data could be used to estimate physical therapist students' risk for academic difficulty.

Subjects: A nationally representative sample of 20 physical therapist education programs provided data on 3,582 students.

Methods: Programs provided data regarding student demographic characteristics, undergraduate grade point average (uGPA), quantitative and verbal Graduate Record Examination scores (qGRE, vGRE), and academic difficulty. Data were analyzed using logistic regression. Rules for predicting risk of academic difficulty were developed.

Results: A prediction rule that included uGPA, vGRE, qGRE, age, and race or ethnicity was developed from the entire sample. Prediction rules for individual programs showed large variation.

Discussion and Conclusion: Undergraduate grade point average, GRE scores, age, and race or ethnicity can be useful for estimating student academic risk. Programs should calculate their own estimates of student risk. Academic programs should use risk estimates in combination with other data to recruit, admit, and retain students.

Top Abstract Introduction Method Results Discussion Conclusion References

 
Professional physical therapist education programs use a variety of measures of academic performance, aptitude, and affective characteristics when selecting students for admission.^1,2 Of these, grade point average (GPA) and standardized test scores are commonly used quantitative indicators.¹ In a survey of allied health education programs, physical therapist program directors ranked overall preadmission GPA and GPA in prerequisite courses as the most important factors for admissions decisions.¹ Although many authors^3–16 have studied the utility of these measures, little guidance exists on whether or how such data may be used to predict performance of individual students in physical therapist education programs. Prediction of academic performance could potentially be important because academic programs could estimate the resources that may be needed for students who eventually encounter academic difficulty. In addition, programs could consider early educational interventions for at-risk students in hopes of reducing students’ risk for academic difficulty.

Numerous authors have studied the relationships among preprofessional GPA, standardized test scores, and academic performance in professional physical therapy curricula. Some early studies examined standardized tests that are now rarely used by physical therapist education programs, such as the Allied Health Professions Admissions Test⁴ or the Scholastic Aptitude Test.⁷ Other, generally more recent, studies have examined the use of GPA and scores on different portions of the Graduate Record Examination (GRE) to predict professional program GPA^6,8,11,16,17 or raw scores on the National Physical Therapist Examination (NPTE).^6,16 Day¹⁷ conducted the only study we found that examined student data from more than one physical therapist education program. Using linear regression, Day found that preprofessional GPA, scores on the analytic portion of the GRE (aGRE), and race or ethnicity accounted for less than 20% of the variance in GPA among 522 students from 4 schools.

We found no studies that accounted for differences based on sex or ethnicity. In other fields, it has been suggested that sex or ethnic differences exist in performance on the GRE,^16,18–20 but such differences do not relate to academic performance.^21,22 Because previous studies have relied on relatively small samples, the effect of demographic characteristics, such as sex and race or ethnicity, on prediction of physical therapist student performance is unclear.

Recruiting and retaining students who are likely to succeed is an important goal for physical therapist education programs. In 2003, 12% of students admitted to physical therapist education programs did not graduate within 150% of the normally expected time frame.²³ When students struggle academically, programs may utilize more resources in providing remedial learning experiences and ensuring student competence. The purpose of this study was to investigate the utility of quantitative admissions data in predicting whether physical therapist students encountered academic difficulty during the professional academic program. Because academic programs make admissions decisions using only prematriculation data, we examined variables for which data were collected prior to students enrolling in professional programs. Data for variables obtained after matriculation (eg, professional program GPA) would potentially confound the findings for prematriculation variables. For the purposes of this study, academic difficulty was defined as being placed on probation, suspension or dismissal from the education program, or repeating courses or units due to poor academic performance.

By collecting data from a national sample of professional Master of Physical Therapy (MPT) and Doctor of Physical Therapy (DPT) programs, we examined the following questions: (1) can preadmission demographic data, GPA, and verbal and quantitative Graduate Record Examination scores (vGRE, qGRE), either in isolation or in combination, be used to predict physical therapist student risk for academic difficulty on a national scale, and (2) does the extent of prediction of academic difficulty vary by program?

Top Abstract Introduction Method Results Discussion Conclusion References

 
Program Recruitment

Physical therapist education programs were eligible for recruitment if they used the GRE in their admissions processes and if they had estimated class sizes of at least 30 students per year for the graduating classes of 2000 to 2004. We chose the years 2000 to 2004 because the most recent data were available for these years; we adjusted for "year" within the analysis to control for any variations among cohorts. Because we were interested in assessing predictors both among and within programs, we recruited only those programs that admitted at least 30 students to allow for a within-program analysis. We estimated that at least 30 students per year (for a total of 150 students) would be the minimum number required to generate reliable predictions in our within-program analysis. Given the number of variables that we examined, this number of students would allow for a minimum of 5 cases per cell in most cases.

The online directory of accredited programs compiled by the Commission on Accreditation in Physical Therapy Education²⁴ was used to identify accredited programs. Of the 209 programs accredited in 2004,²³ the online directory of accredited programs²⁴ provided profiles and contact information for 191 programs. The Internet Web sites for all 191 programs were reviewed to identify those programs that used the GRE and that reported an estimated enrollment of at least 30 students each year. Ninety-five programs were eligible.

Programs were recruited for participation using quota sampling. In quota sampling, subjects are enrolled based on the proportion of key characteristics in the population of interest.²⁵ We planned to recruit one fourth (n=24) of the 95 eligible programs. This approach allowed us to draw a sample that was representative with respect to geography and degree level. Recruiting 24 programs also would provide data on an adequate number of students for statistical analysis.²⁶

The 95 eligible programs were classified into 4 geographic categories (Northeast, South, Midwest, and West) which corresponded to US Census Bureau regions.²⁷ Within each geographic category, programs were further categorized by degree level. Programs that had conferred professional DPT degrees in the years 2000 through 2004 were classified as DPT programs. Programs that conferred professional MPT degrees in the years 2000 through 2004 were classified as MPT programs. Programs that were conferring MPT degrees in 2000 but began conferring DPT degrees during any of the next 4 years were classified as "transition" programs (Tab. 1).

View this table: [in this window] [in a new window]

Table 1. Population and Target Sample Characteristics

Within each of the 12 geographic and degree-level groups, eligible programs were assigned random numbers. These random numbers established the order in which programs would be invited to participate. The target sample included the number of programs in each category noted in the bottom panel of Table 1. In each category, programs were contacted and invited to participate until the target number of programs in that category was achieved. Overall, 19 programs declined, citing time constraints or difficulty compiling the requested information as reasons for not participating. In one of the categories, all of the eligible programs were either recruited or declined before the target number of programs was reached. Therefore, a total of 23 programs initially agreed to participate in the study. Eight of these programs required approval from their own institutional review boards. Human subjects approval was obtained from all 8 institutions using exempt criteria.

Subjects

Participating programs were asked to provide data regarding all students matriculating into the classes scheduled to graduate in the years 2000 through 2004. Three of the participating programs withdrew from the study because of difficulty compiling the needed data. The remaining 20 programs provided data on 3,585 students.

Data Collection

Academic difficulty was the dependent variable in the study. Programs were asked to indicate whether each student encountered academic difficulty in the academic program. As listed in Table 2, program directors or their designee assigned each student to 1 of 6 categories describing the student's progress in the program. We used categorical rankings of academic difficulty instead of professional program GPA because curricula, grading scales, and academic standards may vary from program to program. We believe that categorical ratings of academic difficulty reflect the range of relevant student outcomes. Other authors^10,28,29 have used similar approaches.

View this table: [in this window] [in a new window]

Table 2. Definitions and Coding for Academic Difficulty

Independent variables included undergraduate GPA (uGPA), vGRE, and qGRE. The aGRE was changed from multiple choice to an essay-based assessment in 2002.³⁰ Because most students enrolled in physical therapist education programs during the study period would have taken the older version of the aGRE, this variable was not included in the study. Student demographic data and program degree level also were recorded. These independent variables are outlined in Table 3.

View this table: [in this window] [in a new window]

Table 3. Independent Variablesa

Procedure

After agreeing to participate in the study, program directors were sent a data collection spreadsheet and instructions for completion. Each program completed the spreadsheet and returned it as a password-protected electronic mail attachment. No names, birth dates, or other data that could be used to identify individual students were returned to the investigators.

Data Analysis

Prior to analysis, uGPA was transformed by multiplying by 10, which we believe allows for a clearer interpretation of the odds ratios produced by the logistic regression. Instead of describing a change in odds for every 1.0 change in uGPA (say from a 2.0 to a 3.0), the transformed variable produces an odds ratio based on an uGPA change of 0.1. Similarly, GRE scores were transformed by dividing by 10. The verbal and quantitative portions of the GRE are scored in 10-point increments.³¹ The transformation allows for interpretation of the odds ratios based on these 10-point increments. Data screening procedures included assessment of missing data, ratio of cases to variables, and multicollinearity.^26,32

Between-program analysis.
Logistic regression procedures were used for the entire sample. Logistic regression, for our study, was useful for determining whether independent variables (eg, uGPA or GRE scores) predicted the dichotomous dependent variable of academic difficulty. The dependent variable was dichotomized into students who encountered academic difficulty (ie, academic problems, dismissal, or academic withdrawal; see Tab. 2) and those who did not (ie, graduate, nonacademic delay, or nonacademic withdrawal; see Tab. 2). We dichotomized academic difficulty because there were very few subjects in some categories, notably students who were dismissed (n=51) and students who withdrew for academic reasons (n=21).

Advice on the sample size required for logistic regression is scarce.^33,34 However, Norman and Streiner³³ recommended having 10 times as many subjects as independent variables. This is consistent with Tabachnick and Fidell,²⁶ who recommended having at least 5 subjects per cell. Our study was designed to meet or exceed these recommendations. Prior to logistic regression analysis, cross-tabulations of all combinations of nominal independent variables were performed. To avoid potential errors due to small cell sizes,²⁶ ethnicity was recoded from 7 to 5 levels (black/African American, Hispanic, white/non-Hispanic, Asian/Pacific Islander, and "other"; "white/non-hispanic" was used as the referent category). Because we used program as a control variable, the other program-level covariables (ie, institutional Carnegie classification as listed in the directory of accredited programs²⁴ and geographic region) resulted in cells with no data; thus, they could not be used in the main logistic regression analysis.

We used hierarchical logistic regression because we wanted to control for certain variables before other variables were entered into the model. Hierarchical regression requires entry of variables into the regression model in a prespecified order based on theoretical or evidence-based grounds.³⁵

Specifically, we were interested in the influence of GPA and GRE scores after controlling for potential confounding variables. First, we wanted to control for the effects of the different years in the data set, as well as the different programs and degree levels, so we entered these variables into the model.³⁶ Given that programs vary in the methods used to train students, we suspected that there was variation in the relationship between academic difficulty and GPA and GRE scores across programs, degree levels, and years. These variables are all "group-level" variables. Next, we entered variables measured at the level of the individual—student age, sex, and race or ethnicity. Studies in physical therapy^9,16,17 and other fields^22,37,38 indicate that prediction of academic performance may be influenced by these demographic factors. Lastly, we entered uGPA, qGRE, and vGRE into the model. We entered these variables as a final step because we wanted to control for the previously mentioned group-level and individual-level variables before assessing the effect of uGPA and GRE scores.

The –2 Log-Likelihood test (P<.05) was used to evaluate whether the final model represented improvement over the constant-only model.³⁴ The Hosmer and Lemeshow test (P>.05) was examined to evaluate how well the model fit the data.^26,34,39 Wald statistics and adjusted odds ratios were examined to identify the variables that contributed significantly to the prediction model.³⁴

To provide an interpretation of the ability of uGPA and GRE to predict academic difficulty after adjusting for potential confounding variables, we used receiver operating characteristic (ROC) curves. Receiver operating characteristic curves provide an indication of how well a variable discriminates between students who encountered academic difficulty and students who did not.⁴⁰ For example, if one wanted to know whether a uGPA of less than 3.0 placed a student at increased risk for academic difficulty, a ROC curve could be used to determine whether this cut-point was important for estimating risk of academic difficulty. For a more thorough description of ROC curve analysis, see Deyo and Centor.⁴¹ Receiver operating characteristic curves were plotted for those continuous variables (ie, uGPA, vGRE, qGRE, and age) that were identified as significant predictors of academic difficulty.

We used 2 strategies to develop cut-point thresholds for these variables. First, we divided uGPA, vGRE, qGRE, and age into tertiles, and then we tested the previously described model with the variables coded as tertile scores. We identified 2 cut-points for each variable because we wanted to examine the predictability for reasonably small ranges of scores for each variable (eg, uGPA of 3.0 or less versus a uGPA of 3.01 to 3.5 versus a uGPA of 3.51 or higher). If the tertile method did not result in significant differences among the 3 score ranges, the ROC curve was used to identify a single cut-point threshold that best discriminated between students who encountered difficulty and those who did not.

A final logistic regression model was fit using these discrete variables. This final model was examined for goodness of fit; main effects as well as all 2-way interactions were tested for significance. The variables contributing significantly to the final model were used to develop a prediction rule for academic difficulty.

Clinical prediction rules use clinical data to assist clinicians in making a diagnosis or predicting an outcome.^42,43 We use the term "prediction rule" rather than the term "clinical prediction rule" because, in our case, the prediction rule was designed to assist physical therapist academicians in the interpretation of applicant information.⁴³ We used the ß coefficients from logistic regression procedures to weight the importance of the variables predicting academic difficulty, an approach commonly used to develop clinical prediction rules.^44–46 Finally, we tested the prediction rule using the data from students from all programs.

Within-program analysis.
To further clarify how prediction of academic difficulty may vary by program, the analyses performed on the entire data set were repeated with data from each program. The same variables were used as in the between-program analysis, with a few exceptions. First, controlling for program was not necessary. Second, some programs had few or no students from ethnic groups other than white/non-Hispanic. In these cases, ethnicity was collapsed into fewer categories or eliminated from the analysis, as needed. Similarly, some programs graduated students with only one type of degree (MPT or DPT) during the study period, so the degree variable was eliminated for these programs. Data were analyzed using the Statistical Package for the Social Sciences (SPSS), version 12.0.^*

Top Abstract Introduction Method Results Discussion Conclusion References

 
Population Comparisons

Where possible, the characteristics of the program and student sample were compared with those of the 95 programs eligible for the study. For the remaining population comparisons, we used the total population estimates from all accredited programs²³ because population estimates for the 95 programs that met the eligibility criteria were not available.

The geographic distribution of programs in the sample was similar to that of the 95 programs eligible for the study (Supplemental Fig. 1). In 2004, 53% of accredited programs offered the DPT degree. In the study sample, 45% of the programs conferred DPT degrees in 2004 (Supplemental Fig. 2). The sample included a slightly higher proportion of private institutions than did the populations of both accredited and eligible programs (Supplemental Fig. 3). The proportion of programs in the sample housed at doctoral, master's, and specialized medical or health sciences institutions was very similar to the population of accredited programs (Supplemental Fig. 4).

The median age at admission of students enrolled in the sample programs was 23 years (range=18–52), which is similar to the median age reported by Goldstein and Gandy.⁴⁷ Women represented 65.9% of the sample. For the academic years 1999–2000 and 2001–2002, the sample had a slightly smaller proportion of women than did the population of all accredited programs (Tab. 4). The majority of students in the sample were white and not of Hispanic descent (Tab. 4). The sample had smaller proportions of African-American and Hispanic students than the population of students enrolled in all accredited programs.

View this table: [in this window] [in a new window]

Table 4. Demographic Characteristics of Enrolled Students in All Programs Compared With Sample

Table 5 presents descriptive statistics regarding quantitative admissions data for students in the sample. The mean uGPA of students in the sample was between 3.40 and 3.50 for all 5 cohorts of students, which is consistent with the mean uGPA of students admitted to all accredited programs²³ (Fig. 1).

View this table: [in this window] [in a new window]

Table 5. Quantitative Admissions Data for Students Enrolled in Sample Programsa

View larger version (21K): [in this window] [in a new window]

Figure 1. Mean undergraduate grade point average of physical therapist students by cohort (error bars indicate 95% confidence intervals).

Three of the 3,585 students in the sample were missing data regarding academic difficulty, so they were excluded from further analysis. A total of 9.6% (n=345) of the 3,582 students encountered academic difficulty (Tab. 6). The percentage of students in the sample graduating on time without difficulty was the highest for the 2000 cohort (93.2%) and decreased each year. A similar trend was seen in the population of students enrolled in all accredited programs. Figure 2 compares the percentage of students in the study who graduated on time without difficulty with the graduation rates reported by the American Physical Therapy Association (APTA).²³ For all except the 2001 cohort, there was no significant difference (95% confidence interval [CI]) between the sample and population means.

View this table: [in this window] [in a new window]

Table 6. Academic Difficulty by Cohort

View larger version (14K): [in this window] [in a new window]

Figure 2. Physical therapist student graduation rates by cohort (error bars indicate 95% confidence intervals).

Between-Program Analysis

The results of the hierarchical logistic regression indicated that age, ethnicity, uGPA, vGRE, and qGRE contributed to prediction of academic difficulty after controlling for program, cohort, degree, ethnicity, and sex (Tab. 7). Odds ratios indicate that, as uGPA decreased by 0.10 (ie, from 3.1 to 3.0), the odds of encountering academic difficulty were increased by 15%. As vGRE and qGRE decreased by 10 (ie, from 420 to 410), odds of academic difficulty were increased by 3% and 4%, respectively. Age and ethnicity also contributed significantly to the model. As age at admission increased by 1 year, odds of academic difficulty increased 10%. Odds of academic difficulty were 200% higher for students from ethnic groups other than white/non-Hispanic.

View this table: [in this window] [in a new window]

Table 7. Logistic Regression for Prediction of Academic Difficulty Using Continuous Data for Undergraduate Grade Point Average and Graduate Record Examination Scoresa

Receiver operating characteristic curves were plotted for uGPA, vGRE, qGRE, and age. The area under each curve (Tab. 8) was significantly greater than 0.5 (P<.001), indicating that these variables significantly discriminated among students who did and did not have academic difficulty. Table 9 presents the final logistic regression model after converting age, uGPA, vGRE, and qGRE into discrete variables. Based on ROC curve analyses, uGPA was recoded into 3 discrete categories (3.15 or lower, 3.16–3.49, and 3.50 or higher). Both vGRE and qGRE were recoded into dichotomous variables (vGRE: 400 or lower versus 410 or higher; qGRE: 530 or lower versus 540 or higher). In addition, race or ethnicity was dichotomized into 2 levels representing white/non-Hispanic and "other." We dichotomized the race and ethnicity variables because the students who did not fit the white/non-Hispanic category all had reasonably similar increased risks of academic difficulty (Tab. 7). Being older than 26 years of age, being from a racial or ethnic group other than white/non-Hispanic, having a uGPA of 3.15 or lower, having a vGRE of 400 or lower, and having a qGRE of 530 or lower all were associated with increased odds of academic difficulty. Having an uGPA of 3.50 or higher was associated with a reduced risk of academic difficulty. The model indicates that an interaction between age and uGPA exists: students who were older and who had low uGPA scores were at an increased risk of difficulty, as were students who were older and had higher uGPA scores.

View this table: [in this window] [in a new window]

Table 8. Receiver Operating Characteristic Curve Areasa

View this table: [in this window] [in a new window]

Table 9. Logistic Regression for Prediction of Academic Difficulty Using ROC Curve Cut-points and Data From All Programsa

Table 10 presents the individual items in the prediction rule created by rounding the ß coefficients to the nearest 0.5 from the final logistic regression model presented in Table 9. For example, students with a low uGPA (3.15 or lower) had an increased incidence of academic difficulty with a ß coefficient of 0.357. We rounded this coefficient to +0.5 for the prediction rule. The risk was even more pronounced for older students. A uGPA of 3.51 or higher had a protective effect in younger students, with a ß coefficient of –1.113. We rounded this coefficient to –1.0 for the prediction rule. Older students, however, had a higher incidence of difficulty, even when uGPA was 3.51 or higher. Each student's predicted score was calculated, and then the predicted scores were cross-tabulated with the student's actual difficulty status. The results of the combined scores from the prediction rule are shown in Table 11.

View this table: [in this window] [in a new window]

Table 10. Prediction Rule for Academic Difficultya

View this table: [in this window] [in a new window]

Table 11. Decision Rule Score by Academic Difficulty

Within-Program Analysis

In the between-program models, prediction of academic difficulty varied by program, as evidenced by the ß coefficients in Tables 7 and 9. When logistic regression was applied to data of each program individually, a variety of models emerged. After controlling for covariates, uGPA was the most consistent predictor of academic difficulty, contributing to prediction of difficulty in 12 programs (alone in 7 programs, in combination with vGRE in 3 programs, and in combination with qGRE in 2 programs). Quantitative GRE score alone contributed to prediction of difficulty in 2 programs. For 2 programs, an adequate model could not be fit. For 4 other programs, the logistic regression model included only control variables, such as age and cohort.

The logistic regression output and prediction rules for 3 selected programs are presented in Supplemental Tables 1, 2, and 3. These 3 programs were chosen to illustrate the variation in prediction across the entire sample of participating programs.

Top Abstract Introduction Method Results Discussion Conclusion References

 
Between-Program Analysis

The logistic regression analyses indicate that uGPA, vGRE, and qGRE are independently predictive of academic difficulty when controlling for program, cohort, degree level, ethnicity, and age. These results are consistent with other research in physical therapy and other fields. For instance, Huff and Fang³⁸ found that low scores on the Medical College Admission Test (MCAT) and low preadmission GPA were predictive of academic difficulty in 13,118 medical students. As preadmission GPA increased by 0.1 point, risk of academic difficulty was reduced by 6.7%.³⁸ Rosenfeld et al⁴⁸ used discriminant function analysis and found that preadmission GPA and MCAT scores correctly classified the difficulty status of 76% of the 223 students in their study. Thomas and Draugalis⁴⁹ used multiple regression analysis to develop an equation that predicted 43% of the variance in first-year GPA in a group of 159 pharmacy students; the prediction equation included GPA in prerequisite courses and scores on the Pharmacy College Admissions Test.

Day¹⁷ found that uGPA, scores on the analytic GRE, and ethnicity accounted for 20% of the variance of professional program GPA for a sample of physical therapist students. Thieman et al¹⁶ found that prerequisite course GPA, GRE scores, and age accounted for 37% of the variance in professional program GPA. Work by Day¹⁷ and Thieman et al¹⁶ support the notion that previous academic performance and GRE scores are related to performance in physical therapist professional education. The major difference is that these studies were conducted on relatively small samples. Our study appears to be the first study to suggest that GPA and GRE scores provide reasonably stable population-based estimates of risk. Substantial program-to-program variation exists, however, which suggests to us that individual programs should generate their own risk estimates.

We developed a prediction to identify students who are at risk for academic difficulty. In this sample, nearly 40% (95% CI=32.3–45.1) of students with high prediction rule scores (scores of 2 or higher) encountered academic difficulty. Using this rule would allow early identification of students at risk, enabling programs and students to develop strategies to maximize the chances for student success. Academic programs interested in using this prediction rule should recognize that "program" was an important predictor in our study. The most accurate prediction rule for an individual program would be developed using data collected from that program. In lieu of a program developing its own prediction rule, we suggest the prediction rule reported in Table 10 as a reasonable, though less accurate, alternative.

Influence of Covariates on Academic Performance

Program was an important covariate in the prediction of academic difficulty. There is no prescribed curricular format for physical therapist education programs,⁵⁰ so curricular variation may affect the prevalence of academic difficulty at individual programs. Programs also are responsible for setting their own policies for grading and academic performance requirements, which in turn may affect the number of students encountering academic difficulty as defined in this study. Despite these differences in curricular design and policy, vGRE, qGRE, and uGPA still were predictive of academic difficulty, which suggests that these scores are reasonably powerful predictors of academic difficulty across a broad array of program types in different geographic regions.

Because we could not control for geographic region and Carnegie classification in our main analyses, these variables were tested in separate post hoc analyses. Each of these 2 variables was entered into a single logistic regression model. The Northeast region and doctoral institutions served as the referent categories, respectively. Geographic region was associated with academic difficulty (²=32.81, P<.001). Students in the Midwest had reduced odds of academic difficulty compared with students in the Northeast (odds ratio=0.459, 95% CI=0.317–0.665). Carnegie classification also was associated with academic difficulty (²=7.38, P=.025). Students enrolled in programs at institutions classified as master's institutions had 1.3 times the odds of academic difficulty compared with students at doctoral institutions (95% CI=1.301–1.733). Students enrolled in medical or specialized health sciences institutions had 1.4 times to odds of academic difficulty compared with students at doctoral institutions (95% CI=1.076–1.952).

Student-level characteristics that were significant in the between-program logistic regression model were age and ethnicity. Age contributed to the prediction of academic difficulty (P<.001) in the first logistic regression model developed for all 3,582 students in the sample. The regression model created using dichotomized variables suggests that prediction of academic difficulty is age-dependent. A uGPA of 3.50 or higher was protective against academic difficulty; however, when interacting with age 27 years, this relationship was reversed. Students with this combination of variables had more than twice the odds of difficulty (odds ratio=2.314, 95% CI=1.097–4.881) as compared with younger students with the same GPA. These findings suggest that older students may have other challenges beyond undergraduate academic preparation that affect their performance in graduate-level physical therapist education. For instance, older students may have experienced a longer gap between undergraduate and professional education than their younger counterparts. They also may have different work or family commitments. Programs should be aware that student age is a significant predictor of academic difficulty even after adjusting for program and other student-level variables.

Other studies support the notion that age may affect performance in physical therapist education. Both the studies by Dockter⁶ and Thieman et al¹⁶ showed that age contributed to the prediction of professional program GPA in linear regression models. Hayes et al⁹ suggested that traditional measures of intellectual ability, such as uGPA and standardized test scores, may not be the best predictors of academic performance for older, nontraditional physical therapist students. In a study of 107 students in a baccalaureate physical therapist program, they found that grades in a gross anatomy course and preprofessional GPA predicted professional program GPA for younger students, whereas grades in a gross anatomy course and interview scores predicted professional program GPA for older students.

Some authors^1,2,51 have expressed concerns that GPA and test scores are not accurate predictors of academic performance for students from minority ethnic groups. Student ethnicity emerged as a significant predictor of academic difficulty. Interaction effects between race or ethnicity and the variables uGPA, vGRE, and qGRE were not detected, suggesting that the effects of race or ethnicity were independent of other variables in this study. However, fewer than 3% of the students in the sample were Hispanic or African American. Thus, the statistical power to detect significant interactions between ethnicity and the independent variables may have been limited.

The data collected do not allow inferences regarding why students in some ethnic groups had higher odds of academic difficulty. For example, the higher odds of academic difficulty for some racial or ethnic groups may be related to socioeconomic and educational factors that were not addressed by this study. Further research is needed regarding the relationships among race or ethnicity, quantitative admissions data, and academic difficulty to guide programs in the development and implementation of recruitment and retention strategies for minority students. Important disparities in the representation of some racial or ethnic minorities in physical therapy exist. For example, African Americans represent 12.3% of the US population,⁵² yet only 1.9% of physical therapist members of APTA are listed as African American.⁵³ In our opinion, resolving these types of disparities should be priority issues for academic programs.

Within-Program Analysis

For students in 12 of the 20 programs, uGPA contributed to prediction of academic difficulty. Verbal GRE score was predictive of difficulty in 3 programs, and qGRE score was predictive in 4 programs. Variability in program-specific odds ratios, ROC curve areas, and prediction rules indicates that discrimination between students who had difficulty and students who did not varied widely within programs. These results suggest that program directors should analyze data from their own students to generate program-specific thresholds and rules for prediction of academic difficulty. If the number of students at an individual program is too small to conduct the analysis, we recommend using results from the between-program analysis because they represent the "typical student" in the study sample.

Implications for Academic Programs in Physical Therapy

In 2004, Oeffner⁵⁴ suggested that uniform requirements for physical therapist education programs should be developed. That same year, the Education Section of APTA developed a list of recommended admissions requirements for physical therapist education programs.⁵⁵ These recommendations are broad in that they do not specify minimum GPA requirements, nor do they recommend any single standardized test. The Education Section's recommendations⁵⁵ state that programs should consider a variety of intellectual, nonintellectual, and demographic factors when selecting students. The current study provides strong evidence to support the use of uGPA, vGRE, and qGRE as part of students’ overall admissions portfolio.

Many physical therapist education programs use the GRE in their admissions processes, although it is a general test and is not designed specifically for physical therapist students. An admissions test with content more relevant to physical therapy may improve the ability of the test to predict which students might have difficulty completing physical therapist education. Until a better alternative is identified, this study suggests that vGRE and qGRE scores should be used by academic programs to assist in making admissions and academic training decisions. We are not suggesting that the prediction rule be used, in isolation, to decide which students should or should not be offered admission to an academic program. There was too much unexplained variance in our prediction rule to recommend that it be used for this purpose. Rather, we contend that if programs are aware of a prospective student's modest increase or decrease in risk for academic difficulty, the program will be in a position to better plan that student's educational experience and increase the likelihood of success. All programs need to balance admissions decisions with policies, program and university missions, and societal needs. It is our contention that program missions and societal needs should be given high priority. We believe that the clinical prediction rule we describe can assist programs in better utilizing their educational resources to optimize the chances of success for all students while complying with institutional missions and societal needs.

Limitations

Despite controlling for program, cohort, degree level, and student demographic characteristics, the logistic regression models accounted for only 24% of the variance in odds of academic difficulty. We suspect that a variety of variables that we did not measure explain the remaining variance. First, the analyses did not control for the rigor of students’ undergraduate education. Studies of medical students^38,56,57 have used indexes for this variable to adjust uGPA in their analyses. In addition, programs may have calculated preadmission GPAs in different ways, especially when applicants attended more than one undergraduate institution. Applicants may have taken the GRE more than once. We asked programs to report the scores they used in making their admissions decisions. Consistency in recording these data may have improved the strength of our predictive models.

Our study did not attempt to control for students’ socioeconomic status or psychosocial factors. Studies of students in other fields, such as medicine, suggest that these variables may contribute to academic performance. Fadem and colleagues³⁷ found that parental income was related to performance of 192 medical students on both the MCAT and medical licensure examinations. The authors suggested that students from poorer families may have had fewer educational advantages than their wealthier counterparts. Furthermore, students from families with fewer financial resources may be more likely to work during medical school, which may affect learning and performance.³⁷ In a study of 175 medical students, Hojat and colleagues⁵⁸ demonstrated that measures of psychosocial factors, such as anxiety, depression, locus of control, stress, and self-esteem, improved the prediction of medical school grades over the use of preadmission GPA and MCAT scores. More research of physical therapist students is needed to clarify which socioeconomic and psychosocial factors potentially contribute to students’ performance and the occurrence of academic difficulty.

Other study limitations are related to potential sampling and historical biases. Because it is not possible to determine the specific demographic makeup of the student bodies at the 95 eligible programs, we compared the sample to the population of all students at all accredited programs. Because only larger programs that use the GRE were eligible for the study, these comparisons may not be valid.

With respect to historical bias, it is possible that the results were affected by the profession's rapid transition to the DPT. However, degree level (MPT versus DPT) did not contribute to the prediction of academic difficulty. Concurrently with the transition to the DPT, the number of students applying to physical therapist education programs declined in the early part of the decade.²³ The mean uGPA of students admitted to physical therapist education programs declined slightly over the years of this study. Therefore, changes in the applicant pool may also have introduced bias. This was addressed statistically by controlling for cohort in the between-program analyses and, when possible, in the within-program analyses as well.

Recommendations for Future Research

Because large proportions of the variance in odds of academic difficulty were unexplained in the results of this study, future research should consider other program and student characteristics. A valid method for adjusting for academic rigor would be an important contribution to the program level analysis. In addition, qualitative research methods may help identify additional variables that contribute to prediction of academic difficulty. This approach also may help clarify reasons why students who are older or from minority ethnic groups are at higher risk for academic difficulty than younger students.

The study sample was drawn from a population of 95 programs that were selected for eligibility based on their size and their use of the GRE. These eligible programs represent only half of the accredited physical therapist education programs in the United States. Future research should investigate whether previous grades or other measures of intellectual ability predict academic difficulty for students in programs with smaller class sizes or that use standardized tests other than the GRE.

Top Abstract Introduction Method Results Discussion Conclusion References

 
Physical therapist education programs are charged with the task of producing graduates who can fulfill society's needs for high-quality, contemporary physical therapy care.⁵⁰ This study confirms the utility of uGPA and GRE scores as well as age and race or ethnicity in predicting difficulty among students enrolled in physical therapist programs. However, we concur with other authors^2,31,55,59 who have suggested that previous grades and standardized test scores should be considered in light of other information when making admissions decisions. Because the prediction of student performance varies by program, individual institutions should study their own data to develop guidelines for predicting students’ risk for academic difficulty.

 
All authors provided concept/idea/project design, writing, and data analysis. Dr Utzman provided data collection. Dr Utzman and Dr Riddle provided project management and fund procurement. Dr Riddle and Dr Jewell provided consultation and manuscript review.

This work was completed in partial fulfillment of requirements for Dr Utzman's PhD degree in Health Related Sciences, Medical College of Virginia Campus, Virginia Commonwealth University.

The Institutional Review Board of Virginia Commonwealth University approved this study.

This research was presented at the annual meeting of the Federation of State Boards of Physical Therapy; September 10, 2006; Portland, Ore.

This work was funded, in part, by the Federation of State Boards of Physical Therapy.

^* SPSS Inc, 233 S Wacker Drive, Chicago, IL 60606.

Top Abstract Introduction Method Results Discussion Conclusion References

 

1. Agho AO, Mosley BW, Williams AM. A national survey of current admission practices in selected allied health educational programs. J Allied Health. 1999;28:8–14.[Medline]
2. Scott AH, Chase LM, Lefkowitz R, et al. A national survey of admissions criteria and processes in selected allied health professions. J Allied Health. 1995;24:95–107.[Medline]
3. Baker J, Douphrate D, Ridley D. The use of the Revised PSB-Health Occupations Aptitude Examination as a predictor of success in physical therapy students. Journal of Physical Therapy Education. 1996;10(2):63–67.
4. Balogun JA, Karacoloff LA, Farina NT. Predictors of academic achievement in physical therapy. Phys Ther. 1986;66:976–980.[Abstract/Free Full Text]
5. Balogun JA. Predictors of academic and clinical performance in a baccalaureate physical therapy program. Phys Ther. 1988;68:238–242.[Abstract/Free Full Text]
6. Dockter M. An analysis of physical therapy preadmission factors on academic success and success on the National Licensing Examination. Journal of Physical Therapy Education. 2001;15(1):60–64.
7. Gross MT. Relative value of multiple physical therapy admissions criteria in predicting didactic, clinical, and licensure performance. Journal of Physical Therapy Education 1989;3(1):7–14.
8. Guthrie MR. Physical therapy student selection: analysis of the variables. Journal of Physical Therapy Education. Spring 1990:31–36.
9. Hayes SH, Fiebert IM, Carroll SR, Magill RN. Predictors of academic success in a physical therapy program: is there a difference between traditional and nontraditional students? Journal of Physical Therapy Education. 1997;11(1):10–16.
10. Jewell DV, Riddle DL. A method for predicting a student's risk for academic probation in a professional program in allied health. J Allied Health. 2005;34:17–23.[Medline]
11. Kirchner GL, Holm MB, Ekes AM, Williams RW. Predictors of student success in an entry-level master in physical therapy program. Journal of Physical Therapy Education. 1994;8(2):76–79.
12. Rheault W, Shafernich-Coulson E. Relationship between academic achievement and clinical performance in a physical therapy education program. Phys Ther. 1988;68:378–380.[Abstract/Free Full Text]
13. Roehrig SM. Prediction of licensing examination scores in physical therapy graduates. Phys Ther. 1988;68:694–698.[Abstract/Free Full Text]
14. Roehrig SM. Prediction of student problems in a baccalaureate physical therapy program. Journal of Physical Therapy Education. 1990;4(1):26–30.
15. Templeton MS, Burcham A, Franck L. Predictive study of physical therapy admission variables. J Allied Health. 1994;23:79–87.[Medline]
16. Thieman TJ, Weddle ML, Moore MA. Predicting academic, clinical, and licensure examination performance in a professional (entry-level) master's degree program in physical therapy. Journal of Physical Therapy Education. 2003;17(2):32–37.
17. Day JA. Graduate record examination analytical scores as predictors of academic success in four master's degree physical therapy programs. Phys Ther. 1986;66:1555–1562.[Abstract/Free Full Text]
18. Gallagher A, Bridgeman B, Cahalan C. Effect of Computer-Based Tests on Racial/Ethnic, Gender, and Language Groups. Princeton, NJ: Educational Testing Service; 2000. Report no. 96–21P.
19. Gallagher A, Levin J, Cahalan C. Cognitive Patterns of Gender Differences on Mathematics Admissions Tests. Princeton, NJ: Educational Testing Service; 2002. Report no. 96–17P.
20. Grandy J. Trends and Profiles (Second Edition): Statistics About General Test Examinees by Gender, Age, and Ethnicity. Princeton, NJ: Educational Testing Service; 1999. Report no. RR-99–16.
21. Attiyeh GM. Determinants of Persistence of Graduate Students in Ph.D. Programs. Princeton, NJ: Educational Testing Service; 1999. Report no. 95–18R.
22. House JD. Gender differences in prediction of grade performance from Graduate Record Examination scores. Journal of Psychology. 1994;128:695–698.[Web of Science][Medline]
23. American Physical Therapy Association. Fact Sheet: Physical Therapist Education Programs, 2005. Available at: http://www.apta.org/AM/TemplateRedirect.cfm?template=/CM/ContentDisplay.cfm&ContentID=23836. Accessed February 12, 2006.
24. Commission on Accreditation in Physical Therapy Education. Accredited Physical Therapy Programs, 2004. Available at: http://www.apta.org/Education/accreditation/dir_acc_PT_ed_prog. Accessed May 2, 2004.
25. Portney LG, Watkins MP. Sampling. Foundations of Clinical Research: Applications to Practice. 2nd ed. Upper Saddle River, NJ: Prentice Hall Health; 2000:113–152.
26. Tabachnick BG, Fidell LS. Logistic regression. In: Tabachnick BG, Fidell LS, eds. Using Multivariate Statistics. 4th ed. Boston, Mass: Allyn & Bacon; 2001:517–581.
27. US Census Bureau. Census Regions and Divisions of the United States, 2004. Available at: http://www.census.gov/geo/www/us_regdiv.pdf.
28. Jones RF, Vanyur S. MCAT scores and student progress in medical school. J Med Educ. 1984;59:527–531.[Web of Science][Medline]
29. Schmalz GMA. The use of pre-admission data to predict levels of success in selected allied health students. Occupational Therapy Journal of Research. 1990;10:367–376.[Web of Science]
30. GRE Analytical Writing Measure: An Asset in Admissions Decisions. Princeton, NJ: Educational Testing Service; 2004.
31. 2005–2006 Guide to the Use of Scores. Princeton, NJ: Educational Testing Service; 2005.
32. Tabachnick BG, Fidell LS. Cleaning up your act: screening data prior to analysis. In: Tabachnick BG, Fidell LS, eds. Using Multivariate Statistics. 4th ed. Boston, Mass: Allyn & Bacon; 2001:56–110.
33. Norman GR, Streiner DL. Logistic regression. In: Norman GR, Streiner DL, eds. Biostatistics: The Bare Essentials. 2nd ed. Hamilton, Ontario, Canada: BC Decker Inc; 2000:139–144.
34. Peng CJ, Lee KL, Ingersoll GM. Introduction to logistic regression analysis and reporting. Journal of Educational Research. 2002;96(1):3–14.
35. Cohen J, Cohen P. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences. Hillsdale, NJ: Lawrence Erlbaum Associates; 1983.
36. Tabachnick BG, Fidell LS. Multiple regression. In: Tabachnick BG, Fidell LS, eds. Using Multivariate Statistics. 4th ed. Boston, Mass: Allyn & Bacon; 2001:111–176.
37. Fadem B, Schuchman M, Simring SS. The relationship between parental income and academic performance of medical students. Acad Med. 1995;70:1142–1144.[Web of Science][Medline]
38. Huff KL, Fang D. When are students most at risk of encountering academic difficulty? A study of the 1992 matriculants to U.S. medical schools. Acad Med. 1999;74:454–460.[Web of Science][Medline]
39. Munro BH. Logistic regression. Statistical Methods for Health Care Research. 4th ed. Philadelphia, Pa: Lippincott Williams & Wilkins; 2000:283–302.
40. Hosmer DW, Lemeshow S. Applied Logistic Regression. 2nd ed. New York, NY: Wiley; 2000.
41. Deyo RA, Centor RM. Assessing the responsiveness of functional scales to clinical change: an analogy to diagnostic test performance. J Chronic Dis. 1986;39:897–906.[CrossRef][Web of Science][Medline]
42. Reilly BM, Evans AT. Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med. 2006;144:201–209.[Abstract/Free Full Text]
43. Wasson JH, Sox HC, Neff RK, Goldman L. Clinical prediction rules: applications and methodological standards. N Engl J Med. 1985;313:793–799.[Abstract]
44. Wells PS, Anderson DR, Bormanis J, et al. Value of assessment of pretest probability of deep-vein thrombosis in clinical management. Lancet. 1997;350:1795–1798.[CrossRef][Web of Science][Medline]
45. Solomon DH, Avorn J, Warsi A, et al. Which patients with knee problems are likely to benefit from nonarthroplasty surgery? Development of a clinical prediction rule. Arch Intern Med. 2004;164:509–513.[Abstract/Free Full Text]
46. Leung GM, Rainer TH, Lau FL, et al. A clinical prediction rule for diagnosing severe acute respiratory syndrome in the emergency department. Ann Intern Med. 2004;141:333–342.[Abstract/Free Full Text]
47. Goldstein M, Gandy J. Applicants to professional physical therapist programs in 2000. Journal of Physical Therapy Education. 2001;15(3):9–15.
48. Rosenfeld LM, Hojat M, Veloski JJ, et al. Delays in completing medical school: predictors and outcomes. Teach Learn Med. 1992;4:162–167.
49. Thomas M, Draugalis JL. Utility of the Pharmacy College Admission Test (PCAT): implications for admissions committees. American Journal of Pharmaceutical Education. 2002;66(1):47–51.
50. Commission on Accreditation in Physical Therapy Education. Evaluative Criteria for Accreditation of Education Programs for the Preparation of Physical Therapists. Alexandria, Va: American Physical Therapy Association; 2006.
51. Chapman DM. Noncognitive variables as predictors of academic success for physical therapy students. Journal of Physical Therapy Education. Fall 1989:18–21.
52. Grieco EM, Cassidy RC. Overview of Race and Hispanic Origin. Census 2000 Brief. Washington, DC: Department of Commerce, Economics and Statistics Administration, US Census Bureau; 2000.
53. American Physical Therapy Association. Cultural Competence Strategic Plan, 2002. Available at: http://www.apta.org/AM/Template.cfm?Section=Reources5&CONTENTID=14668&TEMPLATE=/CM/ContentDisplay.cfm. Accessed June 28, 2006.
54. Oeffner JK. Uniform academic prerequisites: action required. Presented at: Combined Sections Meeting of the American Physical Therapy Association; February 4–7, 2004; Nashville, Tenn.
55. Education Section of the American Physical Therapy Association. Admissions Requirements Recommendations, 2004. Available at: http://www.aptaeducation.org/Admiss_Req.pdf. Accessed March 1, 2006.
56. Koenig JA, Sireci SG, Wiley A. Evaluating the predictive validity of MCAT scores across diverse applicant groups. Acad Med. 1998;73:1095–1106.[Web of Science][Medline]
57. Swanson DB, Case SM, Koenig J, Killian CD. Preliminary study of the accuracies of the old and new medical college admission tests for predicting performance on USMLE Step 1. Acad Med. 1996;71(1 suppl):S25–S27.
58. Hojat M, Robeson M, Damjanov I, et al. Students' psychosocial characteristics as predictors of academic performance in medical school. Acad Med. 1993;68:635–637.[Web of Science][Medline]
59. Sedlacek WE. Beyond the Big Test: Noncognitive Assessment in Higher Education. San Francisco, Calif: Jossey-Bass Inc Publishers; 2004.

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

This article has been cited by other articles:

				D. L. Riddle, R. R. Utzman, D. V. Jewell, S. Pearson, and X. Kong Academic Difficulty and Program-Level Variables Predict Performance on the National Physical Therapy Examination for Licensure: A Population-Based Cohort Study Physical Therapy, November 1, 2009; 89(11): 1182 - 1191. [Abstract] [Full Text] [PDF]

				L. R. Kogan, S. M. Stewart, R. Schoenfeld-Tacher, and J. M. Janke Correlations between Pre-Veterinary Course Requirements and Academic Performance in the Veterinary Curriculum: Implications for Admissions J Vet Med Educ, June 1, 2009; 36(2): 158 - 165. [Abstract] [Full Text] [PDF]

				R. Sandstrom Commentary on "Use of Demographic and Quantitative Admissions Data ..." Physical Therapy, September 1, 2007; 87(9): 1194 - 1196. [Full Text] [PDF]

				R. R Utzman, D. L Riddle, and D. V Jewell Author Response Physical Therapy, September 1, 2007; 87(9): 1197 - 1198. [Full Text] [PDF]

				R. R Utzman, D. L Riddle, and D. V Jewell Use of Demographic and Quantitative Admissions Data to Predict Performance on the National Physical Therapy Examination Physical Therapy, September 1, 2007; 87(9): 1181 - 1193. [Abstract] [Full Text] [PDF]

This Article Abstract Full Text (PDF) Supplemental Figures and Tables All Versions of this Article: ptj.20060221v1 87/9/1164    most recent Submit a response Alert me when this article is cited Alert me when Rapid Responses are posted Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Utzman, R. R Articles by Jewell, D. V Search for Related Content PubMed PubMed Citation Articles by Utzman, R. R Articles by Jewell, D. V Related Collections Physical Therapist Education Social Bookmarking                      What's this?