We're putting together information to help you move to virtual advising, deploy nudging, get answers on data science questions, and use your solutions during COVID-19. ACCESS RESOURCES HERE. 

Support is Moving! You can submit a new ticket at support@civitaslearning.com

Welcome
Login
Open navigation

Impact analysis using prediction-based propensity score matching

Using many terms of your institution’s historical student data, the variables that are most predictive of student persistence at your institution are identified and used to create a persistence model. This model is used to deliver an individual persistence predictionfor each currently enrolled student. This prediction indicates the likelihood of a student enrolled in one term persisting to the next term at your institution and staying enrolled past your institution's census date (or add/drop period) or graduating.

When an initiative is submitted to Illume Impact, these predictive variables are used to build a propensity model specific to that initiative. This propensity model assesses a student’s similarity to initiative participants and it is powered by the extensive set of variables in the Civitas data set that have historically been predictive of persistence at your institution. This ensures that when students are matched for comparison, the variables used are the factors most likely to influence whether these students persist.

For a given initiative at your institution (e.g. Freshman Writing Center), each student who meets the eligibility criteria to participate (e.g. first year student, enrolled full time) has a persistence prediction, as well as a propensity score. The propensity score measures the likelihood of a student's participation in treatment. For impact analysis using Illume Impact, propensity scores are determined by measuring the similarity of eligible students who did not participate in the initiative to students who did participate in the initiative.

Next, students are matched on these two dimensions - persistence prediction and propensity score. This two-dimensional matching ensures that the matched pair of students is similar in both their persistence likelihoods and their propensity to participate in the initiative. Each matched pair has one student who participated in the initiative and one student who was eligible but did not participate.

After matching, the impact of the initiative can be estimated by measuring the overall difference between the persistence rate of the matched students who participated in the initiative and the persistence rate of the matched students who were eligible but did not participate in the initiative.

Did you find it helpful? Yes No

Send feedback
Sorry we couldn't be helpful. Help us improve this article with your feedback.