After checking the overall student counts, see the breakdown of these students over the terms of data submitted. This table will allow you to see which terms may have limited data and supplement these terms with additional data before proceeding with impact analysis.
For each included term, you can review:
- Term and Term ID: The name of the term the initiative was offered (e.g. Winter 2016) and the term ID used in your institution's internal systems.
- % of Student IDs Found: Out of the students submitted in your uploaded list, the percentage of student records identified in the Civitas data set collected from your systems (SIS, LMS, etc.).
- Initiative Participants: The number of participants identified from the student list who participated during the indicated term.
- Eligible Comparison Students: The number of eligible students who did not participate identified from the student list who were eligible during the indicated term.
If the number of identified eligible comparison students for a specific term is less than 80% of the number of identified participants, continuing would result in at least 20% of the participants being left out of analysis, which could skew results. If this is the case for one of your initiatives, you'll see a warning in red beneath the term:
The lack of eligible comparison students in this term could be an issue.
Click 'Let's Fix This' to the right of the term details to add eligible comparison students.
You will see a message introducing the options for fixing the term:
We can automatically add the eligible comparison students we recommend to this term. Would you like us to do this, or choose additional eligible comparison students yourself?
Select from these three options:
- Yes, add for me: Illume Impact will automatically add students who met the eligibility criteria during other same-season terms. For example, if Fall 2014 triggered this warning, eligible comparison students will be added from other Fall terms when the initiative was offered. Same-season terms are used to limit confounding factors that could influence analysis, as Fall terms look more similar to other Fall terms than other season terms across institutional operations, student populations, and student outcomes.
- No, I'll choose: You will add other terms to include in the eligible comparison group. If you specified the eligible comparison group in your student list file, only the terms included in that file will appear as options. If you did not specify the eligible comparison group, all terms within the last four years that are available in your institution's Civitas data set will appear as options. Click the checkbox to the left of any available term you want to include in the eligible comparison group and finalize your choices by clicking the Add students button.
- Ignore this term: This term will be excluded from impact analysis. Excluding a term will result in all student records associated with that term being left out of the analysis. If there are many terms of data and a large population of students available for comparison, this may be fine. If there are very few terms and a small set of students, excluding a term could skew impact analysis results.
If you attempt to ignore all available terms, you will not be able to continue with impact analysis. You will be prompted to edit your selections or upload a new list before continuing.
You can continue impact analysis without fixing the indicated terms. However, this is not recommended as the number of students analyzed for the indicated term(s) will be greatly reduced, which could cause inaccurate analysis.
When you have reviewed the student counts for each term the initiative was offered, click Continue to finish data validation.