Customizing Diagnostic reports allows users to investigate the academic growth of groups of students at varying achievement levels for groups of students they select or choose. LEAs/districts and schools can use the filtering options in the diagnostic report to determine and identify patterns or trends of academic growth among students with similar educational opportunities and experiences.
CAUTION: The Diagnostic reports are NOT designed for accountability purposes; they are intended to be used for diagnostic purposes only. This report should be used for the purposes of continuous improvement of professional practice. Any attempt to create, interpret or include a customized diagnostic report for the purposes of a teacher's summative evaluation would be an inappropriate and invalid use of the PVAAS data.
When using a customized Diagnostic report, it is important to consider the students you are selecting. The criteria for the selection of students are defined by you as the user and should be established before selecting the students. These criteria will assist you in interpreting the resulting report.
Remember, this report only serves as one indicator; additional indicators should also be used when exploring the influence of educational programs on student performance. Additionally, care should be taken in using this report, as it does not imply direct causal relationships between any educational variables and student growth. For example, you may have two Math classes – one class in the morning and one class in the afternoon. We may see from generating customized Diagnostic reports that the students in the morning class tend to have more growth than students in the afternoon class. We cannot say, however, that taking the class in the morning "causes" better growth. There are many possible root causes for student outcomes. We can say, however, that there appears to be a relationship between academic growth and when students take the class - morning or afternoon. We would then need to look at the many factors that may contribute to that relationship. Were different instructional strategies used earlier in the day versus later in the day? Was instruction adapted later in the day based upon what was learned from teaching students earlier in the day? Are students being pulled out from classes earlier in the day for various activities? These, as well as other questions, would need to be investigated to get at all possible root causes.
When choosing the criteria for selecting students, you are encouraged to carefully think about the educational programming or opportunities that are being investigated. The clearer the criteria are defined, the clearer the interpretation of the results.
Consider the example of a school that offers tutoring in Math and wishes to select only the 46 students who received tutoring services during the school year. The resulting customized Diagnostic report (graph and table) is seen below. If all 46 students received tutoring support and had similar educational experiences within the classroom, this report indicates that the students in the Lowest achievement group had the largest gains as compared to the students in the Low-Mid and Middle achievement groups. The diagnostic growth colors (shown in the table) indicate students in the Lowest achievement group exceeded the growth standard (i.e., gained ground), while students in the other achievement groups met the growth standard (i.e., maintained).
The educators in this school may wish to look at the list of 23 students in the Lowest achievement group, and ask themselves questions such as:
Did those students have a different tutoring experience than the other 23 students? Or, did all students receive the same tutoring experience that varied in its impact on student growth, depending upon the entering achievement of the student?
Were different instructional strategies used with the 23 students in the Lowest achievement group than with the other 23 students? Or, were the same instructional strategies used with all students, producing different amounts of growth, depending upon the entering achievement of the student?
Did the 23 students in the Lowest achievement group have a need in a specific assessment anchor or in standard(s) that were different from the other 23 students? Or, were the standards or assessment anchors that were taught in the tutoring program more appropriate to the needs of the students in the Lowest achievement group as compared to those in the Low-Mid or Middle achievement groups?
How many hours of tutoring did students receive? Was there large variability in those hours?
Depending on the answers to those questions and the number of students selected, the educators may wish to refine their list and create a new customized Diagnostic report with further defined criteria. For example, they may decide to look further at only students receiving 45 hours or more of tutoring instruction.
Following the example above, the school now creates a customized Diagnostic report for only the 32 students who received a minimum of 45 hours of tutoring. The resulting report (graph and table) can be seen below. From this further customized report, we can clearly see that all three achievement groups of students who participated in at least 45 hours of tutoring (Lowest, Low- Mid, and Middle) exceeded the growth standard (i.e., gained ground).
These two reports combined would provide additional evidence to the educators in this school that not only is the Math tutoring benefitting students, but also the largest impact for growth is occurring for those students who participate in tutoring services for a minimum of 45 hours during the school year.
REMEMBER: When interpreting this report, NO causal relationship between growth and educational variables can be inferred from this report. This report is intended to serve as one indicator.