Keep It Simple: Explain PVAAS in Everyday Language

Just like you do not need to be a mechanic to drive a car, you don't need to be a statistician to use PVAAS! This document provides some clear and brief explanations of statistical terms and concepts related to PVAAS, to increase your confidence and understanding as you examine, discuss, and use the reports.

PVAAS Growth Color Indicators

PVAAS growth colors help us interpret a PVAAS growth, or value-added measure – did the student group maintain, gain, or fall behind in terms of their achievement?

DARK BLUE or LIGHT BLUE – the group of students gained or exceeded the growth standard
GREEN – the group of students maintained their achievement or met the growth standard
YELLOW or RED – the group of students fell behind or did not meet the growth standard

The colors help us know if our schools and our academic programs are "adding value" to the students we teach, to the "raw materials" that walked in our door. The five PVAAS growth color indicators are the same at the LEA/district, school, and teacher levels.

Growth Standard Methodology versus Predictive Methodology

In Pennsylvania, growth is measured in two ways – using the growth standard methodology or the predictive methodology. The Growth Standard Methodology is used when we test students every year, e.g., ELA and Math in grades 4-8. The Predictive Methodology is used for subjects and grades when we don't test students every year, e.g., Science and Keystones.

The process of measuring growth is conceptually similar:

Both entering achievement and exiting achievement calculations for a group of students are made using the student group's entire achievement history, across grades and subjects.
Using both the entering achievement and exiting achievement for the student group, we can calculate a growth, or value-added, measure for that group. PVAAS growth colors help us interpret the growth, or value-added measure – did the student group maintain, gain, or fall behind in terms of their achievement?

DARK BLUE or LIGHT BLUE – the group of students gained
GREEN – the group of students maintained their achievement
YELLOW or RED – the group of students fell behind (or "slipped")

Normal Curve Equivalents (NCEs)

NCEs allow for comparison between tests that do not have the same scales for scores.

PSSA tests do NOT have the same scale across years, grades, and subjects – for example, a 1300 on a PSSA Grade 4 Math test does not mean the same thing as a 1300 on a PSSA Grade 5 Math test.
You cannot do mathematical calculations with scale scores or percentiles; in other words, it is not appropriate to subtract or calculate an average of scale scores or percentiles.
You can do math with NCEs; in other words, it is appropriate to subtract and calculate an average of NCEs.
You can measure growth using differences in NCEs but cannot measure growth using differences in percentiles or scale scores.

Quintiles (found on PVAAS Launchpad and Diagnostic reports)

Quintiles are achievement groups. All students statewide are rank ordered from lowest to highest in terms of achievement. Students are then divided into five equal groups, called quintiles, with 20% of students statewide in each of the five quintile groups, with group 1 being the lowest 20% in the state and group 5 being the highest 20% in the state.

Statewide Data shown in Quintiles

In simplistic terms, we can think of these five quintile achievement groups as:

Group 1 – well below average achievement
Group 2 – below average achievement
Group 3 – average achievement
Group 4 – above average achievement
Group 5 – well above average achievement

Diagnotistic Report Pie Chart Consider the pie chart version of the Diagnostic reports, as they appear on the Launchpad.

The five slices of the pie chart represent the five quintile achievement groups described above for your LEA/district or school.
The SIZE of the slices of the pie chart tells you the proportion of students in YOUR school or LEA/district in that achievement group. The bigger the slice of the pie, the more students in that achievement group in your LEA/district or school.
The COLOR of the slices of the pie chart tells you about the growth that group of students demonstrated, with BLUE indicating students gained, GREEN indicating students maintained their achievement, and YELLOW indicating students fell behind.

Standard Errors

When we measure anything in the real world, there will always be some kind of natural "error" around that measure – this is called the standard error. When we measure student achievement, this same thing happens. Natural error exists around any student assessment that measures achievement.

Take the example of weighing yourself on the bathroom scale. Depending on how straight you stand or where your feet are placed on the scale, your weight could differ slightly with each measurement. Each of these weights would be a good estimate of your actual weight and would give you a good indicator of what you weigh right now. Yet, there could be a pound or two each way in terms of the differences in each of the measurements you may take. That one or two pounds is called the standard error.

As an educator, and perhaps even as a parent, you probably notice this when receiving the PSSA and Keystone report on your student. A range is provided indicating what the student might receive if s/he took an equivalent version of the test. (In other words, the student took a test that covered exactly the same content but included a different set of questions.) It indicates that the score could go up or down by a certain amount – the standard error.
All PVAAS growth measures also have standard errors. We assume in PVAAS that all student groups will maintain their achievement (i.e., GREEN in PVAAS), unless we have enough data or evidence to say that the group gained or fell behind. We use the standard error to help us determine if we have enough data or evidence to say the student group did not maintain their achievement. In other words, we use the standard error in determining the PVAAS growth color indicator.

Average Growth Index

The Average Growth Index (AGI) is a reliable measure of growth that tells us whether a group of students maintained, gained, or fell behind in terms of their achievement.

The AGI is calculated by taking a PVAAS growth measure and dividing by its standard error.
The AGI is a growth value that takes into account the amount of growth a student group has made as well as the amount of data or evidence we have in calculating growth for that group of students.
While a PVAAS growth measure CANNOT be compared across LEAs/districts, schools, teachers, grades, subjects, or years, the AGI CAN be compared across LEAs/districts, schools, teachers, grades, subjects, and years.

Predicted Scores versus Student Projections

These two terms are similar and are often mistakenly used for one another. However, in PVAAS, they are calculated differently and have different purposes.

Predicted scores of students are used in generating a growth measure for a group of students.

These are based on students' own testing history and on the average performance of students in the same cohort (in the same testing year).
These represent what each student is expected to score based on their own achievement and students who took the assessment in the most recent year.
Because predicted scores of students are based on the performance of other students who took the assessment in the same year, it is NOT possible to calculate the predicted score for a student prior to them taking the assessment.

Student projections are calculated for individual students and are used to project the probability of success of a student on a future assessment.

These are based on students' own testing history and on the performance of students in earlier cohorts (in prior years).
These represent how a student is likely to score if the student makes the amount of growth that was typical for students who took the assessment in the most recent year.

​Keep It Simple: Explain PVAAS in Everyday Language