Industry-Based Learning Indicator for the Future Ready PA Index: Guidelines for Data Collection, Monitoring, and Reporting

(This document is not to replace the PIMS Manual for coding.)

The Pennsylvania Department of Education (PDE) is committed to providing regular updates on the implementation of its recently approved Every Student Succeeds Act (ESSA) Consolidated State Plan and the Future Ready PA Index, a comprehensive, public-facing school progress report that increases transparency around school and student group performance. This e-message is the sixth in a series to provide timely information on the transition to the Future Ready PA Index.

As outlined in previous Future Ready updates, indicators fall into three main categories:

1. Statewide Assessment Measures
2. On-Track Measures
3. College and Career Ready Measures

This e-message provides additional details and support on the Industry-Based Learning Indicator, one of six College and Career Measures.

The Industry-Based Learning Indicator is designed to evaluate how students are engaging in work- and classroom-based activities by 12th grade. Effective the 2017-18 school year, all LEAs must report in PIMS whether a 12th grader has met the requirements for the Industry-Based Learning Indicator. As with other data collections, school entities can enter and update student data for this indicator throughout the school year, up until the last collection period (typically June). Reporting requirements have not changed for students enrolled in PDE-approved Career and Technical Education (CTE) programs. Additional information regarding collection procedures, timelines, and technical instructions for submitting data are available in the 2017-18 PIMS User Manual, Vol. 1.

Table 1 illustrates the three options available to LEAs to meet the Industry-Based Learning Indicator requirements.

Table 1: Meeting the Industry-Based Learning Indicator

Frequently-asked questions and answers for the Industry-Based Learning Indicator are below.

What is the formula for calculating this indicator?

Numerator: Sum of the number of 12th graders1 who achieved one or more of the following between 7th grade and 12th grade:

• Scored competent or advanced on Industry Standards-Based Competency Assessments;
• Earned at least one industry-recognized credential; or
• Completed a work-based learning experience

Denominator: Total number of 12th graders

In addition to this calculation, the Future Ready PA Index will report on each of the following:

1. Percentage of 12th graders in approved CTE programs who score competent or advanced on NOCTI/NIMS assessments
2. Percentage of 12th graders in approved CTE program as well as non-CTE 12th graders who earn at least one industry recognized credential
3. Percentage of 12th graders who complete a work-based learning experience

How will school performance be described and displayed on the Future Ready PA Index for the Industry-Based Learning Indicator?

For 2017-2018, this indicator will be displayed as an informational indicator, and will only be reported as a percentage of students who attained the target divided by the total number of students in the cohort.

Starting in 2018-2019, Performance Standards will be developed using the 2017-2018 data as a baseline to measure expected school performance levels. Results will be displayed as follows:

• BLUE for a school that exceeds the Performance Standard
• GREEN for a school that meets the Performance Standard
• RED for a school that does not meet the Performance Standard

• An upward ARROW for an increase in performance from the previous year
• A solid CIRCLE for maintaining the same performance from the previous year
• A downward ARROW for a decrease in performance from the previous year

Advanced on Industry-Based Competency Assessments (NIMS and NOCTI)

Beyond proficiency calculations, it is important to recognize advanced levels of academic achievement. The Department will calculate the percentage of 12th graders scoring advanced on Industry Standards Based Competency Assessments. This is the same measure as is included in the Industry-based Learning Standards, but only for those students who scored at an Advanced level.

Results will be reported as a percentage of students who attain advanced on the assessment.

Where can my school find additional information about industry-based learning experiences and the data collection requirements for reporting purposes?

PDE has provided an Industry-Based Learning Indicator Guidance Document below to assist schools in providing student work-based learning activities, as well as data reporting and monitoring requirements. PDE will develop additional resources made available on PDE’s SAS portal.

Introduction

Pennsylvania’s economic future depends on having a well-educated and skilled workforce prepared to meet the current and projected demands of a global, knowledge-based 21st century economy. To help meet this challenge, the Pennsylvania Department of Education (PDE) is committed to ensuring all students in the commonwealth have access to high-quality learning opportunities that prepare them for meaningful engagement in postsecondary education, in workforce training, in career pathways, and as responsible, involved citizens.1 Regardless of their individual postsecondary plans, all students should graduate from high school with the knowledge, skills, and experiences needed to succeed in the workforce.

Industry-based learning experiences vary in structure, scope, and intensity with the common goal of providing students with the opportunity to connect academic and technical skills to real-world settings. Examples of industry-based learning include site visits, job shadowing, paid and unpaid internships, job training, job shadowing, mentorships, service learning, apprenticeships, or paid employment, among others.

High-quality industry-based learning experiences that are aligned to challenging academic and technical instruction have demonstrated positive short- and long-term impacts on students, helping them develop and enhance their career awareness, skills, and abilities while preparing for meaningful careers.2 Research also suggests that work-based learning experiences are a strong predictor of adult employment success for students with disabilities.3

As part of efforts to improve alignment between education and the needs of business and industry, PDE has worked with partners to highlight recommended strategies and considerations for local education agencies (LEAs) – including school districts, charter schools, and career and technical centers (CTCs) – looking to start or expand work-based learning programs for students.4 The Department has also provided assistance to LEAs to ensure that these initiatives and activities are aligned to Pennsylvania’s Career Education and Work (CEW) Standards, established as required education for all students by the State Board of Education in 2006.5

Purpose

To underscore the importance of connecting students with experiences that connect academic and technical education with workforce demands, the Department included a measure of students’ Industry-based Learning experiences as part of the Future Ready PA Index, the commonwealth’s public-facing school progress report. 

Effective the 2017-18 school year, school entities report student-level data for the Industry- Based Learning Indicator in the Pennsylvania Information Management System (PIMS).6 This document provides guidance for data collection, monitoring, and reporting students for this indicator.

NOTE: PIMS reporting requirements for PDE-approved CTE programs has not changed.

Background

The Industry-Based Learning Indicator identifies the percentage of graduates who demonstrate meaningful engagement in exploration and preparation in industry-based technical skills within grades 7 through 12. The indicator is calculated based on the total number of 12th graders who meet at least one of the following criteria:

• Score competent or advanced on Industry Standards-Based Competency Assessments;
• Earn at least one industry-recognized credential; or
• Complete a work-based learning experience.

The indicator also aims to promote access and inclusion for career readiness activities for historically underserved students, including English learners, students with disabilities, economically disadvantaged students, students of color, and students in nontraditional fields.⁷

Implementation

The Industry-Based Learning Indicator is designed to evaluate how students are engaging in work-based and classroom-based activities by the end of 12th grade. Recognizing that industry-based learning activities should be developed and sustained throughout a student’s secondary level education, the Department recommends that LEAs monitor students’ annual progress towards meeting the indicator as described below.

Note: Reporting requirements have not changed for students enrolled in PDE-approved CTE programs.

Effective the 2017-18 school year, all LEAs must report in PIMS whether a 12th grader has met the requirements for the Industry-Based Learning Indicator. As with other data collections, school entities can enter and update student data for this indicator throughout the school year, up until the last collection period (typically June). (Note: Additional information regarding collection procedures, timelines, and other technical instructions for submitting the data are available in the 2017-18 PIMS User Manual, Vol. 1.)

Table 1 illustrates the three options available to LEAs to meet the Industry-Based Learning Indicator requirements.

Table 1: Meeting the Industry-Based Learning Indicator

LEAs must implement and evaluate all activities counted towards the Industry-Based Learning Indicator with fidelity and rigor, as demonstrated by alignment to CEW standards. Activities should reflect meaningful engagement focused on achieving postsecondary success, that are responsive to the needs of communities and regions, and that are tailored to students’ personal interests and education plans. LEAs are advised to provide a variety of standards-aligned programs and activities, including work-based learning opportunities, to promote career awareness, preparation, readiness, and entrepreneurship. The Department also encourages LEAs to partner with their local workforce development boards (LWDBs), chambers of commerce, Occupational Advisory Councils (OACs), business and industry leaders, postsecondary institutions, and other community partners to connect to local, regional, and state workforce needs. An analysis of regional and statewide workforce data, including current and future projected openings and skills needs, may also inform programming.

Appendix A includes a list of activities, descriptions, and requirements that promote valid and reliable industry-based learning programs. While not exhaustive, these lists are meant to provide examples of activities and programs that are aligned with CEW Standards and would meet the criteria of fulfilling the Industry-Based Learning Indicator. In addition to being standards-aligned, industry-based learning activities should be individualized to each student’s interests and needs and should be designed to ensure that all students – including students with disabilities, English learners, and other traditionally underserved students – are able to access industry-based learning experiences to prepare them for meaningful postsecondary success.

Data Reporting and Monitoring

Data Reporting

LEAs are responsible for reporting individual student data into PIMS to verify which 12th graders met the Industry-Based Learning Indicator criteria. LEAs can submit data regarding students’ status for the Industry-Based Indicator throughout the school year, until the final PIMS collection window in June. (Additional information regarding collection procedures, timelines, and other technical instructions for submitting the data are available in the 2017-18 PIMS User Manual, Vol. 1.)

Reporting CTE Graduates

For students enrolled in PDE-approved CTE programs, reporting requirements have not changed. PDE will use existing CTE Domain templates in PIMS to identify graduates meeting the Industry-Based Learning Indicator criteria either by scoring competent or advanced on the NOCTI/NIMS assessment, earning an industry-recognized credential, or participating in a work-based learning experience.

1. NOCTI/NIMS Assessment – Students scoring competent or advanced on the NOCTI/NIMS will be identified by PDE via the Assessment Fact Template in PIMS. The Assessment Fact Template is a PDE-loaded table, as information is pulled from NOCTI/NIMS. (Reminder: LEAs do not need to enter this information into PIMS for CTE students since it is already reported.)
2. Industry-Recognized Credentials - This information is entered in Field 7 "Industry Credential Code" of the Student Industry Credential template. It is a 3-digit code, identifying the industry certification and the industry certification provider of the credential that the CTE student earned during the reporting period, as a result of the student’s enrollment in the program (CIP CODE) and as reported in Field 5 of the template. (Refer to Appendix Q of the PIMS User Manual (Vol. 2) for a complete list of valid values.)
3. Work-Based Learning Experiences – The LEA will identify students with work-based learning using the following fields in the CTE Student Fact template in PIMS:
• Field 12 - Registered Apprentice Indicator
• Field 13 - Internship Indicator
• Field 14 - Cooperative Work Indicator
• Field 15 - Job Exploration Indicator
• Field 16 - Agriculture Experience Indicator
• Field 17 - School-Sponsored Enterprise Indicator
• Field 22 - Work-Based Experience Indicator as CTE Work-Based Learning

Reporting Non-CTE Graduates

LEAs will report students not enrolled in PDE-approved CTE programs as follows:

1. NOCTI/NIMS Assessment – Non-CTE graduates scoring competent or advanced on the NOCTI/NIMS will be identified by PDE, via the Assessment Fact Template in PIMS. The Assessment Fact Template is a PDE-loaded table, as information is pulled directly from NOCTI/NIMS. The LEA does not enter this information into PIMS.
2. Industry-Recognized Credential and/or Work-based Learning Experiences - Non-CTE graduates with these experiences will be identified using the following fields in the Student Award Fact Template for Non-CTE Industry-Recognized Credentials and Work-Based Learning Experiences:
• Field 5 Award Type - Enter one of the following: "ICN" for Industry-Recognized Credential, or "WBL" for Work-Based Learning Experience.
• Field 4 Award Code - Enter the unique code assigned to identify the subgroup that pertains to the credential or work-based learning experience, as listed below or refer to Appendix AM in the PIMS User Manual, Volume 2.

Table 2 - Non-CTE Work-Based Learning Experiences Codes

LEAs with both CTE- and non-CTE graduates will use a combination of the previous reporting methods. Graduates enrolled in PDE-approved CTE programs will be identified using the existing CTE Domain templates in PIMS; non-CTE graduates will be identified using the Assessment Fact Template or the Student Award Fact Template for Non-CTE Industry-Recognized Credentials and Work-Based Learning Experiences, as described above.

As with all PIMS data reporting, the LEA is responsible for assuring the quality and sufficiency of evidence provided. The PIMS administrator and chief academic officer at the reporting school entity are encouraged to consult the current PIMS user manual for additional information regarding the submission of data for purposes of state and federal accountability to the Department. By signing the assurances included within the Accuracy Certification Statement (ACS) provided during PIMS reporting, the school entity’s chief academic officer verifies the accuracy of the data reported by the school entity, the successful completion of student evidence/artifacts, and the quality of the program.

Monitoring

During statewide assessment monitoring and the evaluation of approved CTE programs, monitors may request documentation to verify the data reported. Monitors may request to see lesson plans/curriculum, course guides, and/or other applicable evidence to support the reported data.

When evaluating evidence provided by school entities, the Department reserves the right to request additional information and make determinations regarding the accuracy and quality of both the school entity’s documentation of evidence as well as the programs and activities counted as evidence of students’ successful attainment of the Industry-Based Learning Indicator.

Appendix A: Work-Based Learning Activities

Work-based learning provides an opportunity for students to reinforce their classroom learning, explore future career fields, and demonstrate their skills in a real-world setting. Listed below are common types of work-based learning activities, along with requirements to qualify for the activities. Guidance on selecting and implementing work-based learning models, as well as key terms and considerations, are available in the PDE Work-Based Learning Toolkit.

Appendix B: Terms and Definitions

Apprentice – A person at least 16 years of age who is engaged in learning a recognized skilled trade through actual work experience under the supervision of a journeyman. The training should be combined with properly coordinated studies of related technical and academic subjects.⁸

Apprenticeship Program – A competency-based program that coordinates and integrates classroom instruction with a structured work-based employment experience designed for students, as defined in Chapter 4.⁹

Career and Technology Center (CTC)/Area Vocational Technical School (AVTS) – A public school that provides vocational-technical education to secondary school students, out-of-school youth and adults in a geographical area comprised and operated by one or more school districts and established under sections 1840-1853 of the School Code (24 P. S. § § 18-1840—18-1853).¹⁰

Career and Technology Education (CTE) – Programs under public supervision and control which provide an organized process of learning experiences designed to develop integrated academic and occupational skills, knowledge, attitudes, work habits, and leadership ability for entry into and advancement within various levels of employment in occupational areas of agriculture, business, marketing and distribution, health, home economics and trade, and industry and for participation in postsecondary education and training.¹¹

Career Education and Work (CEW) Standards – In 2006, the Pennsylvania State Board of Education established regulations (22 Pa. Code Chapter 4) establishing the state Academic Standards for Career Education and Work (CEW standards). In accordance with 22 Pa. Code § 4.12(5), career education and work is defined as "understanding career options in relationship to individual interests, aptitudes, and skills including the relationship between changes in society, technology, government, and economy and their effect on individuals and careers."¹² These standards describe what students should know and do, and provide measuring points at grades 3, 5, 8, and 11 in the following areas:

• Career Awareness and Preparation (Section 13.1);
• Career Acquisition (Getting a Job) (Section 13.2);
• Career Retention and Advancement (Section 13.3); and
• Entrepreneurship (Section 13.4).

Classification of Instructional Programs (CIP) Code – A numerical coding of instructional programs developed by the U.S. Department of Education.¹³

Cooperative Education – A method of education for individuals who, through written cooperative arrangements between a school and employers, receive instruction, including required rigorous and challenging academic courses and related career and technical education instruction, by alternation of study in school with a job in any occupational field.¹⁴

Cooperative Vocational Technical Education – A planned method of instruction developed through a signed, cooperative arrangement among school representatives, students, parents, and employers in the community to provide students with an opportunity to alternate in-school academic and vocational technical instruction in entry-level, paid employment in an occupational field, in which the students total occupational work experience is planned, coordinated, and supervised by the school in close cooperation with the employer, as defined in Chapter 4.¹⁵

Cooperative Education Teacher-Coordinator – A certified professional member of the instructional staff responsible for administering the cooperative education program and/or work-based learning program and resolving all problems that arise between the school and the on-the-job activities of the employed student. The teacher-coordinator acts as a liaison between the school and employers for cooperative education programs or other work-based learning experiences.¹⁶

Fair Labor Standards Act – This act establishes minimum wage, overtime pay, recordkeeping, and youth employment standards affecting employees in the private sector and in federal, state, and local governments.¹⁷ Also see Appendix C.

Industry-Recognized Credential – A portable, industry-recognized credential validating that a student successfully demonstrated skill competencies in a core set of content and performance standards in a specific set of work-related tasks. The tasks and assessment must connect to workforce demands.

National Institute of Metalworking (NIMS) – This credentialing agency is a PDE-approved agency for student occupational competency testing in machining-related approved CTE programs and provides tests and student data for Pennsylvania. NIMS tests include both an online theory component and a performance component.

National Occupational Competency Testing Institute (NOCTI) – This testing agency is a PDE-approved agency for student occupational competency testing and provides job-ready tests and student data to Pennsylvania. NOCTI tests are composed of a multiple-choice component and a performance component. The online multiple-choice component measures the technical knowledge acquired by students. The performance component allows students to demonstrate their acquired skills by completing actual jobs using the tools, materials, machines, and equipment of the occupation.

Registered Apprenticeship Training Program – A program registered with the U.S. Department of Labor or the state apprenticeship agency in accordance with the Act of August 16, 1937, known as the National Apprenticeship Act (29 U.S.C. §50). A registered apprenticeship is conducted or sponsored by an employer, a group of employers, or a joint apprenticeship committee representing both employers and a union and contains all terms and conditions for the qualification, recruitment, selection, employment, and training of apprentices.¹⁸

Registered Apprenticeship – A training plan registered with the Apprenticeship and Training Council of the Commonwealth and evidenced by a certificate of registration or other appropriate document as meeting the apprenticeship standards of the council, as defined by Chapter 339.¹⁹

Teacher-Adviser – A certified professional member of the instructional staff responsible for administering the work-based learning program and resolving all problems that arise between the school and the on-the-job activities of the student. The teacher-adviser acts as a liaison between the school and other work-based learning experiences.

Appendix C: Work Guideline Requirements

The requirements described below apply to paid and unpaid work-based learning experiences. Additional information is available in the PDE Work-Based Learning Toolkit.

Clearances

Employers participating in the program are required to obtain school volunteer background clearances. School volunteers are required to have a Pennsylvania State Police criminal history check, child abuse history certification, and if the mentor has resided in Pennsylvania for fewer than 10 years, a federal criminal history check. If the child interacts with other workers at the employer’s job site, only the assigned employee is required to have clearances if that employee remains in the immediate vicinity of the student and is identified as the responsible adult (PA Department of Education, n.d.).

Pennsylvania Child Labor Act

The Pennsylvania Child Labor Act was enacted to provide for the health, safety, and welfare of minors by forbidding their employment or work in certain establishments and occupations, and under certain specified ages. The Child Labor Act establishes the age limits, hours of employment and the prohibited occupations for students who are residents of the commonwealth.

The provisions of the Child Labor Act apply to all situations in which an employer-employee relationship exits, including all paid work experience as part of work-based learning.

Wages

Consideration must be given to whatever trainees or students participating in unpaid learning experiences might be considered employees with the meaning of the Fair Labor Standards Act (FSLA). No single factor is determinative; it requires an analysis of the circumstances to determining if the student is or is not an employee. If the student is not an employee, then there is no entitle to minimum wage or overtime pay, in accordance with FSLA.

Factors meriting consideration include the following:

1. The extent to which training is tied to formal education program by integrated coursework or academic credit;
2. The extent to which training, even though it includes actual operation of the facilities of the employer, is like that which would be given in a career and technical school;
3. The training is for the benefit of the trainees or students;
4. The trainees or students do not displace regular employees, but work under close supervision and complements the work of paid employees;
5. The employer that provides the training receives no immediate advantage from the activities of the trainees or students and, on occasion, his operations may even be impeded;
6. The trainees or students understand they are not necessarily entitled to a paid job after the training period and;
7. The employer and the trainees or students understand that the trainees or students are not entitled to wages for the time spent in training.

More detailed information is provided in the resource, "Employment Relationship under the Fair Labor Standards Act" available at Pennsylvania Cooperative Education Association.

It is advisable to have the school solicitor assist in the preparation of a comprehensive school policy regarding potential liability in case of an accident or injury to a student participating in an unpaid worksite experience.

• If the student is paid by the workplace, the employer is responsible for all wages and taxes as well as liability and workers’ compensation coverage.
• If the school pays the student, the school district is responsible for all wages and taxes as well as liability and workers’ compensation coverage.
• If the student is unpaid, the school district is responsible for liability coverage.

¹ Pennsylvania’s state definition of "college and career readiness."

² C. Alfeld, et al., Work-Based Learning Opportunities for High School Students, National Institute for Work and Learning, February 2013.

³What to Know about Work-based Learning Experiences for Students and Youth with Disabilities, Federal Partners in Transition, November 2015.

⁴PDE Work-Based Learning​ Toolkit, November 2017

⁵PA Career Standards, PA Department of Education.

⁶ 2017-2018 Pennsylvania Information Management System User Manual Volume 1 v.1.0

⁷ Nontraditional fields of work are those for which individuals from one gender comprise less than 25 percent of the individuals employed in each such occupation or field of work.

⁸http://www.doleta.gov/OA/regdirlist.cfm

⁹ 22 Pa. Code § 4.12(5)

¹⁰ 22 PA Code § 4.3 Definitions

¹¹ 22 PA Code § 4.3 Definitions

¹² Pa. Code § 4.12(5)

¹³https://nces.ed.gov/

¹⁴Cooperative Education Guidelines for Administrators, PDE.

¹⁵Cooperative Education Guidelines for Administrators, PDE.

¹⁶Cooperative Education Guidelines for Administrators, PDE.

¹⁷https://www.dol.gov/whd/flsa/

¹⁸ 22 Pa. Code § 339.31

¹⁹ 22 Pa. Code § 339.31