E-Mail this Page | Print this Page

Education

Assessment System

The unit has developed an assessment system with its professional community that reflects the conceptual framework(s) and professional and state standards. The unit’s system includes a comprehensive and integrated set of evaluation measures that are used to monitor candidate performance and manage and improve operations and programs. Decisions about candidate performance are based on multiple assessments made at admission into programs, at appropriate transition points, and at program completion. Assessments used to determine admission, continuation in, and completion of programs are predictors of candidate success. The unit takes effective steps to eliminate sources of bias in performance assessments and works to establish the fairness, accuracy, and consistency of its assessment procedures.

The purpose of the unit’s assessment system is to articulate the sequential process used to document that candidates at the initial and advanced levels have met the outcomes defined by the conceptual framework and institutional, state, and national standards. Before the development of the formal assessment system, the Unit, through existing evaluation procedures and revisions that were in progress, already had a number of the assessments that were needed to accomplish this goal.

Data however, while being systematically used to make decisions about candidates, were not used systematically to improve programs and unit operations. The development of the assessment system in conjunction with the revision of the conceptual framework has resulted in a stronger system for assessing candidates and a comprehensive system for program improvement. Increased availability of data and a coherent feedback timeline will result in a continuous examination and improvement of initial and advanced programs.

Some elements of the Unit assessment system were developed beginning in 1999-2000 to provide data for the Institutional Report Card required by the Nebraska Department of Education under the federal Title II requirements. The Unit assessment system was further developed during 2003-2004 and continues to be refined. Existing gateways, candidate assessments, and traditional informal practices were analyzed and incorporated into a comprehensive assessment system. Changes include:

· aligning assessment tools and course content with revised conceptual framework and state/national standards,

· aggregating/ disaggregating data on conceptual framework components from the Teacher Intern Checklist and follow up studies,

· gathering and utilizing data from several existing sources such as the "Observation Verification Report", multicultural field experience reflections, and candidate recommendations.

· developing additional assessments (i.e. Teacher Work Sample, EDAD practicum portfolios) or procedures for gathering numerical data on existing assessments (i.e. School Counseling oral examinations),

· and formalizing or developing mechanisms for utilizing data in decision making at various levels.

The current assessment system will evolve based on program needs. As described below, the assessment system includes both internal and external assessments that incorporate professional, state, national and institutional standards. The challenge presently is to effectively manage data from various sources and aggregate and report data in an efficient and timely manner. The system operates effectively in determining student entrance and exit from the various program gateways at both the initial and advanced levels. An accountability management system is in use for initial candidates, but input of data is still labor intensive. Departments monitor advanced program candidate progress, but a unit wide system similar to the one for initial candidates is being planned for advanced candidates. A data (warehousing) management system is being explored to increase efficiency across platforms.

There are a number of opportunities within the system to check for bias, accuracy, consistency, and fairness. Examples for specific programs and areas will follow. Performance data are being collected, as shown in Standard 1 above, and are being used to make decisions about candidates at the gateways of the unit assessment system for initial and advanced programs. The revised CSC conceptual framework and additional performance based measures such as, the teacher work sample (TWS, Rubric) at the initial level and the Master of Education in Curriculum and Instruction--Elementary and Secondary education follow up survey (employer, self) at the advanced level, have only been in effect since the Spring of 2005. As a result, performance data using these measures is somewhat limited. Data from other measures such as Teacher Internship Checklist and some measures of Master of Education in Education Administration--Elementary and Secondary indicate trends. Revision of the conceptual framework further limits comparative uses of the more mature instruments. The assessment system is operational and will increase in efficacy with continued development, data collection, and data analysis.

Initial and Advanced Programs

 The assessment system has been developed over a period of several years with the involvement of the professional education community. Development of the plan began in 1999-2000. The system was totally revised in Fall 2004 after revision of the conceptual framework and the addition of more performance measures. The formal feedback loop was piloted in Spring and Fall of 2005. Data collection and system revision continue based on program and procedural needs.

In 1999, the Dean, Assistant to the Dean, and Project Coordinator of the Nebraska Partnership for Quality Teacher Education (Title II grant) evaluated assessment measures in use. They determined a more cohesive database and additional measures of candidate performance related to the conceptual framework were necessary (1999-2000). Those measures were piloted; data were collected; and a data management system was instituted. Refinements continued. In the fall of 2004 the NCATE Steering Committee directed the development of the current assessment system, building on the past elements. Revisions included incorporating updated assessment tools (follow-up surveys, Teacher Intern Checklists, Teacher Work Sample [TWS], and professional semester feedback forms) that reflect the revised conceptual framework. The NCATE Steering Committee is now comprised of the Dean, School Administrative Coordinator, Education Department Chair (Chair for Standard 1) and the Chair for Standard 2. This committee sought additional input from the Teacher Education Unit Committee. This committee is composed of faculty representatives from throughout the campus who were involved in teacher and other school personnel preparation programs. A number of others from the professional education community have been asked to review and provide input into the development of the assessment system. Representatives from the following groups include but are not limited to:

• Teachers and administrators from cooperating schools

• Elementary candidates and secondary candidates

• Graduate Council – representatives from departments with advanced programs and

graduate candidates.

The conceptual framework was key to the development of the assessment system and its components. Examples that illustrate this alignment include: 1) references to the conceptual framework within the assessment system, 2) evidence of the conceptual framework in expectations of candidates, 3) the sections of the teacher intern evaluation checklists that reflect the conceptual framework (overarching component, leadership, with sections addressing thinking skills, professionalism, methodology/technology, assessment, communication, diversity/human relations), and 4) in the policies and procedures (gateways) that specifically address the knowledge, skills, and dispositions found in the conceptual framework for initial and advanced programs. Unit faculty and administrators established criteria for all program assessments; defined gateways; and established and continue to refine knowledge, skills, and disposition assessments for initial and advanced level candidates. The gateways and assessment system are in accordance with the role, mission, rules and procedures of Chadron State College.

The last major step in the implementation of the assessment system will be the acquisition of a data (warehousing) management system designed to build upon the database developed for the Institutional Report Card. Several existing computer programs were investigated, and continue to be explored.

Candidate Performance Initial Programs

 The assessment system for initial candidates is based on 5 gateways (decision and assessment points): (1) admission to Chadron State College; (2) application to teacher education (candidacy); (3) admission to teacher education program; (4) admission to professional year [4a-admission to the professional semester and 4b-placement in teacher internship; and (5) program completion – graduation, certification, and entry into the profession. Data collected include both internal and external multiple assessments, traditional testing, performance assessments, and recommendations. Detailed gateway charts have been developed and are available in the documents room. Following is a summary of the gateways described in Standard 1-A. (Also available in evidence room.)

Initial programs and assessments are based on the Unit conceptual framework and the Nebraska Department of Education (NDE) professional education and content standards (Rule 24). NDE initial program performance standards are based on national program standards. All unit programs and assessments relate to state and national standards. Course syllabi reflect relationships to the conceptual framework and standards.

Chadron State College’s professional preparation and program is designed to produce Visionary Leaders at the early childhood, elementary, middle, secondary and graduate levels. Content and pedagogical knowledge, skills and dispositions are embedded in program curricula from the Education Department and the eleven (11) departments that offer endorsement programs. Candidates move through the following program gateways:

Gateway 1: Admission to Chadron State College

Gateway 2: Application to Teacher Education Program

Gateway 3: Admission to Teacher Education Program

Gateway 4: Admission to Professional Year (4a and 4b)

Gateway 5: Graduation (Program Completion)

Early childhood, elementary, middle, and secondary program candidates must meet the same requirements at each Gateway point. Early childhood, elementary and middle level candidates are reviewed by the Education Department, while secondary level candidates are reviewed by their respective content area departments.

A variety of assessment techniques are used by the various departments to evaluate candidate knowledge, skills, and dispositions. These include portfolios, microteaching, exams, case studies, simulations, technology demonstrations, collaborative activities, research projects, and class presentations, projects, and discussions. These assessments are documented by candidate work and course syllabi in the documents room. A description of each Gateway and the applicable criteria are presented in Standard I (p. 22-26).

Illustrations of efforts to assure fairness, accuracy, consistency, and non-bias include:

Validity: The portfolio rubric and teacher intern evaluation checklist have been carefully aligned with the "Visionary Leader" conceptual framework. Experienced teachers played a role in the development of the Teacher Work Sample (TWS) and those who have served as raters believe that the TWS is a valid, meaningful assessment that is related to the Nebraska assessment framework. The Nebraska assessment framework resulted from an extended process involving Nebraska Department of Education personnel, all Nebraska higher education institutions, educational service unit representatives, and public educators. Dispositions policy and procedural elements and documents are tied directly to the conceptual framework and NDE rules. Program reviews are based on state standards and NDE Rule 24: Policy and Regulations for Endorsement Programs which in turn are based on national content standards.

Reliability: Faculty and teacher intern supervisors have been trained in observing and for scoring teacher work samples. Teacher intern checklists were recently revised so that data obtained matches the revised conceptual framework. An inter-rater reliability study for the teacher work sample scores will be conducted and continued analysis is planned.

Fairness: Candidates are allowed to appeal denial of admission to program gateways, including teacher intern semester. The TWS (TWS, Rubric)was piloted. Candidates who scored low were permitted to redo sections for which they received a low score. The dispositions assessment is being piloted to assure fairness and non-bias.

Advanced Program Masters Programs

 The assessment system for master’s degree candidates is based on four decision points: (1) admission to graduate study, (2) admission to graduate status, (3) application for oral examination, and (4) program completion. Data collected include both internal and external sources, multiple assessments, traditional testing, performance assessments, and recommendations. Gateways are described in Standard I (p. ). (Expanded descriptions of graduate gateways are available in Standard 1-A and evidence room.)

Some illustrations of efforts to assure fairness, accuracy, consistency, and non-bias include:

Validity: For all master’s candidates, their advisors and faculty committee review portfolios prior to the oral exam. The portfolio self-reflections are tied to the conceptual framework.  Faculty use common rating scales in scoring Education Administration (EDAD) practicum portfolio (notebook), and major written papers and projects in courses to assure assessment of appropriate content and skills. In School Counseling faculty use common rubrics or rating scales in scoring exams, and evaluating counseling skills during practicum and internship and major written papers and projects in courses to assure assessment of appropriate content and skills as well as dispositions.

Reliability: In Education Administration (EDAD), a handbook has been created for field experience supervisors and practicum candidates to standardize the field experience supervision and assessment. EDAD uses two or three readers in the scoring exams. Oral exam committees for Education Administration (EDAD) and Education Curriculum and Instruction (EDCI) candidates are comprised of at least two faculty members from the candidate’s program area and at least one faculty member from outside of the department. Oral exam committees examine the candidate’s professional knowledge, skills, and dispositions; evaluate the candidate’s portfolio; and evaluate the thesis, scholarly project, or field study, depending on the chosen option. School Counseling offers both a handbook for practicum and internship that contain evaluation of skills on a consistent and structured basis for faculty and supervisors. Oral exam committees for School Counseling are comprised of at least two faculty members from the candidate’s program area and at least one faculty member from outside of the department. Oral exam committees examine the candidate’s professional knowledge, skills, and dispositions; evaluate the candidate’s portfolio; and evaluate the thesis, scholarly project, or field study, depending on the chosen option.

Fairness: A common assessment form is used for candidate assessment of courses. The graduate program has an appeal process for candidates who disagree with assessments tied to the gateways. Education Administration (EDAD) uses a universal "candidate rating admittance" form in determining eligibility for programs to ensure that all candidates are judged on the criteria. School Counseling uses a common assessment form for candidate assessment of courses. The graduate program has an appeal process for candidates who disagree with assessments for the counseling program.

Unit Operations

The assessment system not only addresses candidate performance but also unit operations. The Unit seeks fairness, accuracy, consistency, and non-bias of all assessments. The following is a partial list of data and documents that are the basis for evaluation of operation of the unit:

                    Unit Policies and Procedures handbook

 

 

 

 

 

Program and candidate admission data

Teacher Intern Guidebook (cover page, TOC page 1, TOC page 2) (includes the TWS prompt and rubric) [secondary]

Teacher Intern Checklist (includes the TWS prompt and rubric) [elementary]

Psychology and Counseling Department Graduate Student Handbook and Internship Handbook

EDAD program Practicum/Internship Handbook

Performance of initial preparation and advanced program candidates in field experiences

Candidate evaluation of instructors, courses, and field experiences including supervision

Formal candidate complaints and resolutions

Follow-up data from graduates and employers (initial preparation and advanced programs)

Evaluation of faculty and administration as part of CSC personnel processes

Internal program reviews by departments, schools, and college

 

 

 

 

 

 

 Programs and policies are reviewed by appropriate faculty and staff who comprise Gateway Committees, Education Department, Teacher Education Unit Committee, Graduate Council, and the Dean of the School of Professional and Graduate Studies to determine fairness, accuracy,

consistency, and non-bias. Following is a partial list of policies, procedures and opportunities: 

 

 

 

 

 

 

"Screening Committee" (appeals committee) consisting of the Education Department Chair, Director of Field Experience, Registrar, College Nurse, Dean of Students, and one representative of elementary and secondary professional semester faculty review candidate applications that do not meet gateway criteria.

Campus offices provide candidate assistance and accommodations for candidates with disabilities (Student Success Services -- academic counseling, disability/non-traditional candidate programs, tutoring).

All candidates complete the human relations/multicultural course requirement and each program includes multicultural components, to promote non-bias.

Non-bias training is provided by the Education Department to scorers of the Teacher Work Sample.

Statements are placed on syllabi to encourage candidates with disabilities to seek accommodations when needed (as per required college policy).

Cooperating/mentor teacher and college supervisors’ training is conducted to ensure fairness, consistency, etc. with regard to evaluating teacher interns.

Department chairs review formal candidate complaints. If not resolved through the department chair or the Screening Committee appeals process, the Dean reviews and initiates appropriate action.

Candidates are provided due process both at the Unit and college levels.

Candidate dispositions are assessed and data are collected through internal and external means to check for fairness and non-bias.