Unit 5: Evaluation

5.1 Concept, meaning, definition and types of evaluation

5.2 Continuous and comprehensive evaluation (CCE), Formative and Summative Evaluation

5.3 Error Analysis & Error Correction

5.4 Program Monitoring and Importance and type - Record Maintenance

5.5 Report preparation and feedback to the Parents

5.1 Concept, meaning, definition and types of evaluation

It is a technique by which we come to know at what extent the objectives are being achieved.  It is a decision making process which assists to make grade and ranking. According to Barrow and Mc Gee: It is the process of education that involves collection of data from the products which can be used for comparison with preconceived criteria to make judgment.
Evaluation is comparison of performance of a student to a prescribed criteria. Howell and Morehead (1987) state that evaluation is a thoughtful process involving the comparison of the way things are, to the way they should be. During evaluation, we collect the data, summarize the results and compare the results with a set criteria to draw conclusions. For example, an objective for student A is “Reads five words correctly after 15 sessions of teaching”. On evaluation, it was found that she is able to read only 3 words. When we compare, we say that she is able to read only 3 words and she is yet to learn 2 more words. Here, we have compared the performance of the student (the way the things are) to the expected performance (reading 5 words – the way they should be) to find out effectiveness of intervention programme. The following figure explains the evaluation process model.

Standard Behaviour Discrepancy: The behaviour represents what the student is doing and the discrepancy represents how much the student has to achieve to meet the standard. Let us discuss the same example given above in the light of evaluation process model. The “standard” set is reading five words. On evaluation, the “behaviour” shown is reads 3 words. On comparison of the behaviour to the set standard, the “discrepancy” is 2 words. Therefore, the comparison of a student behaviour to a standard is central to the process of evaluation and the evaluator makes judgments based on discrepancy to determine the effectiveness of intervention programme.

TYPES OF EVALUATION

Assessment should be done before and after instruction. Assessment done after instruction is evaluation. Two types of post instructional evaluation procedures are in use in educational evaluation – formative and summative.

Formative Evaluation: Formative evaluation is conducted during the intervention programme actually being implemented. It facilitates periodic assessment to indicate whether the planned instruction is delivered as planned and whether or not the expected progress is being made by the student. Further, ongoing evaluation give the teacher and student immediate feedback on the adequacy or inadequacy of instruction and learning so that deficiencies or gaps can be quickly remediated.

Summative Evaluation : Summative evaluation on the other hand is a long term, final assessment conducted after a completion of unit instruction. It indicates final degree of learning or achievement.

Diagnostic Evaluation. Diagnostic evaluation is directed towards two purposes either for placement of students properly at the outset of an instructional level (such as secondary school),or to discover the underlying cause of deviancies in student learning in any field of study.

Criterion Referenced and Norm Referenced

When we look at the types of evaluation instruments, we can generally classify them into two main groups: Criterion-referenced evaluation and norm-referenced evaluation.

"Norm-Referenced Evaluation:

A norm-referenced test scores a test by comparing a person's performance to others who are similar. You can remember norm-referenced by thinking of the word 'normal.' The object of a norm-referenced test is to compare a person's performance to what is normal for other people like him or her.

Definition: A test or other type of assessment designed to provide a measure of performance that is interpretable in terms of an individual's relative standing in some known group- Gronlund (2000)

Criterion-Referenced Evaluation

Definition: A test or other type of assessment designed to provide a measure of performance that is interpretable in terms of a clearly defined and delimited domain of learning tasks." Gronlund (2000)

A criterion-referenced test is a style of test which uses test scores to generate a statement about the behavior that can be expected of a person with that score. Most tests and quizzes that are written by school teachers can be considered criterion-referenced tests.

Techniques Of Evaluation :

A variety of techniques are employed. Questionnaire, checklist, interview, group discussions evaluation workshops and Delphi techniques are the major one.

a) Observation :   It is related to curriculum transaction. Observation schedule helps the evaluator to focus his attention on the aspects of the process that are most relevant to his investigation. This method gains credibility when it contains both subjectives and objective methods. Interviews and feed-back and other documentary evidences may supplement observations.

b) Questionnaire :  It is used to obtain reaction of curriculum users namely pupils, teachers, administrators, parents and other educational workers concerning various aspects of prescribed curriculum are to be ascertained

c) Check-list : It can be used as a part of questionnaire and interview. It provides numbers of responses out of which most appropriate responses are to be checked by the respondent.

d) Interview :  It is a basic technique of evaluation and for gathering information. It may be formal or informal in nature. The information required should be suitably defined and the presentation of questions should in no case betray and sort of bias the part of the interviewer. e) Workshops & Group discussion :  In this technique, experts are invited at one place to deliberate upon syllabi, materials etc; and to arrive at a consensus regarding the quality of the same. The materials may be evaluated against a set of criteria that might have been prepared by the evaluator

5.2 Continuous and comprehensive evaluation (CCE), Formative and Summative Evaluation

What is CCE?

 CCE refers Continuous & Comprehensive Evaluation, a system of school based assessment that covers all the aspects of a student’s development. It was designed to reduce the student stress related to board exams, and to introduce a uniform and comprehensive pattern for student evaluation across the country. It emphasizes on two broad objectives: (a) Continuity in Evaluation and (b) Assessment of broad based learning. Clearly, it attempts to shift emphasis from ‘testing’ to ‘holistic learning’ with an aim of creating young adults, possessing appropriate skills and desirable qualities in addition to academic excellence.

Objectives of CCE - 

· Encourage development of congnitive skills and de-emphasize rote learningØ

· Make the entire education process a student-centeric activityØ

· Help develop cognitive, psychomotor and interpersonal skillsØ

· Make holistic evaluation an integral part of entire education processØ

· Improve student's accomplishments through regular diagnostics and remedial instructionsØ

· Use evaluation to control quality and maintain desired performanceØ

· Take decisions about the learner, learning process and learning environment by determining social utility, desirability & effectiveness of the programme

Comprehensive Evaluation

1. Scholastic Evaluation

2. Co- Scholastic Evaluation

The ‘comprehensive’ component of CCE takes care of assessment of all round development of the child’s personality. It includes assessment in Scholastic as well as Co-Scholastic aspects of the pupil’s growth.

The second term `comprehensive' means that the scheme attempts to cover both the scholastic and the coscholastic aspects of students' growth and development. Since abilities, attitudes and aptitudes can manifest themselves in forms other then the written word, the term refers to application of variety of tools and techniques (both testing and non-testing) and aims at assessing a learner's development in areas of learning like :

· Knowledge

· Understanding/Comprehension

· Applying

· Analyzing

· Evaluating

· Creating

The scheme is thus a curricular initiative, attempting to shift emphasis from testing to holistic learning. It aims at creating good citizens possessing sound health, appropriate skills and desirable qualities besides academic excellence. It is hoped that this will equip the learners to meet the challenges of life with confidence and success.

Formative Evaluation. It occurs during the course of curriculum development. Its purpose is to contribute to the improvement of the educational programme. The merits of a programme are evaluated during the process of its development. The evaluation results provide information to the programme developers and enable them to correct flaws detected in the programme.

Summative Evaluation. In summative evaluation, the final effects of a curriculum are evaluated on the basis of its stated objectives. It takes place after the curriculum has been fully developed and put into operations.

5.3 Error Analysis & Error Correction

Error analysis

Although CRTs are helpful in determining specific curriculum areas, they do have certain limitations. CRTs will tell what to teach but gives little information on how to teach that skill. To put in another way, most CRTs are interested in measuring the product and not the process. The method that is used to determine the process or strategies a student uses while doing the academic tasks is called error analysis.

Why to do error analysis?

It helps in selection and evaluation of teaching strategies, gathering pre-referral information and planning and evaluation of IEP.

Why should teachers use?

A great deal of information can be obtained from the type of errors a student makes in routine school work. For example, see the following computation.

    38
 + 95
-------
1213
-------

This would indicate that the student does need instruction in renaming because he added the ones and tens columns separately. The student has not applied the step of carrying over. How did you know? Yes! By analyzing the product of the sum. Now you know what to focus on. Consistency in error pattern is an indicator to the teacher on student’s learning style which helps her to adapt teaching style to suit the need.

Error correction

Error correction is a procedure that details what a trainer or program implementer does when the learner engages in an incorrect response during a teaching opportunity. Error correction aims to enhance learning by teaching the learner the appropriate response and increasing the learner’s contact with reinforcement contingencies rather than simply extinguishing errors. This procedure is intended to help learners acquire skills much faster and with less frustration than simply allowing trial and error. In other words, it teaches the learner what to do instead of just allowing them to make mistakes and try to determine the correct response on their own.

Error correction sequences consists of four steps:

1. A student error

2. The teacher’s feedback which may take the form of explicit correction, recast, clarification request, metalinguistic feedback, elicitation or error correction

3. The student’s response, which may or may not still need repair

4. Reinforcement of a correct response by the teacher (on occasion)

Students respond more successfully when the correct form is not supplied for them and there is negotiation of form  ie: with clarification requests, metalinguistic feedback, elicitation or error repetition

5.4 Program Monitoring and Importance and type - Record Maintenance

‘Record maintenance simply means systematically documenting and storing information collected from various sources using appropriate procedures for a predetermined purpose’.

Importance and purpose
The first and foremost task a professional carries out when a client approaches him is to register the person – collecting information such as name, address, purpose of visit and so on. This is the beginning of recording. As you know, this will be followed by a number of assessments based on the needs each of which, is recorded suitably by the respective professionals. When the multi/inter disciplinary team meets, these records serve as a source of information for decision making.

The decisions are again recorded so that the document is available during follow up visits of the client and therefore, continuity can be maintained.

Even if the professionals have changed between visits, the records help the new professionals to follow up without difficulty.

The client and family have a base in the records using which they can discuss for further plans for their child.

If the family shifts to another city or changes the service organization for their child, the written comprehensive report provided by the attending professional will help the new agency understand the case and continue with the training.

If there is any deviation in the implementation from the documented plan of action, the discrepancy can be noted immediately for further programme.

Above all, the documents serve as a link between professionals, among themselves as well as with families. With the legal provisions in order, documents and records are likely to serve as the base for resolving issues related to disability rehabilitation.

Types of records
Now let us look at the specific special educational records and their purposes.

After the team assessment and diagnosis of mental retardation is carried out, the child is referred to a special school or special educational services in non-school setting such as home based or center based training.

Whatever is the setting, records are maintained. Such records are of two types.

1.  Administrative records
2. Educational/Technical records

Administrative records
The administrative records include demographic data, that is, the basic information regarding the child such as name, address, parental occupation, income, family background information, past records of schooling history if any and so on. This may be called registration.

After official formalities, when the child is admitted to the school or the educational programme, there will be a separate entry called `admission entry’ in another register. This is because all the `registered cases’ are not necessarily admitted. Based on the rules and regulations of the organization, the number admitted will be restricted. Other considerations will include age, sex, severity level, SES, availability of seats in a given group and so on.

Now we have two registers – one for registration and one for admission.

The third important administrative record will be for fee collection where entries of various fees, concessions, etc. will be documented.

Other administrative records relevant to the child includes undertaking by the parent on their acceptance/non-acceptance for immediate medical help by the school in case of illness/accident in school, parent’s consent/refusal for including the child in video/photography for publicity purposes and so on. The parent has a right to accept/refuse both of the above mentioned instances which should be honoured. It is always better to get these in writing as it is safe for the organization, in case of legal problems.

Educational/Technical records
Let us now focus on educational or training records, which are essential for monitoring progress. These records include assessment report, attendance register, periodic evaluation procedures and scoring, the individualized educational programmes (IEPs), the record of interaction with parents, task analysis recording, classroom time-tables and the child’s progress report.

A good teacher maintains all of these records, as they help in monitoring progress and analyzing error if any for correction. Let us see how.

5.5 Report preparation and feedback to the Parents

Report Writing- It is a process of preparing a formal written document that describes in detail an event, situation or occurrence, usually as the result of observation or inquiry.

Or simply, it is a written record of the vocational trainee based upon his/her initial assessment done, goal selected, implementation process and subsequent evaluation.

Report writing includes two basic aspects that need to be considered. They are-

1. Report Maintenance –It involves maintaining records of the trainee in written form wherein time duration taken is 6 months.

The purpose of report maintenance is the following-

· It is important for further program  planning, implementation and evaluation.

· It is used as a baseline data estimate for long term changes related to performance of the trainee.

· It acts as a proof for administrative purposes.

· It is useful for parents who want to see the level of performance of their    children.

· It is used to meet the information needs for other educators.

2. Reporting- We are taking this aspect to be done on annual/yearly basis. At vocational level, it is focused mainly on reporting to the administration of the training institution, parents of the trainee and the prospective employer. However, this approach (of reporting to prospective employers) is not followed at prevocational and secondary level.

Feedback

Feedback as an essential component of formative assessment. Good feedback generally focuses on behavior or the outcome of behavior rather than on the inherent characteristics of the person concerned. It leaves that person feeling positive and able to move forward. The timing of the feedback is important. It needs to be given as soon as possible after the event. The greater the delay, the less likely it is that the student will find it useful or be able or inclined to act on it.
Feedback also needs to be clear. Handwritten feedback should be legible. The language should be comprehensive to students. You need to take care with style and tone as misunderstanding can easily arise. This particularly applies when feedback is written.

Characteristics of Good Feedback
A. Good feedback is relevant
1. Feed back needs to be relevant in two way
a) First it should be linked to the assessment criteria for the task. Students should be able to see how well they have met each criterion.
b) Second , feedback needs to be relevant to the individual student, respecting the particular approach they took to the assessment task and other matters unique to the individual such as their previous work and stage of development.
2. You need to recall previous feedback you have given to avoid repetition and ensure variety.
3. It is helpful to keep photocopies of previous assignments or brief summaries of comments previously made.

B. Good feedback is Informative
1. Need to remember that the audience of feedback is students unlike conventional examination marking, your Feedback
is intended to act and in order to do this it has to be sufficiently detailed.
2. The word sufficiently as important as too little information and too much information can be equally ineffective.
3. Information should be given on students strength . this is encouraging and good for motivation , it also helps students plan ahead.
4. Should also indicate priority areas for improvement or development and suggest how students might tackle these.
5. Suggestions for improvement need to be sensitively handled. They should be perceived by students to be attainable within their grasp, as research shows that students are most motivated when they feel they can achieve results with reasonable efforts.
6. Aim to help students move ahead one or two steps.
7. It is generally best to give students an overview of their work, complemented and exemplified by detailed examples such as in the margins of an essay.
C. Good feedback encourages dialogue
Good Feedback is a two way process. You should try to stimulate a response and continuing dialogue – whether this be on the topics that formed the basis of the assignment or the feedback it. You should also try to find out how helpful students find the feedback and how it might be improved in future.
D. Good Feedback encourages self Assessment
i. The importance of the student being proactive in assessment one part of this is their willingness to assess their own work critically and to internalize criteria and standards.
ii. Asking students to send with their assignment their own assessments of them
iii. Asking questions and inviting students to respond.
Thus we find that feed back is an essential component assessment and evaluation.