Thinking Aloud

Let’s start with reiterating a truism: to state that “A has learnt X” necessarily presumes that assessment has happened. Otherwise the statement is an expression of worthless declaration. Let’s also remind ourselves that the goddess of child-centric educational discourse ‘learning’ necessarily implies standards. For A to have leant X means: (i) that A could not do/understand/feel X at some earlier time and now A can do/understand/feel what is involved in X. Doing/understanding/feeling necessarily means coming up to some agreed upon standards. If A has leant to ‘read’ (X) then A has come up to certain standard of making meaning out of black (generally) marks made on some surface. (ii) Some one who is making the statement “A has learnt X” has grounds to declare that, and therefore there is an assessor. To repeat: learning necessarily assumes standards, assessment that those standards are achieved, and an assessor who has ascertained the fact of achieving those standards.

It is commonly stated that education is concerned with learning, and often education is equated with learning. The later is plainly wrong. Learning happens all the time and spontaneously and often unknown even to the learner. To call it all ‘education’ will render the concept of education useless by over generalisation. The first part—education is concerned with learning—is true enough but is often misinterpreted. Education is concerned with development of capabilities to know (understand), to do and to feel in certain ways. Education in how to ‘feel’ includes values, emotions and dispositions. Learning is a—actually the only acceptable—means to achieve the desired standards in chosen areas of knowing, doing and feeling. Therefore, the only useful and defensible meaning one can attach to the phrase “learning assessment” is: assessment of acquired knowledge, abilities (to do) and values and dispositions. All this—in this and the paragraph above—is common knowledge restated to start a little discussion on assessment.

On the basis on the above two paragraph we can say that education per se may be possible without assessment but a statement that ‘some education has happened’ is not. Education is not a one shot activity started and finished on a fine morning. It is a long drawn process; and gradually and cumulatively happens. To know go to the next stage requires assumptions regarding achievements set for the pervious one. Thus assessment is necessary for ascertaining that the desired education is happening. Those who think that one can be certain without assessment that education has happened are deceiving themselves. We can not do away with assessment in education, even if we dislike it; and the animal called assessment as it roams the educational terrains today is dislikeable enough.

Now the question arises who can do assessment and how? And how far is it possible? Let’s note that assessment of knowledge and values involves ascertaining the contents of other’s mind. The assessor is ascertaining the contents of the educatee’s mind. The contents of another mind are not directly available to us, they can not be. We can directly know the contents of our own mind only. So what we do in assessment is watch and record the performance of the educatee and infer the contents of her mind. In case of assessment of skills and abilities (paper folding, applying algorithms in mathematics, carpentry) this performance is ‘doing something’ successfully, as per preset standards. In case of values and dispositions this performance is behaving in a certain manner with others on the basis of some consciously chosen principles. In case of knowledge and understanding ‘doing something’, ‘behaving with others’ and responding to questions set in language.

It seems the performance in skills can be assessed easily and reasonably reliably, as it has heavy dependence on doing and relatively little with understanding. But we do not actually teach or assess skills in schools. The skills we do teach and assess there are mostly intellectual skills (algorithms in mathematics) and they have very important knowledge/understanding aspect to them. Actually most of intellectual skills are of very limited value shorn of understanding. Therefore, educationally important skills have the same challenge as assessment of knowledge and understanding. We will come to that presently.

The moral values (we will not take up dispositions in this little piece) necessarily have two parts: the performance (behaviour with or treatment of others) and intentions. The intentions are governed by general principles. Therefore, assessment of values requires observed data on performance as well as on intentions and principles that are basis of behaviour exhibited in performance. Let’s note that one single observation of behaviour can not be sufficient, therefore, substantial amount of observation is needed. The principles and intentions, again, have the same requirements as those of knowledge and understanding. In addition they have the substantial problem of possible pretention. I can behave in a supposedly correct manner in a give situation deliberately, even if that be counter to my general behaviour and my normal dispositions. Similarly, I can state supposedly correct principles of behaviour (usually called moral values, like sada sach bolo) even if I do not believe in them. As a result, assessment of values becomes extremely difficult without very long association in non-threatening conditions, even if the challenge of assessment of knowledge and understanding is somehow ignored.

Our knowledge, generally speaking, is composed of concepts, relationships between concepts and principles that govern those relationships. Examples: tree, water, air, life, growth etc. are concepts. “Trees need water and air to survive” is a statement of relationships between the concept of tree, need, water, air and survival. This is expressed as a knowledge claim. “All knowledge clams need to be based on evidence” is a principle used in generating, assessing, and accepting knowledge. All this can be expressed in language and as knowledge claims. Therefore, the important question for an assessor becomes: how to ascertain the kind of knowledge claims the educatee as acquired? As mentioned above knowledge clams are internal to a persons mind, we have no direct access to other people’s minds. But knowledge clams are made in language, and we can, presumably, understand each other’s language. Therefore, assessment of knowledge and understanding becomes a paper-pencil test through asking and answering questions.

Now, let’s note a little commonly known fact about knowledge claims: that they never can stand alone, they always depend on each other. My claim (commonly called belief) that “humans are thinking animals” depends on host of other such beliefs. For example: my concepts of humans, thinking, and animals. Also recall that knowledge claims require evidence (justification), they are worthless without justifications. So many other beliefs will be required for justifying my clams that humans are thinking animals. The simple statement that “Humans are thinking animals” is of no educational value unless my assessor knows that (i) I understand the meaning of that statement, (ii) I have adequate evidence for the statement, and that (iii) I understand principles that evidence necessarily required to make a knowledge claim.

All the above would be useless unless the edcatee can make proper use of this in her real life problems. Therefore, the in addition to understanding all the above, the assessor also need to know that I can make use of this repertoire of acquired skills, values and knowledge. This will require setting up real life situation, watching me in those situations and also understand my reasons behind dealing the way I deal with those situations.

To summarise, then, a worthwhile educational assessment requires the following:

1. In skills

1. Performance of a task/act. This will demand detailed observation of performance at multiple times. [Observation]
2. The leaner’s behaviour with others. Again, detailed multiple observation are required. [Observation]
3. Moral principle guiding the behaviour. The assessor will need information on principles behind that behaviour. [Narrative statements of dialogue]
4. Concepts acquired by the learner. [Mostly narrative, may be performance of intellectual tasks.]
5. Knowledge claims acquired by the learner. [Mostly narrative, may be performance of intellectual tasks.]
6. Principles of knowledge claims acquired by the learner. [Mostly narrative, may be performance of intellectual tasks.]
7. Interconnections and mutual interdependence of between the knowledge claims. [Mostly narrative, may be performance of intellectual tasks.]
8. Performance in real life situation, not for proving or showing anything but just living. [Observation]

2. The learner’s knowledge base for that skill. This will require information regarding her beliefs on that particular skill. More complex the skill more complex and larger the set of belief behind it. [Narrative statements or dialogue]
1. In values

3. Sincerity. Information on whether the person assessed is pretending or not. [the assessors judgement arrived on grounds of behaviour and narrative/dialogue. Will always remain doubtful.]
2. Knowledge and understanding

5. [All that is written in 3 (a to d) can be formulated as knowledge claims.]
3. Ability to use skills, vales and knowledge in appropriate situations.

2. And all that is written in points 1 to 3 above.

Now the question arises: Who can do it? And how could all this be done? In the present educational discourse there are too many purposes and too many candidates who want to do learning assessment. In this little piece I will consider the duty and clams of the teacher and claims of the large scale assessors.

First, the claims of the large scale assessors. They usually carry standardised paper-pencil tests, when really large scale multiple choice questions. Observation and dialogue with the learners are used only on a minuscule scale. Through the paper-pencil test one can find out only the statements (knowledge claims) preferred by the learner. Understanding of the entirely of belief system that gives meaning and justification to that statement remains hidden to the large scale assessor. As a result she is assessing what Andrew Davis calls thin knowledge, and can never reach with reasonable reliability to deep knowledge. But it is the deep knowledge that is required in real life situations and further growth of learning. Gerard Lum has convincingly argued that large scale standardised testing methods necessarily have to take what he calls “prescriptive” route to testing. Meaning that there are questions and pre-set answers to them. If the child reproduces the desired answer than gets full marks, is does not gets no marks. This makes the tests ‘highly reliable’ in what they are testing, e.i. ability to produce desired answer. But what is the educational worth of that answer? To understand this he creates some right/wrong scenarios. A good right/wrong scenario in geography context could be: suppose you ask a child “who is the Prime Minister of India?” Suppose further that the child answers: “Dr. Manmohan Singh”. She is of course correct and gave the desired answer. Now suppose that she also believes that Manmohan Singh is from BJP. Then her answer in spite of being correct does not carry much of understanding of the political situation in India. This kind of right/wrong scenarios are a plenty in science, maths and social sciences. But you can never discover them unless you ask further questions depending on the context. The method of asking further questions in context Lum calls “expansive” and opposed to “prescriptive”. Expansive route could be taken only by the person administering the test. And that makes the testing more valid in terms of representing the child’s state of knowledge; but less reliable as the subjectivity of the situation will seep in. There could be other reasons related with cost of large scale testing that do not allow the researchers to use expansive mode. But academically speaking even if they are prepared to bear the additional costs the problem of reliability will remain. Thus, as far as knowledge related issues are [in the list above: 1(b), 2(b and c) and 3 (a to d)] the large scale testing seems to be much less satisfactory than usually assumed.

The remaining: behaviour and values are eve more difficult in large scale testing. Therefore, one can safely conclude that as far as educational worthwhileness of the things that can be tested reliably remains vey low in large scale testing. This raises doubt about the claims made for ‘evidence’ based policy as far as that evidence is generated with such methods. One can of course say that there is a high correlation between the prescriptive standardised testing and the ‘actual’ understanding of the children. But the protagonists of large scale testing build their arguments on the research based evidence, so where is the evidence that proves this later claim?

Now we come to the second candidate as assessor: the teacher. I need not spend much time on arguing that the teacher is best suited for the worthwhile assessment of the learners. A glance at the list we have generated above will suffice to convince one that the required evidence and opportunity for further investigation is readily available to the teachers. How can they do it? Answering that questing will require another short article; but one thing can be said readily: in the process of their actual teaching in the classroom, no separate efforts are required. What one needs to do is understand the pedagogy properly and keep some minimal records. If we consider the activity of teaching then we can immediately see that without observing the child’s ways of activities, and without spotting and eliminating her right/wrong scenarios one can not teach at all. Therefore, continuous assessment is a necessary ingredient of a pedagogy aimed at developing the child’s capabilities and understanding. That would be much of more worth that all the noise being made on continuous comprehensive assessment and so called quality measuring large scale testing.

If this common sense analysis is accepted then the djinn of student assessment as inflated by the large scale standardised testing should be put back into the bottle of pedagogy where the teacher can deal with it suitably. We should all work to drive the standardised testers out of business. Not with the malice for them in our heats; but with suggesting them a better field of research: how to make it possible that the teachers keep reliable records of their own assessment and how to collate them to generate reliable knowledge on the health of schools and even health of the system. Figuratively speaking: how to use the activities of the numerous djinns properly capped in numerous bottles to gain reliable and valid insight into the collective functioning of all these djinns? This is the issue people in love with scale and moved by desire to influence policy should engage with. At present they seem to be marching confidently in the wrong direction.

This entry was posted on Sunday, June 16th, 2013 at 10:33 am and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.