Aklilu Yilma and Klaus Wedekind
SIL International
2002
2
Contents
1 Introduction
2 Oromo Preliminary Form of the SRT
3 Where Are We in the Development of SRT “Sentence Repetition Tests”?
References
3
1 Introduction
Bilingualism has turned out to be a critical factor for several of the language groups surveyed. A solid tool for
mass assessments of bilingualism clearly is what we need. In an earlier report, Aklilu Yilma presented the
preliminary form of the Amharic “Sentence Repetition Test” (SRT). Here he presents the preliminary form of the
Oromo SRT, which is also being developed by him.
To provide an overview, Klaus Wedekind presents the second part of this report displaying the various stages in
developing an SRT test battery.
Marco Last and Deborah Lucassen, a team from Leiden, have kindly responded to a request to elicit 320 words
from Chai for us. The list is included with this report.
2 Oromo Preliminary Form of the SRT
P1–P3: Pretest sentences for warming up:
P1 Fardi kun bay'eesa gulufa
fardi kun baj
/eesa gulufa
horse this well it-gallops
This horse gallops well
P2 Mee kookii keeraa na cuffacciisi
mee kookii keeraa na
tS'uffattSiisi
let peach on me bite-caus.
Please let me bite off a piece from your peach
P3 Hojii argacuun hedduu rakkoo ta'eera
ho
dZii argatSuun hedduu rakkoo ta/eera
job to-get very problem has-become
It has become very difficult to get a job
Sentences 1–49 are the preliminary set of SRT test sentences. These forty-nine sentences are now being
screened (cf., the flow chart, 116–131) to focus on the “most discriminating” fifteen sentences.
1 Guurbaa! eerrbaatta kee nyaadhuutti rafii
guurbaa! eerrbaattakee
¯aaÎuutti rafii
boy! your-supper having-eaten you-sleep
Boy, eat your supper and sleep!
2 Sireen kee nama tokko qofa rafisti
sireenkee nama tokko k'ofa rafisti
bed-your person one only sleep-caus.-she
Your bed is only for one person
3 Manguddoon oromoo jecoota baay'ee beeku
manguddoon oromoo
dZetSoota baaj/ee beeku
elders-of Oromo sayings many they-know
Oromo elders know many sayings
4
4 Dhiinuumma irra firumman jiraacuu wayya
Î
iinuumma irra firumman dZiraatSuu wajja
enmity more-than relative to-live better
Living in friendship is better than enmity
5 Ajaja abbootii ofii kabajuun ni barbacisa
a
dZadZa abbootii ofii kabadZuun nibarbatSisa
order fathers self respect it-is-necessary
It is necessary to respect one's father's orders
6 Fuudhee woggaatti mucaa durbaa tokko dhalce
fuu
Îee woggaatti mutS'aa durbaa tokko ÎaltSe
of-married within-a-year baby girl one give birth-caus.-he
He has got a daughter after a year's marriage
7 Akka ati mul'atte nageenyi haa mul
/atu
akka ati mul
/atte nagee¯i haa mul/atu
as you have-been-seen peace let be-seen
As you come let also peace come
8 Namni seena abootiisaa hinbeekne jaamaa dha
namni seena abootiisaa hinbeekne
dZaamaa Îa
man-nom. history father’s-his who-not-know blind is
A man who does not know his father's history is blind
9 Makiinaan kee nama afur caalaa baacuu hin dandeessu
makiinaankee nama afur
tS'aalaa baatSuu hindandeessu
car-acc.-your person four more-than to-carry she-can-not
Your car can not carry more than four persons
10 Waan ofi jaalatan ormaafis gocuun ni barbacisa
waan ofi
dZaalatan ormaafis gotSuun nibarbatSisa
what self like to-others to-do it-is-essential-caus.
It is essential to do for others what one likes to do for himself
11 Jireenyaa baadiyyaa irra jireenya magaalaa wayya
dZ
iree¯aa baadijjaa irra dZiree¯a magaalaa wajja
to-live country-side from to-live town it-is-better
Living in town is better than living in the countryside
12 Eenyu iyyuu seera olitti jiraacuu hin danda
/u
ee
¯u ijjuu seera olitti dZiraatSuu hindanda/u
who ever law above to-live not-able
Nobody is to live above the law
5
13 Torbanitti altokko mataa koo dhiqacuu na barbaacisa
torbanitti altokko mataakoo
Îik'atSuu na barbaatSisa
in-a-week once head-my to-wash I necessary-caus.
I have to wash my hair once a week
14 Wal galuun, wali yaaduufi wal danda'uu dha
wal galuun, wali jaaduufi wal danda
/uu Îa
each other to-understand one-another to-care-and each-other to-tolerate is
To know one another other is to take care of, and tolerate one another
15 Aadaa isa bade deebisun dirgama keenya dha
aadaa isa bade deebisun dirgama kee
¯a Îa
culture which lost to-return obligation our is
It is our obligation to develop our lost culture
16 Manni warra keenyaa kuunno gama sanaa mul'ata
manni warra kee
¯aa kuunno gama sanaa mul/ata
house-nom. parents our that other side over-there is-seen
Our parents' house is seen over there at the other side (of something)
17 Ijollee kootta! goodaa gubbaa baanee farda gulufna
i
dZollee kootta! goodaa gubbaa baanee farda gulufna
boys you(pl.) come! field up-there going horse we-ride
Come on boys, let us go to up there to the field and ride a horse
18 Kaaleessaa fi har'a utuman gad hin taa'inan oole
kaaleessaa fi har
/a utuman gad hintaa/inan oole
yesterday and today none down not-resting spent-a-day
I spent yesterday and today without resting
19 Aduun ganamaa nioo'iiti malee nama hin gubdu
aduun ganamaa nioo
/iiti malee nama hin gubdu
sun-light morning it-warms-fem. otherwise man she-not-burn
The morning sun warms, otherwise it does not burn man
20 Ani gabaabbaDus kan ana irra utaale hinjiru
ani gabaabba
Îus kan ana irra utaale hindZiru
I-nom. if-short whoever I-dat. on jump-past did-not
Even though I am short, no one has jumped over me
21 Ha'as akkuma kaleesaa caljettani taa'uun sirrii miti
ha
/as akkuma kaleesaa tS'aldZettani taa/uun sirrii miti
today-too like yesterday keep-quiet to-sit right it-is-not
These days too, it is not right to sit quietly like yesterday
6
22 Baga waanuma xaqishu kana irratti wal hubaanne
baga waanuma t'ak'i
Su kana irratti wal hubaanne
it-is-good thing minute this-fem. on each-other know-we
It is good to know each other on this minute problem
23 Seenaa abbooti keenya beekuun nu dargaggootara eegama
seenaa abbooti kee
¯a beekuun nu dargaggootara eegama
history fathers ours to-know we young-pl.-from it-expected
It is expected from us young men to know the history of our fathers
24 Waan fedhanu argacuuf hojji itti cimsuu ni barbaacisa
waan fe
Îanu argatSuuf hodZdZi itti tS'imsuu nibarbaatSisa
something which-wished to-get work on hard it-is-necessary
It is necessary to work hard in order to get what one wishes
25 Midhaan nyaatanii bishaan irratti yoo dhugan gaarii dha
mi
Îaan ¯aatanii biSaan irratti joo Îugan gaarii Îa
grain eat water on-it when drink good is
It is good for us to drink water after having eaten food
26 Bunni yoo nyaata dura dhugani qonqo nama cufa
bunni joo
¯aata dura Îugani k'onk'o nama tS'ufa
coffee when food before if-drunk appetite person it-closes
If you drink coffee before a meal it spoils your appetite
27 Fardis kunoo dirreenis kunoo yoo dhuguma onneen jiraate
fardis kunoo dirreenis kunoo joo
Îuguma onneen dZiraate
horse-too here-it-is plain-too here-it-is when true heart if-exist
Here is a horse and a racing ground for whoever is truly brave
28 Hojin waan natti bayyatuuf beellama siif kennuu hindando'u
ho
dZin waan natti bajjatuuf beellama siif kennuu hindando/u
work that on-me load appointment for-you to-give I-can-not
Since I have a lot of work to do, I cannot give you an appointment
29 Roobni hamaa ogga roobu midhaan akka balleessu ni bekama
roobni hamaa ogga roobu mi
Îaan akka balleessu nibekama
rain-the heavy when it-rains grain as to-destroy it-is-known
It is known that when it rains the rain may destroy the grain
30 Karaa hundumaan mataa ofii sirriitti danda'uun barbaacisaa dha
karaa hundumaan mataa ofii sirriitti danda
/uun barbaatSisaa Îa
way in-every head self correctly to-be-able essential-which is
It is essential to support oneself well enough in every way
7
31 Waan tokko utuu hin haasa'in dura itti yaaduun ni barbaacisa
waan tokko utuu hinhaasa
/in dura itti jaaduun nibarbaatSisa
something one if not-speaking before of to-think it-is-necessary
It is necessary to think over something before talking about it
32 Ummani Oromo aadaa boonsaata'e akka qabu seenaan niibisa
ummani oromo aadaa boonsaata
/e akka k'abu seenaan niibisa
nation Oromo culture proud which-is as-has history-also it-explains
History tells that the Oromo nation also has a glorious culture
33 Dharagägo baay'en yeroo har'a gorsa jaarsolii fudhacuu hin barbaadan
Î
aragägo baaj/en jeroo har/a gorsa dZaarsolii fuÎatSuu hinbarbaadan
youth many time today advice elders to-take they-do-not-want
Nowadays, many youngsters do not want to take the advice of elders
34 Meeshaallee aadaa Oromoo keessaa innitokko qabee akkata'e ni beekama
mee
Saallee aadaa oromoo keessaa innitokko k'abee akkata/e nibeekama
instruments culture Oromo in the-one
Qabee as it-is-known
It is known that Qabee is one of the cultural instruments of Oromo
35 Hojiin isaa kun hiikuu qofa odoo hin taane fooyessuu dha
ho
dZiin isaa kun hiikuu k'ofa odoo hintaane foojessuu Îa
work his this to-translate only do-not to-improve is
His work is not only to translate but also to improve it
36 Yeroo kam iyyuu waranni ilmaan namatti rakkoo hedduu uuma
jeroo kam ijjuu waranni ilmaan namatti rakkoo hedduu uuma
time whenever war sons of-man problem many creates
Whenever there is war, it creates many problems for people
37 Eeboo abbaan koo darbate, baha biiftuu ka'ee dhiha biftuu bu'e
eeboo abbaankoo darbate, baha biiftuu ka
/ee Îiha biftuu bu/e
spear father-my which-he-threw, rise sun having-started sun-set fall-past
The spear my father threw started rising at sunrise and fell at sunset
38 Yoom iyyuu taanaan hojiin jireenya ilmoo nammatiif barbaacissa dha
joom ijjuu taanaan ho
dZiin dZiree¯a ilmoo nammatiif barbaatSissa Îa
when ever if-occur work living son mankind it-necessary is
At all times work is essential for the existence of mankind
39 Ani obboleessa angafa hinqabu garuu obbolaa durbaa laman qaba
ani obboleessa angafa hink'abu garuu obbolaa durbaa laman k'aba
I brother elder do have-not but sisters girl two I-have
I do not have an elder brother, but I have two sisters
8
40 Finfinneen magaalaa guddo sabaa fi sab-lammootni keessatti argamanii dha
finfinneen magaalaa guddo sabaa fi sab-lammootni keessatti argamanii
Îa
Addis Ababa city big nation and nationalities in-the found-which is
Addis Ababa is a big city in which various nations and nationalities live
41 Kan ana irra yaa'u imimmaani, kan ani boo'us garaadhaani
kan ana irra jaa
/u imimmaani, kan ani boo/us garaaÎaani
that I on to-flow tear-the that I cry-caus. from-my-abdomen-is
The tears flow on my cheek, but my crying comes from my deepest inner being
42 Isa boruu rabbitu beeka mee isa har'aa nagaan nu haabulcu
isa boruu rabbitu beeka mee isa har
/aa nagaan nu haabultSu
for tomorrow God knows-he let for today peace-with we pass-the-night
Only God knows about tomorrow, let us pass this night in peace
43 Hanga humni ofii danda'e dalaganii kan hin dandeenye dhiisuu dha
hanga humni ofii danda
/e dalaganii kan hin dandee¯e Îiisuu Îa
amount force self to-be-able to-work that not-able-which to-leave is
It is better to do as much as one can do, and leave aside what one cannot do
44 Kuno ani dhufe mul'adhee waan natti himtu dadaafiitii natti himi
kuno ani
Îufe mul/aÎee waan natti himtu dadaafiitii natti himi
here I come-past appear-passive thing I-to tell-which-you quick-quick I-to tell-me
Here I am, tell me quickly what you are going to tell me
45 Gaararraan waanuma argite fakkaatteetu balaa ishii irra ga' jalaa baati
gaararraan waanuma argite fakkaatteetu balaa i
Sii irra ga/u dZalaa baati
chameleon everything see look-alike danger she from happening under escape
A chameleon escapes from danger by looking like whatever it sees
46 Fardi akka konkolaata konkolaacuu baatu iyyuu dhimma abbaa baasuuf ni gargaara
fardi akka konkolaata konkolaa
tSuu baatu ijjuu Îimma abbaa baasuuf nigargaara
horse-nom. like car to-drive if-not-occur ever affair father to-carry-out it-helps
Even though a horse cannot drive like a car, it helps to carry out the business of its owner
47 Dhukkuba irraa ofeeguuf mataa keenyaa fi naannoo keenyaa qulqullessu nu barbaacisa
Î
ukkuba irraa ofeeguuf mataa kee¯aa fi naannoo kee¯aa k'ulk'ullessu nu barbaatSisa
disease from to-protect head ours and surroundings our cleaning we it-is-necessary
We have to keep ourselves and our surroundings clean in order to be protected from disease
9
48 Yeroon walirraa baqatanii hafan waan darbeef, amma walitti adeemnee
jeroon walirraa bak'atanii hafan waan darbeef, amma walitti adeemnee
time-the each-other keep-away-from to-remain since passed now together we-come-closer
wal gaafacu qabna
wal gaafa
tSu k'abna
each-other to-ask have-to
The time of keeping away from each other has passed, now we have to come closer to each other
and to ask each other questions
49 Barnootini waan gaafa dhalannee jalqaba hanga gaafa duunutti waan nu irra ga'uu dha
barnootini waan gaafa
Îalannee dZalk'aba hanga gaafa duunutti waan nu irra ga/uu Îa
education things day we-born start-raising up-to day we-die things we on which-happen is
Education is whatever happens to us from the day we are born until the day we die
3 Where Are We in the Development of SRT “Sentence Repetition Tests”?
The following flow chart has been distilled from Carla Radloff's SRT book.
There were several reasons for reducing her handbook in this way:
(1) We wanted to understand each single step in its relation to all other steps, and (2) as we are developing SRT
tests for two Ethiopian languages, we want to see, at every point of the work, where exactly we are, and what is left
to be done. In her book, Carla Radloff warns her readers that solid SRT results can only be expected where the
preparation of the SRT test material itself is solid. So (3) we want to make sure all of our preparatory steps are taken
care of and checked out.
In the last few years and months, additional studies have appeared which review, reassess, and refine Carla
Radloff’s original SRT work (e.g., Wetherill 1991, Hatfield et al. 1993, Karan 1993.)
Reservations have been expressed, for instance, about the astonishingly high correlations with other
measurements of L2 proficiency—a correlation as high as r=.94 really is fantastic! (Radloff 1990:69, 93, 165) But
we would hasten to affirm that we are very satisfied with any test which dicriminates above “3” in a nine step
scale
1—provided such a test can be administered to large numbers of testees in actual field work.
SRT can!
Tests which on the field need excessive amounts of time and care are self-defeating: bilingualism needs to be
tested on large numbers of people. A bilingualism test which cannot be administered easily on the field is not
helpful.
SRT is!
2
1
The nine steps of this scale, based on the scale of the “Foreign Service Institute” (FSI), range from 0+ “very
minimal proficiency”, over 1 1+ 2 2+ 3 3+ 4, to 4+ “approachnig native speaker proficiency”.
2
In the subsequent flow chart, all page numbers refer to Radloff 1990.
10
Flow Chart
STEP 1: DEVELOPING A PRELIMINARY SRT TEST
1 The Main Researcher (MR) identifies SRT personnel as listed below:
2 Needed are: 1–2 persons as Research Team (RT), including a “Main Researcher” (MR) (p. 39)
3 Needed are, in addition: 10–-14 Educated Mother Tongue speakers (EMT)
4 RT identifies among these, “field workers” or “raters” (pp. 135–137)
5 Needed are, in addition: >50 speakers whose L2 proficiency is to be “rated”
6 RT and EMT both identify among these about 50 “ratees” (pp. 40, 138)
7 Needed: 1–10 Test Administrators (TA) for step 5 later on only (p. 41)
b
8 RT obtains fifty sentences which fulfill the conditions given (p. 42)
9 RT writes initial phonetic transcription of these sentences
10 RT and EMT record all fifty sentences with three sec. pause between sentences
11 RT and EMT examine and select the most appropriate sentences (p. 43)
12 If EMT rejects a sentence, RT writes down the reasons why
13 RT plays to two EMTs the selected sentences over headphone, one by one
b
14 If EMTs cannot repeat a sentence because it is too long, RT deletes it (p. 44)
b
15 If there are now less than fifty sentences:
go to
a 8
16 RT checks for variety and length again (p. 44):
go to
a 8
go to
a 14
17 RT assembles and cuts apart the selected sentences
18 RT orders them from short to long, counting words (or syllables) (p. 45)
19 RT keeps the three shortest sentences for practice (p. 45)
20 If no short sentences of 5–9 words, RT adds such
21 RT selects one EMT to record the fifty sentences
22 RT and EMT set up equipment in a quiet place under conditions described for quality recording (p. 45)
b
23 EMT reads one sentences at a time, starting with the first of the list.
24 EMT practices the sentence out loud
25 RT and EMT records the sentence in natural standard pronunciation, not pausing
26 RT records three sec. pause after each sentence
b
27 EMT takes the next sentence etc., applying the same procedure to all fifty sentences:
go to
a 23
28 RT produces a high quality copy of this tape
29 RT transcribes the tape phonetically (p. 46)
11
30 RT checks the transcript for critical sounds of minimal difference
31 RT adds a free translation
32 RT adds a word-by-word translation for all sentences
33 RT prepares a transcript in four lines as exemplified (p. 46)
34 RT regards this set of sentences as the
preliminary SRT test for that language.
STEP 2: REPORTED PROFICIENCY EVALUATION (RPE)
35 RT discusses the “RPE” with EMTs and applies the L2 proficiency criteria (pp. 135–138)
36 RT identifies among the EMTs about fifteen “TRAINED FIELD WORKERS” and “RATERS”
37 RT familiarizes these EMTs with FSI / RPE criteria (pp. 147–151)
38 RT discusses with EMTs the criteria of (pp. 147–151)
39 RT and EMTs establish which is the lg. most read by EMT raters
40 RT establishes the need to follow procedures rigorously
41 RT secures addresses of the “RATEES” and makes sure relations to them are close
42 RT and EMTs translate the descriptions (pp. 147–151) in expanded form (p. 135)
43 RT and EMTs work as a team, making translating part of EMTs’ training
44 RT writes the translation in neat form ready for the raters (EMTs)
b
45 RT identifies >15 adult educated “RATERS” among the EMTs (p. 137)
46 Raters must have close relations to 3–5 potential ratees who speak the test language as a second language (L2)
47 RT checks whether these L2 speakers represent all levels of L2 proficiency
48 RT checks whether the EMTs will have access to the ratees for step 3
b
49 If not:
go to
a 35
50 RT rechecks whether the EMTs are adult, mature, accessible MT (p. 137–138)
51 RT writes questionnaire for EMTs (p. 137)
52 RT decides which “RATEES” to involve, on the basis of how frequent and how intensive interactions are
b
53 If not a sufficient number of RATEES are accessible:
go to
a 35
go to
a 45
54 RT and EMT/raters study interview technique and protocol (pp. 142–143)
55 RT explains SRT to EMT/raters purposes of SRT (p. 139)
56 RT shows that the SRT is a tool which helps to understand how people learn languages in different ways
57 RT confirms to the EMTs the confidentiality of ratees' test results
58 RT informs EMTs that SRTs will be taken from the ratees
12
59 RT shows EMTS a copy of the translated criteria which maybe the EMT can familiarize himslef with, overnight
60 RT plans with each rater for one grading and rating session
61 RT sits down with the first EMT rater
62 EMT rater thinks of his 3–5 friends who will be “RATEES”
63 EMT prepares to rank these 3–5 ratees by “PERCEIVED PROFICIENCY” (p. 140)
b
64 EMT ranks all 3–5 ratees for
accent
65 EMT ranks all 3–5 ratees for
grammar
66 EMT ranks all 3–5 ratees for
vocabulary
67 EMT ranks all 3–5 ratees for
fluency
68 EMT ranks all 3–5 ratees for
comprehension
69 EMT rater considers the best ratee and the A–F scale (pp. 142–144)
70 EMT rates him for accent, keeping his rank in mind (p. 141)
71 EMT assigns letters A or A+ etc., accordingly
72 EMT rater considers the weakest ratee and the A–F scale
73 EMT rates him for accent, keeping his rank in mind (p. 141)
74 EMT assigns letters A or A+ etc., accordingly
go to
a 69
75 EMT rater considers 2nd best ratee and the A–F scale
76 EMT rates him for accent, keeping his rank in mind
go to
a 69
77 EMT rater considers each other ratee in turn
78 EMT rates the ratee for accent, keeping his rank in mind
79 EMT assigns letters A or A+ etc., accordingly
b
80 EMT goes through the same routine for
grammar (pp. 142–144):
go to
a 69
81 EMT goes through the same routine for
vocabulary
go to
a 69
82 EMT goes through the same routine for
fluency
go to
a 69
83 EMT goes through the same routine for
comprehension
go to
a 69
84 RT identifies among the EMTs RATERS who should contact RATEES
85 RT or EMT contacts first ratee and gathers information (p. 142)
86 RT fills in questionnaire (p. 142)
b
87 RT administers the SRT test to one ratee at a time, starting with the first ratee (p. 142)
b
88 RT works with next ratee, etc.:
13
go to
a 87
89 RT calculates “WEIGHTED SCORE” for one ratee at a time, starting with the first ratee (p. 143)
b
90 RT does the same for the next ratee etc.:
go to
a 89
b
91 RT converts ratings “A”, “A+”, “B”, “B+” (...) “F” into numbers (p. 144)
92 RT calcualtes the total points score for SRT calibration (p. 142)
93 RT converts the “WEIGHTED SCORES” to RPE proficiency levels (p. 144)
b
94 RT reports about the SRT development in terms of the “
COMPOSITE RPE LEVEL DESCRIPTIONS” (p. 144):
RPE level 0+: Very minimal proficiency.
RPE level 1: Minimal, limited proficiency.
A person at this level has a very heavy accent which makes understanding difficult and forces people to ask for repetition. There seem to be more
mistakes in grammar than correct usage, except for stock phrases. Vocabulary is limited to basic personal and survival areas. Speech is slow
and halting except for short or routine sentences. Understanding is limited to slow, very simple speech, with very frequent repetition and
rephrasing.
RPE level 1+: Limited, basic proficiency.
RPE level 2: Adequate, basic proficiency.
A person at this level has a heavy accent that forces people to concentrate when listening and sometimes causes misunderstanding and gives the
appearance of errors. Some important grammatical rules are not controlled which occasionally causes misunderstanding and even irritation.
Vocabulary is broad enough for daily topics, but limited in some common domains and sometimes inaccurate. Hesitations and jerkiness are
frequent. Sometimes sentences cannot be completed. Understanding is possible if people speak carefully and simplify their speech somewhat,
but they must repeat and/or rephrase frequently.
RPE level 2+: Good, basic proficiency.
RPE level 3: Good, general proficiency.
A person at this stage has a marked “foreign” accent, with occasional mispronunciations, but these do not interfere with understanding. Imperfect
control of some grammatical patterns causes occasional errors, but understanding is not affected. Vocabulary is adequate to cope with varied
social situations and special interests in professional domains with some circumlocutions. Speech is occasionally hesitant and perceptibly nonnative
in speed and evenness. Normal educated speech is understood quite well, with only occasional need for repetition or rephrasing.
RPE level 3+: Very good, general proficiency.
RPE level 4: Excellent proficiency.
A person at this level still has a very slight accent but no longer mispronounces words. No patterns of grammatical error remain, and only rarely
are errors made. Vocabulary is broad and precise, adequate for all technical, social, and practical situations. Only a slight difference in the
speed and evenness of speech separates this speaker from a native speaker. Comprehension is complete except for very slurred or rapid speech
or perhaps uncommon words or idioms.
RPE level 4+: Approaching native speaker proficiency.
b
STEP 3: RATING THE PARTICIPANTS' PROFICIENCIES (RPE)
95 RT identifies “TEST ADMINISTRATOR(S)” (TA) among RTs or EMTs
96 RT ensures that the TA is/are the same for all tests (p. 49)
97 If the TA cannot be the same person(s), establish scoring standards
98 RT prepares a training tape for later field testing and trains RTs/EMTs in procedure
99 RT seeks permission from ten top ratees to tape record responses (p. 50)
100 RT sets up “STANDARD PROCEDURE”: proper introduction/explanation/pauses/confidentiality. (pp. 51–52)
101 RT sets up equipment to record first the speaker who scored highest on RPE
102 RT decides whether there are sufficient easy and difficult sentences
b
103 If not sufficient, RT repeats:
14
go to
a 8
104 RT plots a scattergram (p. 51)
b
105 RT decides whether the differences at high and low level are represented sufficiently
b
106 If not sufficient, EMT selects more sentences:
go to
a 8
go to
a 105
107 If new sentences are added, RT asks same EMT to record them:
go to
a 8
108 If new sentences added, RT tests with high level RPE ratees:
go to
a 104
109 RT does same with next best ratee etc.:
go to
a 104
110 RT asks permission of 2–3 ratees of each level to tape their responses
111 RT defines what is an error (pp. 53–55)
112 RT establishes scoring standards and lg. specific errors
113 RT uses new sheet for each ratee to monitor deviations from SRT stimuli
114 RT underlines expected deviations in the test transcript (p. 173)
115 RT prepares to shorten and calibrate the SRT list of sentences
STEP 4: CALIBRATING THE SRT TEST
116 RT arranges the data of fifty testees who had been rated on RPE (p. 56–57)
117 RT makes this sheet the basis for the subsequent selection process
118 RT arranges data according to total scores, left-right corresponding to strong-weak
119 RT arranges sentences according to total score, top-bottom corresponding to easy-difficult.
120 RT calculates “DISCRIMINATION INDEX” (p. 57)
121 RT lists actual scores in first line
122 RT lists rearranged scores below (p. 58)
123 RT calculates differences below (p. 58)
124 RT calculates “DIFFICULTY LEVEL” (p. 59)
125 RT selects final fifteen sentences (p. 60)
126 RT looks at the “DISCRIMINATION INDEX” of each sentence
127 RT selects fifteen sentences with lowest discrimination index (>0 but <40 p="p">
128 If less than fifteen, RT adds from the rest, getting full range represented
129 If more than fifteen, RT selects by factors like quality of recording, content, etc.
130 If more than fifteen, RT eliminates sentences with difficulty level >.95 or >.90
131 RT considers the resulting set of fifteen sentences the
FINAL FORM OF THE SRT TEST (p. 61)
b
132 RT removes from the chart those rows and sentence totals which were not selected
15
133 RT calculates an “EXTRACTED SCORE” for each person tested (p. 61)
134 RT checks the list of “EXTRACTED SCORES” of all fifty subjects (p. 58)
135 RT selects three “PRACTICE SENTENCES” (p. 62)
136 RT selects the first “PRACTICE SENTENCE” of 5–9 words
137 RT selects the third to have the same length as the 1st of the fifteen test sentences
138 RT selects second to have intermediate length
139 RT records the “PRACTICE SENTENCES”, with three sec pause intervals
140 RT then records fifteen selected SRT sentences
141 RT considers this
THE ACTUAL TEST TAPE (p. 62)
142 RT selects a standard by which to calibrate the SRT test (p. 63)
b
143 RT plots final SRT “EXTRACTED SCORES” (x) versus RPE raw score (y) (p. 63)
144 RT visually examines normality of this “SCATTERGRAM” (x 1–40, y 0–99)
145 RT calculates the “LINE OF REGRESSION” to understand the actual strength of this SRT (pp. 63–65)
b
146 If correlations are too weak, RT repeats:
go to
a 152(??)
147 RT calculates a high point and low point, connects “LINE OF REGRESSION” (p. 108)
148 RT uses formula (p. 66/26) to calculate range of SRT equiv. for RPE scores
b
149 If the curve flattens out at the upper end, RT adds more difficult sentences:
go to
a 8
150 RT calculates “STANDARD ERROR OF ESTIMATE” (SEE), inserting SEE points (p. 67)
151 RT calculates the “COEFFICIENT OF CORRELATION”, ideally r>0.9
b
152 RT improves correlation by examining test procedures and administration
153 RT does some control testing of the final SRT form, various age groups, etc.
154 RT checks where the cut-off point is: maybe 3+; ideally 4
155 RT checks for flattening-out of curve: if so
go to
a 8
go to
a 143
156 RT considers the
SRT test calibrated
STEP 5: TRAINING SRT TEST ADMINISTRATORS TO USE THE TEST
157 RT identifies SRT Test Administrators (TA) and trains them
158 RT and TAs
use the SRT bilingualism test in the field, with reference to FSI standards.
16
References
Hatfield, D., C. Radloff, T. Bergman, M. South, and B. Wetherill. May 1993. “The Cameroon study: A comparison
of second language proficiency testing methods”. ms.
Karan, M. E. March 1993. Towards refining the SRT: Observations drawn from the Sango SRT. ms.
Radloff, Carla. 1990. Sentence repetition testing, Dallas.
Wetherill, B. 1991. Report of an analysis of data from the bilingualism test in Cameroon. ms.
No comments:
Post a Comment