Content uploaded by Musawenkosi Mabaso
Author content
All content in this area was uploaded by Musawenkosi Mabaso on Apr 13, 2016
Content may be subject to copyright.
South African Journal of Psychology 1 –16 © The Author(s) 2015
Reprints and permissions: sagepub.co.uk/journalsPermissions.nav
DOI: 10.1177/0081246315599476 sap.sagepub.com
Predictive power of psychometric
assessments to identify young
learners in need of early
intervention: data from the Birth to
Twenty Plus Cohort, South Africa
Linda Richter1,2, Musawenkosi Mabaso3 and
Celia Hsiao2
Abstract
The use of psychometric assessments during early childhood to predict children’s later outcomes
is vital for early intervention. This study evaluates the predictive power of eight psychometric
assessments administered during early childhood as screening measures for identifying those in
need of early interventions to prevent late school entry and grade repetition. The measures are
the Bayley Scales of Infant Development and the Griffiths Mental Development Scales at 6 months
and 1 year; the Vineland Social Maturity Scale and the Behaviour Screening Questionnaire
at 2 years and 4 years; the Revised Denver Prescreening Developmental Questionnaire at
5 years; and the Conners’ Teacher Rating Scale, the Draw-a-Person, and the Raven’s Coloured
Progressive Matrices at 7 years. We used receiver operating characteristic curve analysis to
examine predictive values of the measures, and the area under the curve to assess sensitivity and
specificity. Findings suggest that with a moderate degree of diagnostic accuracy, the Bayley Scales
of Infant Development at Year 1 with receiver operating characteristic curve (area under the
curve = 0.61; 95% confidence interval: 0.51, 0.71) and the Conners’ Teacher Rating Scale at Year 7
with receiver operating characteristic curve (area under the curve = 0.64; 95% confidence interval:
0.58, 0.70) can be used as screening measures to identify children at risk of late school entry. The
Conners’ Teacher Rating Scale at Year 7 predicted grade repetition with a moderate degree of
accuracy (area under the curve = 0.62; 95% confidence interval: 0.57, 0.67). The only statistically
significant covariate-adjusted model showed that young maternal age (β = –5.25; 95% confidence
1DST-NRF Centre of Excellence in Human Development, University of the Witwatersrand, South Africa
2 MRC/Wits Developmental Pathways for Health Research Unit, Faculty of Health Sciences, University of the
Witwatersrand, South Africa
3HIV/AIDS, STIs and TB, Human Sciences Research Council, South Africa
Corresponding author:
Celia Hsiao, MRC/Wits Developmental Pathways for Health Research Unit, Faculty of Health Sciences, University of the
Witwatersrand, School of Public Health Building, No. 27 St. Andrews Road, Johannesburg 2000, South Africa.
Email: Celia.Hsiao@wits.ac.za
599476SAP0010.1177/0081246315599476South African Journal of PsychologyRichter et al.
research-article2015
Article
at Human Sciences Res Council on January 11, 2016sap.sagepub.comDownloaded from
2 South African Journal of Psychology
interval: –9.62, –0.88) and low socioeconomic status (β = –2.04; 95% confidence interval: –3.76,
–0.33) had a negative influence on the age at school entry as predicted by Bayley Scales of Infant
Development at Year 1. This study is the first of its kind in South Africa, and contributes to the
conceptual and empirical literature on children’s developmental assessment.
Keywords
Early childhood, early intervention, predictive power, psychometric assessments, schooling risk
Globally, an estimated 10% of young children show some form of developmental delay (Rydz,
Shevell, Majnemer, & Oskoiu, 2005). In low- and middle-income (LAMI) countries, the estimates
are as high as 23% (Gottlieb, Maenner, Cappa, & Durkin, 2009). Despite low-income environ-
ments posing greater risks for young children in these settings (Grantham-McGregor et al., 2007),
very little is known about the developmental progress of young children in LAMI countries apart
from, in some settings, basic epidemiological data (Maulik & Darmstadt, 2007; World Health
Organization [WHO], 2012).
In addition, knowledge and instrumentation for developmental assessment in LAMI countries
lag very much behind countries such as the United States, about which the American Academy of
Pediatrics (2001) stated more than a decade ago that ‘the science of developmental testing has
improved in the last 10 years’ (p. 193). The lack of suitable validated instruments is a major barrier
to assessing children’s development in LAMI countries (Engle et al., 2007; Sabanathan, Wills, &
Gladstone, 2015). This includes parent-report questionnaires, despite research in Western countries
that attests to their value (Diamond, 1993). One reason for this is that concerns have been expressed
that parents and other caregivers in LAMI countries may lack knowledge about early child devel-
opment and therefore might not be able to report accurately on their children’s progress (De
Lourdes et al., 2005; Ertem et al., 2007). As a consequence of these measurement challenges, too
little attention is given to children’s difficulties, remedial interventions, support for families, and
advocacy for the early identification and prevention of developmental delays (WHO, 2012).
The early identification of potential developmental delays is regarded as essential for effective
early intervention. However, successful screening and assessment are dependent on formal tools,
standardized for local populations (American Academy of Pediatrics, 2001).
South Africa is no exception with respect to these issues. Barriers to advancements in this field
in South Africa are similar to those experienced in other low-income and some middle-income
countries. They include a limited number of developmental psychologists including professionals
from all cultural groups, access to licensed instruments, translation into multiple official languages
and several dialects, and psychometric expertise. Among the tests that have been normed in South
Africa are the Draw-a-Person (DAP) test (Richter, Griesel, & Wortley, 1989), the Bayley Scales of
Infant Development (BSID; Richter, Griesel, & Rose, 1992), the Griffiths Mental Development
Scales (GMDS; Jacklin and Cockcroft, 2012; Mothuloe et al., 1994), the McCarthy Scales of
Children’s Development (Richter, Griesel, & Rose, 1994), and the Raven’s Coloured Progressive
Matrices (RCPM; Knoetze, Bass, & Steele, 2005). The use of different psychometric assessments
during early childhood development stages as predictors of children’s development later in life is
vital for early intervention. However, studies testing the predictive validity of earlier measures
against later assessments in LAMI countries are rare.
This study evaluates the performance of eight different psychometric measurements administered
to children between 6 months of age and 7 years as predictors of school entry and grade repetition,
at Human Sciences Res Council on January 11, 2016sap.sagepub.comDownloaded from
Richter et al. 3
among a large representative sample of South African children recruited into a prospective birth
cohort study and followed up. Receiver operating characteristic (ROC) curves were used to assess the
predictive power (sensitivity and specificity) of the different psychometric tests.
Method
Participants
This analysis uses data generated in Birth to Twenty Plus (Bt20+), an ongoing longitudinal birth
cohort project examining the socioeconomic, growth, health, development, and overall well-
being of urban South African children and their families. The main motivation for starting Bt20+
was to study children’s physical and social development in an environment of rapid social
change. The Bt20+ cohort was defined by the timing of a singleton birth within a defined period
(from 23rd April to 8th June 1990), as well as continued residence within Soweto-Johannesburg
for at least 6 months after the birth of the child (Richter et al., 2012). During this time, the study
recruited 3273 singleton infants and has followed up about 70% of the children for 25 years to
date. At enrolment, the cohort was demographically representative of the study area with the
majority of participants being Black African and comprising roughly equal numbers of male and
female participants. Detailed accounts of the study have been comprehensively described in
previous publications (Norris, Richter, & Fleetwood, 2007; Richter et al., 2004, 2007, 2011).
Not all children were seen at every data collection point, so the samples for each test overlap to
a greater or lesser extent.
Instruments
The measures selected were those that we had used before, did not require extensive adaptation,
and seemed feasible for a community-based study involving a large number of children, all of
whom had to be assessed with a defined period around their birthdays, which occurred within a
7-week window.
Each of the measures is briefly described below. Where applicable, South African adaptations
for various measures are reported:
BSID: It is a standardized behavioural measure assessing the cognitive, language, and motor
development of children between 2 months and 3 years of age. It is one of the most commonly
used instruments to assess infant development and is frequently regarded as the gold standard.
The BSID has been found to be associated with later academic achievement, measured through
assessments such as the McCarthy Scales of Children’s Abilities, the Wechsler Preschool and
Primary Scale of Intelligence–Revised, and the Preschool Language Scale–Third Edition
(Bradley-Johnson, 2001; e.g., Cooper & Sandler, 1997; Potterton et al., 2009). Individual items
contribute to either a Mental Development Index (MDI) or Psychomotor Development Index
(PDI; Bayley, 1969). Total raw scale scores were converted into South African normalized
standard scores published by Richter, Griesel, & Rose (1992).
GMDS: It is a behavioural measure intended to assess the mental abilities of infants and young
children. Several studies have examined the validity of the GMDS in South Africa. For instance,
Mothuloe, Richter, Barnes, and Schoeman (1994) found that the GMDS was significantly asso-
ciated with the Aptitude Test for School Beginners as well as end-of-year school marks among
a group of Black school beginners. For the first 2 years of life, the GMDS consists of five scales:
locomotor, personal-social, hearing and speech, eye and hand, and performance (Griffiths,
at Human Sciences Res Council on January 11, 2016sap.sagepub.comDownloaded from
4 South African Journal of Psychology
1970). Total raw scale scores were converted into a Mental Age Scale Quotient, and a General
Quotient was calculated for each scale and across all five scales.
Vineland Social Maturity Scale (VSMS): This caregiver report assesses social competence in
individuals from birth to 30 years and has been found to be significantly associated with the
Battelle Developmental Inventory (Berls & McEwen, 1999), which evaluates children’s cogni-
tive, adaptive (self-help), motor, communication, and personal-social development. A study
investigating the social maturity of rural and urban children with mental retardation in South
Africa found a strong association between the VSMS and the RCPM, an assessment of general
intelligence (Pillay, 2003). The VSMS measures five domains of adaptability: daily living skills,
communication, motor skills, socialization, and occupational skills (Doll, 1965). Total raw
scores were converted into a total-by-age score, adjusted for chronological age. Some items
were reworded to facilitate understanding, such as ‘Masticates food’ was reworded to ‘Chews
solid food, like meat or bread’.
Behaviour Screening Questionnaire (BSQ): This 12-item caregiver report questionnaire is
designed to identify preschool children with behaviour difficulties, marked developmental
delay, or physical handicap. Richman and Graham (1971) reported that the BSQ identified 75%
of the children attending a psychiatric clinic. In the present study, BSQ scores were converted
into a total-by-age score, adjusted for chronological age. Where necessary, some items were
rephrased. For instance, at both Years 2 and 4, ‘Appears markedly clumsy for age’ was changed
to ‘Seems clumsy, knocks things over, walks into things and trips frequently’.
Revised Denver Prescreening Developmental Questionnaire (R-DPDQ): This was developed to
identify those children from birth to 6 years who require a more thorough screening with the
Denver Developmental Screening Test (DDST), a tool used to screen cognitive and behavioural
problems in preschool children. Several South African studies have found a strong association
between the DDST and the GMDS (e.g., Hsiao & Richter, 2014; Luiz, Foxcroft, & Tukulu,
2004). Thirty-two culturally appropriate items were included from the R-DPDQ (Frankenburg,
Fandal, & Thornton, 1987) and the Denver Prescreening Developmental Questionnaire II
(Frankenburg, Van Doorninck, Liddell, & Dick, 1976). The items cover personal-social, fine-
motor, gross-motor, and language. Total raw scores were converted into a total-by-age score,
adjusted for chronological age. Some items were adapted to minimize cultural biases, for exam-
ple, ‘What is a hedge?’ was modified to ‘What is a fence’ as local children more commonly see
a fence than a hedge.
Conners’ Teacher Rating Scale (CTRS): This scale assesses teachers’ perceptions of the behav-
iour of children 3 to 12 years old. The 39-item scale assesses children’s conduct, cognitive, anxi-
ety, and social problems in the classroom (Conners, 1970). The CTRS has been found to
correctly discriminate between children with attention deficit/hyperactivity disorder (ADHD)
and normal children (Conners, Sitarenios, Parker, & Epstein, 1998). Scores were converted into
a total-by-age score, adjusted for chronological age.
DAP: This is a widely used procedure, but in this case is a subtest of the McCarthy Scales of
Children’s Abilities, a non-verbal assessment of children’s perceptual performance, general
cognitive, and motor development from 2.5 to 8.5 years (McCarthy, 1972). Specific character-
istics of the content and quality of children’s drawings are assigned a score with a maximum of
20. The total test score was converted into a total-by-age score. Richter, Griesel, and Wortley
(1989) reported a significant positive relationship between DAP scores and school performance
among a large sample of South African children 5 to 8 years of age.
at Human Sciences Res Council on January 11, 2016sap.sagepub.comDownloaded from
Richter et al. 5
RCPM: It is a non-verbal assessment of reasoning for young children 5.5 to 11.5 years (Raven,
Court, & Raven, 1977). It is viewed as a culturally fair measure of non-verbal intelligence and
is widely used across various settings in countries all over the world. Despite early scepticism
about its culture-fairness in South Africa (Owen, 1992), the test has been found to be a signifi-
cant predictor of academic achievement (Rohde & Thompson, 2007) and is highly correlated
with mathematics (Pind, Gunnarsdottir, & Johannesson, 2003). The total test score was con-
verted into a total-by-age score.
Outcome and explanatory variables. The current analytical sample comprises a large representative
sample of South African children on whom developmental measures, outcome, and explanatory
data were available, recruited into a prospective birth cohort study, and followed-up into their
school years. Outcome variables included two binary measures of school performance: delayed
age of entry (entered school at normal age, by age 7 = 0, delayed age of school entry 8 years or
later = 1), and repetition of at least 1 year of school in the first 6 years (did not repeat = 0, repeated = 1).
These data were obtained from mothers and other primary caregivers at annual data collection
rounds, as well as from school reports. Explanatory variables included socio-demographic charac-
teristics of the family, mother, and child.
The age at which each assessment was administered and the sample size for each measure are
shown in Table 1. The sample sizes are roughly even between girls and boys, with a slightly higher
proportion of boys in each age group.
Procedure
The Bt20+ study established an infrastructure and procedures to track children for annual assess-
ments (Norris et al., 2007), and all children were assessed at one of two project field sites (Chris
Hani-Baragwanath Hospital and the Charlotte Maxeke Johannesburg General Hospital). The CTRS
Table 1. Child age at which assessments were conducted and sample size.
Data collection
wave
Measure Sample
size
Age at
assessment
M (SD)
6 months Bayley Scales of Infant Development 559 6.60 (0.69)
Griffiths Mental Development Scales 559 6.60 (0.69)
1 year Bayley Scales of Infant Development 308 1.15 (0.08)
Griffiths Mental Development Scales
2 year Vineland Social Maturity Scale 1 795 2.08 (0.13)
Behaviour Screening Questionnaire 1 797 2.09 (0.13)
4 year Vineland Social Maturity Scale 1 600 4.06 (0.23)
Behaviour Screening Questionnaire 1 598 4.06 (0.23)
5 year Denver Prescreening Developmental
Questionnaire
1 233 5.22 (0.22)
7 year Conners’ Teacher Rating Scale 754 7.85 (0.41)
Draw-a-Person 1 292 7.82 (0.43)
Raven’s Coloured Progressive Matrices 1 312 7.84 (0.43)
7–8 years Age at school entry 967 6.69 (0.78)
13–14 years Mean years repeated by Grade 6 965 0.31 (0.55)
at Human Sciences Res Council on January 11, 2016sap.sagepub.comDownloaded from
6 South African Journal of Psychology
was completed by teachers at school and participating schools were provided with a small food
hamper in appreciation. Families were not offered incentives to bring their children in for data col-
lection rounds but were compensated for their transport costs and provided with a healthy snack
while they waited their turn to be assessed.
Assessment of young children requires patience and skill as babies fall asleep or need to be fed
and changed, and toddlers and preschoolers must be kept interested and engaged. All assessors
were trained to manage these practical issues, as well as to refer children who were identified as
having serious problems to the study counsellor who ensured onward referral to the appropriate
departments in both hospitals. Children were assessed through eight psychometric instruments that
capture one or other aspect of development administered at 6 months, 1 year, 2, 4, 5, and 7 years of
age. The procedures were administered at 6 months and 1 year by specially trained and supervised
postgraduate students in Psychology, and at 2, 4, 5, and 7 years by experienced and specially trained
and supervised research assistants permanently employed by the Bt20+ research programme. The
research staff also collected the demographic information.
Ethical considerations
Ethical approval for the study was obtained from the Committee for Research on Human Subjects
at the University of the Witwatersrand, Johannesburg. For each wave of data collection, children’s
primary caregivers were fully informed about the study procedures and their written informed
consent, and children’s assent, where age-appropriate, was obtained prior to each visit.
Data analysis
The area under ROC curve (AUC) and its 95% confidence interval (CI) were used to assess psy-
chometric assessments of children’s development for predicting delayed age of school entry and
number of years of grade repetition. Essentially, the ROC curve is a graph of the sensitivity against
1-specificity for a binary classification (DeLong, DeLong, & Clarke-Pearson, 1998). An AUC of
0.5 means the indicator is no better than chance, and the closer the AUC is to 1, the better the per-
formance of the indicator. For psychometric tests that performed better based on the AUC value,
covariate-adjusted non-parametric ROC regression models (Janes & Pepe, 2009) were fitted to
account for the possible effects of maternal age, education, socioeconomic status (SES; measured
by asset-based wealth quintiles), and the child’s birth weight with a 0.05 level of statistical signifi-
cance. All statistical analysis was performed using Stata version 12 (Stata Corp. LP, College
Station, TX, USA).
Results
Table 2 summarizes participants’ baseline characteristics. The majority of mothers were Africans,
20–38 years of age, single (never married, separated, divorced, or widowed), had secondary school-
ing, and fell into the middle-wealth quintile. The sample comprised an almost equal number of
boys and girls, most weighing between 2500 and 3999 g at birth.
Table 3 shows models of how the various psychometric assessments from infancy to 7 years
predict delayed age of school entry. AUC estimates for most of the psychometric measures are 0.5
and below reflecting weak predictive power for this outcome. At least two measurements have
AUC estimates with values well above 0.5, the BSID at Year 1 with ROC curve AUC = 0.61 (95%
CI: 0.51, 0.71) and the CTRS at Year 7 with ROC curve AUC = 0.64 (95% CI: 0.58, 0.70). Although
neither model achieved close to 100% sensitivity and 100% specificity, their power to predict
at Human Sciences Res Council on January 11, 2016sap.sagepub.comDownloaded from
Richter et al. 7
delayed age of school entry is above the reference line and therefore unlikely to be due to chance
(see Figures 1(a) and 2(b), respectively).
Table 4 shows models of the power of the various psychometric assessments administered from
infancy to 7 years for predicting repeat of at least one grade of school. Only the CTRS at Year 7 had
an AUC estimate above 0.5 with ROC curve AUC = 0.62 (95% CI: 0.57, 0.67). Although the ROC
curve is below 1, it is well above the reference line for predicting repetition of at least 1 year of
school, indicating that prediction is better than chance (see Figure 2(a)).
Table 5 shows covariate-adjusted models of both delayed age of school entry and repetition of at
least 1 year of schooling. The only statistically significant covariate-adjusted model showed that
young maternal age (β = −5.25; 95% CI: −9.62, −0.88), p = 0.019, and low wealth quintiles (β = −2.04;
95%: −3.76, −0.33), p = 0.020, had a negative influence on the age at school entry as predicted by
BSID at Year 1. However, there was little difference between unadjusted and covariate-adjusted
Table 2. Baseline characteristics of mothers and children on enrolment in the study (n = 3273 unless
otherwise indicated).
Characteristics N%
Population group
African 2568 78%
Other (White, Indian, Mixed Race) 705 22%
Maternal age (in years)
⩽17 92 3%
17–19 392 12%
20–38 2692 82%
⩾39 95 3%
Marital status
Married (any) 1201 37%
Single, separated, divorced, widowed 2072 63%
Education status (n = 2932)
No schooling 247 9%
Primary schooling 208 7%
Secondary schooling 2149 73%
Tertiary education 328 11%
Wealth quintiles (n = 2860)
1 541 19%
2 456 16%
3 870 30%
4 533 19%
5 460 16%
Child characteristics
Child’s sex
Male 1591 49%
Female 1692 51%
Birth weights in grams
<1500 30 1%
1500–2499 322 10%
2500–3999 2827 86%
⩾4000 89 3%
at Human Sciences Res Council on January 11, 2016sap.sagepub.comDownloaded from
8 South African Journal of Psychology
curves (Figure 1(a) and (b), respectively). No other covariates, including those in the model for pre-
dicting repetition of at least 1 school year, had any significant effect in the performance of selected
psychometric assessments. The estimated curves and AUCs were similar (Figures 2(a) and (b) and
3(a) and (b)).
Discussion
The predictive power of early childhood screening assessments is important for the early identifi-
cation and prevention of developmental delays and difficulties in LAMI countries (WHO, 2012).
However, caution has been expressed about the use of infant tests to predict developmental prob-
lems among young children without specifically examining their specificity, sensitivity, and their
association with later functioning (Meisels, 1989). The present analysis took advantage of unique
longitudinal data available from the Bt20+ birth cohort, and evaluated the power of the psychomet-
ric tools used in the study to assess development from infancy through primary schooling to predict
of late age of school entry and repetition of at least 1 year of schooling during the first six grades.
ROC curve analysis, which is commonly used in the fields of medicine and psychology to analyze
predictive quality of different tests (Balsamo et al., 2013), was used to examine the predictive value
of the different psychometric measurements.
The findings suggest that, in infancy, the BSID mental scale at Year 1 was the best predictor of
delayed age of school entry, with a moderate degree of diagnostic accuracy. The BSID is one of the
Table 3. Models of the performance of early psychometric assessments administered at different age
points for predicting delayed age of school entry.
Variables NAUC 95% CI
BSID
Mental (6 months) 347 0.51 0.42–0.61
Mental (1 year) 244 0.61* 0.51–0.71
Motor (6 months) 347 0.42 0.32–0.52
Motor (1 year) 244 0.54 0.43–0.64
GMDS
6 months 347 0.48 0.38–0.57
1 year 244 0.50 0.40–0.60
VSMS
2 years 1 343 0.44 0.39–0.48
4 years 1 329 0.48 0.43–0.53
BSQ
2 years 1 341 0.52 0.48–0.56
4 years 1 329 0.51 0.49–0.56
R-DPDQ (5 years) 1 038 0.31 0.26–0.36
CTRS (7 years) 623 0.64* 0.58–0.70
DAP (7 years) 1 091 0.38 0.34–0.43
RCPM (7 years) 1 113 0.32 0.27–0.36
AUC = the area under the curve and a value of 0.5 means the indicator is no better than chance, and the closer the
AUC is to 1, the better the performance of the indicator; BSID = Bayley Scales of Infant Development; GMDS =
Griffiths Mental Development scales; VSMS = Vineland Social Maturity Scale; BSQ = Behaviour Screening Questionnaire;
R-DPDQ = Revised Denver Prescreening Developmental Questionnaire; CTRS = Conners’ Teacher Rating Scale; DAP
= Draw a Person; RCPM = Raven’s Coloured Progressive Matrices.
*p <.05
at Human Sciences Res Council on January 11, 2016sap.sagepub.comDownloaded from
Richter et al. 9
most widely used measures for the assessment of infant development in relation to later outcomes
as it is often considered the ‘gold standard’, and it has been argued that mental or cognitive growth
rather than motor development may have a moderate degree of predictability (Slater, 1997).
However, there is inconsistent evidence on the predictive value of the BSID-I/BSID-II/Bayley-III
for long-term development of children, with some studies concluding that the BSID-I and BSID-II
have adequate predictive abilities for future functioning (Doyle & Casalaz, 2001; Sajaniemi,
Hakamies-Blomqvist, Katainen, & Von Wendt, 2001; Patrianakos-Hoobler et al., 2009) and other
studies concluding the opposite (Hack et al., 2005; Janssen et al., 2009).
The analysis also revealed that the CTRS at Year 7 was able to predict risk of delayed school
entry and repetition of at least 1 year of schooling with a moderate degree of accuracy. CTRS is
commonly used in classroom and research settings for detecting child behaviour problems related
to ADHD (Purpura & Lonigan, 2009), but some researchers contend that its ability to predict
ADHD is limited (Charach, Chen, Hogg-Johnson, & Schachar, 2009). However, in one very large
study of predictors of later achievement in school, the CTRS was successfully used to identify inat-
tention at school entry as the only learning-related predictor of later poor achievement in school
(Duncan et al., 2007). Increased behavioural problems have been associated in other studies with
delayed school entry and delayed school progress (Byrd, Weitzman, & Auinger, 1997).
Underlying factors that contribute to cognitive development and later poor school outcomes
such as poor performance and grade retention include maternal characteristics such as age, educa-
tion level, occupation, and SES (Shaw, Lawlor, & Najman, 2006; Turley, 2003; Williams et al.,
2012). In this study, the only statistically significant covariate-adjusted model showed that young
maternal age and low SES negatively influence the age at school entry as predicted by BSID at
Year 1. Maternal age is often viewed as a proxy for maternal education, maturity, and therefore
intelligence which might have an indirect effect on the child’s development and performance at
school (Shaw et al., 2006; Turley, 2003; Williams et al., 2012). Considerable evidence suggests
that maternal intelligence is mediated by predisposing variables which are, in turn, conditioned by
Figure 1. Receiver operating characteristic curve for the probability of (a) the unadjusted Bayley Scales of
Infant Development (BSID) administered at Year 1, and (b) covariate-adjusted BSID to predict delayed age
of school entry (the diagonal line represents no prediction value).
at Human Sciences Res Council on January 11, 2016sap.sagepub.comDownloaded from
10 South African Journal of Psychology
Table 4. Models of the performance of early psychometric assessments administered at different ages for
predicting repetition of at least 1 year of schooling.
Variables NAUC 95% CI
BSID
Mental (6 months) 347 0.45 0.40–0.54
Mental (1 year) 242 0.51 0.43–0.59
Motor (6 months) 347 0.48 0.41–0.55
Motor (1 year) 242 0.51 0.43–0.59
GMDS
6 months 347 0.48 0.41–0.55
1 year 242 0.51 0.43–0.58
VSMS
2 years 1 340 0.44 0.41–0.48
4 years 1 325 0.51 0.47–0.54
BSQ
2 years 336 0.51 0.48–0.55
4 years 1 325 0.47 0.44–0.51
R-DPDQ (5 years) 1 036 0.34 0.31–0.39
CTRS (7 years) 621 0.62* 0.57–0.67
DAP (7 years) 1 089 0.42 0.38–0.46
RCPM (7 years) 1 111 0.44 0.40–0.47
AUC = the area under area under the curve and a value of 0.5 means the indicator is no better than chance, and the
closer the AUC is to 1, the better the performance of the indicator; BSID = Bayley Scales of Infant Development;
GMDS = Griffiths Mental Development scales; VSMS = Vineland Social Maturity Scale; BSQ = Behaviour Screening
Questionnaire; R-DPDQ = Revised Denver Prescreening Developmental Questionnaire; CTRS = Conners’ Teacher
Rating Scale; DAP = Draw a Person; RCPM = Raven’s Coloured Progressive Matrices.
*p <.05
Figure 2. Receiver operating characteristic curve for the probability of (a) the unadjusted Conners’
Teacher Rating Scale (CTRS) administered at Year 7, and (b) covariate-adjusted CTRS to predict delayed
age of school entry (the diagonal line represents no prediction value).
at Human Sciences Res Council on January 11, 2016sap.sagepub.comDownloaded from
Richter et al. 11
income or SES, and that the impact of income or SES is moderated by family influences such as
the home environment (Shaw et al., 2006; Turley, 2003; Williams et al., 2012).
This article did not explore in detail why the BSID and the CTRS demonstrated more predictive
power than the other instruments among this sample of children and with these outcomes. There
are both similarities and differences between the tests. The GMDS, like the BSID, is also a test of
early development and they are well correlated; the BSQ and the VSMS tap aspects of attentional
and behavioural control which are also covered in the CTRS, and the RCPM and the DAP were
both administered at 7 years of age, close in time to the outcome measures selected. The BSID, the
GMDS, the DAP, and the RCPM involve direct assessment of the child by a trained tester, while
the VSMS, BSQ, and aspects of the R-DPDQ are based on parental report. Given that it is the
mental scale of the BSID specifically, as opposed to the psychomotor scale, that was predictive of
late school entry, it is possible that items from the more focused domain of early cognitive develop-
ment is more likely to be predictive of school outcomes. Other measures may have included con-
structs that were either not as pertinent, such as the VSMS and BSQ which focus more on social
competence and behavioural problems, or not as focused, such as the GMDS and R-DPDQ which
include other domains of development in addition to cognitive development (e.g., behavioural,
hearing and speech, eye-hand co-ordination, etc.). It is possible that the lack of predictability of the
DAP may be due to the lack of evidence on its application and understanding in the South African
context, where school readiness and repetition may be reliant on factors outside of the scope of this
Table 5. Covariate-adjusted models of early psychometric assessments (selected at AUC > 5) for
predicting school performance (delayed age of school entry and repetition of at least 1 school year).
Delayed age of school entry
BSID (Mental at 1 year) β95% CI p-value
AUC 0.59 – – –
Maternal age −5.25 −9.62 −0.88 0.019
Maternal education 1.91 −0.50 4.33 0.120
Wealth quintiles −2.04 −3.76 −0.33 0.020
Child’s birth weight 0.29 −5.53 6.10 0.922
Intercept 113.63 91.93 135.33 0.000
CTRS (7 years) 0.62
Maternal age 0.73 −3.14 4.60 0.711
Maternal education 1.31 −0.80 3.43 0.223
Wealth quintiles −0.50 −1.93 0.94 0.499
Child’s birth weight −3.78 −9.75 2.19 0.214
Intercept 88.63 66.23 111.03 0.000
Repetition of at least 1 school year
CTRS (7 years) β95% CI p-value
AUC 0.61 – – –
Maternal age −0.02 −3.86 3.82 0.993
Maternal education 1.12 −1.24 3.48 0.352
Wealth quintiles −0.11 −1.69 1.46 0.887
Child’s birth weight −3.19 −9.01 2.63 0.282
Intercept 87.09 65.24 108.95 0.000
at Human Sciences Res Council on January 11, 2016sap.sagepub.comDownloaded from
12 South African Journal of Psychology
Figure 3. Receiver operating characteristic curves for the probability of (a) the unadjusted Conners’
Teacher Rating Scale (CTRS) administered at Year 7, and (b) covariate-adjusted CTRS to predict repetition
of at least 1 year of school (The diagonal line represents no prediction value).
instrument (e.g., specific classroom behaviours). Although the RCPM is considered to be a good
measure of non-verbal intelligence in high income countries and is often used among older chil-
dren and adults, it is possible that the format in which the test is conducted, with children having
to identify a missing shape from a series of patterns, is still novel at the age in which it was assessed
in the current study. The CTRS may have been more accurate in assessing both children’s readiness
and performance in the classrooms because it was not only administered close in time to the out-
comes, but was also completed by teachers who rated the child’s behaviour and performance in the
classroom environment, constructs which more directly tap into the outcome variables. More
detailed psychometric work is underway to identify the items in the BSID and the CTRS which
most contribute to the tests’ predictive power in order to more accurately pinpoint valuable screen-
ers. As they stand, the findings point to a need to strengthen early social service and educational
interventions designed to prevent poor cognitive and behavioural outcomes and increase a child’s
readiness for school and chances of success. This could be done indirectly by targeting young
maternal age to enhance quality of parenting through clinic- and community-based interventions,
coupled with delaying first births and increasing young women’s education levels. Directly, it
could be done by using opportunities in available services to identify children at potential risk, for
example, through the milestones in the Road to Health card used in all public child health facilities,
and providing them and their families with opportunities to address potential learning and behav-
ioural problems. However, more research is needed to identify components of the instruments that
can be used to build better predictive models that enable prediction of later cognitive and behav-
ioural development with sufficient sensitivity and specificity, in order to improve future guidance
and interventions.
The following are some of the limitations to the study: The concept of covariate adjustment has
received little attention in research on child development measures where classification accuracy,
rather than association, is of primary interest (Janes & Pepe, 2009). In the present analysis, there
were little differences between unadjusted and covariate-adjusted ROC curves. The limitations of
at Human Sciences Res Council on January 11, 2016sap.sagepub.comDownloaded from
Richter et al. 13
this approach include the fact that confounding occurs in evaluating predictive accuracy when a
covariate is associated with both binary outcome (age at school entry and grade repetition) and the
psychometric measure. The confounder leads the location of ROC curve to deviate from its true
location in ROC space, which may result in over- or under-estimation of predictive accuracy (Janes
& Pepe, 2009). Nevertheless, rigorous evaluation of the predictive validity of psychometric meas-
urements is essential, and adjustment for known risk factors of childhood development is an impor-
tant component of this evaluation. More studies are needed to explore the association between each
psychometric measurement and ensemble covariates.
Conclusion
The ROC curve analyses used in this study gives a statistical measure of the predictive power of
the different early psychometric tests for later child development measures at a significant level
as compared to nothing at all. Findings suggest that with a moderate degree of diagnostic accu-
racy, the BSID mental scales at Year 1 and CTRS at Year 7 can be used as screening measures
for identifying young children in need of early interventions to prevent late school entry and
repetition of at least 1 year of schooling. This study is the first of its kind in South Africa, and
contributes to the conceptual, empirical, and technical literature on children’s developmental
assessment. The findings are in contrast to the assertion that infant tests are generally poor pre-
dictors of later functioning (Aylward, 2004), and speaks to the longstanding debate on the con-
tinuity of cognitive and other functions from infancy into childhood and adolescence (Lewis,
1973). The study adds valuable information to ongoing research towards implementing screen-
ing assessments for predicting future school performance based on early psychometric measure-
ments, and fits into the problem-solving model and prevention-oriented assessment framework
which focuses on early intervention.
Acknowledgements
The authors would like to thank the participants and their families, and the research assistants involved in the
study.
Funding
This work was supported by the Wellcome Trust (grant number 092097MA) and the Anglo American
Chairman’s Fund (grant number AAC 020419).
References
American Academy of Pediatrics. (2001). Developmental surveillance and screening of infants and young
children. Pediatrics, 108, 192–195.
Aylward, G. (2004). Prediction of function from infancy to early childhood: Implications for pediatric psy-
chology. Journal of Pediatric Psychology, 29, 555–564.
Balsamo, M., Imperatori, C., Sergi, M. R., Murri, M. B., Continisio, M., Tamburello, A., . . . Saggino, A.
(2013). Cognitive vulnerabilities and depression in young adults: An ROC curves analysis. Depression
Research and Treatment, 2013, Article 407602. Retrieved from http://dx.doi.org/10.1155/2013/407602
Bayley, N. (1969). Manual for the Bayley scales of infant development. New York, NY: The Psychological
Corporation.
Berls, A. T., & McEwen, I. R. (1999). Battelle Developmental Inventory. Journal of the American Physical
Therapy Association, 79, 776–783.
Bradley-Johnson, S. (2001). Cognitive assessment for the youngest children: A critical review of tests.
Journal of Psychoeducational Assessment, 19, 19–44.
at Human Sciences Res Council on January 11, 2016sap.sagepub.comDownloaded from
14 South African Journal of Psychology
Byrd, R. S., Weitzman, M., & Auinger, P. (1997). Increased behavioral problems associated with delayed
school entry and delayed school progress. Pediatrics, 100, 654–661.
Charach, A., Chen, S., Hogg-Johnson, S., & Schachar, R. J. (2009). Using the Conners’ Teacher Rating
Scale-Revised in school children referred for assessment. Canadian Journal of Psychiatry, 54, 232–241.
Conners, C. K. (1970). Conner’s Rating Scales–Revised: Technical manual. New York, NY: Multi-Health
Systems.
Conners, C. K., Sitarenios, G., Parker, J. D. A., & Epstein, J. N. (1998). Revision and restandardization of the
Conners Teacher Rating Scale (CTRS-R): Factor structure, reliability, and criterion validity. Journal of
Abnormal Child Psychology, 26, 279–291.
Cooper, P. A., & Sandler, D. L. (1997). Outcome of very low birth weight infants at 12 to 18 months of age
in Soweto, South Africa. Pediatrics, 99, 537–544.
De Lourdes, D. M., de Castro Aerts, D. G., de Souza, R. M., de Carvalho Leite, J. C., Giugliani, E. J., &
Marshall, T. (2005). Social inequalities in maternal opinion of child development in southern Brazil.
Acta Paediatrica, 94, 1006–1008.
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988). Comparing the areas under two or more
correlated receiver operating characteristic curves: A nonparametric approach. Biometrics, 44, 837–845.
Diamond, K. (1993). The role of parents’ observations and concerns in screening for developmental delays in
young children. Topics in Early Childhood Special Education, 13, 68–81.
Doll, E. A. (1965). Vineland Social Maturity Scale: Condensed manual of directions. Circle Pines, MN:
Educational Test Bureau.
Doyle, L. W., & Casalaz, D. (2001). Outcome at 14 years of extremely low birthweight infants: A regional
study. Archives for Disease Childhood Fetal & Neonatal Edition, 85, 159–164.
Duncan, G. J., Dowsett, C. J., Claessens, A., Magnuson, K., Huston, A. C., Klebanov, P., . . . Japel, C. (2007).
School readiness and later achievement. Developmental Psychology, 43, 1428–1446.
Engle, P., Black, M. M., Behrman, J. R., de Mello, M. C., Gertler, P. J., Kapiriri, L., . . . the International Child
Development Steering Group. (2007). Strategies to avoid the loss of developmental potential in more
than 200 million children in the developing world. The Lancet, 369, 229–242.
Ertem, I., Atay, G., Dogan, D. G., Bayhan, A., Bingoler, B. E., Gok, C. G., . . . Isikli, S. (2007). Mothers’
knowledge of young child development in a developing country. Child Care Health and Development,
33, 728–737.
Frankenburg, W. K., Fandal, A. W., & Thornton, S. M. (1987). Revision of Denver Prescreening Developmental
Questionnaire. Journal of Pediatrics, 110, 653–657.
Frankenburg, W. K., Van Doorninck, W. J., Liddell, T. N., & Dick, N. P. (1976). The Denver Prescreening
Developmental Questionnaire. Pediatrics, 57, 744–753.
Gottlieb, C. A., Maenner, M. J., Cappa, C., & Durkin, M. S. (2009). Child disability screening, nutrition, and
early learning in 18 countries with low and middle incomes: Data from the third round of UNICEF’s
Multiple Indicator Cluster Survey (2005–06). The Lancet, 374, 1831–1839.
Grantham-Mcgregor, S., Cheung, Y., Cueto, S., Glewwe, P., Richter, L., & Strupp, L. (2007). Developmental
potential in the first 5 years for children in developing countries. The Lancet, 369, 60–70.
Griffiths, R. (1970). The abilities of young children: A comprehensive system of mental measurement for the
first eight years of life. London, England: Young & Son Ltd.
Hack, M., Taylor, H. G., Drotar, D., Schluchter, M., Cartar, L., Wilson-Costello, D., . . . Morrow, M. (2005).
Poor predictive validity of the Bayley Scales of Infant Development for cognitive function of extremely
low birth weight children at school age. Pediatrics, 116, 333–341.
Hsiao, C., & Richter, L. M. (2014). Early mental development as a predictor of preschool cognitive and
behavioral development in South Africa: The moderating role of maternal education in the birth to
twenty cohort. Infants and Young Children, 27, 74–87.
Jacklin, L., & Cockcroft, K. (2012). The Griffiths Mental Development Scales: An overview and considra-
tion of their relevance for South Africa. In S. Laher & K. Cockcroft (Eds.), Psychological assessment in
South Africa: Research and applications (pp. 169–185). Johannesburg: Wits University Press.
Janes, H., & Pepe, M. S. (2009). Adjusting for covariate effects on classification accuracy using the covariate-
adjusted receiver operating characteristic curve. Biometrika, 96, 371–382.
at Human Sciences Res Council on January 11, 2016sap.sagepub.comDownloaded from
Richter et al. 15
Janssen, A. J., Nijhuis-van der Sanden, M. W., Akkermans, R. P., Tissingh, J., Oostendorp, R. A., & Kollee,
L. A. (2009). A model to predict motor performance in preterm infants at 5 years. Early Human
Development, 85, 599–604.
Knoetze, J., Bass, N., & Steele, G. (2005). The Raven’s Coloured Progressive Matrices: Pilot norms for isiX-
hosa-speaking primary school learners in peri urban Eastern Cape. South African Journal of Psychology,
35, 175–194.
Lewis, M. (1973). Infant intelligence tests: Their use and misuse. Human Development, 16, 108–118.
Luiz, D. M., Foxcroft, C. D., & Tukulu, A. N. (2004). The Denver II and the Griffiths Scales of Mental
Development: A correlational study. Journal of Child and Adolescent Mental Health, 16, 77–81.
McCarthy, D. (1972). Manual for the McCarthy scales of children’s abilities. New York, NY: Psychological
Corporation.
Maulik, P., & Darmstadt, G. (2007). Childhood disability in low- and middle- income countries: Overview
of screening, prevention, services, legislation, and epidemiology. Pediatrics, 120(Suppl. 1), S1–155.
Meisels, S. (1989). Can developmental screening tests identify children who are developmentally at risk?
Pediatrics, 83, 578–585.
Mothuloe, V. B., Richter, L. M., Barnes, C. J., & Schoeman, M. (1994). Griffiths Scales of Mental
Development: A South African validation study. South African Journal of Education, 14, 38–43.
Norris, S., Richter, L., & Fleetwood, S. (2007). Panel studies in developing countries: Case analysis of sample
attrition over the past 16 years within the Birth to Twenty cohort in Johannesburg, South Africa. Journal
of International Development, 19, 1143–1150.
Owen, K. (1992). The suitability of Raven’s standard progressive matrices for various groups in South Africa.
Personality and Individual Differences, 13, 149–159.
Patrianakos-Hoobler, A. I., Small, M. E., Huo, D., Mark, J. D., Plesha-Troyke, S., & Schreiber, M. (2009).
Predicting school readiness from neurodevelopmental assessments at age 2 years after respiratory dis-
tress syndrome in infants born preterm. Developmental Medicine & Child Neurology, 52, 379–385.
Pillay, A. L. (2003). Social competence in rural and urban children with mental retardation: Preliminary find-
ings. South African Journal of Psychology, 33, 176–181.
Pind, J., Gunnarsdottir, E. K., & Johannesson, H. S. (2003). Raven’s Standard Progressive Matrices: New
school age norms and a study of the test’s validity. Personality and Individual Differences, 34, 375–386.
Potterton, J., Stewart, A., Cooper, P., Goldberg, L., Gajdosik, C., & Baillieu, N. (2009). Neurodevelopmental
delay in children infected with human immunodeficiency virus in Soweto, South Africa. Vulnerable
Children and Youth Studies, 4, 48–57.
Purpura, D. J., & Lonigan, C. J. (2009). Conners’ Teacher Rating Scale for preschool children: A revised,
brief, age-specific measure. Journal of Clinical Child & Adolescent Psychology, 38, 263–272.
Raven, J. C., Court, J. H., & Raven, J. (1977). Manual for Raven’s progressive matrices and vocabulary
scales. London, England: H.K. Lewis.
Richman, N., & Graham, P. J. (1971). A behavioural screening questionnaire for use with three-year-old
children. Preliminary findings. Journal of Child Psychology and Psychiatry, 12, 5–33.
Richter, L. M., Griesel, R. D., & Rose, C. (1992). The Bayley scales of infant development: A South African
standardization. South African Journal of Occupational Therapy, 22, 14–25.
Richter, L. M., Griesel, R. D., & Rose, C. (1994). The McCarthy Scales of Children’s Abilities: Adaptation
and norms for use amongst black South African children. South African Journal of Occupational
Therapy, 24, 17–30.
Richter, L. M., Griesel, R. D., & Wortley, M. (1989). The draw-a-man test: A 50 year perspective on drawings
done by black South African children. South African Journal of Psychology, 19, 1–5.
Richter, L. M., Norris, S. A., & De Wet, T. (2002). Transition from birth to ten to birth to twenty: The South
African cohort reaches 13 years of age. Paediatrics and Perinatal Epidemiology, 18, 290–301.
Richter, L., Norris, S., Pettifor, J., Yach, D., & Cameron, N. (2007). Cohort profile: Mandela’s children: The
1990 birth to twenty study in South Africa. International Journal of Epidemiology, 36, 504–511.
Richter, L. M., Victora, C., Hallal, P., Adair, L., Bhargava, S., Fall, C., . . . COHORTS Group. (2011). Cohort
profile: The consortium of health research in transitioning societies (COHORTS). International Journal
of Epidemiology, 41, 621–626. doi:10.1093/ije/dyq251
at Human Sciences Res Council on January 11, 2016sap.sagepub.comDownloaded from
16 South African Journal of Psychology
Rohde, T. E., & Thompson, L. A. (2007). Predicting academic achievement with cognitive ability. Intelligence,
35, 83–92.
Rydz, D., Shevell, M., Majnemer, A., & Oskoiu, M. (2005). Developmental screening. Journal of Child
Neurology, 20, 4–21.
Sabanathan, S., Wills, B., & Gladstone, M. (2015). Child development assessment tools in low- and middle-
income countries: How can we use them more appropriately? Archives of Diseases in Childhood, 100,
482–488.
Sajaniemi, N., Hakamies-Blomqvist, L., Katainen, S., & Von Wendt, L. (2001). Early cognitive and behav-
ioral predictors of later performance: A follow-up study of ELBW children from ages 2 to 4. Early
Childhood Research Quarterly, 16, 343–361.
Shaw, M., Lawlor, D. A., & Najman, J. M. (2006). Teenage children of teenage mothers: Psychological,
behavioural and health outcomes from an Australian prospective longitudinal study. Social Science &
Medicine, 62, 2526–2539.
Slater, A. (1997). Can measures of infant habituation predict later intellectual ability? Archives of Diseases
in Childhood, 77, 474–476.
Turley, R. N. (2003). Are children of young mothers disadvantaged because of their mother’s age or family
background? Child Development, 74, 465–474.
Williams, B. L., Dunlop, A. L., Kramer, M., Dever, B. V., Hogue, C., & Jain, L. (2012). Perinatal origins of
first-grade academic failure: Role of prematurity and maternal Factors. Pediatrics, 131, 693–700.
World Health Organization. (2012). Developmental difficulties in early childhood: Prevention, early identifi-
cation, assessment and intervention in low- and middle income countries: A review. Geneva, Switzerland:
Author.
at Human Sciences Res Council on January 11, 2016sap.sagepub.comDownloaded from
