Psychometrics

Description of The Norm Sample

The most recent statistical validation of the SEI-YV, version V2.1, is based on a sample size of n = 5,715, which dates from 2007 to the beginning of 2011. For statistical validation, the norm base was cleaned from bogus and outlier responses for optimal credibility, as it forms the baseline against which individual and group responses are standardized and compared, resulting in an effective norm base of size n = 5,693.

Ideally, we want the gender and age distribution to be closely equal in all categories and across different countries, which may be achieved with growing an even larger norm base over time, and by managing or encouraging specific projects. Currently, male-female distribution in the norm base shows a ratio of 1:2. In addition, female youth tends to be slightly younger (peaking around 12- 13 years) than male youth (peaking around 13-15 years). The average age of the norm population is 13 years and four months. There are several hundred youth in each of the eight combined gender-age categories in the norm base, which allows for trustworthy statistical comparisons between them. The combined age and gender category distribution in the norm base is as follows:

Gender is fairly equally spread across the countries where the SEI-YV was used across multiple projects, with a notable exception where females dominate, namely Singapore. We need yet more data to draw a conclusion with confidence for the age categories across different countries. So far, attempts to do so are highly dependent on where different practitioners from the same country find local opportunities to use the SEI-YV, which is part of the challenges of growing any young assessment. Given the progress we made since 2007, we are highly encouraged by it.

Country and ethnic representation have expanded considerably in the last four years: 14 different countries (of which nine are sizable) used the SEI-YV over 204 project groups of varying sizes. A third of these projects contained group sizes of 30 or sometimes considerably more students. Within countries, students reported rich ethnicity combinations. While students completed the survey in English, in some countries another language may be the medium for instruction and spoken at home.

Females significantly outperformed males on all scales (EQ components and Life Barometers) despite age distribution, including the validity indicators. Importantly, the found differences lie in the strength of individual EQ component scores, but not necessarily between EQ component scores when they are viewed across as EQ profiles. In contrast to gender, EQ performance does not follow a set score strength pattern across the different age categories, with one exception. Older youth tend to respond less positively, so their scores are slightly lower than younger children. Given that males, who underperformed compared to females, also tend to be a bit older, the found difference in gender performance with regards to EQ component score strength is partially amplified. Keep in mind also that slightly lower EQ among older groups are part of EQ stabilization and hence to be expected. It may be that as children mature, they become more discriminating.

EQ performance between countries with sizable and composite representation within the norm population is not significantly different one from another overall. From some countries, youth from multiple ethnicities were assessed. These characteristics together support the claim that the SEI-YV is largely free from cultural influences, starting with the extreme care we take when formulating and translating survey items. The data in the norm base span a range of socio-economic sectors, types of schools, academic needs, and achievement levels.

Standardization

The demographic characteristics and EQ performance of the norm population lead to the recommendation that score standardization should be based on the norm population in general, rather than being specific to gender and/or age category, until a future date when more research is available. For group reports and individual report comparisons where groups consist of both male and female youth, the use of general population norms are required.

At this time, the data are not weighted for norming purposes as was done in Version 2.0. The main reason for this decision is that given the complexity of multiple demographic characteristics in the norm population, any weighting based on existing frequencies amplify rather than neutralize the skews that exist. For now, we are content with the composite nature of the current demographic distribution while we continue to gather data and grow the norm base in as natural a way as possible. Whereas previous norms may have been slightly lenient overall, the new norms may be slightly strict overall. We believe the norming pendulum will stabilize in the near future, while we are pleased with the progress made over the life span of this young assessment.

Psychometric characteristics of all survey items held remarkably steady over several iterations of statistical validation. No items required major changes, while both the number and order of items appear to function effectively.

The descriptive statistics for the EQ components, Life Barometers, and two validity indicators are shown below. These form the basis for standardizing individual and group scores. Note that there is a tendency towards negative skewedness statistics (i.e., positively skewed distributions). This is normal in self-judged measures where Likert-type response scales are used. In general, skewedness is low and close to zero. The kurtosis (i.e., peakedness) of the distributions varies close to zero across the scales as well, which is desirable.

In the scale of response inconsistency (IC), a higher Kurtosis is desirable. The raw IC value is positively skewed (meaning in general and for the norm population as a whole, the scores are on the low side) and highly peaked (in other words, consistently so). This means that if an individual youth shows a high IC score, this can be interpreted as exceptional and meaningful. More about this will be said in the next section.

Note that the above reported values are raw values, which should not be confused with standardized scores that we use in SEI-YV reports. The use of standardized scores enables us to make direct comparisons between EQ components and Life Barometers related to either individuals or groups of youth, as these are all set against the same baseline provided by the norm population. Think of this as two students, one who got 24 and another who got 30 for two tests. The second student performed better, right? Wrong. Unless we know that the first student’s test was 24 out of 30 and the second student’s test was 30 out of 50, which we can now turn into percentage scores of 80% and 60% respectively (a simplistic version of standardization), can we make a direct and accurate comparison between the two students.