Recent research has documented the importance of school readiness in young children. Children who start school lacking basic skills often continue to show lower achievement throughout schooling (Burchinal, Magnuson, Powell, & Hong, 2015; Józsa, 2016; Eisenberg, Spinrad, & Eggum, 2010; Snow, 2006). Most current assessments of school readiness focus on early measures of pre-academic skills, such as emerging literacy and numeracy. Although these skills are useful in predicting school success, research suggests that approaches to learning, such as executive functions (EF) and mastery motivation (MM), may be even more important (Berhenke, Miller, Brown, Seifer, & Dickstein, 2011). Approaches to learning, an over-arching term for attributes that help children learn, such as enthusiasm, focus, persistence, flexibility, and mastery motivation, form a key dimension of school readiness according to the National Education Goals Panel (Kagan, Moore, & Bredekamp, 1995). In this article, we provide information about a new, computer based assessment of school readiness and early school skills: game-like tasks to assess mastery motivation and executive functions in children aged 3-8. For more information about psychometrics, see Józsa, Barrett, Józsa, Kis, and Morgan (2017).

A rather unique contribution of the school readiness assessment we will discuss here is its incorporation of measures of mastery motivation (MM). In their classic and influential report, Shonkoff and Philips (2000) highlighted MM as a key factor in early development. Morgan, Harmon, and Maslin-Cole (1990) defined it as a multifaceted psychological force that stimulates an individual to attempt to master a skill or task that is at least moderately challenging for him or her. A key feature distinguishing this approach to motivation from others is its focus on persistence on tasks that are at least moderately challenging for a particular individual. Ability to persist in the face of challenge is crucial for school readiness and, even more, for school success.

In spite of the crucial importance of MM, until recently, there have been surprisingly few empirical studies on this approach to motivation. Those that have been done confirm its utility (Busch-Rossnagel & Morgan, 2013; Józsa & Molnár, 2013; Józsa & Morgan, 2014; Józsa, Wang, Barrett, & Morgan, 2014; Morgan, Józsa, & Liao, 2017). MM has an important impact on cognitive development, as well as other domains of development (Busch-Rossnagel & Morgan, 2013; Wang & Barrett, 2013).

Unfortunately, existing behavioral measurements of MM for young children are time-consuming and require training to administer. As a result, they are impractical for teachers in authentic school settings to administer. Although adult-report questionnaires have been developed that are less challenging to administer, they involve perceptions rather than behaviors, relying on adults’ memory and interpretation of relevant events. Perhaps as a result, they often seem to confound motivation and competence (e.g., Józsa & Molnár, 2013; Józsa & Morgan, 2014; Józsa et al. 2014, Morgan, Wang, Liao, & Xu, 2013).

Morgan, Busch-Rossnagel, Maslin-Cole, and Harmon (1992) developed a procedure intended to help separate motivation from the child’s ability, selecting a particular task that was moderately challenging for each individual child, based on objective measures of children’s degree of success on several, increasingly difficult tasks. They operationalized mastery motivation as children’s persistence and pleasure at those moderately difficult tasks. This individualized approach has proved very useful and has been used by a number of researchers measuring mastery motivation in both typically and atypically developing young children (e.g., Gilmore & Cuskelly, 2011; Young & Hauser-Cram, 2006; Wang, Morgan, Hwang, & Liao, 2013; Wang et al., 2016). This same approach was taken in developing the new computer based assessment described in this paper. In the current version, the same tasks are given to all children of a particular age, but the tasks used to measure motivation are individualized, based on that child’s performance (see Józsa et al., 2017). Eventually, the computer will be programmed to actually give children different tasks based on that child’s individual performance on the initial level of the task.

In the past two decades, executive functions have become a major focus of research in psychology, neuroscience, and education because these skills provide an important foundation for learning in education settings (Zelazo, Blair, & Willoughby, 2016). EF refer to cognitive processes that are required for the conscious, top-down control of action, thought, and emotions, and that are associated with neural systems involving the prefrontal cortex (Diamond, 2013; Müller & Kerns, 2015; Zelazo & Müller, 2010). There is general agreement that there are three core EF components (Blair & Diamond, 2008; Tsermentseli & Poland, 2016): inhibitory control, working memory, and cognitive flexibility. EF are essential for mental and physical health; success in school and in life; and also for cognitive, social, and psychological development (Diamond, 2013; Zelazo et al., 2016). EF are central to school readiness and early school achievement (Blair & Raver, 2015). Research has found that EF measured in childhood predict a wide range of important outcomes, including readiness for school (McClelland et al., 2007) and the successful transition to kindergarten (Blair & Razza, 2007); school performance and social competence (Mischel, Shoda, & Rodriguez, 1989). In fact, EF predicted outcomes better than IQ (Zelazo et al., 2016).

Traditionally the role of emotion and motivation in EF has largely been neglected (Peterson & Welsh, 2014). The movement away from a purely cognitive conceptualization of EF can be largely credited to the work of Zelazo, and Müller (2002) in which they proposed that EF varies according to the motivational significance of a situation. They outlined a distinction between cool and hot EF. This broader conceptualization of EF has important implications for research into child development because EF have been found to be a strong predictor of school readiness, academic achievement and social behavior (Brock, Rimm-Kaufman, Nathanson, & Grimm, 2009; Zelazo et al., 2016). However, existing measures of EF do not take into account the role of MM in EF performance.

A large number of studies have highlighted the importance of the preschool-to-school transition (e.g., Burchinal et al., 2015; Eisenberg et al., 2010; Snow, 2006), and schools are increasingly being required to demonstrate their success in helping children make this transition. Researchers have paid increasing attention to identifying the conditions of a successful start in school. Creating instruments for assessing school readiness and monitoring development at the beginning of schooling is important to such initiatives. Although the majority of studies on school readiness assessment have focused on the cognitive domain, recent research identified several other factors, including motivation, executive function, and emotion regulation, which play a crucial role in the preschool to kindergarten transition (e.g., Berhenke et al., 2011; Blasco, Saxton, & Gerrie, 2014; McWayne, Cheung, Wright, & Hahs-Vaughn, 2012).

It is clear that MM and EF are important for school success. In fact, there is evidence that MM and EF are even better predictors of later school performance than IQ (Diamond, 2016; Józsa & Molnár, 2013). Despite their importance, there are no standardized behavioral tests of the MM of children during this critical transition from pre-school to elementary school, and few computer- or tablet-based assessments of EF. Moreover, existing computer-based assessments of EF are either very long and, thus, impractical to add to other assessments, are highly influenced by less relevant skills, such as reaction time, or need to be administered individually by trained examiners.

We have developed an internet-based tablet assessment for 3 to 8 year-old children. Characteristics assessed include (a) mastery motivation (i.e., persistence in searching for letters, numbers, and pictures in an increasingly challenging array); (b) executive functions (working memory, measured by ability to remember locations of pictures; inhibitory control and mental set shifting, measured by increasingly challenging card sorting tasks), and (c) recognition of numbers and letters.

The goal of this paper is to give an overview of the new, computer-based tasks. To help the reader, the paper provides selected examples of the 103 screenshots and accompanying instructions that the computer narrator, Little Bear, gives children, so the reader can better understand the tasks from children’s perspective. The paper also includes tables showing the levels of each task, including the levels for which screen shots are not included here.

We developed seven computer-tablet, game-like tasks for this school readiness assessment. The first two tasks involve recognition of numbers and letters; they are brief assessments of pre-academic abilities. They provide some information about the child’s pre-reading and mathematics readiness skills. These two brief pre-academic competency tasks may also help us distinguish the child’s pre-academic knowledge from their motivation and executive functions.

Tasks 3-5 are designed to measure an important aspect of the child’s MM: persistence while trying to solve a challenging problem. These letter and number search tasks vary in difficulty so that children are given tasks that are easy, moderate, and hard for most children their age. Our search tasks assess the child’s persistent focus on the task in order to find all matches. By relating persistence on the MM tasks to the child’s competence on the EF tasks, we can see the extent to which both types of tasks share the ability to self-regulate and inhibit potential distractions.

Tasks 6 and 7 are designed to assess aspects of EF. Our Picture Memory task, which assesses working memory, requires the child to remember the location of specific pictures in an array of face down picture “cards”, in order to match pairs of pictures. Persistence on this task also provides another measure of MM. Our Size-Shape-Color Game, which is a modified version of the Dimensional Change Card Sort task (Zelazo, 2006) requires the child to not only remember (or, at later levels, figure out) the sorting rules but to respond to multiple rule changes on multiple sorting dimensions, and to inhibit responses consistent with previous rules. Our version has been modified to increase difficulty level at the higher levels, so that difficulty will not be defined by reaction time, as it is on other versions of the DCCS that are designed to be used across a wide age span.

Each of the seven tasks varies in difficulty from easy for 3-year-olds to difficult for 8-year-olds. We break the presentation of the seven tasks into two sessions of approximately 15-20 minutes each. Sessions may be held the same day at different times or on different days, depending on what is more convenient for the children and site involved. The first session includes the pre-academic competencies (number and letter recognition tasks, which are counter-balanced in presentation order) and also the mastery motivation (letter and number search tasks, which are again counterbalanced). Session 2 includes the picture memory and card sort tasks (again counterbalanced), both of which assess executive functions.

Tasks 3-7 could all be considered measures of “Approaches to Learning (ATL)” - non-academic attributes such as engagement, focus, and motivation that are important foundations for success in the classroom setting. One of the strengths of the present assessment is its ability to simultaneously collect data on MM, EF, and competence on the same tasks as well as on others, enabling partialling of each from the other. Tasks 3-5 assess not only MM but some aspects of EF, especially inhibitory control, in that children with lower inhibitory control would be expected to make more mistakes of commission (touching incorrect items). And, tasks 6 and 7 could be viewed as assessing MM because persistent and focused attention is key to doing these tasks successfully.

A summary of the seven tasks and appropriate time needed for each is presented in Table 1. Note that we counterbalance the order of administration of tasks in each session as indicated below.

The assessment does not require children to read, but the computer narrator, Little Bear, speaks in either English or Hungarian based the examiner’s selection. The tasks were developed to be appropriate for both Hungarian and American cultures, and involve pictures of everyday objects and school-related symbols, including letters, numbers, animals, vehicles (boats, cars, and airplanes), and fruits. Children of both languages were readily able to do the easy level of all of the tasks. Currently, we are working on the Hebrew version.

Preliminary data have been collected in Hungary and the U.S. (Barrett & Józsa, 2016; Józsa, Barrett, & Morgan, 2016; Józsa, Barrett, Stevenson, & Morgan, 2016). Significant correlations were found among the measures of persistence: letter search, number search, and picture memory. To assess concurrent validity, teachers rated children’s persistence and mastery pleasure on the Dimension of Mastery Questionnaire (DMQ, Morgan, Busch-Rossnagel, Barrett, & Wang, 2009). Teacher-rated persistence using the DMQ was significantly correlated with persistence on the letter and number search tasks. Teacher-rated mastery pleasure on the DMQ was also significantly correlated with experimenter-rated mastery pleasure. The tasks have good reliabilities and concurrent validity (Józsa, Barrett, & Morgan, 2017; see Józsa et al., 2017 for more details).

The session begins when the test administrator (or teacher) introduces her/himself to the children and explains that they are going to play some games on a computer/tablet. The test administrator fills in the login screen with the experimenter’s user name and password, Child’s ID number, and birth year and month. Note, what the computer says is in quotations and italicized.

Figure 1 appears, and touching the bear starts the narration. Little Bear moves its mouth as a pre-recorded voice says, “Hello! My name is Little Bear. I am going to play with you today.”

Before each task there are training slides; in this case with pictures of five animals (fish, bird, bunny, cat, and mouse) to help the child understand the type of task and provide help if the child does not initially know what to do.

Task 1 or 2. Number Recognition (tasks 1 and 2 are counterbalanced). The task is to see how many numbers the child can correctly identify. After training, “Little Bear” says: “Now we will play a number game. First, I will say a number. Then, you will touch that number on the screen. For example, if I say ‘2’, you will find and touch ‘2’ on the screen. Only touch one number. When you touch it, a new screen will appear and I will tell you a new number.”

Little Bear then says a number and the child’s task is to select it on the screen from an array of numbers and touch it. After the child touches a number, the array disappears, that trial ends, and a new array appears. To assess the child’s number recognition, the numbers get progressively more difficult with each trial. The results of our pilot testing indicate that up to 15 trials and 90 seconds is enough to obtain a good measure of 3–8 year-old children’s level of number recognition. When the child has missed two Number Recognition trials in a row, the task is stopped and the computer goes to the next task. Table 2 shows the 15 levels of the Number Recognition task.

This task assesses how many letters the child can correctly identify. Before Trial 1, ”Little Bear” says: “Now we are going to play a game with letters. For this game, I will tell you the name of a letter. On the screen, touch the letter that you hear. For example, if I say ‘A’, find and touch ‘A’. Only touch one letter. When you touch it, a new screen will appear and I will tell you a new letter to find.”

Little Bear then says a letter and the child finds it in an array of letters and touches it. As with number recognition, after the child touches one letter, all the letters in the array disappear. Then the computer says a new letter. As with numbers, the letter recognition tasks get progressively more difficult as trials progress. Pilot work indicates that at most 15 trials and 90 seconds is enough to obtain a good measure of the child’s knowledge of letters. Table 3 presents these levels. The task ends when the child misses two consecutive letters.

The letter and number search tasks are primarily used to obtain measures of focused persistence on moderately challenging tasks (MM), and they also yield measures of accuracy on the tasks. As Table 4 shows, each child is given one easy, two moderately difficult, and one hard level of each task based on their age, for up to two minutes each. Based on the findings of our initial studies using the assessment, we will modify the computer program so it bases the level each child receives on that child’s performance on the first tasks. Note that the figures and narratives presented here show only some levels of each task. The letter search task is divided into two parts; the more difficult levels (6–8) have a different rule and directions.

The screen shows a target object in the upper left. The middle of the screen displays a 2x4 matrix of 8 pictures, two each of identical pictures of four familiar objects: boat, house, banana, and car. Little Bear says: “Now we are going to play a different game. Over here is a boat (it flashes). Over here there are eight pictures (they flash). Touch all the pictures of the boat.”

If children touch both of the boats, Little Bear says, “That’s right”. If children make a mistake, Little Bear corrects them, saying, “That is a _____, not a boat”. This serves as the training for both search tasks. It occurs before the first search task, whether it is number search or letter search. If the child touches both boats and no other objects, level 1 of the number or letter search starts; if not, another example trial is given.

Tasks 3 and 4 are counterbalanced. Little Bear says: “This is the Number Search game. In this game you will find the numbers. Over here, you see a number (number flashes) that is in a red box. The other numbers are in blue boxes. You will need to touch all of the blue numbers that are exactly the same as the red number. During these games we will not tell you if you have found them all.”

Little Bear appears on the screen and says: “When you think you are done with this level and want to move on to the next, just click on me! I’ll be right here!” (Figures 2 and 3)

Little Bear says, “Now we are going to play a game where you find letters. Over here, you will see a letter (letter flashes) that is in a red box. The other letters are in blue boxes. You will need to touch all of the blue letters that are the same as the red letter.”

“I’m still right here, so when you want to go to the next level, just touch me.”

Then the computer presents the easy level for that child’s age group (see Table 4). The computer then presents any moderate levels for that child’s age group that are no higher than level 5. It does not present levels 6–8 at this time, because additional training is needed for these highest levels. (Figure 4)

Levels 6–8 require that the child find the same letters, even when they appear in a different order. Because the letters do not form words, the order is unimportant. (Because ordering numbers differently always changes the numerical value represented, the assessment does not have this same type of task for number search.) After additional training (with pictures of flowers and boats) to teach children not to consider order in finding matches, these more difficult levels of the letter search are presented by the computer. The child is given these instructions by Little Bear: “Now you get to play the new letter game, which has the same rule as the flower and boat game you just tried. In this game you will find several letters in a red box over here (box flashes). The other letters are in blue boxes. You will need to touch all of the groups of blue letters that are the same letters as the red letters. The blue letters can be in any order as long as they are the same as the red letters. Find JK and also KJ.” (Figure 5)

Each child will receive one task that is typically easy at the child’s age, one moderate task, and one hard task as shown on Table 8.

Tasks 6 and 7 are counterbalanced. In this task the child sees a rectangular array of blank cards, which have pictures on the other side. When the child touches the blank card, the computer turns it over so that the picture is visible. Little Bear explains it as follows: “This is the picture memory game. In this game, you will find pictures that are the same. Touch a card to see what picture it is and then touch another card to try to find the same picture. For example, if you touch a card that is a fish, touch another card to see if it is the other fish. If the other card is also a fish you have found what you are looking for. If you find a picture that isn’t the same, then keep playing.”

If the child doesn’t find the match they are expected to keep trying by touching one card at a time until they find the match. For levels 1-5, when the child touches a matching card, both cards disappear, but when a non-matching card is touched, it flips back. However, in the more difficult levels 6-8, the computer turns over the cards and leaves them in the same place on the screen. “Let’s start. Find all the cards that are the same as each other” (Figure 6).

Children aged 5 years and older will receive at least one task from levels 6–8. The computer will give them instructions about the “new,” harder game where the cards don’t disappear when they are matched (Figure 7).

Table 9 shows all eight levels of the picture memory task, including details about: (a) the number of pairs of pictures, (b) the total number of pictures on the screen, and (c) whether both cards disappear when they are matched or the cards turn back over when matched rather than disappearing.

This is the Modified Dimensional Change Card Sort Task (the Size-Shape-Color game). Figure 8 shows the general design on the screen for these tasks. Note that there is a red sailboat on the bottom of the screen which the child can drag into one of two baskets depending on the sorting dimension specified (the game being played). Instructions vary with the specific task (Figure 8).

Sometimes, the child “plays the shape game”, where the child is told to drag the test card into the basket with the same shape, ignoring color. For example, in the shape game, all of the rabbits go in the basket with the rabbit on it, and all of the boats go in the basket with the boat on it even though the colors don’t match. In the “color game,” all the red boats go in the basket with the red bunny, and all of the blue bunnies go in the basket with blue boat. In the size game, all the big things go in the basket with the big picture on it and all the little things go in the basket with the little picture on it. The child is told whether it is correct on training trials but not on the test trials. Note that the cards to be sorted never exactly match the pictures on the baskets. After training, Little Bear starts the task by saying, “We’re going to play a game with colors and shapes. You will sort ‘pictures’ into two baskets. During each game, we will tell you the rule you will use to sort pictures.”

“Now we are going to play the color game. In the color game, you put all of the red ones in this basket (it flashes) and all of the blue ones in this basket (it flashes). Each time you see a new card, put it in the red basket if it is red and the blue basket if it is blue.”

“Now we are going to play the shape game. Put the flower cards in the flower basket and the airplane in the airplanes basket.”

Level 4 has nine cards to be sorted with two shades of green and two shades of blue. The left basket has a small daisy with one shade of blue on it and the right hand basket has a large airplane with a shade of green. Level 4 is intended to be hard for 3-year-olds, moderately challenging for 5-year-olds, and easy for 7-year-olds. Note that only one picture at a time actually shows at the bottom of the screen.

“This time we will play the color game. All of the blue cards go in the blue basket, and all of the green cards go in the green basket.“ (Figure 9)

Level 4 Post Switch: Using the same two blue and green baskets and nine test cards. “Now, we are playing the opposite color game. In the opposite color game, you put the cards in the basket with the OTHER color. So, the blue cards go in the green basket and the green cards go in the blue basket.”

Level 4 Second Post Switch: Using the same two baskets and nine test cards. “Now, we are going to sometimes play the color game and sometimes the opposite color game. When I say color game, keep playing that game until I say we will play the opposite color game. Keep playing that game until I say we will now play the color game.”

For levels 7 and 8, there are four baskets on the screen and children are instructed to sort the test cards into first one and then the other appropriate basket, based on one of three dimensions: size, color, or number. The computer demonstrates the sorting, but does not verbalize how it is sorting. For example, in level 7a and 8a, the child is shown but not told to sort based on size so a large orange rabbit would go into the basket with the large orange boat and then into the basket with the two large green bunnies. The second test card, which is a small green boat would go into the baskets with the small objects on them (See Table 10). When the child finishes the last executive functions task, Little Bear says “Goodbye”.

The need for tests of children’s motivation and executive functions during this transition to school period is very great. Currently, there are many tests of IQ and basic achievement skills, and there are questionnaire assessments of concepts such as intrinsic motivation, mastery motivation, and executive functions. However, to our knowledge there are no standardized behavioral tests including both children’s mastery motivation and executive functions, and no computer-based assessments of both of these skills. Thus, such a test will fill a void in a very large Hungarian, US, and international market. The preliminary data show good reliabilities and construct validity of the tested tasks.

We are currently creating an android version of the tasks. The android app will enable us to do the tasks even when internet access is inconsistent or unavailable. Because the computer tablet essentially administers age-appropriate tasks and collects the data needed for the analyses, individualized adaptive test administration and data collection will not require much teacher time or training.

Our long-term plan is to make the assessment available to school systems as well as researchers. We believe that the tasks will be useful in schools and for school success research as a crucial part of an assessment of school readiness. Our tasks should also aid in the development of individualized assessment plans for intervention or remediation. Ultimately, the assessment will be standardized and available to schools in Hungary, the US, and other countries and languages.

Much research has documented that high quality early childhood education has an extraordinarily high return on investment, given its association with increased school performance and with decreases in later school drop-out, delinquent and other risky behaviors. Both in Hungary and the US, early childhood education and school readiness are important, especially with regard to access to it by low income families. Both countries value individualized assessments of school readiness and individualized curriculum to remediate any deficiencies. A tablet-based assessment can determine each individual child’s level of development on each task, allowing for individualized remediation and enrichment efforts.

The research was supported by the Hungarian Scientific Research Fund, OTKA-K83850 and Colorado State University Ventures Grants. Krisztián Józsa also was supported by the János Bolyai Research Scholarship of the Hungarian Academy of Sciences. The authors thank Amanda Dillard, Gabriella Józsa, and Noémi Kis for their assistance.