Józsa, K. & Morgan, G. A. (2015). An Improved Measure of Mastery Motivation: Reliability and Validity of the Dimensions of Mastery Questionnaire (DMQ 18) for Preschool Children. Hungarian Educational Research Journal, 5(4), 87-103, DOI :10.14413/herj.2015.04.08.
An Improved Measure of Mastery Motivation: Reliability and Validity of the Dimensions of Mastery Questionnaire (DMQ 18) for Precshool Children
Krisztián Józsa[1] & George A. Morgan[2]
Abstract
Recent research has documented well the importance of mastery motivation in early childhood. Thus, it is important to have valid and reliable instruments to measure this concept. The Dimensions of Mastery Questionnaires (DMQ 17) has been used extensively in Hungary and around the world to assess children’s mastery motivation from ratings by parents, teachers, and school-age children themselves. Based on empirical and theoretical feedback, a revised version of the questionnaire (DMQ 18) was developed in English, Hungarian, Chinese and Spanish. The aim of the present study was to report and evaluate the psychometric properties of DMQ 18. In this first publication about the new DMQ, 211 3-6 year-old Hungarian kindergarten children’s mastery motivation was rated by their teachers, using DMQ 18. Validity was supported by a factor analysis which fit well the theoretical dimensions and by strong correlations between DMQ 18 and the extensively validated DMQ 17 scales. Good internal consistency, interrater reliability, test-retest reliability, and longer-term stability also support the usefulness of DMQ 18. We also discuss the importance and usefulness of measuring mastery motivation in preschool and school age children.
Keywords: motivation, early childhood, psychometrics, mastery motivation, questionnaire, teachers’ ratings, reliability, validity
 
Introduction
The US National Academy of Science report, From Neurons to Neighborhoods (Shonkoff & Phillips, 2000), identified mastery motivation as the intrinsic drive to explore and master one’s environment, and said it is a key developmental concept, which should be included as part of a child’s evaluation. Mastery motivation has two major aspects: instrumental and expressive (Barrett & Morgan, 1995). The instrumental aspect motivates a person to attempt, in a focused and persistent manner, to solve a problem or master a skill or task, which is at least moderately challenging for him or her (Morgan, Harmon and Maslin-Cole 1990). The expressive aspect of mastery motivation produces affective reactions while the person is working at such a task or just after completing it. This affect may or may not be overtly expressed and may assume different forms in different children as the child develops (Barrett and Morgan, 1995).
The Dimensions of Mastery Questionnaire (DMQ) is one of several measurement techniques, including challenging structured tasks and semi-structured play, developed to assess mastery motivation (Busch-Rossnagel & Morgan, 2013). The DMQ assesses mastery motivation by having a parent or teacher rate their perceptions of the child’s behavior (or school-aged children rate their own behavior) in mastery contexts. This article is based primarily on data from DMQ 18 ratings by teachers of the mastery motivation and competence of Hungarian preschool children. The development of the DMQ scales is briefly described in the next section, followed by previous research on reliability and validity and by cultural comparisons of DMQ results from Hungary, China and the US.
Development of the DMQ Questionnaire
The DMQ was developed over the last 30 years. The results of early versions were reported in Morgan et al. (1993). DMQ 17 (Morgan, 1997) was used from 1997 to 2014; it had 45 items rated 1-5, from “not at all like this child” to “exactly like this child,” and 7 scales. Four scales (object-oriented persistence, gross motor persistence, social persistence/mastery motivation with adults, and social persistence/mastery motivation with children) measured the instrumental/persistence aspect mastery motivation. Two scales (mastery pleasure and negative reactions to failure in mastery situations) assessed the expressive/affective aspects of mastery motivation. The final scale was not a measure of mastery motivation, instead it was a brief measure of the child’s general competence or ability.
More than 15,000 children from 6-month to 19-years of age were rated with DMQ17. These include more than 1000 children developing atypically with several types of delays and more than 500 children at risk due to low SES. Geographically and linguistically the children were very diverse. Data included English speakers from the United States, Canada, the UK, and Australia. Chinese speakers were from mainland China and Taiwan. In Hungary, more than 10,000 mostly typically developing school-age children rated themselves and/or were rated by their parents and teachers. In some samples, parents, teachers, and/or children themselves did the ratings, but many children were rated by only one rater.
Overviews of DMQ research on the Hungarian, English, and Chinese samples were published by Józsa (2007), Józsa, Hricsovinyi, & Szenczi (2015), Józsa and Molnár (2013), Józsa, Wang, Barrett, and Morgan (2014);and Morgan, Wang, Liao, and Xu (2013). These papers summarized evidence for reliability and validity, relationships to other variables, and also compared the three cultures at similar ages and across ages. Evidence for reliability is presented before validity in this paper because an instrument cannot be valid unless it is reliable. Thus, reliability is a precursor to validity.
Reliability and Validity of the DMQ 17 Ratings
Internal Consistency. Morgan et al. (2013) presented evidence that each of the four DMQ 17 instrumental/ persistence scales and mastery pleasure scale had acceptable to good internal consistency (alphas > .74) for both English and Chinese parent versions and the English version by teachers. Alphas for the child self-ratings were somewhat lower (.67 - .85) on these five scales. Some of the English-speaking children were 5-7 years old, probably too young to fully understand these self-ratings of their motivation, even when the items were read to them and the tester used visual aids. The alphas for negative reactions to failure and competence, the two shortest scales, ranged from .60 - .86 with a median of .70.
There were also good Cronbach alphas for the Hungarian samples (Józsa, 2007; Józsa & Molnár, 2013), on the four instrumental/ persistence scales and the mastery pleasure scale for teachers and parents. Reliabilities of Hungarian teacher ratings were somewhat higher than those of parents. No significant age differences in alphas were found for either the teacher or the parent samples. However, reliability for student self-ratings were somewhat higher for older school-age groups than younger school-age groups. Development of reading comprehension undoubtedly influences the computed reliability of the questionnaire, and it could be the reason for the increase in reliability indices with age. Total persistence, had an alpha of .92. For the Hungarian sample, Cronbach alphas ranged from .67, to .84 (median .74) with alpha of .88 for total persistence.
Thus, in general, the seven scales of DMQ 17 had minimally adequate to excellent internal consistency. Alphas for teacher ratings were the highest and child self-ratings the lowest, especially for children under 8. Alphas for the four instrumental/persistence scales combined were almost always >.80, even for child self-ratings of young children with disabilities.
Test-retest reliability. Józsa and Molnár (2013) reported test-retest reliabilities, ranging from .61 to .94, for 98 Hungarian teachers, parents, and students on the four instrumental and two expressive scales. The median correlations for these scales were .83, .80, and .74 for teacher, parents, and students, respectively. These test-retest correlations were highest for cognitive/object and gross motor persistence, somewhat lower for the social mastery scales and mastery pleasure, and lowest for negative reactions to failure.
Inter-rater reliability. An analysis of the Hungarian DMQ data was carried out by examining the correlations between the ratings of pairs of teachers who rated the same children but in somewhat different contexts (Józsa & Molnár, 2013). One of the teacher raters was the home-room teacher and the other was a teacher who taught the children in several courses. On total mastery motivation correlations between the ratings by these teachers for children in grades 4 and 8 were moderate, indicating a relatively close correspondence between teacher ratings. However, in grade 10, much lower correlations were found. This may be because in grade 10, the teachers teach the children in only one subject (e.g. math or history) so they know the children in different contexts and less well than the teachers in 4th and 8th grades.
Longer term stability. Huang and Lay (2011) found adequate stability (.62 - .75) of the DMQ 17 over a 5-month period (21-26 months) in their Taiwanese toddlers, except for negative reaction to failure. Moreover, Huang and Lay found somewhat lower, but still relatively high, stability over 32 months. Several other studies of the DMQ 17 have also found stability in parent ratings across a time span of 6 months or more (e.g., Wang, Hwang, Liao, Chen, & Hsieh, 2011; Wang, Morgan, & Biringen, 2013).
In terms of longitudinal studies with longer time periods between measures, Gilmore and Cuskelly (2009) found a significant correlation (r = .45) when children with Down syndrome were rated by a parent on object/cognitive persistence when the children were approximately 5 and 13 years of age. Józsa and Morgan (2014) studied Hungarian children and found a significant correlation of .37 for object/cognitive persistence self-ratings across the four years from 4th to 8th grade.
In summary, both short (test-retest) and longer term stability of parent, teacher, and child-self ratings were acceptable for almost all scales in several studies.
Evidence for convergent validity. Evidence for convergent validity in children developing typically includes Morgan and Bartholomew (1998) who found that elementary school-aged children’s DMQ self-ratings were significantly related to the child’s self-ratings of Harter’s (1981) intrinsic motivation scales and to the child’s ratings of Harter’s (1982) preference for challenge. The DMQ scores have also been correlated with other theoretically related variables. For example, Józsa and Morgan (2014) found significant positive correlations between DMQ ratings of object/cognitive persistence and the child’s grade point average. Gilmore, Cuskelly, and Purdie (2003) found that DMQ ratings at 2 years predicted cognitive, reading, and spelling of typically developing girls at age 8. Thus, there are a number of studies that provide evidence for the convergent validity of the DMQ in children developing typically.
Miller, Ziviani, Ware, and Boyd (2014a) provided evidence that inconsistent and excessively lax parental discipline is related to low mastery motivation in school-age children with cerebral palsy. Wang, Morgan, Hwang, Chen, and Liao (2014) found that mother’s cognitive growth fostering teaching interactions with her toddler who had motor delays was significantly correlated to both DMQ total persistence and mastery pleasure. Thus, there are also a number of studies that provide evidence for convergent validity in children with disabilities as well as children developing typically.
Evidence for criterion-related validity. Several studies have found significant relationships between maternal perceptions of child motivation and tester ratings (or scores) of the child’s mastery motivation in standardized testing sessions, which could be considered a criterion measure. For example, Morgan and Bartholomew (1998) found that children’s DMQ total mastery motivation score was significantly correlated with observed mastery motivation during tasks. Gilmore and Cuskelly (2009) found that parent’s DMQ object/cognitive persistence scores were related to persistence at behavioral tasks for children with Down syndrome at age 5, and such parent DMQ scores at age 13 were related with task persistence.
Factorial evidence for validity. Factor analyses of the parent ratings of the four instrumental and the positive expressive (mastery pleasure) items for English-speaking preschool children conformed quite well to the intended dimensions. The Chinese parent preschool ratings and those for English and Chinese infants factored less well but generally in the planned way. It is likely that the differences from intended factors in the Chinese samples reflected both cultural differences and subtle problems with translation of some DMQ items, which we have tried to correct in the DMQ 18 (Morgan et al., 2013).
To assess factorial validity, Józsa et al. (2014) computed principal axis factor analyses on Hungarian, Chinese, and American school-age children’s data for the 30 positively worded DMQ mastery motivation items from the four instrumental scales and mastery pleasure these analysis were computed for the large combined sample and each of the three countries separately. For the combined sample there was strong factorial evidence for the validity of these five mastery motivation scales because these items had their highest factor loading from the intended factor and there were no factors with cross loadings above .30. Similar results were found for the analyses of each country. Thus, the four persistence scales and mastery pleasure have good factorial validity for school-aged children’s self-rating across samples from these three cultures.
Development of the Current Dimensions of Mastery Questionnaire (DMQ18) from DMQ17
Although the DMQ 17 data provided good evidence for reliability and validity of the scores and useful results in a number of studies, we decided for several reasons to make a revision. First, the second author received feedback from some international users which indicated that the overall instructions, scale anchors and certain items were not clearly understood, so we made some recommended changes to increase clarity.
Second, a major issue was that the reverse coded items clearly caused problems with the ratings for 10 - 20% of the raters who did not seem to read them accurately. We reanalyzed a large sample of Hungarian DMQs to examine the effect of filtering out respondents, who seemed to misread the negatively worded items, on the Cronbach alpha’s of the DMQ 17 scales. As expected, the alphas increased. Thus, we decided to omit the negatively worded items from future analyses and publications; i.e., Wang, Józsa, and Morgan (2014); Józsa, Wang, Barrett, and Morgan (2014). Kis, Józsa, and Dombi (2015) also have pointed out that the negative items do not fit well using confirmatory factor analysis.
The social mastery motivation scales, both persistence with adults and also with peers, produced acceptable to good reliabilities, but some items did not seem age appropriate, the school-age and preschool items seemed to focus more on persistence related to play than seemed desirable for a broad measure of social mastery motivation. Therefore, we developed several new items that were pilot tested in Taiwan and the US.
The negative reaction to failure scale frequently produced relatively low alphas and results that were difficult to interpret. Part of the problem seemed to be that the some of the items were related to frustration or anger while others were more related to sadness or shame and avoidance. In addition to developing and pilot testing several new items, we decided to split these negative reaction items into two subscales for the preschool and school-age versions. Shame/sadness did not seem appropriate for infants so that subscale was not included for the infant version of DMQ 18.
J. Wang et al. (2014) examined the measurement invariance of the self-rated DMQs of children, ages 8 to 19 years from the US, China, and Hungary to see which items did and didn’t work well in all three cultures. Positively worded DMQ 17 items from the four instrumental/persistence aspects of mastery motivation and mastery pleasure were used in these analyses. Items with low factor loadings and/or inappropriate cross-loadings were dropped from further analyses. For the retained items, single-group confirmatory factor analyses were conducted. The Comparative Fit Index (CFI) and the Root Mean Square Error of Approximation (RMSEA), indicated a good fit for a five factor model in each sample. Finally, multiple-group confirmatory factor analyses were conducted to examine the measurement invariance of the remaining self-reported DMQ 17 items among the American, Chinese, and Hungarian samples combined. Measurement invariance was confirmed for these school-age children’s self-ratings. The problematic items were not included in the revised DMQ,
Hwang (Personal communication , May 12, 2015) analyzed the DMQ 17 data for samples of Chinese, American and Hungarian preschool children’s parent ratings using confirmatory factor analysis and structural equation modeling to validate the hypothesized 5-factor structure of the DMQ and the invariance across English-speaking, Chinese-speaking and Hungarian-speaking preschool children. Cross-cultural invariance was found after a few items were deleted. The results support the revised (DMQ18) preschool version of the mastery motivation questionnaire.
Finally, we wanted to be as certain as possible that there was not only linguistic equivalence of the revised items across cultures but that the items were age and culturally appropriate. As mentioned earlier, several of the new items had successfully been tried out in Taiwan and the US. All the new items were translated in Chinese, Spanish, and Hungarian, examined by the authors and checked with some parents and professionals to ensure that the phrases were clear and appropriate. Questions and concerns led to several changes not only in the Chinese, Spanish, and Hungarian but also in the English versions. Thus, the process was similar to back translation plus decentering.
DMQ 18 has four current language versions: English, Chinese, Spanish, and Hungarian (Morgan et al., 2015). There are three parallel age-related versions (infant, preschool, and school-age, which can be rated by adults and by children to rate themselves) in each language. The Infant version is for children of developmental ages approximately 6-23 months rated by an adult. The preschool version is for young children of developmental ages approximately 2-6 years rated by an adult. The school-age by adult version is for students from 1st grade through high school rated by a parent or teacher. Likewise, the school-age self-rating version is for students from 3rd grade through high school. DMQ 18 has the same seven scales at each of the three ages and the same or similar age-appropriate items at each age (Morgan et. al, 2015).
DMQ 18 has some new or revised items but the same scales as DMQ 17, except the negative reaction scale has two subscales. Several Studies using various language and age versions of the DMQ 18 are underway. The current study focused on psychometric properties and is the first to be completed using the revised Dimensions of Mastery Questionnaire (DMQ18). The data are from a sample of Hungarian preschool children.
The aims of the current study are to:
1.      Report and evaluate the internal consistency reliability of the revised Dimensions of the Mastery Questionnaire (DMQ 18).
2.      Examine and evaluate the interrater and test-retest reliability, as well as the longer-term stability of DMQ18.
3.      Examine and evaluate the validity of the DMQ using factor analysis of the DMQ 18 items and correlations between the scale scores of DMQ17 and DMQ18.
Method
Participants
The present study was conducted in Hungary where free universal early childhood education (called kindergarten) is guaranteed by law from age 3. It includes three or more years in a whole day preschool. At least 1 year of preschool is required prior to school entry, but the average time spent in preschool is 3.3 years. Approximately 85% of 3- to 4-year-olds attend kindergarten, as do 91% of 4- to 5-year-olds and 97% of 5- to 6-year-olds.
The sample included 211 Hungarian preschool children 3-6 years of age, who were rated by one of their teachers. The children were from 10 preschool groups at three publically funded kindergartens in two Southern-Hungarian towns. All of the children were native Hungarian speakers; 47% of the sample were girls. The average number of years of parental education was  approximately 11.55 years (SD = 1.79) for fathers and 12.21 for mothers (SD = 2.03). The distribution of the parents’ education in the sample reflected well the national distribution of Hungarian parents (Józsa, 2004); see Table 1.
Table 1. Distribution of Hungarian parents’ level of education (%)
Level of education
Father
Mother
Finished primary school
9
6
Vocational school
33
25
Secondary school
22
37
BA
33
25
MA
3
7
Table 2 shows the number of preschool children in each of the four age groups plus their mean age and standard deviation in months. Note that there are approximately equal numbers of children who were about 3½, 4½, 5½, and 6½ years old, respectively.
Table 2. Means ages and standard deviations (in months) of the 211 Hungarian preschool children
Age
n
M
SD
3-year-olds
58
42.33
2.73
4-year-olds
53
53.44
3.73
5-year-olds
48
65.66
3.55
6-year-olds
52
77.29
2.99
Instruments
The new Dimensions of Mastery Questionnaire (DMQ18) preschool version was the main instrument used for this study. For purposes of construct validity the same children were rated using DMQ17. The seven DMQ18 scales and a common item are presented as follows:
Four scales for the instrumental (persistence) aspects of mastery motivations are:
1.      Cognitive/object persistence scale (5 items for preschoolers)
Common item “Works for a long time trying to do something challenging.”
2.      Gross motor persistence scale (5 items)
Common item “Tries to do well in physical activities even when they are challenging.”
3.      Social persistence/mastery motivation with adults scale (5 items for preschoolers)
Common item “Tries hard to get adults to understand him or her.”
4.      Social persistence/mastery motivation with children/peers scale (6 items)
Common item “Tries to do and say things that keep other children interested.”
Two scales for the expressive aspects of mastery motivation are:
1.      Mastery pleasure scale, positive affect after finishing and/or while working on a task (5 items)
Common item “Gets excited when he or she figures something out.”
2.      Negative reactions to challenge in mastery situations scales (8 items with 2 subscales)
Common item for frustration/anger subscale “Gets frustrated when not able to complete a challenging task.”
Common item for sadness/shame subscale “Looks away when tries but cannot do something.”
One scale to assess competence or the ability to solve problems in contrast to the motivation to master tasks is:
1.      General competence compared to peers scale (5 items)
Common item “Does things that are difficult for children his or her age.”
In addition, a total persistence score was computed based on the average of scales 1-4. Thus, 10 scales are reported in the tables and results; the 7 main scales, total persistence, and the 2 negative reaction subscales.
Procedure
Data collection was conducted in early 2015. One teacher from each of 10 preschool/kindergarten classes rated the 211 children’s mastery motivation using first DMQ 17 and then DMQ 18 about a month apart. The same teachers rated a subgroup of children (n=120) using DMQ 18 again approximately 2 months later; based on this subsample, we computed a measure of longer-term stability. Another subgroup of children (n=133) were rated using DMQ18 by both preschool teachers who taught the child’s class. Based on this subsample interrater reliability was computed. A third subsample (n=58) was rated twice by the second teacher, approximately two weeks apart, in order to assess test-retest reliability. Parents were asked about their level of education.
Data Analysis
The data were analyzed using SPSS. Cronbach alphas, intraclass correlation coefficients, and principal axis factor analysis were computed. Interpretation of coefficients were based on recommendations by the Gliner, Morgan and Leech (2009) research methods book.
Results
The means and standard deviations of the ten DMQ18 scales varied between 3.03 for frustration/anger and 4.10 for mastery pleasure on the 1 to 5 rating scale as shown in Table 3. Thus, on average the items were rated as at least somewhat like this child. All the scales had standard deviations less than 1 and were approximately normally distributed, supporting an assumption for most parametric statistics.
Table 3. Means, standard deviations, and Cronbach alphas for Hungarian preschool children’s Dimensions of Mastery Questionnaire (DMQ 18) scales
Scale
Items
M
SD
Alpha
Persistence scales
Cognitive/object
5
3.58
.81
.93
Gross Motor
5
3.81
.95
.96
Social w. Adults
5
3.52
.91
.91
Social w. Children
6
3.74
.70
.90
Total persistence
21
3.66
.70
.95
Expressive scales
Mastery Pleasure
5
4.10
.64
.90
Negative Reactions
8
3.05
.63
.79
Frustration/anger
4
3.03
.85
.85
Sadness/shame
4
3.07
.60
.54
General Competence
5
3.68
.89
.94
Reliability
Table 3 also shows the Cronbach alphas, a common measure of internal consistency reliability, for each scale. All the reliability coefficients for the persistence scales, mastery pleasure, and general competence were .90 or above, indicating excellent internal consistency. The alpha for the negative reactions to challenge in mastery situations subscale of frustration/anger was .85, and for overall negative reactions alpha was.79, both considered to be acceptable to good. However, the alpha for the negative reactions of sadness/shame subscale was only .54, not acceptable for internal consistency reliability.
The interrater reliabilities shown in Table 4 indicate generally adequate reliability using intraclass correlation coefficients (ICC) based on ratings of 133 children by each child’s two preschool teachers. Reliabilities ranged from inadequate for the two negative reactions subscales to .87 for general competence. Apparently preschool teachers see the negative reactions differently but have little trouble evaluating and agreeing on a child’s ability or competence relative to other children. The ICCs for mastery pleasure and three of the four instrumental persistence scales (cognitive, social with adults, and social with peers) as well as the total persistence scale were between .78 and .85, indicating acceptable to good interrater reliability. Surprisingly, the interrater reliability for gross motor persistence was only .65 indicating marginally acceptable interrater reliability. It seems that teachers have a harder time evaluating their preschool students’ motivation for large motor physical activities.
Table 4. Intraclass correlation coefficients (ICC) to assess interrater reliability, test-retest reliability, and two-month stability and also correlations between the same scales of DMQ17 and DMQ18 for Hungarian preschool children
DMQ 18 scale
Interrater ICC
Test-retest ICC
Stability ICC
r with DMQ17 scales
n
133
58
120
193
Persistence Scale
Cognitive/object
.85
.87
.61
.63
Gross Motor
.65
.84
.57
.60
Social w. Adults
.78
.89
.71
.76
Social w. Children
.79
.89
.71
.65
Total persistence
.83
.89
.72
.76
Expressive scales
Mastery Pleasure
.78
.82
.56
.59
Negative Reactions
.61
.78
.35
.38
Frustration/anger
.52
.76
.43
NA
Sadness/shame
.59
.69
.24
NA
General Competence
.87
.88
.73
.72 
Table 4 also shows the test-retest reliability for each of the scales, based on the ratings of 58 teachers who rated the same child twice approximately two weeks apart. The intraclass correlation coefficients (ICC) were above .80 for all the persistence scales, mastery pleasure and general competence, indicating good test-retest reliability. The overall negative reactions scale (.78) and the frustration/anger subscale (.76) had adequate test-retest reliability but the sadness/shame subscale had an ICC of .69, a marginally acceptable indication of reliability.
Table 4 also shows longer-term stability of 120 children’s motivation based on a teacher’s ratings of each child approximately two months apart. ICC stability coefficients were, as expected, somewhat lower than test-retest coefficients. The persistence scale ICCs varied from .57 for gross motor persistence to .72 for the total persistence. These stability coefficients indicate that the teacher perceives that the relative motivation of children in her class varied somewhat over the two months, with some children being ranked relatively higher on persistence after two months compared to other children in the class and others ranked lower. The 2-month stability scores for the negative reactions to challenge ratings were quite low (.24 - .34). This indicates that teachers perceive these dimensions to be quite unstable, and in addition, the teacher ratings indicated that these negative reactions to challenge, both frustration/anger and sadness/shame, were perceived to have increased significantly on average over the two months.
Validity
A five factor principal axis factor analysis with promax (oblique) rotation was used to see whether the empirical grouping of items fit the theory that there are four distinct but intercorrelated persistence dimensions and a distinct mastery pleasure dimension. Table 5 shows the results of this 5-factor PAF analysis. It indicates an excellent fit of the theory and the empirical data because each of the items in the five scales had high factor loadings (.5 or above) on the appropriate factor, and there were no items that cross-loaded above .4 on another scale. Item 6 did not load above .4 on any scale, which may mean that it should be deleted or rewritten.
Table 5. Principal Axis Factor Analysis of the items from the four persistence scales and mastery pleasure of DMQ18 for 205 Hungarian Preschool children
 
Scales and Items
Gross Motor Persist
Social Persist. Peers
Cognitive Persist.
Mastery Pleasure
Social Persist. Adults
Gross motor persistence
26
Repeats jumping/running skills until can do them
.94
 
 
 
 
 3
Tries to do well at motor activities
.94
 
 
 
 
12
Tries to do well in physical activities
.88
 
 
 
 
36
Tries hard to get better at physical skills
.87
 
 
 
 
38
Tries hard to improve throwing or kicking
.84
 
 
 
 
Social persistence with peers
28
Tries hard to make friends with other kids
 
.94
 
 
 
35
Tries to keep play with kids going
 
.91
 
 
 
32
Tries to get included when children playing
 
.87
 
 
 
 7
Tries to do things to keep children interested
 
.58
 
 
 
25
Tries to understand other children
 
.56
 
 
 
Cognitive persistence
23
Works long to do something challenging
 
 
.87
 
 
17
Tries to complete toys like puzzles
 
 
.85
 
 
14
Tries to complete tasks, even if takes a long time
 
 
.83
 
 
29
Will work a long time to put something together
 
 
.81
 
 
 1
Repeats a new skill until he can do it
 
 
.62
 
 
Mastery Pleasure
18
 
Gets excited when figures out something
 
 
 
.91
 
11
Shows excitement when is successful
 
 
 
.88
 
30
Smiles when makes something happen
 
 
 
.80
 
 2
Smiles broadly after finishing something
 
 
 
.75
 
21
Is pleased when solves a challenging problem
 
 
 
.72
 
Social persistence with adults
33
Tries to figure out what adults like
 
 
 
 
.92
37
Tries hard to understand my feelings
 
 
 
 
.87
15
Tries hard to interest adults in playing
 
 
 
 
.87
22
Tries hard to get adults to understand
 
 
 
 
.51
 8
Tries to keep adults interested in talking
 
 
 
 
.51
Note. Principal axis factor analysis with promax rotation. These five factors account for 71% of the variance. Loading less than .40 have been omitted. Item 6, theoretically intended as a social persistence with peers item, is not shown because it didn’t load above .40 any scale.
Another piece of evidence for the validity of DMQ18 is shown in the right hand column of Table 4. If ratings of the scales scores of DMQ18 are significantly correlated with the same scales on DMQ17, support for construct validity is provided (Gliner et. al, 2009). Such supporting evidence is provided for all the scales except negative reactions to challenge. Note that correlations were not expected to be as high as the reliability coefficients because some scales included some new items, and especially negative reaction items were changed substantially in the development of DMQ18.
Other Results
There were no significant gender differences on any of the mastery motivation scales or on the competence scale. Pairs of age groups (e.g., 4 vs. 5 year-olds) differed significantly at p < .05 on only 10 out of 60 possible post hoc tests comparing the four age groups on the various scales. Although this is somewhat above chance, the only meaningful pattern of paired age differences was that the 6- and 5- year olds were rated higher on mastery pleasure than both the 4- and 3-year old children.
There was only 1 out of 20 significant relationships between mother’s or father’s education and the DMQ scale scores. Father’s with more education were slightly more likely (r = .19, p < .02) to have children rated higher on general competence or ability by their preschool teacher.
Discussion
This study introduces a revised and improved measure of the motivation of children to solve problems and master skills, the Dimensions of Mastery Questionnaire version 18. Data in this study provide evidence that DMQ18 produces reliable and valid information about a child’s mastery motivation. Preschool teachers rated their student’s mastery motivation on four instrumental/persistence dimensions (cognitive, gross motor, social with adults and social with peers) and two expressive/affective dimensions (mastery pleasure and negative reactions to challenging tasks). The teachers also rated the children’s ability or competence relative to other children the same age.
Reliability
Most of the DMQ18 scales provided good evidence to support the three measures of reliability: internal consistency of the scales, interrater, and test-retest. The exception was the negative reaction to challenge scale and, especially the sadness/shame subscale, where some revised items are being considered. Three of the four instrumental/persistence scales, mastery pleasure, and the competence/ability scales provided good evidence for all three measures of reliability, but gross motor persistence provided only marginally adequate evidence for interrater reliability. Preschool teachers seem to have a somewhat difficult time agreeing about their students’ motivation for physical activities. These reliability measures are similar to or higher than those reported by Józsa and Molnár (2013) for Hungarian children and by Morgan et al. (2013) for English and Chinese speaking children.
Stability of ratings over a two-month period was less reliable, perhaps because children changed differentially over that time period. Again, the negative reaction ratings were the least stable, probably indicating both differential changes in the children and reflecting lower general reliability of those ratings. The stability of the persistence scales, mastery pleasure and competence is as good as or better than what might be expected from the literature (Józsa & Morgan, 2014; Wang et al., 2011; Wang et al., 2013).
Validity
Factorial evidence for the validity of the four persistence and mastery pleasure scales was excellent and at least as strong as that provided by Józsa (2007) and Józsa et al. (2014) for DMQ 17. Preliminary evidence for the construct validity of DMQ18 is provided by significant correlations with the same teacher’s ratings of DMQ 17, which has a lot of evidence to support its validity (e.g., Gilmore & Cuskelly, 2009; Józsa & Morgan, 2014; Morgan et al., 2015).
Limitations
Using ratings from questionnaires and inventories, including the DMQ, poses several validity issues. There is always the question of the extent to which the rater’s perceptions of behavior are in agreement with actual behavior; for example when one person (e.g., a teacher) is rating another person’s (e.g., the child’s) behavior. There are also validity concerns whenever an instrument developed in one language and culture is translated and used in another culture. With the Chinese and Hungarian translations of DMQ 17, decentering (adjusting both the English and other language questionnaires) was not used, but we used a procedure similar to decentering in the development of the Hungarian DMQ 18.
Importance and Usefulness of the DMQ 18
Mastery motivation is a fundamental developmental construct that should be used as a key component of a comprehensive evaluation of children’s functioning (Shonkoff & Phillips, 2000). Recent studies have pointed out that mastery motivation has an important role in school success. For example, Józsa, Barrett and Morgan (2015) examined the role of early mastery motivation as a predictor of school success and found that mastery motivation predicted school success even after controlling IQ and parents’ education. Several other studies found strong associations between mastery motivation and both GPA and achievement in school subjects (Józsa, 2007; Józsa & Molnár, 2013; Józsa & Morgan, 2014). These studies highlight the importance of developing intervention programs to improve mastery motivation, which should lead to better school readiness and school performance.
There is also considerable research evidence that intrinsic motivation and mastery motivation decline during the school years (Gottfried et al, 2007; Józsa & Morgan, 2014). This decline is very similar in Hungary, China, and the US (Józsa et al., 2014) and is concerning to educators. Other studies have reported important results about the mastery motivation of children with special needs (Gilmore and Cuskelly, 2009, 2011; Miller et al., 2014a, 2014b; Wang et al., 2013, 2014).
The results of all these studies and others indicate that developing valid measures of children’s mastery motivation is important to understanding many aspects of child development and success in school. Thus, it is important to have valid and reliable measures to assess mastery motivation. DMQ 18 provides strong psychometric evidence to support its usefulness in future studies.
Another important use of the DMQ, and of behavioral tasks of mastery motivation, is as part of a school readiness assessment. Hungarian studies of the widely used DIFER (Nagy, Józsa, Vidákovich, & Fazekasné, 2004a, 2004b) indicate that DIFER is a good predictor of preschool children’s success in primary school (Józsa, 2014; Zentai & Józsa, 2012). The DMQ provides additional information to supplement the more traditional DIFER school readiness data (Józsa, 2004). Our long term goal is to increase the predictive power and usefulness of DIFER by adding a valid measure of mastery motivation to the battery.
 
Acknowledgments
Several colleagues who are not co-authors of this paper played key roles in the development of the DMQ18: Karen Barrett, Hua-Fang Liao, Su-Ying Hwang, Jun Wang, and Pei-Jung Wang. We also want to acknowledge that many researchers around the world provided DMQ data used to support the reliability and validity of DMQ17. The research was supported by the European Union and the State of Hungary, co-financed by the European Social Fund in the framework of TÁMOP-3.1.9-11/1-2012-0001 "Developing Diagnostic Assessments' project”. Krisztián Józsa also was supported by the János Bolyai Research Scholarship of the Hungarian Academy of Sciences.
 
References
Barrett, K. C., & Morgan, G. A. (1995). Continuities and discontinuities in mastery motivation in infancy and toddlerhood: A conceptualization and review. In R. H. MacTurk & G. A. Morgan (Eds.), Mastery motivation: Origins, conceptualizations, and applications (pp. 67–93). Norwood, NJ: Ablex.
Busch-Rossnagel, N. A., & Morgan, G. A. (2013). Introduction to the mastery motivation and self-regulation section. In K. C. Barrett, N. A. Fox, G. A. Morgan, D. J. Fidler, & L. A. Daunhauer (Eds.), Handbook of self-regulatory processes in development: New directions and international perspectives (pp. 247–264). New York, NY: Psychology Press.
Gilmore, L., & Cuskelly, M. (2009). A longitudinal study of motivation and competence in children with Down syndrome: Early childhood to early adolescence. Journal of Intellectual Disability Research, 53, 484–492.
Gilmore, L., & Cuskelly, M. (2011). Observational assessment and maternal reports of motivation in children and adolescents with Down syndrome. American Association on Intellectual and Developmental Disabilities, 116, 153–164.
Gilmore, L., Cuskelly, M., & Purdie, N. (2003). Mastery motivation: Stability and predictive validity from ages two to eight. Early Education and Development, 14, 411–424.
Gliner, J. G., Morgan, G. A., & Leech, N. L. (2009). Research methods in applied settings: An integrated approach to design and analysis. (2nd ed.). New York: Routledge/Taylor & Francis Group.
Gottfried, A. E., Marcoulides, G. A., Gottfried, A. W., Oliver, P. H., & Guerin, D. W. (2007). Multivariate latent change modeling of developmental decline in academic intrinsic math motivation and achievement: Childhood through adolescence. International Journal of Behavioral Development, 31(4), 317−327.
Harter, S. (1981). A new self-report scale of intrinsic versus extrinsic orientation in the classroom: motivational and informational components. Developmental Psychology, 17, 300–312.
Harter, S. (1982). The perceived competence scale for children. Child Development, 53, 87–97.
Huang, S-Y., & Lay, K-L. (2011, October). The psychometric characteristics and long-term stability of the Dimensions of Mastery Questionnaire in Chinese children in Taiwan from 10 months to 53 months (pp. 55). Program and Proceedings of the 12th Early Intervention Conference, Taipei, Taiwan.
Józsa, K. (2004). Az elsõ osztályos tanulók elemi alapkészségeinek fejlettsége: Egy longitudinális kutatás elsõ mérési pontja [Developmental level of first-grade students’ basic skills. The first measurement point of a longitudinal research program]. Iskolakultúra, 14(11), 3–16.
Józsa, K. (2007). Az elsajátítási motiváció (The dimensions of mastery motivation). Budapest, Hungary: Mûszaki Kiadó.
Józsa, K. (2014). A számolás fejlesztése 4–8 éves életkorban [Games and methods for improving pre-math skills in ages of 4–8].Szeged: Mozaik Kiadó. (in Hungarian)
Józsa, K., Barrett, K. C., & Morgan, G. (2015, May). A longitudinal study of the role of mastery motivation in Hungarian children’s school performance. Invited presentation, Colorado State University Department of Human Development and Family Studies. Fort Collins, US. (Submitted for publication)
Józsa K., Hricsovinyi, J., & Szenczi, B. (2015). Számítógép-alapú Elsajátítási motiváció kérdõívek validitása és reliabilitása [Validity and Reliability of the computer-based Dimensions of Mastery Questionnaire]. In B. Csapó & A. Zsolnai (Eds.), Online diagnosztikus mérések az iskola kezdõ szakaszában [Online diagnostics assessment in elementary school] (pp. 123–146). Budapest: Oktatáskutató és Fejlesztõ Intézet.
Józsa, K., & Molnár, E. D. (2013). The relationship between mastery motivation, self-regulated learning and school success: A Hungarian and European perspective. In K. C. Barrett, N. A. Fox, G. A. Morgan, D. J. Fidler, & L. A. Daunhauer (Eds.), Handbook on self-regulatory processes in development: New directions and international perspectives (pp. 265–304). New York, NY: Psychology Press.
Józsa, K., & Morgan, G.A. (2014). Developmental changes in cognitive persistence and academic achievement between grade 4 and grade 8. European Journal of Psychology of Education, 29(3), 521–535.
Józsa, K., Wang, J., Barrett, K. C., & Morgan, G. A. (2014). Age and cultural differences in mastery motivation in American, Chinese, and Hungarian school-age children. Child Development Research. 2014, 16 pp. doi 10.1155/2014/803061
Kis, N., Józsa, K., & Dombi. E. (2015, April). Az Elsajátítási Motiváció Kérdõív tanulói változatának megerõsítõ faktoranalízise [Confirmatory factor analyzes of self-ratings of the Dimensions of Mastery Questionnaire (DMQ 17)]. In Cs. Csíkos, & Z. Gál (Eds.), XIII. Pedagógiai Értékelési Konferencia – 13th Conference on Educational Assessment (pp. 153). Szeged: SZTE Neveléstudományi Doktori Iskola.
Miller, L., Ziviani, J., Ware, R. S., & Boyd, R. N. (2014 a). Mastery motivation in children with congenital hemiplegia: individual and environmental associations. Developmental Medicine and Child Neurology. 56, 267–274. doi: 10.1111/dmcn.12356
Miller, L., Ziviani, J., Ware, R. S., & Boyd, R. N. (2014 b). Mastery motivation as a predictor of occupational performance following upper limb intervention for school-aged children with congenital hemiplegia. Developmental Medicine and Child Neurology. 56, 976-983.doi: 10.1111/dmcn.12471
Morgan, G. A. (1997). Dimensions of Mastery Questionnaire (Database Record). Retrieved from PsycTESTS. doi:10.1037/t04950-000
Morgan, G. A., & Bartholomew, S. (1998). Assessing mastery motivation in 7-and 10-year olds. In: Busch-Rossnagel, N. A. & Morgan, G. A. (chairs): New measures of mastery motivation for infancy through elementary school. Symposium conducted at the Fourth Head Start National Research Conference, Washington (ERIC Document No, ED422-110).
Morgan, G. A., Harmon, R. J., & Maslin-Cole, C. A. (1990). Mastery motivation: Definition and measurement. Early Education and Development, 1, 318–339.
Morgan, G. A., Maslin-Cole, C. A., Harmon, R. J., Busch-Rossnagel, N. A., Jennings, K. D., Hauser-Cram, P., & Brockman, L. (1993). Parent and teacher perceptions of young children’s mastery motivation: Assessment and review of research. In D. Messer (Ed.). Mastery motivation in early childhood: Development, measurement and social processes (pp. 109–131). London: Routledge.
Morgan, G. A., Wang, J., Barrett, K. C., Liao, H.-F., Wang, P.-J., Huang, S.-Y., & Jozsa, K. (2015). The Revised Dimensions of Mastery Questionnaire (DMQ 18). Colorado State University, Fort Collins, CO. https://sites.google.com/a/rams.colostate.edu/georgemorgan/mastery-motivation
Morgan, G. A., Wang, J., Liao, H.-F, & Xu, Q. (2013). Using the Dimensions of Mastery Questionnaire to assess mastery motivation of English- and Chinese- speaking children: Psychometrics and implications for self-regulation. In K. C. Barrett, N. A. Fox, G. A. Morgan, D. J. Fidler, & L. A. Daunhauer (Eds.), Handbook of self-regulatory processes in development: New directions and international perspectives (pp. 305–335). New York, NY: Psychology Press.
Nagy, J., Józsa, K., Vidákovich, T., & Fazekasné Fenyvesi, M. (2004 a). DIFER Programcsomag: Diagnosztikus fejlõdésvizsgáló és kritériumorientált fejlesztõ rendszer 4–8 évesek számára. [DIFER: A diagnostic system of assessment and criterion-referenced improvement of basic skills between the ages of 4–8.] Szeged: Mozaik Kiadó. (in Hungarian)
Nagy, J., Józsa, K., Vidákovich, T., & Fazekasné Fenyvesi, M. (2004 b). Az elemi alapkészégek fejlõdése 4–8 éves életkorban. Az eredményes iskolakezdés hét kritikus alapkészségének országos helyzetképe és a pedagógiai tanulságok [The development of basic skills between the ages of 4 and 8. A national overview of the seven basic skills needed for academic success and their educational consequences]. Szeged: Mozaik Kiadó. (in Hungarian)
Shonkoff, J. P., & Phillips, D. A. (2000). From neurons to neighborhoods: The science of early childhood development. Washington DC: National Academy Press.
Wang, J., Józsa, K., & Morgan, G. A. (2014, May). Measurement invariance across children in US, China, and Hungary: A revised dimensions of Mastery Questionnaire (DMQ). [Summary] Program and Proceedings of the 18th Biennial Developmental Psychology Research Group Conference, Golden, CO.
Wang, J., Morgan, G.A., & Biringen, J. (2013). Mother-infant affect exchange and mastery motivation in toddlerhood. Infant and Child Development, 23 (2), 139–152. doi: 10.1002/icd.1825.
Wang, P.-J., Hwang, A.-W., Liao, H.-F., Chen, P.-C., & Hsieh, W.-S. (2011). The stability of mastery motivation and its relationship with home environment in infants and toddlers. Infant Behavior and Development, 34, 434–442. doi: 10.1016/j.infbeh. 2011.04.005.
Wang, P.-J., Morgan, G. A., Hwang, A.-W., & Liao, H.-F. (2013). Individualized behavioral assessments and maternal ratings of mastery motivation in mental-age matched toddlers with and without motor delays. Physical Therapy, 93, 79–87.
Wang, P.-J., Morgan, G. A., Hwang, A.-W., Chen, L. C., & Liao, H.-F. (2014). Do maternal interactive behaviors correlate with developmental outcomes and mastery motivation in toddlers with and without motor delay? Physical Therapy, 94, 1744–1754.
Zentai, G., & Józsa, K. (2012). Az elemi alapkészségek óvodáskori fejlettségének prediktív ereje az elsõs készségfejlettségre: egy longitudinális vizsgálat eredményei [Predicting grade 1 school achievement from preschool basic skills: A longitudinal study] (pp. 154.) In A. Benedek, P. Tóth, & A. Vedovatti (Eds.), XII. Országos Neveléstudományi Konferencia: A munka és nevelés világa a tudományban. Program és Összefoglalók. Budapest: MTA Pedagógiai Bizottság. (in Hungarian)
 
[1] University of Szeged (Hungary), Email address: jozsa@sol.cc.u-szeged.hu
[2] Colorado State University (US), Email address: George.Morgan@colostate.edu