Salud Mental 2018;

ISSN: 0185-3325

DOI: 10.17711/SM.0185-3325.2018.011

Received: 16 May 2017 Accepted: 2 April 2018

Internal and inter-rater reliability of the ASQ-3 in Mexican preschoolers

Silvia Ortiz-León 1 , Armida Granados-Rojas 2 , Josefa Cavazos-Olivo 3 , Leonardo De Benito-Avendaño 4 , Victor Hugo Obregón-García 5 , Xenia Margarita Duran-Avendaño 6 , Rosario Viridiana Ramírez-Pérez 7 , Juan Alberto Cárdenas-Medina 8 , Blanca Estela Vargas-Terréz 9 , Gerardo Luna-Guevara 1 , Ana Carolina Rodríguez-Machain 1 , Aurora Leonila Jaimes-Medrano 1


1 Departamento de Psiquiatría y Salud Mental, Facultad de Medicina, Universidad Nacional Autónoma de México, Ciudad de México, México.

2 Hospital Psiquiátrico Infantil Juan N. Navarro, Ciudad de México, México.

3 Centro Comunitario de Salud Mental Cuauhtémoc, Ciudad de México, México.

4 Instituto de Neurociencias, Investigación y Desarrollo Emocional, Ciudad de México, México.

5 CISAME Chimalhuacán y CISAME Ixtapaluca, Estado de México, México.

6 Hospital Nacional de Niños Benjamín Bloom (HNNBB), San Salvador, El Salvador.

7 Grupo Paidos, La Paz, Baja California Sur, México.

8 Neurocom, Ciudad de México, México.

9 Universidad La Salle, Unidad Benjamín Franklin, Ciudad de México, México.

Correspondence: Aurora Leonila Jaimes Medrano Departamento de Psiquiatría y Salud Mental, Facultad de Medicina, Universidad Nacional Autónoma de México. Avenida Universidad, Ciudad Universitaria, Delegación Coyoacán, C.P. 04510, Ciudad de México, México. Phone: +52 (55) 5624 - 5069 Email: aurojm@me.com


Abstract:
Introduction. Monitoring child development includes the promotion of development in the healthy child and the detection of delays and early indicators of disorders that begin in the first five years of life through easy-to-use, easy to mark, and low-cost screening tests.
Objective. To evaluate the internal reliability and inter-rater reliability of the ASQ-3 in Mexican preschool children.
Method. The ASQ-3 was applied to parents and/or teachers of 33 - 60 month old children who attended the Centros de Desarrollo Infantil (Child Development Centers-CENDIS) in the public and private sectors of Mexico City.
Results. A total of n = 1052 questionnaires were obtained, grouped into six age groups (33, 36, 42, 48, 54, and 60 months of age) according to the Ages and Stages Questionnaire-3 (ASQ-3). The levels of reliability of the ASQ-3 for each of the age groups were acceptable, with a range of α = .77 to α = .88. Regarding inter-rater reliability (parents vs. teachers), moderate correlation levels were observed.
Discussion and conclusion. The results obtained suggest that this is a screening instrument that reliably evaluates the five areas of development that make up the ASQ-3 in pre-school children. This represents an opportunity to continue studying the psychometric characteristics of validity of this instrument in representative samples of Mexican children to optimize the process of early childhood development monitoring.

Key words: Child development, mass screening, reliability, surveillance.

Resumen:
Introducción. La vigilancia del desarrollo infantil contempla la promoción del desarrollo en el niño sano y la detección de retrasos e indicadores tempranos de trastornos que se inician en los primeros cinco años de vida por medio de pruebas de tamizaje de fácil aplicación, calificación y de bajo costo.
Objetivo. Evaluar la fiabilidad interna y confiabilidad interevaluador del Cuestionario de Edades y Etapas-3 (ASQ-3) en preescolares mexicanos.
Método. Se aplicó el ASQ-3 a padres y/o tutores de niños de edades entre 33 y 60 meses que asistían a Estancias de Desarrollo Infantil (CENDIS) del sector público y privado de la Ciudad de México.
Resultados. Se obtuvieron un total de n = 1052 cuestionarios, agrupados en seis grupos de edad (33, 36, 42, 48, 54 y 60 meses de edad) de acuerdo con el ASQ-3. Los niveles de confiabilidad de los ASQ-3 para cada uno de los grupos de edad fueron aceptables con un rango de α =. 77 a α =. 88. Respecto a la confiabilidad interevaluador (padres vs. maestros), se observaron niveles de correlación moderados.
Discusión y conclusión. Los resultados obtenidos sugieren que es un instrumento de tamizaje que evalúa de forma fidedigna las cinco áreas del desarrollo que conforman el ASQ-3 en los preescolares. Ello representa una oportunidad para seguir estudiando las características psicométricas de validez de este instrumento en muestras representativas de niños mexicanos con la finalidad de optimizar el proceso de vigilancia del desarrollo en edades tempranas.

Palabras clave: Desarrollo infantil, tamizaje, fiabilidad, vigilancia.




INTRODUCTION

Development is defined as the dynamic, continuous acquisition of functions and independent domains that interrelate with each other and the environment in a non-linear way to achieve a balance throughout life. Studies have documented the influence of risk and protective factors that influence the course of early childhood development, which should be taken into account in primary health care (Walker et al., 2011). During this first phase of care, monitoring includes activities related to the promotion of normal development and the detection of delays or problems in the early years of life, which implies the continuous participation mainly of health professionals, parents and teachers (Hutchson & Nicoll, 1988).

Due to the complexity of the interaction between development areas and its path, it has been proposed that a delay in one area correlates with delay in others, which are common underlying manifestations of developmental deficits. Regarding this point, the complex, multifaceted relationship between language, cognition, and motor skills has been described, as a result of which intervention programs must take into account and promote these areas (Wang, Lekhal, Aarø, & Schjølberg, 2014; Piek, Dawson, Smith, & Gasson, 2008; Houwen, Visser, van der Putten, & Vlaskamp, 2016). A delay in acquiring functions of some or all areas of development is associated with the combination of several risk factors encompassing economic, social, environmental, and biological aspects, which can affect the level of a person’s functioning and may or may not cause disability (Narbona & Schlumberger, 2008). Thus, disability is defined as an activity limitation and a participation restriction, caused by a disorder that permanently affects the individual (World Health Organization, 2001).

The health care and wellbeing of children at a very early age are subjects of global interest and have been addressed in various fields of knowledge, such as medical, psychological, social, and economic spheres, in which research has been conducted on care programs and the impact they have had not only in the short term, but also at later stages of human development (Heckman, Pinto, & Savelyev, 2013; Heckman, 2011). In 2007, Chile became a pioneer in Latin America to implement a system for developmental care and monitoring from zero to four years, called Chile Grows with You Comprehensive Child Protection System (CHCC). One of its main objectives was to reduce the prevalence of developmental delay risk through specific actions for children and mothers in vulnerable conditions. This program has been evaluated at various levels through indicators to identify areas of opportunity and improve early childhood care by restructuring child health actions and policies in this country (Atalah et al., 2014), which has prompted other countries in the region to develop similar programs (Bedregal, 2014).

In Mexico, interest for early childhood and concern for its care is based on the Official Norm for Child Health Care (Secretaría de Salud, 1999), which not only establishes disease eradication and control, but also stipulates that all health institutions must periodically monitor the healthy growth and development of children under five through clinical observation. Various organizations and collegiate bodies have established guidelines and policies for child development. The American Academy of Pediatrics suggests continuous surveillance and monitoring of development, taking into account both biological and social risk factors (Bright Futures Steering Committee & Medical Home Initiatives for Children with Special Needs Project Advisory Committee, 2006). For this reason, it has been recommended to use simple, accessible methodologies, with valid, reliable instruments through which specific indicators are obtained on the level achieved in areas of development and risks are detected in the general population, together with developmental disorders.

The literature describes screening tests that require specialized, trained human resources for their application, as well as specific materials and clinical spaces (Figueiras, de Souza, Ríos, & Benguigui, 2006). In Mexico, validated, reliable screening instruments are available to assess early childhood development, such as the Neurobehavioral Assessment of the Development of Infants (VANEDELA, with a sensitivity .79 - .85 and specificity of .71 - .84) (Ruiz, Vázquez, & Sánchez, 2013) and the Child Development Evaluation (EDI, with a sensitivity of .81 and specificity of .61) (Rizzoli-Córdoba et al., 2013). At the same time, other instruments have been developed that do not require prior training for health professionals as parents and/or caregivers are the ones who answer the questionnaires, which involves them by taking into account their observations and concerns about their child’s behavior (Glascoe, 2000), since the information explored everyday activities. This is an advantage since it translates into a reduction of both financial and human costs to achieve one of the objectives of early childhood development monitoring programs. The Ages and Stages Questionnaire, known by its acronym ASQ (Ages and Stages Questionnaires) (Squires, Bricker, & Potter, 1997), was designed in the late 1970´s in USA as a development screening instrument from first month of birth to 66 months. It has been modified over time and is now in its third edition. The ASQ-3 is the version that has been most widely studied and has been translated into various languages and is used in several countries (Charafeddine et al., 2013; Filgueiras, Pires, Maissonette, & Landeira-Fernandez, 2013; Vameghi et al., 2013; Velikonja et al., 2016). Various studies have reported the stability, consistency, and validity of the psychometric characteristics (Dionne, McKinnon, Squires, & Clifford, 2014; Kerstjens et al., 2009; San Antonio, Fenick, Shabanova, Leventhal, & Weitzman, 2014). Regarding the specificity and sensitivi-

ty of ASQ-3, it has been reported that it is an instrument capable of detecting developmental delays at early ages. Some authors report that the ASQ-3 is better at identifying children at high risk (92.3%) than those at low risk (60%) (Ga & Kwon, 2011; Juneja, Mohanty, Jain, & Ramji, 2012). The ASQ-3 has proved its effectiveness detecting delays in children with vulnerable conditions. One study reported that the systematic application of the ASQ-3 in US children in vulnerable conditions facilitated the detection of delays in all domains of development, particularly in areas that are less explored during routine medical examinations, such as problem-solving, the socio-individual area, and fine motor skills (Jee et al., 2010). The ASQ-3 application in Spanish in Argentinian children showed sensitivity indices of 88% and specificity indices of 94%, with positive predictive values of 88% and negative predictive values of 96%. This suggests that the Spanish version is valid for detection of developmental delays or inconsistencies (Romero et al., 2018).

The objective of the first phase of this study was to determine the internal consistency and inter-rater reliability of the Spanish version of the ASQ-3 in Mexican preschoolers. The aim is to continue studying the psychometric properties of this instrument, because it has been reported in the literature as a useful tool for monitoring early childhood development, which would provide opportunities for health and education systems in Mexico.

Method

Participants

Sample consisted of parents and/or guardians of children aged between 33 and 60 months who attended various Child Development Daycare Centers (CENDIS) in the public and private sector of Mexico City for four consecutive years, beginning with the oldest children in 2011 and ending with the youngest preschoolers in 2015. The sample size was set on the basis of criteria for studying the psychometric properties of instruments (Anastasi, 1981; Pick & López, 1994), which stipulate that at least five subjects per item are required for statistical analysis.

Procedure

After authorization had been obtained from the directors of the CENDIS, parents/guardians and teachers were invited to a briefing session at which the importance of evaluating and monitoring areas of development through reliable, valid screening instruments for Mexican children was explained; then they were asked to participate. Parents/guardians who agreed to participate signed a consent letter and complete the sociodemographic data card and the ASQ-3

according to child’s age. Children with identified medical neurological, and psychiatric conditions in sociodemographic data sheet, as well as those whose information was not answered in full were excluded. Inter-rater reliability was carried out with the teachers of each age group and 30 questionnaires were randomly selected. A brief written report of the ASQ-3 results was delivered to each of the participating parents (Figure 1).


view

Instruments

Sociodemographic data sheet: An ad hoc questionnaire was designed on the sociodemographic aspects of the child and his/her parents; as well as the child’s prenatal and postnatal health.

Ages and Stages Questionnaire (ASQ-3): This is a screening instrument that identifies developmental delays in children aged 1 to 66 months. It consists of 21 questionnaires for the age ranges of 2, 4, 6, 8, 9, 10, 12, 14, 16, 18, 20, 22, 24, 27, 30, 33, 36, 42, 48, 54, and 60 months. Each questionnaire contains 30 questions that evaluate five areas of development: problem solving; communication; fine motor development; gross motor development; and socio-individual skills. Three response options are provided: 1. the behavior occurs; 2. sometimes occurs, and 3. has not yet occurred. It has a compilation sheet, in which the scores of the five areas are recorded, and three levels of the total score are represented in graphs: 1. The target level that refers to a score above expectations means that the child’s development matches the time of the evaluation; 2. The gray level shows a score barely above expectations and corresponds to scores with -1 standard deviation of the mean (-1 SD) indicates that the child is still in process in one area and it is suggested that additional activities should be proposed to encourage him or her and monitor the process; and 3. The black level, whose score is below expectations, relates to scores with -2 SD with respect to the average, which is an indicator that an additional, more specific evaluation should be carried out. Another section of the ASQ-3, comprising general observations by the parents, refers to family history, concerns, and recent health problems of the child, which allows for additional complementary information. Lastly, there is a section for the follow-up of ASQ-3, in which the suggestion to undertake additional activities and specific medical evaluations is recorded. Time required for answering and grading the questionnaire is approximately 10 to 15 minutes. The third edition, published in 2009, has a Spanish version designed for a basic reading comprehension level. As mentioned earlier, this instrument has acceptable psychometric levels, with a sensitivity of 86.1% and specificity of 85.6% (Pomés, Squires, & Yovanoff, 2016; Squires & Bricker, 2009).

Ethical considerations

Project was registered and approved by the ethics and research commission of a hospital within federal health system. The objective of the study, confidential handling of data, the benefit they would obtain from a personalized written report of the results of the ASQ-3, and recommendations on parenting strategies were explained to parents/guardians. The principle of autonomy was preserved since the parents who agreed to participate voluntarily signed an informed consent form.

Statistical analysis

Sociodemographic variables were analyzed with summary measures, and scores of each of the ASQ-3 scales were analyzed using central tendency measures. Cronbach’s alpha coefficient was used to measure consistency and Pearson correlation coefficients were used to measure the inter-rater correlation level of total and each one of the questionnaire areas considering significant correlations with the p < .05* y p < .01** values. Confirmatory factorial analysis and the Varimax method were used to identify independence between the factors and evaluate validity of the latter. A discriminant analysis was carried out using the variables of age and area of development in order to determine the level of risk in each area by means of total scores for each of them. They were represented on a table of frequencies and percentages according to age range, areas, and three levels of development (achieved, in process, and requiring specialized evaluation), in order to observe population distribution. Mean and standard deviation of each area and age group were obtained, and children with scores of less than one and two standard deviations were grouped into probable cases. Version 19 of the SPSS program for Windows was used.

Results

The sample comprised parents and guardians (n = 1291), out of which 194 parents declined to participate and 45 questionnaires were eliminated because they were not completely filled out, yielding a total of n = 1052 questionnaires, divided into six age groups (33, 36, 42, 48, 54, and 60 months of age) of the ASQ-3. Of the total sample, 48% (n = 505) were boys and 52% girls (n = 547). The parents’ educational attainment was extremely heterogeneous, a high percentage only completed middle school (Table 1).


view

The reliability levels of the questionnaires for each age group were acceptable, ranging from α = .77 to α = .88. A confirmatory factorial analysis (CFA) was performed to evaluate construct validity, from which five factors were obtained through the Varimax method and Kaiser normalization. The percentages of explained variance ranged from 39.65% to 44.85%, which coincide with theoretical approach of the instrument. In order to under- take a discriminant analysis, the Wilks lambda statistic was obtained, which makes it possible to test the null hypothesis that the centroids of the groups are the same. The statistic obtained in all the tests was lower than the significance level (.05), from which it was concluded that the variables were adequate and significant for discriminating between age groups.

In the inter-rater reliability analysis, the areas having statistical significance are shown below:

• the area of communication for 33 months (p < .05), 54 months (p < .05), and 60 months (p < .01);

• the gross motor area for 36 months (p < .01) and 60 months (p < .01);

• the fine motor area for 33 months (p < .01), 36 months (p < .01), 48 months (p < .01), 54 months (p < .01), and 60 months (p < .05);

• the problem solving area for 33 months (p < .05) and 54 months (p < .05);

• the socio-individual area for 36 months (p < .01), 48 months (p < .05), and 60 months (p < .05).

Correlation values obtained were .426 < - < .85, indicating moderate to good association levels, with the exception of the group aged 42 months, in which there were no significant correlations in the areas evaluated (Table 2).


view

Characterization of the levels obtained

by ASQ-3 area and age group

In the characterization of levels obtained by each area of the ASQ-3 and age group, it was observed that over 70% of children were within the expectations for their age range (white area), except for the 48 and 54-month age groups, which were below 70% in the areas of fine motor skills and problem solving. Moreover, in the 48-month age group socio-individual area, less than 60% of the children were found in the white area. In the group of possible cases, we perceived that areas requiring more stimulation and specialized evaluation are the socio-individual area, problem solving, and fine motor skills, mainly in groups aged 42, 54, and 48 months. Figure 1 shows that approximately 30% of children evaluated failed to achieve expected level of development for their age in any area at the age of 42 months (Table 3 and Figure 2).


view

view

Discussion and conclusion

The results obtained suggest that theASQ-3 is a stable, consistent screening instrument for preschoolers at CENDIS in Mexico City. Levels of reliability for each age group ranged from α = .77 to α = .88, from which it may be inferred that the ASQ-3 possesses a level of accuracy acceptable (Kerlinger, Lee, Pineda, & Mora, 2002). The variation between reliability values reported in this study and those from the article by Hornman, Kerstjens, Winter, Bos, and Reijneveld (2013) can be explained by age variable. However, other authors, such as Vameghi et al. (2013) and Charafeddine et al. (2013), report this variability is due to translations and cultural adaptations of the instrument, as well as higher scores in each of the ASQ-3 areas in comparison with the original version in a US population (Vameghi et al., 2013; Velikonja et al., 2016).

Regarding inter-rater reliability (parents vs. teachers), moderate correlation levels were observed in this study. The areas with greatest correlation occurred at the ages of 54 and 60 months in areas of communication, fine motor and socio-individual skills, as well as gross motor skills for the age of 60 months and problem solving for age of 54 months. In the 42-month age group, no significant correlations were found between parents and teachers in areas of the instrument,which can be explained by findings of the study by Dionne et al. (2014). This study reports high rates of disagreement in areas of communication and problem solving, since it attributes these findings to discrepancy between parent’s expectations of their children development and those from the school system in kindergartens, and the difficulty parents may experience in classifying the frequency of a behavior in the way it is assessed by the ASQ-3 (always or sometimes) (Kerstjens et al., 2009).

According to Squires and Bricker (2009), Filgueiras et al. (2013), and Pomés et al. (2016), the ASQ-3 is easily applied and graded, making possible to distinguish between slight or serious delay problems in areas of development it evaluates.

Likewise, the ASQ-3 is a tool that can be used to periodically monitor early childhood development in pediatric medical care as indicated by the Secretaría de Salud (1999) for timely identification of developmental delays.

Is important to point out that in a developmental evaluation, when an area has not achieved the expected level according to child’s age, this represents a clinical risk and is an indicator that requires attention and stimulation. In this study, we found that areas of fine motor skills, socio-individual, and problem solving at the ages of 42, 48, and 54 months were located in the gray area (in process) or black area (below expectations), indicating need for a specialized clinical evaluation. In this regard, Wang et al. (2014) mentioned that finding two areas that require stimulation or specialized assessment are common underlying manifestations of neurodevelopmental deficits and should not be considered symptoms of separate disorders. Likewise, Houwen et al. (2016) reported that fine and gross motor development are strongly associated with cognition and therefore with language, which underlines the importance of early interventions that promote and stimulate all the areas involved.

Thus, we concluded that ASQ-3 scores are consistent in the age groups evaluated, which indicates the need to continue studying the internal and external validity of the instrument in order to be used in primary health care for development monitoring and timely detection of delays in Mexican children. One limitation of the study is that used the population of CENDIS in various parts of Mexico City and failed to consider other variables been associated with delays in early development, ranging from parent’s level of education and their socioeconomic status, including the mother’s health status during pregnancy, childbirth, postpartum, and the birth of the child ‒such as her weight and height‒. Future directions could meet the need to replicate this study in different populations, checking their psychometric properties and the association between health determinants and developmental delays. It is also essential to specify sensitivity and specificity indices in preschool children.

Funding

This study was funded by the authors themselves.

Conflict of interests

The authors declare they have no conflict of interests.

REFERENCES

Anastasi, A. (1981). Capacitación, avance de las pruebas y capacidades desarrolladas. American Psychologist, 36, 1086-1093.

Atalah, S. E., Cordero, V. M., Guerra, Z. M., Quezada, L. S., Carrasco, F. X., & Romo, M. M. (2014). Monitoreo de los indicadores del Programa “Chile Crece Contigo” 2008-2011. Revista chilena de pediatría, 85(5), 569-577. doi: 10.4067/S0370-41062014000500007

Bedregal, P. (2014). Hacia la renovación en las políticas de infancia en Chile. Revista Chilena de Pediatría, 85(1), 7-11. doi: 10.4067/S0370-41062014000100001

Bright Futures Steering Committee & Medical Home Initiatives for Children With Special Needs Project Advisory Committee. (2006). Identifying infants and young children with developmental disorders in the medical home: An algorithm for developmental surveillance and screening. Pediatrics, 118(1), 405-419. doi: 10.1542/peds.2006-1231

Charafeddine, L., Sinno, D., Ammous, F., Yassin, W., Al-Shaar, L., & Mikati, M. A. (2013). Ages and Stages Questionnaires: Adaptation to an Arabic speaking population and cultural sensitivity. European Journal of Paediatric Neurology, 17(5), 471-478. doi: 10.1016/j.ejpn.2013.03.001

Dionne, C., McKinnon, S., Squires, J., & Clifford, J. (2014). Developmental screening in a Canadian First Nation (Mohawk): psychometric properties and adaptations of ages & stages questionnaires. BMC Pediatrics, 14(1), 23. doi: 10.1186/1471-2431-14-23

Figueiras, A., de Souza, I. C. N., Ríos, V. G., & Benguigui, Y. (2006). Manual para la Vigilancia del Desarrollo Infantil en el contexto de AIEPI. Retrieved from: http://www1.paho.org/hq/dmdocuments/manual-vigilancia-desarrollo-infantil-aiepi-2011.pdf

Filgueiras, A., Pires, P., Maissonette, S., & Landeira-Fernandez, J. (2013). Psychometric properties of the Brazilian-adapted version of the Ages and Stages Questionnaire in public child daycare centers. Early human development, 89(8), 561-576. doi: 10.1016/j.earlhumdev.2013.02.005

Ga, H. Y., & Kwon, J. Y. (2011). A comparison of the Korean ages and stages questionnaires and Denver developmental delay screening test. Annals of Rehabilitation Medicine, 35(3), 369-374. doi: 10.5535/arm.2011.35.3.369

Glascoe, F. P. (2000). Evidence‐based approach to developmental and behavioral surveillance using parent’s concerns. Child: Care, Health and Development, 26(2), 137-149. doi: 10.1046/j.1365-2214.2000.00173.x

Heckman, J. (2011). Policies to foster human capital. Educational Studies, 3, 73-137. doi: 10.17323/1814-9545-2011-3-73-137

Heckman, J., Pinto, R., & Savelyev, P. (2013). Understanding the mechanisms through which an influential early childhood program boosted adult outcomes. The American Economic Review, 103(6), 2052-2086. doi: 10.1257/aer.103.6.2052

Hornman, J., Kerstjens, J. M., de Winter, A. F., Bos, A. F., & Reijneveld, S. A. (2013). Validity and internal consistency of the Ages and Stages Questionnaire 60-month version and the effect of three scoring methods. Early Human Development, 89(12), 1011-1015. doi: 10.1016/j.earlhumdev.2013.08.016

Houwen, S., Visser, L., van der Putten, A., & Vlaskamp, C. (2016). The interrelationships between motor, cognitive, and language development in children with and without intellectual and developmental disabilities. Research in Developmental Disabilities, 53, 19-31. doi: 10.1016/j.ridd.2016.01.012

Hutchson, T., & Nicoll, A. (1988). Developmental screening and surveillance. British journal of Hospital Medicine, 39(1), 22-9.

Jee, S. H., Szilagyi, M., Ovenshire, C., Norton, A., Conn, A. M., Blumkin, A., & Szilagyi, P. G. (2010). Improved detection of developmental delays among young children in foster care. Pediatrics, 125(2), 282-289.

Juneja, M., Mohanty, M., Jain, R., & Ramji, S. (2012). Ages and Stages Questionnaire as a screening tool for developmental delay in Indian children. Indian Pediatrics, 49(6), 457-461. doi: 10.1007/s13312-012-0074-9

Kerlinger, F. N., Lee, H. B., Pineda, L. E., & Mora, M. I. (2002). Investigación del comportamiento. México: Mc Graw-Hill.

Kerstjens, J. M., Bos, A. F., ten Vergert, E. M., de Meer, G., Butcher, P. R., & Reijneveld, S. A. (2009). Support for the global feasibility of the Ages and Stages Questionnaire as developmental screener. Early Human Development, 85(7), 443-447. doi: 10.1016/j.earlhumdev.2009.03.001

Narbona, J., E., & Schlumberger, E. (2008). Retraso Psicomotor. Asociación Española de Pediatría, 21, 152-157. Retrieved from: https://www.aeped.es/sites/default/files/documentos/21-retraso.pdf

Organización Mundial de la Salud. (2001). Clasificación internacional del funcionamiento de la discapacidad y de la salud. Informe de la Secretaría. Retrieved from: http://apps.who.int/iris/bitstream/10665/82187/1/sa5418.pdf

Pick, S., & López, A. (1994). Cómo investigar en ciencias sociales. México: Trillas.

Piek, J. P., Dawson, L., Smith, L. M., & Gasson, N. (2008). The role of early fine and gross motor development on later motor and cognitive ability. Human Movement Science, 27(5), 668-681. doi: 10.1016/j.humov.2007.11.002

Pomés, M., Squires, J., & Yovanoff, P. (2016). Psychometric examination of a Spanish translation of a developmental screening instrument. Journal of Early Childhood Research, 14(2), 132-145. doi: 10.1177/1476718X14529279

Rizzoli-Córdoba, A., Schnaas-Arrieta, L., Liendo-Vallejos, S., Buenrostro-Márquez, G., Romo-Pardo, B., Carreón-García, J., …Robles-Anaya, R. (2013). Validación de un instrumento para la detección oportuna de problemas de desarrollo en menores de 5 años en México. Boletín Médico del Hospital Infantil de México, 70(3), 195-208.

Romero, A. O., Grañana, N., Gaeto, N., Torres, M. L. Á., Zamblera, M. N., Vasconez, M. A., ... & Squires, J. (2018). ASQ-3: Validation of the Ages and Stages Questionnaire for the detection of neurodevelopmental disorders in Argentine children. Archivos Argentinos de Pediatría, 116(1), 7-13.

Ruiz, G. A. A., Vázquez, R. I. M., & Sánchez, C. (2013). Modelo de vigilancia y seguimiento del neurodesarrollo infantil: experiencia en la Clínica de Medicina Familiar Tlalpan. Revista de Especialidades Médico-Quirúrgicas, 18(1), 19-30.

San Antonio, M. C., Fenick, A. M., Shabanova, V., Leventhal, J. M., & Weitzman, C. C. (2014). Developmental screening using the Ages and Stages Questionnaire: standardized versus real-world conditions. Infants & Young Children, 27(2), 111-119. doi: 10.1097/IYC.0000000000000005

Secretaría de Salud. (1999). Norma Oficial Mexicana para la Atención a la Salud del Niño. México, D.F. Retrieved from: http://www.ucol.mx/content/cms/13/file/NOM/NOM_031_SSA2.pdf

Squires, J. & Bricker, D. (2009). Ages and Stages Questionnaires: A Parent-Completed, Child-Monitoring System (3rd ed.) Baltimore: Paul H. Brookes.

Squires, J., Bricker, D., & Potter, L. (1997). Revision of a parent-completed developmental screening tool: Ages and Stages Questionnaires. Journal of Pediatric Psychology, 22(3), 313-328. doi: 10.1093/jpepsy/22.3.313

Vameghi, R., Sajedi, F., Kraskian, M. A., Habiollahi, A., Lornezhad, H. R., & Delavar., B. (2013). Cross-Cultural Adaptation, Validation and Standardization of Ages and Stages Questionnaire (ASQ) in Iranian Children Iranian. Journal of Public Health, 42(5), 522-528.

Velikonja, T., Edbrooke‐Childs, J., Calderon, A., Sleed, M., Brown, A., & Deighton, J. (2016). The psychometric properties of the Ages & Stages Questionnaires for ages 2‐2.5: a systematic review. Child: Care, Health and Development, 43(1), 1-17. doi: 10.1111/cch.12397

Walker, S. P., Wachs, T. D., Grantham-McGregor, S., Black, M. M., Nelson, C. A., Huffman, S. L., … Gardner, J. M. M. (2011). Inequality in early childhood: risk and protective factors for early child development. The Lancet, 378(9799), 1325-1338. doi: 10.1016/S0140-6736(11)60555-2

Wang, M. V., Lekhal, R., Aarø, L. E., & Schjølberg, S. (2014). Co‐occurring development of early childhood communication and motor skills: results from a population‐based longitudinal study. Child: Care, Health and Development, 40(1), 77-84. doi: 10.1111/cch.12003