Salud mental 2018;

ISSN: 0185-3325

DOI: 10.17711/SM.0185-3325.2018.034

Received: 3 May 2018 Accepted: 24 September 2018

Construct validity of a new scale for assessing anger proneness (APS-G)

Cirilo H. García-Cadena 1 , Leopoldo Daniel-González 1 , Adrián Valle de la O 2 , Tomás Caycho-Rodriguez 3 , Arnoldo Téllez López 1


1 Facultad de Psicología, Universidad Autónoma de Nuevo León, Monterrey, Nuevo León, México.

2 Escuela de Medicina y Ciencias de la Salud, Instituto Tecnológico y de Estudios Superiores de Monterrey, Monterrey, Nuevo León, México.

3 Facultad de Psicología, Universidad Privada del Norte, Breña, Perú.

Correspondence: Leopoldo Daniel González Facultad de Psicología, Universidad Autónoma de Nuevo León. Dr. Carlos Canseco 110, Col. Mitras Centro, C.P. 64460, Monterrey, Nuevo León, México. Phone: (81) 8333 - 8233 Email: leopoldo.danielgn@uanl.edu.mx


Abstract:
Introduction. The negative effects of anger on health highlight the value of developing short, reliable, and valid instruments that allow its assessment whether for the purpose of research, clinical diagnosis, and/or evaluation of interventions aimed at reducing it effectively.
Objective. The primary aim of this research was to determine the construct, convergent, and discriminant validity, as well as the factorial invariance, of a new scale to measure anger proneness.
Method. A confirmatory factorial analysis was used.
Results. An online nonprobability sample composed of 457 participants (35.2% men and 64.8% women), with a mean age of 36.87 years (SD = 12.513) was recruited. The unidimensional model of the García’s Brief Scale for Assessing Anger Proneness (APS-G scale) shows a good data fit (df = 2; χ2 = 5.515; χ2/df = 2.575; CFI = .993; GFI = .996; RMSEA = .051; SRMR = .0193). There are factorial, configural, metric, unrestricted, strong, and strict factorial invariances between men and women. Likewise, a positive correlation coefficient exists between the APS-G scale and the State-Trait Anger Expression Inventory-2 (STAXI-2) (r = .561; p < .01); on the other hand, there is a negative correlation coefficient between the studied scale and the Satisfaction with Life Scale (SWLS) (r = -.179; p < .01).
Discussion and conclusion. A new reliable and valid instrument to measure anger proneness has been created, and its use is proposed for research and screening purposes with Spanish-speaking population.

Key words: Anger, personality tests, psychometrics, mental health.

Resumen:
Introducción. Los efectos negativos de la ira sobre la salud apuntan a la importancia de desarrollar instrumentos cortos, confiables y válidos que permitan su evaluación, ya sea con propósitos de investigación, diagnóstico clínico y/o evaluaciones de intervenciones que la reduzcan de manera efectiva.
Objetivo. El objetivo primario fue determinar la validez de constructo, convergente y discriminante, así como la invarianza factorial, de una nueva escala para medir la disposición a la ira.
Método. Se usó un análisis factorial confirmatorio para evaluar la estructura factorial subyacente y para identificar la validez de constructo del modelo de medición.
Resultados. Se reclutó una muestra en línea no probabilística compuesta por 457 participantes (35.2% hombres y 64.8% mujeres), con una edad promedio de 36.87 años (DE = 12.513). El modelo de medición muestra una buena bondad de ajuste para la escala unidimensional EPI-G (gl = 2; χ2 = 5.515; χ2/gl = 2.575; CFI = .993; GFI = .996; RMSEA = .051; SRMR = .0193). Se encuentra invarianza factorial configuracional, métrica, fuerte y estricta entre hombres y mujeres, además de una correlación positiva entre la escala propuesta y la Expresión Externa de la Ira del STAXI-2 (r = .561; p < .01) y negativa con la escala de Satisfacción con la Vida (SWLS) (r = -.179; p < .01).
Discusión y conclusión. Un nuevo instrumento para medir la disposición a la ira fue creado para propósitos de investigación y tamizaje, sobre todo en población hispanohablante.

Palabras clave: Ira, test de personalidad, psicometría, salud mental.




Introduction

Anger is a very important negative emotion, and nowadays it is considered an indicator of mental health, along with depressive mood and anxiety symptoms, within the orthodox psychopathological model (Nadimpalli, Kanaya, McDade, & Kandula, 2016). The concept of anger proneness refers to a relatively enduring and transituational personality trait. On the other hand, state anger refers to a momentary and short-lasting outburst of anger; its intensity might be mild, moderate or, sometimes, severe, and it can be triggered by detriment or harm (being real or imagined), which in turn can cause an impediment to achieve a goal (Spielberger & Reheiser, 2009).

The interbehavioral interpretation of anger (Kantor, 1959, 1969; Ribes-Iñesta, 2007) implies that some persons tend to experience anger proneness under certain specific circumstances which would not cause such a feeling in the case of other people. For instance, some individuals may experience anger proneness when they feel that someone else is asserting the opposite of what they have just previously said or when things do not go the way that such individuals expected. In this emotional behavior, certain environmental conditions (including some specific aspects of each individual) give rise to a discrete episode of anger, and the source of such circumstances is to be found in the reactive biography of the individual.

In terms of form, anger is expressed in a wide variety of behaviors. One of the most important characteristics of anger is the disorganized and chaotic behavior that does not end, as most psychological activities do, in a consummatory response. Precisely because of the absence of a consummatory response, the emotional behavior related to anger is considered as a behavioral segment without an answer. The behavioral segment of anger is clearly distinguished from other kind of behavioral segments because the latter allow a fluid exchange between the individual and their surroundings since there are stimulus and response functions harmoniously linked in an appropriate environment; nevertheless, this does not happen in the irascible behavioral segment (Kantor, 1959, 1969; Ribes-Iñesta, 2007). A high level of anger is a risk factor for the development of heart diseases and stroke (Kitayama et al., 2015; Mostofsky, Penner, & Mittleman, 2014), and hypertension (Ford et al., 2016; May, Sanchez-Gonzalez, Hawkins, Batchelor, & Fincham, 2014).

Recent studies have also linked anger with several psychiatric disorders, such as depression, neuroticism, attention deficit, and hyperactivity disorder, and problems with the consumption of alcoholic beverages (Lubke, Ouwens, deMoor, Trull, & Boomsma, 2015; Morrison, Noel, & Ogle, 2012) and alcohol-related aggression (Giancola, 2002). Likewise, other researchers have also found strong links between anger and psychotic disorder, drug dependence, bipolar disorder, and schizotypal and borderline personality disorders (Okuda et al., 2015).

Therefore, the important negative effects of anger on health highlight the value of developing short, reliable, and valid instruments that allow its assessment whether for the purpose of research, clinical diagnosis, and/or evaluation of interventions aimed at reducing it effectively. Thus, in view of the scarce number of instruments available for its use in the Spanish-speaking population, the aim of this study has been the development of an instrument for the measurement of anger proneness and the assessment of its validity (of construct, convergent, and discriminant, and its reliability) so that it can be used for research and, probably, clinical purposes.

The state-Trait Anger Expression Inventory-2 (STAXI-2) is a widely used instrument to assess anger proneness. There are several studies that have tried to adapt and corroborate the factor structure of STAXI-2 (Spielberger, 1999) and to determine its internal consistency in different Latin samples from Mexico (Alcázar, Deffenbacher, & Byrne, 2011; Moral, González, & Landero, 2010; Moral & Segovia, 2015; Moral, Ramos-Basurto, & Segovia-Chávez, 2015; Oliva, Hernández, & Calleja, 2010). The evidence to support its use has been contradictory in the Mexican studies. Those contradictory findings were to be expected if one considers the ethnocentric premise that posits that psychological traits, and the instruments created to assess them, are necessarily universal.

Mixed results have been found with respect to gender differences in expression of anger. Among non-clinical population, Cubito and Brandon (2000) found that adult women showed higher anger scores than adult men; nevertheless, some researchers have found no differences between men and women with regards to the feeling of anger, although women tend to express it more frequently (Averill, 1983; Evers, Fisher, Rodriguez-Mosquera, & Manstead, 2005). In the case of boys and girls, some studies have not found differences in the level of anger between both genders (Burt, 2014; Pullen et al., 2015), but other studies have found that female adolescents experienced a higher level of anger than male adolescents (Dey, Rahman, Bairagi, & Roy, 2014; Spielberger & Reheiser, 2009).

According to the results shown by this literature, a scale with factorial invariance is needed. Thus, the aims of this research were: a) to develop a valid and reliable instrument to measure anger proneness and b) to find out if García’s Anger-Proneness Scale can measure anger regardless of gender.

Method

Study design

A cross-sectional design has been used.

Participants

A sample of volunteers from Monterrey, Mexico, was recruited through an online, non-probability sampling method.

Measures

The instrument includes socio-demographic questions (age and sex) and the following scales:

a) García’s Anger Proneness Scale (APS-G).This scale, which was created for this study, comprises four items which are evaluated along an agreement, four-point Likert-type scale (4 = Yes; 3 = I think so; 2 = I do not think so; and 1 = No). The sum of the four items yields a total score such that a higher score means greater anger proneness.

b) Sub-scale of External Expression of Anger from STAXI-2 (Spielberger, 1999). It is a self-report scale composed of six items which are evaluated along a four-point Likert-type scale (from 4 = almost always to 1 = almost never). We used the Spanish version elaborated by Miguel-Tobal, Casado, Can-Vindel, and Spielberger (2001). The scale has been validated in general population from Monterrey, Mexico, by Moral and Ramos (2015). This instrument was used to estimate convergent validity of APS-G. In the present study, STAXI-2 showed a one-factor structure and showed good indices of internal consistency (α = .751 and ω = .753).

c) Satisfaction with Life Scale (SWLS) (Diener, Emmons, Larsen, & Griffin, 1985). It is a self-report scale composed of five items which are evaluated along a six-point Likert-type scale (from 6 = disagree strongly to 1 = agree strongly). We used the Spanish version elaborated by Atienza, Pons, Balaguer, & García-Merita (2000). The scale has been validated among Mexican university students by Padrós, Gutiérrez, and Medina (2015). This instrument was used to estimate the discriminant validity of APS-G since it has been previously shown that there exists a negative relationship between anger and life satisfaction (Kerhervé, Gay, & Vrignaud, 2008; Schimmack, Oishi, Fuur, & Funder, 2004). In the present study, this scale showed a one-factor structure and good indices of internal consistency (α = .834 y ω = .835).

Procedure

First, an initial set composed of eight items was created, which were evaluated along a four-point Likert-type scale (4 = Yes; 3 = I think so; 2 = I do not think so; and 1 = No). This instrument was elaborated considering the interbehavioral interpretation of anger, specifically the interactive style of personality construct (Ribes-Iñesta, 2009). The questionnaire was administered at the university classrooms and online. In both cases, it was clarified that the response to the questionnaire would be anonymous and confidential.

Data analysis

Measures of central tendency were calculated for the variable age and, in the case of categorical variable gender, the corresponding percentages were calculated. To determine the final version of García’s Anger-Proneness Scale (APS-G) the following criteria should have to be met:

a) The factor loadings for each one of the items should be equal to or greater than .40 (λ ≥ .40) (Williams, Onsman, & Brown, 2010).

b) To be accepted as a component of APS-G, any factor would have to show a Cronbach’s alpha higher than .70 (Nunnally & Bernstein, 1994).

Before performing a factor analysis, the Kaiser-Meyer-Olkin (KMO) test for sampling adequacy was assessed; a value higher than .60 was considered adequate. Likewise, it is necessary to reject the null hypothesis of equivalence of the correlation matrix to an identity matrix through the Bartlett’s test of sphericity; small values (less than .05) of the significance level indicate that factor analysis may be performed. To identify the factorial structure of APS-G, confirmatory factor analysis (CFA) was performed using unweighted least squares and maximum likelihood as extraction method, respectively. The assumption of multivariate normality was assessed through Mardia’s multivariate kurtosis coefficient; this assumption is fulfilled if its value is lower than 70 (Rodríguez & Ruiz, 2008). The goodness of fit of the model was assessed through the estimation of the following goodness of fit indices: chi-square, Goodness of Fit Index (GFI), Comparative Fit Index (CFI), Root Mean Square Error of Approximation (RMSEA), Standardized Root Mean Square Residual (SRMR), and the relative chi-square (χ2/df); values between two and three, or up to a difference ≤ 5, indicated a good fit of the model. Overall, GFI, and CFI values > .95, as well as RMSEA and SRMR < .08, are considered as indicators of a goodness of fit from good to adequate between the hypothetical model and the data (Hu & Bentler, 1999). In addition, to check factorial equivalence, the Tucker-Lewis Index (TLI), which must be higher than .95 (Ten-Berge, 1986), is considered as the first criterion.

The factorial invariance between men and women was estimated (Byrne, 2008). Internal evidence for convergent validity was obtained by calculating the average variance extracted (AVE) from the factor; AVE > .50 was considered as satisfactory (Fornell & Larcker, 1981). In addition, the evidence for convergent and discriminant validity was obtained by calculating the Pearson´s correlation coefficient between the APS-G score and other measures that are theoretically related to anger proneness, such as the external expression of anger and the satisfaction with life measured through the STAXI-2 and the SWLS. The effect size of the correlations was calculated (r ≥ .20: minimum, ≥ .50: moderate; ≥ .80: strong) and their CIs. The reliability of APS-G scores was estimated through Cronbach’s alpha coefficient (Cronbach, 1951) and McDonald’s omega coefficient (McDonald, 1999).

Finally, the percentage of participants both male and female, that expressed a significant anger proneness, was calculated, and these were defined as those people scoring in the highest third of the distribution (Patrick & Iacono, 1989). Statistical analyses were performed through SPSS and AMOS 24.

Ethical considerations

The names of those responsible for the study and its electronic address were provided so that the participants could request information or support in relation to any question raised by the study. No identification data were asked for to the participants to assure their anonymity, and the confidentiality of the information provided through this questionnaire was guaranteed.

Results

A sample composed of 457 volunteer participants from the general adult population was recruited: 161 women (35.2%) and 296 men (64.8%). The mean age of the whole sample was 36.87 years (range: 18 – 65 years old; SD = 12.51); the mean age of the subsample composed of women was 36.33 years (SD = 12.07) and the mean age of the subsample composed of men was 37.86 years (SD = 13.28). No significant and important differences were found between the mean age of men and women (t234 = 1.249; p = .001; d = .12; 95% CI [-.07, -.31]). Table 1 shows the descriptive statistics of each one of the items composing APS-G. The assumption of multivariate normality was fulfilled (Mardia’s coefficient = .684), thus proving that there will be no problem to continue with the successive statistical analyzes.


view

The Kaiser-Meyer-Olkin (KMO) value was .710 and Bartlett’s test of sphericity was significant (χ2 = 382.642; df = 6; p = .001), and then it was proceeded to perform directly CFA (Suhr, 2006).

Confirmatory factor analysis (CFA)

Confirmatory factor analysis (CFA) was used to estimate the unidimensionality of the construct. The model showed good goodness of fit indices (χ2 = 5.515, df = 2, χ2/df = 2.575, CFI = .993, GFI = .996, RMSEA = .051, SRMR = .019); three out five of these indices were greater for APS-G than for STAXI-2 (GFI = .988, CFI = .985, SRMR = .027). The analysis of standardized factorial loadings (λ) for the one-factor model (item 1 = .69, item 2 = .69, item 3 = .58, item 4 = .58), which are shown in Figure 1 , have the expected direction and an average λ higher than the value recommended by several authors (Williams et al., 2010). Likewise, the one-factor model showed internal evidence of convergent validity (AVE = .62). The four items composing APS-G did not show high correlations (r > .90); thus, it is possible to say there is no multicollinearity (Tabachnick, Fidell, & Osterlind, 2001). These data provide evidence of internal discriminant validity (Merino-Soto, Domínguez-Lara, & Fernández-Arata, 2017).


view

Considering the results shown on Table 2 , the CFA of the one-factor structure for the sample composed of men showed appropriate goodness of fit indices (GFI = .990 and RMSEA = .067). Likewise, the measures of incremental adjustment and parsimony were higher than the parameters of the independent model and were very like the parameters found with the saturated model. On the other hand, the goodness off it indices for the sample composed of women were also favorable (GFI = .993 and RMSEA = .068) and, as in the sample composed of men, the incremental adjustment and parsimony indices for the one-factor model were higher than those for the independent model, and were very similar to the parameters found with the saturated model. In both samples the absolute, incremental, and parsimony indices were adequate.


view

The factorial loadings (λ´s) of each item of APS-G are adequate, both in the sample composed of men (item 1 = .737, item 2 = .742, item 3 = .560, item 4 = .582), as in the sample composed of women (item 1 = .649, item 2 = .663, item 3 = .606, item 4 = .590).

Factorial invariance between men and women

Initially, the value obtained from TLI = .980 offered sufficient evidence to continue the analysis of factorial invariance. Thus, Table 3 shows the calculation of the unconstrained model (M1); the fit indices obtained in this model (χ2 = 9.063, df = 4; χ2/df = 2.266; RMSEA = .048; SRMR= .0302) were adequate (Hu & Bentler, 1999).

The values obtained for the analysis of Configural Invariance (M2), SRMR = .0363, and RMSEA = .027 provided additional evidence to the invariance model. In addition, the value of the Akaike Information Criterion (AICM1-M2 = 5.162) and CFI did not show significant changes in models M1 and M2. Likewise, after applying the criterion of Cheung and Rensvold (2002) for nested models, it was found that the difference between CFIs obtained in the models M1 and M2 was lower than .01 (CFIM1-M2 = .006). Thus, it was possible to accept the Configural Invariance model.

The next model ( Table 3 ) was the Metric Invariance Model (M3), considering the Strong Invariance Model (M4 - intercept values). The values of SRMR = .0366 and RMSEA = .024, in addition to the difference between the CFIs (CFIM3-M4 = .004) and AICs (AICM3-M4 = 2.712), allowed to accept the Metric Invariance model. The next step was to assess the equivalence between M2 and M4. The Strong Invariance Model showed an adequate fit ( Table 3 ) regarding its nesting with M2. The difference found between CFIs was .005; SRMR = .0487 and RMSEA = .024. Therefore, the Strong Invariance Model was accepted, thus concluding the equivalence with respect to the factorial coefficients and the intercept values.


view

Finally, we assessed the Strict Invariance Model (M5). The values shown in Table 3 supported M5. The differences found between CFIs and AICs of M5 and M4 (CFIM5-M4 = .003; AICM4-M5 = 1.98), in addition to the value of RMSEA = .021 and SRMR = .0509, supported M5. Thus, these results confirmed that APS-G has a factorial structure that fits to the proposed invariance models ( Table 3 ).

Comparison of means between men and women

Among the 467 participants, 98 out of 161 male participants (61.5%) and 168 out of 296 female participants (57.1%) obtained a score in the highest third of the distribution. The means between men (M = 10.91; SD = 3.10) and women (M = 11.01; SD = 3.07) were compared. The results showed the absence of significant differences in anger proneness (t455 = -.331; p > .001; d = .03). Practical significance is null (Cohen, 1992).

Convergent and discriminant validity

Correlations between APS-G and STAXI-2 (r = .561; p < .01; 95% CI [.49, .77]), and between APS-G and SWLS (r = -.179; p < .01; 95% CI [.09, .42]) showed the expected direction and were significant. Correlation between APS-G and STAXI-2 showed a medium effect size (r ≥ .50); correlation between APS-G and SWLS showed a trivial effect size (r < .20) (Ferguson, 2009).

Reliability of APS-G

The reliability of APS-G was assessed through alpha (α = .731; 95% CI [.69, .77]) and omega (ω = .732). Alpha for the sample of men was α = .75 (95% CI [.68, .81]) and omega was ω = .753; for the sample of women, alpha was α = .722 (95% CI [.66, .77]) and omega was ω = .722. The values of these coefficients for the whole sample as well as for the men and women samples supported APS-G reliability.

Discussion and conclusion

The primary objective was to analyze construct validity of a brief scale to measure anger proneness. According to CFA, a unifactorial structure was found. APS-G also showed adequate reliability, appropriate goodness of fit, and factorial invariance. Following the methodological recommendation of Campbell and Russo (2001), the use of STAXI-2 and APS-G and the fact of having a moderate correlation between both scales validate the existence of APS-G. Thus, it is possible to assert that APS-G is as good as STAXI-2 in Ibero-American countries. With reference to discriminant validity, SWLS was used because having a satisfactory evaluative conception of one’s life would be in contradiction with the disposition to experience anger; indeed, a negative correlation was found between SWLS and anger proneness, providing validity to APS-G.

Ribes-Iñesta (2009) developed a model to explain personality traits and created the psychological construct known as interactive personality style. García-Cadena, Téllez-López, Ramírez-Aguillón, Ramírez-Hernández, and Pérez-Cota (2016) used it for developing instruments aimed at measuring dispositional optimism, generosity, and forgiveness. Using an interbehavioral interpretation, the four items composing APS-G can be classified into two categories. Items 1 and 2 refer to a generalized transituational disposition to get angry ( Table 1 ). Items 3 and 4 refer to getting angry when any person or situation hinders the achievement of what is wanted. The multiple historical interactions that have been differentially accumulated within a subject´s reactive biography explain why some persons get angry under some circumstances while other individuals show an impressive immutability when they are confronted with the same circumstances.

Several groups of researchers (Shaver, Schwartz, Kirson, & O’Connor, 1987) have proposed that anger is an emotion composed of eight factors, namely: 1. antecedents, 2. cognitive reactions, 3. non-verbal expressions, 4. verbal expressions, 5. body sensations, 6. interpersonal responses, 7. primary self-control, and 8. secondary self-control. Alonso-Arbiol et al. (2011) based their research on that study and created a six-item scale (α = .61) for assessing the factor antecedents, four of which have great similarity with the four items composing APS-G.

This study is the first one that analizes the factorial invariance of a brief scale to assess anger pronenness. Configural, metric, unrestricted, strong, and strict factorial in variances were maintained when comparing men and women, which is a prerequisite for any comparison between groups (Byrne, 2008). This suggests that items composing APS-G measure the same latent variable (Brown, 2006), and data also support the unidimensional structure of APS-G and factorial invariance.

However, a limitation is that the findings support the use of APS-G for research and screening purposes, but not for decision-making in a clinical setting (DeVellis, 2003). Likewise, this study covers only one of the eight components of anger. A final limitation refers to the absence of sensitivity and specificity computations of APS-G. Nevertheless, following the recommendation of Patrick and Iacono (1989), those participants scoring in the highest third of the distribution could be considered as persons possesing the trait of anger proneness.

Therefore, to measure anger proneness with a lower margin of error, it is pertinent to continue investigating other instruments composed of a higher number of items that might contribute to making more informed judgments (Nunnally & Bernstein, 1994). Finally, the conclusions are provisional, until other studies are carried out in similar cultural groups, which would allow adding evidence in favor or against APS-G.

Funding

None.

Conflict of interests

The authors declare no conflict of interests.

REFERENCES

Alcázar, R. J., Deffenbacher, J. L., & Byrne, Z. S. (2011). Assessing the Factor Structure of the Anger Expression Inventory (MLSTAXI) in a Mexican Sample. International Journal of Psychology and Psychological Therapy, 11(2), 307-318.

Alonso-Arbiol, I., van de Vijver, F. J. R., Fernandez, I., Paez, D., Campos, M., & Carrera, P. (2011). Implicit theories about interrelations of anger components in 25 countries. Emotion, 11(1), 1-11. doi: 10.1037/a0020295

Atienza, F., Pons, D., Balaguer, I., & García-Merita, M. (2000). Propiedades psicométricas de la escala de satisfacción con la vida en adolescentes. Psicothema, 12(2), 314-319.

Averill, J. A. (1983). Studies on anger and aggression. American Psychologist, 38(11), 1145-1160. doi: 10.1037/0003-066X.38.11.1145

Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York: Guilford.

Burt, I. (2014). Identifying gender differences in male and female anger among an adolescent population. The Professional Counselor, 4(5), 531-540.

Byrne, B. M. (2008). Testing for multigroup equivalence of a measuring instrument: A walk through the process. Psicothema, 20(4), 872-882.

Campbell, D. T. & Russo, M. J. (2001). Social measurement. California: SAGE.

Cheung, G. W. & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A multidisciplinary journal, 9(2), 233-255. doi: 10.1207/S15328007SEM0902_5

Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159. doi: 10.1037/0033-2909.112.1.155

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of test. Psychometrika, 16(3), 297-334.

Cubito, D. S. & Brandon, K. O. (2000). Psychological adjustment in adult adoptees: assessment of distress, depression, and anger. American Journal of Orthopsychiatry, 70(3), 408-413. doi: 10.1037/h0087856

DeVellis, R. F. (2003). Scale development. Theory and applications. Thousand Oaks: Sage.

Dey, B. K., Rahman, A., Bairagi, A., & Roy, K. (2014). Stress and anger of rural and urban adolescents. Psychology, 5(3), 177-184. doi: 10.4236/psych.2014.53028

Diener, E., Emmons, R., Larsen, R., & Griffin, S. (1985). The satisfaction with life scale. Journal of Personality Assessment, 49(1), 71-75. doi: 10.1207/s15327752jpa4901_13

Evers, C., Fischer, A. H., Rodriguez-Mosquera, P. M., & Manstead, A. S. R. (2005). Anger and social appraisal: A “spicy” sex difference?. Emotion, 5(3), 258-266. doi: 10.1037/1528-3542.5.3.258

Ferguson, C. J. (2009). An effect size primer: A guide for clinicians and researchers. Professional Psychology Research and Practice, 40(5), 532-538. doi: 10.1037/a0015808

Ford, C. D., Sims, M., Higginbotham, J. C., Crowther, M. R., Wyatt, S. B., Musani, S. K., … Parton, J. M. (2016). Psychosocial Factors Are Associated With Blood Pressure Progression Among African Americans in the Jackson Heart Study. American Journal of Hypertension, 29(8), 913-924. doi: 10.1093/ajh/hpw013

Fornell, C. & Larcker, D. F. (1981). Structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39-50. doi: 10.2307/3151312

García-Cadena, C. H., Téllez-López, A., Ramírez-Aguillón, G., Ramírez-Hernández, E., & Pérez-Cota, E. J. (2016). Toward a global conceptualization and measurement in positive Psychology. In R. Bowers (Ed.), Psychological Well-Being. Cultural Influences, Measurement Strategies and Health Implications (pp. 21-42). New York: Nova Science Publishers.

Giancola, P. R. (2002). The influence of trait anger on the alcohol-aggression relation in men and women. Alcoholism: Clinical and Experimental Research, 26(9), 1350-1358.

Hu, L. & Bentler, P. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A multidisciplinary journal, 6(1), 1-55. doi: 10.1080/10705519909540118

Kantor, J. R. (1959). Interbehavioral Psychology: A Sample of Scientific Science Construction. Bloomington: Principia.

Kantor, J. R. (1969). Principles of Psychology. Bloomington: Principia.

Kerhervé, H., Gay, M. C., & Vrignaud, P. (2008). Santé psychique et fardeau des aidants familiaux de personnes atteintes de la maladie d’Alzheimer ou de troubles apparentés. Annales Médico-Psychologiques, 166(4), 251-259. doi: 10.1016/j.amp.2008.01.015

Kitayama, S., Park, J., Boylan, J. M., Miyamoto, Y., Levine, C. S., Markus, H. R., … Ryff, C. (2015). Expression of anger and ill health in two cultures: an examination of inflammation and cardiovascular risk. Psychological Science, 26(2), 211-220. doi: 10.1177/0956797614561268.

Lubke, G. H., Ouwens, K. G., deMoor, M. H., Trull, T. J., & Boomsma, D.I. (2015).Population heterogeneity of trait anger and differential associations of trait anger facets with borderline personality features, neuroticism, depression, Attention Deficit Hyperactivity Disorder (ADHD), and alcohol problems. Psychiatry Research, 230(2), 553-560. doi: 10.1016/j.psychres.2015.10.003

May, R. W., Sanchez-Gonzalez, M. A., Hawkins, K. A., Batchelor, W. B., & Fincham, F. D. (2014). Effect of anger and trait forgiveness on cardiovascular risk in young adult females. American Journal of Cardiology, 114(1), 47-52. doi: 10.1016/j.amjcard.2014.04.007

McDonald, R. P. (1999). Test Theory: A Unified Treatment. New Jersey: Routledge.

Merino-Soto, C., Domínguez-Lara, S., & Fernández-Arata, M. (2017). Validación inicial de una Escala Breve de Satisfacción con los Estudios en estudiantes universitarios de Lima. Educación Médica, 18(1), 74-77. doi: 10.1016/j.edumed.2016.06.016

Miguel-Tobal, J., Casado, M., Can-Vindel, A., & Spielberger, C. (2001). Inventario de Expresión de la Ira Estado-Rasgo STAXI-2. Madrid: TEA.

Moral, J. & Ramos, S. (2015). Estructura factorial y consistencia interna de los ítems de expresión de ira del STAXI-2 en una muestra de población general de la ciudad de Monterrey, México. PSICUMEX, 5(2), 20-39.

Moral, J. & Segovia, M. P. (2015). Propiedades psicométricas de la escala de expresión de la ira en mujeres con VIH. Revista Ciencia UANL, 18(74), 69-74.

Moral, J., González, M., & Landero, R. (2010). Factor structure of the STAXI and its relationship to the burnout in housewives. The Spanish Journal of Psychology, 13(1), 411-423.

Moral, J., Ramos-Basurto, S., & Segovia-Chávez, M. P. (2015). Validación cruzada de la escala de expresión de ira en mujeres con VIH y población general. Revista Iberoamericana de Psicología y Salud, 6(2), 81-89.

Morrison, P. M., Noel, N. E., & Ogle, R. L. (2012). Do angry women choose alcohol?.Addictive Behaviors, 37(8), 908-913. doi: 10.1016/j.addbeh.2012.03.018

Mostofsky, E., Penner, E. A., & Mittleman, M. A. (2014). Outbursts of anger as a trigger of acute cardiovascular events: a systematic review and meta-analysis. European Heart Journal, 35(21), 1404-1410. doi: 10.1093/eurheartj/ehu033

Nadimpalli, S. B., Kanaya, A. M., McDade, T. W., & Kandula, N. R. (2016). Self-Reported Discrimination and Mental Health Among Asian Indians: Cultural Beliefs and Coping Style as Moderators. Asian American Journal of Psychology, 7(3), 185-194. doi: 10.1037/aap0000037

Nunnally, J. C. & Bernstein, I. H. (1994). Psychometric Theory. New York: McGraw-Hill.

Okuda, M., Picazo, J., Olfson, M., Hasin, D. S., Liu, S. M., Bernardi, S., & Blanco, C. (2015). Prevalence and correlates of anger in the community: results from a national survey. CNS Spectrums, 20(2), 130-139. doi: 10.1017/S1092852914000182

Oliva, F. J., Hernández, M. R., & Calleja, N. (2010). Validación de la versión mexicana del Inventario de Expresión de Ira Estado-Rasgo (STAXI-2). Acta Colombiana de Psicología, 13(2), 107-117.

Padrós, F., Gutiérrez, C. Y., & Medina, M. A. (2015). Propiedades psicométricas de la escala de satisfacción con la vida (SWLS) de Diener en población de Michoacán (México). Avances en Psicología Latinoamericana, 33(2), 223-232. doi: 10.12804/apl33.02.2015.04

Patrick, C. J. & Iacono, W. G. (1989). Psychopathy, threat, and polygraph test accuracy. Journal of Applied Psychology, 74(2), 347-355. doi: 10.1037/0021-9010.74.2.347

Pullen, L., Modrcin, M. A., McGuire, S. L., Lane, K., Kearnely, M., & Engle, S. (2015). Anger in Adolescent Communities: How Angry Are They? Pediatric Nursing, 41(3), 135-140.

Ribes-Iñesta, E. (2007). Lenguaje, aprendizaje y conocimiento. Revista Mexicana de Psicología, 24(1), 7-14.

Ribes-Iñesta, E. (2009). La personalidad como organización de los estilos interactivos. Revista Mexicana de Psicología, 26(2), 145-161.

Rodríguez, M. N. & Ruiz, M. A. (2008). Atenuación de la asimetría y de la curtosis de las puntuaciones observadas mediante transformaciones de variables: incidencia sobre la estructura factorial. Psicológica, 29(2), 205-227.

Schimmack, U., Oishi, S., Furr, R. M., & Funder, D. C. (2004). Personality and Life Satisfaction: A Facet-Level Analysis. Personality and Social Psychology Bulletin, 30(8), 1062-1075. doi: 10.1177/0146167204264292

Shaver, P. R., Schwartz, J., Kirson, D., & O’Connor, D. (1987). Emotion knowledge: Further exploration of a prototype approach. Journal of Personality and Social Psychology, 52(6), 1061-1086.

Spielberger, C. D. & Reheiser, E. (2009). Assessment of emotions: Anxiety, anger, depression, and curiosity. Applied psychology: Health and Well-Being, 1(3), 271-302.

Spielberger, C. D. (1999). The State-Trait Anger Expression Inventory-2: Professional Manual. Odessa, FL: Psychological Assessment Resources.

Suhr, D. (2006). Exploratory or Confirmatory Factor Analysis. Proceedings from the SAS Users Group International Conference. Cary: SAS Institute.

Tabachnick, B. G., Fidell, L. S., & Osterlind, S. J. (2001). Using Multivariate Statistics. Boston: Allyn & Bacon.

Ten-Berge, J. M. F. (1986). Some relations between descriptive comparisons of components from different studies. Multivariate Behavioral Research, 21(1), 29-40.

Williams, B., Onsman, A., & Brown, T.(2010). Exploratory factor analysis: A five-step guide for novices. Australasian Journal of Paramedicine, 8(3), 1-13.