J. C. Bunch, University of Florida, bunchj@ufl.edu
Melissa Cater, Louisiana State University, mcater@agcenter.lsu.edu
Shelli D. Rampold, University of Florida, srampold@ufl.edu
Abstract
As transportation and communication systems evolve, the ability of youth to participate in international experience program (IEP) improves. However, not all youth will choose to participate in an IEP even if an IEP is readily available. Understanding the motivating factors for participation in an IEP and deterrents to participation in an IEP can assist program developers better market available experiences. In order to assess factors that inhibit or promote participation in an IEP, valid and reliable instruments are needed. This study describes the development and initial validation of the 4-H International Experience Program Questionnaire, an instrument designed to measure 4-H members’ motivating factors for participation in an IEP and deterrents to participation in an IEP.
Introduction
The need to adopt a global perspective poses challenges for U.S Cooperative Extension, particularly regarding international program opportunities for 4-H members (Etling, Reaman, & Sawi, 1993). Examination of the 4-H members’ perceptions of motivating factors and deterrents associated with participating in an international experience program (IEP) may assist in explaining their intention to participate in an IEP. Understanding members’ intention to participate in an IEP can further international experience (IE) programming efforts and potentially lead to increased participation in an IEP (Stroud, 2010). According to Etling et al. (1993), an examination of the deterrents and benefits associated with such programs is a necessary measure to reducing deterrents and enhancing international programming. Reported deterrents to 4-H members’ participation in an IEP include: (a) financial constraints; (b) lack of information; and (c) lack of support by family, friends or community (Boyd et al., 2001).Despite the deterrents, 4-H members were motivated to participate because they perceived the experience would be worthwhile and impactful to their lives (Boyd et al., 2001). In another study by Cater, Bunch, and Danjean (2016), prior intercultural experiences predicted 4-H members’ intention to participate in an IEP. This effect was partially explained by motivating factors such as youths’ perception that participation in an IEP increased their employability which resulted in an increase in youths’ perceived intention to participate in an IEP; however, deterrent factors such as time, reduced both motivation and intention to participate in an IEP (Cater et al., 2016).
This study seeks to fill a void in the youth international experience literature for valid and reliable survey instruments that measure youths’ attitudes toward an IEP. Specifically, attitudes in this study are operationalized as motivating and deterring factors to participation in an IEP. Little research exists regarding 4-H members’ perceptions of IEPs and their associated perceived motivating factors and deterrents to participation. Much of the existing literature has become dated, and the surrounding body of literature is lacking overall (Arnold, Davis, & Corliss, 2014). Additionally, the methodological approach used in many studies has been limited and lacks the use of formally developed and tested instruments. To date, there is seemingly no valid and reliable instruments to measure 4-H members’ perceptions of motivating factors and deterrents to participation in an IEP. As such, there is a need to develop an instrument to measure these variables.
Conceptual Framework
The Predisposing, Reinforcing, and Enabling Constructs in Educational/Ecological Diagnosis and Evaluation—Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development (PRECEDE-PROCEED) model was used as the guiding framework for this instrument (Green & Kreuter, 2005). The framework encompasses eight phases that provide a structure for understanding social problems, developing and implementing an intervention, and evaluating proximal and distal program outcomes. PRECEDE is dedicated to understanding a problem and includes a series of phases that lead up to an intervention while PROCEED is devoted to the implementation and evaluation of the program.
The questionnaire developed and described in this paper, phase three of the PRECEDE section of the model, Educational/Environmental Diagnosis and Evaluation, was used. This phase of the model consists of predisposing factors, enabling factors, and reinforcing factors. These factors are particularly relevant when studying 4-H members’ intention to participate, and ultimate participation, in an IEP. Predisposing factors include variables like “knowledge, attitudes, beliefs, values, and perceptions that facilitate or hinder motivation” (Green & Kreuter, 2005, p. 14). Enabling factors comprise “skills, resources, or barriers that can help or hinder” (p. 15) change, while reinforcing factors involve influential reward and feedback mechanisms following a behavior change. Predisposing and enabling factors were considered when developing this questionnaire because of their practical relevance to designing programs that promote 4-H member participation in an IEP, creating training for 4-H professionals and volunteers that lead an IEP, and building marketing campaigns for a 4-H IEP.
Purpose and Objectives
The purpose of this study was to develop an instrument that measured 4-H members’ perceived motivating factors for participation in an IEP and deterrents to participation in an IEP. This study was guided by the following objectives:
- Determine if instrument items cluster into latent constructs that can be used to describe participantattitudes.
- Determine if constructs describing participant attitudes are internally consistent.
Methods
Population and Sample
The target population for the study reported here was 7th through 12th grade 4-H members who attended a three-day 4-H conference at [State] University. Responses were collected from 628 of the 789 4-H members for a response rate of 80%. Random assignment was used to divide the 4-H members into two groups. Data from group one (n = 314) were used to develop the instrument, while group two data (n = 314) were used to test a model of prior intercultural experiences, motivating factors to participation in an IEP, deterrents to participation in an IEP, and intentions to participate in an IEP (Cater et al., 2016). The remainder of this paper reports results using data from group one.
The majority of the 4-H members were female (n = 194; 61.8%). Members were predominantly white (n = 237; 77.2%), with 42 (13.7%) identifying their race as African American and the remainder (n= 28; 9.1%) as other. Seven members did not report race. All members were between 13 and 19 years of age (M = 15.4; SD = 1.5). Grade level ranged from 7 to 12 (M = 10.5; SD = 1.4). Twelve members did not report age, and 25 did not report grade level.
Instrument Development
The original instrument chosen for this study was developed to examine the motivators and barriers influencing College of Agriculture undergraduates’ decisions to participate in international experiences (Bunch, Lamm, Israel, & Edwards, 2013; Reiger, n.d.). Since the original study, the instrument was modified and used with college freshmen in agriculture and undergraduates in other Colleges of Agriculture outside of the original study (Bunch, Blackburn, Danjean, Stair, & Blanchard, 2015; Danjean, Bunch, & Blackburn, 2015). Moreover, previous researchers had not reported if latent constructs existed. Because the existing instrument was not designed for middle and high school-aged youth and latent constructs were not reported, the need to modify the instrument for middle and high school-aged students and report latent variables is warranted.
Items for the instrument, 4-H International Experience Program Questionnaire, were modified from questionnaires developed by Bunch et al. (2013) and Reiger (n.d.) to measure motivating factors for participation in an IEP, nine items from the Bunch et al. (2013) were used verbatim (n = 7) or modified (n = 2). The two items were modified by removing the words (a) academic, (b) specialization, (c) academic, (d) advisor, and (e) department and adding the words (a) 4-H, (b) agent, (c) career, (d) interest, (e) international, and (f) setting to construct items that are more appropriate for a high school audience. The item “Get a graduate degree” was not used since it was not relevant to middle and high school students. Response categories were modified by changing the response labels from not important, not very important, somewhat important, and very important to not at all important, somewhat unimportant, somewhat important, very important to create a more balanced set of options.
Deterrents to participation in an IEP were measured using eight items (n = 8) modified from Reiger (n.d.) and seven items (n = 7) modified from Bunch et al. (2013). The eight items were modified by striking words such as (a) degree, (b) credit hours, and (d) major. Further, words were inserted such as (a) international 4-H experiences, (b) 4-H, and (c) programs to make them more relevant to a middle and high school audience. The items “Potential for being victim of crime, terrorism, or unjust government action,” “Don’t have academic qualifications to study abroad,” “Don’t have foreign language skills,” and “Potential for contracting diseases in foreign countries” were not used since they were not deemed relevant to or appropriate for middle and high school students and this study. The seven items from Bunch et al. (2013) were modified by striking the words (a) study abroad, (b) University, (c) academic departments, and (d) professionally. As such, words were inserted to speak to a middle and high school audience. Specific words included were (a) international 4-H experiences, (b) 4-H programs, and (c) academically.
Data Collection
A hard copy questionnaire was distributed to 4-H members on the final evening of the conference. The instruments were distributed by 4-H youth educators who were trained in the data collection protocol. Youth educators received a data distribution checklist outlining the steps in the process. This process included instructions about distributing the instrument, maintaining participants’ right to refuse to complete the instrument, and returning the instrument to the researchers.
Data Analysis
Descriptive statistics were used to summarize demographic data. Since the items and response categories had been substantially altered, the questionnaire was treated as a newly developed instrument. Thus, exploratory factor analysis (EFA) with principal axis factoring and promax rotation was used (Tabachnick & Fidell, 2007). Promax rotation allowed for correlated factors. The appropriateness of the data for exploratory factor analysis was assessed through examination of sample size, communalities, Kaiser-Meyer-Olkin (KMO) statistic, and Bartlett’s test of sphericity. An a priori decision was made to use a 20-to-1 observation-to-item ratio to minimize sampling error (Hair, Black, Babin, & Anderson, 2009). Additionally, extracted communalities would be inspected to establish a range of 0.5 as a minimum (MacCullum, Widaman, Zhang, & Hong, 1999).
Sampling adequacy was assessed through inspection of Bartlett’s test of sphericity and the KMO statistic with an expectation that a significant p-value for Bartlett’s test and a KMO statistic greater than 0.6 would ensure that sufficient correlation existed among items to allow for extraction of factors (Tabachnick & Fidell, 2007). A value of 0.9 or greater and a determinant exceeding zero were set as indicators of multicollinearity or lack thereof, respectively (Field, 2009). The number of factors extracted was determined by using a minimum cut-off value of 1.0 extracted eigenvalue. Internal consistency reliability was assessed using Cronbach’s alpha.
Results
Objective 1: Determine if items cluster into latent constructs
The purpose of objective one was to determine if the items clustered into latent constructs. The sample was reviewed to determine if sample size was large enough to produce a result with precision of loadings and stability that would be replicable across samples. The 15- to-1 observation-to-question ratio in this study was slightly less than adequate to reduce sampling error (Hair, Black, Babin, & Anderson, 2009). However, with a median value of .492, a mean value of .504, and values ranging from .237 to .634, communalities suggested that the sample was within the range to be considered adequate to reduce sampling error (MacCullum, Widaman, Zhang, & Hong,1999).
The KMO was .91 and Bartlett’s test of Sphericity was significant (X2(210) = 2880.9, p < .001), which demonstrated that there was sufficient correlation among the items to make them appropriate for factor analysis (Dziuban & Shirkey, 1974). The correlation matrix was checked for cases of extreme multicollinearity by reviewing the item intercorrelations and the determinant. Item intercorrelations ranged from -.012 to .696, and the determinant was greater than zero, supporting the assumption of an absence of multicollinearity (Field, 2009). An eigenvalue cut-off of 1.0 was used to determine the number of factors. After extraction, two factors that explain 50.37% of the cumulative variance were returned (see Table 1).
Table 1 Summed Squared Factor Loadings and Total Variance Explained for Items in the 4-H International Experience Program Questionnaire
| Factor | Eigenvalues | Percentage of Variance |
|---|---|---|
| 1 – Motivating Factors for IEP Participation | 6.88 | 32.7 |
| 2 – Deterrents to IEP Participation | 3.70 | 17.60 |
The original instrument contained 15 items representing the factor Deterrents to IEP Participation. Examination of the pattern matrix yielded three items (11B, 11F, and 11G) that contributed little to the factor with loadings less than .40, thus they were removed from the construct. Twelve items remained in Factor 1 (see Table 2). The factor represented members’ perceptions of deterrents for participation in an IEP. Regarding Factor 2, the original instrument contained nine items. All nine items remained in the factor Motivating Factors for IEP Participation and displayed pattern matrix loadings greater than .40 (see Table 2). This factor represented members’ perceptions of motivating factors for participating in an IEP. The correlation between the two factors was -.27.
Table 2 Pattern Matrix, Communalities, and Structure Matrix for the Motivating Factors for and Deterrents to IEP Participation Subscales
| Variable | Pattern Matrix | Structure Matrix | |||
|---|---|---|---|---|---|
| Factor 1a | Factor 2b | h2 | Factor 1a | Factor 2b | |
| Deterrents Item 1 | .50 | -.08 | .32 | .48 | .06 |
| Deterrents Item 2 | .71 | -.02 | .46 | .70 | .18 |
| Deterrents Item 3 | .65 | -.02 | .45 | .64 | .15 |
| Deterrents Item 4 | .67 | -.00 | .51 | .67 | .18 |
| Deterrents Item 5 | .76 | .09 | .60 | .79 | .29 |
| Deterrents Item 6 | .70 | -.06 | .57 | .68 | .13 |
| Deterrents Item 7 | .76 | .02 | .63 | .76 | .22 |
| Deterrents Item 8 | .76 | .05 | .47 | .77 | .25 |
| Deterrents Item 9 | .77 | .08 | .44 | .79 | .29 |
| Deterrents Item 10 | .80 | -.05 | .24 | .79 | .17 |
| Deterrents Item 11 | .76 | .01 | .49 | .76 | .22 |
| Deterrents Item 12 | .68 | -.08 | .41 | .66 | .10 |
| Motivating Item 1 | -.00 | .57 | .45 | .15 | .57 |
| Motivating Item 2 | .01 | .67 | .63 | .20 | .68 |
| Motivating Item 3 | .01 | .67 | .47 | .19 | .68 |
| Motivating Item 4 | .03 | .70 | .58 | .22 | .71 |
| Motivating Item 5 | .02 | .77 | .59 | .23 | .78 |
| Motivating Item 6 | .00 | .75 | .63 | .21 | .75 |
| Motivating Item 7 | .06 | .78 | .62 | .27 | .79 |
| Motivating Item 8 | -.08 | .70 | .58 | .11 | .68 |
| Motivating Item 9 | -.13 | .68 | .44 | .05 | .65 |
Objective 2: Determine the internal consistency reliability of constructs
Objective two sought to establish the internal consistency reliability of each construct. Motivating Factors for IEP Participation had a Cronbach’s alpha reliability coefficient of 0.92with a 95% confidence interval of 0.91 to 0.93. The reliability coefficient for Deterrents to IEP Participation was 0.89 with a 95% confidence interval of 0.87 to 0.91.
Conclusions, Recommendations, and Implications
This study describes the development of an instrument to measure the perceived motivating factors and deterrents of 4-H members regarding their participation in an IEP. The instrument consists of 21 items and two constructs. Specifically, there are 12 items included in construct one (Deterrents to International Experience Program Participation) and nine items in construct two (Motivating Factors for International Experience Program Participation). While many studies have focused on college students (Briers, Shinn, & Nguyen, 2010; Bunch et al., 2013; Bunch et al., 2015; Danjean et al., 2015), few quantitative measures exist for use with 4-H youth audiences. This instrument may be particularly useful within a context where information concerning the predisposing and enabling factors that guide youths’ intentions, and ultimately their behaviors, informs program planning.
The results of this study suggest that the instrument is a viable tool for use in assessing 4- H members’ perceived motivating factors and deterrents to participating in an IEP. EFA established initial construct validity with two factors extracted. The quality of the solution is affected by factors like sample size, inter-item correlations, variance explained by the solution, and factor loadings. While the sample size was slightly less than ideal, as dictated by conventional wisdom (Hair, Black, Babin, & Anderson, 2009), it met the criteria for acceptability under data-driven criteria with communalities meeting the minimum standard.
Regarding limitations to the study, it should be noted that the cumulative percentage of variance extracted for the scale was 50.37%. According to Pett, Lackey, and Sullivan (2003), there are no specific guidelines for minimum variance extraction; however, 60% is typically deemed acceptable (Hair, Black, Babin, & Anderson, 2009). With Factor 1 loadings ranging from .5 to .8 and Factor 2 loadings ranging from .57 to .77, the quality of the factor solution is well within accepted guidelines (Thompson, 2004). As researchers continue to investigate the perceived motivating factors and deterrents among 4-H members regarding their participation in an IEP, additional items should be included in the scale.
Future research is needed to confirm construct validity in diverse populations. Convergent validity of the instrument should be established by looking at the relationship between high motivation and high perceived deterrents to participation and actual behavior (i.e., participating in an IEP). High motivation should predict higher participation behavior while high perceived deterrents should predict low participation behavior. While convergent validity was not established with this study, the weak relationship between the motivating factors and deterrents constructs implies divergent validity; however, readers are cautioned that convergent validity of the constructs with some other measure (e.g., actual IEP participation) should be established. Additionally, future research may focus on the addition of one or more constructs to improve the cumulative percent of variance explained.
Note: Interested individuals may contact J.C. Bunch for information about using the questionnaire.
References
Arnold, M., Davis, J., & Corliss, A. (2014). From 4-H international youth exchange to global citizen: Common pathways of ten past program participants. Journal of Youth Development, 9(2), 85–98.
Borgers, N., Hox, J., & Sikkel, D. (2004). Response effects in surveys on children and adolescents: The effect of number of response options, negative wording, and neutral midpoint. Quantity and Quality, 38, 17-33.
Boyd, B. L., Giebler, C., Hince, M., Liu, Y., Mehta, N., Rash, C., … Yanta, Y. (2001). Does study abroad make a difference? An impact assessment of the international 4-H youth exchange program. Journal of Extension, 39(5), 1–7.
Briers, G. E., Shinn, G. C., & Nguyen, A. N. (2010). Through students’ eyes: Perceptions and aspirations of college of agriculture and life science students regarding international education experiences. Journal of International Agricultural and Extension Education, (17)2, 5–20. doi: 10.5191.jiaee.2010.17201
Bunch, J. C., Blackburn, J. J., Danjean, S. E., Stair, K. E., & Blanchard, L. D. (2015). Examining Louisiana State University College of Agriculture students’ perceived motivators and barriers to participation in international experiences. Journal of International Agricultural and Extension Education, 22(3), 69–82. doi: 10.5191/jiaee.2015.22305.
Bunch, J. C., Lamm, A. J., Israel, G. D., & Edwards, M. C. (2013). Assessing the motivators and barriers influencing undergraduate students’ choices to participate in international experiences. Journal of Agricultural Education, 54(2), 217–231. doi: 10.5032/jae.2013.02217.
Cater, M., Bunch, J. C., & Danjean, S. E. (2016). Assessing the prior intercultural experiences, motivating factors and deterrents influencing 4-H members’ intention to participate in an international experience program. Journal of International Agricultural and Extension Education, 23(2), 25–37. doi: 10.5191/jiaee.23202.
Danjean, S. E., Bunch, J. C., & Blackburn, J. J. (2015). Examining the motivations and barriers influencing the decisions of Louisiana State University College of Agriculture freshmen to participate in international experiences. Journal of International Agricultural and Extension Education, 22(1), 49–62. doi: 10.5191/jiaee.2015.22104.
Dziuban, C., & Shirkey, E. (1974) When is a correlation matrix appropriate for factor analysis? Some decision rules. Psychological Bulletin, 81(6), 358-361.
Etling, A., Reaman, K. K., & Sawi, G. E. (1993). Overcoming barriers to a global outlook in 4- H. Journal of Extension, 31(2). Retrieved from http://www.joe.org/joe/1993summer/intl2.php
Field, A. (2009). Discovering statistics using SPSS (3rd ed.). Thousand Oaks, CA: Sage.
Greer, L., & Kreuter, M. (2005). Health program planning: An educational and ecologica approach (4th Ed.). Boston, MA: McGraw Hill.
Hair, J., Black, W., Babin, B., & Anderson, R. (2009). Multivariate data analysis (7th ed.). Upper Saddle River, NJ: Prentice-Hall.
Institute of International Education (IIE) (2013). Open doors report. Retrieved from http://www.iie.org/en/Research-and-Publications/Open-Doors/Data
MacCullum, R., Widaman, K., Zhang, S., & Hong, S. (1999). Sample size in factor analysis. Psychological Methods, 4(1), 84-99. doi: 10.1037/1082-989X.4.1.84
Pett, M., Lackey, N. & Sullivan, J. (2003). Making sense of factor analysis. Thousand Oaks, CA: Sage.
Rieger, M. (n.d.). University of Florida College of Agricultural and Life Sciences study abroad interest survey. Unpublished manuscript, University of Florida.
Stroud, A. H. (2010). Who plans (not) to study abroad? An examination of U.S. student intent. Journal of Studies in International Education, 14(5), 491–507. doi: 10.1177/1028315309357942
Tabachnick, B., & Fidell, L. (2007). Using multivariate statistics. Boston, MA: Pearson Education, Inc.
Thompson, B. (2004). Exploratory and confirmatory factor analysis: Understanding concepts and applications. Washington DC: American Psychological Association.
