The International GTA Problem:

A Paper Presented
At the
Association for Institutional Research
39^th Annual Forum
Seattle, Washington May 30 – June 2, 1999

By

Laurinda G. Smith, Statistical and Information Officer
Kansas State University
Manhattan, KS

Ronald G. Downey, Associate Provost/Director
Kansas State University
Manhattan, KS

Kelline S. Cox, Assistant Director
Kansas State University
Manhattan, KS

The International GTA Problem: A New Approach
Abstract

In response to criticism of international GTAs’ teaching, a communication survey to be completed by students was developed. The communication survey results from 114 native and non-native English speaking GTAs (217 sections, 4,651 students) were very encouraging. Nine GTAs (8%) were identified as having problems, two (2%) of which were native English speakers. Extensive analyses found that prediction of communication problems was related to many more factors than English skills. The results of this survey have clearly demonstrated the complexities of the issues when we refer to the "international" GTA problem. The fundamental problem is that a limited number of GTAs, both native and non-native English speakers, may lack the basic skills to effectively interact with students in the classroom.

The International GTA Problem: A New Approach

In the United States, roughly 40% of the undergraduate courses in research and comprehensive universities are taught by graduate teaching assistants (GTAs) and approximately 60% of the introductory courses for first- and second-year undergraduates classes are taught by GTAs (Wert, 1998, p. xvii). While many of introductory classes are taught by GTAs, systematic attempts to evaluate their success in the classroom are generally lacking (Smith, 1992). With the trend of increasing tuition and the public outcry about research institutions’ commitment to undergraduate education, concerns about GTAs’ teaching skills in general, and non-native English speaking GTAs specifically, seem hardly surprising (Bailey, 1984; Chism, 1987; Managan, 1992).

In fact, college students’ concerns about international GTAs has continued to fester for more than a quarter of a century and has been a continuing source of public comment and embarrassment to faculty and administration. While research (e.g., Jacobs & Friedman, 1988) shows that students in classes of international GTAs do as well as others, this has little effect on dampening the concerns and often has led to counter-charges by faculty of student xenophobia.

Due to the growing criticism from students and the public about non-native English speaking GTAs, many states either passed laws or implemented system-wide programs aimed at assessing their English language skills (Smith, 1992). Some approaches include a requirement that non-native English speakers complete training programs or short courses to develop language and other skills. Some programs focus on the discipline-specific as well as the cross-disciplinary needs. Smith suggests that no single design for these English assessments and training programs is the best; the more effective programs take into account a thorough understanding of the structure, culture, and needs of an institution.

In response to the concerns about non-native English speaking GTAs, for many years a system for assessing non-native English GTAs’ language skills has been in place at a mid-sized university (the University). The state governing board of six universities addressed the issue of English assessment with a three-tier approach. In this approach, all prospective non-native English speaking GTAs who will have teaching responsibilities must: 1) achieve a score of 550 on the TOEFL (Test of English as a Foreign Language) exam before being admitted into the graduate program; 2) take and pass the SPEAK (Speaking Proficiency English Assessment Kit); and 3) have their English skills assessed by a panel of faculty and students. In addition, each department provides training for their GTAs. The training ranges from: 1) very limited training, consisting of a syllabus and textbook; 2) extensive training programs for GTAs prior to class, including several weeks of instruction, monitoring by faculty (mentoring program), and continued training through formal courses (e.g., English, Humanities); and (3) not allowing students to teach until they have completed their master’s program, taken a course in college teaching, and have taught under the direct supervision of faculty (e.g., Psychology). In spite of the extensive attempts to select and train GTAs, the complaints continued. The continuing concerns suggested other sources for the problem.

Past research (Chism, 1987; Rubin, 1992) and actions taken to address the non-native English speaking GTA issue suggest the basis for complaints may have a source(s) other than English language skills. For example, the performance of students in classes taught by non-native English speakers is comparable to that of students in classes taught by native English speakers (Jacobs & Friedman, 1988). Despite our best efforts to screen for English language skills, the complaints persisted, leading to the question, "What really is the problem?"

Within this context, the University’s Provost asked what could be done to address the concerns of students, their parents, and the public. In response, a new approach and process was developed. This new approach and process included the following major elements:

1. The basic skill(s) of concern was the ability of the GTAs to "communicate" effectively in the classroom. The best source of information on GTAs’ communication skills was their students.
2. In response to international community concerns, the communication skills of all GTAs (native or non-native English speaking) needed to be assessed.
3. The survey would only be given to GTAs teaching in the classroom for the first time (first-time GTAs).
4. The approach was to be done very early in the semester in case remedial help was needed.
5. The procedures needed to identify GTAs who had communication problems versus those who were judged effective but less skilled. Any GTAs who were identified as having a communication problem needed to have an opportunity to improve.
6. After an opportunity to improve, GTAs identified as having a communication problem were re-evaluated. GTAs who failed to improve to the specified level were reassigned.

At the beginning of each academic semester, GTAs teaching in classrooms for the first time were identified by the departments. Each class section taught by a first-time GTA were surveyed. The survey used was an adaptation of an existing communication skills survey (Hoyt, 1978). Students were given an opportunity to complete the survey evaluating the GTA on thirteen items. Using a scale of 1 to 5 (Definitely False to Definitely True), students were asked: "To what extent are the following statements true of your instructor?"

1. enunciation is clear;
2. speaks too rapidly;
3. voice is loud enough;
4. tries to cover too much material in most class sessions;
5. expresses ideas and thoughts clearly;
6. presentations lack good organization;
7. understands student questions or comments;
8. lacks skill in explaining difficult concepts; and
9. uses too advanced a vocabulary.

Using the same rating scale (Definitely False to Definitely True), students were then instructed to "Please give frank answers to the following four questions:"

10. I answered the previous nine questions as fairly as I could;
11. I have attended this class regularly;
12. I have made a good effort to be attentive in this class; and
13. the instructor’s lack of communication skills is seriously hampering my learning in this course.

Since the survey was developed to provide important feedback for both the GTA and the departments and as a means for identifying GTAs with major problems, the evaluation was administered early in the academic semester (i.e., during the third and fourth weeks of classes). The timelines of the results would allow time to remedy any potential problem. Using a criterion-referenced approach to measurement, a certain threshold would trigger the identification of a GTA with a serious communication problem. The threshold was attained when a plurality of the students completing the survey for a class rated three or more items as negative. The items included the first nine communication skills plus survey item 13 (overall item). All results and reports were considered confidential and were distributed only to the GTA and, if a communication problem was identified, to their Department Head, the Dean of the College, and the Provost. If a GTA was identified as having a major communication problem, surveys were re-administered three weeks later (these results are not part of this report).

The results from a pilot project in the fall of 1996 were very encouraging (Signore, Gibbons, & Downey, 1996). A total of 44 sections were surveyed; 4 of the 19 non-native and 1 of the 25 native English-speaking GTAs were identified as having problems with their classroom communication. Based on these preliminary results, a full-scale administration was conducted in the fall of 1997. A total of 114 GTAs teaching in the classroom for the first time were surveyed, representing 4,651 students in 217 individual classes (sections). This report provides an analysis of the outcomes of the GTA communication survey including: 1) summarizing the results of the survey; 2) comparing the communication survey items and their interrelationships; and 3) associating a variety of demographic information with the survey items to gain a better understanding of the sources of the communication problems that were identified.

Table 1 provides the means and standard deviations for the 217 sections for items 1 to 9 and item 13. The number of major communication problems was lower than anyone expected; nine GTAs (8%) were identified as having major communication problems (i.e., three or more items, where the plurality of the students gave negative ratings). Of these nine GTAs, two (2%) were native English speakers. The identification of English speaking GTAs with communication problems was consistent with the pilot results and supports the assumption that issues other than English speaking skills were sources for those problems.

Table 1

Results for the Fall 1997 GTA Communication Survey (N=217)

Table 2 provides the intercorrelations of the variables included in the analysis: gender; time of day class was held; building; type of class (lab or lecture); was English the native language of the GTA; academic discipline; and the number of students in the class. The correlations for discipline and building were based on dummy coded information (i.e., a series of 0 and 1 variables were created). The values for discipline and building with other variables represent the regression of the dummy codes on the other variables. The value for discipline with building is the redundancy value from a canonical correlation of the dummy codes for each of the variables. Some interesting patterns emerged between and among the predictors. Building and academic discipline were strongly related to each other (i.e., departments teach most classes in the same building) and had significant relationships with all of the other predictors. The associations for building tended to be stronger than those for academic discipline and reflected the greater degree of differentiation.

Table 2

Intercorrelations of Demographic (Predictor) Variables

Table 3 provides the R² for each of the demographic items on each of the 10 communication items. All of the survey items were significantly related at a low (e.g., r = .29) to high (e.g., r = .91) range with most being in the moderate or higher range (e.g., r = +. 60). Since some of the demographic items were dummy coded, some of the values are correlations and some are regressions. Three of the seven predictors (i.e., demographic items) were found to have substantial relationships (one R² reached .60) with students’ responses to the GTA survey items. The English speaking background of the GTA, the academic discipline, and the building where the class was taught were all found to be predictive of the survey items. Items assessing the more basic communication skills (e.g., enunciates clearly) were more related to the English speaking variable, while items assessing presentation skills (e.g., expresses ideas and thoughts clearly) were more related to discipline and building. Gender, class type (lab versus lecture), and class size had small to moderate relationships with some of the survey items (all R²s were less than .16). A series of hierarchical regressions using the GTAs English speaking background, academic discipline, and building found that each contributed unique information in predicting a GTA’s communication ratings. English speaking background contributed more on predicting the basic communication skills, and building and academic discipline contributed more on the classroom presentation skills.

Table 3

Regression Results – R² (N=217)

*Denotes a significant result p<.05.

The development and administration of the GTA Communication Survey provided students, the GTAs, the departments, and the University with many benefits. What started out as an attempt to deal with the "international GTA problem" yielded results beyond this issue. While the language background of a GTA was related to communication skills in the classroom, other factors (e.g., discipline, lab versus lecture, class size) were also explicitly important and seemed to justify continuing the administration of the Communication Survey to all first-time GTAs. Having a majority of students in a class reporting a major concern about the quality of communication was not isolated to non-native English speakers. The very low number of non-native English speaking GTAs with major communication difficulties is a testament to the effectiveness of the screening procedures used to ensure basic English skills (e.g., testing of speaking skills). However, even with the small number of problems, the number of students affected in these classes (i.e., several hundreds) is unacceptable.

Our work in this area has suggested additional factors that may cause or contribute to the communication problem. The following provides an informal list of some of the findings and benefits of the GTA Communication Survey.

• It identified other important individual skills and structural conditions that are important in communicating with students. For example, departmental GTA training, prior teaching experience, social skills of the GTA, difficulty of the discipline, cultural differences in interacting in a class, et cetera.
• It allowed a close examination of inconsistent results from multiple sections for a GTA where the inexperienced GTA lost control of the classroom.
• It allowed a basis for following up on complaints from students about a GTA’s communication skills.
• The evaluation of first-time GTAs prompted departments to supply training specifically for first-time GTAs.
• The evaluation process sent a message to all GTAs that they must be prepared for the classroom setting.
• It is an outward and visible sign to students, faculty, departments, colleges, parents, and our governing board about our concern and commitment to a quality teaching environment.
• Finally, the improvement of GTAs’ communication skills will better prepare those GTAs planning to enter academia upon completion of their graduate studies.

The GTA Communication Survey has not been a static process. Each year since 1996 there has been re-evaluations of the process. The focus of the first two years was identifying GTAs with communication problems and dealing with them. This past year, the Faculty Senate has become involved in revising the form and procedures. The changes involved, though, did not alter the basic meaning or results. In addition, the approach has become developmental for all GTAs. The reporting format was modified to provide normative information and appropriate on-campus resources to aid the GTAs to improve. It is also anticipated that the Faculty Senate will endorse the program and/or add it to the Faculty Handbook.

The success of all GTAs clearly depends on the cooperation and coordination of administration, faculty, and students and requires clear plans and procedures to facilitate success. The results have clearly demonstrated the complexities of the issues when we refer to the "international" GTA problem. When faced with a non-native English speaker, students tended, for lack of a better reason, to attribute the problem to the GTA’s accent. What has become very evident to us is the importance of seeing the issues in the broader context of concern for quality teaching. What we first thought was a major problem, became a manageable one when we realized that English language difficulties were not the root of students’ concerns. Often the students were unable to articulate that their GTA could not answer their questions or did not have organized lectures. The fundamental problem is that a limited number of GTAs, both native and non-native English speakers, may lack the basic skills to effectively interact with students in the classroom.

Bailey, K M. (1984). The foreign TA problem. In K. Bailey, F. Pialorsi, and J. Zukowski/Faust (eds.), Foreign teaching assistants in U.S. universities (pp. 3-15). Washington DC: National Association for Foreign Student Affairs.

Chism, N. V. N. (Ed.). (1987), International teaching assistants. Institutional responsibilities and responses in the employment and education of teaching assistants. (p. 249). Columbus, OH: The Ohio State University: Center for Teaching Excellence.

Hoyt, D. P. (1978). Communication survey. Unpublished materials, Kansas State University: Manhattan, KS.

Jacobs, L. C., & Friedman, C. B. (1988). Student achievement under foreign teaching associates compared with native teaching associates. Journal of Higher Education, 59, 551-563.

Mangan, K. S. (1992, Mar. 4). Colleges expand efforts to help teaching assistants learn to teach. The Chronicle of Higher Education, p. A17.

Rubin, D. L. (1992). Non-language factors affecting undergraduates’ judgements of non- native English-speaking teaching assistants. Research in Higher Education, 33, 4.

Signore, C. E., Gibbons, J. A., & Downey, R. G. (1996). A study of the first-year communication skills of non-native and native english-speaking graduate teaching assistants (Research Rep. No. 76). Manhattan, KS: Kansas State University, Planning and Analysis.

Smith, R. M., Byrd, P., Nelson, G. L., Barrett, R. P., and Constantinides, J. C. (1992, Dec.). Crossing pedagogical oceans: International teaching assistants in U.S. undergraduate education. (ASHE-ERIC Higher Education Report No. 8). Washington, D.C.; The George Washington University, School of Education and Human Development.

Wert, E. (1998). Foreward. In Marincovich, M., Prostko, J., & Stout, F. (Eds.). (1998). The professional development of graduate teaching assistants. Bolton, MA: Anker Publishing Company, Inc.