The Effectiveness of Thinking Maps in Developing Secondary Students' listening Comprehension

Manal Lotfy Kabesh
( )
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
The Effectiveness of Thinking Maps in Developing Secondary Students' listening Comprehension
Manal Lotfy Kabesh
Ph.D. holder from Ain-Shams University /Faculty of Education
National Center for Educational Research and Development.
Abstract
This study aimed at investigating the impact of implementing Thinking Maps in teaching listening comprehension to secondary students. A group of 50 secondary stage students were selected from one of Cairo language schools. The quasi-experimental design called the non-equivalent group design was employed in the present study where two intact classes were randomly selected to represent both groups. The experimental group received training through the proposed Thinking Maps strategy. The control group received regular listening comprehension instruction. A pre/post listening comprehension test was given to the two groups before and after the treatment. The results of the t-tests proved to be statistically consistent with the hypotheses. The results revealed that the experimental group outperformed the control group in the listening comprehension skills. Implementing Thinking Maps changed the classroom from a place where listening comprehension was taught to one where opportunities to think broadly and flexibly away from the constraints of their minds were provided. This proved that Thinking Maps not only developed participants' listening comprehension skill, but also raised their visual awareness.
Keywords: Thinking Maps, listening comprehension, secondary stage,
Introduction
Among the main four English language skills, listening is of vital importance. It is a deliberate conscious act that goes beyond the simple reception of sounds to a deeper processing level aiming at reconstructing the original intentions of speakers. In addition to being a complex development skill that requires the mastery of different sub-skills, listening is also an active process of reconstructing the original intentions of speakers using cues from contextual information and from existing knowledge (O'Malley, Chamot & Kupper, 1989, 434; Scarcella & Oxford, 1992, 142; Rubin, 1994; Nunan, 1997, 6; Chung, 2002,231 and Vandergrift, 2003, 427). In this sense, the listening process involves various sub-processes, which demand both physical and mental activity on the part of the listeners; such as perception of the spoken stimuli and assigning meaning to them (Jalong, 1991, & Kingen, 2000).
(114)
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
It is worth mentioning that listening has been in the shadow of the other skills' areas ; for a long time, it was simply a means to an end (Grant, 1997, 11). Hence, it is not surprising to find Feyten (1991,173) describing listening as " an overlooked dimension in language acquisition " or to find Mendelsohn (1995) and Nunan(1999) referring to listening as the ' Cinderella skill ".
It should be clearly emphasized that problems involved in teaching EFL listening comprehension might be partly due to the difficult nature of the listening skill itself (Helal, 2006,4). According to Oxford (1993,206) and Rubin (1995,8), listening is the skill that makes the heaviest processing demands because learners must store information in their short term memory at the same time as they are working to understand this information. Moreover, one of the largest inhibitors for ESL/EFL students during listening comprehension is having and facing a "mental block". Discovering that they can not understand or follow what is being said , students either tune out or get caught up in an internal dialogue trying to translate specific words included in the oral texts. This in turn, negatively affects both their attitude towards the listening comprehension process and their performance during listening tasks (Beare,2005,1).
Confirming the idea that EFL listening is a difficult activity for most students, Underwood (1990,16-20) states that EFL students encounter several problems when they attempt to listen. These problems include learners' (a) lack of control over the speed at which speakers speak, (b) inability to get utterances repeated especially in real life communication situations, (c) limited vocabulary, (d) lack of suitable background knowledge of the context of the listening material in a way that negatively affects their interpretation of the spoken discourse, (e) inability to concentrate for longer periods of time, especially when they find the listening topic boring, (f) failure to recognize 'signals' of spoken texts such as pause, gestures, sudden increased loudness, clear change of pitch or intonation patterns or the use of certain coherence markers that reveal the organization of the spoken discourses and (g) traditional established learning habits that make them falsely think that they need to understand the literal meaning of every single word in order to understand a given utterance. All the above mentioned points were empirically confirmed in Hasan's study (2000) on causes of EFL learners' difficulties in listening comprehension, as well as (Nunan,1997,6)& Jun,2000,1). Listening is considered a highly integrative skill and research has also stressed the role of listening in facilitating second language acquisition (Richards, 2008,86 & Fang, 2008). Listening is thought of as both interactive and constructive. It should be emphasized that problems involved in teaching
Manal Lotfy Kabesh
( )
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
EFL listening comprehension might be partly due to the difficult nature of the listening skill itself.
Nevertheless, based on a pilot study conducted by the researcher based on an interview and an informal questionnaire administered to 18 English language teachers and supervisors. This questionnaire and interview aimed at identifying the methods and techniques currently employed in teaching listening comprehension skills among secondary students. In addition, the researcher asked 20 secondary students to answer an English listening comprehension test including the literal, inferential, and critical language listening comprehension sub-skills. The results of the pilot study revealed that listening comprehension is the least skill which secondary students show real desire to carry out, which may be due to the prevailing traditional method of teaching listening comprehension. Also, students rarely practise any pre-listening activities that facilitates their meaning -making processes of spoken texts. They immediately launch in listening to the tape once or twice at most and then do the post- listening activity. This negatively affects the development of students' listening comprehension skills and decrease their motivation to practise listening activities.
Moreover, 80% of those teachers and supervisors do not have adequate knowledge of the listening comprehension sub-skills necessary for secondary stage students. Most of them focused on literal comprehension skills including identifying the main idea and identifying specific details, neglecting other important levels of listening comprehension skills, i.e, inferential and critical listening and their sub-skills including making inferences, drawing conclusions, inferring the speakers' attitude, mood and intentions. In addition, 90% of EFL teachers do not teach students the distinctive features of the spoken English language such as the different intonation and stress patterns as they are not evaluated according to the formal written examination system. Moreover, 90% of those EFL teachers and supervisors expressed their need for better teaching strategies that would increase their students' motivation to actively engage in listening tasks and consequently improve their listening comprehension skills. This problem was further emphasized by reviewing previous related studies in the Egyptian context such as the studies of Abou Hadid (2000), Al Hariree (2004), and Helal (2006)These studies emphasized the fact that EFL listening comprehension skills are neglected in Egyptian secondary classes, despite their importance to other language skills. Also, These studies highlighted the need for adopting better teaching strategies to foster EFL secondary students; listening comprehension skills.
(116)
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
Statement of the problem
The study problem can be stated in secondary stage students' poor mastery of the necessary EFL listening comprehension skills. This might be attributed to many factors. Among these factors are the traditional methods of teaching listening still adopted by most EFL secondary school teachers and the absence of motivating pre-listening activities that can build and activate students' background knowledge necessary for the successfully interpreting and making meaning of spoken discourses, and engage learners in the thinking and listening comprehension process.
Questions of the study
The current study attempted to provide answers to the following main question: What is the effectiveness of a proposed program based on Thinking Maps strategy on developing EFL listening comprehension skills among secondary stage students?
The following sub-questions were derived from this main question:
1-What are the listening comprehension skills necessary for secondary English language students?
2-What are the theoretical bases and principles for using the Thinking Maps strategy on developing the listening comprehension skills for secondary English language students?
3-What are the activities to be included in the proposed strategy?
4-How far is the proposed strategy effective in developing the listening comprehension skills among secondary English language students?
The study hypotheses:
The study hypotheses were as follows:
Hypothesis one:
There is a statistically significant differences between the mean scores of the experimental group exposed to the suggested Thinking Maps program and the control group receiving regular instruction on the post-test in overall listening comprehension in favor of the experimental group.
Hypothesis two:
There are statistically significant differences between the mean scores of the experimental group on the listening comprehension pre-and post-tests in listening comprehension sub-skills in favor of the post -test scores.
Manal Lotfy Kabesh
( )
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
Delimitations of the study:
This study was confined to:
1-Two intact secondary classes in Maadi Canal language school(An Egyptian National institution).
2-Developing listening comprehension sub-skills necessary for secondary students. These include literal, inferential as well as critical listening comprehension skills.
3-A duration of six weeks.
Significance of the study
The significance of the present lies in the fact that: 1- It is an attempt to overcome the current shortcomings in teaching EFL listening comprehension skills for secondary school students. 2-It is also an attempt to investigate the effectiveness of a proposed program based on Thinking Maps strategy in developing EFL listening comprehension skill.
Thinking Maps
David Hyerle (1996, 2-3) created a set of eight thinking-process maps, incorporating various types of graphic representations. He explains that despite the wealth of knowledge his students displayed on their semantic maps they were ultimately confused about how to further organize, analyze,and evaluate their representations. They could brainstorm exciting and imaginative ideas, but they were less capable at following through with an organized, coherent piece of writing. In order to know what happens to the brain after the storm, he became immersed in the thinking process approach to curriculum, and devised a language of eight related visual tools- what he calls Thinking Maps. (see figs. 1&2).
(118)
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
Fig. (1): Overview of 8 Thinking Maps
Fig. (2): Thinking Maps and Thinking Processes
These forms are designed to help students generate and organize their thoughts and ideas, either on paper or by using the software, and construct simple to complex models. Each Thinking Map corresponds to a single thinking process:
-Circle map : helps define words or things in context and presents points of view.
-Bubble map: describes emotional, sensory, and logical qualities.
Manal Lotfy Kabesh
( )
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
-Double bubble map : compares and contrasts qualities.
-Tree map : shows the relationships between main ideas and supporting details.
-Flow map : shows events as a sequence.
-Multi-flow map : shows causes and effects and helps predict outcomes.
-Brace map : shows physical structures and part-whole relationships.
-Bridge map :helps to transfer or form analogies and metaphors.
Teachers are trained to introduce students to all eight maps as a related set of tools for content learning. They then show the students how to use these maps as needed, isolated or together. Teachers can do this in a short time because each map is a concrete tool rather than an abstract definition. These Thinking Maps are consistent yet flexible, with developmental, integrative, and reflective attributes that can be used interdependently as a tool to transform knowledge into learning across content across and age levels(Hyerle, 2009). Each of the eight Thinking Maps is theoretically grounded in a fundamental cognitive process or thinking skill.
Awareness of five critical attributes of Thinking Maps will clarify how all maps work, and how they work together.Thinking Maps have five qualities:- consistency: Thinking Maps have a consistent form that visually reflects the cognitive skill being defined, for example, the process of sequencing is represented by the Flow Map starting with one box and one arrow. This is the graphic primitive upon which the map is used to show linear concepts. Thus, a Flow Map might show just the three boxes, with key information written inside, showing the beginning, middle, and end of a story.-Flexibility: Thinking Maps are flexible in form and in the number of ways the map can grow and be configured. So a Flow Map of a story may start at the beginning but grow in complexity to show many stages and sub- stages of the story. This map could be drawn rising from the bottom left to top right of the page, reflecting the rising action of a story.- Development: Any learner may begin with a blank sheet of paper and expand the map to show his or her thinking. The learner and the content of the learning determine the complexity of the map, and produces a different configuration.- Integration: Thinking processes and content knowledge are two key dimensions of integration. All the maps may be used and integrated together. Using the story example, a learner can use the Flow Map to show a plot, a Double Bubble Map to show a comparison of characters, and then a Tree Map to identify the main idea and supporting details.-Reflection: As a language learner,
(120)
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
the maps unveil what and how one is thinking in patterns. Not only can the learner reflect upon the pattern of content, but also reflects on and informally assesses the content learning and thinking processes of the learner (Novak & Gowin, 1984).
According to Sherman(2011) thinking Maps fall into the Postmodern Constructivism Theory. In the Postmodern Constructivism theory of Education, learners often use their prior experiences, and when using one of the eight fundamental thought processes and Thinking Maps, learners often do use their background knowledge to continue constructing new meaning and knowledge. Students are also encouraged to create and form their own opinions after self-reflection and creating Thinking Maps, as well as to engage and discuss their individual maps with their peers, even if their ideas differ. Another aspect of the Postmodern Constructivism theory of Education that is seen by the use of Thinking Maps is that they promote critical Thinking and problem solving.
Thinking Maps provides visual images which can be linked to verbal information. Students' thinking is transmitted into visual images when students verbalize their thinking, either externally or internally. By combining linguistic with nonlinguistic representations of what is to be learned, cognitive development and comprehension will be enhanced. Furthermore, the creation of nonlinguistic representations explicitly, increases and stimulates brain activity (Gerlic & Jausovec, 1999).
Thinking Maps are based on eight cognitive skills. Each visual representation is linked to a specific thought process. By connecting a dynamic visual design with a specific thought process, students create mental visual patterns for thinking on these eight cognitive skills. There are several commonalities among various types of graphic representations, such as to differentiate ideas and concepts (Clark & Paivio,1991), to trigger prior knowledge and to evaluate students' understandings or misunderstandings. Despite common characteristics, various types of graphic representations with different purposes exist. They can be categorized as brainstorming webs, graphic organizers, concept maps, and thinking maps (Hyrele, 2009).
Rationale for using Thinking Maps
Thinking Maps force students to think things through different ways. The more connections, associations, groups and other components they organize, transform, arrange and sort through accordingly, the more pieces of the puzzle start to come together in different ways. Thinking Maps frees students' minds from the limiting constraints of the thought- process and opens their perspectives to new possibilities by helping them to expand their understanding of concepts, elements and ideas from a
Manal Lotfy Kabesh
( )
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
variety of different angles. It forces them to think because there are no longer any short-term memory limitations- they are free to think broadly and flexibly away from the constraints of their minds.
Through his research, Hyerle (2011), found that Thinking Maps help close the gap, as they can help students below grade level self-regulate their own learning and be more successful because Thinking Maps serve as a device for mediating thinking, listening, speaking, reading, writing, problem solving, and acquiring new knowledge. Camp (2007, 155) stated that "thinking in maps is substantively different from thinking in sentences". This concept in turns, supported Hyerle's (2011) idea that Thinking Maps posses an artistic and kinesthetic component, where students can feel free to express their ideas in a "drawing", or map, instead of using complete written sentences.
Hyerle,(1993,7-241) argues that just giving students ready-made graphic organizers is not enough. We must go beyond helping students graphically organize information, and students must become independent critical thinkers. In essence, students using Thinking Maps learn what thinking looks like. Students hook a visual image , because the brain loves pictures, with an abstract concept (like cause and effect reasoning) in order to make thinking more visual.
Connecting brain research (Wolfe(2001), Jensen(2002), Sousa (2000), Goleman (2002)) with lesson planning and Thinking Maps as effective instructional strategies that support the current research are all based on asking the following questions : How does the brain decide what to pay attention to ? How much new information can the brain process and how long can the brain stay focused? What brain functions cause information to " move " from short term memory to long term memory? How can Thinking Maps be utilized to enhance thinking and learning?
Research tells us that the majority of students in a regular classroom need to see information in order to learn it. Visual thinking and learning help students clarify their thoughts. Hyerle (1996) clarifies that by using visual tools corresponding to thinking processes, students can organize their ideas on paper or by computer and improve their reading, writing and thinking skills. He maintains that seeing is understanding: students see how ideas are connected and realize how information can be grouped and organized. With visual learning , new concepts are more thoroughly and easily understood when they are linked to prior knowledge.
Thinking Maps help students organize and analyze information. Students better remember information when it is learned both visually and verbally as this leads them to make connections, understand relationships and recall details (Benson, 1997). Abdel Aziz (2013) investigated the effectiveness of a Mind Map strategy on developing third
(122)
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
year preparatory stage students' reading comprehension skills. The results of the study showed evidence that Mind Maps help students to generate and organize ideas in a visual map. Also, they assure that visual mapping help students to develop their thinking skills, sorting of information and ideas, and understood and retained information when ideas, words and concepts were associated with visual figures.
Moi and Lain (2007) used Mind Maps as one of the comprehension strategies. They assured that Mind Maps help break the complexity of the text passage into words for easier understanding and enhance remembering the comprehension passage better through establishing links, pattern, relationship among concepts and information. Sunseri (2011, 5 -35) examined the scaffolding strategies inherent in the Thinking Maps program to see if students' compositions were more organized if they used a Thinking Map in responding to a writing prompt. She found that Thinking Maps can help students demonstrate their knowledge by helping them organize their thoughts so they can write effective expository composition.
Furthermore, Cooks and Sunseri (2014) examined the impact of several writing strategies on English language learners' (ELLS) writing skills, including prewriting strategies and scaffolding strategies inherent in the Thinking Maps program. Results showed that the overall average of students' writing score in the areas of "ideas" and "organization" increased, while the overall averages were below the proficiency level(3.0), these writing strategies can be seen as having a positive impact on ELLS' writing skills.
Ghoneim (2012), conducted a study to develop the inferential reading comprehension of first year preparatory students through the eight Thinking Maps to improve the higher order thinking skills. Results showed that using Thinking Maps was effective in developing the higher order thinking skills in reading comprehension.Ghoneim,2012,173 -174) highlighted that the Circle Map proved to be an effective brainstorming tool for the students, as it served as a classroom mirror, reflecting the fluency and flexibility of students' thinking, ideas, and information at the moment in time. The map also served as a window, providing a means to access and asses the students' thinking and prior knowledge. For the students, the Bubble Map was not a static "fill-in -the bubbles worksheet". Instead, it became a dynamic, versatile, open-ended graphic with a certain "elasticity" that could be stretched in tandem with their thinking. The double Bubble Map was clearly a more :user-friendly tool for students to combine more clearly the points of similarity and differences. The other Thinking Maps (Tree Map, Flow Map, Multi-Flow
Manal Lotfy Kabesh
( )
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
Map, Brace Map, and Bridge Map)helped to organize students; thoughts in a visual manner.
Listening is considered a highly integrative skill and research has also stressed the role of listening in facilitating second language acquisition (Richards, 2008,86 & Fang,2008). Listening is thought of as both interactive and constructive. From an information processing perspective, listening consists of three simultaneously occurring phases : perception (storing sound in memory), parsing (converting sound to mental representation to the listener's prior knowledge (Lynch, 2005,4). Findings from research suggest that using Thinking Maps facilitates greater listening comprehension. Thinking Maps help students structure and order their thinking by creating a visual representation of concepts and their understanding.
Method
Participants
A group of (50) secondary students were selected from one of Cairo language schools, namely Canal school (Formal institution) , in the year 2016 - 2017 (25 students in the experimental group and 25 students in the control group). Students' age in both groups ranged from sixteen to eighteen years old. All students in the sample of the current study had been learning English as a foreign language for fifteen years.
Research Design
The quasi-experimental design called the non-equivalent group design was employed in the present study. This design is identical to the pre-posttest control/experimental group design in all aspects except that intact groups rather than randomly assigned ones are used, creating a control problem in terms of selection bias. This makes the use of a pre-test necessary for this particular design. In this study, two intact classes were randomly selected to represent the experimental and the control groups. The experimental group received training throughout the proposed Thinking Maps program for developing the listening comprehension skills.
On the other hand, students in the control group received regular instruction. A pre/post listening comprehension test was given to the two groups before and after the treatment. The independent variable was the suggested program based on Thinking Maps, and the dependent variables were the pre-post test, and the listening comprehension skills.
(124)
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
Measures:
The present study made use of three main tools:
 A listening comprehension skills checklist.
 A pre-post listening comprehension test.
 A proposed program based on Thinking Maps.
a- The listening comprehension skills checklist
The checklist was meant to determine the most important listening comprehension skills necessary for the secondary stage. It was designed in light of the objectives included in the Ministry of Education document and the previous literature and related studies concerned with developing listening comprehension skills at this stage for ESL/EFL students, the checklist was submitted to a panel of jury in the field of EFL methods of teaching to determine the degree of importance of each skill.
The listening comprehension skills selected by the study according to their high percentages were as follows:
(1) Literal comprehension:
 Identifying specific stated information or details.
 Identifying organizational patterns within a spoken discourse (including explicit cause and effect relationship and the chronological order of events).
(2)Inferential comprehension :
 Interpreting the uses of different intonation patterns, tones, and stresses which give clues to meaning, social setting and speakers' attitudes..
 Guessing the meaning of unknown words or phrases in a given utterance using semantic, syntactic, contextual and phonological clues.
 Making inferences.
(3) Critical Comprehension:
 Drawing conclusion.
b- The listening comprehension test
Objectives of the test
A pre/post listening comprehension test was constructed and administered to the two groups by the researcher. It was used prior to the program implementation to make sure that students of both groups were at the
Manal Lotfy Kabesh
( )
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
same level before starting the experiment, and hence, the progress achieved by the experimental group would be attributed to the suggested Thinking Maps program they had been exposed to. As a post-test, it was used to investigate the effectiveness of the proposed Thinking Maps program, on developing the selected listening comprehension skills.
The equivalency of both the control and experimental groups was established through analyzing their pre-test results. The following table shows that there were no statistically significant differences between the control and the experimental groups on the pre-test in listening comprehension.
Table (1)
t-test results of the pre-test of both the control and experimental groups
Listening comprehension Skills
sub skills
group
N
M
S.D.
T value
D.F.
Sig. at 0.05
Literal comprehension
Identifying specific stated information or details
experimental
25
1.780
.5788
.726
48
Not sig.
control
25
1.660
.5902
Identifying organizational patterns within a spoken discourse
experimental
25
1.760
.5795
.865
48
Not sig.
control
25
1.620
.5642
Sum
experimental
25
3.5400
1.09848
.819
48
Not sig.
control
25
3.2800
1.14637
Inferential comprehension
Interpreting the uses of different intonation
experimental
25
2.640
.5107
.978
48
Not sig.
control
25
2.780
.5017
Guessing the meaning of unknown words or phrases
experimental
25
1.440
.5066
.863
48
Not sig.
control
25
1.320
.4761
Making inferences
experimental
25
2.040
.4062
.359
48
Not sig.
control
25
2.000
.3819
Sum
experimental
25
6.1200
.75388
.097
48
Not sig.
control
25
6.1000
.70711
Critical comprehension
Drawing conclusion
experimental
25
1.280
.4583
.596
48
Not sig.
control
25
1.360
.4899
Over all sum
experimental
25
10.9400
1.64773
.454
48
Not sig.
control
25
10.7400
1.45860
As shown in table (1), t-values are (0.819) for literal listening comprehension, and (0.097) for inferential listening comprehension, and (0.596) for critical listening comprehension, which are not statistically significant at 0.05 level. Thus, it can be concluded that the two groups
(126)
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
were almost at the same level of performance in listening comprehension skills .
Description of the test
The pre-post test consisted of three varied listening texts selected from different sources and followed by both multiple choice and open-ended questions. The listening texts included in the text final version were all adapted and recorded at normal pace to suit Egyptian EFL secondary stage students' linguistic proficiency level. Students were allowed to listen to each spoken text twice. For the first time, they were free to take notes if they liked or just listen attentively. After the second time of listening to the spoken texts, students were asked to answer the questions in a separate answer sheet especially designed for that purpose.
Test validity and reliability
To measure the test content validity, the first version of the test was given to 5 EFL supervisors and teachers to evaluate it in terms of content appropriateness, and suitability of the test to the students' level. Moreover, they were asked to evaluate the test as a whole in terms of : (a) correctness, (b) number of items and (c) suitability of the listening texts and the test items to the secondary stage students' linguistic level. The test proved to be a valid one, as it measured what it was intended to measure in most cases.
In order to establish the test reliability, the test-retest method was employed with an interval of two weeks on a sample of 25 secondary students of the pilot study. The Pearson's correlation coefficients was used to calculate the reliability coefficient which was(0. 90), and which was relatively high. Therefore, the test could be considered a reliable one for the purpose of the current study.
Piloting the test
The test was piloted on a group of 15 students randomly selected to determine item difficulty, and the suitability of the task selected to extract the expected skills. Results revealed that the majority of students obtained low scores. Moreover, students reported that listening comprehension questions that required them to infer information not explicitly stated in the spoken texts were more difficult for them than listening comprehension questions whose answers were directly and clearly stated in the texts. This may be due to the fact that they were accustomed to answering the literal listening comprehension questions included in their textbook. In addition, it was estimated that a period of 60 minutes would
Manal Lotfy Kabesh
( )
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
provide ample time to complete the test. This time was estimated in the following way :
Time taken by the fastest student + time of the slowest student
2
The pre-test was administered to both groups in normal classroom conditions three days prior to the experiment. The post-rest was administered four days after the experiment ended.
Scoring
Students' answers to the pre/post -listening comprehension test were hand-scored by the researcher. Test scoring did not require another rater because all test items whether multiple choice or open- ended questions were controlled. For multiple choice questions, one score was given for each correct answer, while zero was given for double, wrong or left answers.
As for open-ended items, they were allocated two scores as they required students to provide two pieces of information, with one score assigned to each required piece of information. Zero was given for completely wrong or left question. Students' errors in grammar, vocabulary, spelling and punctuation were disregarded.
Suggested Thinking Maps Program
Learning objectives
Students were expected to master the identified listening comprehension skills by the end of the treatment.
Duration of the program
The Thinking Maps program is taught in six weeks. The first classroom period was an introductory one aiming at introducing students to the aims of the listening comprehension activities they were going to practise ,and Thinking Maps activities they were expected to do.
Validity of the program
The Thinking Maps program's validity Sample activities were given to EFL specialists who approved them, and suggested some modifications.
45 +75
= 60 minutes
2
(128)
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
The panel of jury approved the Thinking Maps as valid and suggested allocating more time to teaching the listening comprehension activities.
Piloting the program
After designing the program and modifying it according to the panel of jury suggestions ,a small pilot study was carried out, 20 students were selected from the secondary stage and were asked to perform three activities, the pilot study aimed at:- determining the time taken by the students to listen to the spoken texts and practise the different activities. -Getting an indication to what extent they enjoyed the activities and how far students were enthusiastic about the listening comprehension and Thinking Maps activities.-Getting an indication concerning the suitability of the program content and activities to students' linguistic proficiency level.
Description of the Thinking Maps listening comprehension program:
It was taken into consideration that the three different levels of listening comprehension should be stressed in each lesson through practising a varied range of listening comprehension activities. However, the choice of a particular literal, inferential or critical listening comprehension skill to be developed in a given lesson was affected by the nature of information included in the spoken texts and in their genre(i.e., narrative, expository or a dialogue).
The researcher followed a proposed teaching strategy throughout the program implementation. Assumptions of this strategy were as follows:
(a)Prior to listening, building and activating students' prior knowledge is necessary for the active reconstruction of the original intentions of speakers. This can be done by using (Thinking Maps : Circle Map, Bubble Map, Double Bubble Map, Tree Map, Brace Map, Flow Map, Multi-Flow Map, and Bridge Map) visual tools that correspond to the thinking processes and a combination of activities including prediction, brainstorming, and different Thinking Maps as tools. With visual learning new concepts are more thoroughly and easily understood when they are linked to prior knowledge.
(b)The 'top-down' and the 'bottom-up" processing models are interactive and complementary for the successful of spoken discourses.
(c) Listening comprehension should be primarily taught as a process of making meaning that proceeds from whole to parts. In this sense, attention is focused on teaching listening comprehension skills in context when they become relevant or when they occur naturally in the texts to
Manal Lotfy Kabesh
( )
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
which learners listen. This should be done using varied during and post listening activities that integrate the four language skills.
According to this teaching strategy, each listening comprehension lesson was divided to three phases. These phases were as follows:
1- Pre-listening phase.
2- During listening phase.
3- post-listening phase.
These phases could be described as follows:
1- Pre-listening phase:
-The teacher prepared students for the listening task through activating their prior knowledge about the topic, context and cultural aspects of the spoken text. This was achieved through encouraging students' active participation in pre-listening activities beginning with predicting the content of the spoken text using the title and accompanying pictures. The teacher used Thinking Maps to further activate and build students' background knowledge about the content of the spoken text whether in pairs, in small groups or individually.
-The teacher helped students' build and activate their prior knowledge about one of the distinctive features of spoken language relevant to the successful comprehension of the spoken text they were going to listen to. This was achieved through previewing and listening to (a) specific minimal pairs, (b) certain confusing sounds or (c) a certain intonation or stress pattern that would be heard later on in the spoken text. Hence, such an activity aimed at fostering students' bottom-up processing of the given text.
-The teacher encouraged students to listen attentively to the spoken text using active listening and monitoring strategies including: (1)paying attention,(2)selective listening focusing only on relevant information and filtering out any irrelevant information,(3) recognizing key words, (4) matching the listening material against learners' own experience, background knowledge and preconceptions, (5) predicting what the speaker is going to say next,(6) using pauses to think about what had been heard,(7) taking notes, (8) trying to visualize elements of the spoken text and (10) inferring implied information about speakers and their situation.
2-During listening Phase:
- Students listened attentively to the spoken text to see whether their prediction about the spoken text were confirmed or not. They also listened carefully to find more information about the content of the spoken text to develop their Thinking Maps.
(130)
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
-Students used active listening and monitoring strategies including their use of relevant prior knowledge cues included in that text to guess the meaning of unknown words or expressions and to make meaning of the whole spoken text.ge of the content and cultural aspects of the spoken text and the syntactic,semantic as well as the phonological cues included in that text to guess the meaning of unknown words or expressions and to make meaning of the whole spoken text.
-Students practise the necessary literal and inferential listening comprehension sub-skills in context using a combination of during-listening activities for each listening comprehension level and sub-skill as follows:
-Literal Comprehension:
-Identifying specific stated information or details.
Students listened to the given spoken discourse and (a) completed texts (or filled gaps),(b) extracted evidence from the spoken texts confirming their sound predictions made prior to listening, (c) completed the story Incidents Predictions/ Confirmation charts(Flow Map), (d) answered true/false questions then corrected false statements.(e) chose sentences according to a speaker's viewpoints from a given table, (f) labeled pictures based on an a given spoken text.
-Identifying organizational patterns within spoken discourse.
Students listened to spoken discourses and (a) answered open-ended questions, or (b) completed a cause-effect chart to practise identifying explicit cause- effect relationships. In addition, students listened to spoken texts, rearranged scrambled sentences, and filled in Series of Events Charts (Flow Map or Multi-Flow Map) or Bridging Snapshots to practise identifying the chronological order of events.
-Inferential Comprehension:
-Guessing the meaning of unknown words or phrases in the spoken text.
Students (a) completed different definition maps (Circle Map) and/or (b) answered multiple choice questions, then circled the word(s) from the sentences that helped them figure out the meaning of unknown words or phrases.
-Interpreting the uses of different intonation patterns, tones and stresses.
Students worked in groups or in pairs to answer open-ended questions addressing their active interpretation of stresses, and intonation patterns
Manal Lotfy Kabesh
( )
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
included in the given spoken discourses to infer speakers' attitudes and feelings.
3-Post-Listening Phase
-Identifying specific stated information or details & Identifying organizational patterns within spoken discourses.
Students completed and extended different Thinking Maps to include new information acquired during listening.
Inferential Comprehension:
-Making inferences.
Students worked in groups or in pairs to (a) answer open-ended and multiple-choice questions, and to (b) judge the correctness of given sets of inferences based on the spoken texts.
Critical Comprehension:
Drawing Conclusion:
Initially, the teacher asked students to draw conclusions based on the spoken texts at hand. Then, the teacher discussed these conclusions with her students showing them how they needed to reflect on the information included in the spoken texts and on their background information to draw sound conclusions. Furthermore, students discussed together each conclusion and provided evidence from the spoken text supporting each conclusion.
Instructional Aids
The following aids were used during implementing the current strategy:
 The white board and illustrations used prior to listening to activate students’ background knowledge.
 The audio CD-ROMs and cassette audio tapes of the listening materials.
 The worksheets including relevant Thinking Maps..
 Hand-outs used by students including different activities focusing on the different listening comprehension sub-skills.
Evaluation
Formative evaluation took the form of M.C.Q and open-ended questions following each passage. On the other hand, summative evaluation included the listening comprehension test utilized to measure the achievement of the intended goals after the treatment.
(132)
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
Results
Results of the study are presented in terms of the study hypotheses.
Hypothesis one:
There are statistically significant differences between the mean scores of the experimental group exposed to the suggested Thinking Maps program and the control group receiving regular instruction on the post test in listening comprehension in favor of the experimental group.
t-tests for independent samples were conducted in order to compare the post-test scores of the experimental and control groups in listening comprehension skills (literal comprehension, inferential comprehension, and critical comprehension). The results of the t-test proved to be statistically consistent with the above stated hypothesis. Therefore, the first hypothesis was supported. Table (2) shows this statistical significance.
To investigate the differences between both the experimental and control groups with respect to each determined listening comprehension sub-skills, t-tests for independent samples were used and they revealed statistically significant differences at 0.05 level.
Table (2)
t-test results of the post-test of both the control and experimental
groups in listening comprehension skills.
Listening comprehension Skills
sub skills
group
N
M
S.D.
T value
D.F.
Sig. at 0.05
Effect size
Literal comprehension
Identifying specific stated information or details
experimental
25
2.920
.2769
7.941
48
Sig.
2.29
control
25
2.240
.3266
Identifying organizational patterns within a spoken discourse
experimental
25
2.820
.2843
6.695
48
Sig.
1.93
control
25
2.100
.4564
Sum
experimental
25
5.7400
.50249
8.101
48
Sig.
2.34
control
25
4.3400
.70297
Inferential comprehension
Interpreting the uses of different intonation
experimental
25
3.900
.4082
8.115
48
Sig.
43.2
control
25
2.400
.8292
Guessing the meaning of unknown words or phrases
experimental
25
2.360
.4899
6.857
48
Sig.
83.1
control
25
1.400
.5000
Making inferences
experimental
25
2.800
.2887
7.794
48
Sig.
2.25
control
25
1.900
.5000
Sum
experimental
25
9.0600
.78156
12.180
48
Sig.
3.52
control
25
5.7000
1.13652
Critical comprehension
Drawing conclusion
experimental
25
2.480
.5099
8.752
48
Sig.
2.53
control
25
1.280
.4583
Over all sum
experimental
25
17.2800
1.39254
14.248
48
Sig.
4.11
control
25
11.3200
1.56045
The above table shows that there were statistically significant differences at 0.05 level between the mean scores of the control and
Manal Lotfy Kabesh
( )
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
experimental groups on the post - test in each listening comprehension skills in favor of the experimental group, since the estimated t-values for the literal listening comprehension were (7.941) for identifying specific stated information or details, and (6.695) for identifying organizational patterns within a spoken discourse, and for the inferential listening comprehension (8.115) for interpreting the uses of different intonation, and (6.857) for guessing the meaning of unknown words or phrases, and (7.794) for making inferences, and for the critical listening comprehension (8.752) for drawing conclusion. Moreover, the effect size values (2.34 ), (3.52) and (2.53) for the literal, inferential and critical listening comprehension skills respectively reveal that the proposed Thinking Maps program had a large effect on the experimental group students' listening comprehension skills on the post-test as compared to those of the control group receiving regular instruction.
Hypothesis two:
There are statistically significant differences between the mean scores of the experimental group on the pre-post- tests in listening comprehension skills in favor of the post-test scores.
In order to verify the validity of this hypothesis, t-tests for paired samples were used. The t-test results proved that there were statistically significant differences between the pre-posttests mean scores of the experimental group in listening comprehension skills (literal comprehension, inferential comprehension, and critical comprehension). In other words, the results of the t-tests proved to be statistically consistent with the above stated hypothesis. Hence, the second hypothesis was accepted.
To investigate the differences between the experimental group students' listening comprehension skills before and after being exposed to the Thinking Maps program, t-tests for paired samples were used for each determined listening comprehension skill. The following table shows the existence of statistically significant differences at 0.05 level in this respect.
(134)
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
Table (3)
t-test results comparing the pre-test vs. post-test means for the experimental
group in each listening comprehension skill.
Listening comprehension Skills
sub skills
group
M
S.D.
Mean diff.
S.D. for diff.
T value
D.F.
Sig. at 0.05
Effect size
Literal comprehension
Identifying specific stated information or details
experimental
2.920
.2769
1.140
.7147
7.975
24
Sig.
3.26
control
1.780
.5788
Identifying organizational patterns within a spoken discourse
experimental
2.820
.2843
1.060
.6970
7.604
24
Sig.
3.10
control
1.760
.5795
Sum
experimental
5.7400
.50249
2.200
1.323
8.315
24
Sig.
3.39
control
3.5400
1.0985
Inferential comprehension
Interpreting the uses of different intonation
experimental
3.900
.4082
1.260
.6633
9.498
24
Sig.
3.88
control
2.640
.5107
Guessing the meaning of unknown words or phrases
experimental
2.360
.4899
.9200
.5715
8.048
24
Sig.
3.29
control
1.440
.5066
Making inferences
experimental
2.800
.2887
.7600
.5025
7.562
24
Sig.
3.09
control
2.040
.4062
Sum
experimental
9.0600
.78156
2.940
1.244
11.814
24
Sig.
4.82
control
6.1200
.75388
Critical comprehension
Drawing conclusion
experimental
2.480
.5099
1.200
.7071
8.485
24
Sig.
3.46
control
1.280
.4583
Over all sum
experimental
17.280
1.3924
6.340
2.564
12.360
24
Sig.
5.05
control
10.940
1.6473
The above table shows that there were statistically significant differences at 0.05 level between the mean scores of the experimental group on the pre-post tests in favor of the post-test in the six listening comprehension skills, since the estimated t-values for the literal listening comprehension were (7.975) for identifying specific stated information or details, and (7.604) for identifying organizational patterns within a spoken discourse, and for the inferential listening comprehension (9.498) for interpreting the uses of different intonation, and (8.048) for guessing the meaning of unknown words or phrases, and (7.562) for making inferences, and for the critical listening comprehension (8.485) for drawing conclusion. Moreover, the effect size values (3.39 ), (4.82) and (3.46) for the literal, inferential and critical listening comprehension skills respectively reveal that the proposed Thinking Maps program had a large effect on the experimental group students' listening comprehension skills on the post-test as compared to the pre-test.
Discussion
Implementing Thinking Maps helped students learn more effectively and efficiently, enabling learning objectives to be covered in less time and with great retention. Thinking Maps are used by teachers to assess
Manal Lotfy Kabesh
( )
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
progress, gauge knowledge, track performance, and even assess their own lessons as they discover what students have learned from class.
Students found that ideas can be connected and realized how information can be grouped and organized with visual learning and Thinking Maps. Moreover, new concepts are more thoroughly and easily understood when they are linked to prior knowledge, this is consistent with Hyerle(1996),Hyerle (2011),& Gerlic& Jausovec (1999).
It is noteworthy that benefits of using Thinking Maps have become apparent throughout the program as students and teachers share a common language that improves communication and facilitates the learning process. Moreover, most students demonstrated improvement in their ability to organize their thoughts. Also, the quality of learning has been taken to a higher level, activities have become more meaningful and relevant, this is consistent with the study of Goneim(2012).
Many students' attitudes have become more positive toward learning. Thinking Maps help students organize and analyze information. Students better remember information when it is learned both visually and verbally as this leads them to make connections, understand relationships and recall details ,this is consistent with(Benson, 1997,and Abdel Aziz,2013).
It is noteworthy that Mind Maps help break the complexity of the text passage into words for easier understanding and enhance remembering the comprehension passage better through establishing links, pattern, relationship among concepts and information, this is consistent with(Moi and Lain, 2007 & Sunseri,2011).
Using Thinking Maps facilitates greater listening comprehension. Thinking Maps help students structure and order their thinking by creating a visual representation of concepts and their understanding. As highlighted in the current study, implementing Thinking Maps proved to be influential in enhancing students; motivation and involvement in listening activities, thus fostering their listening comprehension skills, this is consistent with Helal(2006).
Recommendations and suggestions for further research :
 Fostering Thinking Maps awareness and developing an Mapping should be a part of early childhood educators' commitment to helping children move towards using their fullest potential.
 Teachers should play a vital role in encouraging Thinking Maps to enhance learning, and must create an atmosphere that encourages creativity as well as Mapping development in the classroom.
(136)
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
Therefore, teachers must prepare conditions or settings that enhance creative and aesthetic development.
 .Listening comprehension instruction should be given more attention in our EFL classes. More time and efforts should be exerted to develop this main skill and its sub-skills.
 In order to foster an Thinking Maps pedagogy in the classroom, students must develop an awareness of their own responses, affective, imaginative, and intellectual, to listening comprehension and other English language skills.
 Teaching listening comprehension must be engaging as well as challenging.
 More studies are needed to compare the relative efficacy of using Thinking Maps to address different genres and other language skills.
 Further research is necessary to explore and investigate the effectiveness of applying similar Thinking Maps activities with different stages and student populations over a longer period of time on students' skills especially on listening comprehension sub-skills.
Manal Lotfy Kabesh
( )
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
References
Abdel Aziz, M.(2013). Mind Mapping and Its Effect on Developing Reading Comprehension of Preparatory stage Students. Occasional Papers In the Development of English Language Education. Volume 55 January2013. Ain Shams University. Faculty of Education. Center for Developing English Language Teaching.
Benson, P. (1997). Problems in Picturing Text : A Study of Visual / Verbal Problem Solving. Technical Communication Quarterly,6(2),141-161.
Camp, E. (2007). Thinking with Maps. Philosophical Perspectives.21, Issue 1 (145-182).
Chamot, A. (1990). Learning Strategy Instruction in the Foreign Language Classroom : Listening. The Interstate Research Associates, MClean,VA. Center for International Education, Washington DC. Retrieved from ERIC database (ED:343441).
Cooks, J. & Sunseri, A. (2014). Leveling the Playing Field: The Efficacy of Thinking Maps on English Language Learner Students' Writing. The CATESOL Journal 25.1.201333/2014.
Fang, X. (2008). Listening Comprehension in EFL Teaching. US-China Foreign Language,6(1),21-29.
Feyten, J. (1991). The Power of Listening Ability :An Overlooked Dimension in Language Acquisition .The Modern Language Journal. Vol.7, No.2, 173-180.
Ghoneim, N. (2012). The Effectiveness of Thinking Maps on the Preparatory Students' Development of EFL Inferential Reading Comprehension. Occasional Papers.In the Development of English Language Education. Ain Shams University, Faculty of Education, Center for Developing English Language Teaching, Cairo, Volume 54,158-176.
(138)
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
Goleman, D. (2000). Primal Leadership: Realizing the Power of Emotional Intelligence. Boston, MA: Harvard Business School Press.
Grant, L. (1997). Listening to English in the 21st Century: The Need for Learner Strategies. In C. Zaher (Ed.)Proceedings of the Third EFL Skills Conference : New Directions in Listening. 10-18. The Center for Adult and Continuing Education, The American University in Cairo, Cairo, 3rd-5th. December, 1996.
Hasan, A.(2000). Learners Perceptions of Listening Comprehension Problems. Language, Culture and Curriculum.Vol.13,No.2. 137-153.
Helal, E. (2006). The Effectiveness of a proposed Program based on Schema Theory in Developing English Language Listening Comprehension Skills among First year Secondary Students (Unpublished Ph.D. thesis). Women's College for Arts,Ain Shams University, Egypt.
Hyerle, D. (2011) Student Successes with Thinking Maps, Second Edition. Thousand Oaks. Corwin Press.
Hyerle, D. (1996). Thinking Maps: Seeing is Understanding. Educational Leadership. vol. 53, No. 4, January 1996.
Hyerle, D., Costa, A.L. & Marzano, R. L. (2009). Visual Tools for Transforming information Into Knowledge. Corwin Press. Second Edition.
Hyerle, D. (1993). Thinking Maps as Tools for Multiple Modes of Understanding. Ph.D. Thesis in Education. University of California. Berkeley. School of Education.
Hyerle, D. (2009). Bifocal Assessment in the Cognitive Age: Thinking Maps for Assessing Content Learning and Cognitive Processes. The New Hampshire Journal of Education (Plymouth State University and ASCD),32-38. Retrieved from http://www.thinkingfoundation.org/david/books/T
Manal Lotfy Kabesh
( )
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
MapsAssessJournal.pdf.
Hyrele, D. (1996).Visual Tools for Constructing Knowledge. Alexandria,VA: Association for Curriculum and Development.
Hyrele, D. & Yeager, C. (2007). Thinking Maps: A Language For Learning. Cary, NC: Thinking Maps.
Jalongo, M. (19991).Strategies for Developing Children's Listening Skills. Bloomington, IN.:Phi Delta Kappa Educational Foundation
Jensen, E.(2004). Brain Compatible Strategies. San Diego, CA. The Brain Store.
Jun, Z. (October 2000). Warm-up Exercises in Listening Classes. The Internet TESL Journal. Vol. VI, No.10, 1-3.
Kingen, S. (2000). Teaching Language Arts in Middle Schools. Connecting and Communicating. New Jersey: Lawrence Erlbaum Associates.
Lynch, T. (2005).Listening In Kaplan, R.,Grabe, W. Swain, M, & Tucker, R. (Eds.).The Oxford handbook of Applied Linguistics. (pp, 39-46). Oxford University Press.
Mendelsohn, D. (1995). Applying Learning Strategies in the Second/ Foreign Language Listening Comprehension Lesson. In D. Mendelsohn & J. Rubin (Eds.) A Guide for the Teaching of Second Language Listening.132-150. San Diego, CA: Dominie Press, Inc.
Moi,W. & Lain,O.(2007). Introducing Mind Map in Comprehension. Educational Research Association (Singapore).
Novak, J. D. & Gowin, B.D. (1984). Learning How to Learn. Cambridge: Cambridge University Press.
Nunan, D. (1997).Approaches to Teaching Listening in the Language Classroom. A Paper presented to the 1997 Korea TESOL Conference.
Nunan, D. (1999). Second Language Teaching & Learning. Boston :
(140)
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
Heinle & Heinle Publishers.
Oxford, R. (1993). Research Update on L2 Listening. System. Vol. 21, No. 2, 205-211.
Richards, J.C.(2008). Teaching Listening and Speaking from Theory to Practice. Cambridge: Cambridge University Press.
Rubin, J. (1994).A Review of Second Language Learning listening Comprehension Research. The Modern Language Journal.Vol.78,No.2, 199-221.
Scarcella, R. & Oxford, R.(1992).The Tapestry Approach of Language Learning. The Individual in the Communicative Classroom. Boston: Heinle & Heinle Publishers.
Sherman, L. (2011). Postmodern Constructivist Pedagogy for Teaching and Learning Cooperatively on the Web. Cyber Psychology & Behavior. 3, 51-57.
Sousa, D. (2000). How the Brain Learns. Alexandria, VA: Association for Supervision and Curriculum Development.
Sunseri, A. (2011). The Impact of Thinking Maps on Elementary Students' Expository Texts. Ph.D. faculty of San Francisco State University.
Torky, Sh.(2010).The Effectiveness of a Strategy based on Students' Generated Questions in Developing University EFL Students' Listening Comprehension Skills. Studies in Curriculum and Instruction. Egyptian Council for Curriculum and Instruction, 166,2-44.
Underwood, M. (1990)Teaching Listening.(2nd. edition). London : Longman.
Vandergrift, L. (March 2003).From Prediction through Reflection : Guiding Students through the process of L2 Listening. The Canadian Modern Language Review,Vol.59,No.3, 425-440.
Wolfe, P.(2001). Brain Matters : Translating Research into Classroom Practice Alexandria, VA. Association for Supervision and Curriculum Development.
Manal Lotfy Kabesh
( )
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721
Yaw, TH.,(2014). The Impact of Primary Students' Multiple Intelligences on Motivation in Thinking Maps Classroom. M.SC. Faculty of Cognitive Sciences and Human Development. University Malaysia Sarawak.
(142)
Occasional Papers
Vol. 62(Dec. 2016)
ISSN 1110-2721