«The Purpose of Focus Groups in Ascertaining Learner Satisfaction with a Virtual Learning Environment Yana I Tainsh University of Greenwich, London ...»
The Purpose of Focus Groups in Ascertaining Learner
Satisfaction with a Virtual Learning Environment
Yana I Tainsh
University of Greenwich, London
Abstract: This paper examines the contribution of focus groups in evaluating learner satisfaction with a Virtual
Learning Environment (VLE). It explores the views of a group of introductory level Post Compulsory Education
learners that have a history of disaffection, impoverished learning and challenged written and communication skills. The outcome of this study will be used to inform future VLE material design for inclusion in a School policy document. Additionally, the findings will contribute to the development of both a broader range of discrete ICT programs delivered by a VLE and embedded ICT within a range of vocational qualifications across the Post Compulsory Education Vocational Curriculum.
Key words: virtual learning environment, focus group, disaffection, impoverished learning, satisfaction, post compulsory education, policy document.
1. Introduction The use of a Virtual Learning Environment (VLE) in Post Compulsory Education in Further Education Colleges (FE) has been increasing incrementally over the last five years. Having begun life in Universities and the more ‘traditional’ higher education institutions, VLEs are flexible, accessible and encourage the development of communities of practice. They encourage group activities, peer support and electronic delivery of learning but are not intrinsically designed to aid those learners at the lower end of the academic spectrum. This paper presents a case study of learners on an introductory (Level 1) FE course in ICT ascertaining their level of user satisfaction with a VLE. The outcome of this study will be used to inform future VLE material design for inclusion in a School policy document. Additionally, the findings will contribute to the development of both a broader range of discrete ICT programs delivered by a VLE and embedded ICT within a range of vocational qualifications across the Post Compulsory Education Vocational Curriculum.
Focus groups were chosen as the method of data collection for this study, based on work by Morgan (1988), using a group of learners whose academic backgrounds are similar in qualification and educational history and because they ‘generate hypotheses that derive from the insights and data from the group’ (Morgan 1988, Krueger 1988). Focus groups have been in evidence since the 1920s.
At that time, they were in the guise of survey questionnaires related mainly to products and the customer requirements of a product. During World War II and up until the 1970s, focus groups were used for market research to elicit wants and needs. From the 1980s onwards, the use of focus groups has been used mainly in the health arena and in examining social issues. Since then social scientists and program evaluators have found focus groups to be useful in understanding how or why people hold certain beliefs about a topic or program of interest. Krueger and Casey (2000) identified that focus groups can be used for program development and evaluation, planning, and needs assessment. Powell and Single define a focus group as ‘a group of individuals selected and assembled by researchers to discuss and comment on, from personal experience, the topic that is the subject of the research’ (1996) and rely on ‘interaction within the group based on topics that are supplied by the researcher’ (Morgan 1997) confirming that focus groups are an ideal opportunity to elicit information from learners in a safe, non-threatening environment (Krueger 1988). Morgan (1998, pp58) says that ‘the conversations in focus groups can be a gold mine of information about the ways that people behave and the motivations that underline these behaviours.’ David Morgan’s book ‘The Focus Group Guidebook’ (1998) has been used throughout this study as the empirical work containing the methodological approach and the validated data-gathering instrument for the Focus Groups. When designing the VLE interface, factors such as navigatability, learning activities and resources for use within the computer-mediated environment (as is a VLE) were instrumental in the learner satisfaction of the VLE. The work of Robert Gagné (1965) is critical in any study using computers for learning and illustrates the importance of his Instructional Design (ID) ISSN 1479-4403 157 ©Academic Conferences Ltd
Reference this paper as:
Tainsh Y I (2007) “The Purpose of Focus Groups in Ascertaining Learner Satisfaction with a Virtual Learning Environment” The Electronic Journal of e-Learning Volume 5 Issue 2, pp 157 - 164, available online at www.ejel.org Electronic Journal of e-Learning Volume 5 Issue 2 2007 (157 - 164) to an underlying theory of computer-based learning. Gagnés theory stipulates that there are several different types or levels of learning. The significance of these classifications is that each different type requires different types of instruction. Gagne identifies five major categories of learning: verbal information, intellectual skills, cognitive strategies, motor skills and attitudes (web link accessed on 18/06/06). Gagné’s Nine Events of Instruction (a sequence of learning events borne out of his ID theory) addresses learners in terms of the logical steps that are mapped to the way they learn. The events are; gain attention, inform the learner of the objective, stimulate their recall of prior learning, present the stimulus, provide learner guidance, elicit performance, give feedback, assess performance and enhance retention and transfer of learning. Using this premise, designing a VLE with learners for whom written communication is difficult and numerical skills underdeveloped, requires ‘constant revising and adjusting of our uses of technology to better meet the needs of the program and our students’ (Bucci et al 2003), yet is ideally matched to Gagné’s theory of developing computerbased learning using his ID to change the capabilities of the learners.
By the very nature of the flexibility that can be achieved in designing a VLE, a vast range of stimulus can be embedded and these will address the learners preferred learning style and learning processes, which are not completely understood, and are different in detail from one person to another (Bostock 2005). This is clearly evidenced in this case study and its learners, where a detailed analysis of pre-VLE knowledge was essential as well as a breakdown of the learning style of the learners so as to maximise their achievement and success against specified learning objectives. The materials available on the VLE for this case study only mapped as far as Instructional Event 6 - Elicit performance (practice). Future studies are planned whereby the full set of Instructional Events is included in all ICT courses delivered via a VLE. An intrinsic part of the study was the collection of data from pre- and on-course diagnostic tests of the learners in the study particularly their literacy and numeracy level (www.keyskills4u.com), preferred learning style (www.vark-learn.co.uk) and a review of their educational history (Individual Learning Plan). Results identified that more than 30% presented with challenged written and communication skills that would require specialist intervention and support throughout their studies. The VARK (Fleming 2001) online multiple-choice learning style assessment profiled the learners learning styles as mainly kinaesthetic and auditory. Typical of auditory learners would be a preference to attend lectures, listen to speaking and like to read aloud.
For kinaesthetic learners, their preference is to be ‘hands-on’ with practical activities and watch (and be part of) demonstrations. The learners’ prior educational histories contained instances of exclusion and disaffection with their 11-16 education, many learners not completing their year 10 or 11 studies.
Their home postcodes indicating that they live in some of the lower socio-economic housing, mostly council owned, with 3 learners living in ‘poor’ housing in southeast London. All learners in the study completed a pre-VLE questionnaire (Table 3) that identified their personal profile, general level of computing experience (including using the internet/web), concerns they had about using a VLE in their learning and were asked to rank a number of learning activities/resources they had previously used in their educational history. This was used to inform the design of the interface of the VLE and the resources/activities within it. Care was taken in the design process such that there was ease of navigation for the learner, and a simple hierarchical structure for the relationship between the course elements to encourage the learners to make appropriate connections within or between each resource and/or activity. Usability, flexibility and pedagogy attributes were considered at all stages of the VLE design as content management shortcomings militate against making any improvements once the VLE is ‘live' especially if those improvements involve structure (Vogel 2006).
2. Methodology This study used a mixed methods data collection approach where agency was given to both quantitative (Likert-scale questionnaire) and qualitative data (Glaser and Strauss 1967) thereby triangulating the results. Creswell (2003, p208) makes clear that the perceived legitimacy with which the mixed methods approach is being promoted is expanding. This is supported by detailed reference to mixed methods studies in areas as diverse as occupational therapy and AIDS prevention. A new ‘Handbook of Mixed Methods in the Social and Behavioural Sciences’ (Tashakkori and Teddlie, 2003) is cited within Creswell (ibid) as the foremost publication in the field of mixed methods research. In this case study, 39 students were actively recruited and all completed the pre-VLE questionnaire before being exposed to the VLE itself. The issue of ethics is of paramount consideration in a focus group study. Informed consent must be gained prior to any focus group activity in either written or oral form. The participants must be told of the consequences of the research and care must be taken to reduce any harmful effects of the research on the participants. The usefulness of the research cannot
be under-estimated and this must be brought to bear on the participants in terms of their benefit and that they can get involved in the change that will result from the study findings. Learners in this study signed an agreement that gave permission for transcription and hard copy storage of the focus group discussions. All participants in the study had sight of the transcripts at the final stage before coding and were able to discuss changes they felt needed to made where meaning or incorrect transcription had occurred. Whilst the learners knew each other as a group on the same ICT Program, the focus group selection process ensured that the ‘friendship groups’ that existed were separated to allow full and frank discussions to take place (Krueger and Casey 2000). Focus groups involve not only ‘vertical interaction’ or interaction between the moderator and the interviewees, but also ‘horizontal interaction’ among the group participants (Denzin and Lincoln 2003). Data emerges from the interaction of the learners in the focus groups, in a language that is native to them alone (Fine 1994). This then has to be delicately decoded to elicit the themes contained therein. Kitzinger (1994) explains that the group situation creates the group’s own hierarchy of importance, their own words and language.
Focus groups are a collectivistic method as defined by Denzin and Lincoln (2003) based on theoretical and methodological considerations. For example; consideration has to be given to how many groups will be held? How many people will be involved in each group? The focus group is an unstructured interview guide with introductory questions. There is an overt need to state that there are no ‘right’ or ‘wrong’ answers. That it is perfectly acceptable (in fact preferred) if there is disagreement on topics. There is much controversy about prior analysis by the researcher of the situation in which the subjects have been involved. Merton and Kendall say: ‘Foreknowledge of the situation obviously reduces the tasks confronting the investigator, since the interview need not be devoted to discovering the objective nature of the situation. Equipped in advance with a content analysis, the interviewer can readily distinguish the objective facts of the case from the subjective definitions of the situation’ (Merton and Kendall, 1946). In a study by Hart (2001), 8 focus groups were used and looked at the experiences and impressions and relationship teenagers in public schools in the United States had with computers. The learners were in public school in Florida, Maryland and Illinois. One of the most surprising findings was that the learners felt that the quality of their education depended on the teacher and not the technology as a better way to learn. This is supported in part by Ainley et al (2000), whose studies showed that learners, teachers and parents felt that computers have a positive effect on learning. US Research shows that the presence of computers and Internet at home has a strong positive association with academic outcomes of school children, particularly children from disadvantaged backgrounds (Wilhelm et al 2002). Interestingly, a study by researchers at the National Centre for Social and Economic Modelling (NATSEM) found that educational attainment of an individual was a stronger predictor of having home computers and the Internet than income (Hellwig and Lloyd 2000).
3. Research design
The group of 39 learners in this case study exhibited the following characteristics:
5 Concerns that learner had about using a VLE in their learning The focus group meetings were held during the learner’s weekly group tutorial sessions (one hour long) over a period of six weeks. Rooms were allocated that were comfortable and non-threatening and informal debriefs were set up so the participants could wind down post-focus group session. Data was collected against the questions asked using notes and then (latterly) recordings that were transcribed and coded. Full focus group questions reproduced below.
Table 4: Focus group questions (based on Morgan 1998, and Vogel 2006) Question Question asked/data collected no.