Business schools are exploring new pedagogical approaches to learning in order to deal with challenges such as increased class sizes, limited funding support, and difficulties in facilitating and encouraging active participation and learning among a diverse cohort of students. This paper reports on a study of the effectiveness of a pedagogical approach that blends online discussion board and case study. Analysing quantity and quality of online postings and comparing accounting students’ performance with previous cohort, this study observes a significant improvement in student learning. Appropriate design and delivery strategies and clear assessment criteria for assessment and use have provided an effective learning vehicle for students, helped them overcome their own language related barriers, and encouraged them to participate in a nonthreatening environment. This approach further complemented the benefits of peer-to-peer learning and case study pedagogy. Reported increase in workload for students and marking load for academics and measuring the value of learning, however, are some of the challenges that need further attention by researchers.
The benefits of using online discussion forums and peer to peer learning for enhancing student learning are well known. Other than in full online courses, their adoption in traditional learning environments complementing face-to-face teaching is increasing [
Further, difficulties of devising an online assessment and incorporating the benefits of case study pedagogy into this online forum have posed further challenges to educators in the design and effective management of these activities [
Increasing class sizes, reducing resources, and widening diversity of students’ cohorts have all placed demands on higher education and stimulated them to explore new pedagogies [
Online discussion forums are considered an extension of traditional learning that promotes dialogue, reflection, knowledge construction, and self-assessment [
Two studies have examined how students perceive online case method pedagogy in business education. According to study by Jonassen and Kwon [
In addition to the above, other studies have reported challenges of online learning. For example, Valaitis et al. [
Threaded forums are one of the most commonly used environments for online discussion, and, though well supported by learning theories, several limitations are identified by researchers. Excessive focus on new posts, overlapping exchanges and hierarchical structure of discussion threads [
Even though learning through online discussion forums was studied by several researchers as shown above, research that focuses on the online case-method pedagogy and assessment is limited. Further, findings on the effectiveness of online discussion forum are also inconsistent and depend on the nature of discussion environments. The experimental, quasiexperimental, and descriptive studies relevant to this question have not produced coherent knowledge about how and when online discussion groups perform better or worse [
Lack of participation by a majority and dominance of a few handful students are the main challenge in asynchronous discussion forums. As assessment is the currency that students deal in [
Even though collaborative learning is critically important, and that the technology is a powerful enabler that match what is needed for discussion and collaboration, the extent to which asynchronous discussion forums succeeded in enabling learning is not known and lacks sufficient empirical evidence where a case study assessment is incorporated. Rather than comparing face-to-face and online groups, unlike previous studies, this study directly investigates online groups engaged in case based discussions. The findings of this study will contribute to the literature on case study pedagogy in online context, an area where existing research findings are limited [
An online discussion of case study is designed as an important assessment component and incorporated in this unit using some of the strategies suggested by Rollag [
The conceptual content that is covered in this subject includes accounting processes including accounts receivable, accounts payable, expenditure cycles, and costing processes and other business processes including procurement and sales. In addition, it also includes making a business case for the selection and implementation of a packaged business software, that is, enterprise system. The unit incorporates variety of teaching methods that include lectures, case study discussions, laboratory demonstrations of the SAP software, and hands-on exercises on the SAP software and emphasizes both conceptual and theoretical content as well as procedural knowledge of the software.
This online case study discussion is one of the key assessment components and weighs 25% of the total assessment. It requires students to answer the questions that deal with the challenges in the selection, evaluation, and implementation of a packaged business enterprise system software and challenges in postimplementation. The objective was to help students demonstrate their ability to work collaboratively using online discussion forum moderated by the lecturer and resolve their differences in developing an optimal solution using synergies generated through collective wisdom and peer-to-peer learning. It requires students to participate in an online discussion forum on a weekly basis in a semester that spans 13 weeks. During this time, they are expected to demonstrate their individual ability to apply the concepts and theories of enterprise systems to a business organization and suggest workable solutions to the problems/issues raised in a comprehensive case study.
At the beginning of the semester, a sheet detailing the expectations of students and the assessment criteria was given to students. This assessment criterion was designed considering the learning objectives of this assessment component, practicality of administering the grading, and key principles of assessment design. The dimensions and their descriptors for grading purpose were developed from several sources such as Oxford Brookes University’s business assessment criteria grid, Washington State University’s guide to rating integrative and critical thinking, and this university business school graduate attributes. The criteria thus developed consist of five dimensions—(i) identification of issues and problems, (ii) consideration of context and application of theories and principles, (iii) analysis of data and evidence in the case study, (iv) effective written communication, and (v) responsive contribution and integration.
Students were given a grid that explains each of the dimensions in a scale of 1 to 6 (1 to 2 = Poor; 3 to 4 = Average; and 5 to 6 = Excellent) with some descriptors. For example, the dimension of “responsive contribution and integration” is described as “Poor” when the student “
Four questions were set on the case study. In order to narrow down their focus on one aspect at a time and to concentrate on the issue at hand, these questions were posted one after another in the online blackboard so that students do not have any idea of the questions and specific details that would appear subsequently. In addition, the objective was to synchronize the questions with the lecture materials and to manage the flow of the discussion. Students, however, can see all the responses to the earlier questions and see how it is building up concepts. Facilitator was monitoring the discussion and, depending upon the need, provided clarifications, asked leading questions, corrected major deviations, and kept the discussion focused on the main issues.
The first question seeks students’ analysis on the challenges the case study firm faced in the selection and implementation of an enterprise system (ES) software solution and evaluation of implementation strategy and software-business fit. The second question deals with the evaluation of the firm’s attempt to change business processes, problems it faced, and its relationship with the adoption of enterprise system and the strategies employed. The third question requires students to analyse the current organizational structure of the firm and develop appropriate mapping structure to be able to configure the enterprise system for the necessary processes and the fourth question involves analysis of the suitability and challenges in selecting and implementing extensions to the enterprise system.
Each student is expected to post up to a maximum of 4 responses for each question. Even though this limit is set, the number of postings per question is not a criterion in the assessment and focus is on the quality of postings. Opportunity to post four times is expected to encourage students to reflect on their responses, read others’ responses, and build on them before posting the next one. In addition, they were encouraged to read widely outside the text book and lecture notes and provide references to their responses in the forum. Students were expected to post their responses within two weeks in the relevant thread created in the blackboard discussion forum for the first three questions and one week for the fourth question. Once the deadline has passed for a particular question, students were not allowed to edit their responses and/or submit additional response. They, however, are allowed to see the responses already posted by all the students and use that knowledge while answering the next question.
A time line of the online discussion of case study is presented in Figure
Timeline for the online case study discussion.
Students’ responses for each of the questions were the primary data used in this study. This data was analysed by two persons independent of each other—the author/facilitator and a research assistant. This research assistant was a qualified educator, specialized in online learning, and was working in the teaching and learning unit of the business school. The objective of this analysis was to assess the quality of responses and online discussion interaction. Rather than positioning the discussion interaction as a dependent variable along with the learning outcomes, a post hoc measurement approach was used in this study [
Quality of responses by each of the students for each question was analysed for assessment purposes as well as for evidence to support learning. While the first approach helped in awarding assessment mark to each of the responses, the second approach had helped in identifying any evidence to peer-to-peer and collaborative learning, data analysis skills, using and citing other sources, pooling of different ideas, and building on others’ contributions. A comparison of the analysis revealed 81% consistency in the rating of both evaluators (i.e., 155 responses, out of 192 responses, 4 questions
In addition to students’ responses, their perception of the entire process was also collected using semistructured interviews. The approach taken in the analysis was to analyse the responses of students to semistructured questions, to examine their feedback on the effectiveness of this online assessment task, and to assess its pedagogical benefits. Further in the interviews, participants were asked to give their views on the structure of the learning environment including the online discussion forum, appropriateness of the weightage assigned to this assessment task, timing and level of difficulty of the case study questions, and participation by the academic as a moderator and their perception of collaborative and peer-to-peer learning. From a total sample of 48 students, 12 students have volunteered to be interviewed and provided such qualitative feedback.
Demographics of the 48 respondents participated in this study are presented in Table
Demographics of respondents.
Characteristic of respondents | Percentage | |
---|---|---|
1 | Proportion of international students | 68% |
2 | Students from accounting discipline | 61% |
2 | Students from other disciplines such as information technology, business information systems, and logistics | 39% |
3 | Percentage of students currently employed | 38% |
4 | Percentage of students below 25 years of age | 65% |
5 | Percentage of students with previous work experience | 42% |
6 | Percentage of students who have completed more than 4 business related units/subjects | 53% |
7 | Percentage of students who have completed at least one IT/IS related subjects | 88% |
Previous knowledge of business functions, business processes, information systems, and technologies gained in other units is expected to determine the prior content knowledge student may have and therefore could have an influence on their learning experience in this unit. Therefore, data on the completion of previous units before enrolling in this unit was collected and presented in Table
Each participant is expected to submit responses to four questions posted in the online discussion forum. There were 268 valid responses for all the four questions, that is, on average, 67 responses per question or 1.40 responses per question per student. As shown in Table
Number of responses posted by student.
Number of responses posted | Question 1 | Question 2 | Question 3 | Question 4 | ||||
---|---|---|---|---|---|---|---|---|
No. | % | No. | % | No. | % | No. | % | |
Zero | 1 | 2% | 0 | 0% | 0 | 0% | 0 | 0% |
One | 42 | 88% | 36 | 71% | 28 | 60% | 40 | 83% |
Two | 4 | 8% | 8 | 17% | 13 | 25% | 8 | 17% |
Three | 1 | 2% | 3 | 6% | 7 | 11% | 0 | 0% |
Four | 0 | 0% | 1 | 2% | 1 | 2% | 0 | 0% |
Except for the fourth question, all the remaining questions are content based and deal with subject specific issues such as enterprise systems implementation challenges, accounting related business processes, challenges of integration, and critical success factors. The fourth question also deals with the adoption of enterprise system and requires students to develop an artefact, a mapping structure for implementing accounting processes (accounts receivable, accounts payable, and general ledger). Even though each student is allowed to post up to four responses to each of the questions, very few posted more than two responses as shown in Table
In terms of timing, about 50% of the respondents posted their response on the due date, while about 25% of students posted in the first two days, as shown in Table
Timing of responses posted by students.
Timing of responses posted by students | Q1 | Q2 | Q3 | Q4 |
---|---|---|---|---|
First response posted in the first two days after question is made available | 20% | 28% | 37% | 39% |
First response posted on the due date | 55% | 40% | 32% | 32% |
Response posted between day 3 and the last day | 25% | 32% | 31% | 30% |
|
||||
Total responses | 100% | 100% | 100% | 100% |
Analysis of the responses suggests that students were hesitant to submit their comments in the initial stages and are generally waiting for a leader to post first. About 20% of the respondents submitted their first response in the first two days while about 55% of the students submitted their response on the last day for the first question. This, however, has improved as students become familiar with the process and started probably realizing the value of collaborative learning. For the third and fourth questions, number of students who submitted their response on the first day increased from 20% to 37%, while the percentage of students who submitted their response to the fourth question on the last day has improved from 55% to 32%. Thus, there is clear improvement in the frequency of participation and the intensity of participation in the online discussions. Importantly, there is no topic decay as observed in previous studies [
Based on the criteria set out for assessment purposes, the quality of written responses was analysed and a summary is presented in Table
Summary of analysis.
Content analysis | Q1 | Q2 | Q3 | Q4 |
---|---|---|---|---|
No evidence that the student read/understood others’ ideas/contributions | 56% | 44% | 54% | 34% |
Responding to others contribution by elaboration/critique or building-up | 15% | 21% | 31% | 36% |
Evidence of good analysis of the case study data | 32% | 41% | 39% | 44% |
Evidence of using and citing other sources/references in the response | 17% | 18% | 21% | 24% |
Demonstrable understanding of the questions and identification of issues in case study | 21% | 27% | 33% | 39% |
Effective written communication | 64% | 72% | 56% | 71% |
Evidence of pooling different ideas, expertise, and information in the development of an artifact |
NA | NA | 45% | NA |
Analysis of the responses as shown in Table
It is necessary to measure the impact of this initiative on learning. Academic performance, though, is considered one of the variables that could measure the effectiveness of learning; it is not possible to conclusively prove this without a control group. Though not the most appropriate method, grading of the current cohort of students in this assessment task (i.e., case study) is compared with the performance of a previous cohort of students in the same unit with the same case study. In the past, students were asked to submit a written analysis of the case study after face-to-face discussion in the classroom. With the characteristics of the cohort of students considered generally similar every semester, a comparison is made. Simple comparison reveals improvement in the performance of students in this assessment component. Table
Performance in the assessment task.
Performance in the assessment task | Current cohort of students (with online discussion forum) | Old cohort* | ||||
---|---|---|---|---|---|---|
Q1 | Q2 | Q3 | Q4 | Overall | ||
Average and below average performance | 60% | 46% | 40% | 38% | 46% | 77% |
Credit performance | 34% | 40% | 38% | 36% | 37% | 16% |
Distinction performance | 6% | 14% | 22% | 26% | 17% | 7% |
|
||||||
Total | 10 | 100 | 100 | 100 | 100 | 100 |
Grading data shown in Table
In order to determine whether this model of learning (case study analysis through online discussion forum) has contributed to improvement in overall learning or not, a comparison was made. Given the absence of control group, a simple comparison with the previous cohorts of students where no such asynchronous medium was used is made, fully acknowledging that this is not the appropriate method. Except for this assessment component, all the remaining aspects of the unit were exactly similar to the current one in terms of the learning outcomes, content, delivery methods, and general assessment components. For the previous cohort of students, a case study analysis was used as an assessment component, while the same case study analysis is used to the current cohort of students using asynchronous medium for discussion.
Using independent samples
Variable (posttest) | Group |
|
Mean | S.D. |
|
|
---|---|---|---|---|---|---|
Performance in the case study task | Online assessment (experimental group) | 48 | 65.94 | 10.193 | 5.065* | 0.000 |
Normal assessment (control group) | 42 | 56.43 | 7.096 | |||
|
||||||
Overall teaching evaluation score | Online assessment (experimental group) | 48 | 4.50 | 0.505 | 3.698* | 0.000 |
Normal assessment (control group) | 42 | 4.10 | 0.532 |
Subject evaluation score in this unit has marginally improved from 4.1 to 4.52 in a scale of 1 to 5. The curriculum and delivery of this unit are quite different from other standard units delivered by this school. This unit incorporates SAP, an enterprise system software solution, and teaches students the concepts of business process integration and enterprise systems along with SAP software skills. The curriculum incorporates hands-on laboratory sessions and these hands-on components typically made this unit interesting and contributed to good evaluations, that is, 4.10, before introducing this online discussion assessment. The introduction of online discussion forum has further contributed to improvement in learning as well as in students’ satisfaction. In qualitative feedback, a significant number of students (62% of them) pointed out online discussion forum as an important learning component. Informal feedback and comments from students also reflect a generally positive endorsement to the online discussion. They have observed that the exchange of views and ability to asynchronously post their responses online have encouraged them to interact more and better and facilitated their learning. They have acknowledged the benefits of peer-to-peer learning which they believe would not have happened without this online discussion. This online discussion has also helped them better in deriving the benefits of case study pedagogy. Thus, significant improvement in the quality of responses and observed improvement in students’ academic performance on this particular assessment task indicate the contribution of this strategy to deep learning, given the structure of the assessment and content.
In spite of positive endorsement and observed improvement in the learning, some challenges are identified. They include reported increase in students’ workload because of the online discussion forum, inability to freely express their ideas and views because of the assessment involved, restriction to the number of postings per student per question, inadequate number of responses by some students, poor integration and synthesis of others’ ideas by students, insufficient moderation by the lecturer, and some subject related issues. About 85% of the participants believed that the weightage given to the assessment task was appropriate and motivated them to actively engage in the discussion and learning. Some students, however, viewed the weight given to this single assessment component (25% of the overall assessment weight) as a major constraint in freeing up the discussion. Further, academic administration of this online discussion forum has been a challenge. With each student posting on average 1.4 postings per question, the total number of postings has reached about 270 and enormously increased the marking load to the academic. In addition, the need to monitor the discussion forum, edit some unhelpful comments, redirect the debate to the issues relating to the subject content, and provide hints has added to the normal teaching and administrative load. In addition to this, some of the typical challenges in case study pedagogy have also resurfaced during the discussion. They include request for one single answer to the case study questions, inability to comprehend the case study materials by some, inadequate English language skills of students, inability to apply theoretical frameworks, and skills in scoping out, assumptions, and analysis.
Teaching in emerging disciplines such as business process management and enterprise systems in business schools is challenging because of the dynamic nature of the subject content, diverse nature of student cohorts, nonavailability of localized case studies, required multidisciplinary focus, and other institutional factors such as large class sizes and increased competition for scarce resources and students. Integrating e-learning technologies such as discussion forums with the traditional case method pedagogy has the potential to change educational processes and enhance the quality of learning in this environment. Combining online discussion forum and case study analysis offered students a high-quality learning environment. According to study, this blended model encouraged collaborative learning and contributed to improvement in cognitive learning. By transferring discussions to an online environment, opportunities for learning from each other have improved. Data suggests that students perform better in an environment where a mixture of classroom and online technologies is employed and there is no decay in the interest on the topic. In general, students perceived improvement in learning because of the online environment and appreciated the opportunities it has provided for deeper understanding of content based issues, appreciation of multiple views, and reflection of their own learning and perceptions. Though it is early days in understanding the effectiveness of these blended learning models, their contribution to the improvements in the quality of discussion and the depth of learning in a higher educational environment appear to be significant and can be further expanded by imparting better argumentation and integrative skills to students. The workload involved in setting up, monitoring, and assessing the online discussions, though, is significant; it is possible to make it less burdensome by efficient design and structure of the initiative using guidelines well established now and with appropriate faculty workload policies. Further studies on the influence of students’ knowledge and skills, development of argumentation skills for students and moderators’ skills, and lecturers and students’ actual use of synchronous and asynchronous communications, interactions, and reflections would help in designing a more effective online discussion environment and assessment strategies in business schools.
The author declares that there is no conflict of interests regarding the publication of this paper.