The Impact of Instructional Methods on Critical Thinking : A Comparison of Problem-Based Learning and Conventional Approach in Engineering Education

Recent research reveals a gap in determining the effect of problem-based learning (PBL) on students’ critical thinking ability. Existing evidences from previous research have been equivocal to be deemed conclusive, especially in across multiple educational disciplines. This paper investigates the effect of PBL on students’ critical thinking ability, as compared to conventional approach. The study employed an experimental pretest and posttest with control group design. The participants were 53 undergraduate students in electrical engineering course, who attended Electrical Technology Module (ET101) in their first semester. Participants completed the pretest and posttests of the translated version of the Cornell Critical Thinking Test Specimen set. The results suggested that students’ critical thinking ability in the PBL group had not been significantly different from their counterparts in the conventional approach group. This paper further discusses the implication of problem-based learning in engineering education.


Introduction
Critical thinking is in the family of higher cognitive abilities, along with creative thinking, problem solving, and decision making [1].Critical and creative thinking are intimately connected and integrated in producing an effective thinking and problem solving [2].In light to the recent development in engineering workplace, educating students with critical thinking might become crucial, in order to produce competence engineers.For that reason, teaching higher order cognitive abilities such as critical thinking ability has always been the ultimate goal of education [3], especially in engineering education.
Teaching critical thinking is as important as an individual being educated [4].Evidence suggests that the complex cognitive abilities can systematically be taught [5].Thus, in the recent development of pedagogical approach, one new method that has been claimed promoting students' critical thinking ability is using problem based learning (PBL).This method is generally agreed to have important implications for transfer of knowledge and application of problem solv-ing skills to novel situations [6].This assertion has been supported by several previous research reports [7][8][9].
PBL is often theorized to promote students' critical thinking, especially reasoning skills [10,11].PBL is anchored by Students Centered Learning approach that follows constructivist learning theory principles [12,13].In this context, knowledge acquisition becomes one of the prerequisites in developing students' critical thinking ability [13,14].According to Winterton et al. [14], knowledge and working memory play major roles in the acquisition of complex cognitive skills.This is particularly true since knowledge is operational and working within a social and attitudinal environment.The development of students' critical thinking ability however depends on willingness and an awareness of own thinking (self-reflection), as well as foundation skills [15].
A review on the recent research reveals a gap in determining the effect of PBL on students' critical thinking ability.Existing evidences from previous research have been equivocal to be deemed conclusive.In this context, several studies that related PBL and critical thinking resulted with positive findings.For example, study by Derry et al. [16], a qualitative study result, indicated that students improved their ability to scientifically and statistically reason in problem solving in a new PBL statistical course.Similar findings were also illustrated in Tiwari et al. [7], one year long of PBL treatment enhanced students' critical thinking ability, compared to conventional approach.Also, in Iwaoka et al. [17], students in Food Science and Human Nutrition Course indicated a significant increase in their critical thinking in several years of comparison of conventional approach.
On the other hands, several studies also resulted with negative findings or no significant difference of two groups' comparison in investigating the effects of PBL on students' critical thinking ability.This was illustrated in Polanco et al. [18] study; PBL did not change the first-and second-year undergraduate students' critical thinking ability in Mexican universities.Similarly, Choi [19] studied the effect of PBL on a Nursing Process Course in a quasi-experimental study.The result indicated no significant difference between the pretest and posttest data, for students' critical thinking aspect.In Sulaiman's [20] study, the results also indicated no significant difference for critical thinking, for both the control and the experimental groups.
In addition, the studies on critical thinking were mostly done in mathematical field [21].The studies that investigated the link between PBL and critical thinking in across disciplines and populations reveal a scarcity, especially studies in a control and experimental condition [22].Therefore, the main purpose of this paper is to investigate the effect of PBL on students' critical thinking ability, as compared to conventional approach in the electrical engineering course.This study uses a special design of PBL procedures from existing literature that focuses on three aspects, namely, the problem orientation, the method of facilitation, and the strategy of assessment.First, this paper presents a little introduction to critical thinking.Then, this paper presents and discusses the result of the experimental research on critical thinking, as compared to conventional approach.The result indicated PBL had not significantly enhanced students' critical thinking ability, as compared to conventional approach in an electrical engineering module.

Critical Thinking
Critical thinking is an ability to justify and reflect on what an individual believes.This, in light with Ennis et al. [23], points that critical thinking as a reasonable and reflective thinking that focuses on deciding what to believe or do.Critical thinking is an analytical process of arriving at judgments that is directed by a specific end purpose to arrive at a logical, rational, and reasonable problem solution [2,24,25].Some authors explain that critical thinking is the process of an individual taught to reason for improving the solution [26].Thus, the analytical process of reasoning must arrive at logical, rational, and reasonable judgments, within a given framework, and must agree with specific principles of thinking [27], as proposed by Facione [25] (i) Analysis = identifying and examining ideas and arguments.
(iii) Interpretation = clarifying meaning through categorization and translation.
Based on this conceptual definition, critical thinking ability is possibly nurtured in PBL, through the process of problem solving, particularly in a group brainstorming session [13,28].During this session, students critically consider one best possible solution for the problem at hand.The process is mediated by a facilitator, who is responsible for probing their metacognitive thinking, in making any decision [28].It is believed that probing questions may engage students in a systematic cognitive process that promotes the development of the students' reasoning ability.
In addition, the processes, such as discussion, debating, sharing, and teaching each other, create a platform for students to experience an environment that is conducive for critical thinking to grow [28].Similarly, students develop their critical thinking, especially their reasoning skills, through the process of interaction, reflection, and feedback, during problem solving or from the formative assessment process [12].Unlike in conventional approach, students undergo a process of learning through listening, which is doubted to provide an effective stimulus for students' critical thinking development.
Within this capacity, a strong basis exists that supports PBL's contribution to students' higher order cognitive abilities, especially critical thinking ability.The concept of "learning by doing" in the PBL approach is actually based on Experiential Learning Theory, in which students learn thinking strategies by solving a problem [13].The facilitator then stimulates students' critical thinking in looking for a best solution, which is also in line with the concept of "scaffolding" from the Constructivist Learning Theory [13,28].

Research Design and
Sampling.An experimental, pre-test and post-test with control group design were implemented in July 2011 in the Department of Electrical Engineering in the Malaysian Polytechnic.The participants were 53 undergraduate students attended Electrical Technology Module (ET101) in their first semester.They were selected according to two stages of cluster sampling technique [29].In the first stage of sampling, two out of twenty-two polytechnics were selected, namely, Polytechnic A (N = 27) and Polytechnic B (N = 26).In the second stage of sampling, a class (element) in each of these polytechnics was randomly selected according to lecturers, without studying students' characteristics at first place.These classes were randomly assigned to either the experimental (Polytechnic A) or the control group (Polytechnic B).Both groups were then exposed to ten weeks of instructions (PBL versus conventional approach), given a pre-test in the first weeks and post-test in the twelve weeks.
In this study, the homogeneity of two classes was confirmed by the pre-test data (Levene's statistic = .777;P = .383).
In Malaysia, there were a total of 27 polytechnics, where 22 of them have Electrical Engineering Department (clusters), in which the selected module is offered.As control factors, two polytechnics were selected, in order to avoid samples and treatment diffusion.These polytechnics were equivalent, for they operate in the same working days, state events, and holidays.A class in each polytechnic was assumed to be homogeneous which can be justified in the following reasons: firstly, students' entry requirements and intake for polytechnics were standardized and centralized [30].Secondly, students were randomly placed in a particular class due to the higher number of students' intake regardless of students' particular characteristics.Ethically, students were made to understand that SCL is implemented according to a new polytechnic' educational reformation in 2010 [30].

Procedures.
The unit of instructions used for this study involves two selected units of the Electrical Technology (ET101) Module syllabus (Units 3 and 4).
(i) An introduction to electric circuit.(ii) DC equivalent circuit and network theorems.
These two major units accommodate the timeframes of 14 hours of lecture and 20 hours of laboratory practical, within eight to ten weeks period.In this study, the timeframe for these units of instruction were prefixed into ten weeks duration in both groups.Students were first given a pre-test before the treatment begins, and the post-test followed up after completing the ten weeks of treatment in both groups.
Control Group.In brief, the procedures in the control group were retained as according to the existing setting.Teaching Units 3 and 4, the lecture was for two-hours meeting session in the class with additional two hours for laboratory session within a week.In this case, the lecturer was typically delivering information and facts, explaining the terms, symbols, concepts, and procedures.Students acted as passive learners.In certain learning topics, for example, the "Kirchhoff 's Law", the lecturer had to introduce the theorem before showing some examples of application and calculation on white board.Due to the nature of these topics content with large amount of concepts and principles, the teaching approach using lecturing has always been the primary method of instruction.The instruction went on for 10 weeks parallel with the experimental group.
Experimental Group.The PBL procedures were appropriately designed based on existing pioneer models such as the Aalborg Model, McMaster Model, and Republic Polytechnic Model.In this context, the researcher identified several critical components that became key success factors in implementing PBL [31].During ten weeks of PBL treatment, students were scheduled to solve five problems related to these two topics in the ET101 module.Within these, students were scheduled to have two-weeks problem-solving period to complete one cycle of PBL procedures.The first week's sessions were generally devoted to groups receiving problem scenario.The second week was devoted to assessment activities.The subject-centric problem was used as a trigger [32] and a ten-minute mini lecture was used to fill the gaps within the subject-centric problem [13].
In the first tutorial session, students were divided into heterogeneous group consisting of four to five members each [33].A leader was then appointed and rotated for each PBL problem.All groups received the problem in the form of written scenarios.Each group was given several documents, including a problem analysis table, humanistic skills rubric, process skills rubric, and grading forms.A facilitator then conducted a mini lecture to introduce the problem, explain several important concepts, and to explain the students' role.Then, the PBL groups immediately began work to understand the problem.
During problem solving, the floating facilitator concept was applied [13].The facilitator moved from one group to another to probe and facilitate students through the process of understanding the problem [28].In between meeting sessions, students were encouraged to conduct an independent self-study and independent group discussion [28].Students were also free to collaborate with relevant experts [33].
In the last meeting session, the major activities involved information sharing, assessment, and feedback process [34].The groups took turns to present their solution proposal.These short presentations were conducted in a group-based format, with all group members presenting their part [33].While this was going on, other groups performed peer assessments to evaluate the other groups' performances.At the end of the presentation session, students were asked to rate their team members' performance, according to the rubric rating scale [35].The facilitator immediately provided feedback to each group [13].The facilitator and the students then generalized the learning experience, relevant to the learning outcomes.

Instruments. The translated version of the Cornell Critical Thinking Test Specimen (CCTTS) (with permission)
was used in determining students' critical thinking changes before and after treatment.The CCTTS comprises of 52 items multiple choices and measures critical thinking based on inference, according to induction, deduction, credibility, observation, meaning, assumption, and disposition [23].The CCTTS is appropriate for an experimental study, in order to see improvement on critical thinking [23], as well as comparable to the study samples.
In order to maintain the validity and reliability of the original instrument, the translation and backtranslation was performed by two independent English language experts [36].Then, both instruments were then compared and sought for critical thinking expert validation.The interitem consistency was according to Kuder-Richardson 20 or Alpha, estimated coefficients were 0.60.The value is acceptable, since the estimate range of reliability for CCTTS was 0.49 to 0.87 [23].

Result
A total of 50 participants completed the pretests and posttests, where three students (Experimental group = 2; Control group = 1) have dropped out from polytechnic.Within these, males as dominant represent 84% (42) and females represent 16% (8).However, gender was equal in both groups.The majority of students (Experimental group = 17; Control group = 16) were technically grounded from previous school, while other students were from nontechnical streams such as arts and sciences.
A hypothesis to be tested was as following: there is no significant difference in mean critical thinking ability between the control and experimental groups.In order to test this hypothesis, a number of factors were held constant.These include the lecturers, students, polytechnics administration, instructions duration, and the topics of instruction.Analysis of variance (ANOVA) was used for testing of pretested homogeneity.The result indicated that both groups pretest on critical thinking were not significantly different F(1, 46) = .221,P > .05,which allows for comparison to be made using Analysis of covariance (ANCOVA).The descriptive statistic and ANCOVA were used to analyze the complete pairs of data (N = 50).The result is given in Tables 1 and 2.
Table 1 generally indicates the mean score of the posttest on the CCTTS in both groups slightly different from the mean score of control group that was 20.20 (SD = 3.07) and the experimental group was 20.36 (SD = 2.71).In Table 2, the result from the ANCOVA indicates that there was no significant difference in mean critical thinking ability between the control and experimental group [F(1, 47) = .005,P > .05].The null hypothesis was therefore accepted; the result indicated that students had similar ability in critical thinking; either they were taught using PBL or conventional approach.

Discussion and Implication
In the present study, the result indicated that students had similar ability in critical thinking; either they were taught using PBL or conventional approach.The result is not aligned with the theory from several authors [10,11].The result however can be justified according to several reasons.In PBL, critical thinking ability is possibly nurtured through several processes such as discussion, brainstorming session, debate session, interaction, reflection, feedback, and teaching each other [12,13,28].In this study, some of these processes might be less effective with respect to study samples.Implementing PBL in the first-year students was a challenging task for facilitator, since most of students might be influenced by the previous conventional method.Facilitator provided a platform (knowledge and foundation skills), but critical thinking development depends on willingness and an awareness of own thinking (self-reflection) [15].Additionally, Ho [37] stresses that information is shared during learning process, but knowledge is personally constructed by learners.
Furthermore, the variable such as critical thinking is a subjective and thus difficult to be determined when critical thinking occurrs [15].There is also general agreement in the literature that critical thinking cannot be narrowly defined 49 * Significant at P < .05. [23,24].This implies the area and the scopes measured in this study that might also justify this study finding.The study however reaffirmed previous study findings from multiple educational levels and disciplines [18][19][20].In the context of study in electrical engineering in polytechnics, students in PBL group did not enhance their critical thinking compared to their counterparts in conventional approach.
From other points of views, another possible reason is that the critical thinking for PBL students did not increase as expected, due to the treatment duration.In this study, ten weeks might be insufficient for PBL to enhance students' critical thinking ability.The literature suggests that PBL could be more effective over a longer-term duration [7].In Tiwari et al.'s study, the first-year students in PBL were given two semesters treatment and the data were collected until students become in their third year of the course.This has provided a space for students learn critical thinking through PBL process to such as reasoning throughout the three year of the course.

Conclusion
In this study, PBL was specially designed and implemented in Electrical Technology Module, one of the modules in the polytechnic's Electrical Engineering Course, as a comparison to conventional approach.The design was using PBL key elements that were customized from existing pioneer PBL models.Within this, students in the experimental group underwent PBL procedures, while students in the control group were using conventional approach such as lecturing method.In regards to the study samples, it can be concluded that PBL does not enhance students' critical thinking ability as compared to conventional approach, in Electrical Engineering Module.
Further study is warranted to investigate the effect of PBL on students' critical thinking in a longitudinal design.A continual exposure to PBL processes should also be effective in nurturing students' critical thinking.The implication is that the PBL procedures used in this study might be useful for educators in polytechnic, with some modifications of the original framework.It is suggested that future research replicates the study and modifies the proposed PBL procedures to focus on the process of stimulating students' meta-cognition, as well as increase students' awareness of their critical thinking.These are in respect to the processing such as discussion, brainstorming session, debate session, interaction, reflection, feedback, and teaching each other.In this capacity, it is also important to identify which components in PBL process that are really effective in stimulating each of the variables of interest, such as critical thinking.

Table 2 :
Summary of ANCOVA for the comparison of critical thinking ability scores.