Assessment and Classification of Service Learning: A Case Study of CS/EE Students

This study investigates the undergraduate students in computer science/electric engineering (CS/EE) in Taiwan to measure their perceived benefits from the experiences in service learning coursework. In addition, the confidence of their professional disciplines and its correlation with service learning experiences are examined. The results show that students take positive attitudes toward service learning and their perceived benefits from service learning are correlated with their confidence in professional disciplines. Furthermore, this study designs the knowledge model by Bayesian network (BN) classifiers and term frequency-inverse document frequency (TFIDF) for counseling students on the optimal choice of service learning.


Introduction
Educators in universities have recognized a fact that merely professional knowledge is not sufficient in real-world competition. In addition to professional know-how, the key success factors to a remarkable career include integrity, leadership, interpersonal skills, sympathy to others, and sense of social responsibilities. So students are motivated to provide voluntary services for the public and communities that can play a key role as connections to their social networks. Service learning is a form of experiential education in which students engage in activities that address human and community needs together with opportunities designed to promote student learning and development [1].
Service learning has a long history in the United States and has been recognized as key curriculum component in universities [2][3][4][5][6]. The forms of service learning include homeless shelters, collecting and handing out food, battered women shelters, hot lines, school tutors, and habitat for humanity [7]. Benefits of service learning have been mentioned in various investigations. Toncar et al. [4] developed the service learning benefit (SELEB) as a scale that measures student perceptions of service learning experiences. Werder and Strand [5] later assess the effectiveness of service learning in the public relations capstone course by measuring perceived student learning outcomes. Gallini and Moely [8] find that the service learning courses promoted interpersonal, community, and academic engagement, were academic challenging, and encouraged students continued study at the university (retention).
On the other hand, the challenges of service learning have been reported. Some people argue that it is neither the college's place nor position to require service learning for students. Faculty also faces the challenge that the community should not be treated as a laboratory. Besides it is difficult to prove that service learning experiences are related to the students' curriculum [7].
This study intends to aim at students' perception of service learning experiences. Among the researches on 2 The Scientific World Journal measuring the perceived outcomes, most investigate the students in social science disciplines while the benefits for engineering students are to be explored. This study first investigates the undergraduate students in computer science/electric engineering (CS/EE) to measure their perceived benefits from service learning coursework. Besides, the benefits of service learning and their correlation between the professional disciplines are tested. Furthermore, this study advances one further step to construct the counseling knowledge base using Bayesian networks. We analyze students' profiles and their feedback by term frequency-inverse document frequency and design the Bayesian network that provides advices to students for their optimal type of service learning.
Background of the Knowledge Base. Bayesian networks (BN) [9][10][11] are directed acyclic graphs (DAG) in which the nodes represent the variables, the arcs represent the direct causal influences between the linked variables, and the influences may be quantified by conditional probabilities. They are widely used knowledge representation and reasoning models in various domains under uncertainty. Since an expert system requires both predictive and diagnostic information, two types of reasoning are common in Bayesian networks, deduction and abduction. Deduction, or prediction, is a logical process from a hypothesis to deduce evidence where probabilistic relationships are involved. Abduction, or diagnosis, is a logical process that hypothetically explains experimental observations [11]. The naïve Bayesian network (NBN) model is named by Titterington et al. [12] for its simplicity. In a NBN model, the target variable of interest is normally defined as the root without a parent node.
Term frequency-inverse document frequency (TFIDF) [13][14][15] is the formal measure concerning how concentrated into relatively few documents the occurrences of a given word are. Suppose we have a collection of documents. Define to be the frequency (number of occurrences) of term (word) in document . Then, the term frequency TF is defined as follows: that is, the term frequency of term in document is normalized by dividing it by the maximum number of occurrences of any term (perhaps excluding stop words) in the same document. Thus, the most frequent term in document gets a TF of 1, and other terms get fractions as their term frequency for this document.
Suppose term appears in of the documents in the collection. Then IDF = log 2 ( / ). The TFIDF score for term in document is then defined to be TF × IDF . The terms with the highest TFIDF score are often the terms that best characterize the topic of the document. The measure TFIDF identifies words in a set of documents that are useful for determining the topic of each document [15].
The remainder of this paper is organized as follow. Section 2 first develops the research design for the benefits Perceived effect of professional discipline on service learning Perceived effect of service learning on professional confidence of service learning. Section 3 then addresses the statistics and hypotheses testing. The classification model of service learning is built at the second stage in Section 4. Finally we give the discussions and conclusions in the fifth section.

The Framework for Benefit Measurement
In the first stage, this study aims at the benefits of service learning and their relationships between professional capabilities. The purposes of this stage are (1) exploring the benefits of service learning course by measuring the attitudes of students in CS/EE, (2) examining students' confidence in the fields of CS/EE, (3) testing the correlation between the benefits of service learning and students' professional confidence, and (4) examining the relation between the benefits/confidence and the perceived mutual effects between service learning and professional confidence. The research framework is shown as Figure 1.
Based on the framework in Figure 1, we first test the perceived benefits of service learning and professional learning. To test the correlation between the benefits of service learning and the confidence in professional discipline, the following hypotheses are tested.
H 3 : students perceive that their capabilities and experiences gained from service learning assist in learning the professional discipline.
H 4 : students perceive that their capabilities in the professional discipline assist in the service learning coursework.
The Scientific World Journal 3 Besides, we test the following hypotheses on the mediating effects between the perceived benefits of two types of learning and their correlation.
H 5 : students who perceived high benefits from service learning are likely to perceive high benefits from professional discipline (and vice versa).
H 6a : students who perceived high benefits from service learning are likely to perceive that service learning assists in professional learning.
H 6b : students who perceived high benefits from service learning are likely to perceive that professional learning assists in service learning.
H 7a : students who perceived high confidence of professional learning are likely to perceive that professional learning assists in service learning.
H 7b : students who perceived high confidence of professional learning are likely to perceive that service learning assists in professional learning.

The Sample.
We survey the attitudes of undergraduate students in CS/EE at one leading national university in Eastern Taiwan. The samples are students registered in service learning as a requisite course for two semesters during 2010-2012. In each semester they are required to provide at least 18 hours voluntary services on or off campus, under supervision of the faculty. On completion of the coursework, the students are invited to fill the questionnaires composed of 20 questions on service learning, 9 questions on professional learning, 2 questions on service experiences, and a set of questions about personal information. The total sample size is 256, including 143 in electric engineering and 113 in computer science. Finally the effective sample size is 116, among which 59 (50.86%) in electric engineering and 57 (49.14%) in computer science.

Questionnaire Design.
The questionnaire is prepared based on [4,5] as in Table 1. Questions in the first section construct the students' perceived benefits from service learning, and the items in the second section measure the students' perceived capabilities from professional learning. All the questions are answered with the Likert-type 7-point scale, where 1 means "strongly disagree" and 7 means "strongly agree. " The results of Part 1 are processed by factor analysis and yield a four-factor solution. The rotated factor matrix is shown in Table 2, where all variables load significantly at least one factor. Usually factor loadings exceeding 0.5 suggest significance in practice. All factors load exceeding 0.5 except item 19 (organizational skills) and item 15 (sensitivity to the plight of others) that are slightly below 0.5. So we accept the results of factor analysis and name the factors as "practical skills and development, " "critical thinking, " "leadership and interpersonal skills, " and "citizenship, " respectively. The descriptive statistics are organized by factors as in Table 3.

Hypothesis Testing.
To test the hypotheses H 1 to H 4 , we take the Student's -test for "mean ≥ 5" and "mean ≥ 4.5" at the 99% confidence level, respectively. From Part 1 of Table 3, most items support H 1 , except the item in factor 4 "understanding cultural and racial differences. " The responses arise likely because most students did not work in an international environment. In Part 2, 6 out of 9 items are rejected with "mean ≥ 5" but all accepted with "mean ≥ 4.5. " The results show that undergraduate students in CS/EE are fairly confident of their professional major, not at a strong level. As for the interactions between service learning and professional disciplines, both items in Part 3 reject H 3 and H 4 , respectively.
To test H 5 , the Pearson correlation between the benefits from service learning and those from professional learning is 0.483. The correlation indicates that the benefits from two types of learning are positively correlated in a moderate manner, so H 5 is not rejected.
For H 6a , the correlation between perceived benefits from service learning and the first item of Part 3 is 0.677, which supports H 6a . Similarly, the correlation between perceived benefits from service learning and the second item of Part 3 is 0.631, so H 6b is supported.
To test H 7a , the correlation between perceived benefits from professional discipline and the first item of Part 3 is 0.630, and the correlation between perceived benefits from professional learning and the second item of Part 3 is 0.572. So H 7a and H 7b are both supported.
Notably there is one open question in Part 4 "please describe your experiences in service learning in words. " In next section we will describe how to analyze the responses to the open question and how to build the classification model with the personal features and the outcomes of document analysis.

Knowledge Model of Service Learning
For counseling students on service learning, this section develops the classification model by the naïve Bayesian network. In the naïve Bayesian network, the target variable (root) is students' perceived benefit level and the predicting variables are measured from the items in Part 4, including personal features and internal responses of the students.
This study uses TFIDF to extract a set of words that are frequent in the students' answers. At first we select 10 2 = 100 words. After computing the weight (importance) of the words and eliminating the "stop words" (articles, prepositions, and other auxiliary words), a compact set of 20 words were extracted. These 20 words are further factorized and analyzed, among which six terms are finally selected: "learning, " "service, " "profession, " "process, " "document," and "capability. " The target variable and predicting variables for the Bayesian network on service learning are summarized in 4 The Scientific World Journal Table 1: The structure of the questionnaire. Part 1: benefits from service learning (1) Personal growth (2) Ability to work with others (3) Leadership skills (4) Communications skills (5) Understanding cultural and racial differences (6) Social responsibility and citizenship skills (7) Community involvement (8) Applying knowledge to the "real world" (9) Problem analysis and critical thinking (10) Social self-confidence (11) Conflict resolution (12) Ability to assume personal responsibility (13) Development of caring relationships (14) Gaining the trust of others (15) Sensitivity to the plight of others (16) Workplace skills (17) Ability to make a difference in the community (18) Skills in learning from experience (19) Organizational skills (20) Connecting theory and practice Part 2: confidence from professional learning (1) Basis in CS/EE, mathematics, science, and engineering (2) Design/implementation of experiments, analysis, and explanation of results (3) Capability for using programming languages, application systems, and instruments of CS/EE (4) Capability for hardware and software development (5) Computer hardware design and computer networks (6) Teamwork and communication skills (7) Discovery, understanding, integration, and problem-solving in CS/EE (8) Understanding the impacts of CS/EE on environments and continuous learning (9) Understanding the ethical and social responsibilities of CS/EE Part 3: correlation between service learning and professional discipline (1) The capability and experience gained from service learning assist in learning the professional discipline.
(2) The capability and experience in the professional discipline assist in the service learning coursework.

Part 4: personal information
Please describe your experiences in service learning in words, for example, the remarkable interactions, how your experiences from service learning assisted professional learning, or the contributions of your professional background that might help in service learning  Table 4. The structure of the Bayesian network is depicted as Figure 2.
To estimate the parameters, this study adopts the 5-fold cross validation. The dataset is first divided randomly into five subsamples. Then in every fold, four subsets are input for learning and one subset is used for testing under various scenarios. Before learning the conditional probabilities, the continuous root variable needs to be discretized. We partition The Scientific World Journal 5  · · · · · · · · · · · · Figure 2: The Bayesian network on service learning benefits. the domain of the root into 3 and 5 subranges and compare the classification accuracy by 3 and 5 states, respectively. The results of tests are summarized in Table 5.

Discussions and Conclusions
This study investigates the undergraduate students in CS/EE to measure their perceived benefits from the experiences in service learning coursework. By the questionnaire, we explore the significance of the perceived benefits from service learning and the correlation between their professional disciplines.
The results show that the students perceive significant benefits from service learning, which is consistent with the results of [4,5]. But the students recognize fair confidence from professional learning. Because nearly 67.24% of the subjects are first-year or sophomore-year students, the capabilities perceived from professional discipline manifest their premature background. Besides, the perceived mutual effects between service learning and professional learning are denied. However, we find that the perceived benefits from both service learning and professional learning are positively correlated to the effect of service learning (professional learning) on professional learning (service learning). The results are potentially compatible with Gallini and Moely's finding [8] that the service learning courses promoted interpersonal, community, and academic engagement, were academic challenging, and encouraged their continued study at the university (retention). So we claim that students' perceived benefits from service learning and professional confidence are correlated with their mutual effects in a mediated way. Once students perceive high benefits from service learning or professional learning, they can also reinforce the confidence in either service learning or professional discipline.
Besides, we design the classification model using naïve Bayesian networks for predicting the benefits of service learning that students perceive. By linking students' personal profiles and their feedback on service learning experiences, 6 The Scientific World Journal Activities planning: 9 (7.76%) Others: 11 (9.48%) * Rejected by the -test "mean ≥ 5" but accepted by hypotheses "mean ≥ 4.5", 99% confidence. * * Rejected by the -test "mean ≥ 4.5", 99% confidence. * * * Sum beyond 100% because a student may attend services in various types or places.  we may predict their attitudes (level of perceived benefits) toward service learning. Future studies are suggested to enrich the predicting variables from students' profile and internal responses. Further the classification model can be extended as a knowledge base in the counseling systems for students learning and career path.