Assessing the Effect of Laboratory Activities on Core Curricular Units of anEngineeringMaster’sProgram:AMultivariateAnalysis

'is paper studies the effect of laboratory activities on the results obtained by the students on 7 core curricular units (CUs) of an IntegratedMaster inMechanical Engineering. Each CU is characterized bymeans of 4 performance indices, over the 6-year period 2014/2015–2019/2020. Firstly, individual indices in a per semester basis are compared. Secondly, the CUs are regarded as objects defined in a 4-dimensional space of features, and the multidimensional scaling (MDS) technique is adopted for clustering and computer visualization. 'e MDS is powerful for analyzing the multivariate dataset, unveiling patterns not perceived by standard statistical methods.


Introduction
Laboratory training is fundamental in engineering education [1][2][3][4][5][6][7][8][9][10], but understanding how the experimental activities contribute to meaningful learning and to achieve the goals established for a specific curricular unit (CU) is challenging [11][12][13][14]. Researchers often classify the educational engineering laboratories as follows: (i) hands-on with real equipment; (ii) simulated or virtual; and (iii) remote [15][16][17][18][19]. Despite this strict classification, many laboratory environments adopt a blended approach, where hands-on, virtual, and remote experiments are combined together, trying to simultaneously potentiate the advantages and mitigate the drawbacks of each one [20,21]. e laboratory experiments envisage strengthening: (i) the conceptual understanding, namely, the students' capabilities to understand and to solve problems related to the fundamental concepts presented in the theoretical classes; (ii) the design skills, meaning the students' skills to solve open-ended problems through the design and creation of new processes; (iii) the social skills, which mean the students' abilities to learn how to undertake engineering-related work in groups; and (iv) the professional skills, namely, the students' capabilities to become familiar with the technical expertise that they will need in their professional activities [1,2]. e best procedure to accomplish these four objectives in the context of undergraduate engineering programs is an open issue, which is hardened by multiple specific factors, as the teaching/ learning subjects, the students' backgrounds, or the number of students enrolled in the CU. e Integrated Master in Mechanical Engineering (MIEM) at the Faculty of Engineering of the University of Porto (FEUP) prepares students so that, after completing the cycle of studies (CS), they can initiate a professional career in applied and/or research activities in Mechanical Engineering or related fields. To accomplish such objective, MIEM provides the students a solid educational framework with a rich scientific background and emphasis on the interaction between theory and practice within the main domains of Mechanical Engineering. e importance of laboratory classes has always been recognized by those responsible for the CS, and these classes were implemented on most CUs of MIEM. In fact, the laboratory classes were early considered as an asset that allowed the students to acquire an important training component in technological areas and motivated them greatly to engage in the learning processes. e laboratory classes were also intended to give the students the opportunity to work in teams and an avenue to communicate their ideas in a concise, objective, and pragmatic way, as well as to develop their critical analysis of results. e increase in the laboratory component of the MIEM CUs was successively recommended by the accreditation bodies, namely, for the CUs in the areas of Applied Mechanics and Fluids and Energy, which are core areas of Mechanical Engineering. All CUs of these scientific areas included in the common part of the CS syllabus had no laboratory activities. In some of them, the students had extra difficulties, which translated in getting worse results than those verified in other CUs of the CS program.
ese CUs had in common the adoption of teaching methods based on theoretical classes for the exposition of subjects and theoretical-practical classes for problem solving. Moreover, they had a high number of enrolled students and high failure rates. To mitigate these issues, while avoiding changing the MIEM syllabus, which would have involved a slow and bureaucratic administrative process, the direction of the CS, with the support of the Mechanical Engineering Department (DEMec), decided to implement mandatory laboratory activities in some CUs of the second up to sixth semesters of MIEM, starting in the academic year 2016/2017. e procedure adopted, still in practice, requires that the students should carry out a set of laboratory works throughout the semester. Such activities are performed in small team groups, following the guidelines provided by the CU staff, with the objective of applying the knowledge acquired at the theoretical and theoreticalpractical classes. e laboratorial experiments should follow a script with precise instructions and be completed in a limited time, while the students are supported by monitors, who typically are students of the last semester of MIEM. e students are assessed via small written reports, one per experiment.
In this work the effect of laboratory activities on the results obtained by the students on 7 core CUs of MIEM is assessed.
ese CUs are {Mechanics I, Solids Mechanics, Mechanics III}, to be denoted by {MI, SM, MIII}, and { ermodynamics II, Fluid Mechanics I, Fluid Mechanics II, Heat Transfer}, to be denoted by {TII, FMI, FMII, HT}. e two sets belong to the areas of Applied Mechanics and Fluids and Energy, respectively. Each CU is characterized by the 4 performance indices {passed/enrolled students' ratio, passed/assessed students' ratio, arithmetic mean of the passed students' marks, arithmetic mean of the assessed students' marks}, to be denoted by {Pas/Enr, Pas/Ass, Mean Pas, Mean Ass}, over the 6-year period 2014/2015-2019/ 2020. Firstly, the performance indices characterizing each CU are compared with those of the CUs in the same semester, and their time evolution is analyzed in an individual basis. Secondly, the CUs in a given semester are considered objects defined in a 4-dimensional space of features, and the multidimensional scaling (MDS) technique is adopted for dimensionality reduction, clustering, and computer visualization of patterns.
is is accomplished by choosing a suitable distance metric for object-to-object comparison and Procrustes analysis for obtaining meaningful maps of objects (that is, CUs) which are able to unveil possible hidden relationships between the data.
Having these ideas in mind, the paper organization is as follows. In Section 2, the MIEM program and the CUs are introduced briefly. In Section 3, a statistical study is performed in a per semester basis, to understand the time evolution of the results obtained in each CU. In Section 4, a multivariate analysis is conducted to unveil possible patterns embedded in the data, which are difficult or impossible to be noticed when adopting simple statistics. Finally, in Section 5, the main conclusions are summarized. M � 7 CUs in the set {MI, TII, SM, FMI, FMII, MIII, HT} and they have adopted identical teaching methods based on theoretical classes for subject exposition and theoreticalpractical classes for problem solving. Experimental activities have been negligible. e considerable numbers of students enrolled, between 202 and 275 in year 2019-2020, and the high failure rates in the CUs became a concern to the CS and DEMec directors. erefore, in an attempt to relieve the problem, laboratory activities were adopted in 2016/2017, for {MI, SM, MIII, FMII, HT}, and in 2017/2018, for {TII, FMI}. Such laboratory activities require that the students should carry out predefined laboratory experiments throughout the semester. e work is performed in small groups, according to provided guidelines and in-loco staff support, which envisages consolidating the subjects taught in both the theoretical and the theoretical-practical classes.

Statistical Analysis of the Students' Performance
In this Section, the 4 performance indices {Pas/Enr, Pas/Ass, Mean Pas, Mean Ass} that characterize each CU over the 6 academic years 2014/2015-2019/2020 are compared, in an individual basis, with those that characterize the CUs in the same semester. is means that MI is compared with the CUs in S 2 , TII with the CUs in S 3 , and so on, and the comparison involves one index at a time. Figure 1 depicts the performance indices of the CUs versus time. e red lines represent the values of the performance indices of each indicated CU, and the black lines represent the arithmetic means of the performance indices of the CUs in the same semester, either S 2 , . . ., S 6 , respectively. e bottom and top edges of the blue boxes represent the 25 and 75 percentiles, while the central marks stand for the medians.
e whiskers extend between the extreme data points, when excluding the outliers, and the outliers are represented by means of the symbol "o." We verify that most CUs' performance indices follow the trend exhibited by the CUs in their semester, over the entire period 2014/ 2015-2019/2020. is would mean that the effect of the laboratory activities is either negligible or hidden. However, some CUs reveal complex relationships for certain performance indices, as those depicted in the graphs denoted by (xii), (xiv), and (xv), and further analysis is needed.
To get more solid conclusions, we calculate the Pearson correlation ρ xy as follows: represents the values of a performance index {Pas/Enr, Pas/Ass, Mean Pas, Mean Ass} for a given CU and y � [y 1 , . . . , y 6 ] stands for the arithmetic mean of the index for the CUs that belong to the same semester, in the 6-year period 2014/2015-2019/2020. e symbol μ(•) denotes the arithmetic mean of the argument. Table 2 summarizes the results obtained. We verify that ρ xy is high for most CUs, with the exception of {SM, FMI, MIII}, which exhibit small values of ρ xy for certain performance indices. is confirms the rough assumptions made previously through simple observation of the graphs since a low ρ xy translates into discrepancies between the trends exhibited by a specific CU and the CUs in the same semester.

Multivariate Analysis of the
Students' Performance e analysis performed in Section 3 using simple statistics showed that not all performance indices lead to identical results. erefore, examining the indices separately may reveal inconsistencies and bring the analysis inconclusive. To mitigate this issue, in the follow-up, a multivariate analysis is performed, meaning that all features characterizing the data are considered simultaneously.
Several mathematical techniques are available for multivariate data analysis. e principal component analysis (PCA) [22], additive trees [23], cluster analysis [24], factor analysis [25], and multiple regression analysis [26], among others [27], can be cited. Herein, the MDS [28-31] is adopted. MDS has advantages over other methods, such as PCA, which has difficulties to identify clusters surrounded by rings of points. Moreover, MDS is a more general projection method than PCA [28]. [28][29][30][31]. MDS detects meaningful fundamental dimensions that allow the researcher to visualize (dis)similarities between the objects of interest. Given M objects in a q-dimensional space, the algorithm receives as input a M × M  matrix Δ � [δ ij ] of object-to-object (dis)similarities, calculated based on a distance metric. e MDS assigns coordinates to each object in a p-dimensional space, with p ≤ q, and organizes the set in a symmetric matrix Φ � [ϕ ij ] in order to reproduce Δ as best as possible. is is accomplished by means of a function optimization procedure that tests several configurations. A common index to assess how well a given configuration in Φ reproduces the original distances matrix Δ is the raw stress as follows:

MDS Data Clustering and Visualization. MDS is a computational technique for visualizing information
where f(δ ij ) stands for a linear or nonlinear function. e smaller the stress, R, the better is the fit between Φ and Δ. For p ≤ 3, the objects can be represented in a graph that can be analyzed in the perspective of the objects location, meaning that similar objects appear close to each other, while dissimilar objects are located distant from each other. e orientation of the final MDS chart has no special meaning, and the map is insensitive to translation, rotation, and magnification.
For each year, 2014/2015-2019/2020, the data characterizing the CUs in a semester are organized in a 5 × 4 array,  Ass}. e array W is firstly normalized to prevent numerical saturation. erefore, the columns of W, to be represented by u k , are expressed as follows: is leads to a normalized array W, where σ(•) represents the standard deviation of the argument. en, the rows of W, to be represented by v i � [v i1 , . . . , v i4 ], are used to compute the matrix Δ � [δ ij ], i, j � 1, ..., 5, where δ ij denotes the Euclidean distance: Finally, Δ is processed through the MDS algorithm to generate the maps of objects that represent the CUs in per semester and per year basis. erefore, MI is similar to TI, and the two CUs are quite different from the pair AMII and S. Often, CPI is close to MAII and to S but reveals some variability. In conclusion, over the period 2014/2015-2019/2020, MI did not change its relative positioning with respect to the other CUs in the semester S 2 , meaning that no effect of the laboratory activities on MI can be noted. erefore, TII keeps its relative positioning with respect to the other CUs in the semester S 3 , meaning, as before, that no effect of the laboratory activities on the CU can be noted. Figure 4 portraits the MDS maps of the CUs in semester S 4 . In this case, the behavior of the CUs is almost random, which inhibits any attempt to interpret their dynamics in the period 2014/2015-2019/2020.
In Figure 5, we verify the existence of 2 clusters every year. One includes {CADM, NMM}, and the other comprises {FMII, FPS, SMI}. ese clusters correspond to CUs with the best and the worst results in the semester S 5 . e patterns formed by the CUs are stable over the period 2014/ 2015-2019/2020. us, regarding FMII, it means that the effect of the laboratory activities cannot be distinguished. In synthesis, the multivariate analysis performed unveils some evidences about the behavior of the CUs, but in most cases the results are still somewhat inconclusive.

MDS with Procrustes Data Clustering and Visualization.
To improve the visualization and ease the detection of possible relationships between the data, each set of maps depicted in Figures 2-6 can be replaced by one chart that collapses all information embedded in the set. Hereafter, this is accomplished by means of Procrustes analysis. Procrustes is a numerical technique that takes a collection of shapes and transforms them for maximum superposition, using the linear transformations O � {translation, reflection, orthogonal rotation, magnification/reduction of size} [29,[32][33][34].
For the CUs in each semester S 2 , . . ., S 6 , Procrustes involves four iterative steps: (i) the user selects a reference MDS map (in the present case, the one corresponding to the year 2014/2015 is adopted but any available instance can be used); (ii) the algorithm superimposes all remaining maps into the current reference using the transformations in O; (iii) it computes the 'mean form' of the current set of superimposed maps; and (iv) it calculates the distance between the "mean form" and the reference instances and compares such distance with a given threshold value. If the distance is greater than the threshold, then the algorithm sets the reference to the "mean form" and steps to (ii). Otherwise, it keeps the reference and steps to (ii). At the end, a global representation of all maps that best conforms them is obtained.    (ii) e CUs in S 3 are portrait in Figure 8.  erefore, it can be concluded that MFI is not improving, but SM is getting better results;  (iv) In Figure 10, which portraits the dynamics of the CUs in semester S 5 , it can be seen that there are 2 CUs with much better performance than the others. e best {CADM, NMM} are on the right-hand side, while the worst {FMII, FPS, SMI} evolve on the lefthand side of the MDS map. Regarding FMII, it seems to exhibit a timid movement toward the set {CADM, NMM}, which translates into a slight improvement on the results in the UC; (v) Figure 11 shows the MDS of the CUs in semester S 6   {IAS, MPI}. It can be seen that HT evolves within a relative small volume, with no significant variation on the results obtained over the period 2014/ 2015-2019/2020. e CU MIII evolves approaching the middle of the map, meaning that the results are getting better.

Conclusions
e effect of laboratory activities implemented in 7 core CUs of the MIEM degree at FEUP was investigated in the perspective of the results achieved by the students. e CUs were characterized by 4 performance indices in the 6-year time horizon 2014/2015-2019/2020. In a first phase, the analysis of single indices revealed inconclusive and pointed towards the need of using multivariate analytical models to compare and visualize the data. In a second phase, the CUs were considered as objects defined in the 4-dimensional space of features, and the MDS was adopted for dimensionality reduction, clustering, and data visualization. Finally, Procrustes processing was used to achieve meaningful maps of objects which are able to unveil all hidden relationships between the data. e MDS and Procrustes showed that, for now, the impact of the laboratory activities in the results of most CUs is insipient. However, for SM, FMII, and MIII, the effect seems to be positive since the results in these CUs have been improving in the last few years. Nevertheless, a longer period of analysis is needed to reach more solid conclusions.

Data Availability
e data used to support the findings of this study are available from the author upon request.

Conflicts of Interest
e authors declare that they have no conflicts of interest.