Conceptual Integration andMeasurement of Epistemological and Ontological Beliefs in Educational Research

is paper examines the conceptualization and measurement of epistemological and ontological phenomena and makes recommendations for improving the conceptual framework and methodological assessment of these phenomena. I discuss the ways educational researchers have studied beliefs and how this research can be improved through a comprehensive conceptual framework and better measurement. is paper provides de�nitions of epistemological and ontological beliefs and world views, discusses six complementary strategies for assessing these beliefs, compares the strengths of these strategies, and provides examples of how they have been used in the research literature.is paper discusses challenges related to the development of a comprehensive theoretical framework for beliefs, as well as ways to improve measurement of these beliefs and summarizes six emergent themes.

Research on personal epistemologies and ontologies has been difficult to conduct and interpret at times for conceptual and methodological reasons [36].From a conceptual standpoint, there is disagreement about the underlying structure of epistemological beliefs [12,29,[41][42][43].In addition, and of special signi�cance in this paper, there has not been a clear articulation of the relationship between personal epistemology and ontology [38,39,44].From a methodological standpoint, �ndings have been inconsistent and measurement problems have been a persistent problem for researchers [43,45].Future research would bene�t from a uni�ed conceptual framework that links epistemology and ontology, as well as a systematical methodological measurement plan that facilitates comparisons of epistemological and ontological phenomena across studies [29,46,47].
Researchers have used six different measurement strategies for assessing epistemological and ontological beliefs and world views, including multi-item questionnaires [19,[48][49][50], interviews [20,25,26,51,52], vignettes that characterize prototypical epistemological world views [29,53], essays, journals, and storyboards [54][55][56], concept maps [28,57], and multidimensional scaling of epistemological and ontological relativism [58].A number of questions arise regarding the strengths and weaknesses of these measurements, as well as their interrelationships.Very few studies have addressed these questions; thus, it is unclear whether the measurement strategies used in contemporary epistemological and ontological research measure the same constructs and phenomena, and in the event they measure different constructs, the extent to which these constructs are related.
occurs in a quick or not-at-all fashion), and innate ability (i.e., the ability to acquire knowledge is endowed at birth).�urrently, there is debate as to whether Schommer's �ve beliefs constitute genuine epistemological dimensions [12,43].Most researchers agree that beliefs about the certainty and simplicity of knowledge constitute genuine epistemological beliefs.Hofer [41] also has suggested the origin of knowledge as a genuine epistemological belief.
I use the term epistemological world view to refer to an individual's collective beliefs about the nature and acquisition of knowledge.I use term synonymously with similar terms used in the literature such as personal epistemology [22,49] and epistemological stances [25] that collectively refer to a set of beliefs or a personal theory about knowledge and knowledge justi�cation.I assume that an epistemological world view includes all of one's explicit and implicit beliefs, attitudes, and assumptions about the acquisition, structure, representation, and application of knowledge [12,29,59,64].
It is important to distinguish clearly between epistemological beliefs and epistemological world views.e former consist of speci�c beliefs about a particular dimension of knowledge such as its certainty, simplicity, origin, or justi�cation.e latter consist of a set of beliefs that collectively de�ne one's attitudes about the nature and acquisition of knowledge.Each adult presumably has a set of epistemological beliefs that are included within an epistemological world view, which also may include other beliefs such as how epistemological beliefs are acquired and develop, and how these beliefs change over time [65][66][67].Both epistemological beliefs and world views may be tacit or explicit [1].Presently, it is unclear whether tacit beliefs affect cognition and decision making more or less than explicit beliefs, or whether they are easier or more difficult to change [24,68].
Ontology is used in its broadest sense to refer to the nature of reality and being [4,[6][7][8].Typical ontological questions include what is reality and what can be known about reality?Ontology usually is discussed independent of epistemology, but at some level, the two are related because beliefs about how we come to know reality necessarily involve epistemological assumptions.I make the assumption that epistemological and ontological beliefs are related to some extent in most teachers' minds, although this issue has not been resolved empirically [44].
e term ontological belief has not been used widely in educational and psychological research, at least compared to the study of epistemological beliefs.However, just as researchers have argued for the existence of separate epistemological beliefs such as the origin or certainty of knowledge, it seems plausible that individuals would hold multiple ontological beliefs about the origin, permanence, and changeability of reality and being [69].
I use the term ontological world view to refer to an individual's collective beliefs about the nature of reality and being.I assume that an individual's ontological beliefs collectively comprise a personal ontology.Like epistemology, ontological beliefs and world views may be tacit or explicit in part or whole.I also assume the epistemological and ontological world views work in tandem to determine an individual's beliefs about learning and instruction.Indeed, many graduate T 1: �ey terms, de�nitions, and sources.
Philosophers and researchers oen distinguish between three competing theories of justi�cation referred to as realist, contextualist, and relativist positions that are relevant to a better understanding of epistemological and ontological research [1,5,15,16,46,61,73].A realist position assumes that there is a �xed, core body of knowledge that is best acquired through experts via transmission and reconstruction, which are in a more or less unchanging reality.For example, teachers with a realist world view may tend to view students as passive recipients of a preestablished knowledge base.A contextualist position assumes that learners construct shared understandings in supportive contexts in which teachers serve as facilitators.Teachers with a contextualist world view are less concerned with the type of knowledge that students construct, than the process by which they construct that knowledge, and the degree to which that knowledge has authentic application to the context it is learned in [4,29].Contextualist teachers also assume that a student's reality is idiosyncratic and changing, even though they would likely endorse the view that reality is a consensual social agreement in most contexts [8,51,74].For this reason, a student may possess "multiple epistemologies" that change as the context of understanding changes [75,76].In contrast, a relativist position assumes that each learner constructs a unique knowledge base that is different but equal to other learners' knowledge.Teachers with relativist world views deny the primacy of their own knowledge and emphasize their role in creating an environment where students can learn to think independently.ey also assume that knowledge must differ among students to the extent that the reality in which students construct and apply knowledge is different among students.For present purposes, we use the terms realist, contextualist, and relativist as equivalent to the terms dualist, multiplist, and relativist which have been used widely in the literature [16,18,69,77].We use the former because these terms are used exclusively in extended philosophical treatments of epistemology [1,5,34,59].
Many studies have used two or more of the six measurement strategies, typically with questionnaires or interviews as the primary data collection strategy.In addition, measurement strategies appeared to be chosen based on the research question and goals of the research.e majority of studies have focused on a set of separate epistemological beliefs and therefore have used multi-item questionnaires to assess these beliefs.
Schommer [19] proposed �ve independent beliefs based on the work of Perry [18] pertaining to certain knowledge (i.e., absolute knowledge exists and will eventually be known), simple knowledge (i.e., knowledge consists of discrete facts), omniscient authority (i.e., authorities have access to otherwise inaccessible knowledge), quick learning (i.e., learning occurs in a quick or not-at-all fashion), and innate ability (i.e., the ability to acquire knowledge is endowed at birth).Currently, there is debate as to whether Schommer's �ve beliefs constitute genuine epistemological dimensions [12].Most researchers agree that beliefs about the certainty and simplicity of knowledge constitute genuine epistemological beliefs.In contrast, many researchers argue that beliefs about innate ability re�ect some other, nonepistemological dimension [12].
Schommer [19] developed the Epistemological Questionnaire (EQ) to assess the �ve dimensions described above.e EQ consisted of 62 simple statements that individuals responded to using a �ve-point Likert scale, indicating the extent to which they agreed or disagreed with the statement (see Appendix A, e.g.).e EQ has been used frequently over the past 15 years by Schommer and other researchers [22,67,94,95].Results have been somewhat mixed in a variety of ways.First, factor analyses typically reported less than �ve interpretable factors [49,50].Second, some analyses have yielded factors with an interpretation that was not predicted by Schommer [19].For example, a factor might pertain to the source of knowledge or have item loadings that are difficult to interpret in a manner consistent with Schommer's proposed �ve-factor structure.ird, the factors reported in the analyses oen had a small number of items with acceptable factor loadings and therefore had low or unacceptable reliability coefficients.Fourth, solutions usually explained a relatively small proportion of sample variance (e.g., 20% to 35%), which raised concerns about the construct validity of the questionnaire.
Schraw et al. [50] developed a modi�ed version of Schommer's EQ called the Epistemic Beliefs Inventory (EBI) to address these methodological problems.e EBI contained 32 items based on the �ve factors proposed by Schommer [19] (see Appendix A, e.g.).e EBI typically yielded the �ve proposed factors; however, reliabilities tended to be low (e.g., .50 to .65) and results varied depending on the age and gender of the sample.In addition, like the EQ, the EBI explained a relatively small proportion of sample variation (e.g., usually less than 40%).
Hofer [49] proposed an alternative four-factor framework and developed an instrument called the Epistemological Beliefs Questionnaire (EBQ) to assess these factors.She proposed four factors, which were subsumed under two general dimensions referred to as the nature of knowing and the process of knowing.e former refers to what knowledge is presumed to be, while the latter refers to how one comes to know and understand knowledge.e "nature" dimension included two factors called certainty of knowledge (i.e., the degree to which one sees knowledge as �xed versus �uid and changeable) and simplicity of knowledge (i.e., the degree to which knowledge is viewed as individual facts versus complex, interrelated concepts).e "process" dimension included two factors called source of knowledge (i.e., the extent to which credible knowledge is self-or other generated) and justi�cation of knowing (i.e., the rules and criteria that individuals use to evaluate knowledge claims).
Hofer [49] reported four empirically derived factors that differed somewhat from the four proposed factors described above (see Appendix A, e.g.).Her four observed factors included certainty of knowledge, source of knowledge, justi-�cation, and attainment of truth.A "simplicity of knowledge" factor was not observed, whereas an "attainment of truth" factor was observed.e attainment of truth factor was interpreted as the extent to which experts can attain deep knowledge (i.e., "truth") within their area of expertise.Like the EQ and EBI, the EBQ explained approximately 45% of sample variance, had several factors with few items that loaded satisfactorily, and had factors with low reliability coefficients.
Bromme and colleagues [48,96] have taken a somewhat different approach to measuring epistemological beliefs.ey created a 24-item semantic differential type instrument that measures connotative aspects of student's epistemological beliefs using bipolar adjective pairs that were subsumed under three hypothesized epistemological factors, including certainty, simplicity, and source.For example, the certainty included the certain-uncertain pair; simplicity included the simple-complex pair; and the source factor included the constructed-preexisting pair.Two studies using university students enrolled in either plant biology or chemistry yielded two reliable factors referred to as texture and variability.Texture assessed the complexity and sophistication of beliefs, whereas variability assessed the changeability and permanence of beliefs.
us far, self-report questionnaires that have been developed to assess multiple epistemological beliefs have contributed to the literature in two important ways.One contribution of this research is that there has been a great deal of productive discussion regarding the set of constructs that comprise the domain of epistemological beliefs [2,12,49,[97][98][99].ese discussions are crucial to mapping the possible domain of epistemological beliefs both conceptually and methodologically.A second contribution has been the preliminary �ndings concerning the relationships among epistemological beliefs and a variety of outcomes variables such as age, education level, gender, moral reasoning skills, and academic achievement [67,[100][101][102].Epistemological beliefs that are more relativistic tend to be associated with better problem solving [20,80,86,103], higher self-regulation [2, 99, 104-107], metacognition [91,96,108], constructivist teaching practices [29,89,109,110], and academic career choices [111,112].However, despite positive correlations, the effect size tends to be small in most cases, usually in the .05 to .15 range in terms of variance explained in the outcome measure of interest.
Questionnaires also have experienced a number of difficulties [84].One is disagreement about the number and content of constructs being investigated.Of the four instruments described above, two proposed �ve factors, one proposed four factors, and one proposed three factors.ere has not been a systematic comparison of the instruments; thus, it is unclear which factor structure is best supported by data.A second problem is that factor analytic solutions frequently have failed to replicate [50].is suggests that regardless of proposed factor structure, instruments vary considerably from sample to sample.A third problem is that each factor usually explains a small proportion of sample variance (i.e., less than 15%), while the overall instrument (i.e., all factors combined) likewise explains a small proportion of sample variance (i.e., less than 50%).A fourth problem is that individual factors frequently have marginal reliabilities (i.e., less than .70).Overall, these �ndings suggest that questionnaires should be interpreted carefully within the context in which the data is collected [44].

Measuring Epistemological World Views: Interviews, Vignettes, Essays, and Concept Maps.
A growing body of research has investigated epistemological beliefs and world views using interviews [25,26,52,77,109,[113][114][115][116][117], vignettes [29,53], essays, journals, and storyboards [54][55][56], and concept maps [28,57].ese studies usually differ in terms of methodology and the main research question from studies relying primarily on questionnaire data.Questionnaire studies have focused on using quantitative independent and dependent variables to examine the correlations between variables.In contrast, interview and essay studies have focused on qualitative outcome variables that are used to understand beliefs and world views in greater detail [118,119].ese studies oen focus on how different beliefs and world views affect curriculum and pedagogical choices from the perspective of teachers or students.However, it is important to note that many studies use some combination of the above [53], sometimes with questionnaires as well [77].
Interviews have been used extensively in epistemological research, accounting for approximately 50 to 60 percent of the research in the current review of the literature.Johnston et al. [25] interviewed teachers as part of four case studies to examine teaching differences between teachers with a constructed knowing versus a received knowing epistemological stance.e former emphasizes context-bound, person-constructed knowledge in a student-centered classroom, while the later emphasizes universal, factual knowledge in a teachercentered classroom.Teachers with constructed knowing stances were more likely to emphasize the role of student interaction with teachers and other students.In addition, constructivist teachers used a wider repertoire of teaching strategies and content sources in the classroom [120,121].Similar results have been reported by a variety of other studies that have reported a positive relationship between relativist epistemologies and constructivist classroom practices [51,109,113,116,[122][123][124].
Brownlee et al. [54] examined preservice teachers' epistemological beliefs and whether the promotion of re�ective practice in the classroom changed beliefs in a yearlong teacher education program.Teachers in the re�ective practice condition reported more growth in epistemological beliefs than teachers in the nonre�ective condition.Beliefs changed with respect to teachers expressing more uncertainty, more subjectivity, and greater �exibility in personal beliefs.Although beliefs became more relativistic for all students, changes among the re�ection group were signi�cantly greater than the comparison group.Marra [51] reported similar �ndings in a study of how constructivist instruction affected the development of graduate student teachers at a university.Teachers reported a variety of changes aer the course, but especially changes in epistemological and pedagogical beliefs.Teachers adopted constructivist beliefs that emphasized the role of student interactions.In addition, graduate student teachers adopted stronger contextualist beliefs about the complexity and certainty of knowledge.Overall, a wide variety of studies have reported that re�ection promoted more sophisticated epistemological beliefs [11,14,31,118,125].
Wilson [77] used interviews and surveys to examine whether the amount of education was related to teachers' epistemological stances.Teachers were classi�ed as dualist, multiplist, or relativist.Teachers with graduate degrees tended to be in the multiplist and relativist groups rather than the dualist group.In contrast, teachers with a baccalaureate or less were more likely to be in the dualist group.Tsai and colleagues [52,126,127] reported similar �ndings in a study of Taiwanese science teachers.Teachers with more sophisticated epistemological beliefs and world views were more likely to endorse student-centered instructional practices that emphasized critical reasoning.Teachers with less sophisticated beliefs focused on traditional curriculum, student testing, and mastery of basic science concepts.Kang and Wallace [26] also reported parallel �ndings when examining the relationship between epistemological beliefs and laboratory activities in American high school science classrooms.Lidar et al. [27] found that teachers with more sophisticated personal epistemologies used a greater number of epistemological moves in their science classrooms.Epistemological moves consisted of cognitive activities designed to promote deeper learning and re�ection, including generating, constructing, and reconstructing.ey also reported that the relative success of different epistemological moves depended in large part on contextually speci�c factors such as student knowledge, complexity of activity, and sophistication of students' conceptual understanding.
Vignettes have been used by several researchers in conjunction with interviews and essays.Joram [53] examined preservice teachers' , practicing teachers' , and professors' beliefs about what counts as legitimate knowledge.Participants read two different vignettes that described the effect of research on classroom teaching.In the teaching vignette, research supported classroom practices, whereas in the research vignette, research contradicted classroom practice.Participants read each vignette individually and responded to probe questions about the credibility of evidence and the extent to which research informs teaching.Two �ndings were of special importance.One was that preservice and practicing teachers emphasized the situation speci�c nature of knowledge, whereas professors emphasized the general nature or knowledge and general learning principles to a much greater extent.A second �nding was that students were much more likely to believe that knowledge is transmitted, whereas professors strongly believed that knowledge is constructed or discovered.Joram [53] concluded that students and beginning teachers hold a view of educational knowledge as noncumulative, speci�c, and transmitted, and di�cult to falsify; whereas experienced teachers and professors hold a view of educational knowledge as cumulative, changeable, generalizable, and falsi�able.
Schraw and Olafson [29] created three vignettes shown in Appendix B that assessed realist, contextualist, and relativist world views of knowledge and teaching.Practicing teachers read each vignette and rated the degree to which they agreed with its position.Teachers also completed the Epistemological Belief Inventory [50], the Need for Cognition Scale [128], and a teacher motivation scale that assessed intrinsic and extrinsic motivation to teach.ree �ndings were of special note.One was that teachers strongly supported the contextualist and relativist world views, but not the realist world view.A second �nding was that the need for cognition was correlated positively with the contextualist and relativist world views, but not the realist world view.A third �nding was that years of teaching were correlated positively with the realist world view, but negatively with the contextualist and relativist world view.is suggested that teachers with long service (i.e., greater than 15 years) are more likely to endorse a realist world view and that world views may migrate over time from a contextualist to a realist position.
Essays, journals, and pictorial storyboards have been used as well to explore world views and justi�cation of beliefs.Sandoval [56] investigated whether instruction and electronic journaling using Explanation Constructor promoted change in epistemological beliefs.High school biology students completed �ve hours of computer-based instruction and used an electronic journal in small groups to explain and justify responses.Students produced increasingly sophisticated explanations and justi�cations as their opportunity to engage in group inquiry increased [118].Several other researchers have used written explanations as data.Brownlee et al. [54] asked teachers to write a description and explanation of their epistemological beliefs.She found that students experienced di�culty doing so and oen reported beliefs that con�icted with classroom practice.Asli Özgün-Koca and Ilhan Şen [28] used journals for preservice teachers to justify beliefs outlined in self-generated concept maps about "effective instruction." Olafson and Schraw [55] asked teachers to construct storyboards of a salient teaching episode and to re�ect on the teaching experience.Storyboards consisted of digital photographs and a written description of the progression of events.Participants also were asked to provide written re�ections that considered the ways in which their practices were indicative of their beliefs about teaching and learning.e vast majority of participants endorsed contextualist world views of teaching in their re�ections and provided evidence in the storyboards (e.g., collaborative work, student choice) that was consistent with their views.
Concept maps also have been used in several studies as a way to measure conceptual understanding of a domain and to assess epistemological world views [20,28,57].Asli Özgün-Koca and Ilhan Şen [28] investigated preservice teachers' beliefs about effective instruction.Participants were asked to construct concept maps that linked important concepts such as content and pedagogical knowledge together.Interviews and journals were used aer the construction of concept maps to better understand beliefs embedded within the concept map, the development of beliefs, as well as justi�cation for each participant's beliefs.Results suggested that preservice teachers' have incomplete knowledge about teaching, value student-centered learning, and are concerned about their lack of domain expertise, especially in math and science.
Two other studies utilized concept maps, primarily as a means to evaluate content knowledge and conceptual change over time [20,57].Liu [57] studies 15 high school science students who constructed concept maps as part of their chemistry classroom.Liu used these maps to evaluate the number and sophistication of ontological categories (e.g., chemical processes, interactions).He concluded that ongoing mapping led to greater conceptual change and a richer understanding of information within the targeted ontological categories.Yang [20] used concept maps in conjunction with a questionnaire to determine the conceptual understanding of 10th grade science students.An analysis of concept maps revealed growth over time due to instruction but was not linked to the relative sophistication of each student's epistemological views.

Measuring Ontological Beliefs.
Ontology is a core �eld of study in philosophy [7].Nevertheless, little theoretical or empirical research has been conducted on students' or teachers' ontological beliefs by educational and psychological researchers.Notable exceptions include review and analysis articles by Greene et al. [38], Packer and Goicoechea [39], and Ponterotto [8].Greene et al. [38] proposed a four-group classi�cation system in which individuals are characterized by realism, dogmatism, skepticism, or rationalism based on different modes of epistemological and ontological cognition.Packer and Goicoechea [39] explored the epistemological and ontological differences between sociocultural and constructivists views of learning.Ponterotto [8] compared the epistemological and ontological differences among four competing paradigms used in counseling psychology.
An electronic literature search revealed three empirical studies that examined ontological beliefs.ree of these studies considered the conceptual role of ontological beliefs [57,110,129] but did not provide quantitative measures of ontological beliefs or world views.Liu [57] investigated whether instruction about how to construct concept maps led to conceptual change in a 12th grade chemistry classroom as measured by changes in student's concept maps of the target domain.In addition, �nal concept maps were analyzed with respect to the sophistication of ontological categories within the map.Liu [57] concluded that instruction produced more sophisticated ontological conceptual of the domain of knowledge.She [129] examined whether a Dual-Situated Learning Model (DSLM) that challenged students' beliefs fostered radical conceptual change in ontological categories in among 9th grade students studying heat transfer.
Slotta and Chi [40] investigated the effect of ontological training in promoting conceptual change among physics students.Ontological training consisted of a computerized instructional module that helped students understand four ontological principles related to emergent processes in physical phenomena, which focused on changes that are system wide, equilibrium seeking, simultaneous and independent, and ongoing.College students were classi�ed based on interviews into expert and novice levels of ontological commitment.Results indicated that experts performed better than novices on the posttest as well as the completeness of verbal explanations.e experimental group also outperformed the control on the posttest and verbal explanations.ese �ndings indicated that explicit training helps students become of implicit ontological categories and to learn information and evaluation skills better than the control group.
Olafson and colleagues [55,58,118] asked teachers to situate themselves using a two-dimensional Cartesian coordinate system with respect to epistemological and ontological beliefs.Olafson et al. [118] referred to this method as the fourquadrant scale (see Figure 1).Teachers enrolled in college graduate classes were asked to read written instructions, listen to additional verbal instructions, then situate themselves in one of the four quadrants by making an "" that best corresponded to the extent to which they endorse relativist versus realist epistemological and ontological world views.Teachers also participated in interviews to explain and justify their choices.Several �ndings were of interest.One was a statistically signi�cant correlation of .47 between the epistemological and ontological dimensions.A second �nding was that approximately 90 percent of teachers reported being ontological relativists to one degree or another, whereas about 40 percent reported being epistemological realists to one degree or another.A third �nding was that teachers were inconsistent between their four-quadrant ratings and verbal justi�cations.Speci�cally, although many teachers professed to be highly relativist on the four-quadrant rating, they described teacher-centered realist classroom practices in the verbal interviews.is inconsistency between stated beliefs and practices has been noted previously by researchers [23,24,130,131].

Summary of Measurement
Strategies. is section iden-ti�ed six different measurement strategies used in previous research, including questionnaires, interviews, vignettes, essays, concept maps, and multidimensional scaling methods.Most studies used questionnaires or interviews, with some studies using two or more of these strategies together.Measurement strategies were selected based on the main goals of the study; thus, it is difficult to compare across participants (i.e., students or teachers) phenomena of interest (i.e., beliefs or world views).e vast majority of the studies focused on epistemological phenomena, and only one of these studies investigated both epistemology and ontology simultaneously.
Overall, the extant literature suggests six general conclusions.One is that beliefs and world views can be measured using different types of measurement strategies.Questionnaires, vignettes, and scales were used to measure both epistemological and ontological beliefs and world views.Interviews, journals, and concept maps were used to explore these beliefs in greater detail.Although the bulk of the existing research has focused on epistemological beliefs measured through questionnaires, researcher could use any of the six measurement strategies with either epistemological or ontological phenomena.
A second conclusion is that beliefs and world views are multidimensional and complex [48,49,67,132].For example, many practicing teachers endorsed both contextualist and relativist stances [118,133,134].Both students and teachers clearly hold a variety of beliefs that are situated within epistemological and ontological world views.ese beliefs appear to change over time due to a variety of factors.In addition, beliefs and world views oen are not fully explicit.Several studies suggest that instruction makes beliefs more explicit and more amenable to change.
A third conclusion is that beliefs and world views affect teaching practice [21,31,46].Most studies consistently report that realist perspectives are linked to teacher-centered classroom that emphasize a core body of knowledge and teacherdirected instruction.In contrast, relativist perspectives tend to be link to student-centered classrooms characterized by collaborative learning, tutors, and independent learning opportunities.Nevertheless, a number of studies also reported important inconsistencies between stated beliefs and practices [24,74,118,131].Speci�cally, some teachers profess relativist, student-centered beliefs, but report a strong reliance on teacher-directed instruction, off-the-shelf curriculum, and an emphasis on testing and accountability, though inconsistencies exist.Many practicing teachers also face signi�cant dilemmas with respect to identifying and implementing their personal beliefs [134].
A fourth conclusion is that beliefs and world views are context bound [10,69,92,135].Many teachers and student reported that learning goals and practices were affected by institutional and classroom expectations and developed slowly throughout their university training [136][137][138] and �rst years of teaching [92,139].In addition, objective measurements across multiple domains usually suggested that beliefs changed as a function of the academic domain, as well as the relative sophistication of the materials being studied within that domain [55,97,140].
A �h conclusion is that instruction and short-term intervention designed to promote conceptual change, re�ection, and examination of personal beliefs has a positive effect on epistemological beliefs and classroom practice [141].For example, Olafson et al. [118] found that a 16-week collegelevel class that included discussion of beliefs, re�ective writing and discussion activities, and action research projects in school settings affected beliefs and signi�cantly changes attitudes about teaching practices.Generally, individuals become more likely to use contextualist, student-centered teaching practices and focus on large-grain thinking skills [21,42,73,74,114].A variety of studies have reported similar �ndings [142][143][144][145].
A sixth conclusion is that teachers' beliefs and classroom practices affect students' beliefs and learning [146][147][148].A variety of studies have found that teachers' epistemological beliefs are related positively to student epistemological development and learning outcomes and shape students' emerging conceptions of knowledge [27,51,127].Weinstock and Roth [149] also showed that teachers' epistemological beliefs are related positively to student autonomy.

A Comparison of Constructs and Measurement Strategies Used in Previous Research
Research to date has focused primarily on individual epistemological beliefs, a global epistemological world view, or a global ontological world view.ere are no empirical studies that we located that examined multidimensional ontological beliefs analogous to the multibelief models proposed by Schommer [19] or Hofer [49].Most studies have relied on one main measurement strategy using self-report questionnaires; however, approximately 40 percent of studies used a second data collection method such as an interview.Different measurement strategies have been used for different research purposes.Table 2 shows that studies focusing on students' epistemological beliefs have relied primarily on questionnaires.In contrast, studies that addressed teachers' world views have relied primarily on vignettes, interviews, essays, or some combination of these methods [26,28,54].e few available studies of ontology have used a variety of methods, including interviews [129], an analysis of concept maps [57], short answer responses and interviews [40], and multidimensional scales [58,118].ere are no studies in the literature that systematically compared across different measurement strategies.
Table 3 summarizes the construct being assessed, design of the measurement tool, and relative strengths for each of the six measurement strategies described here.Each measurement strategy differs with respect to the ease of administration, type of information, and richness of information that it provides.All of the strategies shown in Table 3 complement one another and can be used together in virtually any study.Indeed, a number of the studies reported above used multiple strategies such as questionnaires, interviews, and essays [114]; concept maps, interviews, and journals [28]; or questionnaires, multidimensional scales, and interviews [118].
Questionnaires and interviews clearly are the strategies of choice.Questionnaires are quick and easy to administer and amenable to sophisticated statistical analyses of reliability, validity, and the relationship of each variable to other variables.In contrast, interviews are time consuming to administer and score, but provide extremely rich data about beliefs, belief change, and how beliefs are related to classroom practices.
is review foregoes an in-depth discussion of each of the six strategies, which are described at length in many research textbooks [4,[151][152][153], focusing instead on three important omissions in the literature.One is the absence of a questionnaire that measures ontological beliefs.Hofer and Pintrich [12] noted previously that many epistemological questionnaires also appear to measure ontological beliefs and assumptions.Notwithstanding this claim, there is no scale in the current literature that focuses on this goal, provides psychometrically reliable and valid scales, and most importantly provides a detailed discussion of the ontological construct(s) being measured.
A second omission is the lack of cross-validation research within and between studies.Studies that used multiple measurement strategies typically reported the results from each strategy in an autonomous manner.ere was very limited discussion regarding the consistency across strategies and rarely any quantitative data (e.g., correlations or interrater agreement) across methods.In addition, it is not clear based on these studies whether one strategy is better suited to investigate different types of epistemological and ontological beliefs and world views.

Instruments
Epistemological Questionnaire [19] Interviews [25,51,52,54] Four quadrant scale [58] Epistemological Belief Inventory [150] Vignettes [29,53] Epistemological Beliefs Questionnaire [49] Essays and journals [28,56] Connotative Aspects of Epistemological Beliefs [48] Concept maps [28,57] T A third omission is the lack of what I refer to as universal measurement design principles, by which I mean using a measurement strategy in a consistent manner across studies.ere are at least four ways this might be accomplished.One way is thorough de�nition of constructs being investigated and the domain of study.Many studies provided a poor or missing description of critical constructs that were being investigated.A second way is through content consistency, by which I mean using the same items on a questionnaire or structured interviews as other researchers.One strategy would be to use two separate self-report questionnaires with a common set of linking items.A third way is to use consistent administration procedures such as order of presentation, instructions provided to participants, amount of time to complete the assessments, and conditions under which the data is collected.A fourth way is through consistent use of scoring procedures.Indeed rubrics and scoring methods were poorly described in the research, either within a study or across studies using the same methods.Structured interviews typically differed and used very different coding schemes.For all of these reasons, it may be difficult to generalize across studies, even those using similar methodology and measurement strategies.

The Challenge of Conceptual Integration
is section considers six ways to promote better conceptual understanding of epistemological and ontological beliefs.ese include developing a conceptual framework, distinguishing between tacit versus explicit beliefs, examining whether beliefs are domain speci�c versus domain general, examining in more detail the relationship between beliefs and learning, investigating how teachers' beliefs are related to classroom practices, and tracking the development of teachers' beliefs.

Develop an Integrated Conceptual
Framework.Previous research has not provided an integrated conceptual framework for understanding epistemological and ontological beliefs.Hofer [41] and Hofer and Pintrich [12] presented a well-articulated conceptual framework for conducting epistemological research and raised important questions about the boundary between epistemology and ontology.Bendixen and Rule [66], Muis [105], and Schommer-Aikins [67] also provided models that link epistemological beliefs to other related constructs such as metacognition and self-regulation.However, with the exception of Packer and Goicoechea [39] and Ponterotto [8], there has not been a clear explication of ontological beliefs.
I propose the framework shown in Figure 2 as a preliminary guide for conceptualizing and investigating core areas of epistemology and ontology in future research.Figure 2 considers both epistemological and ontological beliefs under the general category of teacher and student beliefs, which might also include self-efficacy, goal orientation, and attributions beliefs [154,155].
ree distinct categories of inquiry are relevant to understanding epistemological beliefs, including types of knowledge and ways of knowing, sources of knowledge and evidence, and justi�cation of knowledge.Beliefs within each of these categories may differ in important ways [156].Hofer and Pintrich [12] provided an in-depth discussion of these three questions a decade ago; however, the literature would bene�t from an updated discussion based on research that has occurred over the last decade.Each of these categories could be partitioned further into more speci�c questions.For example, researchers could create a separate taxonomy of types of knowledge (e.g., [157]) and beliefs about different types of knowledge (e.g., [19]).A separate taxonomy could be created for ways of knowing (e.g., [158,159]).Similarly, sources of knowledge and issues surrounding the justi�cation of knowledge could be partitioned into �ner grained subcategories as shown in Table 1.
Two categories of inquiry are relevant to ontology, including beliefs about reality and beliefs about the nature of being [8,39].Beliefs about reality may include questions about permanence and evidential support for one or more perceived realities.Beliefs about personal being may include perceptions, beliefs, and attitudes teachers have about themselves as teachers, as well as the fundamental nature of schooling.As with epistemological categories, ontological categories could be partitioned into subcategories.For example, very little is known about teachers' beliefs about the reality of the classroom.A codi�ed taxonomy of these beliefs and their implications for classroom instruction would make a very important contribution.Similarly, researchers lack a comprehensive framework for understanding teachers' sense of being teachers.
Of equal importance to any conceptual framework of epistemology and ontology is what the two domains of belief have in common.Regarding education, virtually nothing has been written about the development and nature of teachers' "metabeliefs, " by which I mean an integrated belief system that includes personal epistemology, ontology, self-concept, and other core beliefs that teachers bring to their classroom practice.I assume that similar experiences affect every aspect of a teacher's beliefs.Unfortunately, researchers have little awareness of whether one set of beliefs (e.g., ontology) has a causal impact on other beliefs (e.g., epistemology), whether effects are fully reciprocal, or whether foundational beliefs such as epistemology and ontology are independent.

Tacit versus Explicit Beliefs.
Students and teachers hold many beliefs, some of which are explicit, and some of which are more implicit.Little is known currently about the explicitness of epistemological and ontological beliefs, or whether explicitness affects re�ection on beliefs and classroom decisions.Most researchers assume that these beliefs are at least partially implicit [12,29,66,160].
Of special interest is how implicit beliefs become explicit over time and what fosters this change.Pirttilä-Backman and Kajanne [64] found that Finnish adults' beliefs became more explicit in large part due to continued education.is �nding is consistent with Kuhn [16] and Baxter-Magolda [11].A number of studies have reported epistemological change and greater explicitness awareness of one's beliefs, due to classroom re�ection between teachers and students [27], constructivist activities [51,117], doubt [66,161,162], and cognitive con�ict [68,156,[163][164][165].

5.�. ���ain�Speci�c versus ���ain��eneral Beliefs.
Researchers have addressed whether epistemological beliefs and world views are domain speci�c or domain general in nature.Some researchers argue that beliefs are domain speci�c [41].On this view, one's epistemological beliefs are inseparable from one' domain-speci�c knowledge, and the beliefs that one holds in one domain are not related to beliefs in another domain, but especially when comparing familiar versus unfamiliar domains.In general, the domain-speci�c hypothesis is consistent with models of domain learning [166] and expertise [167] that suggest expertise is encapsulated within a domain and does not readily transfer to other domains.
An alternative view is that epistemological beliefs tend to be domain general [16,67].On this view, epistemological beliefs develop as general dispositions about the world; thus, one's assumptions about the origin and legitimacy of knowledge are likely to be similar in different domains.is view is consistent with model of domain general selfregulation [105,168] and metacognition [78] that postulates higher-order skills such as planning and monitoring that generalize in skilled learners to all domains.
Muis et al. [106] reviewed 19 empirical studies and concluded that beliefs are both domain speci�c and domain general.ey proposed a model in which individuals hold domain speci�c beliefs that differ between domains, while also holding domain general beliefs that are more or less consistent across a unique culture.ey also proposed a developmental sequence in which domain-speci�c beliefs may contribute to broader domain general beliefs.

Student Beliefs and Learning.
Previous research has provided a great deal of information regarding the effect the epistemological beliefs have on student learning [105].For example, epistemological beliefs are related to overall learning [2,67,169], deeper learning [26,56,86], evaluation of evidence [20], academic career choices [112], increased self-regulation [27,104,105,170], and teacher-student interactions [52].In general, students and teachers with contextualist or relativist beliefs learn more and outperform others with realist beliefs [16,25,52,67].Similar �ndings have been reported with teachers, although fewer studies have been conducted compared to college students [55].
Virtually all previous studies have focused on epistemological beliefs without considering ontological beliefs.One issue is whether the empirical relationships between ontological beliefs and learning outcomes are comparable to the effect of epistemological beliefs on learning.One possibility is that ontological beliefs, but especially teachers' ontological beliefs about the reality of teaching, would play an instrumental role in curriculum and pedagogy decisions.

Teacher Beliefs and Classroom
Practices.Few studies have examined teachers' epistemological world views in detail, and none that we know of have examined ontological beliefs.It is possible in some cases that preservice training has little effect on preservice teachers' world views, in part because younger students may not be developmentally ready to consider difficult to resolve epistemological and ontological issues [21,24,34,51,77,155,171].However, it is more likely that the type of preservice training a student receives affect his or her beliefs differently [172,173].
Existing studies suggest that preservice teachers bene�t from an explicit discussion of beliefs, especially those related to student-centered teaching [21,54].One of the most effective belief-change strategies is re�ection on teaching practice through collaborative discussions and re�ective journaling [32,51,114], in part, because it makes competing world views more public to preservice teachers [53].ese activities appear to give preservice teachers a greater awareness of their existing beliefs, as well as an opportunity to discuss alternative beliefs with other students and their teacher.

e Development of Teacher
Beliefs.Previous research suggests that teachers develop over time regarding teaching practices and that their beliefs change as part of their development [54,143,155].Very little is known currently regarding the development of teachers' epistemological and ontological beliefs [65].One issue is whether the two beliefs develop in tandem or follow separate trajectories.A second issue is the general trend in development [123,141].It seems reasonable to assume that beliefs become more relativist over time, but may become stable at some point, or perhaps move from a relativist to realist world views past a certain number of years of experience [174,175].Cross-sectional studies would help address these questions, although longitudinal studies would be especially useful for understanding the development of beliefs.
One important need is a developmental model of belief development that includes a wide range of sociocultural factors [67].Several questions seem especially germane, including the impact of different variables (e.g., family, education) on belief development, whether epistemological and ontological beliefs develop in synchrony, and whether there is a zone of readiness [51] that facilitates belief development.
One requirement of such a model is that it is tested on epistemological and ontological beliefs.Such a model should explain change and development across the two domains of belief or demonstrate that the two domains differ with respect to their developmental trajectories.

The Challenge of Improving Measurement
Most research described in this paper is based on the use of questionnaires to assess epistemological beliefs [12,44,67,105,106,132,150].I make a variety of suggestions in the following for improving the measurement of beliefs, including the development of new instruments, utilizing new methods, improving reliability of assessments, and providing relevant validity evidence.

Develop New Instruments.
Most research utilizes selfreport questionnaires of epistemological beliefs.is situation could be improved by developing a questionnaire to assess ontological beliefs that assess beliefs about reality and beliefs about personal being as suggested in Figure 2. is would enable researchers to assess both epistemological and ontological beliefs simultaneously from the same individuals.Researchers also should standardize other assessments such as interviews to compare epistemological and ontological beliefs within and between individuals.Speci�cally, interviews could use a common format and a common scoring rubric to more effectively compare results.

Utilize New Methods.
Most studies underutilize the type of measurements used to study beliefs.Researchers should explore the use of concept maps [20,28,57], multidimensional scaling [58], Vignettes [53,118], and essays, journals, and storyboards [54][55][56].A variety of other methodologies could be employed as well such as Q-sorts [176,177] in which teachers prioritized beliefs and beeper studies [178] in which individuals are stopped while engaged in vivo in real-time activities in order to report beliefs and ongoing measurable activities.Using a semantic differential tool like that used in Stahl and Bromme [48] provides a quick and easy data-collection strategy.Diaries, journals, videos, and portfolios [179] also have been underutilized in beliefs research in a manner that eliminates "context-stripping" from the interpretation of actions and teacher choices in actual classroom contexts.

Improve Reliability. e reliability of instruments used
to assess beliefs oen is low [44].is could be improved by additional instrument development work that provides better construct de�nitions and additional assessment items [152].Reliability could be improved as well as standardizing interview formats and rubrics used to score interviews [4].
Another way to improve reliability and interpretability is to use a universal measurement design across the �ve subdomains shown in Figure 2. Ideally, the same data collection method (e.g., interviews, concept maps) could be used to assess epistemological and ontological beliefs at the same time.is would enable researchers to compare beliefs in a more reliable manner.
A third strategy is to use statistical techniques that partition difference types of variation among variables.For example, structural equation modeling enables researchers to simultaneously estimate the error variance in a variable (i.e., variable 1) and the variance that is shared between variable 1 and variable 2. Unfortunately, few studies have used statistical techniques to compare either the similarity between parallel measures of beliefs [84] or to model the measurement error of individual variables in a complex structural model.

Provide Relevant Validity Evidence.
It is ironic that research on epistemological and ontological beliefs rarely discusses that extent to which constructs are clearly de�ned and assessed in an appropriate manner.Even a simple threestep approach can improve this situation immensely.e �rst step is to describe the organizational framework and dimensionality of beliefs in a manner consistent with Figure 2. Doing so enables the researcher to describe the number of constructs being measured.A second step is to provide clear de�nitions of the constructs of interest such as those in Table 1.A third step is to describe theoretically the relationships among the variables in step 1; that is, how variables , , and  are related to one another.Ideally, this should be done using a structural model that illustrates the direction and strength of the predicted relationships between variables.
Validity would be improved if researchers used multiple measures to gather convergent evidence of each type of targeted belief.I encourage researchers to do so to develop what has been known historically as a nomothetic net [9] that integrates theory and measurement into a single conceptual framework that can be evaluated using a multiple-trait, multiple-measure validation strategy that uses multiple traits (e.g., strategy use, planning, monitoring, interest, and selfefficacy) and multiple methods (e.g., self-report and thinkalouds).For example, on-line strategy use could be evaluated based on think-aloud reports, eye tracking data that supports or fails to support intended strategy use described before the study, interactions with tutors, as well as palette choices made in real time.Using multiple measures and methods should enhance the interpretative validity of conclusions by combining and synthesizing multiple sources of data and allowing researchers to examine the concurrent validity (i.e., real-time relationships) among different measures.

Summary
is paper addressed three main goals.e �rst was to provide de�nitions for terms frequently encountered in epistemological and ontological research.Table 1 provides detailed de�nitions and appropriate sources.e second goal was to review existing measurement strategies.Table 3 compared and contrasted six strategies in current use.A third goal was to elaborate on challenges of conceptual integration for future research.Figure 2 provided an integrated conceptual framework.
Six themes emerged from this work.One was the lack of clear, universally shared de�nitions of epistemological and ontological phenomena.Table 1 provided de�nitions to resolve this problem.A second theme was that educational researchers have all but ignored the role of ontological beliefs in classroom practice, focusing instead on epistemological beliefs.I argued that epistemological and ontological beliefs differ from one another and must be taken into consideration collectively.A third theme was that researchers have underutilized assessment strategies, relying primarily on questionnaires and interviews.Several additional suggestions were made, including the use of Q-sorts, beepers, journals, and portfolios to augment existing measurements.A fourth theme was that a uni�ed theoretical framework is needed to advance theory and research.I proposed a framework in Figure 2 consistent with the existing literature.A �h theme was that future research should consider the roles of epistemological and ontological beliefs, booth collectively and separately.It is unclear currently whether either type of belief is more important in terms of classroom practice and develops in a similar manner.A sixth theme was that better measurement in the form of new and more valid measures would advance our understanding of beliefs.I summarized four ways that researchers may do so in the future.

Appendices A. Examples from Epistemological Questionnaires
Schommer's Epistemological Questionnaire Successful students understand things quickly.
Scientists can ultimately get to the truth.
e only thing that is certain is uncertainty itself.
e most important part of scienti�c work is original thinking.

Schraw et al. 's Epistemological Beliefs Inventory
When someone in authority tells me what to do, I usually do it.
e moral rules I live by apply to everyone.
What is true today will be true tomorrow.
Most things worth knowing are easy to understand.
Smart people are born that way.

Hofer's Epistemological Beliefs Questionnaire
All experts in this �eld understand the �eld in the same way.
First-hand experience is the best way of knowing something in the �eld.
Sometimes you have to accept answers from the experts in this �eld, even if you do not understand them.
Experts in this �eld can ultimately get to the truth.

B. Three Epistemological World Views
B.1.Realist Worldview.ere is a core body of knowledge in my classroom that each student must learn.Some of it is factual, but some of it is based on broad concepts and principles that everyone agrees on.is knowledge does not change much over time and represents the accumulation of important truths and understanding in my discipline.It is important for students to acquire this knowledge exactly as it is.e best way to acquire this knowledge is through an expert like me because I have a much better sense than they do of what is important to learn.It is unlikely that students could really create this knowledge on their own, so learning it from me quicker and more efficient.For this reason, it is important to me to assume a take-charge attitude so students can learn as much as possible.It is important to me that everyone comes away from my class with the big picture.It is my job to present the big picture clearly.
B.2. Contextualist Worldview.Students are encouraged to develop their own understanding in my classroom so knowledge is personally useful to them.However, the fact that students are expected to construct their own understanding does not mean that all understandings are equally valid.While I believe that knowledge is subject to interpretation, I also believe that some conclusions are better than others.
Students need to understand how to gather and evaluate evidence so they can distinguish good from poor arguments.I can teach them some of these skills, but some they will have to learn by working with other students, or on their own.I believe that each student will bring a unique and valuable perspective with them.I try to structure my class so that students will pool their resources and come to the best understanding possible.

B.3. Relativist Worldview.
Students in my class need to understand that there are a variety of different ways to understand things.Knowledge comes and goes, and what the so-called experts consider the truth today will be viewed with suspicion tomorrow.Even people who spend years studying a topic disagree about what things mean, and in the long run, one opinion is as good as another.is means that students have to learn to think for themselves, question the knowledge and authority of others, and evaluate how what they know affects their life.Knowledge has to be used wisely so no one is le out or exploited by society.For these reasons, I do not believe that I can really teach my students what is important, since they all need to know different things.ey have to �gure it out on their own, taking into account the events that shape their lives, even if the uncertainty of living in a world with con�icting views of truth bothers them.What I know and believe should not really in�uence my students.My job is to create an environment where students can learn to think independently and take nothing for granted.

T 2 :
Constructs and instruments used in research.
3: Six measurement strategies used in research.