Vol 4 no 1 april 2014 by ijlter.org

IJLTER.ORG

p-ISSN: 1694-2493 e-ISSN: 1694-2116

International Journal of Learning, Teaching And Educational Research

Vol.4 No.1

PUBLISHER London Consulting Ltd District of Flacq Republic of Mauritius www.ijlter.org

International Journal of Learning, Teaching and Educational Research

The International Journal of Learning, Teaching and Educational Research is an open-access journal which has been established for the disChief Editor Dr. Antonio Silva Sprock, Universidad Central de semination of state-of-the-art knowledge in the Venezuela, Venezuela, Bolivarian Republic of field of education, learning and teaching. IJLTER welcomes research articles from academics, edEditorial Board ucators, teachers, trainers and other practitionProf. Cecilia Junio Sabio ers on all aspects of education to publish high Prof. Judith Serah K. Achoka quality peer-reviewed papers. Papers for publiProf. Mojeed Kolawole Akinsola Dr Jonathan Glazzard cation in the International Journal of Learning, Dr Marius Costel Esi Teaching and Educational Research are selected Dr Katarzyna Peoples through precise peer-review to ensure quality, Dr Christopher David Thompson originality, appropriateness, significance and Dr Arif Sikander readability. Authors are solicited to contribute Dr Jelena Zascerinska to this journal by submitting articles that illusDr Gabor Kiss trate research results, projects, original surveys Dr Trish Julie Rooney Dr Esteban Vázquez-Cano and case studies that describe significant adDr Barry Chametzky vances in the fields of education, training, eDr Giorgio Poletti learning, etc. Authors are invited to submit paDr Chi Man Tsui pers to this journal through the ONLINE submisDr Alexander Franco sion system. Submissions must be original and Dr Habil Beata Stachowiak should not have been published previously or Dr Afsaneh Sharif be under consideration for publication while Dr Ronel Callaghan Dr Haim Shaked being evaluated by IJLTER. Dr Edith Uzoma Umeh Dr Amel Thafer Alshehry Dr Gail Dianna Caruth Dr Menelaos Emmanouel Sarris Dr Anabelie Villa Valdez Dr Özcan Özyurt Assistant Professor Dr Selma Kara Associate Professor Dr Habila Elisha Zuya

VOLUME 4

NUMBER 1

April 2014

Table of Contents Angovian Methods for Standard Setting in Medical Education: Can They Ever Be Criterion Referenced? ............. 1 Brian Chapman Development Model of Learning Objects Based on the Instructional Techniques Recommendation ....................... 27 Antonio Silva Sprock, Julio Cesar Ponce Gallegos and María Dolores Villalpando Calderón Influential Factors in Modelling SPARK Science Learning System ............................................................................... 36 Marie Paz E. Morales Investigating Reliability and Validity for the Construct of Inferential Statistics ......................................................... 51 Saras Krishnan and Noraini Idris Influence of Head Teachers‟ Management Styles on Teacher Motivation in Selected Senior High Schools in the Sunyani Municipality of Ghana ......................................................................................................................................... 61 Magdalene Brown Anthony Akwesi Owusu Comparison and Properties of Correlational and Agreement Methods for Determining Whether or Not to Report Subtest Scores ....................................................................................................................................................................... 61 Oksana Babenko, PhD. and W. Todd Rogers, PhD Analysis of Achievement Tests in Secondary Chemistry and Biology ......................................................................... 75 Allen A. Espinosa, Maria Michelle V. Junio, May C. Manla, Vivian Mary S. Palma, John Lou S. Lucenari and Amelia E. Punzalan Towards Developing a Proposed Model of TeachingLearning Process Based on the Best Practices in Chemistry Laboratory Instruction ......................................................................................................................................................... 83 Paz B. Reyes, Rebecca C. Nueva España and Rene R. Belecina

International Journal of Learning, Teaching and Educational Research Vol. 4, No.1, pp. 1-26, April 2014

Angovian Methods for Standard Setting in Medical Education: Can They Ever Be Criterion Referenced? Brian Chapman School of Rural Health (Churchill)1 Faculty of Medicine, Nursing and Health Sciences Monash University Churchill, Victoria, Australia Abstract. This paper presents a discussion of Angovian methods of standard setting – methods which are widely used with the intent of defining criterion-referenced absolute standards for tests in medical education. Most practitioners, although purporting to pursue absolute, criterion-referenced standards, have unwittingly slipped into focussing on norm-referenced concepts of „borderline‟ students and their predicted ability to answer assessment items in a test. This slippage has been facilitated by a shift in language from the original concept of „minimally acceptable‟ persons to the modern concept of „borderline‟ persons. The inability of university academics to predict accurately the performance of „borderline‟ graduate-entry medical students is illustrated by presentation of data obtained from three successive cohorts of a small regional medical school during the years 2010-2012. Other data are presented to show how student performance, both „borderline‟ and general, can be significantly altered by switching from didactic lectures to tutorials preceded by task-based active learning. A protocol, based on a stricter interpretation of what is meant by a „minimally acceptable‟ person, is suggested for moving towards a more criterion-referenced standard for a test based on the curriculum‟s learning objectives. Nonetheless, the fallibility of criterion-referenced standard-setting processes means that norm-referenced relative standards may need to be brought into play to deal with anomalous grade results should they arise. The ideal of defining an absolute criterion-referenced standard for a test, using the most commonly implemented Angovian method, is probably as least as unattainable for graduate-entry medicine as it has been previously shown to be for secondary school science. Keywords: standard setting; Angoff method; medical education; normreferenced standard; criterion-referenced standard

Formerly Gippsland Medical School (2007-2013).

1. Introduction Medical schools are required to define standards of quality assurance in the assessment of medical trainees such that society can have confidence in the professional competence of medical graduates once they are registered to practice. To this end, the quality of a medical curriculum is defined by the clarity and comprehensiveness of its stated learning objectives, and the efficacy of the teaching and learning processes directed towards the attainment of those objectives is assessed using a variety of measuring instruments. These instruments may include written examinations comprising multiple-choice questions (MCQs), extended matching questions (EMQs) and short-answer questions (SAQs), viva voce examinations such as the Objective Structured Clinical Examination (OSCE), or a variety of essays and assignments. Quality assurance then focuses on defining a minimally acceptable standard of competence for each assessment question or task encountered by the students as they progress through the course. In common with most licensing and certifying operations, it is desired to define an absolute standard of competence for assessing the quality of a medical graduate rather than a relative standard expressed by comparison either with other candidates in a given cohort or with the performance of preceding cohorts. The definition of an absolute standard is called criterion referencing while the use of a relative standard is called norm referencing; application of criterion referencing is intended to establish minimum standards of competence and this is widely held to be preferable to norm referencing (Searle, 2000; Norcini, 2003; Downing, Tekian, & Yudkowsky, 2006). A cursory glance at the literature on assessment in medical education will reveal the widespread use of methods for standard setting attributed to William H. Angoff (1919-1993), a researcher at the Educational Testing Service in the United States for 43 years, whose main contributions to educational research and practice were focussed on the measurements used in testing and scoring. The key reference cited for this attribution is Chapter 15 „Scales, Norms, and Equivalent Scores‟ in Educational Measurement, Second Edition, edited by Robert L. Thorndike (Angoff, 1971). Yet, within this 93-page chapter, as many people have noted over the years (e.g., Zieky, 1995, pp.8-9; Cizek & Bunch, 2007, p.81), Angoff‟s original description is very short, comprising no more than nine sentences of text distributed between two paragraphs and an associated footnote. The purpose of this paper is to present a critical discussion of the rationale, intent and implementation of the most widely used of the several variants of Angoff‟s (1971) original suggestion that have emerged. Preparation for this discussion reveals that very little can be said today in criticism of Angovian procedures that has not been said before. This suggests that much current practice is based on pragmatism, allowing standard setting to proceed for any number of reasons, including ignorance or wilful disregard of the many objections and concerns that have been raised in the past. It is not the aim of this discussion to review this literature or to rehearse old arguments. Rather, the original contribution sought here is to illuminate the discussion by identifying the problems of linguistic imprecision and conceptual vagueness that have ©2014 The author and IJLTER.ORG. All rights reserved.

interacted and confounded the most well-intentioned quests for defining absolute assessment standards in medical education. Samples of original assessment data are included to illuminate the discussion further. The detailed analysis and discussion will be usefully facilitated by distinguishing two methods of standard-setting contained in Angoff‟s (1971) original description: Angoff‟s Text Method (ATM), and Angoff‟s Footnote Method (AFM).

2. Angoff’s Text Method (ATM) The context for Angoff‟s original description is the standard-setting problem of finding a suitable „pass mark‟ and „honours mark‟ on a scale applied to a test, such cut scores being “decided on the basis of careful review and scrutiny of the items themselves” (Angoff, 1971, p.514). The aim was to establish standards that would be independent of normative data relating to actual “performance as it exists” (p.514). This raises immediately the problem of determining whether a test is criterion referenced or norm referenced. Although Angoff doesn‟t express the issue in these words, it seems that he is striving to define a criterionreferenced standard that will stand immutable in the face of actual performance data. To this end, Angoff‟s (1971, pp.514-515) two paragraphs specify ATM as follows: A systematic procedure for deciding on the minimum raw scores for passing and honors might be developed as follows: keeping the hypothetical “minimally acceptable person” in mind, one could go through the test item by item and decide whether such a person could answer correctly each item under consideration. If a score of one is given for each item [p.515] answered correctly by the hypothetical person and a score of zero is given for each item answered incorrectly by that person, the sum of the item scores will equal the raw score carried by the “minimally acceptable person.” A similar procedure could be followed for the hypothetical “lowest honors person.” With a number of judges independently making these judgments it would be possible to decide by consensus on the nature of the scaled score conversion without actually administering the test. If desired, the results of this consensus could later be compared with the number and percentage of examinees who actually earned passing and honors grades [emphasis added].

Looking back over the ensuing four decades or more since the above words were written, it may be safely concluded that the thinking of Angoff in this specific instance, and of all those who have subsequently used Angovian methods of standard setting, has been dominated by the view that assessment “should be objective, measurable and „certain‟ (and therefore that assessment can be made reliable and valid)” (Williams, 2008, p.402). This is implicit in the notion that an absolute standard for a test can be set “without actually administering the test” and that such a standard might be compared with “performance as it exists” … “if desired”. ©2014 The author and IJLTER.ORG. All rights reserved.

However, Angoff‟s (1971) prescription is far from being a robust example of criterion referencing in action. This is because of the different constructions that might reasonably be placed upon the first paragraph. 2.1 Angoff’s First Paragraph The wording of this paragraph, as quoted above, is insufficiently precise to yield a single, unambiguous reading. As Zieky (1995, p.9 footnote 4) has noted, Angoff‟s (1971) use of the word „could‟ is frustrating in its lack of precision and its uncertain distinction from alternatives such as „should‟ or „would‟.2 On this matter Impara and Plake (1997, p.363 note 1) also observe that „should‟ is typically interpreted as a higher target than „would‟. In the same footnote 4 of Zieky (1995, p.9), we find that, when Zieky personally asked Angoff in the early 1980s which of „could‟ or „would‟ was correct, Angoff “replied that he did not think it mattered very much”. This is very illuminating and it suggests that Angoff did not think it important to develop a full appreciation of the importance of linguistic precision in defining an unambiguous method for establishing a criterion-referenced standard.3 In that sense, therefore, Angoff (1971) did not set his method on a sufficiently firm foundation. However, let us choose the least vague of the three alternatives – would – and see where that leads us. The criterion-referenced standard-setting prescription of ATM then becomes: Keeping the hypothetical “minimally acceptable person” in mind, one could go through the test item by item and decide whether such a person would answer correctly each item under consideration.

There remains a problem with the dual focus of the procedure. Does the focus lie on the concept of the minimally acceptable person or on the content of each item? The crucial linguistic watershed here is Angoff‟s (1971) concept of the “minimally acceptable person”. It is possible to construct this concept in two ways, one lending itself more readily to criterion referencing than the other. A. Firstly, it may be given a more criterion-referenced construction by implicit reverse engineering of the text. In short, by definition, a “minimally acceptable person” will answer correctly every item that the assessors have identified as embracing the „minimally acceptable performance‟ criteria for the test. So the pass mark becomes the sum of all the marks deriving from such „minimally acceptable performance‟ items. The procedure under this construction would be to ask of the item, not whether a “minimally acceptable person” could answer it correctly, but whether it encapsulates an element of „minimally acceptable

2Angoff

(1971) uses „would‟ in the “slight variation” of ATM represented by AFM.

As suggested later (see Footnote 8), there is no reason why Angoff (1971) should have given these matters any more thought or space than he did within the context of his original article. This is the responsibility of contemporary users of Angovian methods. 3

performance‟. It seems that this construction has been attempted rarely, if at all. B. Alternatively, it may be given a less criterion-referenced construction by allowing the difficulty of the item (as distinct from its criterion-referenced content) to weigh in the assessors‟ estimates as to whether or not a “minimally acceptable person” would answer the item correctly. This question cannot be answered with any certainty because the focus has moved away from the item‟s content to the ability of a “minimally acceptable person” to answer the item successfully. Any estimate of this ability must necessarily take into account a margin for error that such guesswork entails. This construction inevitably tends towards norm referencing, where the „norm‟ is a person or group of persons of indeterminate worthiness of passing a test or progressing to the next level. We shall return to the more criterion-referenced option later in the discussion but, for now, we must deal with the fact that the overwhelming majority of practitioners have not placed such an interpretation on the prescription of ATM. Two factors seem to have generated this situation: one deriving directly from Angoff‟s footnote to his first paragraph; the other deriving from, and perhaps concealed by, the subtle shift in language from that used by Angoff (1971) to that used widely today. Let us deal with the footnote first.

3. Angoff’s Footnote Method (AFM) As an exemplar of criterion referencing, Angoff‟s prescription stumbles at the first hurdle through the barely perceptible „sleight-of-hand‟ that occurs as we switch from Angoff‟s first paragraph to its associated footnote. Angoff (1971, p.515) offers an alternative to the procedure outlined in the first paragraph of ATM by specifying AFM as follows: A slight variation of this procedure is to ask each judge to state the probability that the “minimally acceptable person” would answer each item correctly. In effect, the judges would think of a number of minimally acceptable persons, instead of only one such person, and would estimate the proportion of minimally acceptable persons who could answer each item correctly. The sum of these probabilities, or proportions, would then represent the minimally acceptable score. A parallel procedure, of course, would be followed for the lowest honors score.

This footnote has achieved a prominence far outweighing its casual inclusion in the original article, specifying the most widely-used procedure for implementation of a method purported to produce a criterion-referenced standard that seeks to establish competence (Mills & Melican, 1988; Norcini, 2003; Amin, Chong, & Khoo, 2006; Lypson, Downing, Gruppen, & Yudkowsky, 2013). Nonetheless, AFM cannot be regarded as a „slight variation‟ of ATM if ATM is implemented according to the strategy given in Section 2.1 A above. According to a strictly criterion-referenced view of „minimally acceptable‟ for the ©2014 The author and IJLTER.ORG. All rights reserved.

establishment of a pass mark, each item in a test must be given a binary value of 0 or 1 as a multiplier of the respective mark attached to the item (1 for all „must know‟ items, 0 for all other items). But this view cannot accommodate a compromise notion of „probability‟ where the value of the probability may take non-binary values other than 0 or 1. The fact that Angoff (1971) regarded AFM as a „slight variation‟ of ATM suggests that he may not have appreciated the extent to which he lost sight of his assumed criterion-referenced goal almost as soon as he tried to illustrate how it might be achieved. While most applications of Angoff‟s methods have adopted the AFM approach of assigning probabilities over a continuous range between 0 or 1, judges have clearly found the procedure difficult (Norcini, 2003, p.466). Such difficulties apparently led to the „re-discovery‟ of ATM by Impara and Plake (1997), mistakenly reported by Jalili, Hejri, and Norcini (2011) as being proposed in 1997 as “another variation” of the Angoff method, now called the Yes/No Angoff method. In turn, difficulties with applying the Yes/No method to test items then led to the emergence of a “Three Layered Angoff” (TLA) method in which the ratings are Yes = 1, No = 0 and Maybe = 0.5 (Yudkowsky, Downing, & Popescu, 2008; Jalili et al., 2011). The introduction of the “Maybe” category shows plainly that the focus has shifted from strict criterion referencing to some kind of norm referencing, the norm in this case being the examiner-conjured virtual image of a „minimally acceptable‟ examinee. “Maybe” is not a category to which examiners should be in the habit of consigning significant chunks of their curriculum or batches of their questions for criterion referencing, but it is certainly a category that would be heavily populated by examiners attempting to predict the performance of students who cannot be judged with confidence as being of either pass-grade or fail-grade quality. The intended criterion-referenced goal has been obscured by the attention given to the probability that an ill-defined subset of candidates could or would answer each item on a test successfully. Thus, it emerges that the AFM approach cannot be regarded as a „slight variation‟ of the ATM approach, as presented by Angoff (1971), unless the ATM approach is also interpreted as a norm-referenced method, whereupon the ATM approach emerges as a particular extreme case of the more general AFM approach. Somewhere between the extreme binary approach of ATM and the widely-used continuum approach of AFM lies the more recent „variation‟, the Yes/No/Maybe approach (Yudkowsky et al., 2008; Jalili et al., 2011). But, whichever of these three approaches has been used, it would appear that the original goal of achieving an absolute criterion-referenced standard has been obscured by the aforementioned subtle change in language to which we now turn. 3.1 Softening the language of ATM and AFM While a central feature of Angoff‟s methods of standard setting is the definition of “minimally acceptable” persons, contemporary literature speaks, instead, of “borderline” persons. The difference between these labels reflects a shift from a sterner view of what is “minimally acceptable” to a more nebulous concept of what characterises a “borderline” student. This shift has blurred the conceptual ©2014 The author and IJLTER.ORG. All rights reserved.

distinction between “minimally acceptable” (more attuned to criterion referencing) and “borderline” (more attuned to norm referencing). As discussed earlier, it is possible to put a criterion-referenced construction on the usage “minimally acceptable person” by defining such a person in terms of what knowledge, understanding and skills are required. By contrast, it does not seem possible to put such a construction on the concept of a „borderline person‟. On the contrary, the concept would appear from the outset to be norm referenced. Thus, the modern linguistic trend of substituting „borderline‟ for “minimally acceptable” has facilitated and completed the confounding of norm referencing and criterion referencing in the standard-setting process.

4. Angoff’s Second Paragraph 4.1 The Quest for Consensus Angoff‟s (1971) second paragraph, reproduced above, describes how a consensus about the standards might be achieved among several independent “judges”. This raises a number of procedural possibilities and issues according to the nature of the judging panel. Let us consider two extreme situations: A: a panel comprising judges having equal expertise in relation to all n test items; B: a panel comprising judges having expertise limited to different subsets of the n test items, such expertise showing overlap between individual judges and ranging from total overlap to zero overlap. In most medical schools running an integrated curriculum, it would be reasonable to expect that any given panel of judges will lie somewhere between these two extremes and, in practically all cases, much closer to the latter extreme. A. Consensus among totally independent experts In this idealised extreme case, each judge would be equally expert both on the entire content of the test and on the matter of assigning to each item an estimate for the proportion of borderline persons who would answer the item correctly (hereinafter called the “BL value” or, simply, the “BL”). However, real-world variability between judges might be expected to yield some slight variation in the BL estimates, resulting in some items being assigned different BL values by different judges, i.e., the creation of a subset of inconsistently estimated items m. This would indicate a need to achieve a consensus by sacrificing post hoc the absolute independence of the judges by averaging their estimates for each of the m items for which their estimates differed. B. Consensus among inter-dependent partial experts This situation is fraught with difficulty because the partial expertise is rarely manifest as complete expertise on a subset of n and zero expertise on the remainder. Rather, there will be a gradient of expertise for each judge, ranging from complete to zero among the n items. This difficulty could be overcome if each judge self-disqualified on each item for which less than complete expertise was possessed. However, in practice, consensus in these cases can only be reached among non-independent judges by having the less expert judges yield ©2014 The author and IJLTER.ORG. All rights reserved.

to the opinions expressed by the more expert judges on each item. While the intrusion of human nature will be an inescapable complication of this process, it is perhaps to be preferred over an alternative procedure in which each item would be assigned a BL according to an average of independently derived expert and inexpert inputs. 4.2 What is purported to happen Nowadays, much emphasis is placed on training members of standard-setting panels in the art of grasping the concept of a „borderline‟ student. For example, a typical description of the Angoff standard-setting process by Norcini (2003, p.465), under the heading Angoff’s method, reads as follows: Judges are asked to first define the characteristics of a borderline group of examinees (a group with a 50% chance of passing). They then consider the difficulty and importance of the first item on the test. Each judge estimates what percentage of the hypothetical borderline examinees will respond correctly to the item. This judgement is often informed by data on the performance of the examinees. The judges discuss their estimates and are free to change them, and then proceed in the same manner through the remainder of the items on the test. The judges‟ estimates are averaged for each item and the cutpoint is set at the sum of these averages.

Despite the fact that this description is not without its problems, both logical and logistical, it is a fair description of what participants in contemporary standardsetting sessions in medical education imagine that they are doing. The problems are sufficiently important to warrant detailed dissection of this description, sentence by sentence. Judges are asked to first define the characteristics of a borderline group of examinees (a group with a 50% chance of passing).

This process of „definition‟ is quite illusory. The word „borderline‟ embodies the uncertainty attaching to persons who cannot be characterised as being clearly acceptable (deserving to pass) or clearly unacceptable (deserving to fail). Given this uncertainty, on what basis can such a group be said to have a 50% chance of passing? Only if the uncertainty of the examiners is symmetrical, i.e., if every conceivable type of person who is neither „clearly acceptable‟ nor „clearly unacceptable‟, is equally likely to have been wrongly excluded from either of these two categories. Nonetheless, regardless of these issues, the focus is clearly on a subset of the cohort of examinees (whether actual, anticipated or imaginary), i.e., a normreferenced focus, not a criterion-referenced focus. Moreover, given that judges are asked to conceive of such students as a hypothetical abstraction, based on their prior teaching and assessment experience, they can do nothing other than conjure up a norm-referenced concept (the „borderline‟ person) and apply to it their own subjective guesswork. Given the difficulty in establishing a criterionreferenced absolute standard derived from such norm-referenced guesswork, it is not surprising that a desire has arisen among examiners to seek the protection ©2014 The author and IJLTER.ORG. All rights reserved.

and reassurance of group consensus among as large a collection of judges as can be mustered to define a standard for any given test. They then consider the difficulty and importance of the first item on the test.

This sentence directly and unnecessarily confounds the concepts of „difficulty‟ and „importance‟. In fact, it is possible that these two concepts, in relation to individual test items, might be either essentially identical or totally unrelated. For example, the difficulty of an item might be judged to be a direct reflection of its importance, i.e., it is important for the item to be difficult as a property of its defining a minimally acceptable criterion of coping with difficulty. On the other hand, an item might be extremely important yet trivially easy for a correctly trained candidate, so that importance and difficulty are totally unrelated. Moreover, items that are unlikely to be answered successfully by any but the most exceptional candidates may possibly be very difficult yet essentially unimportant. Each judge estimates what percentage of the hypothetical borderline examinees will respond correctly to the item.

This is clearly a norm-referenced judgment, with no necessary link to any sense of importance of the item (criterion referencing). As already noted, the formation of such estimates is reported to be difficult (Lorge & Kruglov, 1953; Bejar, 1983; Impara & Plake, 1998; Norcini, 2003) although claims have been made that judges benefit from an iterative process whereby they can „learn‟ from the estimates of their fellow judges formed during previous standardisation sessions dealing with the same assessment test (Cizek & Bunch, 2007, p.84). However, except where special funding for educational research projects is available, iterative standard setting is beyond the resources and the practical exigencies of most medical schools. Moreover, when the test is a major examination of an integrated curriculum (reflecting current trends in medical education), it is not always possible to convene judging panels in which all specialities within the curriculum are adequately represented, let alone having multiple experts on each area capable of working out a consensus on the respective estimates. At this point it is worth commending to the reader‟s attention the salutary study of AFM by Impara and Plake (1998) in which they “tested the ability of 26 classroom teachers to estimate item performance for two groups of their students on a locally developed district-wide science test.” They found that “teachers‟ estimates of the average proportion correct” for „borderline‟ students “were for the most part quite inaccurate”, with only 23% of 1300 estimates focussed on these students being “accurate (defined as within .10 of actual item performance.” Their concluding paragraph is worth quoting in full (Impara & Plake, 1998, p.80):

The most salient conclusion we can draw from this study is that the use of a judgmental standard setting procedure that requires judges to estimate proportion-correct values, such as that proposed by Angoff (1971), may be questionable. The teachers in this study performed the estimation task in such a way that if their performance estimates were used to set a standard, the validity of the standard used to identify borderline students would be in question. If teachers who have been with their students for most of the school year are unable to estimate student performance accurately using a test that is familiar to them, how can we expect other judges who may be less familiar with examinees to estimate item performance on a test those judges may never have seen before? (emphasis added)

Returning to our dissection of Norcini‟s (2003, p.465) description, we find: This judgement is often informed by data on the performance of the examinees.

In our experience of standard setting, the practice of setting borderlines by reference to item statistics pertaining to past examinee cohorts ranges between two extremes: A. During a standard-setting session, this practice is usually frowned upon as being norm referenced, the purpose of the standard setting being to establish a criterion-referenced pass mark. It is difficult at such sessions to establish acceptance of the fact that the participants are, in fact, being asked to predict a number that should be highly correlated with, and reasonably close to, the actual statistically derived proportion of LOW4 students (as determined by post-test item analysis) that will be discovered to answer the item correctly. Attempts to establish the truth of this identity before the test is administered will often be contradicted by remarks such as, “No, that‟s norm referencing. We are criterion referencing.” Such remarks are untrue, but they frequently win the day, presumably because the abovementioned confounding of difficulty and importance is fairly pervasive. B. During results review, this practice might be encouraged if, as could be the case, reversion to norm referencing for a difficult item would lower the pass mark. For the record, this procedure has not been used at Gippsland Medical School. While these extremes are mutually incompatible, they illustrate ways in which the standard-setting process can become compromised in practice.

The „LOW‟ subset of a cohort of examinees is the bottom 27% (approximately) as measured over the whole examination, including all multiple-choice questions and extended matching questions, but excluding short-answer questions (SAQs). The „LOW‟ success rate for a given question is the proportion of the „LOW‟ subset who answer the question correctly. 4

The judges discuss their estimates and are free to change them, and then proceed in the same manner through the remainder of the items on the test.

This part of the process has already been considered above under section 4.1 B. The judges‟ estimates are averaged for each item and the cutpoint is set at the sum of these averages.

As already noted, the cutpoint may be subject to alteration when the results of the examination are reviewed. 4.2.1 What actually happens The description here is suggested to be typical of what happens routinely in medical schools that apply the Angoff Footnote Method (AFM) of standard setting. It may not be typical of what happens in standard-setting sessions that form part of specially resourced educational research projects. A standard-setting session is usually held several days after a draft of the examination paper has been pre-circulated among all academics involved in teaching and/or examining the unit. For most items, each BL has already been supplied by the respective item‟s author. In many cases, prior statistics deriving from item analysis are also attached to individual items that have been used in previous examinations. Academics are encouraged to review the examination paper thoroughly, focussing particularly on their own areas of expertise, advising of any errors or recommendations for improvement, reviewing the BLs supplied and, where not supplied, suggesting BL scores. Academics who are unable to attend the standard-setting session are encouraged to submit their comments in writing so that they may be considered at the meeting. At the meeting, attention is focussed only on those items that have been flagged for attention in the period prior to the meeting. It is noteworthy that concord between the BLs and their respective LOW success rates (where item statistics are available from prior examinations) is frequently absent. Where the disparity is severe, there may be a consensus at the meeting to alter the BL, but we have rarely seen any BL set lower than 0.3 prior to the examination, despite many LOW success rates being 0.2 or less (see Figure 1, lower right panel). This disparity between BL predictions and actual LOW success rates finds resonance in the following quote from Norcini (2003, pp.465-466): Angoff‟s method is relatively easy to use, there is a sizeable body of research to support it, and it is frequently applied in licensing and certifying settings. It also has the virtue of focusing attention on each of the questions and thus can be very helpful from a test development perspective. This method produces absolute standards, so it is best suited to tests that seek to establish competence. However, judges sometimes feel as though there is no firm basis for their estimates and application of the method can be tiresome for longer tests. ©2014 The author and IJLTER.ORG. All rights reserved.

While the standard-setting process (regardless of the method used) is certainly most helpful from a test development perspective, it cannot be supported that AFM produces absolute standards. Indeed, there is no firm basis for the estimates, as will now be discussed.

5. Angoff’s Footnote Method in Action in a Small Medical School Let us now turn to some assessment data obtained in three successive years from examinations set for first-year graduate-entry students at Monash University‟s Gippsland Medical School in 2010-2012. Prior to each examination, a standardsetting session was held in which a panel of interdependent partial experts was asked to reach a consensus, item by item, on estimating the proportion of borderline candidates who would answer each item correctly for multiple-choice questions (MCQs) and extended-matching questions (EMQs).5 5.1 Relation of BL estimates to different subsets of examinees Figure 1 shows data plots relating the BL estimate associated with each MCQ or EMQ and the respective performance (P) success rate for the entire student cohort (PALL) and for the three subsets of candidates identified by statistical analysis of the results (PHIGH, PMID and PLOW; see Footnote 4). All the data represented in Figure 1 were obtained from the 2010 cohort of 76 examinees answering 83 questions in the mid-semester 1 examination. Thus, although each data plot contains 83 data points, far fewer than this number are visible owing to superposition of many data points. Figure 1 also shows the results of analysing the data using linear regression of P values on respective BL estimates. While there is no reason to expect uncomplicated linear correlations, it is reasonable to expect that the values of both the slope and R2 of the linear regression would be greatest for the LOW subset and least for the HIGH subset. It is also to be expected, as found, that the ordinate intercept of the regression should be close to zero for the LOW subset and significantly greater than zero for the HIGH subset. Consistent with these expectations are the intermediate values of slope, R2 and ordinate intercept observed for the MID subset and for the whole cohort (ALL). Despite the fulfilment of these expectations, it is clear that the BL estimates arrived at through a consensual implementation of AFM are almost randomly and widely inaccurate; the performance success rates span much more than half the available range between 0 and 1 for almost all the BL estimate levels provided. The overall impression gleaned from these data is that academics‟ BL estimates are extremely poor predictors of examinee performance, and this impression is at its strongest in relation to the performance of the LOW subset.

They were also asked to estimate the average mark likely to be obtained by a borderline candidate on each item for Short Answer Questions (SAQs), but this category of estimates is not included in the present analysis. 5

This impression of gross inaccuracy is maintained as we refine the focus to the „Borderline‟6 subset as will be shown in the next subsection.

Figure 1: Data plots of performance rates, P, vs BL estimates for students answering 83 questions in a mid-semester 1 examination in 2010. The plots are for the entire cohort of 76 students (ALL, upper left), for the top 20 students (HIGH, lower left), for the 35 midrange students (MID, upper right) and the bottom 21 students (LOW, lower right).The linear trend line, linear regression equation and R2 value are included on each respective plot.

5.2 Accuracy of BL estimates for the ‘Borderline’ subset of examinees This 2010 examination was given again in 2011 and 2012 with essentially the same group of academics producing very similar styles of questions. The data plots relating the BL estimate associated with each MCQ or EMQ and the respective performance (P) success rate for the „Borderline‟ subsets are shown in Figure 2 for eleven examinees in 2010 (upper left panel), three examinees in 2011 (upper right panel) and for nine examinees in 2012 (lower panel). For these particular data plots, it is highly appropriate to perform linear regression analysis on the relations between BL estimate and performance because the estimate is explicitly purported to predict the actual performance of the identified „Borderline‟ students. The persistently poor correlations already observed in the data plots of Figure 1 are also observed in the data shown in Figure 2, indicating that the academics‟ predictive ability did not improve when only the „Borderline‟ data were included, and nor did it improve with experience. In all three years it was often „Borderline‟ students are defined as examinees whose results fall within a range from one SEM above to two SEMs below the overall BL determined for the test, where the SEM is the standard error of measurement of examinees‟ scores. 6

found that „Borderline‟ performance values spanning the entire possible range between 0 and 1 could be returned for groups of items assigned any given BL value by the examiners. Moreover, the highly respectable slope and ordinate intercept seen from the 2010 „Borderline‟ subset were not seen in the two following years.

Figure 2: Data plots of performance rates, P vs BL estimates for ‘Borderline’ students answering questions in a mid-semester 1 examination in 2010 (83 questions, 11 examinees, upper left), in 2011 (82 questions, 3 examinees, upper right) and in 2012 (95 questions, 9 examinees, lower). The linear trend line, linear regression equation and R2 value are included on each respective plot.

Figure 3 shows analysis of data gleaned from the same cohort as presented for the mid-semester 1 examination in 2012 (Figure 2 lower), showing analyses of data from the end-semester 1 (left) and mid-semester 2 (right) examinations from the same year. Interestingly, this cohort provided no data for such analysis in the 2012 end-semester 2 examination because all the students passed. That is, for the 2012 cohort, the numbers of „Borderline‟ students identified by using Angoff‟s Footnote Method for the four successive mid-semester and endsemester examinations were 9, 9, 15 and 0, respectively. Any discussion or further exploration of this interesting finding would stray too far from the focus of the present critique and so will not be pursued here. It might be objected that the small numbers of identified „Borderline‟ examinees in the examination data reported here (ranging from 3 to 15 per examination, with one instance of zero) militate against drawing firm conclusions. However, the conclusions pertain to the accuracy of predictions of BL values for large numbers of questions, ranging from 75 to 95 per examination. Almost by definition, one hopes, the numbers of „Borderline‟ candidates identified among cohorts of graduate-entry medical students should be small. The critique concerns the accuracy of the large number of predictions about individual ©2014 The author and IJLTER.ORG. All rights reserved.

questions that are made in relation to the actual performance on those questions by the small numbers of identified „Borderline‟ candidates.

Figure 3: Data plots of performance rates, P vs BL estimates for ‘Borderline’ students answering questions in an end-semester 1 examination (87 questions, 9 examinees, left) and a mid-semester 2 examination (75 questions, 15 examinees, right) in 2012. The linear trend line, linear regression equation and R 2 value are included on each respective plot.

5.3 A want of feasibility and credibility for Angoff’s Footnote Method While this direct comparison of BL values with „Borderline‟ performance values has been held to be a way of providing “useful checks on the passing score that is being chosen” (e.g., Kane, 1994, p.447), it seems likely that the poor predictive powers of academics makes the use of AFM an exercise in futility. The only comfort that can be taken from the data shown in Figures 2 and 3 is that the large errors of BL estimation are so randomly distributed that there may be no systematic error in the passing scores actually obtained by using AFM in this way. Thus the attempt to make accurate predictions, though futile and unsuccessful, may actually do little harm. This record of poor prediction of BL values by academics involved in delivering an integrated medical curriculum should come as no surprise to anyone who has absorbed the results and conclusions of the study by Impara and Plake (1998) cited earlier. The least that can be suggested is that these BL predictions cannot be held up as exemplars of the goal of defining criterion-referenced absolute standards that are reliable and valid. On the contrary, the BL estimates of „Borderline‟ examinees‟ performances shown in Figures 2 and 3 are visibly unreliable and invalid. In fact, the data shown in Figures 1 to 3 would seem, at worst, to vindicate Glass‟s (1978, p.259) forlorn conclusion that “setting performance standards on tests and exercises by known methods is a waste of time or worse” and, at best, to resonate with Impara and Plake‟s (1998) finding that even schoolteachers who are deeply familiar with a long-established test and their successive student cohorts are unable to predict the performance of „borderline‟ students in such a way as to produce a reliable criterion-based standard. As already noted, this study is not the first to question the feasibility and credibility of the Angoff process; nonetheless there seems to have been a wellestablished historical tendency for educators to proceed blithely ahead with their ©2014 The author and IJLTER.ORG. All rights reserved.

theories and methods without due acknowledgement of precedence or criticism.7 The data plotted in Figures 1 to 3 cast grave doubt on Norcini‟s (2003, p.466) claim that the Angoff (Footnote) Method produces absolute standards. The standards set for individual items do not seem to correlate in any reassuring way with the degree of difficulty of the items, as encountered by either the LOW subset of examinees or the TOTAL cohort. As for correlation with the importance of the items (cf. Norcini, 2003, p.465 and earlier discussion in Section 4.2), that would seem to be a matter totally beyond analysis. The data reported here are consistent neither with the findings of Shepard (1994), who found that judges tend to overestimate low performers and underestimate high performers, nor with the opposite finding reported by Impara and Plake (1998, p.75) who found that teachers systematically underestimated the performance of „borderline‟ students. It seems that examiners are as likely to underestimate as to overestimate performance success rates over a wide range of BL estimate values. These disparate findings would appear to provide further evidence that absolute standard setting by such prediction is unattainable.

6. A Possible Criterion-Referenced Implementation of ATM Let us now consider how ATM might be interpreted and implemented using the more criterion-referenced construction of the concept of the “minimally acceptable person” suggested in Section 2.1 A. In such an approach there must be a strictly criterion-referenced focus on the content of the assessment items and not a norm-referenced focus on the performance probabilities of examinees. Let us consider a test comprising 100 items, each item carrying 1 mark, with no possibility of scoring fractional marks on any of the items. That is, for each item, a correct answer scores 1 while an incorrect answer scores zero. The method is to “go through the test item by item and decide whether” a “minimally acceptable person ... could answer correctly each item under consideration.”

7Zieky

(1995, p.10) records a disturbing fact about the landmark publications of Angoff (1971), Hills (1971) and Glaser and Nitko (1971), all appearing in the same 2nd Edition of Educational Measurement. They all failed to recognise the pioneering work of Nedelsky (1954) published in Educational and Psychological Measurement. Zieky (1995, p.6) notes that “concepts described by Nedelsky are found in more recent descriptions of methods of setting standards”, and he finds that Nedelsky‟s concept of a „borderline‟ student “corresponds to Angoff‟s (1971) „minimally acceptable person‟, to Ebel‟s (1972) „minimally qualified (barely passing) applicant,‟ and to the members of the „borderline group‟ described by Zieky and Livingston (1977).” Despite the importance of Nedelsky‟s (1954) work, Zieky (1995, p.10) notes with interest that “neither Angoff nor Hills nor Glaser and Nitko referenced Nedelsky‟s article „Absolute Grading Standards for Objective Tests.‟ The article was clearly relevant to the problems addressed in their chapters and had been printed in a major journal about 17 years earlier.” ©2014 The author and IJLTER.ORG. All rights reserved.

6.1 Unambiguous criterion referencing: focus on the test By this more criterion-referenced definition, a “minimally acceptable person” must know, understand or accomplish certain well-defined facts, concepts or procedures, respectively. In other words, a “minimally acceptable person” must demonstrate mastery of certain well-defined criteria. Once the criteria have been defined according to the objectives of the curriculum, the determination of a criterion-referenced pass mark (or honours mark) becomes straightforward and unambiguous. By thus substituting must for „could‟ in ATM, the implication is that, of all the n items in a test, a given item, i, encapsulates required material if no person failing to show mastery of this material (i.e., failing to score 1 for item i) should be allowed to pass the test. The sum of marks attaching to all such required items, p, is therefore the pass mark defining the mastery requirement of the “minimally acceptable person”. To follow Angoff‟s suggestion that a similar procedure could be followed for the hypothetical lowest honours person, all that is required is that, in addition to the p „must know‟ items already identified as required material for the “minimally acceptable person”, a further h „should know‟ items be identified as required material for the lowest honours person. It then follows that, for a test containing a total of n items, there will be a number of items, x = n – p – h, that will be answered correctly only by exceptional candidates („nice to know‟). This proposed distribution of the n test items is summarised in Table 1. These considerations of minimally acceptable or exceptional performance can be applied equally to test items whether they encapsulate required knowledge, required understanding, required skills, or some combination of these attributes. It is important to note, therefore, that this method of standard setting is focussed entirely upon the test items insofar as they are identified as encapsulating required material at whatever level; the focus is not upon the examinee. Table 1 Item type

Description

„Must know/understand/accomplish‟ – all items for which mastery is required by a minimally acceptable person.

„Should know/understand/accomplish‟ – all further items for which mastery is required by the lowest honours person.

„Nice to know/understand/accomplish‟ – these items are unlikely to be answered correctly by any but exceptional persons.

n =p+h+x

Total number of items (marks)

To construct such a test in which the pass mark is 50% and the honours mark is 85%, it is sufficient to ensure that p items carry 50% of the marks and h items carry 35% of the marks. In our example test, comprising 100 equally weighted items (1 mark per item), such a standard would be set by setting p = 50 and h = 35. But note that this implies that 50 p-category items, 35 h-category items and ©2014 The author and IJLTER.ORG. All rights reserved.

15 x-category items have been pre-identified independently and brought together to produce such a combination of 100 items for the test so that such a standard obtains for the test. Other combinations of test items could be put together prior to standardisation, following which the standard setting would determine different values of p, h and x. These could then be assigned different marking weights so as to produce, if so desired, cutpoints at the 50% and 85% boundaries. The above model is simply offered as a suggestion as to how a more consciously criterion-referenced approach to standard setting might be developed. It is not seen as a pure model; such a thing would seem to be unattainable. For example, it is possible for a candidate to score p marks while answering some of the pcategory questions incorrectly, the balance of the marks coming from correct answers to questions in other categories. When one allows for the intrusion of guesswork into candidatesâ&#x20AC;&#x; answers, including the unknowable proportions of informed and uninformed guesswork, the criterion-referenced goal of an absolute standard becomes even more illusory. Rather, this method of standard setting is offered as an approach to the development of tests that are explicitly tied to curriculum objectives and that allow for the capture of information about ranking of candidates in relation to the attainment of those objectives. 6.2 Teach to the objectives and test the objectives Let us now turn to a simple observation, drawn from experience, that highlights the difficulty in believing that AFM can produce an absolute standard of any kind. At Gippsland Medical School in the years 2008, 2009 and 2010, the electrophysiological material on propagation of the nerve action potential was delivered as a didactic lecture to first-year graduate-entry medical students. Among other things, the lecture dealt with the passive electrical properties (resistances, capacitances) of long cylindrical nerve axons and the effect of myelination on those properties. This topic is clinically important because of the disease state of multiple sclerosis in which nerve axons lose their myelin sheaths, leading to motor disability and death. The lecture was always supported by comprehensive, detailed lecture notes made available online in two formats: as a linear text document and as an interactive 3-layered hypertext application. However, it was consistently found that students had difficulty in assimilating and applying the concepts underlying the effects of myelination or demyelination on electrical signalling in nerves. A typical multiple-choice question was set in 2010 as follows (correct answer in bold type): What is the effect of myelination on a nerve fibre? A. It increases the membrane resistance while reducing the membrane capacitance B. It reduces the membrane resistance while increasing the membrane capacitance C. It reduces both the membrane resistance and the membrane capacitance D. It increases both the membrane resistance and the membrane capacitance ÂŠ2014 The author and IJLTER.ORG. All rights reserved.

E. It has no effect on either the membrane resistance or the membrane capacitance, provided the influence of the electrogenic pump is ignored

The statistical item analysis for this question in 2010 was as follows: ITEM 51: DIF=0.513, RPB= 0.478, CRPB= 0.427 (95% CON= 0.223, 0.595) RBIS= 0.599, CRBIS= 0.535, IRI=0.239 GROUP N INV NF OMIT A* B C D TOTAL 76 0 0 0 0.51 0.36 0.08 0.05 HIGH 20 0 0.75 0.10 0.10 0.05 MID 35 0 0.60 0.37 0.00 0.03 LOW 21 0 0.14 0.57 0.19 0.10 TEST SCORE MEAN %: 76 65 66 65 DISCRIMINATING POWER 0.61 -0.47 -0.09 -0.05 STANDARD ERROR OF D.P. 0.16 0.15 0.11 0.08

While it was regarded as disappointing that only 51% of the class answered the question correctly, this was consistent with observations in the preceding two years where students found this topic quite difficult. Note, however, that the question showed a high discriminating power of 0.61, with 75% of the HIGH group, 60% of the MID group and only 14% of the LOW group answering correctly. The underlined part of the analysis shows the performance of the LOW group of students (the 21 lowest performers on the overall examination out of a total cohort of 76 students). In 2011 it was decided to replace the respective didactic lecture with a compulsory Tutorial for which students had to prepare answers to six set tasks. The six tasks were assigned for presentation among the cohort‟s six ProblemBased Learning (PBL) groups for cooperative preparation of a presentation to be posted online a few days before the tutorial. All students were required to prepare for the tutorial by studying all six of the online presentations. At the tutorial, students were selected at random to present their findings to the tutorial group (Presenters) or discuss the findings of other students (Discussants). In short, active learning was enforced in 2011, unlike in previous years where a didactic lecture was used. The same linear text document and interactive 3-layered hypertext application were used to support the tutorial as for the previous year‟s lecture. The relevant tutorial task was given as follows: The Effect of Myelination on Passive Electrical Properties of Nerve Axons a. Explain how myelin is formed in the central and peripheral nervous systems. b. What are the principal features of a node of Ranvier? c. How does myelination influence:  membrane resistance?  membrane capacitance? d. Therefore, how does myelination affect the conduction velocity of a nerve fibre?

When the same MCQ was set in the 2011 examination it was given a BL score of 0.2 using AFM, based on the 2010 item analysis (i.e., LOW = 0.14). In the event, the item analysis in 2011 was as follows: ©2014 The author and IJLTER.ORG. All rights reserved.

ITEM 11: DIF=0.841, RPB= 0.222, CRPB= 0.176 (95% CON= -0.034, 0.372) RBIS= 0.334, CRBIS= 0.266, IRI=0.081 GROUP N INV NF OMIT A* B C D TOTAL 88 0 0 0 0.84 0.09 0.02 0.05 HIGH 25 0 1.00 0.00 0.00 0.00 MID 40 0 0.77 0.10 0.05 0.08 LOW 23 0 0.78 0.17 0.00 0.04 TEST SCORE MEAN %: 70 62 66 67 DISCRIMINATING POWER 0.22 -0.17 0.00 -0.04 STANDARD ERROR OF D.P. 0.09 0.08 0.00 0.04

Thus, for this particular question, there was a very large improvement in performance in 2011 relative to 2010 across the entire cohort, with 84% of the class answering the question correctly. Moreover, the question‟s discriminating power became much lower (0.22), with 100% of the HIGH group, 77% of the MID group and 78% of the LOW group answering correctly. The underlined part of the analysis shows the performance of the LOW group of students (the 23 lowest performers on the overall examination out of a total cohort of 88 students). When the same question was run in the corresponding 2012 examination after delivering the material using the same 2011 active learning model, its performance statistics were as follows: ITEM 12: DIF=0.721, RPB= 0.216, CRPB= 0.161 (95% CON= -0.053, 0.360) RBIS= 0.289, CRBIS= 0.214, IRI=0.097 GROUP N INV NF OMIT A* B C D TOTAL 86 0 0 0 0.72 0.14 0.03 0.09 HIGH 24 0 0.83 0.13 0.00 0.04 MID 38 0 0.74 0.13 0.00 0.13 LOW 24 0 0.58 0.17 0.13 0.08 TEST SCORE MEAN %: 69 66 58 68 DISCRIMINATING POWER 0.25 -0.04 -0.13 -0.04 STANDARD ERROR OF D.P. 0.13 0.10 0.07 0.07

E 0.01 0.00 0.00 0.04 60 -0.04 0.04

This represents a slight drop in performance in 2012 relative to 2011, but still much improved relative to 2010 across the entire cohort, with 72% of the class answering the question correctly. The question‟s discriminating power increased slightly (0.25), with 83% of the HIGH group, 74% of the MID group and 58% of the LOW group answering correctly. The underlined part of the analysis shows the performance of the LOW group of students (the 24 lowest performers on the overall examination out of a total cohort of 86 students). These observations on the performance statistics of a single question show that the BL estimations cannot possibly produce the claimed absolute standard of criterion referencing, and this remains true however the observations are interpreted. They could have been due in part to having superior cohorts of students in 2011 and 2012 relative to previous years, and it must be acknowledged that the level of competition and the cut-off scores associated ©2014 The author and IJLTER.ORG. All rights reserved.

with gaining admission as graduate-entry medical students are increasing year by year. However, it is strongly suggested here that most of the differences are due to the substitution of active learning for what, in the past, had been largely passive learning; this is hardly a surprising result. Whatever the relative contributions of these two causes to these observations might be, the fact remains that the claim (Norcini, 2003, pp.465-466) of setting absolute standards in BL predictions using the Angoff Footnote Method is entirely without foundation. The performance of „borderline‟ students is more dependent on the methods of teaching and learning applied than it is on the intrinsic difficulty of the content. This result certainly accords with the conclusion of Glass (1978, p.239), who wrote: The vagaries of teaching and measurement are so poorly understood that the a priori statement of performance standards is foolhardy.

However, the suggested remedy for these problems is not to seek some unattainable absolute standard, but to apply criterion-referenced (i.e., curriculum-determined) standards of relative importance to test items, ensuring that all items test the objectives, and then teach to the objectives. 6.3 Constructing a criterion-referenced standard for a test The following checklist of requirements is suggested for producing a curriculum-determined standard according to the criterion-referenced interpretation of ATM in which a test comprises a mixture of items in the p, h, and x categories described above in Section 6.1 and Table 1: 

Avoid undue dependence on centralised question banks; there can be no certainty that the questions have been composed in relation to the currently operative learning objectives, or with a view to accommodating the problems of standard setting addressed in this paper. Some such questions may prove useful, but only after they have been subjected to careful scrutiny to decide how they might be assigned among the p, h and x categories.



Before presenting the formal teaching/learning opportunity (lecture, tutorial, practical class and any supporting handouts, online documentation, etc.), identify the respective learning objectives and construct at least one item in each category to address each learning objective as follows: o

Category p – Must know or Must understand or Must accomplish – A person who fails to answer such questions correctly is not yet ready to proceed to the next year level. These questions cover essential information, understanding and skills. Such material is not necessarily easy, although in many cases it may be.

Category h – Should know or Should understand or Should accomplish – A person who fails to answer such questions correctly is unworthy of attaining a Credit, but such failure is not a significant impediment to progression to the next year level. These questions may be more challenging than those in the preceding category, but should not venture beyond material that has been presented in teaching/learning sessions or documented in online course materials.

Category x – Nice to know or Nice to understand or Nice to accomplish – A person who answers one quarter of such questions correctly is worthy of attaining a Distinction; a person who answers half of such questions correctly is worthy of attaining a High Distinction. These questions should be relevant to the material covered in the respective session but not necessarily covered in detail or even directly mentioned at all. They could explore material that a good student might be expected to pick up through further self-directed study.



When presenting the formal teaching/learning opportunity and preparing any supporting online documentation, it is important to teach to the objectives with respect to the p-category and h-category items. This underlines the importance of identifying the learning objectives and preparing the targeted test items before preparing the associated teaching and learning resources.



As with conventional application of the various Angovian methods, examiners should have the opportunity to submit their questions to colleagues for feedback on their individual standard-setting judgments (in this case, the distribution into p, h and x categories). In particular, there may be need for review of the assignment of questions among the three categories both prior to, and following, the administration of the test. This would provide essential information regarding the accuracy and feasibility of such attempts to set relative standards unambiguously.



When the test results are known, review the allocation of students among the various grades and, if the results of applying the implicit ATMderived criterion-referenced standards are unacceptable, then let the allocation of grades be influenced by norm-referenced considerations. If too much norm-referenced adjustment has to be made to the ATMderived standard, then use that information to guide an inquiry into the construction and allocation of questions among the three categories, the teaching and learning resources provided, and the communication of information to students.

‘Borderline’ Students in Graduate-Entry Medical Schools

Competition to gain admission to graduate-entry medical courses is very strong in Australia. Those who succeed do so by obtaining increasingly high marks in ©2014 The author and IJLTER.ORG. All rights reserved.

the Graduate Australian Medical School Admissions Test (GAMSAT) examination. Given that such students have already demonstrated academic success at tertiary level, that they remain sufficiently motivated to study medicine and that, at Gippsland Medical School and many other schools, they are further assessed at interview, there is good reason to suppose that every student gaining admission to graduate-entry medicine should be able to proceed to graduation in the minimum time. That is, we should not expect to find more than a handful of „borderline‟ students in each cohort and we should not be surprised occasionally to find none at all. On the contrary, such expectations flow naturally from the confidence we place in the selection process for graduate-entry medicine. The reasonable default expectation is that all graduate-entry medical students should pass. It follows that the existence of „borderline‟ students in graduate-entry medical cohorts simply demonstrates that the selection process is imperfect and unable to guarantee an absolute standard. Norm referencing will find such students out, as it always has. It would seem unrealistic and unproductive to search for an absolute objective standard such as would relieve examiners of the need to take responsibility for exercising subjective judgments, when required, to deal with „borderline‟ students. As Kane (1994, p.427) observed: We create the standard; there is no gold standard for us to find, and the choices we make about where to set the standard are matters of judgment.

Conclusion

As this discussion has been more critical than supportive of existing applications of Angoff‟s methods and their derivatives, it is important to clarify that this was never intended to be a direct criticism of Angoff (1971)8. Rather, it has been a discussion of the possibility that succeeding generations of educationists may have been too uncritical in their application of a method that was originally offered quite casually as a brief incidental insertion within a very large chapter devoted to other assessment issues. It is suggested that: 

Generations of educators who have sought to implement Angovian methods for defining criterion-referenced standards have, whether consciously or not, been guided by a belief that an objective absolute standard is achievable.



This belief has been undermined from the outset by:

8Angoff

is even reported to have claimed in the early 1980s that the true originator of the method attributed to him was the American mathematician Ledyard Tucker, as recorded by Jaeger (1989, p.493) and Zieky (1995, p. 9 footnote 3). ©2014 The author and IJLTER.ORG. All rights reserved.

blurring the focus on criterion referencing with an insufficiently precise definition of a “minimally acceptable person”, and

replacement of the already ill-defined concept of a “minimally acceptable person” with the norm-referenced concept of a „borderline‟ person.



Rather than Angoff‟s Footnote Method (AFM) being a “slight variation” of Angoff‟s Text Method (ATM), the dominant interpretation and implementation of Angovian methods of the past forty years reveal ATM to be an extreme, binary example of the more generally used probability continuum of AFM, all such applications being norm referenced by focussing on the examinee rather than on the curriculum.



Unless standard setting takes place in the context of a generously funded educational research project, the search for consensus among panels of independent experts is not feasible in the routine management of assessment of an integrated curriculum in a graduate-entry medical school.



Academics‟ predictions of „borderline‟ student performance are manifestly inaccurate in a small rural medical school and are unlikely anywhere to be more accurate than the documented inaccuracy reported among secondary school science educators (Impara and Plake, 1998). However, the prediction errors, though wide-ranging, are possibly sufficiently random to generate no serious systematic error in the performance estimates for „borderline‟ students averaged over a test comprising many assessment items. Thus, the inaccuracy of the predictions may not do significant harm, even though the expenditure of resources in generating them may not be justified.



A more rigorously prescriptive interpretation of ATM raises the possibility of applying criterion-referenced (i.e., curriculum-determined) standards directly to test items, with items distributed into „must know‟, „should know‟ and „nice to know‟ categories.



While standard setting should be done as objectively as possible, the intrusion of imperfection in student selection procedures and assessment procedures will always require examiners to take responsibility for exercising subjective judgments.

Acknowledgements Thanks are due to Professor William Hart and Associate Professor Elmer Villanueva for their helpful comments on earlier drafts of this paper.

References Amin, Z., Chong, Y.S., & Khoo H.E. (2006). Practical Guide to Medical Student Assessment. London, England: World Scientific Publishing Co. Angoff, W. H. (1971). Scales, Norms, and Equivalent Scores. In R. L. Thorndike (Ed.), Educational Measurement, 2nd edn, (pp. 508-600). Washington, DC: American Council on Education. Bejar, I. I. (1983). Subject matter experts‟ assessment of item statistics. Applied Psychological Measurement, 7(3), 303-310. Cizek, G .J., & Bunch, M.B. (2007). Standard Setting: A Guide to Establishing and Evaluating Performance Standards for Tests. Thousand Oaks, CA: Sage Publications. Downing, S. M., Tekian, A., & Yudkowsky, R. (2006). Procedures for establishing defensible absolute passing scores on performance examinations in health professions education. Teaching and Learning in Medicine, 18(1), 50-57. Ebel, R. L. (1972). Essentials of educational measurement, 2nd edn. Englewood Cliffs, NJ: Prentice-Hall. Glaser, R., & Nitko, A. J. (1971). Measurement in learning and instruction. In R. L. Thorndike (Ed.), Educational Measurement, 2nd edn, (pp. 625-670). Washington, DC: American Council on Education. Glass, G. V. (1978). Standards and criteria. Journal of Educational Measurement, 15(4), 237261. Hills, J. R. (1971). Use of measurement in selection and placement. In R. L. Thorndike (Ed.), Educational Measurement, 2nd edn, (pp. 680-732). Washington, DC: American Council on Education. Impara, J. C., & Plake, B. S. (1997). Standard Setting: An Alternative Approach. Journal of Educational Measurement, 34(4), 353-366. Impara, J. C. & Plake, B. S. (1998). Teachers‟ Ability to Estimate Item Difficulty: A Test of the Assumptions in the Angoff Standard Setting Method. Journal of Educational Measurement, 35(1), 69-81. Jaeger, R. M. (1989). Certification of student competence. In R. L. Linn (Ed.), Educational measurement, 3rdedn, (pp. 485-514). New York, NY: American Council on Education/Macmillan. Jalili, M., Hejri, S. M., & Norcini, J. J. (2011). Comparison of two methods of standard setting: the performance of the three-level Angoff method. Medical Education, 45(12), 1199-1208. Kane, M. T. (1994). Validating the performance standards associated with passing scores. Review of Educational Research, 64(3), 425-461. Lorge, I., & Kruglov, L.K. (1953). The improvement of the estimates of test difficulty. Educational and Psychological Measurement, 13(1), 34-46. Lypson, M. L., Downing, S. M., Gruppen, L. D., & Yudkowsky, R. (2013). Applying the Bookmark method to medical education: Standard setting for an aseptic technique station. Medical Teacher, 35, 581-585. Mills, C. N., & Melican, G. J. (1988). Estimating and adjusting cut off scores: Features of selected methods. Applied Measurement in Education, 1(3), 261-275. Nedelsky, L. (1954). Absolute grading standards for objective tests. Educational and Psychological Measurement, 14(1), 3-19. Norcini, J. J. (2003). Setting standards on educational tests. Medical Education, 37(5), 464469. Searle, J. (2000). Defining competency – the role of standard setting. Medical Education, 34(5), 363-366. Shepard, L. A. (1994, October). Implications for standard setting of the NAE evaluation of NAEP achievement levels. Paper presented at the Joint Conference on Standard Setting for Large Scale Assessments, National Assessment Governing Board. Washington, DC: National Center for Educational Statistics. ©2014 The author and IJLTER.ORG. All rights reserved.

Williams, P. (2008). Assessing context-based learning: not only rigorous but also relevant. Assessment & Evaluation in Higher Education, 33(4), 395-408. Yudkowsky, R., Downing, S. M., & Popescu, M. (2008). Setting standards for performance tests: a pilot study of a three-level Angoff method. Academic Medicine, 83(10), S13-S16. Zieky, M. J. (1995). A historical perspective on setting standards. In Proceedings of the Joint Conference on Standard Setting for Large Scale Assessments (pp. 1-38). Washington, DC: National Assessment Governing Board and National Center for Educational Statistics. Zieky, M. J. and Livingston, S. A. (1977). Manual for setting standards on the Basic Skills Assessment tests. Princeton, NJ: Educational Testing Service.

International Journal of Learning, Teaching and Educational Research Vol. 4, No.1, pp. 27-35, April 2014

Development Model of Learning Objects Based on the Instructional Techniques Recommendation Antonio Silva Sprock Universidad Central de Venezuela Caracas, 1043, Venezuela Julio Cesar Ponce Gallegos and María Dolores Villalpando Calderón Universidad Autónoma de Aguascalientes Aguascalientes, Ags, C.P.20131, México

Abstract. This paper presents the progress of the proposal for a model of support in the development of Learning Objects. It incorporates the most appropriate instructional techniques to the cognitive processes involved in the student learning objectives proposed by the teacher, and learning styles of students in order to create the Learning Object. The proposed model is based on Felder-Silverman learning style model (Felder & Silverman, 1988) and the cognitive processes proposed by Margarita de Sanchez (1991). The paper presents the proposed model, the cognitive processes studied, learning styles, instructional techniques included in the study and the relationship of the techniques with cognitive processes and cognitive styles of learning. Finally, it shows the mathematical model and prototype implementation of the mathematical model. Keywords: Learning Objects; Cognitive Processes; Learning Styles; Instructional Techniques.

1. Introduction Learning Objects (LO) are considered as the design paradigm of digital educational resources that can be updated, reused and maintained over time (Hernández & Silva, 2001). It should be noted that there is no single LO definition. One important definition is given by David Wiley (2000) who describes the LO like elements of a new type of computer-based instruction and based on the object orientation paradigm, so that the LO can be used in different contexts of study. Polsani (2003) indicates that it is a self-contained unit and learning, predisposed for reuse. The LO are interactive and educational resources in digital format, developed with the purpose of being reused in different educational contexts, with the same instructional need, this being its main feature, for promoting learning.

The reuse of LO is achieved by the introduction of self-descriptive information expressed into metadata, these are a set of attributes or elements necessary to describe the object, with the metadata, you have a first approach to the LO, knowing its main features, such as name, location, author, language, keywords, etc. However, because a LO is a software product for educational purposes, it is feasible to consider pedagogical, technology and Human Computer Interaction (HCI) aspects in its design.

1.1. Cognitive Learning Process These processes operate in the mental processes of acquiring new information, organization, retrieval or activation in memory. Thus they are related to regulatory processes that govern and control the mental processes involved in learning and thinking in general, affecting several activities of information processing, with special emphasis on learning complex (Rivas, 2008). The cognitive psychological processes are essential for the implementation of complex academic tasks (DĂaz-Barriga & HernĂĄndez, 2010). The basic psychological processes mentioned by Margarita Amestoy de Sanchez (1991), are: Observation, Comparison and Relationship, Simple Classification, Sorting, Hierarchical Classification, Analysis, Synthesis and Evaluation. These psychological processes are closely related to the instructional learning objective to be achieved in the design of teaching and learning process and can associate certain verbs used when generating the objectives. Every psychological process defined by Margarita de Sanchez (1991, 1991a, 1993) is described below: 1. Observation: to identify, to name, to describe, to discuss, to list, to locate, to characterize, to observe, to define, to label, to collect. 2. Comparison and Relationships: to interpret, to summarize, to associate, to differentiate, to distinguish, to compare, to relate, to merge. 3. Simple Classification: to categorize, to sort, to group, to sort, to select, to divide, to tabular. 4. Sort: to sequence, to serialize, to sort 5. Hierarchical Classification: to rank, to structure, to combine, to integrate. 6. Analysis: to connect, to predict, to extend, to interpret, to discuss, to display, to report, to experiment, to discover, to solve, to calculate, to analyze, to discriminate, to induce. 7. Synthesis: to estimate, to summarize, to apply, to demonstrate, to plan, to generalize, to complete, to illustrate, to explain, to show, to build, to infer, to create, to design, to invent, to develop, to modify, to formulate, to rewrite, to replace, to integrate, to use, to form, to deduct. 8. Evaluation: to test, to measure, to recommend, to judge, to explain, to evaluate, to criticize, to justify, to support, to persuade, to conclude, to predict, to argue, to feed back.

1.2. Learning Styles Learning Styles are a sort of personal variables that lay somewhere between intelligence and personality and explain the individual different ways of approaching, planning, and answering to the learning challenges (Kolb, 1984). ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

The Learning Styles included cognitive and affective features. Cognitive features are related to how students structure the content, form and use concepts, interpret information, and solve problems. The affective features are related to the motivations and expectations that influence learning, while physiological features are related to gender and bio rhythms, such as the sleep-wake of the student (Woolfolk, 2006). There are many classification models of learning styles, such as David Kolb model (1976), model of Ned Herrmann Brain Quadrants (Herrmann, 1982, 1990) model of NLP Bandler and Grinder (1982), model Multiple Intelligences Howard Gardner (1983), model of the cerebral hemispheres of Bernice McCarthy (1987) and the model of learning styles Felder and Silverman (1988), among others. In this work we used the model of Felder and Silverman, as a model currently working in the area of the LO (Capuano et all, 2005), (Graf, 2005), (Mustaro & Frango, 2006), (Graf and Kinshuk, 2006, 2009), (Chang et all, 2009), (Popescu, Badica and Moraret, 2010), (Alharbi et all, 2011). 1. The model of Felder and Silverman (1988) classifies learning styles based on five dimensions: 2. Sensitive-Intuitive: the sensitive student prefers to learn by studying facts that deal with aspects of daily life and the intuitive student through the study of abstract concepts. 3. Visual-Verbal: the visual student prefers to learn using visual teaching aids while the verbal student prefers to do it by listening or written form. 4. Inductive-Deductive: The best form for understanding the information for the inductive student is when he sees facts and observes and then infer the principles or generalizations, and the deductive student prefers to deduce consequences and applications. 5. Sequential-Global: the sequential student prefers to learn by following a sequential order and the global student prefers to follow a general schema that allows to visualize a whole instead of its compounding parts 6. Active-Reflective: the active student prefers to learn by doing activities and the reflective student through reasoning on things.

1.3. Instructional Techniques Instructional or teaching techniques are procedures structured logically and psychologically for directing student learning, but in a limited or in a phase of the study of a topic, such as presentation, elaboration, synthesis or critique of it (Nérici, 1992). The technique is less extensive than that of an instructional method and strategy. It is related to the form of immediate presentation of content. It corresponds to the mode of action, objectively, to achieve a goal and fulfill a definite purpose of teaching. It is part of the method in the learning implementation (Nérici, 1992). For example, a case study, projects.

2. The Problem Students, depending on their learning style, use in a conscious form, controlled and deliberate, procedures (sets of steps, operations, or skills) to learn and solve problems, i.e. structure their learning strategy (Díaz-Barriga & Hernández, © 2014 The authors and IJLTER.ORG. All rights reserved.

2010). The effectiveness thereof depends largely on the instructional strategy used (Ossandón & Castillo, 2006), in fact instructional strategies do not work in all situations to develop with any content. The LO are computer and educational resources at the same time, and often in their design the Pedagogical Dimension issues are not considered. People consider models and technical standards that ensure interoperability characteristics, accessibility, reusability, adaptability and durability. For this reason, we must also consider the pedagogical characteristics in the LO (Hernández, 2009), this means, the LO must serve to different types of users, considering the individual characteristics of each and adapting instructional activities according to the learning styles (Arias, Moreno & Ovalle, 2009). The instructional activities are implemented following instructional techniques; these techniques are part of the instructional strategies. You could say that the strategy is realized and made effective through the methods and teaching techniques (Nérici, 1992). Each instructional technique is assigned different degrees of adequacy and effectiveness in the teaching and learning, according to each learning style. Therefore, learning styles are very important in the teaching and learning process (Paredes, 2008). Felder and Silverman (1988) for example, argue that students with a strong preference for a learning style may have difficulties in the process if the learning environment does not suit their learning style. Similarly, the Pedagogical Dimension of the LO’s considers the proposed objectives, which are closely related to the cognitive processes that must operate in the mental processes of acquisition of new information, for their organization, recovery or activation in memory. Like learning styles, cognitive processes are also crucial in the selection of instructional techniques, because this has different degrees of effectiveness for each cognitive process. From these perspectives, the LO design is a challenge for a teacher, who must also choose the content, use instructional techniques, based on the student characteristics from the standpoint of the learning style of the user (Ossandón & Castillo, 2006) , and cognitive processes related to learning objective of the student, defined at the beginning of the design of LO. For all the above, what can be recommended to the LO developers in terms of the most appropriate instructional techniques to learning styles and cognitive processes involved in the learning objective?.

3. The Model In response to the above question, a model for LO development is proposed, based on the assessment of instructional techniques (Figure. 1). The teacher, through a learning platform, defines learning objectives, and then this platform selects cognitive processes involved in the objectives set by the teacher, also the teacher defines student’s learning style to whom the LO is directed and finally from a platform selects from a population of 36 instructional techniques, the techniques that best suit to the cognitive processes and learning styles selected. © 2014 The authors and IJLTER.ORG. All rights reserved.

The teacher can structure instructional strategies, using the techniques indicated and then include the activities in the LO, according to the techniques. The technology platform uses a mathematical model to select the most appropriate techniques to learning styles and cognitive processes involved.

Teacher

Learning Objetives

Cognitive Process

Learning Styles

Mathematical Model

defines

Technology Platform

Instructional Techniques

Teacher

structure

Learning Activities, Evaluation Activities

Instructional Strategies

Learning Objects

Figure 1: Development Model of LO, Based on the Instructional Techniques Recommendation.

4. The Model As noted in the previous section, the selection of instructional techniques is performed using a mathematical model, which assigns a value to each technique according to the sum of adequacy factors of each technique to each selected cognitive process and learning style indicated by the teacher. The adjustment factor for each instructional technique to each learning style and each cognitive process is in the range of [2,10]. Equation (1) presents the mathematical model which calculates the value and shows the first three instructional techniques most suitable to each cognitive process (taking only the technical adjustment factor which is greater than 8), in a descending order. 8

j 1

k 1

ti´   (ti, pj )   (ti, ek )

(1)

Where:

ti  T , i  1, , 36 pj  P , j  1, , 8 ek  E, k  1, , 4

 (t , p)  Adjustment Factor of the Instructional Techniques t respect to Cognitive Process p

 (t , k )  Adjustment Factor of the Instructional Techniques t respect to Learning Style e

5. Results Below it is shown the screen where the teacher indicates the instructional objectives (see Figure.2), then the technology platform shows the cognitive © 2014 The authors and IJLTER.ORG. All rights reserved.

processes (De SĂĄnchez, 1991), associated with the instructional objectives given by the teacher (see Figure.3) and the teacher selects the learning style according to Felder and Silverman model (Felder & Silverman, 1988).

Figure. 2. Instructional objectives indication.

Figure. 3: Cognitive Processes associated with the instructional objectives and learning styles selection. Process

Instructional Technique Total valor of Technique

Simple Classification workshop

study conducted

management notes

workshop

study conducted

pre questions

workshop

study conducted

management notes

Hierarchical Classification

Analysis

Figure. 4. Results of the evaluation of instructional techniques.

The latter figure shows to the left the cognitive processes associated with instructional objectives defined by the teacher, in this case the processes: Simple Classification, Hierarchical Classification and Analysis. For each process, it shows the three instructional techniques rated to each cognitive process. The assessment of each instructional technique, as mentioned, is associated with adjustment factors in each dimension of learning style and the factors chosen for adaptation to the cognitive processes involved. The results show the technical factors in the dimensions of learning style. It is observed that the most valued technique for the Simple classification is "workshop", whose total value is 79, which means that it fits within the value of 40 in the dimensions of the selected learning styles and a value of 39 to cognitive processes. ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

At the end the cognitive processes show the valuation of all instructional techniques included in the model. Note that the technique "Workshop" is, in general, the most valued, and properly applied in each cognitive process involved. However, the technique "addressed Study", the second highest score among all techniques, is not suitable for the hierarchical classification process. Similarly, valuation techniques which put them in third place, "prior organizers" and "Underline", respectively, are not suitable to any of the cognitive processes involved.

6. Conclusion Once the teacher selects the learning styles and verifies the cognitive processes associated with the learning objectives for students, he activates the evaluation of techniques, obtaining the best instructional techniques to be used in the development of LO (see Figure. 4). The article presents the evaluation of instructional techniques according to the valuation calculated by its relevance to the learning styles according to Felder and Silverman model (Felder & Silverman, 1988) and cognitive processes proposed by Margarita De Sánchez (1991). The Felder and Silverman model has been widely used to determine the LO suitability and of teaching resources in general. Similarly, cognitive processes defined by Margarita Sanchez is adapted to cognitive theory, emphasizing the internal forms of assimilation and processing of information.The evaluation of instructional techniques is based on the implementation of the proposed mathematical model, using the stored factors of each technique with respect to its suitability for cognitive process and learning style, these factors can be modified and better adjust by expert teachers. The proposed model may be incorporated into a LO generator, which permits the use of predesigned templates for each specific instructional technique and directed to the teacher, for the design and construction of LO.

References Alharbi, A., Paul, D., Henskens, F. and Hannaford, M. (2011). An Investigation into the Learning Styles and Self-Regulated Learning Strategies for Computer Science Students. Proceedings ASCILITE 2011, Hobart, Tasmania, Australia. Accessed May 20, 2012, from: http://www.ascilite.org.au/conferences/hobart11/downloads/papers/Alharbi -full.pdf. Arias, F., Moreno, J. and Ovalle, D. (2009). Modelo para la Selección de Objetos de Aprendizaje Adaptados a los Estilos de los Estudiantes. Revista Avances en Sistemas e Informática. Vol.6 – N°1, junio 2009. Medellín, Colombia. ISSN: 16577663. Accessed February 2, 2011, from: http://www.revista.unal.edu.co/index.php/avances/article/viewFile/14445/1 5360. Bandler, R. and Grinder, J. (1982). De sapos a príncipes. Editorial Cuatro Vientos. Capuano, N., Gaeta, M., Micarelli, A. and Sangineto, E. (2005). Automatic student personalization in preferred learning categories. In: 3rd International Conference on Universal Access in Human-Computer Interaction. Las Vegas, Nevada, USA. Accessed May 10, 2012, from: © 2014 The authors and IJLTER.ORG. All rights reserved.

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.70.8877&rep=rep1 &type=pdf. Chang, YC., Kao, WY., Chu, CP. and Chiu, CH. (2009). A learning style classification mechanism for e-learning. Computers & Education. Volume 53, Issue 2, September 2009, Pages 273–285. Accessed April 2, 2012, from: http://www.sciencedirect.com/science/article/pii/S036013150900044X. De Sánchez, M. (1991). Procesos Básicos del pensamiento. México, Trillas. De Sánchez, M. (1991a). Procesos directivos, ejecutivos y de adquisición de conocimiento. Guía del instructor. México, Trillas. De Sánchez, M. (1993). Planifica y decide. México, Trillas. Díaz-Barriga, F. and Hernández, G. (2010). Estrategias docentes para un aprendizaje significativo. 3ª Edición. McGraw-HILL, México. Felder, R. and Silverman, L. (1988). Learning and Teaching Styles in Engineering Education," Engr. Education, 78(7), 674-681. Accessed September 18, 2011, from: http://www4.ncsu.edu/unity/lockers/users/f/felder/public/Papers/LS1988.pdf. García, J. (2006). Estilos de Aprendizaje. Web de Jose Luis García Cue. Accessed December 12, 2010, from: http://www.jlgcue.es. Gardner, H. (1983). Frames of Mind: The Theory of Multiple Intelligences. New York: Basic Books, Division of Harper Collins Publishers. Graf, S. (2005). Fostering Adaptivity in E-Learning Platforms: A Meta-Model Supporting Adaptive Courses. CELDA 2005, pp.440-443, Portugal. Accessed May 13, 2012, from: http://sgraf.athabascau.ca/publications/graf_ CELDA05.pdf. Graf, S. and Kinshuk, K. (2006). Considering Learning Styles in Learning Management Systems: Investigating the Behavior of Students in an Online Course. Semantic Media Adaptation and Personalization, 2006. SMAP '06. First International Workshop on, vol., no., pp.25-30. Accessed May 12, 2012, from: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4041954&isnumbe r=4041941. Graf, S. and Kinshuk, K. (2009). Advanced Adaptivity in Learning Management Systems by Considering Learning Styles. Proceedings of the 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology, Volume 03, Pages 235-238. Accessed May 12, 2012, from: http://dl.acm.org/citation.cfm?id=1632300. Hernández, Y. and Silva, A. (2011). Una Experiencia Tecnopedagógica en la Construcción de Objetos de Aprendizaje Web para la Enseñanza de la Matemática Básica. Revista de Tecnología de Información y Comunicación en Educación Eduweb. Vol 5 No 1. Junio 2011. Accessed November 18, 2011, from: http://servicio.bc.uc.edu.ve/educacion/eduweb/vol5n1/art4.pdf. Herrmann, N. (1982). The Creative brain. NASSP Bulletin, 31-45. Herrmann, N. (1990). The Creative Brain. Brain Books, Lake Lure, North Carolina. Kolb, D. (1976). The Learning Style Inventory: Technical Manual, Boston, Ma.: McBer. McCarthy, B. (1987). The 4MAT system: Teaching to learning styles with right/left mode techniques. Barrington, IL: Excel, Inc. Mustaro, P. and Frango, I. (2006). Learning Objects: Adaptive Retrieval through Learning Styles. Interdisciplinary Journal of Knowledge and Learning Objects, Volume 2. Accessed January 20, 2012, from: http://www.ijello.org/Volume2/v2p035-046Mustaro.pdf. Nérici, I. (1992). Hacia una didáctica general dinámica. 3ª Edición. Kapelusz, Argentina. Ossandón, Y. and Castillo, P. (2006). Propuesta para el Diseño de Objetos de Aprendizaje. Revista Facultad de Ingeniería. Universidad del Tarapacá, vol. 14 Nº 1, pp. 36-48. Accessed July 13, 2011, from:

http://www.scielo.cl/scielo.php?pid=S071813372006000100005&script=sci_artte xt. Paredes, P. (2008). Una Propuesta de Incorporación de los Estilos de Aprendizaje a los Modelos de Usuario en Sistemas de Enseñanza Adaptativos. Tesis Doctoral. Universidad Autónoma de Madrid. Departamento de Ingeniería Informática. Madrid, España. Accessed March 20, 2011, from: http://arantxa.ii.uam.es/~pparedes/tesis.pdf. Polsani, P. (2003). Use and Abuse of Reusable Learning Journal of Digital Information, Volume 3 Issue 4, Article No. 164. Accessed March 23, 2011, from: http://journals.tdl.org/jodi/article/viewArticle/89/88. Popescu, E., Badica, C. and Moraret, L. (2010). Accommodating Learning Styles in an Adaptive Educational System. Informatica, International Journal of Computing and Infomatics, 34 (2010). 451–462. Accessed April 2, 2012, from: http://www. informatica.si/vol34.htm #No1 Rivas, M. (2008). Procesos Cognitivos y Aprendizaje Significativo. Madrid. Comunidad Autónoma. Servicio de Documentación y Publicaciones. Wiley, D. (2000). Connecting learning objects to instructional design theory: A definition, a metaphor, and a taxonomy. In D. A. Wiley (Ed.), The Instructional Use of Learning Objects. Accessed March 03, 2011, from: http://reusability.org/read/chapters/wiley.doc. Woolfolk, A. (2006). Psicología Educativa. 9ª Edición. Pearson Educación, México.

International Journal of Learning, Teaching and Educational Research Vol. 4, No.1, pp. 36-50, April 2014

Influential Factors in Modelling SPARK Science Learning System Marie Paz E. Morales Educational Policy Research and Development Center Philippine Normal University 1000 Manila, Philippines

Abstract. The study is focused on the exploration of influential factors in modelling PASCO-designed technology in science classes. Mixed method was employed to critically explore how the SPARK Science Learning System is meaningfully integrated into the teaching of selected topics in Earth and Environmental Science. The SPARK Science learning system is an all-in-one mobile device that integrates the power of probe ware with inquiry-based content and assessment. It is a device that includes a large, full-color display, finger-touch navigation and data collection and analysis capabilities designed to become a discoverybased science learning environment. It provides both the teacher and the students the embedded support for exploring science concepts. Results show that there is a significant gain in student achievement with the integration of SPARK Science learning system. Significant positive correlation is observed between post-test and intrinsic motivation. Correlation between post-test and evaluation and correlation between intrinsic motivation and evaluation, however, posit non-statistically significant correlation. Mapped advantages and disadvantages of using the technology resulted to recurring themes for framework design of using the SPARK Science Learning System to further institute its effect in the curriculum as a precursor towards envisioning the 21 st century learning. Keywords: Environmental Science; Technology Integration, Pedagogy

Introduction Dramatic technological revolution ushered the new millennium. Focus on digitization and technology use has been the subject of several researchers because of this trend. In many countries, today’s students are referred to as “digital natives” and today’s educators as “digital immigrants.” Thus, there is a need for teachers to work closely with students whose entire lives have been immersed in the 21st century media culture. This enculturation of students as digital natives is described as P21 or better known as “Partnership for 21st Century Skills” (Kellner 2002).

It is well-established by researches that integrating technology into the curriculum and instruction will bring about significant student achievement and therefore deepunderstanding of concepts (Clark, 2010). He claimed, however, that technology has to be integrated meaningfully into the curriculum and instruction, for probable positive impact on student learning and achievement. “Meaningful integration” of technology refers to the process of matching the most effective tool with the most effective pedagogy to achieve the learning goals of a particular lesson. Each tool brings different opportunities to the learning environment and involves a different set of skills on the part of teachers and students. Each can play a unique role in the learning process when used at the appropriate time, under the most suitable learning conditions. It is simply the degree to which a particular technology’s capabilities are matched with the expected learning outcomes and supported by fitting pedagogy that will determine the impact that technology has on learning and achievement (Clark, 2010). This match of the technological tool with pedagogy and curriculum is the main focus of the study. Further, the research would want to establish that this match is feasibly achieved by the attributes of the teachers as the “digital immigrants” working collaboratively with the students as the “digital natives” to help foster the intended partnership and be confluent with the P21 flow. The purpose of this research is to investigate the use of SPARK Science learning system in Earth and Environmental Science classes. The specific research objectives are to: 1. Determine the effect of using the SPARK devices on student motivation; 2. Establish the effect of using SPARK on student achievement; 3. Identify the influential factors in modelling integration of technology (SPARK Science learning system) in science classes; and 4. Design a framework to integrate technology in science classes and adopt them to the 21st century learning.

Framework and Literature In the 21st century framework, the definition shifts to learning towards learning technologies and on how instructional technologies can best serve learning. The Association for Educational Communications and Technology (AECT 2003) defines educational technology as "the study and ethical practice of facilitating learning and improving performance by creating, using and managing appropriate technological processes and resources” (Richey, Silber, & Ely, 2008). A revisit of important attributes of learning such as motivation and preliminary attempts of technology integration can explore the initial results of the benefits of technology in the curriculum. On motivation One of the many aspects that can help foster better achievement by students in the classroom, according to Slavin (2003), is motivation. He defined motivation as “what gets you going, keeps you going, and determines where you want to go”. Many researchers (Brookhart et al. 2006; Palmer 2005; and Mazer, Murphy © 2014 The author and IJLTER.ORG. All rights reserved.

& Simonds, 2009) provide an impression that motivation is the key component in reaching a high level of student achievement. In a study conducted by Martin (2006), he argued that if students set meaningful goals that are attainable, they will progressively achieve higher results. There is a need to provide students with a distinct set of goals that can help them be motivated. He further suggested that if students have predetermined goals they will strive for personal bests with a higher level of motivation. Teachers can play a large role in determining the motivation level of the students in the class. Studies on the effects of teacher self-disclosure on student motivation using Facebook web-based software as medium for disclosure conclude that students were more motivated when their teacher shared some personal information about themselves. However, some disadvantages of this self-disclosure surfaced with too much self-disclosure which led to non-elicitation of same motivation (Mazer, Murphy & Simonds, 2009). On technology integration and learning Educational technology has been defined in numerous ways. It usually highlights the teacher and the pedagogies that might be employed on the learner. In the 20th century, four paradigm shifts are characterized as the physical science or media view; the communications and systems concept; the behavioral science-based view; and the cognitive science perspective. Each of these shifts has different philosophical and theoretical orientations that affected theory, practice and definitions of educational technology (Saetller, 2004). Several studies have been conducted on the goodness and effectiveness of technology as integrated into the curriculum or instruction. According to Floyd et al. (2008), integration of technological advances should be a major part in designing the most effective and innovative emergent technology literacy intervention. Successful technology integration, according to Mishra & Kohler (2006), requires that educators blend strong content knowledge with appropriate pedagogical strategy. From which they were able to come up with TechnologyPedagogy-and–Content Knowledge or TPACK framework. This highlights P21 or known as Partnership for the 21st Century Skills, which focused on “meaningful” integration of technology. As expressed by Clark (2010), integrating technology in meaningful ways involves matching instructional tools with curricular goals, desired student outcomes and instructional practice. Choosing the “right” tool for a learning task requires not only familiarity with the kinds of tools available, but also depends upon an understanding of how those tools can support the development of desired knowledge and skills. As with any tool selected for any purpose, the choice of what technology to use and how to use it must be guided by a set of beliefs-a vision-for how learning is best supported. Though technology integration is foreseen as a way of attaining meaningful learning on the part of the digital natives, there were several studies marking the disadvantages of technology integration. One of which was noted by Schmidt in Veitch (2010) who voiced a concern that people might be losing deep-reading skills, as they spend less time reading long-form literature passages. This probably has an effect on cognition and reading, although no one really knows what that does. Gasser and Palfrey (2009) identified multitasking © 2014 The author and IJLTER.ORG. All rights reserved.

as a skill developed when students are engaged in technology integration. They claimed multitasking does not render learning impossible. It does not even necessarily make it more difficult to accomplish tasks. Multitasking is likely to change learning qualitatively by making the learner rely on different memory systems that vary in flexibility when it comes to the use of knowledge. However, they also mentioned that the loss of attention and the time spent switching from task to task is likely to have an adverse effect on digital natives' ability to learn complex new facts and concepts. Some of these issues and concerns of technology integration into the curriculum were addressed by Siemen’s (2005a) theory of connectivism, where he claimed that technology has also contributed to a rise in informal learning where the majority of education no longer occurs in formal settings but through learning communities of practice, personal networks and through completion of work related task. In contrast to established theories of learning, the essence of connectivism is that learning is viewed as a connections/network-forming process (Siemens, 2005b). Meaningful technology integration touches ground on motivation and appropriate use of tools to match the learners and pedagogy at hand. The information provided by this research is of value to science teachers working on similar objectives. This also allows science teachers to explore and improve their motivation techniques which may later lead to a deep conceptual understanding of the subject matter. Further, the results would help establish effectiveness of technology-inspired science classroom in trying to be at par with the 21st century learning.

Methodology The study used mixed methods in order to gather data and pertinent observations regarding the use of technology in science classroom. Presented below is the summary of the study including different stages, data gathering procedures, participants and statistical analysis. Qualitative method was used to validate quantitative results derived from the investigation. Table 1: Summary of the Methodology Stages of the Study

Data Collection

Instrument/Tool 

 Preparation and PreImplementation



 During Instruction PostImplementation



SPARK Science Learning System Orientation Administration of pre-tests

Lesson Sessions using SPARK Science Learning System Administration



 

Literacy and Technology Checklist Intrinsic Motivation Inventory Achievement Test Evaluation Form



SPARK Science Learning System



Intrinsic

Data Analysis 



Average ratings and Aiken’s content validity coefficient Averages of technology literacy constructs

Paired sample

of post-tests, postimplementation interviews

  

Motivation Inventory Achievement Test Evaluation Form Interview protocol

 

t-test Correlation Interview transcriptions

Participants The participants of this study included one intact class of tertiary students who were specializing in physics and were enrolled in both Computer Literacy 1 and Earth and Environmental Science classes. These are pre-service physics students who qualified as Philippine government scholars in science teaching. They enjoy the consortium benefits with the De La Salle University, Manila. As government scholars, these students were nationally selected from different science oriented and non-science oriented high schools all over the Philippines. They also enjoy the benefits of the grant and are envisioned to be the future Physics teachers. Materials and Instruments SPARK Learning System - This is an all-in-one mobile device that integrates the power of probe ware with inquiry-based content and assessment. The device includes a large, full-color display, finger-touch navigation and data collection and analysis capabilities. It is designed to become a discovery-based science learning tool, providing both the teacher and the students the embedded support for exploring science concepts. It has more than 60 free pre-installed SPARK-labs which are standard-based guided inquiry labs in a unique electronic notebook format that integrates background content, data collection, analysis, and assessment. (PASCO Scientific, 2008). Literacy and Technology Checklist - This instrument established the students’ knowledge and know how in technology, literacy and web expertise, which is requisite to the use of SPARK Science Learning system. The three major parts of the instrument are background information, technology component, and literacy & web expertise. The second part highlights technology component using a fourpoint Likert scale system. In addition, the other components are in openapproach. Intrinsic Motivation Inventory - This is a multidimensional instrument intended to appraise participants’ subjective experience related to a target experiment in laboratory sessions. It has been used in several experiments related to intrinsic motivation and self-regulation (Gottfried, 1985). There are several versions of this inventory. The two versions were used in the study are the full 45-item tool that completes the 7 subscales, and the 25-item version that was used in the internalization study, including the three subscales of value/usefulness, interest/enjoyment, and perceived choice. Achievement Test - The achievement test is a 19-item test, which has undergone content validation by three science experts and science educators. Item analysis procedures have reduced the number of questions of the set from 25-items to 19items. The test covered topics on radiation and insolation which are the major topics on which the SPARK Science Learning System were integrated. © 2014 The author and IJLTER.ORG. All rights reserved.

Evaluation Form - This is a 13-item survey in Likert scale intended to identify the insights of the students on the use of SPARK Science Learning System as a technology in the teaching and learning of science concepts. This postimplementation tool was administered to students where they were asked to tick on the appropriate cell. Part of the tool included questions related to the advantages and disadvantages of using SPARK Science Learning System in open-ended format.

Procedure Preparation and pre-implementation Pre-implementation commenced with the preparation of the equipment and the instruments needed for the study. Correspondence with De La Salle University, physics laboratory technicians and computer literacy instructor of the participants was done prior to implementation of the technology integration. As pre-intervention procedure, Literacy and Technology Checklist, Intrinsic Motivation Inventory, and Pre-Test (Achievement Test) were administered. Profiling of students was conducted to determine their background information and their technological literacy. Since every participant is a government scholar in physics teaching, these students are highly motivated to study Physics. Thus, SPARK Science Learning System was integrated to Earth Science lessons instead of lessons in Physics. Table 1. Technology Literacy Checklist Access on Technology

Experience with Technology

Computer Literacy and Web Expertise

0.7

0.9

0.8

0.7

0.9

0.6

Female

Marikina Science High School Ramon Magsaysay Cubao High School Tala High School

0.7

0.8

0.5

Female

LPNHS (main)

1.0

0.9

0.8

Female

DARSSTHS

1.0

0.8

0.6

Female

1.0

0.8

0.4

Female

1.0

0.8

0.5

Female

1.0

0.8

0.7

Male

Patoc National High School Ramon Magsaysay Cubao High School Sorsogon national high school Jonu Rural School

1.0

0.7

0.4

R10

Female

1.0

0.7

R11

Female

1.0

0.8

R12

Female

Muntinlupa Science High School Rizal National High School Lagro High School

1.0

0.8

R13

Female

NOHS

0.7

0.6

0.5

R14

Female

Jose P. Laurel High School

1.0

R15

Female

Rosario

0.7

0.8

0.7

Respondent

Gender

High School

Male

R3 R4

National

High

School R16

Female

San Jose National High School Pasay City South High School Ramon Magsaysay Cubao High School Cavite National High School Paranaque national High School-Lahuerta Cavite National High School MORMS

1.0

0.8

0.5

R17

Male

1.0

0.9

0.8

R18

Male

1.0

0.8

0.7

R19

Female

1.0

0.9

0.8

R20

Female

1.0

0.8

0.7

R21

Female

1.0

0.9

0.7

R22

Female

R23

Female

1.0

0.8

0.4

1.0

0.8

0.7

1.0

0.9

0.7

Male

Mount Carmel School Of Infanta Binan National High school Baclaran high School

R24

Male

R25 R26

1.0

0.9

0.7

Male

DARSSTHS

1.0

0.8

0.7

R27

Male

Paranaque National High School-Lahuerta

1.0

0.8

0.6

0.9

0.8

0.7

AVERAGE

Table 1 shows the background information and the summary of the technology literacy of the participants. The indices were computed as ratios of the averages of student ratings based on a four-point Likert scale and the theoretical average in each of the constructs: Access with technology, experience with technology, and computer literacy and web expertise. All values are close to 1 which connotes that all students are technologically literate enough ready to use the SPARK Science Learning System. These students were products of public high schools directly administered and monitored by the department of education. Everyone graduated either from science oriented schools, science high schools or department of science and technology-science education institute node schools. These participants can be said to be at par with one another in terms of learning experiences. Further, it can be inferred that majority of these students have access to computers with internet capabilities. This may be through the Learning Resource Center provided by the department of science and technology, the Philippine Normal University and the consortium benefits with the De La Salle University, Manila. The majority of the participants use computers and other computer related technology for personal interest and lesson-related activities which make their technology usage a part of their daily routine. This means that they are wellinformed in manipulation of devices and technology which has the same features as that of a computer. They can be considered ready users of the SPARK Science Learning System.

During instructions The succeeding sessions were focused on the integration of the SPARK Science learning system to two major topics in Science 3 (Earth and Environmental Science). The two major topics: radiation and insolation, in the course syllabus of Science 3 (Earth and Environmental Science) were selected for the purpose of the study. Session plans were prepared to map out the integration and instruction of the selected topics. The implementation of the integration of the SPARK Science Learning System in selected topics was conducted in several sessions. The first session highlighted the orientation on the SPARK Science Learning System. This orientation was conducted at the Philippine Normal University. In this session the researcher presented the visual reference, the userâ&#x20AC;&#x2122;s guide, and the quick start guide to the participants. Discussions on how to use the instruments and some comparison with the classical laboratory procedure were also presented and discussed with the students. The first impression of the students was that the instrument maybe very expensive. They expressed some anxiety on the use for reasons that they may damage the said instrument. Further discussions on the said instrument was done by comparing SPARK Science Learning System with some common and familiar technology these students are adapted to like the touch screen mobile phones and PSPs which helped them concretely visualize the introduced technology (SPARK Science Learning System). The succeeding sessions were hands-on orientation on the instrument and integration of the SPARK Science Learning System on selected topics in Science 3 (Earth and Environmental Science) - Radiation and Insolation. The integration procedure followed pedagogically accepted process as presented in the session plans prepared by the researcher and content validated by experts including the researcherâ&#x20AC;&#x2122;s consultant. Within the short span of time students were able to come up with good results using the SPARK Science Learning System. Post-Implementation To determine the effect of the SPARK Science Learning System, an achievement test was administered to the participants after implementation of SPARK integration. Post-test results were statistically compared to the results obtained in the pretest to determine gains if any in student achievement. Intact group pretest-post-test design was used in the study. One limitation of the study is identifying comparable set of participants. Thus, only anecdotal comparison of the student achievement using SPARK Science Learning System with student achievement without the integration was done to validate the significant statistical difference in the pre- and post-test results on student content knowledge. The intrinsic motivation inventory and evaluation tool were administered to determine whether the students were intrinsically motivated by the SPARK Science Learning System integration. Correlation of post-test with intrinsic motivation, post-test with evaluation, and intrinsic motivation with evaluation was done to identify the factors that may have influenced the gains in ÂŠ 2014 The author and IJLTER.ORG. All rights reserved.

student achievement. Transcriptions of interviews and annotation of verbatim answers on the open-ended questionnaire part of the evaluation were used to further identify influential factors in the design of framework on technology integration.

Results The primary goals of this study are to establish the effect of using SPARK Science Learning system on student achievement; to determine the effect of using the SPARK devices on student motivation; to identify the influential factors in modelling SPARK Science learning system in science classes; and design framework to integrate technology in science classes and adopt them to the 21st century learning. Results of the study are presented according to these major goals. On the effect of SPARK Science Learning System on student achievement To determine if there was a significant gain in studentsâ&#x20AC;&#x2122; content knowledge, statistical comparison of the pre-test and post-test of the participants through paired sample t-test was done as presented in Table 2.

Pair Pre-Test and Post-Test

Table 2. Paired Sample Statistics Pre Test Post Test N Mean Mean 25 9.00 13.60

p-value 0.00*

(*)Significant at 0.05

The participants performed better in the post test as compared to the pre-test with the implementation of the SPARK Science Learning System. The difference in the pre-test mean and the post-test mean was statistically significant with a pvalue of less than 0.05 (p-value = 0.00 < 0.05). As targeted, the integration of SPARK Science Learning System has brought about significant gains in the student achievement. This implies that the integration of SPARK Science Learning System in selected topics in Earth and Environmental Science is highly effective. Anecdotal comparison of student achievement using SPARK Science Learning System with student achievement without the integration was also done to validate the significant statistical difference in the pre- and post-test. Students from other classes encountered difficulty in meaning making when it comes to learning the concepts of Earth and environmental science. They usually scored lower in examinations given to them. They are not that active during class discussions and more often they encountered erroneous sets of data when performing comparable experiments with those done by the participants.

On the effect of SPARK Science learning system on student motivation Table 3. Correlation of SPARK Evaluation, Post-Test and Intrinsic Motivation Categories

Post-Test

Evaluation

Intrinsic Motivation

Mean Pearson Post-Test Evaluation Intrinsic Motivation Model Summary**

13.64

4.75

5.68

(*) Significant at 0.05

1.00 -0.063 0.618*

-0.063 1.00 -0.353 0.464**

0.618* -0.353 1.00

** Predictors: Post-Test, Evaluation & Intrinsic Motivation

Table 3 presents post-test mean value of 13.64 out of the 19-item test of the participants. This means that the participants were able to correctly answer more than 70% of the items about radiation and insolation through the integration of SPARK Science Learning System. Evaluation of the SPARK Science Learning System has a high mean value (4.75 out of 5). This connotes that participants express positive attitude towards the use of SPARK Science Learning System in learning science concepts. Intrinsic motivation has moderate mean value of 5.68 out of 7. Significant positive correlation is observed between post-test and intrinsic motivation. The other pairs: post-test & evaluation and intrinsic motivation & evaluation posit non-statistically significant correlation. Low positive correlation of three variables: post-test, evaluation, and intrinsic motivation presented in the “model summary” was observed with an R-value of 0.464. This is lower than the usually accepted value of 0.5. This implies that there may be other constructs of learning that are better predictors of student achievement other than the evaluation of the technology (SPARK) and the post-experimental intrinsic motivation.

Discussions The high mean value of the evaluation of the SPARK Science Learning System is complemented by the student answers in the open-ended portion of the evaluation. They positively identified several advantages of using the device as follows: “Learners will now find it easy and fun to do experiment. The results will be no doubt accurate.” “The SPARK is very useful during the experiments; students can easily record data accurately while doing the graphs and tables at the same time.” “Besides from being handy, it is also good in understanding a concept because the background gave the information about the concept and after this is a follow up question that will help the student think.” “It gives background concepts on the activity to be performed and asks questions to tests our knowledge on the topic.” “Results are readily seen…continuous to record data and can be saved.” © 2014 The author and IJLTER.ORG. All rights reserved.

“The device can be easily manipulated. It provides learners with necessary guide questions that directly lead to further understanding of the lesson and its concepts.” „The concepts are already stated in the activities.” “It‟s accurate, innovative, safe.” Similar answers were provided by selected students during the postimplementation interview. They pointed out how the SPARK Science Learning System was helpful and engaging to students. They attested that SPARK Science Learning System is novel to them and is very visual in perspective, which matches their learning needs and style. “Na-amaze ako mam sa nagagawa ng instrument or device.” (I was amazed with what the instrument can do.) “Yes mam, the SPARK Learning System helped a lot. I was able to answer the follow up questions with ease and also the evaluation questions.” Mam sometimes it‟s hard to learn using books alone because they are not that much available or engaging, unlike the SPARK, it has a way of making interactions work out. “Yes mam, sa tulad ko po na madali makaintindi pag may illustrations mas maganda para sa amin ang mga ganitong device para mas maintindihan and concepts.” (Yes Mam, for student like me who hardly understands concepts in science but can possibly do so with good visuals.) “I would recommend the use of SPARK Learning system but in partnership with written outputs, written graphs and computations.” However, students have also identified several areas of weaknesses and improvement in integrating SPARK Science Learning System in science lessons to make learning much more meaningful and appreciated by them. In the postinstruction interview, these pre-service students believe that the full potential of SPARK Science Learning System may be achieved in combination with other written curriculum materials. The positive correlation of post-test and intrinsic motivation could mean that they were already highly motivated in the subject area as they are science-oriented students but this intrinsic motivation is hardly identified with the integration of the SPARK Science Learning System. This result is complemented by students’ answers when asked about some disadvantages of using the SPARK Science Learning System as follows: “Graphing skills of the students and manipulating data may be affected negatively.” “Less interaction or cooperation among students since it can be done individually.” “The students will be lazy and always depends on the SPARK.” “The students might just rely on the tool in graphing and not do it manually.” “There will be little interaction between the teachers and the learners. Learners will only depend on the approaches.” © 2014 The author and IJLTER.ORG. All rights reserved.

From the transcriptions, students still hold on to the ideology of learning by doing. Graphical skills may not be developed if graphs are automatically done by the instrument. Since everything becomes automated, students seem to exert less effort and they perceive this as being lazy or not being able to give their best shot in an activity. They further claim that the tool may just develop dependency of the students to equipment rather than on their own skills. Although they were working in groups and the nature of the course is collaborative and inquirybased, they feel that interactions within the group for them to nurture relationships and build their socialization skills are fewer with the tool at hand. They also experience less interaction with the teacher since all answers regarding the topics presented can be understood using the SPARK Science Learning System. In terms of student achievement, integration implementing SPARK Science Learning System was a success. The integration brought about significant and meaningful learning on the part of the participants. Motivation, on the other hand, did not positively correlate with student perceptions on implementing the technology. The same non-correlation result was found between student achievement and student perception on the integration of SPARK Science learning system. This suggests that some other factors were able to influence the motivation of students to learn such as other pedagogical techniques, teaching and learning of other important science process skills that the technology is incapable of doing, and learning environment. These factors were identified by the participants in their verbatim answers in the open-response part of the evaluation tool. Implementing the SPARK Science Learning System could touch grounds on learning and innovation skills, which focus on creativity, critical thinking, communication and collaboration. This is a good foundation in preparation for the shift towards P21 or 21st Century Learning. Embedded in the learning system are activities that could promote the needed attributes of students to attain learning and innovation skills. With the technology, students could be able to exhibit a range of functional and critical thinking skills related to information, media and technology. The use of the SPARK Science learning system gives students more opportunities to develop skills related to information, media and technology. Life and career skills are also needed for students to navigate the complex life and work environments in the globally competitive information age. This can be achieved by combining the SPARK Science learning system with other curriculum materials that may develop the latter identified skills. These are the needed skills of a new generation student to be able to adapt and be a successful citizen. These are the bases of identifying the influential factors needed in modelling the SPARK Science learning system in science classrooms.

Pedagogical Techniques Collaborative Self-Directed Culture and Cross-Culture Based Content-Based Inquiry-Based

    

   

Assessment

21st Century Skills

Other Curriculum Material



Graphical Analysis Written Assessment Computations Use of indigenous tools



Learning and Innovation Skills Information, Media and Technology Skills Life and Career Skills

 

SPARK Integrated Assessment Performance-Based Assessment

Learning Environment    

Balance of dependence and independence Interdependence Socialization Manipulative and hands-on learning

Figure 1: Framework of Implementing Technology Integration

The framework presented in Figure 1 shows all the influential factors of modelling the SPARK Science Learning System. It was identified that integration of the SPARK Science Learning System was effective to a certain extent. The low correlation observed between evaluation of the technology & intrinsic motivation and evaluation of the technology with post-test result led to the idea that positive results may not be solely attributed to the integration of the SPARK Science Learning System in the pedagogy. It was noted that probable combination of other curriculum materials, proper learning environment, proper planning of integration process, and other forms of assessment could lead to much more meaningful integration of the SPARK Science Learning System. As claimed by Mishra & Kohler (2006), successful technology integration requires that educators blend strong content knowledge with appropriate pedagogical strategy. From the study, it can be gleaned that factors that influence the significant effect of integrating technology are clustered into four. These are pedagogical techniques, other curriculum materials, assessment procedures, and learning environment. To achieve full meaning of technology integration, combinatorial presentation of the four constructs with the integration would achieve meaningful learning. This is known as Technology-Pedagogy-and– Content Knowledge or TPACK on which the designed model is aligned.

Conclusions and Recommendations The foci of this study were to establish the effectiveness of the integration of the SPARK Science Learning System on selected topics in Earth and Environmental Science. It is intended to determine whether or not integration of SPARK Science Learning system positively affects student motivation eventually leading to student achievement; to identify influential factors on modelling technology and to design framework of technology integration. The intervention administered was effective because it led to a significant gain in the pre-test and post-test mean difference on student knowledge of the content. This implied a meaningful integration of the SPARK Science Learning System on Earth and Environmental Science. The integration of the SPARK Learning System also had positive effects on student post-experimental intrinsic motivation and was evaluated positively by the respondents. These were separately manifested in the means or averages of the data sets. However, it was noted, that two of the three variables: post-test, evaluation, and post-experimental intrinsic motivation had low positive correlation. It can be inferred that although the integration was effective, constructs other than student motivation and evaluation of the integrations contributed to the mean gain in the pre-test and post-test difference. Post-instruction interviews with the students provided other details of the low correlation. Further, influential factors that are needed in the much more meaningful integration of the SPARK Science learning system were deduced from the post-implementation interviews and open-responses of the students in the evaluation. These factors were noted as inputs to the design of framework which captured all study results for meaningful integration of technology (SPARK) leading to development of 21st century skills as preparation to P21 learning. Integrating technology is not just using the technology. It is a special skill of combining the technology with other learning constructs such as curriculum materials, pedagogy, assessment and learning environment to achieve the full potential of the technology to induce learning to the students. Though the study is able to enhance student achievement and has provided framework for meaningful integration of technology, there are still some limitations. The identified participants were government scholars in the field of sciences thus they are already science enthusiasts. Large gains in terms of student achievement and learning motivation may be deduced if the study is replicated to a group of non-science students. An experimental design with a control group may be adopted to compare student achievement with the integration to those without technology integration. The focus of the experimental study would be cognition, process skills and affective domains of learning. Pre-and post-implementation interviews may also be conducted highlighting motivation constructs and not only focused on perception of the students on the use of the technology. Classroom observations can be done to determine other significant observations which may not be provided by interviews and test results. Replication of the study is needed to fully establish effectiveness of meaningful integration of technology in learning science. A study to test the designed model may help launch meaningful integration of technology that leads to the ÂŠ 2014 The author and IJLTER.ORG. All rights reserved.

development of the 21st century skills. Teacher education curriculum designers may look into the feasibility of the model or framework in developing preservice students’ TPACK that would greatly support the development of teachers’ competencies that would help mold the 21st century learners.

References Brookhart, S.M., Walsh, J.M., & Zientarski, W.A. (2006). The Dynamics of Motivation and Efforts for Classroom Assessments in Middle School Science and Social Studies. Applied Measurement in Education, 19(2), 151-184. Clark, J. (2010). Best Practices Research Summary. Sun Associates 2010. Retrieved November 1, 2012 from www.sun-associates.com Floyd, K. et.al. (2008). Assistive Technology and Emergent Literacy for Preschoolers: A Literature Review. Assistive Technology Outcomes and Benefits, 5(1), 92-102. Gasser, U., & Palfrey, J. (2009). Mastering Multitasking. Educational Leadership, 66(6), 15-19. Gottfried, A. E. (1985). Academic Intrinsic Motivation in Elementary and Junior High School Students. Journal of Educational Psychology, 77, 631-635. International Technology Education Association. (© 2003). Advancing Excellence in Technology Literacy: Student Assessment, Professional Development, and Program Standards. Retrieved October 15, 2011from www.iteawww.org\ Kellner, D. (2002). New Media and New Literacies: Restructuring Education for the New Millennium. Retrieved March 4, 2012 from http://pages.gseisucla.edu/faculty/kellner. Martin, A. J. (2006). The Relation between Teachers’ Perceptions of Student Motivation and Engagement and Teachers’ Enjoyment of and Confidence in Teaching. AsiaPacific Journal of Teacher Education, 34, 73-93. Mazer, J., Murphy, R., & Simonds, C. (2009). The Effects of Teacher Self-disclosure via Facebook on Teacher Credibility. Learning, Media and Technology, 34(2), 175-183. Mishra, P., & Koehler, M. J. (2006). Technological Pedagogical Content Knowledge: A new framework for teacher knowledge. Teachers College Record. 108(6), 1017-1054. Palmer, D. (2005). A Motivational View of Constructivist-Informed Teaching. International Journal of Science Education, 27(1), 1853-1881. _______(2008). PASCO Scientific. Retrieved December 15, 2011 from http://www.pasco.com/prodCatalog/PS/PS-2008_spark-science-learningsystem/index.cfm Richey, R. C., Silber, K. H., & Ely, D. P. (2008). Reflections on the 2008 AECT definitions of the field. TechTrends, 52(1), 24-25. Saettler, P. (2004). The Evolution of American Educational Technology. Greenwich, CT: Information Age Publishing. Siemens, G. (2005a, January). Connectivism: A learning theory for the digital age. International Journal of Instructional Technology and Distance Learning, 2(1). Retrieved December 30, 2011 from http://www.itdl.org/ Journal/Jan_05/index.htm. Siemens, G. (2005b). Learning Development Model: Bridging Learning Design and Modern Knowledge Needs. Elearnspace. Retrieved October 25, 2011 from http://www.elearnspace.org/Articles/ldc.htm Slavin, R. (2003). Educational Psychology, Theory and Practice (7 thed.). Boston, MA: Allyn and Bacon. Veitch, M. (2010). Google's 'Deep Reading' Fears Lost in Shallows. CIO Insider. Retrieved December 20, 2011 from http://www.cio.co.uk/blogs/cio-news-view/googlesdeep-reading-fears-lost-in-shallows/ © 2014 The author and IJLTER.ORG. All rights reserved.

International Journal of Learning, Teaching and Educational Research Vol. 4, No.1, pp. 51-60, April 2014

Investigating Reliability and Validity for the Construct of Inferential Statistics Saras Krishnan and Noraini Idris University of Malaya Kuala Lumpur, Malaysia

Abstract.A hierarchical construct to assess and describe students’ learning of inferential statistics has been previously developed using the Rasch analysis. In particular, the Rasch Partial Credit Model was instrumental in identifying the number of strata in the construct and in establishing the reliability and validity of the instrument used. In this paper, the analysis is replicated with a different sample of students to investigate if the reliability and validity still hold. Keywords: assessment; Partial Credit Model; Rasch analysis

Introduction Past studies in various aspects of inferential statistics provide evidence of students’ continual difficulties in learning the many aspects and concepts of inferential statistics (e.g., Francis, Kokonis,& Lipson, 2007; Weinberg, Wiesner,& Pfaff, 2010). This situation is worrying since inferential statistics is taught in a majority of courses,and the knowledge and skills of inferential statistics will be required at one time or another by the students. Despite the many studies of students’ learning of various topics of inferential statistics, at present there is need for more research in this area(Smith, 2008; Sotos, Vanhoof, Noortgate,& Onghena, 2009). A construct of learning to describe students’ understanding of inferential statistics in hierarchical levels has been developed as part of the main author’s postgraduate research. The developmental process of this construct is discussed in Krishnan and Idris (2013a). Discussion included the use of Rasch analysis in establishing the reliability and the validity of the results, and in determining the number of levels in the construct. Further, another paper discussed the use of thehierarchical construct to investigate students’ learning of inferential statistics (Krishnan & Idris, 2013b). In this paper, we investigate the reliability and validity of the results using a different sample of students with the same sample size. The purpose of this investigation is to determine if the conditions of reliability and validity are still fulfilled when a different sample of students is used.

Literature Review Issues of concernin the assessments in statistics education included the assessment of various statistics topics and the assessment of aspects of statistics that are indicative of students’ varying levels of understanding (Bude, 2006). Another concern is that assessment of students’ statistical learning is yet to be adequately addressed (Smith, 2008). With regards to these concerns, several attempts have been made to assess and describe students’ learning of statistics in hierarchical stages of understanding. The statistical literacy construct developed by Watson and Callingham (2003) describes students’ understanding of statistics involving average and chance, sampling and inference, representation of data, and variation. The two frameworks for the statistical literacy construct are Biggs and Collis’ (1982, 1991) SOLO Taxonomy and Watson’s (1997) three tier statistical literacy model. Other constructs that assessed students’ learning of statistics have basically evolved from Watson and Callingham’s construct (e.g., Callingham, 2009; Kaplan & Thorpe, 2010; Watson, Kelly,& Izard, 2005). On the other hand, Kataoka, da Silva, Vendramini, and Cazorla (n.d.) used the SOLO Taxonomy to categorize students’ responses to a statistics questionnaire but did not offer any learning construct in their study. The Construct of Inferential Statistics (Krishnan & Idris, 2013b)contains six hierarchical levels that describe students’ understanding of inferential statistics in increasing complexity as we ascend the levels. For instance, the first level involves students’ ability to identify inferential terminologies and symbols when presented in contextual form while the fifth level involves understanding of sampling, students’ ability to infer in different contexts, and knowledge and understanding of inferential procedures and concepts. The different constructs of statistics have been developed using the Rasch model for analysis of data. Rasch model has been particularly useful in statistics assessments in determining students’ levels of understanding of various statistics concepts.Apart from that, Rasch analysis has also been used to investigate students’ understanding of basic statistical concepts (Kassim, Ismail, Mahmud,& Zainol, 2010),and to investigate attitude and knowledge of statistics among postgraduate students (Mahmud, 2011). There are two important reasons for using the Rasch analysis in our studies. First, Rasch analysis is used to determine the number of strata or levels of the construct in describing the stages of students’ understanding of inferential statistics. Second, Rasch analysis is used to establish the reliability and validity of the instrument and the sample of students. The first reason is facilitated by the use of the item separation reliability and the item-person map. The second reason is facilitated by the use of the fit analysis primarily the table of summary statistics and the table of misfit order of items. Explanation on these can be found in Krishnan and Idris (2013a, 2013b). The item-person maps are not included in this paper due to the irrelevancy to the discussion here.

Among the different types of Rasch models, the Partial Credit Model (Masters, 1982) is especially instrumental in our research because it accommodates items that have different hierarchical scoring categories. In other words, the Rasch Partial Credit Model allows the dichotomous and polytomous items to be put together in the same instrument (Bond & Fox, 2007). Thus, it is a model particularly practical and instrumental in education assessments because it is common for students to provide partly correct answers to any questions in a written assessment.

Methodology Research design Descriptive research design, in particular the cross-sectional survey method was used to collect quantitative data. Descriptive research primarily describes a current state of affairs usually with the use of visual aids (Knupfer & McLellan, 2001). Our studies employed the descriptive research design because we want to describe categories of information relating to students’ understanding of inferential statistics with the aid of the item-person map in the Rasch analysis.

Instrumentation The instrument used to collect data in this study is a task-based questionnaire on inferential statistics. Three progressive sets of pilot studies were conducted in developing the instrument. At each stage the instrument was further improved to meet the criteria of Rasch analysis particularly in terms of the reliability and validity of the instrument. In addition, the language and the structure of the questions were also modified to be able to elicit more valid responses from the students. The purpose of the first pilot study was to collect baseline data to get an idea of the possible responses to the questionnaire and possible problems in coding these responses. The second and third pilot studies had a more definite purpose of investigating the quality of the instrument whereby items that do not meet the conditions of reliability and validity are either removed from the instrument or are restructured. The results of these pilot studies are reported in Krishnan and Idris (2013a). The final instrument named as the Questionnaire for the Construct of Inferential Statistics contained 10 main items and 21 items altogether and is a task-based questionnaire that allows students to give open-ended responses. As such, we are able to gather a multitude of different responses and can perceive a greater variability of students’ learning of inferential statistics in the higher education. As of now, we are not able to furnish the questionnaire due to the unpublished status of the first author’s thesis.

Data collection The actual data collection process was carried out over a period of 6 months.Two factors contributing to the duration is the availability of the students and authorization from the higher education institutions in concern. © 2014 The authors and IJLTER.ORG. All rights reserved.

Each data collection required 40 minutes where in the first 10 minutes the students were briefed about the purpose of the data collection and were given the necessary instructions. Then, students had 30 minutes to respond to the items in the questionnaire individually. Data collection involved 150 students in each sample. In using Rasch analysis, there are no specific requirements for the sample size. In general, the sample size is large enough if the item reliability is not less than 0.90.

Samples of study Malaysia is a country in the South East Asia with a population of various ethnic, cultural and lingual backgrounds. The many ethnic groups predominantly consist of the Malay, Chinese and Indian races. The national language is the Malay language while English is widely used as the second language. The two main higher education providers in Malaysia are the government (60%) and the private sector (40%). Notwithstanding, the number of students opting for a private education has been increasing over the years (Krishnan & Idris, 2013c). Purposive sampling has been used to identify the samples of students from the different higher education institutions. Sample 1 is made up of students from one private and one semi-private higher education institutions from two different states in the central region of the country. The private higher education institution was founded more than a quarter century ago and at present offers a range of programs from pre-university studies to postgraduate courses. The students for this study are taken from one pre-university program and two different degree programs from this private higher education institution. The semi-private higher education institution has been in operation longer than the private higher education institution, having evolved from a training centre to a full fledge higher education provider. Some of the courses available at this institution are architecture, communication studies and dentistry. The students for this study are taken from an external pre-university program at this semiprivate higher education institution, which is a different pre-university program than the one from the private higher education institution. Sample 2 consists of students from a public higher education institution in the northern region that offers a range of undergraduate and postgraduate programs in pedagogy in the different faculties it houses. The 150 students sampled from this higher education institution belong to the same diploma program and is taught by the same instructor in four separate classes. Although the official medium of instruction at this institution is English, the Malay language was often used because the students are largely from the Malay language speaking background and thus have limited English speaking and writing capabilities. The teaching materials too are sometimes provided in dual languages to compensate studentsâ&#x20AC;&#x2122; English language inadequacy. The defining differences between these two samples are: (i) gender, (ii) ethnicity, and (iii) English language capability. Table 1 shows the composition of students in the samples according to this segregation. In comparison, both samples have ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

more female students than the male students. The largest ethnic group for Sample 1 is Chinese while the largest ethnic group for Sample 2 is Malay. On the other hand, the smallest ethnic group for Sample 1 is other ethnicity while the smallest ethnic group for Sample 2 is Indian. Further, a small percentage of the students in Sample 1 maintained that they have good English speaking and writing capabilities whereas for Sample 2 the studentsâ&#x20AC;&#x2122; English capability ranged from moderate to poor. None of the students in Sample 2 have good English speaking or writing capability. In fact, for both samples, the largest percentages of students have moderate speaking and writing capabilities of the English language. Table 1: Composition of students in the samples

Gender Ethnicity

Spoken English

Written English

Male Female Malay Chinese Indian Others Good Moderate Poor Good Moderate Poor

Sample 1 42.7% 57.3% 29.3% 56% 8% 6.7% 14.7% 67.3% 18% 15.3% 66% 18.7%

Sample 2 25.3% 74.7% 88.7% 2% 0.7% 8.7% 0% 81.3% 18.7% 0% 87.3% 12.7%

Analysis of Results Table 2 shows the reliability and fit indices for Sample 1. These results have been discussed in earlier paper that described the development of the hierarchical construct (Krishnan & Idris, 2013b). The purpose of this study is to investigate the results of these indices for a different sample, Sample 2. The item separation reliability determines the breadth of the items whereby a value more than 1.00 indicates that the items have enough breadth as with the case of Sample 1. In a similar manner, the person separation reliability must be more than 1.00 to warrant that the students are measured across the continuum. This condition has been met by Sample 1. Table 2: Reliability and fit indices for Sample 1

Item separation reliability Item infit mean square Item reliability Person separation reliability Person infit mean square Person reliability Cronbachâ&#x20AC;&#x2122;s alpha

6.48 1.00 (s.d. 0.08) 0.98 1.77 1.03 (s.d. 0.33) 0.76 0.75

The item infit mean square and the person infit mean square must be in the range of 1.00 to 1.20 to be reckoned as acceptable. Value less than 1.00 means that the responses are too predictable. It also suggests the presence of redundant items. On the other hand, value more than 1.20 suggests unpredictable responses or inappropriate response patterns. Meanwhile, the standard deviation must be smaller than 2.00 to indicate little misfit.Both the item infit mean square and the person infit mean square for Sample 1 as well as their standard deviation values met the required conditions. As mentioned in Krishnan and Idris (2013b) there is no hard and fast rule on the acceptable range of the fit statistics and different researchers have complied with different ranges of these values. Discussion on the possible different values of the fit statistics can be found in Green and Frantom (2002), and Linacre (2002). In addition, the item reliability, the person reliability and Cronbach’s alpha in Table 1 are more than 0.70. The item reliability and the person reliability values are equivalent to the value of Cronbach’s alpha, said Green and Frantom (2002). In this study, Cronbach’s alpha of 0.70 is used as an acceptable reliability coefficient (Nunnaly, 1978; Santos, 1999). For Sample 2, some of the aforementioned conditions were met whereas others were not. First, the item separation reliability of 3.98 and the person separation reliability of 1.03 both satisfy the condition that these values must be more than 1.00. However, they are lower than the values for Sample 1. This observation suggests that the spread of the items and students in Sample 2 is smaller compared to Sample 1. The item infit mean square for Sample 2 is in the stipulated range of between 1.00 and 1.20 but the person infit mean square does not fulfil this condition. Likewise, the item reliability is more than 0.70 but the person reliability is not. The Cronbach’s alpha too does not meet the condition of reliability. Table 3: Reliability and fit indices for Sample 2

Item separation reliability Item infit mean square Item reliability Person separation reliability Person infit mean square Person reliability Cronbach’s alpha

3.98 1.00 (s.d. 0.10) 0.94 1.03 0.98 (s.d. 0.45) 0.51 0.58

The misfit order of items for analysis of both samples is displayed in Table 4. The infit mean square values (denoted by MNSQ) and the infit z-standardized values (denoted by ZSTD) are investigated to establish the validity of an instrument whereby the conditions for validity are: (i) MNSQvalues between 0.70 and 1.33 (Watson & Callingham, 2003), and (ii) ZSTD values between -2.00 and +2.00 for samples of sizes between n = 30 and n = 300 (Bond & Fox, 2007).

Table 4 shows that both conditions have been met for Sample 1 and Sample 2. For Sample 1 the MNSQ values ranged from 0.86 to 1.17 and the ZSTD values ranged from -1.60 to 1.40 (Krishnan & Idris, 2103b). Meanwhile, for Sample 2 the MNSQ values ranged from 0.79 to 1.27 and the ZSTD valuesranged from -0.80 to 1.40. Table 4: Misfit order of items

Sample 1 MNSQ ZSTD 1.15 1.40 1.12 1.30 1.17 1.20 1.07 0.60 1.13 1.10 0.94 -0.60 0.94 -0.60 1.00 0.00 0.92 -0.40 1.03 0.40 1.01 0.20 0.96 -0.40 0.99 -0.10 0.99 -0.10 0.98 -0.10 0.98 -0.10 0.97 -0.20 0.94 -1.20 0.91 -0.90 0.88 -1.10 0.86 -1.60

Sample 2 MNSQ ZSTD 0.92 -0.20 1.27 1.00 1.07 0.90 1.14 1.40 1.09 0.60 1.00 0.10 1.07 0.30 1.04 0.70 1.04 0.40 1.01 0.20 1.02 0.20 1.01 0.10 1.00 0.30 1.00 0.30 0.99 -0.10 0.92 -0.80 0.98 0.00 0.96 -0.10 0.89 -1.30 0.89 -0.40 0.79 -0.60

Overall, the analyses from Sample 1 and Sample 2 reveal that the reliability and validity of the instrument has been established regardless of the sample diversity. Especially the results of analysis of Sample 2 corroborate the quality of the Questionnaire for the Construct of Inferential Statistics because the conditions of reliability and validity have been met by the instrument despite the students in Sample 2 not fulfilling the conditions of reliability.

Conclusion Statistics assessment has evolved in the past 40 years (Jolliffe, 2007) from assessing students’ knowledge of statistical formulas to assessing students’ understanding of statistical concepts. The various existing constructs to assess students’ learning of statistics are largely concerned with students’ understanding of the descriptive statistics. We have developed a construct to assess students’ learning of the inferential statistics in the higher education contexts and have discussed the development of this construct in earlier papers (Krishnan & Idris, 2013a, 2013b). © 2014 The authors and IJLTER.ORG. All rights reserved.

The development of the construct of inferential statistics basically supports the requirement to increase the number of literature in the area of students’ learning and understanding of inferential statistics because studies in this area are still scarce at present (Smith, 2008).The construct of inferential statistics can be utilized by statistics educators to improve students’ understanding of the logic of statistical investigations and the need to infer from samples to populations. It can also aid in developing students’ deep and connected understanding of inferential statistics. By identifying the different levels of students’ understanding of inferential statistics, instructors can focus on the development and improvement of students’ understanding of the levels in concern. In this paper, we investigated if the reliability and validity achieved earlier is maintained if a different sample of students were used. It was found that although the sample of students in this study did not meet some conditions of reliability, the reliability and validity of the instrument was fulfilled. This served to verify the validation of the instrument and subsequently the validation of the construct. However, this study did not investigate if gender, ethnicity or language capability or a combination of them affected the results. We suspect language or rather lack of it could have played a major role because a number of questionnaires from Sample 2 were barely attempted and some students conceded that it was because they did not know how to explain their reasoning in English. Lesser (2010) believes that student diversity interacts with the learning of statistics and it is important for instructors to use student diversity as an opportunity instead of obstacle. Lesser and Winsor (2009) also believe that language is an important factor in students’ performance but found that at present there is lack of research on statistics learning involving English articulateness. Future possible work with respect to this paper is to investigate in detail how students’ different language capabilities affect the reliability and validity of the construct.

References Biggs, J.B.,& Collis, K.F. (1982). Evaluating the quality of learning: The SOLO taxonomy. New York, NY: Academic Press. Biggs, J.B.,& Collis, K.F. (1991). Multimodal learning and the quality of intelligent behaviour. Intelligence: Reconceptualization and measurement, 57-76. Bond, T.G.,& Fox, C.M. (2007). Applying The Rasch Model: Fundamental Measurement in the Human Science. Mahwah, NJ: Lawrence Erlbaum Associates. Bude, L. (2006). Assessing students’ understanding of statistics. In Proceedings of the 7th Annual Meeting of the International Conference on Teaching Statistics. Brazil: ISI. Retrieved April 16, 2010 from www.stat.auckland.ac.nz/~iase/publications/17/6G3_BUDE.pdf Callingham, R. (2009). Using Rasch Measurement to Identify Cross-cultural Aspects of Statistical Literacy.Changing Climates: Education for Sustainable Futures, 1, 1-10. Francis, G., Kokonis, S.,& Lipson, K. (2007). Enhancing Student Understanding in Statistical Inference-Assessing the Effectiveness of a Computer Interaction. IASE/ISI Satellite. Retrieved February 20, 2011 from http://iase-web.org/documents/papers/sat2007/Francis_et_al.pdf

Green, K. E., & Frantom, C. G. (2002). Survey Development and validation with the Rasch Model. Paper presented at theInternational Conference on Questionnaire Development, Evaluation, and Testing. Charleston, SC.Retrieved February 6, 2013 from http://www.jpsm.umd.edu/gdet/final_pdf_papers/green.pdf Jolliffe, F. (2007). The Changing Brave New World Of Statistics Assessment. IASE/ISI Satellite. Retrieved February 20, 2011 from http://www.stat.auckland.ac.nz/~iase/publications/sat07/Jolliffe.pdf Kaplan, J.J.,& Thorpe, J. (2010). Post Secondary and Adult Statistical Literacy: Assessing Beyond the Classroom. In Proceedings of the Eighth International Conference on Teaching Statistics. Netherlands: ISI.Retrieved September 2, 2011 from http://iase-web.org/documents/papers/icots8/ICOTS8_5E3_KAPLAN.pdf Kassim, N.A., Ismail, N.Z., Mahmud, Z., & Zainol, M.S. (2010). Measuring Students Understanding of Statistical Concepts using Rasch Measurement. International Journal of Innovation, Management and Technology, 1(1), 13-19. Kataoka, V.Y., da Silva, C.B., Vendramini, C.,& Cazorla, I. (n.d.). Using Rasch Partial Credit Model to analyze the Responses of Brazilian Undergraduate students to a Statistics Questionnaire. Retrieved June 1, 2014 from http://www.cerme7.univ.rzeszow.pl Knupfer, N.N.,& McLellan, H. (2001). Descriptive Research Methodologies. In D.H. Jonassen (Ed.), Handbook of Research for Educational Communications and Technology (pp. 1196-1213). Mahwah, NJ: Lawrence Erlbaum Associates. Krishnan, S., & Idris, N. (2013a).The Development of an Assessment Construct for Inferential Statistics. Paper presented at the International Conference on Assessment for Higher Education Across Domains and Skills (AHEADS2013). Kuala Lumpur, Malaysia. Krishnan, S., & Idris, N. (2013b). The Use of a Hierarchical Construct to Investigate Students’ Learning of Inferential Statistics. In Proceedings of the Joint IASE/IAOS Satellite Conference(pp. 1-8).Macao, China. August 2013. Krishnan, S., & Idris, N. (2013c). The Use of Graphics Calculator in a Matriculation Statistics Classroom: A Malaysian Perspective. Technology Innovations in Statistics Education, 7(2), 1-13. Lesser, L.M. (2010). Equity and the Increasingly Diverse Tertiary Student Population: Challenges and Opportunities in Statistics Education. In Proceedings of the Eighth International Conference on Teaching Statistics (pp. 1-6).Netherlands: ISI. Retrieved December 10, 2013 from http://iase-web.org/documents/papers/icots8/ICOTS8_3G3_LESSER.pdf Lesser, L.M., & Winsor, M.S. (2009). English language learners in introductory statistics: Lessons learned from an exploratory case study of two pre-service teachers. Statistics Education Research Journal, 8(2), 5-32. Linacre, J.M. (2002). What do Infit and Outfit, Mean-Square and Standardized Mean. Rasch Measurement Transactions, 16(2). Mahmud, Z. (2011). Diagnosis of Perceived Attitude, Importance, and Knowledge in Statistics Based on Rasch Probabilistic Model. International Journal of Applied Mathematics and Informatics, 5, 291-298. Masters, G.N. (1982). A Rasch model for partial credit scoring. Pschometrika, 47, 149-174. Nunnaly, J. (1978). Psychometric theory. New York, NY: McGraw Hill. Santos, J.R.A. (1999). Cronbach’s alpha: A tool for assessing the reliability of scales. Journal of extension, 37(2), 1-5. Smith, T.M. (2008). An Investigation into Student Understanding of Statistical Hypothesis Testing. Doctoral dissertation. University of Maryland. Sotos, C., Vanhoof, S., Noortgate, W.,& Onghena, P. (2009). How confident are students in their Misconceptions about Hypothesis Tests?. Journal of Statistics Education, 17(2). © 2014 The authors and IJLTER.ORG. All rights reserved.

Watson, J.M. (1997). Assessing statistical literacy using the media. In I. Gal & J.B. Garfield (Eds.),The Assessment Challenge in Statistics Education (pp.107-121). Netherlands: IOS Press. Watson, J.M.,& Callingham, R.A. (2003). Statistical literacy: A complex hierarchical construct. Statistics Education Research Journal, 2(2), 3-46. Watson, J.M., Kelly, B.A.,& Izard, J.F. (2005). Statistical Literacy over a Decade. Building connections: Theory, research and practice, 1, 775-782. Weinberg, A., Wiesner, E.,& Pfaff, T.J. (2010). Using Informal Inferential Reasoning to Develop Formal Concepts : Analyzing an Activity. Journal of Statistics Education, 18(2), 1-23.

International Journal of Learning, Teaching and Educational Research Vol. 4, No.1, pp. 61-75, April 2014

Influence of Head Teachers‟ Management Styles on Teacher Motivation in Selected Senior High Schools in the Sunyani Municipality of Ghana Magdalene Brown Anthony Akwesi Owusu Department of Arts and Social Sciences Education University of Cape Coast Ghana Abstract. Senior High School administration in Ghana is bedeviled with a lot of problems most of which emanate from deficiencies in management styles of heads of these institutions. There have been times when teachers had to stage open protests to register their displeasure about head teachers‟ management styles which they often described as administrative and managerial incompetency. This situation has often led to low morale among teachers and therefore this study focuses on finding out the effects of head teachers‟ management styles on teachers‟ motivation in SHSs in Ghana. The study also attempts to look at and explain how head teachers‟ informal relationship with teachers serves as a motivation for teachers to work to achieve institutional goals. The study sampled 100 senior high school teachers and 10 head teachers in the Sunyani Metropolis. These were sampled using the quota and simple random sampling procedures. The study, among other things, revealed that most teachers see their heads as bosses and not as friends. Again, it was revealed that though teachers were involved in decision making, the actual setting of objectives for a school were left in the hands of the heads and the school management team. It was therefore recommended for head teachers to adopt the Management by Walking About and Management by Objective so as to get closer to their teachers in the running of schools.

Key words: Management by Objective; Management by Walking About; Motivation

Introduction In every organisation for which the school is not an exception, management is expected to produce results. These results do not just happen overnight. They demand great efforts by the leaders who in turn, are to spearhead the affairs of the organisation. A high degree of workplace spirituality and spiritual leadership, as a driver of organizational commitment and productivity, is important to enhancing organizational performance (Fry & Matherly, 2006).The person at the helm of affairs is usually the manager (Ekeland, 2005). The

managers of various organisations were seen as not sociable and distant people who are seen once in a while either at a meeting or for specific programmes. Also, the norm was that the communication process was a downward one whereby decisions are taken by managers and pushed down to the subordinates. However, for the objective of an organisation to be achieved, it demands the collective efforts of both managers and employees. This case is not different when it comes to school management. If the management of any school will achieve results, the efforts of the teachers must be appreciated. For managers to achieve their organizational targets, services of people are imperative which in the school setting will be teachers (Thomson, 1998). Managers can only use this tool (people) effectively when they instill in them a sense of commitment and the desire to accomplish organizational goals. Again, if this tool can be used well, their efforts should be controlled and coordinated toward goal accomplishment. The manager in all these should give subordinates the opportunity to increase their skills and abilities in contributing to achieving the organization‟s aspirations. The individual‟s style will be use based on a combination of their beliefs, values and preferences, as well as the organizational culture and norms which will encourage some styles and discourage others (Almansour, 2012). Teacher Motivation Aacha, (2010) in a study alluded to the fact that teacher motivation has become an important issue since teachers‟ preoccupation is to transmit skills, knowledge and attitudes to learners. Teachers who are satisfied with their job tend to give off their best and can go a long way to influence students‟ performance (Mertler, 1992). Motivation guide people‟s actions and behaviours toward achievement of some goals (Analoui, 2000). In the world work and the school setting, motivation can be perceived in two distinct contexts: intrinsic and extrinsic motivation (Sansone & Harackiewicz, 2000). The former emanates from within the individual and positively influences their behavior, achievement (Ryan & Deci, 2000). The latter on the contrary, comes as result of influences from the external environment which acts as stimulus. Thousands of studies have been conducted following Thorndike‟s (1911). On such study was the Emery Air Freight study carried out by Hamner and Hamner (1976) and Komaki (1982) on how behaviors change by the manipulation of extrinsic factors. With intrinsic motivation, one performs an act for its own sake rather than being urged by an external factor. The issue of intrinsic motivation is a vital concept which has been amply dealt by White (1959), Maslow (1943) and Alderfer (1969). A research paper on extrinsic and intrinsic motivation of teachers that was measured in terms of teaching work satisfaction alluded to a number of intrinsic motivational variables. Some of these included satisfaction derived from teaching, setbacks in teaching, the competition in teaching, recognition, career advancement and, control over others. The paper also viewed the extrinsic motivation of teachers in terms of salaries, free accommodation, free meals, weekly duty allowances and extra teaching load allowances, advance payments in case of financial problems, leave of absence and free medical care among others.

Teachers, per their job make or unmake a society and this is why their work output is of great interest to everybody in society. Oxford Advanced Learner„s Dictionary defines performance as - the act or process of carrying out something or execution of an action or a repetitive act or fulfillment or implementation (Hornby, 2000). In this respect, teacher performance connotes the teachers‟ role of teaching students in class and outside the class. The key aspects of teaching involve the use of instructional materials, teaching methods, regular assessment of students, making lesson plans, assessment of pupils, conduct of fieldwork, teachers‟ participation in sports, attending school assembly and guidance and counseling. Therefore, teacher job performance is the teacher„s ability to integrate the experience, teaching methods, instructional materials, knowledge and skills in delivering subject matter to students in and outside the classroom. Teacher performance in this study was measured by regular and early reporting at school, participation in extra-curricular activities, supervision of school activities, adequate teaching preparation (schemes of work, lesson plans), marking and general punctuality among others. Teaching is a mass occupation, which accounts for one-half to two-thirds of public sector employment in most developing countries (Bennell, 2004). While other professions (engineers, medical doctors and lawyers) enjoy a high degree of self-regulation and are successful in maintaining high barriers to entry in terms of qualification requirements and registration, teachers on the other hand, tend to have weak, state-dominated professional organizations with several trade unions. In addition, public sector recruitment freezes in many countries during the 1990s have seriously constrained the growth of the number of teachers in government or government-aided schools. Although such events are expected to have affected the morale of teachers to perform, detailed studies examining the effect of motivation on performance of teachers are still scanty. Available evidence, however, reveals that the teaching force has expanded rapidly in only a relatively few countries (most notably Bangladesh, Malawi, Ethiopia, Eritrea, Mozambique and Uganda). Bennell (2004) indicates that teaching has become employment of the last resort among university graduates and secondary school leavers. About one-half of junior secondary school leavers in Malawi and Tanzania who finished school in 1990 were employed as teachers in 2001. The corresponding figure for Uganda is a staggering 81 per cent (AlSamarrai & Bennell, 2003). Consequently, teachers often lack a strong, long term commitment to teaching as a vocation. Besides, the status and pay of primary school teachers compared to secondary school teachers is generally much lower in developing countries. Thus, in the absence of alternative employment opportunities, becoming a secondary school teacher is the main avenue for social and economic advancement for the most able primary school teachers. This has had important implications for intrinsic motivation of primary school teachers. Literature have reviewed various management styles adopted by management but this study sought to look at how head teachers as managers of schools are able to adopt the management by objective and management by walking about style in their management practices and its effects on teacher motivation. Management by walking/wandering about, according to Peters and Waterman

(1982) “is an unstructured approach to hands on, direct participation by the managers in the work-related affairs of their subordinates, in contrast to rigid and distant management”. For teachers to be motivated extrinsically to give off their best, the heads‟ management style contribute immensely (Holten, Dent & Rabbett, 2009; Aacha, 2010). The issue of management style and employee performance has gained so much ground especially the western world. Many researches conducted in the field of Business world have proven that the management styles of managers greatly influence the motivation of subordinates. However, when it comes to the teaching field it remains unclear how the styles of management of heads have effects on teacher motivation. The output of the Ghanaian teacher is said to be going down in recent years. Most of these teachers are now performing below expectation just because people claim that their incomes are low but this problem could be due to so many factors and one such factor could be the management styles adopted by their heads. Management styles which are adopted by head teachers usually affect the performance of teachers in that they either positively or negatively on teacher motivation to give off their best. Also most research work on management styles based on the popular management styles as the democratic, autocractic, laissez-faire etc. but much has not been done when it comes to the management by objective and management by walking about. If head teachers as school managers, sit jointly together with their teacher to set specific institutional objectives to be accomplished within a certain specific time frame, all players buy into the vision, hands are put on deck, objectives are attained. The emphasis of this management style is that goals jointly set by teachers and heads within the school boosts teacher morale. Also, the informal relationship of head teachers‟ informal towards their teachers serves as a motivation for teachers to work to achieve institutional goals. The normal style of the heads is seeing them in their offices performing their role. The teachers see their heads only on formal grounds like meetings or being summoned to see the head in his or her office. Again, what normally happens is that objectives are set by the management board and the heads normally bring them to staff meetings for discussion. The question therefore is “are the heads aware there is a management technique like these and are the head teachers actually using the management style of walking/wandering around or management by walking about and what effects have these styles on teacher motivation?. Also do the head teachers actually involve their teachers in setting objectives? To help unfold these issues this research is aimed at finding the effects of the heads management style on teacher motivation. The purpose of the study was to identify the management styles of head teachers and their effects on teacher motivation. The research will therefore bring to light the importance of these two vital management styles that can be adopted by managers of schools to increase teacher motivation. The study focused only Senior High Schools in Brong Ahafo and not the entire country; hence the generalization of the result might not be

easy since the condition pertaining to one region might not be the same for another region. The study was confined to only Brong Ahafo Region and not the entire country. Again, the study did not address other management styles adopted by managers in their day-to-day activities. Research Questions To what extent does head teachers‟ management style (MBO and MBWA) influence teachers‟ motivation to perform? Research Hypotheses H0: Head teachers‟ management style has no significant effect on teacher motivation to perform.

Overview of Literature This chapter sought to review literature on the two management styles (Management by Objective and Management by walking/wandering around and motivation. Management by objective (MBO) Management By Objective theory was introduced to the business science in the mid of 1950s by Peter Drucker, a prominent management scientist. Drucker‟s (1954) concept of the MBOs introduction was the apparent importance of businesses‟ clear objectives to their profitability, productivity, share market increase and concurring reputation. According to Drucker (1954), the procedure of objectives‟ setting and progress‟ monitoring are determining factors towards the function of organizations, thus these factor should permeate the entire organization, from top to bottom. To achieve this, it is important for top level management to jointly set organizational targets with their subordinates. Management by objective is a system whereby the superior and subordinates of an organisation jointly identify its common goals, define individual‟s responsibility in terms of expected results. It makes a demand on every manager to set targets to be achieved in the future and inspires subordinates to persistently ask what can be done (Thomson, 1998). He went further to assert that organizations do not exist for their own sake but for a purpose. And to achieve this purpose, higher level managers should set attainable and specific objectives by involving subordinates at all levels of decision making in attaining targets. Organizations which do not adopt the MBO strategy normally risk having a downward decision-making approach. That is, the goals are set by top management and are handed down to subordinates for implementation. The lack of participation by subordinates which is usually manifested by noncommitment in the decision-making process hampers the full realization of organizational goals and objects. According to Thomson this does not encourage commitment on the part of the subordinates. MBO gained attention because it focuses on objectives and results which a manager wants to achieve in specific time as well as focusing on participative management

MBO is a result centered managerial approach for the effective utilization of material, physical and human resources of the organization. MBO tries to combine long term goals of organization with short term goals. MBO does not only focus on goals but also on effective performance. It also focuses of the participation of employees in goal setting process. In the school Management by Objective could be applied. The head can adopt the participative management approach where formulation of objective becomes a shared activity between the head and the teachers. This could be exhibited in the nature of decision making in the schools. Staff members involved decision making generally develop a high level of satisfaction. Teachers must be recognised and respected by the administration for their expertise (Conley, Schmidle, & Shedd, 1988). Teachers are empowered to act as leaders in concert with the principal. Liontos (1993) suggested that a shared decision-making strategy has the potency to improve the quality of decisions thereby increasing decision acceptance rates, boost staff morale , increase staff efficiency, staff commitment and teamwork; build trust among staff; help staff acquire new skills and increase overall school effectiveness. Management by Wandering About/ walking Around The term „Management by Walking‟ (MBWA) was defined by Peters and Waterman (1982) as a style of management whereby managers wander around in an unstructured manner at the workplace randomly with a view to checking with employees, equipment, about the status of ongoing work. This definition emphasizes „wandering‟ as a random movement in a workplace rather than a carefully plan visits by managers to employees at a more systematic and prearranged times and venue. The advantage with this style of management is that a manager is more likely to increase productivity and total quality in the management of the organization in contrast to staying glued in a specific office and waiting for employees, or the delivery of feedback reports on what goes on in the organization. This term was first used by executives of Hewlett-Packard company, in the 1970s (Mears, 2009). MBWA was used by the above company executives in boosting morale within the company. Following success in its usage, the style was embedded into the culture of the organization. However, Hewlett-Packard company was not the only company in those days that adopted this style of management where managers made unstructured visits to subordinates in the workplace. For instance, Tom Peters and Robert H. Waterman have earlier used this term in their book: „In Search of Excellence: lessons from America's best-run companies‟ in 1982. According to Hinners (2009) management by walking about connotes a sense of purposeful random visit by a manager to learn at first hand, the working conditions which have beneficial and fruitful consequences for organizational growth. He explained how he adopted the MBWA in his managerial role when he was appointed the director for Smithsonian National Air and Space Museum. He commented on how MBWA has served as an eyeopener in his career. He explained that by walking around the working sites and asking questions he was able to get vital information he believes would never had come at a formal meeting. Roaming about at the work site helps the © 2014 The authors and IJLTER.ORG. All rights reserved

manager to learn on a daily basis procedures and processes in the organization. He explained that subordinates see you (manager) as a superior; they will be more willing to open up to tell you (the manager) the happenings and problems within the organization before some of them become problems. Furthermore, as subordinates gets to know their boss better, they tend trust him/her more. Another advantage of adopting MBWA is that subordinates will be ready to share more vital information with the manager and this can help break down barriers to communication. Again, visits by the manager to different departments can help him/her discover and fix problems earlier in efficient manner. According to Hinners (2009), communication in MBWA thus becomes spontaneous and symbiotic and also gives room for on-the-job mentoring. These features described earlier differentiate MBWA from other management styles. This approach guarantees a fantastic multiplier effect as far as information diffusion is concerned. Word gets around rapidly and potentially makes people (workers or subordinates) feel comfortable coming unannounced to offices of their bosses (heads) to get some tidbit off their chest. Another dimension of this approach is the use of norm, a most unusual leadership style which makes use of profound technical skills, and an ability to analyze audience using appropriate humor and presentation skills. It enables subordinates to put on a “can do” spirit attitude: after all, nothing is impossible and challenging “what we do”. It is a wander why many managers do not adopt this style (ie MBWA). Nonetheless, MBWA is not a panacea to all managerial problems. To make MBWA successful, other management tools and styles must be adopted concurrently. As is true of any individual management tool, it must be augmented and complemented by a host of other proven techniques. Management by Walking About can easily be adopted by heads of schools where they could pay random visit to classrooms to interact with the teachers and be able to get hands on information about the activities that go on in the classrooms. The two approaches are expected, when used properly to motivate teachers to function effectively and efficiently as teachers. Figure 1 gives a diagrammatic representation of the two styles opined by Oates (1977) and Hinners (2009).

Figure 1. The process and impact of MBO and MBWA in an organization (Author derived)

Methodology Research Design The cross sectional survey design of the descriptive design was adopted for this study. The cross sectional survey design was employed in carrying out the study. This design was adopted because it affords the researcher the opportunity to observe, assess and describe the extent to which the management styles of head teachers in the Senior High Schools in the Brong Ahafo Region motivate the teachers to perform. Population of the Study The population comprised 10 senior high school head teachers and 633 teachers in the Sunyani Municipality. The sample for this study comprised 100 teachers who were randomly selected from all the public senior high schools in the Sunyani Municipality. The census survey was used to select 10 head teachers from all the selected senior high schools in the Sunyani municipality. Instrument for Data Collection According to Gay (1992), all research studies involve data collection with the help of research instruments. For this reason the researchers decided to use questionnaire for both the head teachers and the teachers. The questionnaire was divided into three sections. Section A was the introductory section containing an item on teachers‟ education zones. Section B comprised 7 items on the relevance of MBO and teacher motivation, while section C had 12 items on the benefits of MBO to school management. The items in the questionnaire were both closed ended and open ended questions. Validity and Reliability of Instrument The instrument was given to expert to ascertain its content and face validity. Changes were effected to improve upon the instrument. The instrument was pilot tested thereafter. The purpose of the pilot testing according to Brown (2012) was to gain an insight into the relative strengths and weakness of the research instrument in order to make possible improvement prior to the main study. One hundred and ten (110) questionnaires were administered to both head teachers and teachers in the Sunyani Municipality. This area was chosen because the researchers believe that the head teachers and teachers in this district bore similar characteristics in terms of qualification and experience with other teachers. The data gathered were analysed and the Cronbach‟s alpha was established for each of the items. Both questionnaires had a Cronbach‟s alpha value of 0.80 which was respectably reliable in every sense.

Results and Discussion Data Analysis The outcome of the study has been presented and discussed in the following section. The study examined the extent to which head teachers management style (MBO and MBWA) influenced teachers‟ motivation to perform and whether head teachers‟ management style had any significant effect on teacher motivation to perform. © 2014 The authors and IJLTER.ORG. All rights reserved

Table 1: Management by Objective Statement

Strongly Agree No. (%) 13(13)

Agree No. (%)

Disagree No. (%)

Mean

15(15)

Strongly Disagree No. (%) 0(0)

72(72)

55(55)

17(17)

15(15)

13(13)

3.22

My head rewards for the accomplishment of goals and this serves as a motivation for me

0(0)

17(17)

28(28)

55(55)

1.62

I have been involved in establishing disciplinary policies and so I am motivated

0(0)

13(13)

87(87)

0(0)

2.13

I have been involved in planning structural facilities for the school and this is an incentive to work hard.

0(0)

13(13)

32(32)

55(55)

1.58

I have been involved in planning new projects for the school and so I give off my best

27(27)

13(13)

32(32)

28(28)

2.39

I have been involved in resolving staff disputes and this encourages me to give off my best

44(44)

41(41)

15(15)

0(0)

3.29

My head teacher brings specific goals and measures for discussion during staff meetings and this motivates me to give off my best I am liable for the accomplishment of goals

Total

2.98

2.46

It could be deduced from table 1 that the heads bring specific goals to staff meeting for discussion because majority, 72(72%) agreed to that statements. Again it could be observed that 55(55%) of the responded agreed that their heads hold them responsible for the accomplishment of those goals brought for discussion. However, issues concerned with disciplinary policies, a clear majority disagreed that they are involved in such decisions and this clearly discouraged a lot of them. Also, on the issues on planning of structural facilities and new projects, a greater number of the respondents disagreed with those issues. This is a clear indication that most of the decisions taking in the schools are solely taken by the head teachers. Again, on the issue of whether the respondents were involved in planning disciplinary policies, majority, 87(87%) indicated that they were not involved. Druckerâ&#x20AC;&#x;s (1954) who introduced the concept of Management By Objective was of the view that for an organisation such as the school to be able to achieve its objectives successfully it is important that top level management (head teachers) jointly set organisational objective

with their subordinates (teachers). Liontos (1993) stated that a shared decisionmaking strategy has the potency to improve the quality of decisions thereby increasing decision acceptance rates, boost staff morale , increase staff efficiency, staff commitment and teamwork; build trust among staff; help staff acquire new skills and increase overall school effectiveness. Table 2: Management by walking about Strongly Agree Disagree Agree No. (%) No. (%) No. (%) My head frequently visits my 44(44) 13(13) 43(43) class Statement

Strongly Disagree No. (%) 0(0)

Mean

3.01

I feel intimidated when I see my head around my class

0(0)

31(31)

41(41)

28(28)

2.03

I feel relaxed when my head comes to my class to visit me

30(30)

42(42)

0(0)

28(28)

3.02

I am able to share problems with my head more when he/she comes to visit me in my class

27(27)

30(30)

43(43)

0(0)

2.84

My head is often curious to know how some topics are taught when he/she visits my class.

0(0)

30(30)

70(70)

0(0)

2.30

My head often uses informal visits to discuss formal issues with me

0(0)

44(44)

56(56)

0(0)

2.88

I am able to share personal issues during informal discussions with my head

45(45)

26(26)

29(29)

0(0)

3.16

My headâ&#x20AC;&#x;s informal visits approach reduces the cumbersome bureaucratic procedures. Total

24(24)

41(41)

29(29)

6(6)

2.83

2.75

Out of the 100 teachers employed in this study only 44(44%) indicated that their heads frequently visit them in their class. Again, on the issue of whether the head is often curious to know how some topics are taught when they visit their classes, 70(70%) respondents disagreed. However, on the issue of whether the heads informal visits reduces the cumbersome bureaucratic procedures, majority, 65 (65%) of the respondents agreed. The result in Table 2 indicates that heads were not so much involved in management by walking about. Management by walking about involves the manager visiting work sites frequently, discussing issues with subordinates. According to Hinners (2009), visits by the manager to different departments can help him/her discover and

fix problems earlier in efficient manner. According to Hinners (2009) is of the view that communication in MBWA thus becomes spontaneous and symbiotic and also gives room for on-the-job mentoring. This means that if head teachers frequently visits their teachers in their classroom they will be able identify problems and help to give coaching when it becomes necessary. From the results, it is clear that study examined the extent to which head teachers management style (MBO and MBWA) influenced teachers‟ motivation to perform and whether head teachers‟ management style had any significant effect on teacher motivation to perform. Table 3: Correlation between head teachers’ management style and teacher motivation to perform Variables Management style of head teachers

N 100

Mean 15.92

SD 3.01

Teacher motivation to perform

100

21.77

4.97

R .263

P- value 0.008

*P<0.05 This test was conducted to determine the relationship between head teachers management style and teacher motivation to perform. Testing was done at 0.05 level of significance (95% confidence level). The hypothesis was a non-directional one (2-tailed). From Table 3, management style of head teachers, r (100) = .263, P = .008, given that ∞ = 0.05 and P = 0.23. Since P < ∝, the result is statistically significant. The null hypothesis is therefore rejected and a conclusion made that, there is s significant relationship between management style of head teachers and teacher motivation to perform. This outcome confirms the suggestion by Liontos (1993) that the management style of head teachers has the ability to improve the quality of decisions; increase the decision‟s acceptance rate and provide avenues for learning new skills.; inspire staff, instill commitment and teamwork; build trust among staff; and increase overall school effectiveness. Head teachers Five head teachers were involved in this study. On the issue of whether the heads bring specific goals to staff meeting for discussion, all the five heads responded in the affirmative. Again, on the issue of whether they hold teachers responsible for the accomplishment of goal, 3 out of the 5 respondents indicated that they do hold the teachers responsible because in their view if one is involved in the decision making process then that person should be held responsible for its accomplishment. When asked whether they involve their teachers in the planning of structural facilities, and designing of new disciplinary policies, 2 indicated that those areas are managerial in nature and has got nothing to do with teachers, 2 also indicated that disciplinary policies are mostly designed by the central government and not at the school level. © 2014 The authors and IJLTER.ORG. All rights reserved

On the issue of whether the heads frequently visits their heads, 3 indicated that they sometimes visit their teachers in their classes. The 2 indicated that they had more office work to perform and this makes it difficult for them to be visiting their teachers when they are teaching.

Conclusions There is no doubt that the management style adopted by head teachers either motivates teachers to perform or demotivate them in the discharge of their duties. One cannot imagine how chaotic society would have been if there were no leaders to manage affairs of organisations. People would have been without mission and direction and development would not have been thought of. In the school setting, head teachers are seen as managers and their management style can greatly influence the performance of their teachers. The two management styles: management by objective and management by walking about are often adopted by some heads educational institutions even though these heads are not in a position to christen them as this study has done through review of literature. Head teachers do not use management by walking about as a style in the discharge of their duties. The question now is: Are the head teachers even aware there is a style of that nature and its effective implementation could yield positive outcome in the school setting? The conclusions drawn from this study indicated that management by objective as a management style was adopted by the headmasters. Irrespective of the form the MBO takes within the school set up, it is basically a style that helps to direct head teachersâ&#x20AC;&#x; attention toward results and force teachers of a school to commit themselves to achieving specific goals and facilitating their thinking of future needs and the objectives to set. Again, the MBO approach can provide the head teachers with greater measures of the tools they need to make in the best interest of the schoolâ&#x20AC;&#x;s progress. The head teacher, as a manager can gain maximum cooperation and desire to contribute from subordinates (teachers) by making them to feel that the objectives they are working toward were not taking by some people and just handed to them but are really part because they played a part in setting them, and also giving teachers a sense of belonging in the school setting by making clear how their objectives fit into the overall goal of the school. And, the approach helps to inject life into the school that comes with the energy produced as stakeholders strive to achieve its goals. Finally, it helps heads in the school setting gain better control and coordination toward goal attainment by having a clearer understanding of who is doing what and how the parts all fit together. It also helps when the heads have teachers who are more likely to control and coordinate their own activities because they know what will help and what will hinder their goal achievement. MBO easily is often misused in that what is supposed to be a system that allows for dialogue and growth between head teachers and teachers with a view to achieving results often reduces into a school system in which the heads sometimes put too much pressure on the teachers to produce results forgetting

that school success comes as a result of a lot of factors. Sometimes, heads with good intentions are prone to misuse MBO due to their lack of human skills or knowledge. Finally, many heads fail to see that MBO is one out of thousand management styles that could be adopted in solving management problems in our schools. Heads of SHS must in their attempt to use this approach guard against some of the pitfalls that have created rather ravaging effects in the use of MBO. This explains why this study advocates for a blend of the styles in order to tap their full positive effects in our schools.

Recommendations 1. It is recommended that heads of schools be given in-service training to educate them on how to blend the use of the two management styles to motivate their teachers to perform. In this regard, the situational leadership style based on the interaction among the dimensions of relationship behavior and task behavior, as well as follower readiness or maturity for performing a certain task is strongly recommended during such in-service training sessions. Potential managers of schools must be made aware that their teachers are the most critical factor in ensuring leadership success in their schools. 2. Again, heads of the various schools should be able to delegate some of their duties to other assistant heads and other teachers to give them greater opportunities to walk around the schools to see for themselves what really goes on in the classrooms. Heads must first see themselves as â&#x20AC;&#x17E;first among their equalsâ&#x20AC;&#x; and fully give teachers the respect they deserve. By definition, delegation is the transfer of authority to make decisions and complete specific tasks. Learning how to delegate is one of the most important skills for managers and leaders to possess. Strong delegation techniques can help managers save time, motivate people, and train people, as well as these techniques can enable managers to take on new opportunities. However, the lack of delegation practices often leaves people frustrated, unmotivated, and under-trained, while the manager remains overworked. Delegation is a skill that enables managers to achieve more without burning themselves out. This is one single surest way of motivating or driving a person to do something. Much of the driven are the thought of a potential reward, or a consequence of not doing something. 3. The main concepts in delegation such as authority, responsibility, and accountability for the task changes should be hammered when a head decides to delegate a task. Before delegating a task, it is important to understand how it affects these three concepts. Authority is the power given to a person or group of people to act and make decisions within designated boundaries. When delegating a task, the authority is shared between the head teacher and the teacher receiving the delegation. Also, responsibility refers to the act of carrying out the task. When delegating a task, the head teacher and the teacher receiving the delegation share the responsibility of completing the work. The head teacher has the responsibility of providing instructions on what work needs to be done,

while the teacher receiving the delegation is responsible for figuring out how the task should be completed. Accountability is the act of being liable for a person‟s actions and decisions. During delegation of a task, the accountability of the task transfers from the head teacher to the teacher receiving the delegation and actually completing the work. Any positive or negative consequences associated with the teacher‟s performance are ultimately the responsibility of the head teacher. Understanding these basic concepts would equip heads with some skills in delegating tasks in the school effectively. 4. Efforts should be made to continuously involve the teachers in the decision making process in the school to boost their motivation to perform. Heads must consciously tell their teachers after making the decision and announce it to the staff with a clear direction. Once that has been ensured, the head should attempt to gain commitment of the staff by "selling" the positive aspects of the decision. The heads should then invite input into the decision while retaining authority to make the final decision themselves. Finally, invite teachers to make inputs into the decision with. In the process, the head should consider themselves as having a voice equal to those of the subordinates.

References Aacha, M. (2010). Motivation and the performance of primary school teachers in Uganda: A case of Kimaanya-Kyabakuza division, Makasa district. Unpublished dissertation, Makerere university, Kampala. Alderfer, C. P. (1969). An empirical test of a new theory of human needs. Organisational Behavior and Human Performance, 4, 143-175. Almansour, Y. M. (2012). The relationship between leadership styles and motivation of managers conceptual framework. International Refereed Research Journal, 3(1), 1 – 6. Al-Samarrai, S. & Bennell, P. (2003). Where Has All The Education Gone On Africa? Employment Outcomes Among Secondary School And University Leavers. Brighton: IDS/KSD. Analoui, F (2000). What motivates senior managers? The case of Romania. Journal of Managerial Psychology, 15(4), 324-340. Armstrong, M. (2004). How to be an even better manager. London: Kogan Page Press. Bennell, P (2004). Teacher motivation and incentives in Sub-Saharan Africa and Asia. Brighton: Knowledge and Skills for Development. Brown, M. (2012). Motivational factors as determinant of performance among business Students in the SHSs in the Cape Coast metropolis, Unpublished master‟s thesis, University of Cape Coast, Cape Coast. Conley, S. C., Schmidle, T., & Shedd, J. B. (1988). Teacher participation in the Management of school systems. Teachers College Record, 90, 259-280. Drucker, P. (1954). The practice of management. New York: Harper & Row. Ekeland, T., P. (2005). The relationship among affective organisational commitment, transformational leadership style, and unit organizational effectiveness within the corps of cadets at Texas A & M University. Unpublished Doctorate Dissertation, Texas A & M University, Texas. Fry, L. & Matherly, L. (2006). Workplace spirituality, spiritual leadership, and performance excellence. In S. Roglberg & C. Reeve (Eds.), The Encyclopedia of

Industrial and Organisational Psychology. San Francisco: Sage Publishing. Gay, L. R. (1992). Educational research. New York, NY: Maxwell Macmillan International. Hamner, W. C., & Hamner, E. P. (1976). Behavior modification and the bottom line. Organizational Dynamics, 4(4), 3-21. Hertzberg, F. (1966). Work and the nature of man. Cleveland: World Publishing. Hornby, A., S. (2000). Oxford Advanced Learner„s Dictionary of Current English. Oxford University Press. Hinners, N. W. (2009). Management by wandering around. NASA‟s Mars Exploration Programme Analysis Group, University of Colorado Aerospace Engineering Sciences Department. Retrieved November 16, 2013 from http://www.pmhut.com/ Holton, V., Dent, F., & Rabbetts, J. (2009). Motivation and employee engagement in the 21st century: A survey of management views. An Ashridge Report. Humble, J. (1968). Improving business results. New York: McGraw-Hill. Humble, J. (1970). Management by objectives in action. New York: McGraw-Hill. Komaki, J. (1982). Managerial effectiveness: Potential contributions of the behavioral approach. Journal of Organizational Behavior Management, 3, 71-83. Liontos, L. B. (1993). Shared decision-making. Eugene, OR: Oregon School Study Council, University of Oregon, OSSC Bulletin Series. Maslow, A. H. (1943). A theory of human motivation. Psychological Review, 50, 370-396. McGregor, D. (1966). Leadership and motivation. Cambridge, MA: M.I.T. Press. Mears, M. (2009). Leadership Elements: A guide to building trust , p.51, Google Books link: BooksG-TOC-51 Mertler, H. (1992), Value make the Company. Harvard Business Review. Odiorne, G. (1970). Management by objectives. New York: Pitman. Peters, T. & Waterman, R. (1982). In Search of Excellence. New York: Random House. Reddin, W.J. (1971). Managerial effectiveness. New York: McGraw-Hill. Ryan, R. M., & Deci, E. L. (2000). When rewards compete with nature: the undermining of intrinsic motivation and self-regulation. In C. Sansone & J. M. Harackiewicz (Eds.), Intrinsic and extrinsic motivation: The search for optimal motivation and performance (pp. 14-54). San Francisco: Academic Press. Sansone, C., & Harackiewicz, J. M. (2000). Intrinsic and extrinsic motivation: The search for optimal motivation and performance. San Diego: Academic Press. Thorndike, E. L. (1911). Animal intelligence. New York: Macmillan. White, R. (1959). Motivation reconsidered: The concept of competence. Psychological Review, 66, 297-333.

61 International Journal of Learning, Teaching and Educational Research Vol. 4, No.1, pp. 61-74, April 2014

Comparison and Properties of Correlational and Agreement Methods for Determining Whether or Not to Report Subtest Scores Oksana Babenko, PhD W. Todd Rogers, PhD University of Alberta Edmonton, Canada

Abstract. Large-scale testing agencies often report subtest scores in addition to reporting the total test score. But is there evidence that subtests reveal differences in student performances? Three methods for determining whether subscore reporting is warranted were examined and evaluated using large-scale data as well as samples of various sizes for Reading and Mathematics assessments. Results revealed that subtests did not differ among themselves and added no value over the total test. The method statistics were determined to be accurate and precise estimators of the population parameters. Implications for subscore reporting are discussed. Keywords: subscore reporting; accuracy; precision; large-scale assessment

Introduction Results from a large-scale achievement test can be reported in the form of the total test score and, if justified, as a series of subtest scores together with the total test score. The common practice is to report the total score as a summary of achievement of the total domain tested. However, large-scale testing agencies have increasingly adopted the practice of reporting subtest scores in addition to reporting the total test score because of the potential diagnostic value of subtest scores (Wainer, Sheehan, & Wang, 2000; Tate, 2004; Sinharay, Haberman, & Puhan, 2007; Yao & Boughton, 2007; Sinharay, Puhan, & Haberman, 2009; Sinharay, 2010). Part of the argument put forward to support subscore reporting is based on the fact that the items comprising a test are referenced to a curriculum that is multidimensional in nature, with each dimension characterized by specific content and/or cognitive skills. For example, items on a Mathematics achievement test can be referenced to (a) content areas, such as number sense and numeration, measurement, geometry and spatial sense, patterning and algebra, and data management and probability, and/or (b) cognitive skills, such as knowledge and understanding, application, and problem solving. In test development, the table of specifications serves to ensure that the test reflects the multidimensionality of the curriculum. However, what needs to be recognized is that there must be evidence that the variables or skills ÂŠ 2014 The author and IJLTER.ORG. All rights reserved.

62 measured by the subtests are indeed sufficiently distinct to warrant reporting scores from the subtests. Additionally, while the number of test items typically reflects the proportional weighting given to each cell within the table of specifications, the total number of items included in the test is limited by the amount of available test administration time. Consequently, more often than not the number of items for each dimension in the table of specifications is not sufficient to achieve a high degree of reliability or a low error of measurement. Despite these cautions, officials at large-scale assessment agencies still want to report subtest scores even though no deliberate attempt was made to ensure that (a) the variables (e.g., number sense and numeration) assessed by subtests are distinct and not highly related, and (b) there is a sufficient number of items for each subtest to ensure high reliability. The evidence that is usually used to determine if the variables are sufficiently distinct is the correlation among the subtest scores whereas the internal consistency of the items in each subtest provides evidence of subscore reliability. What is desired are low subtest correlations and high subtest internal consistencies (American Educational Research Association, American Psychological Association, & National Council on Measurement and Evaluation, 1999; Wainer, Vevea, Camacho, Reeve, et al., 2001; Tate, 2004). Various methods for determining whether subtest scores are distinct and/or add value over and above the total test score have been developed. These methods include the agreement method (Kelley, 1923; see also Gulliksen, 1951; Lord & Novick, 1968; Ryan, 2003; Haladyna & Kramer, 2004), correlations corrected for attenuation due to unreliability of the measures (McPeek, Altman, Wallmark, & Wingersky, 1976; Harris & Hanson, 1991; Haladyna & Kramer, 2004), factor analytic method (McPeek et al., 1976; Grandy, 1992), statistical model fit (Harris & Hanson, 1991), and, in the case of determining only whether a subtest has value over the total test, the proportional reduction of the mean squared error (Haberman, 2005, 2008; Sinharay, Haberman, & Puhan, 2007). Three of these methods were considered in the present study: Kelley’s agreement method (KR; Kelley, 1923), correlations corrected for attenuation ( c ˆ jk ; McPeek, et al., 1976), and the proportional reduction of the mean squared error (PRMSE; Haberman, 2005; Sinharay, Haberman, & Puhan, 2007). The agreement method takes into account the actual differences between observed scores on subtests j and k expressed in the same score metric (Kelley, 1923; Gulliksen, 1951; Lord & Novick, 1968; Ryan, 2003; Haladyna & Kramer, 2004). Working with z-scores (   0 ;   1 ) or scores in some other standardized metric to remove the effects of different means and standard deviations of subtests j and k, the difference, d i , between two standard scores for student i is given by: di  zij  zik , where

zij

is the observed standard score of student i on subtest j, and

zik is the observed

standard score of student i on subtest k. If the estimated standard error of the difference for a given student, 2   jj   kk , where  jj and  kk are the reliabilities of subtests j and k, and the © 2014 The author and IJLTER.ORG. All rights reserved.

63 estimated standard deviation of the obtained differences for the group of students,

2  2 jk ,

where  jk is the correlation between the scores on subtests j and k, are close in value, then the obtained differences are no greater than what would be expected by chance (Kelley, 1923, p. 329). In order to determine directly the percentages of students with differences beyond what would be expected by chance, Kelley computed the proportion of cases in excess of the chance as a function of the ratio:

KR 

2   jj   kk

(Kelley, 1923, p. 330).

2  2  jk

Kelley (1923) illustrated his agreement method with the eight subtests of the Stanford Achievement Test Battery and found 10% to 44% of the students had differences beyond chance for 36 pairs of subtests (p. 331). Values of KR closer to one led to small proportions of students with differences beyond chance; values of KR further from one (i.e., closer to zero) led to larger proportions of students with differences beyond chance. The correlation corrected for attenuation due to unreliability, c  jk , is given by: c

 jk

 jk 

 jj kk

where  jk is the uncorrected correlation between the scores on subtests j and k, and  jj and  kk are the internal consistency estimates (Cronbach, 1951) for subtests j and k, respectively. If c  jk is less than 0.90, then it is concluded that student performances on subtests j and k differ and that reporting of subtest scores is warranted (McPeek et al., 1976; Haladyna & Kramer, 2004). For example, Haladyna and Kramer (2004) used the c  jk method to determine whether subtest scores on a basic biomedical science test revealed any differences in examinees’ performances. They found that the corrected correlations were higher than 0.90, suggesting a high degree of similarity in examinees’ performances on the subtests of the test. The proportional reduction of the mean squared error method involves predicting the true scores on subtest j from the observed scores on subtest j and from the total test score:  ij   j   jj (sij   j ) (1) and

 iX   j  s x t



X

( xi   X ) ,

(2)

where  ij and  iX are, respectively, the true score for student i on subtest j when predicted from the observed subtest score and the true score of student i on subtest j when predicted from the total test X score for student i;  j and  X are the means of subtest j and the total test X;

sij and xi are the observed scores on subtest j and the total test X, respectively, for student i;

 X is the standard deviation of the total test scores, and    j  jj , where  j is the standard j

deviation of the scores on subtest j; and s x  s2 x  XX , where t

 s2 x is computed as outlined in

Haberman (2005). The corresponding mean squared errors (MSE) are given by:

and

, where

σ τ2j is the true score variance for subtest j. The MSE when τ j is simply predicted from

s j is the subtest score error variance,  e2   2j (1   jj ) . j

The proportional reduction of the mean squared errors when the true score is predicted from a subtest score using equation (1) is given for each subtest by:

σe2 - MSEτ / s PRMSE = σe2 τ/s

The PRMSE when the true score is predicted from the total test score is computed in the same way but using the MSEτ / x as the base. If PRMSEτ / s > PRMSEτ/x , then reporting the scores for subtest j adds value over reporting only the total test scores (Haberman, 2005, 2008; Sinharay et al., 2007, 2009; Lyren, 2009; Sinharay, 2010). Haberman (2008) used the PRMSE method to determine whether or not the subtest scores on SAT I “had added value over and above the value of the total score” and found that “none of the section scores of SAT I math or SAT I verbal provide any appreciable information concerning an examinee that is not already provided by the math or verbal total score” (p. 221). Using the PRMSE method, Sinhary (2010) examined 25 operational tests to see if the subtests within each test had added value over the full test. He found that 16 of the 25 tests had no subtest scores with added value even though subtest scores were reported in many cases. Of the remaining nine tests, some but not all of the subtests had added value. However, it should be noted that in contrast to correlations corrected for attenuation, the PRMSE does not compare subtest scores to determine if they are distinct from one another. In contrast to correlations corrected for attenuation and proportional reduction of the mean squared error methods, which do not specifically look at the agreement between two observed scores obtained from two subtests, the agreement method takes into account the actual differences between observed scores on subtests j and k expressed in the same score metric. In the case of the c  jk method, if differences among the subtests are revealed, then the agreement method will need to be used to determine which students have pairs of scores that differ. In the case of the PRMSE method, if a subtest is found to have value over the total test, then the agreement method will need to be used to determine which students have subtest scores that differ from the total test. Thus, it seems reasonable to use the Kelley’s agreement method alone. © 2014 The author and IJLTER.ORG. All rights reserved.

65 Hence, one purpose of the present study was to determine whether Kelley’s agreement and correlations corrected for attenuation methods would lead to the same or different decision regarding the identification of pairs of distinct subtests and whether Kelley’s agreement and proportional reduction of the mean squared error methods would lead to the same or different decision about subtests having added value over the total test. If the decisions were the same in both, then the agreement method could simply be used. A second purpose of the study was to examine the accuracy and precision of the statistics used in the KR, c  jk , and PRMSE methods. No studies were found in the published literature that comparatively examined accuracy and precision of the statistics used in these methods. If one method produced biased or imprecise estimates, then different decisions could be made when using samples rather than the population. However, if the method produced unbiased and precise estimates, then the decisions made would not be due to bias or impreciseness.

Method The two data sets used in the study were population data sets for the Junior (Grade 6) Englishlanguage Reading and Mathematics assessments conducted by the Education Quality and Accountability Office (EQAO) in Ontario, Canada (www.eqao.com). EQAO conducts annual province-wide assessments in both of Canada’s official languages (English and French) at the Primary (Grade 3) and Junior (Grade 6) levels in the areas of Reading, Writing, and Mathematics, at the Grade 9 level in Academic Mathematics and Applied Mathematics, and at the Grade 10 level in Literacy (Reading and Writing). Results are reported at the provincial, district, school, and student levels and are publically available on the EQAO website, with emphasis on progress from the previous year. EQAO requested that the present study be conducted to determine if reporting subtest scores was justified given no explicit attempt was made to develop subtests with psychometric characteristics that allowed subtest score reporting.

Description of the Reading Test The English-language Reading test items are referenced to three knowledge and skills categories as specified in the curriculum for the province: Explicit Information, Implicit Information, and Connections. The items in the Explicit Information subtest require students to detect and understand information and ideas stated explicitly in a variety of text types identified in the provincial curriculum. The items in the Implicit Information subtest probe students’ understanding of implicitly stated information and ideas. The items in the Connections subtest require students to demonstrate their understanding of text passages by connecting, comparing, and contrasting the ideas presented in the passages and drawing upon their own knowledge, experience and insights, other texts, and the world around them. Thus, the three subtests can be ordered in terms of complexity, with the Explicit Information and Connections subtests at the lowest and the highest levels of complexity, respectively. The Explicit Information subtest contains six multiple-choice items, the Implicit Information subtest contains 14 multiple-choice items and four open-response items, and the Connections subtest © 2014 The author and IJLTER.ORG. All rights reserved.

66 contains six multiple-choice items and six open-response items. The 10 open-response items are scored using four-point scoring rubrics.

Description of the Mathematics Test In contrast to the Reading assessment, the items on the Mathematics assessment are referenced by content areas (i.e., strands) and by cognitive skills as specified in the mathematics curriculum. The five content areas include: Number Sense and Numeration (8 items involving estimation, rate, ratio, and use of fractions), Measurement (8 items involving the use of area relationships, understanding of the dimensions of the shapes needed to calculate their areas, and the conversion of metric area units), Geometry and Spatial Sense (6 items dealing with the identification, performance and description of transformations, the identification of angles, and accurate use of rulers and protractors), Patterning and Algebra (7 items dealing with growing patterns, use of diagrams, tables and number sequences to represent the stages of patterns), and Data Management and Probability (7 items involving concepts of probability, predicting and representing the probability of an outcome, comparing probabilities using common representations (e.g., common denominators, percents or decimals), and interpreting graphs). The five content areas are not ordered in terms of complexity. Cognitive skills are divided into three categories: Knowledge and Understanding (8 items), Application (15 items), and Problem Solving (13 items). The items referenced to the Knowledge and Understanding category require students to demonstrate subject specific content (knowledge) and the comprehension of its meaning and significance (understanding). The Application items require students to select and fit an appropriate mathematical tool or get the necessary information. The Problem Solving items require students to select and sequence a variety of tools to solve a problem and demonstrate a critical-thinking process. That is, to answer Problem Solving items, students need to make a plan. In contrast to the content subtests, the cognitive subtests can be ordered in terms of complexity, with the Knowledge and Understanding subtest and the Problem Solving subtest being at the lowest and the highest levels of complexity, respectively. The total number of items on the Mathematics assessment is 36, including 8 open-response items scored using a four-point scoring rubric and distributed such that each content subtest has at least one open-response item.

Analyses The analyses were conducted in two main stages corresponding to the two purposes of the study. First, the responses of the population of students were analysed to obtain the population value of each test statistic for each of the three detection methods. Following this, the analyses were repeated for 1,000 replicated independent samples of five different sizes – 250, 500, 1,000, 2,000, and 5,000 – randomly drawn from the population with replacement to (1) determine the effect of sample size on the accuracy and precision of the estimators, and then to (2) assess the consistency of the decisions made using the three detection methods in light of the findings about accuracy and precision. At the first stage, means and standard deviations of the distributions of sample statistics were used to evaluate the three detection methods with respect to their accuracy and precision. At the second stage, the KR and c  jk methods were applied, first, at the population level to determine if the subtests were distinct, and then applied to each © 2014 The author and IJLTER.ORG. All rights reserved.

67 of 1,000 replicated samples for each of the five sample sizes to see if the same decision was made. The PRMSE method was applied at the population level and then for each replicated sample to see if the subtests added value over the total test. The consistency of the decisions made was assessed using the percentage of samples that led to the same decision that was made at the population level.

Results and Discussion Psychometric Properties of the Reading and Mathematics Tests The psychometric properties of the Junior Reading and Mathematics subtests and the total tests are provided in Table 1 for the population of students. The means and standard deviations are reported in the observed score units and as percentages (in parentheses). Table 1. Means, Standard Deviations, and Internal Consistencies for Reading and Mathematics Tests Subtest/Total Test

k/msa

6/6 18/30 12/30 36/60

4.59 (76.5) 20.92 (69.7) 17.16 (57.2) 42.68 (64.7)

1.28 (21.3) 4.42 (14.7) 4.33 (14.4) 8.97 (13.6)

Content Area Numeration (N) Measurement (M) Algebra (A) Probability (P) Geometry (G) Total Test (TT)

8/14 8/11 7/10 7/13 6/12 36/60

8.44 (60.3) 6.34 (57.6) 6.63 (66.3) 7.20 (55.4) 7.18 (59.8) 35.79 (59.7)

3.02 (21.6) 2.67 (24.3) 2.14 (21.4) 2.71 (20.8) 2.80 (23.3) 11.20 (18.7)

Cognitive Skill Know/Understand (K/U) Application (A) Problem Solving (PS) Total Test (TT)

8/8 15/24 13/28 36/60

5.45 (68.0) 14.98 (62.4) 15.36 (54.8) 35.79 (59.7)

1.87 (23.4) 4.91 (20.5) 5.42 (19.4) 11.20 (18.7)

Reading, N = 128,089 Explicit Information (EI) Implicit Information (II) Connections (C) Total Test (TT)

Correlations

sX EI 0.47b

II 0.59 0.76

C 0.52 0.74 0.74

TT 0.69 0.93 0.92 0.87

N 0.63

M 0.66 0.63

A 0.62 0.59 0.58

P 0.67 0.63 0.62 0.61

K/U 0.60

A 0.70 0.75

PS 0.67 0.79 0.78

TT 0.80 0.94 0.94 0.89

Mathematics, N = 127,596 G 0.63 0.63 0.59 0.62 0.60

TT 0.87 0.84 0.80 0.85 0.83 0.89

is number of items in a subscale or the total test and ms is the maximum score given the use of dichotomously scored multiple-choice items and polytomously scored open-response items. b Internal consistencies of the subtests and the total test are shown in italics along the principal diagonal of each correlation panel.

Reading. The mean percentages revealed that studentsâ&#x20AC;&#x2122; performance declined on the original three subtests as the complexity of the constructs increased. The standard deviations (percentages) were essentially the same for the Implicit Information and Connections subtests, which are at the two higher levels of complexity, but smaller than for the Explicit Information subtest, likely because of the smaller number of items and, therefore, total points for this subtest. As shown along the main diagonal of the correlation matrix on the right side of Table 1, ÂŠ 2014 The author and IJLTER.ORG. All rights reserved.

68 the internal consistency (alpha; Cronbach, 1951) of the Explicit Information subtest was much lower than the internal consistencies of the Implicit Information and Connections subtests, which were essentially the same. The low reliability of the Explicit Information is due to the relatively small number of items (6) in this subtest in comparison to the other subtests (18 and 12, respectively). The estimate of the internal consistency of the total test was 0.87, reflecting the typical practice mentioned above of ensuring that the total test reliability is at an acceptable high level. The values of the correlations were either greater than the corresponding reliabilities or close in value, which suggests that the three procedures examined in this study will show that the subtests are not distinct and the subtests do not add value over and above the total test. Mathematics content area. The mean percentages revealed that the mean for the Algebra subtest was the highest, the mean on the Probability subtest was the lowest, and the means of the other three subtests were between and essentially the same. The standard deviations were somewhat larger for the Measurement and Geometry subtests than the standard deviations for the Numeration, Algebra, and Probability subtests, which were essentially the same. Given the numbers of items in each subtest did not differ much as they did in the case of Reading and Mathematics cognitive skills, the internal consistencies of the five content subtests were essentially the same, ranging from 0.58 to 0.63. However, as with Reading, the values of the correlations were close to the values of the reliabilities, suggesting again that the three procedures examined in this study will show that the subtests are not distinct and the subtests do not add value over and above the total test. Further, some of the values of KR and c  jk will exceed 1.00, which theoretically should not happen. Mathematics cognitive skills. Similar to Reading, the students’ performance on the three mathematics cognitive subtests declined as the level of required thinking increased from knowledge and understanding to application to problem solving. The standard deviations were essentially the same for the Application and Problem Solving subtests, which are of higher complexity, but smaller than the standard deviation for the knowledge and understanding subtest, again likely because of the smaller number of items in the latter subtest. The internal consistency of the knowledge and understanding subtest, 0.60, was lower than the internal consistencies of the Application and Problem Solving subtests, which were more alike, 0.75 and 0.78, respectively. The somewhat low value of reliability for the knowledge and understanding subtest was likely due to the relatively smaller number of items (8) in this subtest as compared to the numbers of items in the other two subtests (15 and 13, respectively). The estimate of the internal consistency of the total test was 0.89, again reflecting the typical practice mentioned above of ensuring that the total test reliability is at an acceptable high level. Again, we see as for the Reading and Mathematics content areas that the values of the correlations are close to the values of the reliabilities, suggesting that the three procedures examined in this study will show that the subtests are not distinct and the subtests do not add value over and above the total test, with some of the values of KR and c  jk will exceed 1.00.

Accuracy and Precision of the Estimators The means and standard deviations of the 1,000 replications for each sample size are reported in Table 2 for the KR method, Table 3 for the c  jk method, and Table 4 for the PRMSE method. As shown in Table 2, all but two of the means of the sampling distributions of 1,000 replications across the pairs and sample sizes were within 0.01 of the corresponding population value of KR for each subtest pair and sample size (the difference is 0.02 for the Numeration and Probability subtest pair, with n = 250 and n = 500). The standard error of KR decreased as the sample size increased. For example, for n = 250 the standard errors were between 0.043 and 0.049, whereas for n = 5,000 the standard errors were between 0.009 and 0.011. Table 2. Accuracy and Precision: Kelley’s Ratio (KR) Sample Size 250 500 1,000 2,000

Subtest Pairs Reading Exp Info–Imp Info 0.97 (0.049)a Exp Info–Con 0.92 (0.046) Imp Info–Con 0.96 (0.043) Mathematics Content Area Num–Mea 1.04 (0.051) Num–Alg 1.03 (0.050) Num–Prob 1.09 (0.050) Num–Geo 1.03 (0.049) Mea–Alg 0.98 (0.045) Mea–Prob 1.02 (0.046) Mea–Geo 1.02 (0.052) Alg–Prob 1.03 (0.049) Alg–Geo 1.00 (0.047) Prob–Geo 1.03 (0.048) Mathematics Cognitive Skill Kno/Und–App 1.04 (0.052) Kno/Und–Prob Sol 0.98 (0.049) App–Prob Sol 1.05 (0.048) a

5,000

Population

0.97 (0.035) 0.91 (0.032) 0.95 (0.032)

0.97 (0.024) 0.91 (0.022) 0.95 (0.021)

0.97 (0.017) 0.91 (0.015) 0.95 (0.015)

0.97 (0.011) 0.91 (0.010) 0.95 (0.009)

0.97 0.91 0.96

1.04 (0.035) 1.03 (0.034) 1.09 (0.037) 1.03 (0.037) 0.98 (0.033) 1.02 (0.032) 1.03 (0.035) 1.03 (0.033) 1.00 (0.033) 1.03 (0.034)

1.04 (0.025) 1.03 (0.025) 1.08 (0.026) 1.03 (0.025) 0.98 (0.023) 1.02 (0.024) 1.02 (0.026) 1.03 (0.024) 1.00 (0.023) 1.03 (0.023)

1.04 (0.017) 1.03 (0.016) 1.08 (0.026) 1.02 (0.017) 0.98 (0.016) 1.02 (0.017) 1.02 (0.017) 1.03 (0.017) 1.00 (0.017) 1.03 (0.017)

1.04 (0.011) 1.03 (0.011) 1.08 (0.011) 1.02 (0.011) 0.98 (0.010) 1.02 (0.010) 1.02 (0.011) 1.03 (0.010) 1.00 (0.010) 1.03 (0.011)

1.04 1.02 1.07 1.02 0.98 1.01 1.02 1.03 1.00 1.02

1.04 (0.035) 0.98 (0.031) 1.05 (0.035)

1.04 (0.026) 0.98 (0.024) 1.05 (0.024)

1.03 (0.017) 0.97 (0.018) 1.05 (0.018)

1.04 (0.011) 0.97 (0.011) 1.05 (0.011)

1.04 0.97 1.05

The first value is the mean and the value in parentheses is the standard deviation of the sampling distribution (i.e., standard error) of 1,000 replications.

The means of the sampling distributions of population values of

ˆ jk were within 0.01 of the corresponding

 jk for all the pairs of subtests and sample sizes (see Table 3). The

standard errors of sample estimators decreased as the sample size increased. For n = 250, the standard errors ranged between 0.029 and 0.081, whereas for n = 5,000, the standard errors were as low as 0.007 and as high as 0.017. Given the low reliability of the Explicit Information subtest in the Reading assessment, the standard errors for the pairs involving this subtest were consistently higher than the standard errors for the remaining pairs of subtests.

70 Similar to KR and c ˆ jk , the means of the distributions of sample estimators of PRMSEτ / s and

PRMSEτ/x were within 0.01 of the corresponding population values for all four subtests (Table 4). The standard errors of sample estimators were the largest when the Explicit Information subtest was considered (e.g., the standard error PRMSEE  0.100 for n = 250) but decreased as the sample size increased, ranging between 0.003 and 0.020 for n = 5,000. Taken together, the results provided in Tables 2, 3, and 4 reveal that sample estimates of KR, c  jk , and PRMSEτ / s and PRMSEτ/x are accurate and precise. Therefore, any differences among the three detection methods used for the detection of subtest differences or subtest-total test differences are not confounded by presence of biased or imprecise estimators. Table 3. Accuracy and Precision: Correlation Corrected for Attenuation ( c  jk ) Subtest Pairs 250 Reading Exp–Imp Info 0.99 (0.078)a Exp Info–Con 0.90 (0.081) Imp Info–Con 0.97 (0.029) Mathematics Content Area Num–Mea 1.05 (0.054) Num–Alg 1.04 (0.062) Num–Prob 1.09 (0.051) Num–Geo 1.03 (0.058) Mea–Alg 0.98 (0.062) Mea–Prob 1.02 (0.055) Mea–Geo 1.02 (0.062) Alg–Prob 1.04 (0.064) Alg–Geo 0.99 (0.066) Prob–Geo 1.04 (0.059) Mathematics Cognitive Skill Kno/Und–App 1.04 (0.047) Kno/Und–Prob Sol 0.99 (0.048) App–Prob Sol 1.03 (0.026)

500

Sample size 1,000

2,000

5,000

Population

0.99 (0.051) 0.89 (0.056) 0.97 (0.021)

0.99 (0.037) 0.89 (0.039) 0.97 (0.015)

0.98 (0.026) 0.89 (0.017) 0.97 (0.011)

0.98(0.016) 0.89 (0.017) 0.97 (0.007)

0.98 0.89 0.97

1.05 (0.037) 1.04 (0.042) 1.09 (0.038) 1.03 (0.040) 0.97 (0.045) 1.02 (0.038) 1.03 (0.041) 1.04 (0.043) 1.00 (0.046) 1.03 (0.042)

1.05 (0.026) 1.04 (0.031) 1.09 (0.027) 1.03 (0.029) 0.97 (0.032) 1.02 (0.028) 1.03 (0.031) 1.04 (0.030) 0.99 (0.033) 1.03 (0.029)

1.05 (0.018) 1.03 (0.020) 1.09 (0.018) 1.03 (0.021) 0.97 (0.022) 1.02 (0.020) 1.03 (0.020) 1.04 (0.022) 0.99 (0.023) 1.03 (0.021)

1.04 (0.012) 1.03 (0.014) 1.09 (0.012) 1.03 (0.013) 0.97 (0.014) 1.02 (0.013) 1.03 (0.013) 1.04 (0.013) 0.99 (0.014) 1.03 (0.014)

1.04 1.03 1.09 1.03 0.97 1.02 1.03 1.04 0.99 1.03

1.04 (0.031) 0.99 (0.031) 1.03 (0.018)

1.04 (0.023) 0.98 (0.023) 1.03 (0.013)

1.04 (0.016) 0.98 (0.016) 1.03 (0.009)

1.04 (0.010) 0.98 (0.010) 1.03 (0.006)

1.04 0.98 1.03

The first value is the mean and the value in parentheses is the standard deviation of the sampling distribution (i.e., standard error) of 1,000 replications. a

Detection of Performance Differences and Consistency of Decisions As foreshadowed in the presentation of the psychometric properties of the subtests and total test and as revealed by the results in Tables 2, 3, and 4, the subtests were determined to be not distinct nor did the subtests add value over the total test. The values of KR were close to one with one exception (Reading, Explicit Information and Connections subtests; Table 2). Further, 11 of the 16 KR values exceeded one, which theoretically should not happen. For the agreement procedure to work, the sum of the reliabilities of the two subtests has to be greater than two times the correlation between the two subtests being compared. This was not the case with the © 2014 The author and IJLTER.ORG. All rights reserved.

71 subtests considered in the present study, with the sum of the reliabilities in the 11 cases being less than two times the corresponding correlations. The decision rule for the method of correlations corrected for attenuation is a value less than 0.90 indicates that the two subtests being correlated are sufficiently different to warrant reporting the scores on each (McPeek et al., 1976). With one possible exception (Reading, Explicit Information and Connections subtests; Table 3), the values of c ˆ jk exceeded 0.95, with 10 of the 16 values being greater than 1.00, which theoretically should not happen. The decision rule for the PRMSE method is: if PRMSEτ / s > PRMSEτ/x , then the subtest has added value over and above the total test and, therefore, the score on the subtest should be reported. As shown in Table 4, for all subtests, PRMSEτ / s < PRMSEτ/x . For both the c  jk and PRMSE methods, the reliabilities of the subtests must be high, which was not the case in the present study. In the case of Reading, with perhaps one exception, the decisions made using population values of KR and c  jk were that the subtests did not differ, and the population values of PRMSEτ / s and PRMSEτ/x indicated that the three subtests did not add value over the total test. For the Explicit Information and Connections pair of subtests, KR suggested that there was a difference beyond chance for 5% of the students, and that the value of c  jk , 0.89, was just less than 0.90. The sample data revealed that with exception of two subtest pairs, Explicit Information and Connections and Explicit Information and Implicit Information with n = 250 and n = 500, the same decision was made using sample data for at least 91% of the replications using the KR, c  jk , and PRMSE methods across the different sample sizes. In the case of the Explicit Information and Connections pair, the decision consistency for c ˆ jk varied from 51.4% to 78.8% across the five sample sizes (i.e., 514 of the 1,000 replications led to the same decision made at the population level). This finding is attributable to the low reliability of the Explicit Information subtest, 0.47, and the observation that the value of c  jk was only 0.01 below the decision value of 0.90. In the case of the Explicit and Implicit Information pairs, the decision consistency for n = 250 was 90.5% and for n = 500, 95.1%, while for n  1, 000 the decision consistencies were 99.1%, 100%, and 100%. In the case of Mathematics, KR and

 jk indicated that there were no distinct subtests and

PRMSEτ / s and PRMSEτ/x indicated that no subtest added value over the total test. Further, the majority of values for KR were greater than 1.00 due to the fact that the sum of the reliabilities was greater than two times the uncorrected correlation. Similarly, the majority of the values for ˆ c  jk were greater than 1.00 due to the fact that the square root of the product of the reliabilities was less than the uncorrected correlation between the pairs of subtests. The sample data revealed that, with three exceptions, Measurement and Algebra with n = 250 and n = 500 and Algebra and Geometry with n = 250, the same decision was made using sample data for at least 97% of replications using the KR and c  jk methods and, in the case of the subtest-total test pairs, the PRMSE method. The exceptions included the Algebra subtest, which had the lowest © 2014 The author and IJLTER.ORG. All rights reserved.

72 reliability out of the all Mathematics subtests. Again, as for Reading, the sample values of the correlations were close to the sample values of the reliabilities, and in the majority of cases the two times the correlation exceeded the sum of the reliabilities, leading to sample estimates greater than 1.00. Table 4. Accuracy and Precision: Proportional Reduction of the Mean Squared Error (PRMSE) Sample Size Population Subtest/PRMSE 250 500 1,000 2,000 5,000 Reading Exp Info

PRMSEτ / s PRMSEτ/x

0.47 (0.054)a

0.47 (0.038)

0.47 (0.027)

0.47 (0.020)

0.47 (0.012)

0.47

0.80 (0.100)

0.79 (0.065)

0.79 (0.047)

0.79 (0.033)

0.79 (0.020)

0.79

Imp Info

PRMSEτ / s PRMSEτ/x

0.76 (0.023)

0.76 (0.016)

0.77 (0.011)

0.77 (0.008)

0.77 (0.005)

0.77

0.87 (0.017)

0.87 (0.013)

0.87 (0.008)

0.87 (0.006)

0.87 (0.004)

0.87

Con

PRMSEτ / s PRMSEτ/x

0.73 (0.023)

0.73 (0.016)

0.73 (0.011)

0.74 (0.008)

0.74 (0.005)

0.74

0.85 (0.020)

0.85 (0.014)

0.85 (0.010)

0.85 (0.007)

0.85 (0.004)

0.85

Mathematics Content Area Num

Mea

Alg

Prob

Geo

PRMSEτ / s PRMSEτ/x

0.63 (0.028)

0.63 (0.019)

0.63 (0.015)

0.63 (0.010)

0.63 (0.006)

0.63

0.94 (0.037)

0.94 (0.025)

0.94 (0.019)

0.94 (0.012)

0.94 (0.008)

0.94

PRMSEτ / s PRMSEτ/x

0.63 (0.029)

0.63 (0.024)

0.63 (0.015)

0.63 (0.010)

0.63 (0.006)

0.63

0.90 (0.042)

0.90 (0.031)

0.90 (0.021)

0.89 (0.014)

0.89 (0.009)

0.89

PRMSEτ / s PRMSEτ/x

0.58 (0.038)

0.58 (0.025)

0.58 (0.019)

0.58 (0.013)

0.58 (0.008)

0.58

0.89 (0.055)

0.88 (0.039)

0.88 (0.027)

0.88 (0.019)

0.88 (0.012)

0.88

PRMSEτ / s PRMSEτ/x

0.61 (0.032)

0.61 (0.022)

0.61 (0.019)

0.61 (0.011)

0.61 (0.007)

0.61

0.94 (0.042)

0.93 (0.029)

0.93 (0.020)

0.93 (0.015)

0.93 (0.009)

0.93

PRMSEτ / s PRMSEτ/x

0.60 (0.033)

0.60 (0.024)

0.60 (0.017)

0.60 (0.011)

0.60 (0.007)

0.60

0.90 (0.045)

0.90 (0.031)

0.90 (0.023)

0.90 (0.016)

0.90 (0.010)

0.90

Mathematics Cognitive Skill Kno/ Und

PRMSEτ / s PRMSEτ/x

0.60 (0.038)

0.60 (0.026)

0.60 (0.019)

0.60 (0.013)

0.60 (0.008)

0.60

0.90 (0.055)

0.89 (0.036)

0.89 (0.027)

0.89 (0.018)

0.89 (0.012)

0.89

App

PRMSEτ / s PRMSEτ/x

0.75 (0.019)

0.75 (0.013)

0.75 (0.010)

0.75 (0.007)

0.75 (0.004)

0.75

0.91 (0.016)

0.91 (0.011)

0.91 (0.008)

0.91 (0.005)

0.91 (0.004)

0.91

PRMSEτ / s PRMSEτ/x

0.78 (0.017)

0.78 (0.012)

0.78 (0.009)

0.78 (0.006)

0.78 (0.004)

0.78

0.89 (0.014)

0.90 (0.010)

0.90 (0.007)

0.90 (0.005)

0.90 (0.003)

0.90

Prob Sol

The first value is the mean and the value in parentheses is the standard deviation of the sampling distribution (i.e., standard error) of 1,000 replications.

73 Taken together, the results provided in Tables 2, 3, and 4 revealed that there were no differences among the abilities of the three detection methods to detect subtest differences or subtest-total test differences. Kelleyâ&#x20AC;&#x2122;s agreement and correlations corrected for attenuation methods led to the same decisions regarding the identification of pairs of distinct subtests. Likewise, Kelleyâ&#x20AC;&#x2122;s agreement and proportional reduction of the mean squared error methods led to the same decisions about subtests having added value over the total test. Specifically, the decisions were that the subtests did not differ among themselves and the subtests did not add value over the total test.

Conclusion and Implications for Practice Whether or not to report subtest results is an important topic that has immediate practical implications. Given a profile of subtest scores, teachers and school counsellors can identify areas of strength and areas that need to be addressed for individual students. Similarly, changes in curriculum and instruction designed to maintain strength and address issues at the school and class levels can be made to improve student learning and achievement. Subscore reporting will most likely be enhanced if subtests are specifically developed to measure a multidimensional construct or domain. The subdomains to be assessed must be clearly defined and, if supportable, weakly to moderately correlated. The number of items used to assess each dimension or subdomain must be large enough to ensure an adequate level of reliability. The correlations between the subtests examined in the present study were moderate to moderately strong and the reliabilities of the subtests were not high, resulting in reliabilities and correlations being similar in value. But it seems reasonable to assume that the values of the correlations for the pairs of subtests in the present study are likely to be found in other large-scale assessments of achievement in the school system. Consequently, given this assumption, it is necessary to increase the reliabilities of the subtests. For example, assuming the median observed correlation among Mathematics content subtests in the present study, 0.63, the percentage of students who would be identified with subtest differences beyond chance using the agreement method would be approximately 5% if the reliability of the two subtests was 0.70, 9% if the reliability of the two subtests was 0.75, 15% if the reliability of the two subtests was 0.80, and 20% if the reliability of the two subtests was 0.85. Likewise, for the correlations corrected for attenuation and the proportional reduction of the mean squared error methods, pairs of subtests are most likely to be found distinct and subtests are most likely to have value over and above the total test if the subtests have relatively high reliabilities and the true subtest scores and the true total scores have only moderate correlations. The results for replicated random samples (n = 250, 500, 1,000, 2,000, and 5,000) revealed that the statistics for the three detection methods were accurate and precise estimators of the corresponding population parameters.

References American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334. Grandy, J. (1992). Construct validity study of the NTE core battery using confirmatory factor analysis. (ETS Research Report No. RR-92-03). Princeton, NJ: Educational Testing Service. Gulliksen, H. (1950, 1967). Theory of mental tests. New York: John Wiley & Sons, Inc. Haberman, S. J. (2005). When can subscores have value? (ETS Research Report No. RR-05-08). Princeton, NJ: Educational Testing Service. Haberman, S. J. (2008). Subscores and validity. (ETS Research Report No. RR-08-64). Princeton, NJ: Educational Testing Service. Haladyna, T. M. & Kramer, G. A. (2004). The validity of subscores for a credentialing test. Evaluation and the Health Professions, 27, 349–368. Harris, D. J. & Hanson, B. A. (1991, April). Methods of examining the usefulness of subscores. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago. Kelley, T. L. (1923). A new method for determining the significance of differences in intelligence and achievement scores. Journal of Educational Psychology, 14, 300–303. Lord, F. M. & Novick, M. R. (1968). Statistical theories of mental test scores. New York: Addison–Wesley. Lyrén, P. E. (2009). Reporting subscores from college admission tests. Practical Assessment, Research and Evaluation, 14(4), 1–10. McPeek, M., Altman, R., Wallmark, M., & Wingersky, B. C. (1976). An investigation of the feasibility of obtaining additional subscores on the GRE Advanced Psychology Test (GRE Board Professional Report No. 74 - 4P). Princeton, NJ: Educational Testing Service. (ERIC Document No. ED163090) Ryan, J. (2003). An analysis of item mapping and test reporting strategies. Greensboro, NC: South Carolina Department of Education. Sinharay, S. (2010). How often do subscores have added value? Results from operational and simulated data. Journal of Educational Measurement, 47, 150–174. Sinharay, S., Haberman, S. J., & Puhan, G. (2007). Subscores based on classical test theory: To report or not to report. Educational Measurement: Issues and Practice, 26, 21–28. Sinharay, S., Puhan, G., & Haberman, S. (2009). Reporting diagnostic scores: Temptations, pitfalls, and some solutions. Paper presented at the National Council on Measurement in Education, San Diego, CA, USA. Tate, R. L. (2004). Implications of multidimensionality for total score and subscore performance. Applied Measurement in Education, 17, 89–112. Wainer, H., Sheehan, K. M., & Wang, X. (2000). Some paths toward making Praxis scores more useful. Journal of Educational Measurement, 37, 113–140. Wainer, H., Vevea, J. L., Camacho, F., Reeve, B. B., Rosa, K., Nelson, L., Swygert, K. A., & Thissen, D. (2001). Augmented scores –“borrowing strength” to compute scores based on small numbers of items. In Test Scoring (pp. 343–387). Mahwah, NJ: Lawrence Erlbaum Associates. Yao, L. & Boughton, K. A. (2007). A multidimensional item response modeling approach for improving subtest proficiency estimation and classification. Applied Psychological Measurement, 31, 83–105.

International Journal of Learning, Teaching and Educational Research Vol. 4, No. 1, pp. 75-82, April 2014

Analysis of Achievement Tests in Secondary Chemistry and Biology Allen A. Espinosa Faculty of Science, Technology and Mathematics, College of Teacher Development, Philippine Normal University, Manila, Philippines Maria Michelle V. Junio Makati Science High School, Osias St., Poblacion Makati City, Philippines May C. Manla St. Louise de Marillac School of Tabaco, Ziga Avenue, Tabaco City, Albay, Philippines Vivian Mary S. Palma NOH-School for Crippled Children, Banawe St., Quezon City, Philippines John Lou S. Lucenari San Jose National High School, Rodriguez Rizal, Philippines Amelia E. Punzalan University of the Philippines National Institute for Science and Mathematics Education Development, Diliman, Quezon City, Philippines

Abstract. The study analyzes the performances of students in a regular high school, a special education school and a special science high school of different regions in the division achievement tests in Chemistry for school year 2009-2010. It also identifies the content of the division achievement tests in chemistry as well as in biology. The biology achievement test serves as a benchmark on how sophomores are being prepared, prior to taking a chemistry course. Test items in the division achievement tests are classified according to the level of thinking being developed. Difficulty indices for each item in the Chemistry achievement tests are also determined. The study found out that both Chemistry and Biology achievement tests focus on factual knowledge which promotes lower order thinking skills. It also found out that the special school and regular high schoolâ&#x20AC;&#x2122;s performances in the achievement test still fit a bell-shaped normal curve while a special science high school has a skewed to the right curve. Performances of students are also affected by face validity. Keywords: achievement test in chemistry; achievement test in biology ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

Background of the Study Teachers use different assessment tools in assessing their students. One of the assessment tools that teachers use is the achievement test. Tatum (2010) defined achievement tests as examinations that are designed to determine the degree of knowledge and proficiency exhibited by an individual in a specific area or set of areas. In the Philippines, the achievement test is given towards the end of the school year. This is usually given in February or March when all the expected learning competencies given by the Department of Education have been tackled already. Achievement tests in the public schools are usually provided by the Department of Education, at both division and national levels. The National Educational Testing and Research Center of the Department of Education is the office in charge of administering the achievement test. In the Division of City Schools in Quezon City, the division achievement is made by master teachers in the field and by the Division Supervisor of the area (Division of City Schools â&#x20AC;&#x201C; Quezon City, 2010). These reasons have prompted the researchers to conduct a study regarding division achievement tests in different regions. This study aims to determine and analyze the performances of students in a regular high school, a special school and a special science high school of different regions in the division achievement tests in Chemistry for school year 2009-2010. Moreover, it also identifies the content of the division achievement tests in Chemistry. It also attempts to compare the content of the division achievement test in Biology on how teachers in the second year level prepare students for the Chemistry course that they will be taking after Biology. After the analysis, the study also aims to propose a plan of action in increasing the performance of students in division achievement tests.

Methodology The researchers gathered Chemistry division achievement tests of different regions where they are teaching. These regions include the National Capital Region (represented by Makati and Quezon City) and Region IV-A (represented by Rizal). One of the researchers teaches Biology and was able to gather a Biology division achievement test in Region V (represented by Albay). In National Capital Region and Region IV-A, the researchers were able to gather actual data of studentsâ&#x20AC;&#x2122; responses on the said Chemistry division achievement tests while in Region V, the researchers were not able to gather actual student responses. The researchers would only use the Biology division achievement test to test the level of thinking that it is promoting in relation to preparing the students to a Chemistry course.

After gathering the needed data, the researchers made an item analysis (see appendices) wherein each question in the division achievement test was categorized into three. These are factual knowledge; conceptual understanding; and reasoning and analysis. After classifying each item, the researchers counted the number of students who were able to give a correct response. From this, they computed for the index of difficulty of each item by dividing the total number of students who were able to give a correct response by the total number of examinees. From the item analysis, the researchers made a table summarizing the levels of thinking and a table summarizing the difficulty index.

Results and Discussion The following data were collected from the item analysis done. Table 1. Summary of Level of Thinking Number of Items (Percentage) per Region Thinking NCR Levels V Quezon Makati IV-A City City 16 18 22 Factual 31 (32%) (36%) (37%) (62%) Knowledge 19 15 21 Conceptual 11 (30%) (35%) (22%) Understanding (38%) 15 17 17 Reasoning and 8 (30%) (34%) (28%) (16%) Analysis Total Number 60 50 50 50 of Items Table 1 shows the summary of levels of thinking in the division achievement tests in Chemistry in Regions IV-A and the National Capital Region. It also shows the summary of level of thinking in the division achievement in Biology for region V. From the table, it is noticeable that in general, division achievement tests give much emphasis on factual knowledge, thereby focusing only on lower order thinking skills. The skills being developed under this category are recalling or recognizing, defining, describing, and using tools and procedures (Mullis, et. al., 2003). The next skill that is given emphasis is conceptual understanding which focuses on illustrating with examples, comparing or contrasting or classifying, representing or modeling, relating, extracting or applying information, finding solutions and explaining (Mullis, et. al., 2003). The thinking level that is given least emphasis is reasoning and analysis which develop the higher order thinking skills of students like analyzing, interpreting or ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

solving problems, integrating or synthesizing, hypothesizing or predicting data, drawing conclusions, generalizing, evaluating and justifying (Mullis, et. al., 2003). One will also notice that the Biology division achievement test (as shown by Region V) is similar with the Chemistry division achievement test (as shown by NCR and Region IV-A) in terms of thinking level content. Both give much emphasis on factual knowledge and less on reasoning and analysis. It is also very evident that the Biology achievement test gives a very big emphasis on factual knowledge having more that 50% of the test items. The 1986 Constitution of the Republic of the Philippines mandates all schools to “encourage critical and creative thinking” (Constitution of the Republic of the Philippines, 2005, p. 55) among all Filipino students. Moreover, the 2002 Basic Education Curriculum prescribes the use of inquiry in teaching science to promote higher order thinking skills, such as critical and creative thinking (Department of Education, 2002). Table 2. Summary of Difficulty Index Number of Items (Percentage) per Region Difficulty Index NCR Range IV-A Quezon Makati City City 0.00 – 4 (8%) 2 (4%) 1 (2%) 0.04 0.05 – 0.09

0.10 – 0.14

2 (4%)

0.15 – 0.19

9 (18%)

0.20 – 0.24

0.25 – 0.29

1 (2%)

0.30 – 0.34

13 (26%)

0.35 – 0.39

0.40 – 0.44

1 (2%)

0.45 – 0.49

2 (3%) 0 1 (2%) 3 (5%) 2 (3%) 2 (3%) 5 (8%) 6 (10%) 6 (10%)

0.50 – 0.54

7 (14%)

1 (2%)

0.55 – 0.59

0.60 – 0.64

1 (2%)

0.65 – 0.69

7 (14%)

2 (4%)

0.70 – 0.74

2 (4%)

0.75 – 0.79

1 (2%)

0.80 – 0.84

7 (14%)

1 (2%)

0.85 – 0.89

5 (10%)

0.90 – 0.94

8 (16%)

0.95 – 0.99

6 (12%)

1.00

3 (6%)

17 (34%)

6 (10%) 5 (8%) 3 (5%) 3 (5%) 5 (8%) 2 (4%) 4 (8%) 2 (3%) 2 (3%) 0 0

Frequency of Occurence

Table 2 shows the summary of index of difficulty across two regions. 0.4 0.35 0.3 0.25

Quezon City Makati City Rizal

0.2 0.15 0.1 0.05 0 1 3

9 11 13 15 17

Difficulty Index Interval

Figure 1. Difficulty Index Across Regions You will notice that if you graph (see figure 1) the index of difficulty for Quezon City (NCR) and Region IV-A, you will be able to get a bell-shaped normal curve,

meaning, the scores are equally distributed. The representative school from Quezon City (NCR) is a special school while the representative school from Region IV-A is a regular high school. But if you will graph the index of difficulty for Makati City (NCR), you will not be able to get a bell-shaped normal curve since the scores are not equally distributed. The curve will be skewed to the right, meaning, more students were able to get a correct response to each item in the achievement test. This is because the representative school from Makati City (NCR) is a special science high school. The items manifesting a very high difficulty index show that the each item is easy and that it needs to be replaced or restructured. On the other hand, the items manifesting a very low difficulty index indicate that the said item is difficult and that it needs to be replaced or restructured, as well. A rough "rule-of-thumb" is that if the item difficulty is more than .75, it is an easy item; if the difficulty is below .25, it is a difficult item (Classroom Assessment, 2010). In other cases, a very low difficulty index would mean that the topic was not tackled in the class. The result shows that even if it is the governmentâ&#x20AC;&#x2122;s primary battle cry to produce Filipino citizens with higher order thinking skills, the present classroom instruction proves to be unsuccessful in developing higher order thinking skills among Filipino students. Florencio Abad, the then- secretary of the Department of Education, mentioned that the mastery levels in Science, Mathematics and English are devastating (Abad, 2005, p.8). Abad is referring to the performance of Filipino high school students in various competency- based examinations in 2004. The bad performance of the Philippines in the Trends in International Mathematics and Science Study (TIMSS) in 2003, ranking near the bottom also shows that Filipino students are weak in terms of higher order thinking abilities because the test required more skills in reasoning and analysis rather than conceptual understanding and factual knowledge (Martin, et. al., 2004). Table 3. Inconsistencies in the 2009-2010 Division Achievement Tests in Chemistry Item Numbers by Regions (Total Items =Percentage) Category NCR IV-A Quezon Makati City City 6, 9, 11, 14, 15, 16, Improper 22, 23, 25, arrangement 0 0 27, 35, 37, of choices 40, 42, 46 (15=30%) Improper 13 0 0 positioning of (1=2%) ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

a figure in a test question 2, 3, 10, 11, 12, 13, 30 (7=14%) 35, 45, 48 (6%)

35, 45, 48 (6%)

49 (2%)

2 (1=2%)

6 (1=2%)

Typographical Error

5, 23, 38, 46 (4=8%)

2, 3, 11, 9, 30, 32, 37, 45, 46 (18%)

6, 30, 41, 54, 52 (8%)

Unclear figure

13 (1=2%)

Incomplete question Misleading question Misleading choices Parallelism in construction of choices

6, 12 (4%)

21, 30, 41, 43, 52 (5=8%) 56 (1=2%) 6, 45, 53 (3=5%)

Another factor affecting student achievement in the division achievement test in Chemistry is the face validity of the test itself. Face validity is concerned with how a measure or procedure appears (Colorado State University, 2010). Table Three shows inconsistencies in the 2009-2010 division achievement tests in chemistry. These could be factors that affect the scores of students who took the achievement tests.

Conclusion and Recommendations As seen in the summary table for levels of thinking, division achievement tests in Chemistry and Biology give much emphasis on factual knowledge, which focuses on developing lower order thinking skills only. The least given emphasis is reasoning and analysis, which focuses on higher order thinking skills. As a result, Filipino high schools students manifested a poor performance in international competency- based examinations, as well as in division and national achievement tests. There are test items that are not face validated and content validated. This also affects the performance of the students in the division achievement tests. Results of the Chemistry achievement test vary from one type of school to another. The special science high schoolâ&#x20AC;&#x2122;s performance is far better than both the special school and regular high school. Against that, the graphs of the difficulty index for the special school and regular high school are bell-shaped normal curve, meaning, the scores are still equally distributed. To improve the performance of Filipino high school students in international competency- based examinations, the following are recommended:

a. Test items in the division and national achievement tests should be face and content validated; b.The number of test items in the division and national achievement tests for reasoning and analysis should be augmented; c. Classroom instruction should encourage creative and critical thinking; and d. Teachers should be trained in using classroom instruction that encourages creative and critical thinking.

References Abad, F. (2005). Why the crisis in education. BizNews Asia, 3(26), pp. 8-12. Colorado State University (2010). Retrieved May 18, http://writing.colostate.edu/guides/research/relval/com2b2.cfm

2010

from

Constitutions of the Philippines (2005). Manila: Anvil Publishing. Department of Education (2002). Basic Education Curriculum Operations Handbook. Manila, Philippines: Department of Education. Division of City Schools – Quezon City (2010). National Achievement Test. Retrieved May 17, 2010 from http://www.cityschoolsqc.ph/innovation.asp Martin, M.O., Mullis, I. V.S., Gonzales, E.J., Gregory, K.D., Smith, T.A., & Chrostowski, S.J. (2004). TIMSS 2003: International science report; findings from IEA’s report of the Trends in International Mathematics and Science Study. Chestnut Hill, MA: The International Study Center, Lynch School of Education, Boston College. Mullis, I.V.S., Martin, M.O., Smith, T.A., Garden, R.A., Gregory, K.D., Gonzales, E.J., Chrostowski, S.J. (2003). TIMSS Assessment Frameworks and Specifications 2nd Ed. Boston: TIMSS International Study Center, pp. 63-68. Ramirez, Rachel Patricia and Mildred Ganaden (2008). Creative Activities and Student’s Higher Order Thinking Skills. Quezon City: Education Quarterly, Volume 66 (1), pp. 22-23. Tatum, Malcolm (2010). What is an Achievement Test. Retrieved May 16, 2010 from http://www.wisegeek.com/what-is-an-achievement-test.htm

International Journal of Learning, Teaching and Educational Research Vol. 4, No. 1, pp. 83-166, April 2014

Towards Developing a Proposed Model of TeachingLearning Process Based on the Best Practices in Chemistry Laboratory Instruction Paz B. Reyes Lyceum of the Philippines University, Batangas City, Philippines Rebecca C. Nueva España College of Graduate Studies and Teacher Education Research, Philippine Normal University, Manila, Philippines Rene R. Belecina College of Graduate Studies and Teacher Education Research, Philippine Normal University, Manila, Philippines Abstract. This study investigated the teaching practices employed by the faculty of the Lyceum University System in teaching chemistry laboratory in order to attain the seven goals of laboratory instruction: (2) mastery of subject matter; (b) scientific reasoning; (c) understanding complexity and ambiguity of empirical work; (d) practical skills; (e) understanding the nature of science; (f) interest in science and in learning science; and (g) teamwork skills. It also determined the extent by which the attainment of the goals of science laboratory instruction was manifested in the students‟ (a) attitude and motivation; (b) laboratory skills; and (c) achievement. Finally, a proposed model of teaching-learning process in chemistry laboratory instruction was developed based on the identified best teaching practices. The qualitative-quantitative methods of research particularly the descriptive design were used. To gather data, interview was conducted to separate groups of students and faculty. Further, classroom observations and questionnaires were conducted and administered to gather other pertinent data. The subjects of the study were eighty students enrolled in General Chemistry during the second semester of the school year 2011-2012 and 4 chemistry instructors. The chemistry instructors were chosen from each of the four universities included in the Lyceum University System. With the aim of determining the best teaching practices employed by the faculty in teaching chemistry laboratory, five instruments were developed and validated by experts: Focus Group Interview Questionnaire for faculty and for Students; Observation Checklist; Attitude/Motivation Instrument; Practical Test; and the Achievement Test. The data analysis made use of frequency, percentage, mean and standard deviation. The results of the study revealed that the teaching practices of the chemistry faculty of the Lyceum University System were based on the university vision, mission goals and objectives and therefore attained the seven goals of the science laboratory instruction. Likewise, the students acquired a positive attitude towards chemistry, high competency in laboratory skills and average level of achievement in the subject. It can be deduced from the findings that indeed the best practices of the faculty © 2014 The authors and IJLTER.ORG. All rights reserved.

in teaching chemistry laboratory are those practices where students engaged in experiential learning, active learning, meaningful learning, and cooperative learning. It was manifested in the students‟ attitude/motivation, laboratory skills and achievement as shown by their interest in chemistry and in learning chemistry, their cognitive and manipulative skills and their understanding of the concept. The use of the proposed model of constructivist teaching-learning process is recommended for an effective chemistry laboratory instruction. Keywords: best practices in chemistry laboratory instruction, teaching-learning model in chemistry

Introduction Science and technology play a major role in man‟s quest for quality of life which subsequently causes a great impact to society. Science is relatively an experimental field, and most of the time learning its concepts and skills happen in the laboratory. Investigating scientific phenomena and testing hypotheses begin with making observations and gaining reasons for or describing observed situations. As such, it is the supreme art of the science teacher to awaken a child‟s curiosity and enkindle the eagerness to explore, to search for knowledge, truth and harmony. To Petrucci, Herring, Madura & Bissonnette (2002), inculcating scientific discipline among learners reflect a response to a higher goal of learning. With different governments in the world trying to redefine and fine tune education, it is therefore imperative to develop more capacity building in science and technology. In the Philippines, for instance the 1986 Constitution provides support for science and technology. Article XIV reads: “Science and Technology are essential for national development and progress. The state shall give priority to research and development, invention, innovation and their utilization; and to science and technology education, training and services. It shall support indigeneous, appropriate, and self – reliant scientific and technological capabilities, and their application to the country’s productive system and national life.” Among the different scientific discipline, Chemistry is regarded as an active and continually growing science and is focussed on realms of both nature and society. Chang (2009) posited that “Chemistry is every bit a modern science, an experimental science that rests on a foundation of precise vocabulary and established methods.” Following this line of thinking, chemistry must be concerned not only with the teaching of concepts, but also of laboratory skills. Therefore, to study chemistry is to understand how concepts are translated into application in a laboratory setting. Among the scientific changes in the concept of chemistry teaching in the last decade is the emphasis of laboratory work as an efficient and meaningful technique in learning science concepts. This change is patterned on Salandanan‟s (2002) definition of experiment which is a mean of illustrating the basic concepts of science and giving a clear view of the topics studied in class. Wink, Gislason, & Kuehn (2000) further validated that through experimentation, knowledge can be formed and in some instances erroneous beliefs that have been passed down by authority can be discarded.

Still, chemistry is not complete without laboratory works because it is where students can discover things for themselves, where they can be actively involved in identifying and using varied chemicals With the onset of modern learning styles and modes of education, it is imperative to consider the different learning tasks in teaching college chemistry to make the system more responsive to the demands of the 21st century. In fact, at the collegiate level, all students should have opportunities to experience more meaningful science laboratory investigations. Such laboratory experiences should aim to address how students should be taught how to work independently and collaboratively as well as incorporate and critique scientific studies published. Moreover, laboratory experiences must teach students how to develop scientific reasoning and appropriate laboratory techniques to define and solve problems, and finally, to draw and evaluate conclusions based on quantitative evidences. Laboratories should correlate closely with lectures and should not be separate activities. This fact reflects that exposure to rigorous, inquiry – based laboratories at the college level duplicates the same experience the science teachers had when they took their undergraduate studies. On the part of the educators, it is not enough for chemistry teachers to simply give facts, figures, concepts, theories, laws and other data, but they should be concerned with incorporating new teaching methods into their laboratory activities and development of courses with more realistic expectations of student involvement in experimental designs, data analyses and data interpretations. Linking laboratory activities which the students really enjoy provides a wider span of meaningful learning and development for both teachers and students (National Science Teachers Association, 2007). To achieve such meaningful learning, laboratory instruction must be designed in a way that it will develop not only the cognitive but also the manipulative skills of students. Again, the argument remains the same, it is not enough for students to learn the concept, it must be taught together with the process. As active participants in science laboratories, students gain a deeper sense of understanding and a greater confidence in their learning. With the acknowledged importance of a laboratory experience for all students, it is necessary for instructors to think clearly about the elements that could help achieve an effective laboratory experience. For instance, it is of great importance to know what techniques can be utilized to encourage students to confidently contribute to their laboratory groups. Corollary to this, which scientific skills and procedures must be practiced and mastered by students to achieve that level of confidence during laboratory works. Still another important point to emphasize are the kinds of instruments in a science laboratory which students should be familiar with. Similarly, understanding the importance of the “other laboratory skills” such as communication (written and oral), teamwork, ethics, fairness, and responsibility should be taught to maximize students‟ participation in performing experiments. In a study conducted by Narayan (2005), she suggested that for students to be engaged in science, they needed to be involved in “learning to use language, think and act in ways that enable one to be identified as a member of the scientific literate community and participate in © 2014 The authors and IJLTER.ORG. All rights reserved.

the activities of that community”. She added that learning occurs more effectively when the student is socialized into a community of practice that he is immersed in. Most science educators encourage fellow teachers to provide students with access to more authentic science activities. Queries on possible steps to take on how to improve the delivery of science lessons and skills remains to be the primary objective in redevelopment and restructuring of pedagogical practices in science classrooms The Lyceum of the Philippines University, one of the country‟s premier institutions, set the standards of commitment in pursuing excellence in education. Guided by its vision, mission and core values, the university offers various science programs which include laboratories as an integral part of the curriculum. The researcher, as a science educator, is concerned with the meaningful learning of students in chemistry. With her several years of teaching chemistry laboratory, the issue on what strategies to use to help students develop positive attitude towards chemistry and become independent learners who are ready to face the challenges of the 21st century science education, has been her problem. Thus, the researcher attempted to investigate the best teaching practices that will focus on the attainment of the goals of chemistry laboratory instruction which aims to develop positive attitude and high motivation of students as well as competency on their laboratory skills which could lead to high probability of achievement. Conceptual Framework This study is anchored on Piaget‟s Theory of Constructivism which encourages learning through collaboration and interchange among the students themselves. Piaget (Muijs and Reynolds, 2011) suggested that students construct new knowledge from their experiences through “accomodation and assimilation.” Constructivism as a learning theory views learning as a process in which “students actively construct or build new ideas and concepts based upon prior knowledge and new information.” Further, it suggests that instruction should follow some basic principles such as; (1) children should be allowed to make mistakes and correct these on their own thereby enabling them to accommodate, assimilate and reconstruct knowledge on their own; discovery learning is emphasized; (2) the process of experimentation at all stages is important; and (3) knowledge is always a construction by the learner which involves operative processes that lead to transformation of reality, either in action or thought therefore experimentation should be done continually. The constructivist teacher encourages students to discover principles and construct knowledge within a given framework or structure by helping students connect with prior knowledge and experiences while new information is being presented. Through constructivism students can dispense their misconceptions and build a correct understanding. Constructivism is a conceptual basis of this study because practices in chemistry laboratory instruction if tailored on the elements of constructivism will lead to the attainment of the seven goals of science laboratory instruction and will develop positive attitude, competency in laboratory skills and high achievement of students © 2014 The authors and IJLTER.ORG. All rights reserved.

In Shulman‟s view as cited by Rowan, Schilling, Ball & Miller ( 2011), the trend in education is one that addresses the pedagogical content knowledge (PCK). PCK is a form of practical knowledge that entails, among other things: (a) knowledge of how to structure and represent academic content for direct teaching to students; (b) knowledge of the common conceptions, misconceptions, and difficulties that students encounter when learning particular content; and (c) knowledge of the specific teaching strategies that can be used to address students‟ learning needs in particular classroom circumstances. PCK is concerned with the representation and formulation of concepts, pedagogical techniques, knowledge of what makes concepts difficult or easy to learn, knowledge of students‟ prior knowledge and theories of epistemology. It further views the knowledge of what the students bring to the learning situation, knowledge that might be either facilitative or dysfunctional for a particular learning task at hand. This knowledge of students includes their strategies, prior conceptions (both "naïve" and instructionally produced); misconceptions students are likely to have about a particular domain and potential misapplications of prior knowledge. PCK represents the blending of content and pedagogy into an understanding of how particular aspects of subject matter are organized, adapted, and represented for instruction. Finally, Rowan, et al. (2011) argued that "pedagogical content knowledge" reflects the content knowledge that deals with the teaching process, including "the ways of representing and formulating the subject that make it comprehensible to others.” In a larger vantage and scope, therefore, a constructivist chemistry teacher implement the best teaching practices in chemistry laboratory if he/she has a knowledge of both content and pedagogy. This idea makes not only constructivism but also pedagogical content knowledge as the conceptual bases of this study. Studies have revealed that there are seven goals of laboratory instruction in Science Education (Singer, Hilton, & Schweingruber, 2005 and Jona, Adsit & Powell, 2008). These goals include a) enhancing mastery of subject matter b) developing scientific reasoning c) understanding the complexity and ambiguity of empirical work d) developing practical skills e) understanding the nature of science f) cultivating interest in science and interest in learning science and g) developing teamwork skills. These goals were achieved in the classroom, according to Jona, et al. (2008) if a student shows mastery of subject matter by readily remembering and understanding the concepts taught. On the same level, if a student manifests the ability to apply the knowledge acquired, then the student has mastery of subject matter. Scientific reasoning is manifested in the students‟ ability to explain, predict and control the occurrence of events. Students understand the complexity and ambiguity of empirical work if they can address the challenges inherent in directly observing and manipulating the material world. Practical skills are developed if students can use scientific equipment correctly and safely, make observations, take measurements and carry out well-defined scientific procedures. The nature of science is being understood if students can interpret data from the material world and they can discover that different people may interpret the same data differently. Interest in science and interest in learning science may be reflected from the positive attitude and high motivation of students. Teamwork skills are developed if students have the ability to collaborate effectively with others. To achieve all these goals, therefore, teachers must implement practices to address positive attitude, high motivation, competency in laboratory skills and high achievement in the subject.

Figure 1 shows the conceptual paradigm of the study. The study focused on the investigation of the best practices in teaching chemistry laboratory with the purpose of attaining the seven goals of laboratory instruction in science education. Chemistry Laboratory Instruction must be student–centered which is focused on how students are motivated to acquire positive attitudes towards chemistry, competencies in lab skills and high achievement. To achieve this goal, teachers must integrate the seven goals of science lab instruction in their practices to address a student-centered classroom setting.

Teaching practices to attain:   

  



Mastery of subject matter Scientific reasoning Understanding complexity and ambiguity of empirical work Practical skills Understanding nature of science Interest in science and learning science Teamwork skills

A Model Students‟:   

Attitude/ motivation Laboratory skills Achievement

Of Teaching-Learning Process In Chemistry Laboratory Instruction

Figure 1. Conceptual Paradigm of the Best Teaching-Learning Process in Chemistry Laboratory Instruction

The first box in the paradigm reflects the teaching practices of chemistry laboratory teachers intended to attain the seven goals of science laboratory instruction. These goals include enhancing mastery of subject matter, developing scientific reasoning, understanding the © 2014 The authors and IJLTER.ORG. All rights reserved.

complexity and ambiguity of empirical work, developing practical skills, understanding the nature of science, cultivating interest in science and interest in learning science, and developing teamwork skills. The second box on the other hand, shows the goals of science laboratory instruction are achieved by the correct pedagogical practices of the faculty. Towards the end, a model of teaching-learning process in chemistry laboratory instruction was developed from the investigated best teaching practices leading to the attainment of the seven goals which are then manifested in the attitude, motivation, laboratory skills and achievement of students. Statement of the Problem The main objective of the study is to propose a model of a teaching-learning process based on the identified best practices in chemistry laboratory instruction. 1.

Specifically, the study sought answers to the following questions: What are the teaching practices employed by the faculty in teaching chemistry laboratory in order to attain the seven goals of science laboratory instruction? 1.1 mastery of subject matter 1.2 scientific reasoning 1.3 understanding of the complexity and ambiguity of empirical work 1.4 practical skills 1.5 understanding of the nature of science 1.6 interest in science and interest in learning science 1.7 teamwork skills To what extent do students manifest the attainment of the goals of science laboratory instruction in some student-related parameters? 2.1 attitude and motivation 2.2 laboratory skills 2.3 achievement Based from the findings of the study, what model of teaching-learning process in chemistry laboratory instruction may be proposed to attain the goals of science laboratory instruction?

Goals of Laboratory Instruction in Science Education. A better science program is said to be that which includes laboratories and other forms of scientific investigations. Scientific investigations must be conducted in accordance with the goals of laboratory instruction in science education. Singer, et al. (2005) reported in the Americaâ&#x20AC;&#x;s Lab Report of the National Research Council the seven goals for scientific investigation as the desired targets of a comprehensive science program. These goals, he added should be taught in every science laboratory and these include the attainment of a) mastery of subject matter b) scientific reasoning c) the understanding of the complexity and ambiguity of empirical work d) practical skills e) the understanding of the nature of science f) interest in science and interest in learning science g) teamwork skills. Jona, et al. (2008) discussed further each of the seven goals of scientific investigation.

Mastery of subject matter. Similar to other disciplines, science aims to teach both established facts and concepts (content) and the skills used by professionals in that field (process). The National Research Council (2005) found that in typical programs, content and process are taught separately. However, modern educational theorists view them as related educational goals because according to Newmann and Wehlage as cited by Jona, et al. (2008), kinesthetic activities and other active learning experience help students in learning the content of the subject matter. Therefore, mastery of subject matter could be attained if concept and process are taught simultaneously. This goes exactly the opposite of typical laboratory experiences where students perform a process without a clear understanding of the relation of that process to content. For example, when students perform titration they do not understand completely why they are doing it or they cannot explain the results in terms of scientific concepts. Furthermore, typical laboratories attempt to demonstrate scientific concepts by making students follow set processes in a recipe type just to confirm something that has been already taught. Inquiry activities, on the other hand, that include manipulation of ideas rather than materials and procedures enhance student understanding of facts and concepts. Corollary to this, an integrated learning program was proposed by the National Research Council (2005). This program makes use of a constructivist approach which according to Teachnology (2007) attempts to make students observe and draw conclusions about concepts prior to receiving explicit instruction. The program consists of instructional design which will improve student mastery of subject matter. This includes the close integration of investigative activities into content, a merging of content instruction and process instruction, and reflection on the meaning of the learning activity once it is completed. Scientific reasoning. Students should be taught the various kinds of scientific processes and valid reasoning principles and at the same time must be given the opportunity to practice these reasoning skills. To achieve this, laboratory instructions must be planned so that students can be encouraged to participate in designing the process of investigation, making them draw and support conclusions. In a direct contrast, a typical science course, students do not develop scientific reasoning skills because they were not given time for planning investigation or interpreting results. Experts take such scenario due to the focus of instruction only on learning content and laboratory experiences focuses upon following specified procedures. With the integrated learning program, a variety of skills associated with scientific reasoning can be developed among students. According to the National Research Council (2005), these include the ability to identify questions and concepts leading to scientific investigations, design and conduct scientific investigations, develop and revise scientific explanations and models; recognize and analyze alternative explanations and models, and make and defend a scientific argument, including writing, reviewing information, using scientific language appropriately, constructing a reasoned argument, and responding to critical comments. It is very well expected that a well designed science course should consider a core scientific process which deal with the ability of students to construct scientific arguments. In this process, students must be taught to design experiments, make predictions, interpret and explain data, recognize discrepancies between predicted and observed outcomes, design good experiments. ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

Instructional practice is effective only when students learned how to relate theoretical claims with evidences gathered from their laboratory investigations. Understanding of the complexity and ambiguity of empirical work. To Singer, et al. (2005), scientific investigations should be properly designed in such a manner that students are able to expect outcomes or experimental results contradicting widely accepted scientific principle. Students must understand that even the same experiments may lead to different results if performed at different times or by different people. Similarly, students should not be confused with the misconceptions that science is a collection of clearly defined laboratory procedures whose outcome firmly support received instruction, instead students must know how to deal with these complexities and ambiguities of empirical work as one aspect of the nature of science. Researchers believe that a well designed scientific investigation program must include opportunities for students to be involved in activities like troubleshooting of laboratory equipment, rechecking data observations and analysis, examining the parameters, assumption and study definitions in contradictory studies, and generally performing the kind of follow-up investigations done within the scientific community. Further, such program must allow students to understand measurement error and interpret and aggregate the resulting data. One technique for an instructional designer is to allow students who are working in a team to perform activities independently, compare results and then discuss and account for discrepancies. Students must be allowed to make mistakes and correct them on their own. Experimental errors are not hindrances to learning, but they are opportunities for greater learning. So instead of working hard to remove complexities and ambiguities, laboratory instructors should include the expectation of experimental errors in their instruction. Practical skills. Although practical skills refer to the proper use of scientific equipment and the conventions of science such as measuring, observing and other science processes, it is not enough for students to know how to use tools and follow correct procedures in scientific investigations. Rather, what is important is they know how to apply effectively the appropriate scientific processes to a new investigation so that they can make accurate observations and follow accepted procedures to ensure valid results. Understanding the nature of science. In a typical science course, students do not realize that science is a human endeavor that seeks to understand the material world and that scientific theories, models and explanations change over time on the basis of new evidence. They simply see that science is a collection of laws and facts without really understanding how existing concepts came into being, how existing ideas are reshaped with new discoveries, how accepted theory differs from wild guess or from firm facts, and how new concepts and theories emerge through investigations. In an integrated learning program, laboratory teachers should explicitly teach concepts in the instructional phase and aim to reinforce the understanding of the concepts through an investigative process. To achieve this, various instructional strategies, such as constructivist ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

activities and activities that allow students to create their own scientific investigations to solve problems, must be implemented in their instruction. Students should be given metacognitive assignments which will allow them to reflect on their learning and to relate their experiences to scientific principles and procedures. Interest in science and interest in learning science. Instruction is considered effective if it cultivates an interest in the subject and motivates students to continue learning more about the subject. This is made possible in science laboratory instruction by applying the five principles of authentic instruction. Scientific investigations and other laboratory activities are applications of authentic instruction which are consistent with the integrated learning process. The five principles of authentic instruction in science include: a) higher order thinking skills b) depth of knowledge c) connectedness to the world beyond the classroom d) substantive conversation and e) social support for student achievement. According to Fraser as cited by Jona, et al. (2008) an extensive study involving multiple countries (including the United States) indicated that positive student attitudes toward science are strongly associated with cohesiveness (the extent to which students know, help, and are supportive of one another) and integration (the extent to which laboratory activities are integrated with non-laboratory and theory classes). Teamwork skills. To National Research Council (2005), scientific investigations promote a student‟s ability to collaborate effectively with others in carrying out complex tasks, to share the work of the task, to assume different roles at different times, and to contribute and respond to ideas. It is important in a learning community to have teamwork and collaboration among members. A well-designed collaborative authentic instruction can enhance student learning in contrast to a poorly designed collaborative process that undermine instruction and student achievement. Teamwork skills therefore, must be integrated as part of the instructional process by introducing investigate processes early in the course. Still in the same argument, teamwork is a part of authentic instruction where substantive conversation requires interaction among members. There is high level of substantive conversation if 1) there is considerable interaction about the ideas of a topic (the talk is about disciplined subject matter and includes indicators of higher-order thinking such as making distinctions, applying ideas, forming generalizations, raising questions, and not just reporting experiences, facts, definitions, or procedures) 2) there is sharing of ideas which is evident in exchanges that are not completely scripted or controlled (as in a teacher-led recitation). Sharing is best illustrated when participants explain themselves or ask questions in complete sentences and when they respond directly to comments of previous speakers 3) The dialogue builds coherently on participants‟ ideas to promote improved collective understanding of a theme or topic. In a true collaborative work, students share ideas about hypotheses, procedures and conclusions, directly contradicting how students in a typical laboratory experience work in group to divide limited laboratory equipment and space among a large number of students. The Role of Laboratory Instruction in Science Education The National Science Teachers Association (NSTA, 2007) defined school laboratory investigations as “an experience in the laboratory, classroom, or the field that provides students © 2014 The authors and IJLTER.ORG. All rights reserved.

with opportunities to interact directly with natural phenomena or with data collected by using tools, materials, data collection techniques, and models.â&#x20AC;? From these investigations, evidences are collected through observations which become the basis in generating scientific theories and scientific laws. In the entire process of investigations, students are expected to acquire skills and knowledge such as the ability to design investigations, engage in scientific reasoning, manipulate equipment, record data, analyze results, and discuss their findings. To achieve this purpose of instruction, National Science Teachers Association (2007) recommend the inquirybased laboratory investigation which is the process of asking questions and conducting experiments as a way to understand the natural world. Inquiry-based laboratory investigations provide instruction with a priority on making observations and gathering evidence, much of which students experience in the lab or the field, to help students develop a deep understanding of the science content, as well as an understanding of the nature of science, the attitudes of science, and the skills of scientific reasoning. To address this, integration of inquiry-based laboratory investigations in the science lesson in every level of education starting from preschool to higher education should be achieved. As students move up to higher grades, the level of complexity of laboratory investigations should also increase. In the preschool and elementary level, students should be given opportunities to investigate appropriate questions, analyze the results of laboratory investigations, debate what the evidence means, construct an understanding of science

concepts, and apply these concepts to the world around them. As the students move up to the high school level, students should develop a growing understanding of the complexity and ambiguity of empirical work, as well as the skills to calibrate and troubleshoot equipment, understand measurement error; and have the skills to aggregate, interpret, and present the resulting data. They should also improve their ability to collaborate effectively with others in carrying out complex tasks, share the work of the task, assume different roles at different times, and contribute and respond to ideas. At the tertiary level, students must learn how to work independently and collaboratively, incorporate and critique the published work of others in their communications, use scientific reasoning and appropriate laboratory techniques to define and solve problems, and draw and evaluate conclusions based on quantitative evidence. To Domin (2009), all the expected outcomes are possible in science education. Learning about the methods and processes of scientific research (science process) and the knowledge derived through this process (science content) are expected as a well developed science education curriculum.. Science process involves direct interactions with the natural world in order to explain natural phenomena. Further, science education should include opportunities for students to learn about both the process and content of science and this is possible only through laboratory experiences. Laboratory Experiences and Student Learning The Committee on High School Laboratory, as cited by Singer, et al. (2005) in Americaâ&#x20AC;&#x;s Lab Report, pointed out the importance of laboratory experiences of students in attaining the seven ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

goals of laboratory instruction in science education. In typical laboratory experiences, students are engaged in one or two experiments followed by assessment to determine whether their understanding of science concept had increased. The Committee on High School Laboratory recommended integration of laboratory experiences into instructional sequences in order to help students progress toward science learning goals. In this way, student‟s learning about the concepts and processes of science are also integrated. This integration is referred to by the committee as the “integrated instructional units” which is designed to be learner-centered. Integrated instructional units believed that effective instruction begin with what learners bring to the setting including cultural practices and beliefs, as well as knowledge of academic content. Students based their preconceptions of the natural phenomena on their everyday experiences in the world. These preconceptions are often reasonable and can provide satisfactory everyday explanations to students, but they do not always match scientific explanations thereby considered as intuitive ideas. Teachers are challenged with these intuitive ideas of students, they are challenged to help students move towards a more scientific understanding through change in and not merely an addition to what students notice and understand about the world. The principle behind the integrated instructional units is that learning is enhanced when the environment is knowledge-centered. In the knowledge-centered environment, students learn with understanding rather than simply acquiring sets of disconnected facts and skills. There are two bodies of knowledge in science with which students must be engaged to – one is knowledge of accepted scientific ideas about natural phenomena and the other is understanding of what it means to “do science”. These two aspects of science are reflected in the goals of laboratory experiences, which include mastery of subject matter (accepted scientific ideas about phenomena) and several goals related to the processes of science (understanding the complexity of empirical work, development of scientific reasoning). Student thinking about science shows a progression of ideas about scientific knowledge and how it is justified. At the first stage, students perceive scientific knowledge as right or wrong. Later, students characterize discrepant ideas and evidence as “mere opinion.” Eventually, students recognize scientific knowledge as being justified by evidence derived through rigorous research. Metacognitive strategies when implemented in a knowledge-centered environment will enable students to reflect on their own learning progress, to identify, monitor and regulate their own thinking and learning which in turn will facilitate their learning. To be effective problem solvers and learners, students need to determine what they already know and what else they need to know in any given situation, including when things are not going as expected. The basic metacognitive strategies include: (1) connecting new information to former knowledge, (2) selecting thinking strategies deliberately, and (3) monitoring one‟s progress during problem solving. Furthermore, in a knowledge-centered learning, the practices and activities in which people engaged in while learning, shape what they learn. Transfer (the ability to apply learning in varying situations) is made possible to the extent that knowledge and learning are grounded in multiple contexts such as what transpires in the laboratory. Through multiple contexts, students can develop a deeper understanding of the concept and its use aside from they can acquire the ability to transfer what has been learned in one context to others. © 2014 The authors and IJLTER.ORG. All rights reserved.

Learning is enhanced in a community setting, when students and teachers share norms that value knowledge and participation (Cobb, Stephan, Clain & Gravemeijer, 2001). Such norms, Cobb further argued, increase people‟s opportunities and motivation to interact, receive feedback, and learn. It is said learning is enhanced when students have multiple opportunities to articulate their ideas to peers and to hear and discuss others‟ ideas. Such scenario can be achieved in integrated instructional units which are combined laboratory experiences and other types of science learning activities which may include lectures, reading, and discussion. If a classroom addresses quality norms, students are given more opportunities to frame their own research questions, design and execute experiments, gather and analyze data, and construct arguments and conclusions as they carry out investigations thereby making them independent learners. On the other hand, diagnostic and formative assessments are embedded into the instructional sequences and can be used to gauge student‟s developing understanding and to promote their self-reflection on their thinking. The National Research Council (2005), in its report considered four principles of instructional design that can help laboratory experiences achieve their intended learning goals. These principles are (1) instructions must be design with clear learning outcomes in mind, (2) they must be thoughtfully sequenced into the flow of classroom science instruction, (3) they must integrate learning of science content with learning about the processes of science, and (4) they must incorporate ongoing student reflection and discussion. Combined with the seven goals, these principles offer better chances for students to experience worthwhile laboratory experiences. They provide a framework for curriculum developers, administrators, and teachers to use in reconsidering how laboratory experiences can be successfully incorporated into science courses. Laboratory Instruction in Chemistry As an experimental science, Chemistry depends heavily on experimental work as a strategy for teaching scientific principles and concepts and its development and application demand a high standard of performance of laboratory activities. Other than the expected results in student learning, laboratory activities allow students to appreciate and experience the constraints, potential and tensions of an investigative process which can only be experienced in the laboratory. The laboratory is the most attractive place for the students to develop and show applications for general principles and techniques, says Wink, et al. (2000). It allows students to experience how the solution of real problems by people in all walks of life requires a thorough understanding of general chemistry principles. Bishop, Bishop & Whitten (2000) pointed out that the best way to acquire a deep, clear understanding of the nature of chemistry is in “hands-on” laboratory experiments with the real chemicals and real equipment which chemists use. There are four reasons, according to Zulueta and Guimbatan (2002) for using the laboratory as a method of instruction in science, in particular, in teaching chemistry. Laboratory instruction in chemistry gives opportunities to students to manipulate concrete objects; participate actively; develop scientific competencies and motivation. Science involves the learning of highly complex and abstract subject matter. By allowing the students to have “hands-on experience”, they understand and use scientific principles learned from the opportunity to manipulate actual © 2014 The authors and IJLTER.ORG. All rights reserved.

objects and materials. Further, participating in a laboratory exercise gives students an appreciation of the methods of science and promotes problem-solving and other analytic competencies that can be generalized and applied to other areas. Students enjoy goal-oriented activities and practical work where they can see the relevance of abstract concepts and principles; and consequently, become interested in sciences and are motivated to learn more about discipline. In addition, Walton as cited by Corpuz, Rimas, Galangco & Bautista (2003), gives the aims of laboratory method as to give firsthand experience in the laboratory which may increase student interest; to provide student participation in original research and to develop skills in the use of laboratory equipment and instruments.

Best Practices in Science Teaching The National Research Council (2005) identified pedagogical practices which are considered truly best practices or authentic best practices in science teaching. Used in the science classrooms, there are significant evidences that show how these practices help students in learning better. These authentic best practices include: 1) Engaging resilient preconceptions. Students upon coming into the classroom have already initial understanding and preconceptions about the topic to be discuss. These preconceptions often limit what a student can learn so that it is important for a teacher to identify, confront and resolve this initial understanding. 2) Organizing knowledge around core concepts. All these best practices aim to increase understanding and retention of concepts among students by carefully and scientifically organizing information. For instance, students can readily remember a concept if they are taught how to recognize a certain pattern. In this way, teachers provide a foundation of factual knowledge and conceptual understanding to students. 3) Supporting metacognition and student self-regulation. This is making students assess themselves as to what they know and what they don‟t know. This could be done by requiring students to make a reflection that summarizes what they have learned or by administering a pre-test. In this way, students can take control of their learning. On the other hand, Minstrell and Kraus (2005) enumerated the so-called best practices in science teaching which are based on ideology rather than on the findings of empirical research. These ideal practices which are often closely associated with students‟ success are: 

Establishing and maintaining classroom environments that are: o o o o

learner centered -- identifying, confronting, and resolving preconceptions, and beginning instruction with what students know. knowledge centered -- focus on how something is know as much as what is known, and provide examples of what mastery looks like. assessment centered -- make frequent attempts to make students' thinking and learning visible as a guide for further instruction. community centered -- encourages a culture of questioning, including a bit of risk taking and respect for others

                                        

Using an empirical approach Regularly employ active learning strategies Employ inquiry labs Talk about the nature of science Provide meaningful, engaged learning for all students. Provide an active approach to learning that includes a strong emphasis on student interaction with phenomena. Clear and explicit linkage between representations and phenomena represented. Engage students in challenging, authentic, interdisciplinary tasks. Provide opportunities for students to observe, explore, and test hypotheses. Eliminate discipline boundaries when natural, logical, and appropriate. Encourage the students' imagination, logic, and open-mindedness. Incorporate the content and processes of science giving due regard to science teaching standards. Give due regard to affective as well as cognitive domain. Link scientific concepts and processes with prior learning in science and other disciplines. Using a constructivist approach. Depth and breadth of coverage are reasonably balanced. Goals of tasks are conceptual and conceptual means are required to accomplish them. Assigning manageable tasks Setting high expectations Engage all learners in meaningful scientific tasks involving high-order thinking skills. Providing and receiving feedback Accommodating student learning styles Teaching in a way that is consistent with student development Including real-world applications in the learning process Using individual and group motivation Moving from concrete to abstract Requiring practice of learned skills Employing learning cycles - observation, generalization, verification, application Making use of multiple intelligences Establishing conducive learning environments Encouraging student evaluation of alternative hypotheses Addressing conceptual goals and means Eliciting and addressing misconceptions Promoting critical thinking Creating, sharing, and using scoring rubrics Aligning objectives, instruction, and assessment Focusing on depth in addition to breadth of coverage Placing strong emphasis on interaction with phenomena Making clear and explicit linkage of representations to phenomena Using multiple representations of physical phenomena Employing Socratic dialogues

Different approaches are employed by laboratory instructors in teaching chemistry laboratory, but all these approaches have an end goal of making students enjoy, understand, participate and develop skills. Aquino (2003) enumerated the three objectives of science teaching as the development of science process skills, scientific attitudes, and literacy. On the other hand, Salandanan (2002) stated that students should be able to achieve the different goals of science teaching such as the development of scientific attitudes and values; enhancement of skills in employing a systematic and scientific methodology; gaining an understanding of functional knowledge; arousing further interest in science-based pursuits; and development of desirable social attitudes. In any teaching strategy in the laboratory, students must be given the opportunity to develop critical and analytical mind. Teachers must engage in practices that will arouse the curiosity of students by encouraging them to ask questions. Creativity and resourcefulness could be inculcated into the students by stimulating them to generate new ideas and original ways of doing things. Other wholesome attitudes should be develop by students through awakening their interest and keeping them highly motivated to inquire about occurrence in the natural environment. They must learn to make fair and unbiased decisions, accept evidences, suggestions, and alternatives in the light of new discoveries. They must relentlessly pursue an investigation and be responsible enough to complete an assigned task despite constraints. They must have constant practice in experiencing step-by-step procedure to find answers to their endless questions. Encouraging students to participate actively in planned experiments might be daunting for the students, however it will definitely enable them to acquire functional knowledge which can be applied in solving problem situations in the environment instead of knowledge which is merely memorized and easily forgotten. Another best practice in the teaching of science is the treatment of life experiences as necessary tools to motivate students to participate in classroom activities. These learning experiences might include joining movements in science promotions, protection and care of the environment and natural resources, as well as helping decide on issues of nationwide interest. Experts are saying that given the correct avenue to express and explore these life experiences, there is a strong chance that students will decide to pursue a science profession in the future and will develop a feeling of gratitude and appreciation for the advances in science and technology that continue to raise the present quality of life. Further, when students are allowed to experience more group investigations, positive attitudes could be developed among students such as tolerance, respect for the opinions of others and willingness to accept criticisms and suggestions, learning to cooperate with others, willingness to share findings and resources and the readiness to extend expertise.

Laboratory Method. The laboratory method according to Acero, et al. (2000) deals with experimentation, observation or application by individuals or small groups dealing with actual materials. There are two types of laboratory method; the experimental and the observational method. They differ in aims and emphasis in the sense that experimental method aims to train pupils in problem-solving with incidental acquisition of information and motor skill while observational method aims on the acquisition of facts. As to emphasis, experimental method is focussed on discovery, original procedure, analysis, and solution of problems while in ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

observational the emphasis is on the acquisition of facts through activities such as visits to museums, exhibits, and art galleries, watching demonstration, listening to lectures, viewing films, and going on field trips. Laboratory method consists of three steps: the introductory step, work period and the culminating activities. In the process, introductory step aimed at orientation and motivation where determination of work to be done is presented. The work period is defined as a supervised period where students gain experience in the scientific procedure, handling raw materials, and using tools while working on the same problem or on different problems on their own. In the culminating activities, students discuss and organize their individual findings after completion of the work. They present the results by a) explaining the nature and importance of the problem the group had worked on; b) reporting data gathered on other findings; c) presenting illustrative materials or special contributions; d) special reporting and exhibition of work by those with individual projects; and finally, e) exhibiting various projects and explanations by their sponsors. To Hidalgo (2000), classroom strategy which uses laboratory method has an advantage over other methods because students are learning by doing since actual experience is vivid. When this happens, the learning gained is retained longer; reality is more vivid than any symbol; and it is a direct preparation for a new way of life. Hidalgo (2000) gave suggestions to the laboratory teachers on the better way of handling laboratory classes. For an efficient delivery of lesson, Hidalgo (2000) opines that teachers should adapt laboratory exercises to the needs, interest and capacities of students. To address reflected thinking, laboratory exercise must grow out of problems so that a recipe-type activity is not recommended. Another good practice for a laboratory teacher is to require students to keep a laboratory notebook where they can record not only the results of their investigations but also the learning they got from the experiment. For his part, Domin (2009) described four types of laboratory instructions: expository (traditional), problem based, discovery and inquiry-based. Domin (2009) explains that they differ in outcome, approach and procedure. Expository, problem based and discovery laboratories all have predetermined outcomes because he emphasized, the expected results are already known. However, the limitation is pegged on the fact that it is only the teacher who knows the outcome in the discovery and problem based unlike in the expository where both the teacher and students know the expected result before doing any activity. In the inquiry-based instructions, meanwhile, neither the teacher nor the students know the outcome of the experiment. Given this, therefore, the choice of instruction is crucial to achieve life-long learning among the students. The study conducted by the National Science Teachers Association (2007) reveals that laboratory investigations must be adapted to the age and ability levels of students. They should not be the recipe-type activity that is somewhat related to the instructional sequence of the topics discussed in the lecture. Well designed laboratory investigations are those in which the objectives of the activity are clearly communicated to students and which focus on science processes and integrate student reflection and discussion. Finally, the designed objectives give ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

100

the students the opportunities to develop safe and conscientious laboratory habits and procedures. Experimenting. An experiment (Salandanan, 2002) is “described as a learning activity wherein a student investigates a problem by manipulating a variable.” Salandanan (2002) listed the reasons why experiments are done: a) to develop basic science process skills; b) to cultivate an inquiry mind; c) to acquire higher-order thinking skills like critical thinking, creativity and inventiveness; d) “learning by doing” forms part of the students attitudes, habits, and ways of reacting; e) to internalize and instantly apply in solving problem situations f) to replace hearsay, superstitions, and unfounded beliefs by more objective assessment and evaluation; g) to make students appreciate and be grateful for the achievements of scientists; and h) to make students responsible for their own learning by completing the assigned tasks. For an experiment to be successful, it is argued that the students must clearly understand the problem; variables must be tested one at a time; students must participate actively in manipulating tools, materials and equipment. Any absent component would mean that the classroom will not be successful. On the part of the teacher, the teacher must be a keen observant who can easily spot incorrect steps and procedure to be able to encourage the use of improvised materials to promote resourcefulness and creativity and to underscore the important elements of classroom setting.

Demonstration Method. Still, Hidalgo (2000) defined demonstration method as the “planned manipulation of materials and equipment to the end that students are able to observe all or at least some of the manifestations of one or more scientific principles operating within a phenomenon.” Demonstration method differs from experimenting because it is content oriented while experimenting is process oriented. It simply reinforces the previous learning and aims toward a summary of ideas while experimenting aims to solve problems and gain new learning. Therefore, Hidalgo (2000) summarizes, “demonstration method is used when time and equipment are limited and the process can be described as complicated or difficult while experimenting is used when the purpose of the classroom is to develop resourcefulness among students.” Demonstration method is considered as an excellent method to motivate and arouse the interests of students in introducing any new lesson. On the other hand, Garcia as cited by Acero, et al. (2000) called demonstration method as imitative method where learning a skill is faster and more effective since the students are shown how the job is done by using the actual tools, machines, and materials. Discovery Method. The discovery method is a teaching strategy in which objectives help the student to learn through self-discovery (Corpuz, et al., 2003). In this type of classroom, the teacher prepares a class situation where students are led to find answers or solutions to a problem on their own. Further, discovery method employs the inductive approach wherein the teacher asks thought-provoking questions before performing an experiment to allow for self introspection and analysis.

101

The discovery classroom, says Hidalgo (2000) demands for the teacher to cultivate among the students an attitude of trying to solve problems on their own even if it would result in failure than not trying at all. Hidalgo (2000) further suggested that teachers allow for independent learning to allow for more learning. With all these benefits an educator gets from the discovery method, Corpuz, et al. (2003) opined that it is an extremely effective method. The most important gain it gives to the learners is the feeling of satisfaction and joy for the students in discovering new learning and concepts. Inquiry Approach. The goal of inquiry teaching according to Salandanan (2002) is “to make children learn how scientists learn, and in the process, learn science.” Further, it aims to encourage students to rely to a greater extent on their own resources. In fact, in using the inquiry approach, learning leads to the attainment of one of the most significant outcomes of science teaching- that is, developing a scientific mind while not undermining the desirable social values. Students develop traits such as critical-mindedness, objectivity, and rationality while engaged in an inquiry approach. They become more cooperative, tolerant, and considerate in dealing with others because of constant involvement in group activities, thereby making them highly motivated. In his study, Straatman (2006) made use of inquiry methods in laboratory activities and demonstrations along with traditional teacher-focused methods. A variety of data collection methods were used to investigate changes that occurred in his approach to teaching chemistry especially in relation to questioning strategies. The study revealed effects of inquiry techniques on students‟ problem-solving and logical thinking skills. His study enabled him to take a more in depth look at how teaching methods affect student learning. Process Method. Hidalgo (2000) defined process as “a method of doing something; a systematic and interdependent action of things related to a discovery approach where at the end things are attained.” In the science classroom, scientific activities are processes or methods and the scientific information is the product of the process. Laboratory investigation, for example, is a process of making a discovery where students become more actively involved. Small Group Instruction. This method of instruction enables the teachers to give more individual attention to each student‟s learning needs. In small groups, the students become more actively involved in their own learning and participate more freely in discussions. (Hidalgo, 2000). In this method, students leadership is develop aside from they learn the skills of discussion and group processes. This method enhances cooperation, team work, and group motivation among students. Principles of Best Practice Learning Zemelman, Daniels & Hyde (2005) claims that there are classroom practices which need to be enhanced to be implemented more often while there are those which should be implemented in a lesser frequency. In a bigger concept, therefore, there must be less implementation of teacherdirected instruction like lecturing; student passivity like sitting, listening, receiving, and absorbing information; presentational or one-way transmission of information from teacher to student; prizing and rewarding of silence in the classroom; classroom time devoted to fill-in© 2014 The authors and IJLTER.ORG. All rights reserved.

102

the-blank worksheets, dittos, workbooks, and other “seatwork”; student time spent reading textbooks and basal readers; attempts by teachers to thinly “cover” large amounts of material in every subject area; rote memorization of facts and details; emphasis on the competition and grades in school; tracking or leveling students into “ability groups”; use of pull-out special programs; and use of and reliance on standardized tests. In contrast, he recommended the frequent or more implementation of experiential, inductive, hands-on learning; active learning, with all the attendant noise and movement of students doing, talking, and collaborating; diverse roles for teachers, including coaching, demonstrating, and modeling; emphasis on higher-order thinking; learning a field‟s key concepts and principles; deep study of a smaller number of topics, so that students internalize the field‟s way of inquiry; reading of real texts: whole books, primary sources, and nonfiction materials; responsibility transferred to students for their work like goal setting, record keeping, monitoring, sharing, exhibiting, and evaluating; choice for students (e.g., choosing their own books, writing topics, team partners, and research projects); enacting and modeling of the principles of democracy in school; attention to affective needs and varying cognitive styles of individual students; cooperative, collaborative activity; developing the classroom as an interdependent community; heterogeneous classrooms where individual needs are met through individualized activities, not segregation of bodies; delivery of special help to students in regular classrooms; varied and cooperative roles for teachers, parents, and administrators; reliance on descriptive evaluations of student growth, including observational/anecdotal records, conference notes, and performance assessment rubrics. From those practices which he recommended to be implemented more and those which need to be implemented less, he then identified thirteen principles characterizing a model education. These principles are interrelated and are actually influencing each other. These practices, he pointed out are student-centered, experiential holistic, authentic, challenging, cognitive, developmental, constructivist, expressive, reflective, social, collaborative and democratic. In a student-centered learning environment, teachers must consider the real interest of students, taking their own questions into precedence over other selected content. At some point, it involves building on the natural curiosity of students and asking them what they want to learn. Teachers must guide their students in solving their own questions by structuring for them widening circles of experiences and investigations. At this point, the teachers serve as facilitators understanding deeply the needs and experiences of their students in order to design enjoyable and engaging activities. In an experiential learning, students are given the opportunities to experience the most powerful and natural form of learning which is acquired through doing instead of just hearing. Students must be engaged in active, hands-on, and concrete experiences such as conducting experiments, going on field trips to investigate natural settings, pollution problems, and laboratories at nearby factories, universities or hospitals. If planned properly, holistic learning is possible when students encounter whole ideas, events, and materials in purposeful contexts, instead of studying isolated subparts from actual use. Information and ideas must not be presented to students in small “building blocks” because this © 2014 The authors and IJLTER.ORG. All rights reserved.

103

part-to-whole approach undercuts motivation for learning since students don‟t perceive why they are doing such things. Students should not be deprived of an essential condition for learning, that is encountering material in its full, lifelike context. Linking learning to real life concepts is considered authentic teaching which integrates real, rich complex ideas and materials in contrast to the lessons or textbooks that disempower students. Experts all agree that learning becomes meaningful when students are faced with genuine challenges, choices and responsibility while learning independently. There is cognitive learning when students acquire true understanding of concepts using higher order thinking associated with various fields of inquiry and through self monitoring of their cognition and mental processes. Learning activities should fit the developmental level of students, therefore they must be taught in a constructivist approach by making them recreate and reinvent every cognitive system they encounter. Students must also be trained to employ regularly the whole range of communicative media to fully express ideas, construct meaning and remember information. Opportunities to reflect, debrief, and abstract from students‟ experiences what they have felt and thought and learned should also be provided for a more effective learning environment. Teachers need to create classroom interactions which are socially constructed to show that learning can be achieved through collaboration to eliminate competition and individualistic approaches. Democratic learning makes the classroom a model community where students learn what they live as citizens of the school. Principles of Effective Laboratory Experiences In 2008, Jona, et al. Emphasized the four curriculum standards that were identified as principles of effective laboratory experiences by the National Research Council (2005). These are clearly communicated purposes; sequenced into the flow of instruction; integrated learning of science concepts and processes; and ongoing discussion and reflection. In National Research Council‟s landmark study (2005), laboratory experiences are considered effective if they have clear learning goals that guide the design of the learning experience. The teacher must communicate clearly the purpose of the activity so that students can successfully carry it out and achieve the desired goals set. It is recommended to design an inquiry activity where students learn specific concepts which are clearly communicated to them throughout the learning and discovery process. At the end, they will be assessed on their ability to achieve the instructional purpose of the activity. Still in the results of the study, laboratory experiences are said to be sequenced into the flow of classroom science instruction if they are explicitly linked to what has come before and what will come after. Scientific investigations when integrated into a well-designed sequence of instruction will serve as an instructional purpose that is consistent with the objectives of the learning unit. To achieve this, laboratory teachers must give their students ample time to discuss the activities they are engaged in during laboratory period and reflect on the meaning they can make out of them. They must also be given opportunities to formulate hypotheses before experimentation so that they can reflect on their ideas after the complex process of experimentation. A knowledge-centered environment is created when students reflect on their own learning progress, when they identify, monitor and regulate their own thinking and © 2014 The authors and IJLTER.ORG. All rights reserved.

104

learning. Metacognitive strategies must be implemented by laboratory teachers in a way that students can determine what they already know and what else they need to know in any given situation including when things are not going as expected. Teaching of Nature of Science Crowther, Lederman & Lederman (2005) suggested some teaching strategies that will highlight the teaching of nature of science. Teachers can design lessons around science topics or concepts that have changed over time and the instruction must be explicit on how knowledge has changed and why. Through this, students will learn that scientific knowledge in and of itself is not static and that with new information, scientific theories can change. However, students must be taught also that some laws in science have stood the test of time. In teaching scientific laws, teachers must emphasize how these laws describe nature and how things act under certain conditions. Also, the manner a teacher poses questions inside the classroom lead to investigation and experiments then will eventually lead to conclusions - but still there are many different pathways that scientists take. In conclusion, it is incorrect to assume that all scientific investigations follow the same set and sequence of steps. One of the reasons why knowledge is subject to change is that these different types of investigations provide different information and evidence concerning the natural world, hence, different learning outcomes. Motivation and Attitude Toward Science Students in the tertiary level of education are required to take an introductory science course as general education subject. It is not surprising that some students do not succeed because they just enroll the subject for requirement purposes rather than taking the subject because they have a passionate interest for learning it. It is the concern of the science instructor to shape the attitudes of students toward science so that they leave their classes with positive views of the discipline. A possible reason why students seem not to learn much concepts on science subjects is that most instructors focus primarily on content of the subject instead of helping learners cultivate a holistic attitude towards the subject. The pedagogic strategies of the teacher also play an important role in how students will appreciate science subjects. Still, there are other factors which may result in negative attitudes towards science: â&#x20AC;&#x153;lack of needed skills to learn and apply scientific concepts, lack of motivation to work hard in science classes, home backgrounds, school and classroom environments, biases of peer groups, the media's portrayal of scientists, and students' perceptions of rewards associated with learning, science anxiety, the fear of science learning, and apprehension toward scientists and sciencerelated activities.â&#x20AC;?

Previous experiences of students also affect the attitude of students toward science as a subject and as a body of knowledge. It has been argued that to build motivation and positive attitude among students, there must be a good understanding of the content being taught, therefore the teacher must find ways to probe knowledge which the students have previously constructed. If ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

105

the prior knowledge of the student is insufficient, inaccurate and in conflict with what is being taught, the teacher, then must guide the students in reconstructing their knowledge. Learning that will give the opportunity to student to reconstruct their own conceptual knowledge and understanding leads to a lasting improvement in students' attitudes greater chances of success in their studies and lives. The attitude of a student toward a subject has something to do with the motivation made by the teacher in introducing a lesson. Positive attitudes develop if a student is highly motivated and this can be done by the teacher through improving the teaching practices and by showing to the students the relevance of the topic to their everyday lives. Teachers must create a learning environment that will encourage and inspire the students not only to come to class regularly but also to have a desire to learn and enjoy learning. Movahedzadeh (2011) suggested some teaching practices that will motivate the students and will lead to their positive attitude towards the subject. Teachers must consider the preconceptions of students regarding the topic by asking them of their personal views so that diversity of views in the class lead to a deeper discussion about the process of doing science, the application of scientific discoveries, and the impact of science on society. The relevance of science can be further emphasized to students by mobilizing the scientific and engineering research community. When students are given access to practicing scientist and engineers who can provide them with valuable information on careers and studies, students would increase their interest and enthusiasm in learning science concepts. For the students, inviting experts in the class would help to put the subject into context and make classroom activities more exciting. It is not only school visits of professionals but also visit of students to the workplaces of these professionals that will help them to learn about and understand specific professions. Brodie (2006) even proposed some projects that will increase student participation, motivation and success by involving the whole scientific community. These are the “Researchers in Residence Project,” “Express Yourself Conferences” and creating Centre for Science Education. In the “Researchers in Residence Project,” creative research talents such as PhD students and post-doctoral researchers in science, technology, engineering and mathematics share their passion for their field of specialization to the students for the purpose of igniting a fresh interest for science among young people. Through the involvement of these research talents who act as positive role models, significant change occur in how people view scientific researches, scientists, and technical aspects of science. The project dubbed as “Express Yourself Conferences” hopes to enable students to present the findings of their own science investigations. In these designed conferences, students are given opportunities to communicate and share their ideas with other students, teachers and researchers; present research papers in seminars chaired by researchers in residence; present and host displays of their investigations; and participate in other activities, such as discussing their work with experienced researchers, attending keynote lectures and demonstrations, and participating in practical workshops. On the other hand, the Centre for Science Education is created for the purpose of inspiring and capturing the imagination of young people in science © 2014 The authors and IJLTER.ORG. All rights reserved.

106

through the development of „creativity-rich‟ resources and activities. In this way, students will be motivated to pursue science courses and to be successful in their chosen career. With the same purpose and intention, Wilson, Cordry & Uline (2004) said that participation of students in science fairs will promote positive attitude towards science because in doing science fair projects, students find enjoyment in applying scientific method thereby promoting their interest in science. It also develop student‟s sense of personal capabilities and qualities and appreciation for nature and the relevance of science in daily life. A study was conducted by Hall (2006) on the techniques to encourage students to confidently contribute to their lab groups in science classrooms. She observed her students during laboratory and analyzed how they conduct themselves. A new system was implemented where students were assigned specific roles during laboratory and a grade was given based on their level of contribution to the lab group. As a generalization, her data showed a positive relationship between the implemented treatment and more active science lab participation.

Another study, that of Washtak in 2006, examined the use of technology in both the lecture and laboratory settings of a high school chemistry class. The focus of her study was on the student motivation and ability to learn from technology. As variables of the study, PowerPoint, SMART Board and computer animations were used in the lecture setting. Logger Pro software and individual laptops were used in the laboratory setting. The assessment techniques included pre- and post-tests, surveys, teacher journal, analysis of specific test and laboratory questions, student interviews and comparison of test scores, results of which found students motivated by technology and were able to learn from them. In an action research spearheaded by Nordick (2006), he introduced a unit plan that included detailed lecture guides. Each guide contained objectives, key terms, and important topics for students to follow during lectures. Such lecture guides were organized into unit plans and presented to the students prior to the beginning of each unit. In another study, the effect of having older students teach science concepts to younger students was given focus. Muchmore (2006) looked into the argument whether students reach higher levels of achievement when they take on the teacher role versus that of student roles. After observing a dramatic rise in student participation in cooperative groups, Muchmore (2006) recognized the importance of active student engagement and responsibility for learning. Evidence indicated that peer assisted learning did, in fact, increase student interest, however, a thorough investigation on better ways for students to retain learning was recommended. Mentzer (2006) also explored on how student motivation, as defined by validity and selfefficacy, was affected by journaling, and if that motivation affected the time it took students to get ready for class. His study revealed that a short period intended for journal writing before classes begin, spell a big difference in students‟ learning.

107

Eastwell and Rennie (2002) for their part used a quasi-longitudinal case study to determine the effects on secondary students of participation in a program of enrichment and extracurricular science activities in terms of their interest and enjoyment in being involved in science activities, their motivation to continue to participate in science, and their perceptions about scientists and about the role of science in society. A strong positive relationship was found between changes in students‟ interest and enjoyment and changes in their motivation, and both these variables increased, in an overall sense for the combined student population, during the study period. All students generally held a high perception of both the normality of scientists and the importance of science in society throughout the study period. Participation in science activities impacted overall positively, but to varying extents for different activities, on all four dependent variables. Suggestions for the structure and/or conduct of competitions, excursions, and practical work, including the design of museum exhibits, and implications for further research are presented. A study with focus on integrating graphic organizers in the attitude, perception and achievement of students in chemistry was done by Torres (2009). The results of the study proved that student‟s prior knowledge of outlining format allowed them to more easily utilize and organize information. In addition, the sequencing and planning of instruction by the teacher in an outline format allowed students to extract and synthesis information in an organized manner. Science Laboratory Skills A well-designed laboratory activity has the potential to motivate students, support meaningful learning of concepts, and develop manipulative competencies among students. According to Moni, Hryeiw, Poronnik, Lluka & Moni (2007), students must be taught of the differences among “knowing about” a topic, “knowing how” to complete a skill, “showing how” to complete a skill, and finally “doing” the skill. This is possible through integration of skills development with conceptual learning. Skills were considered as “embedded elements of the more complex laboratory practices of problem-based or case-based inquiry learning tasks.” (Moni, et al., 2007). Skills can be further differentiated from practices by saying that skills represent “hands-on” or “doing” while practices represent the combination of “hands-on‟ and “minds-on.” With this difference, teachers must teach skills to students with the expectation that competencies in skills would support open-ended, student-driven explorations. Kanli and Yagbasan (2008) identified laboratory principles for teachers which cover laboratory teaching approaches to develop science process skills and conceptual achievement among students. According to them, teachers must practice the principles of knowing how to excite, explore, explain, elaborate, extend, exchange, and evaluate students. They enumerated teaching practices under each principle. To excite students, teachers must provide the students with the thoughts of the first scientist and make them feel like them; intrigue to ensure students‟ participation (a simulation may be watched about experiments); make a spark about the subject; try to discover what students know about new concept or subject; ask questions that may confuse minds (create unbalance); and ask questions about misconception. In the field, science teachers explore when they provide environment for concrete, tangible activities that include skills and concepts; ask probing question; listen and observe students; just © 2014 The authors and IJLTER.ORG. All rights reserved.

108

play the role of a good adviser or coach in students‟ journey to cognitive balance; create a rubric that will evaluate the skills of students about determining variables and establishing hypothesis; and ensure that the students save the data they acquired correctly. Another laboratory principle students must learn from the classroom interaction is the ability to explain. Teachers must encourage students to explain and determine concepts; demand explanations and proofs from students; emphasize that students use the data they acquired to make reasonable explanations; and bring forward new concepts by taking students prior experiences and making explanations and definitions. This is possible, Kanli and Yagbasan (2008) noted if the teacher explores on the elaboration of ideas and topics in the classroom. A teacher can elaborate the topic if she encourages students to apply concepts and skills into new situations; demand from students to use concepts, explanations and definitions with the previously acquired ones. A teacher values exploration in her classroom if the students become aware that proofs and data are necessary in proving one‟s opinion or point. The classroom, as a main source of knowledge, must ensure that the students extend their learning by guiding them in associating present concepts with other fields and/or other concepts/subjects. Further, a teacher may also ask research questions to help them in associating the current lesson to concepts/subjects of other fields. Teachers must also teach to their students the principle of exchange by preparing proper environment for students to discuss their ideas with their friends; observing and listening to the students who are sharing their knowledge; and ensuring the interaction within student groups, competing student ideas. Lastly, teachers must know how to evaluate their students‟ learning by observing students that apply new concepts and skills; evaluating knowledge and skills of students; searching the reasons of students‟ changes of attitudes and ideas; letting the students evaluate their knowledge and group process skills; and asking the open-end questions such as “Why did you think like that?”, “What is your proof for this?”, “What do you know about ....?”, “How do you explain...?” Achievement in Science Successful teaching and learning of science is the product of the correct use of an appropriate teaching method whose objectives focus on the high achievement of the learners. One of the challenges of a science teacher is how to facilitate learning which will address the difficulties of the learners in assimilating concepts. Wachanga and Mwangi (2004) stated that knowledge about teaching methods affect students‟ learning may help educators in selecting methods that will improve teaching quality, effectiveness, and accountability to learners and the public. To be an effective teacher, students must be given opportunities to learn and technically manage instruction. Effective learning and students‟ achievement will be enhanced if students are allowed to use their hands, eyes, ears and their mind. In the Cooperative Class Experiment Teaching Method, Wachanga and Mwangi (2004) reflected on how students acquire greater mastery of the subject matter because of peer teaching. Aside from generating better intergroup relations, “the shared responsibility and interaction in this method result in better self-images for students with histories of poor achievement” (Wachanga © 2014 The authors and IJLTER.ORG. All rights reserved.

109

and Mwangi, 2004). Slow learners, moreover, benefit in this cooperative learning method because fast learners share their ideas so that the others learn the ideas in depth and remember them longer. In a science classroom, on the other hand, well-managed laboratory activities in enhancing student‟s learning of science concepts result in an enhanced interest in science (Claveria, 2002). Students prefer experimenting, demonstrating, film showing using instructional media, making diagrams, drawings, painting and sketches, gathering clippings of inventions and significant science events classifying plants and animals through pictures and observing prepared slides and specimens to enhance student‟s interest in science, collecting rocks, taking photographs of nature, collecting variations of plants and animals and preparing botanical gardens. Furthermore, she recommended that student‟s interest in science should be maintained by inviting young scientists from whom they can draw further inspiration. The teachers should be updated on the new techniques and trends in teaching science subjects, so they could be more creative and resourceful. In addition, the teachers should teach their students to improvise science materials that they can use in the laboratory activities. To prove such theory, Ricardo (2008) conducted a study on how chemistry is taught in the public secondary schools to determine the factors affecting students‟ performance. To gather enough data, Ricardo (2008) did actual observations, supplemented by interviews with the teacher, head teacher, and principal. In his observation, the predominant teaching strategy used by most teachers is still lecture-discussion, a traditional method. A significant correlation existed between the performance of students in the achievement test and the method by which chemistry is taught. The effects of Task-Oriented Learning Approach (TOLA) on chemistry achievement among selfhandicapping students were analyzed by Reyes in 2002. TOLA consisted of several components like task-on-activity, developing collaborative skills, classroom management strategies and self-assessment. Self-handicapping students are those who are complaining about the subject, avoiding seeking help, avoiding taking risk in difficult task, withdrawing effort, reducing performance setting, lacking in preparation, procrastinating and making excuses. The results showed that TOLA consistently improved the achievement of selfhandicapping students in chemistry. Roble-Estrella (2009) suggested some teaching strategies in chemistry laboratory that will enhance the performance of students in the laboratory in her research. For her, the teacher must state the objectives clearly in every laboratory activity; emphasize major ideas as they are presented; provide step-by-step directions when necessary; check for understanding at intervals before proceeding to the next major idea or concept; provide concrete examples to explain and reinforce information; use appropriate scientific vocabulary; must be specific and precise by referring to concrete objects and events; ask questions or obtain work samples before proceeding to the next procedure; call on slower students and non-volunteers and print out necessary parts of the activity. Her study supports other researches on best practices to enhance the learning capabilities of students enrolled in science subjects.

110

In the study by Palada (2002), the level of performance of students in chemistry is found to be significantly influenced by their perception about chemistry and by the teacher‟s most preferred teaching practice as perceived by the students. She further concluded that teaching practice influences the academic performance outcome of the teaching process. “Knowledge of content and understanding of student‟s strengths and weaknesses along with appropriate teaching practices can improve teaching and result to higher student‟s achievement” (Palada, 2002). For her part, Leonor (2007) used the Scientific Inquiry Method to determine the extent of academic performance in chemistry of students. She enumerated techniques which are considered to be included in the scientific inquiry method. These include the limitation in the use of lecture and direct instruction in presenting the lesson; use of student‟s prior knowledge as basis for introducing new concepts; exploring student‟s interest to make learning relevant and meaningful; using inquiries and investigations to anchor new information to previously held knowledge; initiating classroom dialogue and discourse by posing essential or starter questions; asking questions that require higher order thinking skills and critical skills; using wait time techniques appropriately and not interrupting students in the middle of their questions or answer; rephrasing students‟ questions and answers; establishing everyday routines for group interaction; arranging student‟s desks for collaborative work in small groups; focusing the lesson on engaging and relevant problem-solving situations; encouraging students to design and carry out their own investigations; integrating science content with process skills and problem solving strategies; valuing student‟s responses and viewing wrong answers as an open door to their misconceptions; encouraging students to use concept maps; graphic organizers; and drawings of models to explain and demonstrate newly acquired knowledge. Synthesis The present study attempted to explore the different practices in chemistry laboratory instruction that were expected to attain positive attitudes of students towards learning chemistry, competencies in the laboratory skills and high achievement in the subject. In particular, this present study bears similar concepts and focus to that of Hall (2006); Washtak (2006); Mentzer (2006); Eastwell and Rennie (2002); and Claveria (2002) in that they looked into how teachers deliver their lessons for the enhancement of the interest of students in learning science. There is a similarity between the present study and the studies of Palada (2002); Straatman (2006); Nordick (2006); Ricardo (2008); and Roble-Estrella (2009) because they focussed on the teaching strategies for the development of various skills of students in science subjects. Lastly, Reyes (2002); Muchmore (2006); Leonor (2007) and Torres (2009) focussed their studies on how teaching practices influence the achievement of the students in science subject similar to the present study. Research Design This study employed the descriptive design, particularly the qualitative-quantitative method of research which according to Alasuutari (2004) is that type of research which “involves checking of data collected via one method with data collected using another.” The main objective of the © 2014 The authors and IJLTER.ORG. All rights reserved.

111

study was to propose a model of a teaching-learning process based on the best practices in chemistry laboratory instruction. The descriptive design was the most appropriate design to be used in determining the best practices in chemistry laboratory instruction that will attain the goals of science lab instruction. These were investigated through qualitative method in which a focus group interview of faculty and students followed by class observations were conducted. Quantitative method was used in determining the students‟ manifestation of the attainment of the seven goals of science laboratory instruction through administration of instruments such as Attitude/Motivation Instrument, Practical Test and Achievement Test.

Research Locale This study was conducted at Lyceum of the Philippines University in Batangas, Laguna, Cavite and Manila. LPU, an institution of higher learning, inspired by the ideals of former Philippine President Jose P. Laurel, is committed to the advancement of his philosophy and values “Veritas et Fortitudo” (truth and fortitude) and “Pro Deo et Patria” (for God and Country). Guided by its vision and mission, it aims to provide quality education through its three-fold function of instruction, research and community extension. It offers various programs in science where General Chemistry is one of the basic subjects. General Chemistry is a five unit subject comprising of three hours lecture and three hours laboratory in a week. Participants of the Study This study involved teachers and students from the four universities of the Lyceum University System (LPU in Batangas, Laguna, Cavite and Manila). The participants were chemistry faculty and their students enrolled in General Chemistry during the second semester of the school year 2011 - 2012. The chosen faculty have been teaching chemistry for a minimum of three years and were either chemical engineers or chemists by profession. Most of them have Masters‟ degrees. The students, on the other hand, belong to degree programs such as B.S. Physical Therapy, B.S. Psychology, B.S. Engineering, and A.B. Mass Communication. Majority of the studentrespondents were first time takers of the subject, however, some of them were repeaters. Two to six chemistry faculty from each university and a group of three to nine chemistry students participated in the focus group interview. The profile of faculty was secured from the Human Resource Office of the university to find out who among them are science majors. The faculty-respondents were selected from the faculty of sciences. This faculty is responsible for teaching chemistry in the university. The students who were interviewed were selected from the students who were enrolled in General Chemistry during the second semester. About thirty percent of the class or three to nine students were selected randomly from the class based on the total number of students in each chemistry class. Class observations were also done to further validate data gathered from interviews and focus group discussions. A total of four faculty from the four universities were observed and a total of 80 students responded to the administered questionnaires. The Achievement Test, Practical Test and Attitude/Motivation Questionnaire were administered to the intact class of the observed faculty at the end of the semester. © 2014 The authors and IJLTER.ORG. All rights reserved.

112

Data Gathering Instrument To approximate the best teaching practices employed by the faculty in teaching chemistry laboratory, five instruments were developed by the researcher and were content validated by experts. They were developed after a thorough review of conceptual literature about best teaching practices. Focus Group Interview Questionnaire for Faculty and for Students. This instrument was prepared to approximate if the chemistry faculty is implementing practices in teaching chemistry laboratory which will lead to the attainment of the seven goals of science laboratory instruction (See Appendix H- Appendix I ). It was a structured questionnaire consisting of open-ended questions which were answered by a group of chemistry faculty and a group of students. The items were classified according to the goals of science lab instruction. The items were intended to get information on the enhancement of mastery of subject matter, the development of scientific reasoning, the development of the students‟ understanding of the complexity and ambiguity of empirical work, the development of practical skills, the students‟ understanding of the nature of science, the cultivation of interest in science and in interest in learning science, and the development of teamwork skills. The questionnaire was presented to the advisers for comments and suggestions and then to three chemistry experts for face and content validity. Observation Checklist. This checklist was used in conducting unannounced class observation to validate if the faculty concerned is practicing the good instruction in conducting chemistry laboratory classes mentioned in focus group discussions and to see if the seven goals of science instruction are manifested in their teaching practices (See Appendix J). The items in the observation checklist were classified according to the goals of science laboratory instruction. The content of the checklist was similar to the content of the focus group interview questionnaire. The items were enumerated so that the observer can take note of the practices implemented by the faculty after thirty minutes, after two hours and at the end of the class. Like that of the previous instrument, the questionnaire was presented to the advisers for comments and suggestions and to three chemistry experts for face and content validity. The following scale range was used to interpret the data gathered by the questionnaire. Scale Range Verbal Interpretation 2.28 – 3.00 always practiced 1.52 – 2.27 often practiced 0.76 – 1.51 sometimes practiced 0 – 0.75 never practiced Note: A mean of 1.52 to 3.00 is an indication that the faculty is implementing the best teaching practices that will lead to the attainment of the seven goals of science lab instruction. Attitude/Motivation Instrument. To approximate the extent by which students manifest the attainment of the goals of science laboratory instruction in their attitude and motivation, an attitude/motivation instrument was constructed by the researcher. This Likert Scale Instrument (See Appendix L) consisted of 15 items which were categorized as to the views of students about learning chemistry (attitude) or interest in learning chemistry (motivation). Seven items © 2014 The authors and IJLTER.ORG. All rights reserved.

113

of the questionnaire determined the student‟s attitude towards chemistry and eight items determined their motivation. The instrument also approximates the students‟ understanding of the nature of science. The students were asked to chose from a five-scale option such as strongly disagree, disagree, neither agree nor disagree, agree and strongly agree. Equally, face and content validity were established for this instrument. Interpretation of the data yielded by this attitude/motivation questionnaire was based on the following scale range: Scale Range 4.51 – 5.00

Verbal Interpretation Very positive attitude Very highly motivated Very much understood

3.51 – 4.50

Positive attitude Highly motivated Much understood

2.51 – 3.50

Moderately positive attitude Moderately motivated Moderately understood

1.51 – 2.50

Negative attitude Lowly motivated Not so understood

1.00 – 1.50

Very negative attitude Very lowly motivated Not understood Note: A mean of 3.51 to 5.00 is an indication of a best teaching practice of a faculty that will lead to the attainment of interest in science and in learning science and understanding of the nature of science. Practical Test. To measure the extent by which students manifest the attainment of the goals of science laboratory instruction in their laboratory skills, a Practical Test was constructed (See Appendix K). This Practical Test approximates not only the practical skills of the students but also their teamwork skills and understanding of the complexity and ambiguity of empirical work in a chemistry classroom. The instrument used in this test was a checklist which included items on common laboratory techniques such as handling liquids and measuring volume, handling solids and weighing, bunsen burner manipulation, heating substances in a test tube, doing evaporation, and doing filtration. It also included items that indicate if students consider safety precautions in performing experiments, if they exhibit teamwork skills and understand complexity and ambiguity of empirical work. Attached to the checklist was the list of materials and the procedure followed by the students in taking the Practical Test. The questionnaire was presented to the advisers for comments and suggestions. To ensure that the questionnaire was valid three chemistry experts were consulted for face and content validity.

114

The practical skills, teamwork skills, and the students‟ understanding of the complexity and ambiguity of empirical work was categorized and verbally interpreted as recommended by experts as: Scale Range Verbal Interpretation 76 % - 100 % Highly competent / Highly understood 51 % - 75 % Competent / Understood 26 % - 50 % Moderately competent/Moderately Understood 0 – 25 % Not competent / not understood Note: A scale range of 51% to 100% is an indication that the faculty is implementing best teaching practices that will attain practical skills, teamwork skills and understanding complexity and ambiguity of empirical work. Achievement Test. This was a Concept-Application-Procedural (CAP) Test developed to approximate whether the students had attain the mastery of subject matter and scientific reasoning (See Appendix M). Twenty one (21) items of this multiple choice test were categorized under mastery of subject matter while thirty eight items on scientific reasoning. Each question consisted of four options of which students encircled the correct answer. The questions were taken from the seven experiments performed during the semester such as Measurement; Changes in Matter; Classifications of Matter; Laws of Chemical Change; Types of Chemical Reactions; Solutions; and Classes of Compounds. Similarly, this questionnaire was subjected to face and content validity.The original 100 items were then reduced to fifty nine items after item analysis was done. The researcher also deleted the items which were not relevant to the topics. The data gathered by this instrument was interpreted using the following scale: Scale Range 76 % - 100 % 51 % - 75 % 26 % - 50 % 0 – 25 %

Verbal Interpretation high level of mastery / high level of scientific reasoning average level of mastery / average level of scientific reasoning low level of mastery / low level of scientific reasoning no mastery / no scientific reasoning

Note: A scale range of 51% to 100% is an indication that the faculty is implementing the best teaching practices that will lead to attainment of mastery of subject matter and scientific reasoning.

Data Gathering Procedure The study was conducted in successive phases. The details of activities can be referred to the Gantt Chart of Activities. Phase I - Planning Stage. This stage involved the review of literature on the study and development of the instruments used in the study. A thorough reading of books, journals, theses and dissertations together with internet resources was made to gather theories and © 2014 The authors and IJLTER.ORG. All rights reserved.

115

concepts related to best teaching practices. From the constructs gathered, five instruments were developed such as Focus Group Interview Questionnaire for Students and Faculty, Observation Checklist, Attitude/ Motivation Instrument, Practical Test and Achievement Test. The prepared instruments were presented to the advisers for comments and suggestions. To ensure that the instruments were valid, chemistry experts were consulted for face and content validity. Comments, suggestions and recommendations were considered to refine the instruments. It took the whole of the first semester to develop and validate the instruments. Phase I also entailed the securing of approval from Lyceum of the Phil. University in Batangas, Laguna, Cavite and Manila to conduct the study in their respective locuses. Phase II - Gathering of Qualitative Data on Best Teaching Practices. This phase consisted of the focus group interview and class observations. Separate focus group interview for the faculty and students were conducted at the beginning of the second semester. Since the classes for the second semester started on the second week of November 2011, the focus group interview for faculty in the four universities was conducted on the last week of November of the same year. Focus group interview for students was conducted on the first week of December, 2011. Two to six chemistry faculty in each university were interviewed while a group of three to nine chemistry students were included in the focus group interview for the students. The students interviewed were the current students of the teachers who were observed for the study. The interview lasted for one hour. It was video-taped and the responses gathered were analyzed and interpreted qualitatively. The interview and focus group discussions were done to identify the teaching practices implemented by the faculty in order to attain the seven goals of science lab instruction. To validate that the faculty was implementing the teaching practices that will attain the goals of science laboratory instruction, unannounced class observations were also conducted. Only one faculty per university was observed for this purpose. Class observation in each university was conducted for three different experiments performed by the class. Each observation lasted for three hours and was documented by photographs and video tapes. During the observation, the observable items from the instruction made by the faculty were checked. Each faculty got a score of one every time the item was observed and a perfect score of three if during the three observations made, the faculty always demonstrated such item. The faculty got zero if the item was never observed. The mean of the scores for each item of the four observed faculty was computed and was verbally interpreted according to the scale range recommended by experts. Class observation started on the second week of December 2011 and lasted until the last week of February 2012. Phase III - Gathering of Quantitative Data on Best Teaching Practices. This phase consisted of determining whether the observed faculty implemented the goals of science laboratory instruction as manifested in the studentsâ&#x20AC;&#x; attitude, motivation, laboratory skills and achievement. The students were asked to answer Attitude/Motivation Test, did Practical Test and answered the Achievement Test at the end of the semester. The Practical Test was given on the first and second week of March 2012 while the Attitude/Motivation Test was administered together with the Achievement Test on the third week of March of the same year.

116

The Practical Test was conducted on the intact class of the observed faculty with the assistance of their teacher. The class was divided into groups and each group was given an allotted time of thirty minutes to finish the test including instructions on how they will do the test. Each student was given a copy of the procedure before the test. All materials such as laboratory equipments and reagents were made available already on the work table before the start of the test. During the test, the students were rated by checking the items which were observed from the group. The number of groups who demonstrated a particular skill was counted and the corresponding percentage out of the 18 groups of students was computed. The Attitude/Motivation instrument was administered to the intact class of the observed faculty in each university at the end of the semester for a period of thirty minutes including instructions on how students will answer the instrument. For scoring purposes, a score of 5 was given if the respondent strongly agree to the item, 4 if agree, 3 if neither agree nor disagree, 2 if disagree and 1 if strongly disagree. The mean of the scores for each item of the 80 student respondents was computed and was verbally interpreted according to the scale range recommended by experts. The Achievement Test was administered to the intact class of the observed faculty at the end of the semester for a period of ninety minutes including the instructions on how students will answer the test. The perfect score for mastery of subject matter was 21 and for scientific reasoning was 38. The mean of the scores of the 80 student respondents and the corresponding percentage were computed and verbally interpreted according to the scale range recommended by experts. Data Analysis Procedure Content analysis of the responses of students and faculty in the FGI were done by deduction and induction. From here, it was determined whether the teaching practices implemented by the faculty conformed with the seven goals of science lab instruction. To analyze the teaching practices observed among the faculty, the researcher made use of statistical mean. Frequency, percent, mean and standard deviation were used to describe the extent by which students manifest the attainment of the goals of science lab instruction in their attitude and motivation, lab skills and achievement. These statistical treatments were used to analyze the responses and performance of students in the Attitude/Motivation Test, Practical Test, and CAP test. Results and Discussion I.

Teaching Practices Employed by the Faculty in Teaching Chemistry Laboratory in Order to Attain: A. Mastery of Subject Matter

Table 1. Practices Employed by Faculty in Designing their Laboratory Instruction Group of Respondents Faculty of LPU 1

Laboratory Practices of Faculty Basing on the syllabus; patterned with the sequencing of topics discussed in the lecture; making it simultaneous with the topic in the lecture; following the recipe-type of procedure

117

Students of LPU 1

Faculty of LPU 2

Students of LPU 2

Faculty of LPU 3

Students of LPU 3

Faculty of LPU 4

Students of LPU 4

Requiring students to do experiments which are similar to the topics discussed in the lecture; simply following the step-by-step procedure in the lab manual Doing experiments after discussing the concept in the lecture; performing experiments which are related and simultaneous with the topics discussed in the lecture; following the recipe type Discussing first the concept in the lecture and then applying it in the experiment; performing all the experiments related to the topics in the lecture; following the given procedures in the lab manual Performing experiments while discussion of the topics in the lecture is going on; designing it in a way that thereâ&#x20AC;&#x;s close supervision of studentsâ&#x20AC;&#x; learning simultaneously in lecture and lab; sometimes implementing discovery approach with investigative type of procedure Discussing the topic while performing experiment that is related to it; sometimes asking students to perform first the procedure given in the lab manual and then asking students what they learned from the result. Applying deductive approach; providing procedures in the laboratory manual; simply following the cook book style in the manual; doing experiments simultaneously with the discussed topics in lecture Requiring students to have individual laboratory manual; assigning students to read the experiment corresponding to the topic in the previous lecture before coming to class so that students can readily perform the written procedures while doing the experiment

Table 1 presents the practices employed by the faculty in designing their laboratory instruction. It shows that all of the four groups of faculty design their laboratory instruction in relation to the topics discussed in the lecture. They plan the science experiment according to how the topics in the classroom lectures were designed. Three out of the four groups make the experiment procedure in recipe type or cook book style, while only one group of faculty said that they sometimes implement the discovery approach where the procedures are following the investigative approach. They require their students to use laboratory manual where the list of experiments are conforming to the sequence of lecture topics indicated in the syllabus. One group of faculty discusses the topic during the lecture sessions of the class so that the students will be guided properly come laboratory periods. All of the faculty-respondents plan the experiment to be done alongside the lectures because to them, the laboratory manuals were developed in relation to the topics discussed in the lecture. The presence of laboratory manuals, according to the teacher-respondents, allows them to follow the recipe type of experiment procedure. Some teachers are planning to shift to the investigative type, while some others are already implementing the discovery approach but still with close supervision of students. The group of faculty who are implementing the discovery approach are those with fewer students. As part of their strategy, the teachers conduct both ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

118

lecture and laboratory classes in the laboratory room for the chance to add to the discussion while experiments are on-going. To them such style of handling laboratory instruction is aimed to have a close supervision of students‟ learning both in theory and in practice. The data gathered indicate that the teachers relate laboratory activities closely with lectures in order to help students progress toward science learning goals. This conforms with the statement of the National Science Teachers Association (2005) that “laboratories and lectures are not separate activities.” In the same scope, it also complies with the recommendation of the Committee on High School Laboratory to integrate laboratory experiences into instructional sequences (Singer, et al., 2005). On the other hand, the reason for most of the teachers following the recipe-type of experiment procedure is because of lack of time to venture into investigative type of experiment. To the teacher-respondents, the pressure of trying to finish the syllabus for one semester marks the biggest hurdle for them. This finding is similar to the description of a typical science course given by Jona, et al. (2008) that students do not have enough time for planning investigation or interpreting results because teachers bombard them with so many lessons from the syllabus. As reflected in the FGI with students, all the chemistry laboratory teachers from the four universities require their students to perform experiments in relation to the topics being discussed in their lecture. The lecturers are all following the given procedures included in their laboratory manual. Three out of the four groups of students have discussion of the concept in their lecture period prior to the performance of the experiment for their laboratory session. Only one group of students was required to perform experiment while the discussion of the topic is going on. In some cases, this same group of students sometimes perform the given procedure in their laboratory manual first before their teacher ask them on the learnings they get from the results of the experiment. Still, table 1 indicates that most of the times teachers provide their students with prior knowledge of the concept of the experiment either by discussing it in the lecture before conducting the laboratory class or by assigning the students to read about the experiment before coming to the laboratory class. It could be that these teachers want that the learning gained by their students in the lecture could be retain longer through hands-on activities, or it could be that from the learning the students acquired in their lecture, the teacher can enable the students to reconstruct their previous knowledge by doing experiments. These findings conform with Hidalgo‟s (2000) statement that in the laboratory method, “the learning gained by students is retained longer because the students are learning by doing.” Table 2 shows the practices of faculty in designing experiment if the necessary material and equipment is not enough for the entire class. Table 2. Practices of Faculty in Designing the Experiment if the Necessary Material is Only Limited to One or Two Groups

119

Group of Respondents Faculty of LPU 1

Students of LPU 1

Faculty of LPU 2 Students of LPU 2

Faculty of LPU 3

Students of LPU 3

Faculty of LPU 4

Students of LPU 4

Laboratory Practices of Faculty Demonstration of experiment by the teacher; replacing materials; improvising instruments; not skipping any experiment in the manual for the reasons that instruments/materials are not available By rotation, by sharing, the teacher does not skip experiments when instruments/materials are not available; there is an alternative material/instrument for the unavailable one Not skipping experiment because materials and instruments are enough for the students The teacher divide the class two batches so that materials will be accommodated by all the students; unavailability of materials happen rarely; providing all materials/instruments so that no experiment is omitted Encouraging students to be resourceful; not skipping experiment due to unavailability of materials/instruments Does not happen that the materials /instruments are limited or not available because there are very few students in the class; doing all experiments in the lab manual Performing all experiments because all materials and instruments are available; delegating a representative student to demonstrate the procedure or sharing the materials to each group if the material is limited The teacher ask the most intelligent student to do the experiment in front of the class; the students are required to perform all the experiments in the lab

The table shows that all of the four groups of faculty do not skip experiments even though the instruments/materials needed are not available. Two groups of faculty do the experiment through demonstration of the procedure either by the teacher or by a representative student from the class. On the other hand, two other groups of faculty experienced having enough materials and instruments and are available to their students. One group of faculty pushed their students to be resourceful while one group improvised the unavailable instrument and replaced the unavailable or limited materials. Another group of faculty simply shared whatever material is available to the students. All the faculty-respondents performed the experiments in the laboratory even if there is unavailability of instruments/materials because in their university the faculty themselves develop the laboratory manual so that all the needed instruments/materials were requested and were provided at the start of the semester. In case of limited number of equipment, the faculty concerned preferred to use the demonstration method of laboratory instruction to bridge ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

120

the learning of students. This finding is similar to what Acero, et al. (2000) cited that demonstration method is an imitative method where learning a skill is faster and more effective if students are shown how the job is done by using the actual tools, machines and materials. One of the results of the focus group interview, students point out that, the teachers require them to do all the experiments included in their laboratory manual and the unavailability of materials/instruments is not a reason for them to forego any experiment. If ever the necessary material is only limited to one or two groups, their teacher finds ways to address the problem. One group of students cited that if the materials/instruments are unavailable, their teacher makes the class to share the limited material or use the limited instrument by rotation. In their classrooms, there is always an alternative material/instrument for the unavailable one. Another group of students pointed out that they rarely encounter unavailability of materials/instruments; if ever the material is limited, their teacher divides the class into two batches so that the limited material can be used by all members of the class. Another group of students pointed out that they never experienced limited or unavailable materials/instruments because they are very few in the class. On the other hand, the teacher of one group of students asks the most intelligent student among them to do the experiment in front of the class. Table 2 also reveals that the teachers implement various techniques in handling problems such as unavailability or limited materials/instruments because these teachers have knowledge of the specific teaching strategies that can be used to address studentsâ&#x20AC;&#x; learning needs given particular classroom circumstances like lack of materials or equipment. These teachers do not skip experiments when materials/instruments are not available because they want to inculcate to their students the value of resourcefulness which is one of the scientific attitudes a student needs to develop. This is one of the forms of practical knowledge in the pedagogical content knowledge (PCK) of Shulman as cited by Rowan, et al. (2011). The practices of the faculty on how they begin their laboratory class on the other hand, are presented in Table 3. The table shows that all the four groups of faculty begin their laboratory class by presenting the objectives of the experiment to their students. This is done when teachers state the goals and objectives of the experiment themselves or letting students read the objectives stated in the laboratory manual, or asking the students to think of more objectives aside from those stated in the laboratory manual. Table 3. Practices of Faculty on How they Start their Laboratory Class Group of Respondents Faculty of LPU 1

Students of LPU 1

Laboratory Practices of Faculty Giving a brief description of what the experiment is all about, how to do it, and what to expect from it; stating clearly the goal of the experiment Giving a brief summary of the experiment for the day is; the teacher states the objectives of the experiment; asking questions for students to discover the possible result of experiment Describing the procedure of the experiment;

121

Faculty of LPU 2

Students of LPU 2

Faculty of LPU 3

Students of LPU 3

Faculty of LPU 4

Students of LPU 4

emphasizing the safety precautions; checking materials and lab gowns; letting students read the objective of the experiment since it is already in the manual The teacher introduces the experiment; making students read the manual since the objectives are already in the lab manual and then explaining the procedure Asking a series of questions so that students will begin thinking about the possible outcome of the experiment; telling the students the purpose of doing the experiment Allowing students to discover the outcome of the experiment; allowing students to expound on the objective of the experiment Reminding the students about laboratory policies such as proper arrangement of chairs and bags; asking students to think of more objectives of the experiment aside from those stated in the lab manual; asking students on how to apply the concept they learned from their lecture on the experiment Reviewing the topics discussed in the lecture and then asking students to relate it to experiment

Table 3 shows that two groups of faculty give descriptions of the procedure of the experiment to the students while two other groups remind the students on safety precautions and laboratory policies before any laboratory work is done. Two groups of faculty begin their laboratory class by asking questions to make students think of the possible outcome of the experiment or to let them connect the concept learned from their lecture to the experiment they will perform. The responses of the faculty as shown in the table indicate that these teachers employ teaching practices which aim for the success of an experiment because they make their students understand the problem clearly before doing the experiment. This is in consonance to NSTAâ&#x20AC;&#x;s (2007) description of a well- designed laboratory instruction where the objective of the activity is clearly communicated to students and gives opportunities to students to develop safe and conscientious laboratory habits and procedures. As observed by the students, the teachers of the three groups of students clarify the objectives to the entire class at the beginning of the class. For the purpose of clarity, teachers of the two groups of students introduce the experiment or give a brief summary of the experiment. One group of students was allowed by their teacher to discover the outcome of the experiment on their own while one group of students was asked by their teacher to relate the topics discussed in their lecture to the experiment performed. The data prove that the teachers implement a properly designed laboratory instruction because the students are allowed to expect outcomes or experimental results which may or may not be ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

122

contradictory to the concept learned from their lecture. This conforms to one of the four principles of instructional design which according to NRC (2005) can help laboratory experiences achieve their intended learning goal. According to this principle, instructions must be design with clear learning outcomes in mind in order to attain the desired learning objective. The practices of the faculty in conducting pre-lab discussion are shown in Table 4. It can be noted from the table that all the four groups of faculty list down vocabulary or terms related to the experiment, especially those important terms or concepts, the title of the experiment, materials needed, and even the set-up of the procedure is drawn on the board. Two groups of faculty use their studentsâ&#x20AC;&#x; prior knowledge and previous concepts learned as a basis for introducing new concepts. It is done by asking students of their ideas about the experiment before introducing it or by reviewing or recalling the previous concepts learned so that students can connect the gap between prior knowledge and the topic to be introduced while two groups of faculty allow their students to draw hypothesis about concepts by demonstrating selected procedures or by basing the hypothesis from their prior knowledge. The responses of the faculty prove that they are implementing good practices in conducting pre-lab discussion because they provide a foundation of factual knowledge and conceptual understanding to students. It is responsive to the pedagogical practices identified by NRC (2005) which are considered authentic best practices in science teaching. Table 4. Practices of Faculty in Conducting Pre-lab Discussion

Group of Respondents Faculty of LPU 1

Students of LPU 1

Laboratory Practices of Faculty Asking students of their ideas about the experiment before introducing it; writing important terms or concepts on the board The teacher stimulate the discussion by allowing students ask questions; allowing students to state their previous knowledge and are elaborated by the teacher; writing definitions of terms and keywords on the board

Faculty of LPU 2

Demonstrating selected procedures so that students can draw hypothesis from it; writing the title of the experiment and the materials needed on the board

Students of LPU 2

Demonstrating delicate procedures while the rest of the procedures are done by students at their own ; writing concepts on the board

Faculty of LPU 3

Writing vocabulary or terms related to the experiment on the board; asking students to formulate hypothesis about the experiment based on their prior knowledge

123

Students of LPU 3

Allowing students to make a guess on the result of the experiment before doing it by themselves; listing down vocabulary or terms related to the experiment on the board Reviewing or recalling the previous concepts learned to link previous topic to new topic or to have continuity of their previous knowledge; letting students connect the gap between prior knowledge and the topic to be introduced; writing meaning of terms on the experiment together with drawing of set-up on the board

One of thes e aut Faculty of LPU 4 hen tic best prac tice s is Students of LPU 4 Asking students to recall and relate the concepts learned eng from the lecture to the experiment; drawing the set-up of agin apparatuses on the board g resilient preconceptions where teachers identify, confront and resolve this initial understanding of the students. Another authentic best practice is organizing knowledge around core concepts which aims to increase understanding and retention of concepts among students. Cited in the responses of students, the teachers of the four groups of students conduct their prelab discussion by first writing on the board the keywords, definitions of terms, concepts and setup of the apparatuses to be used in their experiment. Two groups of students were asked to recall and state their previous knowledge before the conduct of the experiment. The teacher of one group of students demonstrates the varied procedures of their experiment while one group was asked by their teacher to infer on the result of the experiment before doing the actual experiment. The data revealed that the teachers practice a properly designed pre-lab discussion because they help students move towards a more scientific understanding of what students understand about the prior concepts they learned. This finding affirms Singerâ&#x20AC;&#x;s, et al. (2005) idea of an integrated instructional units which she believes is an effective instruction because it begins with what learners bring to the setting such as knowledge of academic content. The practices of the faculty-respondents in supervising or guiding students in the process of performing the experiment are shown in Table 5. From the table, it is clear that all of the four groups of faculty supervise and guide their students in the process of performing the experiment by moving around the laboratory room to check if each group of students does the experiment properly. Two groups of faculty never allow their students to manipulate the procedures and materials while one group of faculty allows students to manipulate the procedure and materials if they have their approval. Another group of faculty allows their students to manipulate procedures and materials depending on the situation such as when materials are not available so that the students are allowed to use replacement. Table 5. Practices of Faculty in Supervising or Guiding Students in the Process of Performing Experiment ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

124

Group of Respondents Faculty of LPU 1

Students of LPU 1

Laboratory Practices of Faculty Moving around the room, checking each group if they follow the correct procedure; entertaining questions from students; not allowing students to manipulate procedures and materials Asking questions upon seeing that students are not following the procedure correctly; roaming around the room; entertaining questions; consent is being asked from the teacher if students want to manipulate procedure or materials

Faculty of LPU 2

Going from one group of students to another group; seeing if they are doing the experiment properly; never allowing students to manipulate procedures and materials

Students of LPU 2

The teacher is moving around and checking each group if they are doing the experiment correctly; not allowing any manipulation of procedure or materials

Faculty of LPU 3

Going around and checking each group as to how they do the experiment; not allowing students to manipulate the procedure and the materials without their approval

Students of LPU 3

Moving around; not allowing students manipulation of procedure nor materials

Faculty of LPU 4

Making rounds; discussing with each group one at a time; allowing students to manipulate procedures and materials depending on the situation; using replacement if materials are not available

Students of LPU 4

Going from one group to another finding out if students are having problem with the experiment; allowing students to improvised unavailable instrument or use substitute materials

Based on the responses presented, it seems that these teachers employ the method of small group instruction by moving from one group to another because they want to give attention to each studentsâ&#x20AC;&#x; learning needs. This is in response to Hidalgoâ&#x20AC;&#x;s (2000) idea of small group instruction where students become more actively involved in their own learning and participate more freely in discussions.

125

As observed by students, the teachers of the four groups of students supervise students in doing experiment by moving around the room to check whether each group is doing the procedure properly. Two out of the four groups of students are not allowed to manipulate procedure or materials, while one group ask consent from their teacher when they want to do some manipulations. Only one group of students was allowed to improvised instrument or use substitute materials. Most of the practices of faculty, observed by the students, reveal that they make close supervision of the students while doing experiment because of the high demand for success. This is in consonance to what Salandanan (2002) emphasized when she said that for an experiment to be successful, the teacher must be a keen observant who can easily spot incorrect steps and procedure. Table 6 shows the practices of faculty in conducting a post –lab discussion. Table 6. Practices of Faculty in Conducting a Post-lab Discussion Group of Respondents Faculty of LPU 1

Students of LPU 1

Faculty of LPU 2

Laboratory Practices of Faculty Asking the students to relate the result of the experiment to their previous discussion; giving the positive result which may or may not be contrary to the result obtained by students; allowing students to compare their results to find out their mistakes; giving post-lab quiz in the next lab period Discussing partial result before the expt; discussing the result of experiment after the expt if there‟s enough time; giving correct result; allowing students to compare result; giving post- lab quiz the following meeting Calling all members of the group to discuss their result; discussing only questions in the manuals; allowing students to compare results per procedure with each group; giving post-lab quiz the following meeting

Students of LPU 2

Asking students about their ideas on the result of the expt; allowing students compare ideas; telling which is the correct result

Faculty of LPU 3

Conducting post-lab discussion during lecture time; requiring students to report the result; allowing students to compare results but not letting them change their result; giving post-lab quiz during lecture

Students of LPU 3

Allowing students to compare results with other groups and if there are discrepancies making students retest or repeat the procedure; giving post-lab quiz during lecture time Conducting a post-lab discussion a meeting after;

126

Faculty of LPU 4

providing more questions to students if the result deviate from expected result; allowing students to compare results to identify why they are wrong and how they can correct their mistakes; giving post-lab quiz after post-lab discussion

Students of LPU 4

Allowing comparison of result to know their mistakes; giving quiz after discussion of result

All of the four groups of faculty require their students to do the discussion by asking them to relate the result to their previous discussion, by reporting, or by providing more questions for students to answer. All of them allowed their students to compare result with other groups of students and all of them give a post –lab quiz after the post-lab discussion. It is reflected in the table that the teacher-respondents conduct post- lab discussions to clarify the results in relation to the lecture. They require their students to do the discussion with the aim of teaching their students communicative skills. This is similar to the laboratory principle identified by Kanli and Yagbasan (2008) which states that teachers must encourage students to explain by demanding proofs or making students use the data they acquired in making reasonable explanations. As observed by students, all the four groups of students were allowed by their teachers to compare the results of experiment with those of their classmates. Three groups of students were given a post-lab quiz after the post-lab discussion. Only two groups were given the correct result of the experiment by their teacher while the other two groups of students were allowed by their teacher to find their own mistakes. It is also evident from the table that the teachers conduct a post-lab discussion in order to provide the necessary connection between the result of the experiment and the appropriate science concepts. Students were allowed to compare results with their classmates so that they can identify and correct their own mistakes. This conforms with Piaget‟s Theory of Constructivism where children are allowed to make mistakes and correct these on their own thereby enabling them to accommodate, assimilate and reconstruct knowledge on their own (Muijs and Reynolds, 2011). Table 7 presents the data gathered from class observations to determine Table 7. Practices Observed Among the Faculty that will Enhance the Mastery of Subject Matter Practices Mean Verbal Interpretation 1. Giving a brief description of the experiment 3.00 for the day 2. Stating the goals and objectives of the 3.00 experiment 3. Letting the students think of the objectives in 0 © 2014 The authors and IJLTER.ORG. All rights reserved.

Always Always Never

127

doing the experiment 4. Asking a series of questions for the students to begin thinking about the topic 5. Presenting challenging questions to draw out the preconceptions of the students 6. Conducting a pre-lab discussion 7. Using the studentsâ&#x20AC;&#x; prior knowledge as a basis for introducing new concepts 8. Allowing the students make observations and draw conclusion about concepts prior to giving explicit instruction 9. Listing down vocabulary or terms related to the experiment 10. Supervising/guiding the students in the process of performing the experiment 11. Allowing the students explore ideas rather than manipulate material and procedures 12. Making the experiment procedure in recipe type 13. Making the experiment procedure in an openended or investigative type 14. Conducting a post-lab discussion 15. Discussing the results of the experiment in relation to lecture content 16. Administering a post-lab quiz right after the experiment 17. Providing guide questions for students to answer 18. Allowing students to compare results with other groups and making sense of the collective data of the class 19. Providing the necessary connection with the results of the experiment and the appropriate science concepts OVERALL MEAN

2.75

Always

2.00

Often

3.00 2.75

Always Always

2.50

Always

3.00

Always

3.00

Always

1.00

Sometimes

3.00

Always

Never

3.00 3.00

Always Always

1.50

Sometimes

3.00

Always

2.75

Always

3.00

Always

2.38

ALWAYS

The practices implemented by the faculty in enhancing mastery of subject matter. The data reveal that teachers always implement the given practices with a mean of 3.00. Practices like giving a brief description of the experiment for the day, stating the goals and objectives of the experiment, conducting a pre-lab discussion, listing down vocabulary or terms related to the experiment, supervising/guiding the students in the process of performing the experiment, making the experiment procedure in a recipe type, conducting a post-lab discussion, discussing the results of the experiment in relation to lecture content, providing guide questions for students to answer and providing the necessary connection with the results of the experiment ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

128

and the appropriate science concepts are marked as common practices among the teacherrespondents. Practices with a mean of 2.75 on the other hand, are always implemented by the teachers. These practices are asking a series of questions for the students to begin thinking about the topic, using the studentsâ&#x20AC;&#x; prior knowledge as a basis for introducing new concepts, and allowing students to compare results with other groups and making sense of the collective data of the class. The teachers sometimes implement those practices with a mean of 1.50 and 1.00. These are administering a post-lab quiz right after the experiment and allowing students to explore ideas rather than manipulating materials and procedures. Practices with a mean of 0 and a standard deviation of 0 are never implemented by the teachers. These are letting the students think of the objectives in doing the experiment and making the experiment procedure in an open-ended or investigative type. In general, the overall mean of 2.38 was an indication that the teachers always implement practices that enhance the mastery of subject matter among students. It could mean that these teachers exert efforts in teaching the content and process as related educational goals because they want their students to readily understand and apply the concept they acquired. This finding is similar to what Jona, et al. (2008) stated that mastery of subject matter could be attained if concepts and processes are taught simultaneously so that students perform a process with a clear understanding of the relation of that process to content. B. Scientific Reasoning The practices employed by faculty in developing scientific reasoning among their students are revealed in Table 8. The table reveals that all the four groups of faculty require scientific explanations for the result of experiment. Two groups of faculty give on-the-spot questions while performing experiment. One group of faculty require students to submit a reflection paper while the other group of faculty allow their students to reflect by sharing with other students what they have learned from the experiment. One group of faculty let their students check their data, analyze and repeat the procedure when they got a wrong result while another group of faculty ask their students to trace all the errors. It appears from the table that the teachers implement practices that develop scientific reasoning of students because they require students to make scientific explanations of the occurrence of events as in the result of the experiment. They teach their students valid reasoning principles and at the same time give opportunities to their students to practice these reasoning skills. This is in consonance with the integrated learning program of NRC (2005) which is considered an effective instructional practice because students can relate theoretical claims with evidences gathered from laboratory investigation. Table 8 presents the practices of faculty in developing scientific reasoning among students. As observed by students, all the teachers of the four groups of students require them to analyze and explain the data, their observations and discuss the results of the laboratory activity. Two groups of students are given on- the- spot questions asking them the reason for doing such procedure. Two groups of students are asked to do error analysis and to find out the sources of ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

129

error while two groups of students are required to submit reflective essay or narrative reflection of what they learned. The practices of the faculty as observed by the students revealed that their teachers implement practices that develop scientific reasoning among them because they were taught how to explain and give reasons for what they are doing. Some of their teachers allow them to reflect on their own learning so that these teachers practice metacognitive strategies. Metacognitive strategies according to Singer, et al. (2005) when implemented in a knowledge-centered environment will enable students to reflect on their own learning progress, to identify, monitor and regulate their own thinking and learning which in turn will facilitate their learning.

Table 8. Practices of Faculty in Developing Scientific Reasoning Among Students Group of Respondents Faculty of LPU 1

Students of LPU 1

Faculty of LPU 2

Laboratory Practices of Faculty Allowing students to develop scientific explanations for the result of the experiment; asking on-the-spot questions while performing experiment; asking students to defend why they got such a result; requiring students to submit a reflection paper on what they learned Asking students why they are doing such procedure and what they observe; asking the class to explain how they arrived to the result; requiring students to pass a reflective essay for the whole semester Giving on-the-spot questions while doing the experiment to check if they are following procedure correctly; asking only those who are not following correct procedure; tracing all the errors before allowing students to develop their scientific explanation

Students of LPU 2

Asking students to do error analysis; asking students to explain and analyze the data obtained; on-the-spot questions are addressed to the idle member of the group

Faculty of LPU 3

Asking the students to explain why they were not able to produce the result; letting students check their data, analyze and repeat the procedure

Students of LPU 3

Requiring students to analyze and explain the data and graphs obtained and to discuss the results; asking students to find out the sources of error

130

Faculty of LPU 4

Making students defend their result based on the laws and principles studied; allowing students to make reflection by sharing with other groups what they learned from the experiment

Students of LPU 4

Asking students to submit a report sheet for every experiment which is a narrative reflection of what students learned from the experiment; asking students to discuss the observations

In addition to this, Kanli and Yagbasan (2008) said that teachers must demand explanations and proofs from students and at the same time emphasize that students should use the data they acquired to make reasonable explanations. Table 9 presents the data gathered from class observations to determine the practices employed by the faculty in their instruction to develop scientific reasoning among their students. Table 9. Practices Observed among the Faculty that will Develop Scientific Reasoning of Students Laboratory Practices

Mean Verbal Interpretation

1. 2. 3.

Encouraging the students to design and conduct scientific investigations Requiring students to identify questions and concepts that guide scientific investigation Giving on-the-spot questions to check the understanding of students of why they are doing such procedure Allowing students to develop and revise scientific explanations and models or recognize and analyze alternative explanations and models Allowing students make and defend a scientific argument by reviewing information, using scientific language appropriately, constructing a reasoned argument, and responding to critical comments Requiring students to explain/analyze their data, discuss the results including graphs and do error analysis Requiring students to make reflection where

0.25

Never

0.25

Never

3.00

Always

0.50

Never

0.75

Never

3.00

Always

0.50

Never

131

they will defend their conclusions based on data and analysis of data, compare results with other sources and explain differences OVERALL MEAN

1.18

Sometimes

As shown in the table, the practices such as giving on- the-spot questions to check on the understanding of the students of why they are doing such procedure, and requiring students to explain/analyze their data, discuss the results including graphs and do error analysis received a mean of 3.00 which shows that the teachers always practice them. Allowing students to make and defend a scientific argument by reviewing information, using scientific language appropriately, constructing a reasoned argument, and responding to critical comments got a mean of 0.75 indicating that it is never implemented by the faculty. Practices with a mean of 0.50 and 0.25 are never implemented by the faculty. These refer to allowing students to develop and revise scientific explanations and models and recognize and analyze alternative explanations and models; requiring students to make reflection where they will defend their conclusions based on data and analysis of data, compare results with other sources and explain differences; encouraging students to design and conduct scientific investigations; and requiring students to identify questions and concepts that guide scientific investigation. In general, the overall mean of 1.18 was an indication that the teachers sometimes implement practices that develop scientific reasoning of their students. It could mean that these teachers cannot employ the inquiry method of laboratory instruction which allow their students to make their own investigation because they lack enough time to conduct such investigation since they are pressured to finish the syllabus before the end of the semester. This is contrary to the idea of Salandanan (2002) that “in the inquiry approach, the reasoning skills of the students are improved upon learning how to investigate and discover new information.” C.

Understanding Complexity and Ambiguity of Empirical Work

Table 10 shows the practices of faculty in developing the students‟ understanding of the complexity and ambiguity of empirical work. Two groups of faculty avoid errors due to equipment failure by allowing the technician do the checking and troubleshooting of equipment before an experiment, whereas only one group of faculty teach students how to troubleshoot equipment. Table 10. Practices of Faculty in Developing Students’ Understanding ofComplexity and Ambiguity of Empirical Work Group of Respondents Faculty of LPU 1

Laboratory Practices of Faculty Explaining the cause of error; teaching students in troubleshooting equipments in order to avoid errors; asking students to make several trials to check precision and accuracy

132

Students of LPU 1

Teaching students on the maintenance and proper care of equipments; requiring students to repeat the procedure if the data is not precise or accurate; explaining that errors cannot be avoided

Faculty of LPU 2

Asking the technician to do the calibration and troubleshooting of equipments; giving hints to students to account for discrepancies like for example in the unit conversion

Students of LPU 2

Replacing equipments which are not functioning; giving the students clues when there are deviations from expected values

Faculty of LPU 3

Checking the equipment before using; doing a dry-run of the experiment before asking students to perform; letting students compare their data with the standard

Students of LPU 3

Making alternative equipments available if there is malfunctioning of equipments; giving already the expected value before the experiment Explaining to students that there are conditions which may affect the result; asking the technician to check the equipments before lending to students and do the troubleshooting of malfunctioning equipments; requiring students to make three trials for every measurement

Faculty of LPU 4

Students of LPU 4

Asking the students to replace the malfunctioning equipments; explaining the factors which may cause error

The table also shows that two groups of faculty explain the cause of error and clarified that there are conditions which may affect the result. Two groups of faculty require their students to make several trials in order to check for precision and accuracy of data while only one group of faculty gives hints to students to account for discrepancies. Still another group of faculty allows students compare their data with the standard. The data indicate that the faculty design their laboratory instruction in such a way that students are able to expect outcomes or experimental results which are contradictory to the accepted scientific principle. This conforms to Jona, et al. (2008) statement that experimental errors are not hindrances to learning, but they are opportunities for greater learning. So instead of working hard to remove complexities and ambiguities, laboratory instructors should include the expectation of experimental errors in their instruction. ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

133

As observed by students, the teachers of the three groups of students replace the malfunctioning equipment while only one group of students were taught how to maintain and give proper care to equipment. The teacher of one group of students explains that errors cannot be avoided while the teacher of another group explains the factors which may cause errors. One group of student said that they are given the expected value before doing the experiment while another group said that they are required to repeat the procedure if the data is not precise or accurate and still another group said that their teacher give clues when there are deviations from expected values. The data confirm that the teachers implement practices which enable the students to find solutions to problems encountered while performing experiments. They make their students understand that even the same experiment may lead to different results if performed at different times or by different people. According to NRC (2005), a well designed scientific investigation program must include opportunities for students to be involved in activities like rechecking data observations and analysis and performing the kind of follow-up investigations that will validate the result of the investigation. The teaching practices observed among faculty to attain the understanding of complexity and ambiguity of empirical work of students are shown in Table 11. Teaching practices with a mean of 2.75 and 2.50 show that they are always implemented by the faculty-respondents. These practices are giving some clues to students to account for the discrepancy; making the students take notice of serious experimental errors due to equipment failure; emphasizing the need to compare data from standards or controls; making students take notice of precision issues and accuracy issues where accuracy depends on the standardized calibration; and letting students notice deviations from expected values. The faculty-respondents often implement those practices that got a mean of 2.25 and 2.00. These practices are helping students learn to address the challenges inherent in directly observing and manipulating the material world, including troubleshooting equipments used to make observations, understanding measurement error, and interpreting and aggregating the resulting data; emphasizing to students that random error is a normal part of the data and the data must have random error that cannot be eliminated through careful data collection; and allowing students to check repeatability of data they gathered. Table 11. Practices Observed among the Faculty to Develop the Studentsâ&#x20AC;&#x2122; Understanding of Complexity and Ambiguity of Empirical Work Laboratory Practices

1. Helping students learn to address the challenges inherent in directly observing and manipulating the material world, including troubleshooting equipment used to make observations, understanding measurement error, and interpreting and aggregating the ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

Mean

Verbal Interpretation

2.25

Often

134

resulting data 2. Making students take notice of precision issues and accuracy issues where accuracy depends on the standardized calibration 3. Emphasizing to students that random error is a normal part of the data and data must have random error that cannot be eliminated through careful data collection 4. Giving some clues to students to account for the discrepancy 5. Making the students take notice of serious experimental errors due to equipment failure 6. Letting students notice deviations from expected values 7. Allowing students to check repeatability of data they gathered 8. Emphasizing the need to compare data from standards or controls OVERALL MEAN

2.50

Always

2.00

Often

2.75

Always

2.75

Always

2.50

Always

2.00

Often

2.75

Always

2.44

Always

The overall mean of 2.44 was an indication that the teachers always implement practices that will attain the understanding of the complexity and ambiguity of empirical work of students. This shows that the teachers help their students find solutions to problems encountered while performing experiments because they have the pedagogical content knowledge which according to Shulmanâ&#x20AC;&#x;s view as cited by Rowan, et al. (2011) is the knowledge of a teacher of the difficulties that students encounter when learning particular content. D. Practical Skills Table 12 shows the practices of faculty in developing the practical skills of students. All the four groups of faculty teach their students on the proper use of laboratory equipment and then check if they acquired the skills on its use by means of practical tests. All of them are particular with safety precautions such as the use of laboratory gowns. Two groups of faculty check the data and observations recorded by their students. Two groups of faculty check whether their students read the procedure before coming to class by giving a pre-lab quiz or by looking at the amount of reagents they are getting and the sequence of steps they are following. From their practices, it is clear that the faculty implement teaching practices which develop the practical skills of their students because they are concerned not only with the proper use of equipments but also with the safety of the students in following correct procedure. With this concern, students can apply effectively the appropriate practical skills acquired to a new investigation similar to the inquiry-based laboratory investigation recommended by NSTA (2007) where students learn appropriate laboratory techniques to define and solve problems. ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

135

Table 12. Practices of Faculty in Developing Practical Skills of Students Group of Respondents Faculty of LPU 1

Students of LPU 1

Faculty of LPU 2

Students of LPU 2

Laboratory Practices of Faculty Training students on the proper use of lab equipments such as measuring devices; checking whether students are recording their observations correctly; checking the use of lab gowns and reminding of other safety precautions Explaining to students the use of an instrument demonstrating how it is used and its proper care and maintenance; putting deductions on those who are not wearing lab gowns Demonstrating to students the proper operation of lab equipments per group; checking the data of students in their manual; checking whether they read the procedure before the experiment by looking at the amount or reagents they are getting and the sequence of steps they are following; constantly reminding them of safety precautions Explaining the procedure at the same time teaching students how to use the equipment; checking whether students acquire the skill in using equipment during practical exams; implements wearing of lab gown is a must; moving around to check the data recorded in the manual

Faculty of LPU 3

Giving precautionary measures for every experiment; giving practical test to determine if students learned the skills in using lab equipment

Students of LPU 3

Emphasizing the precautions written on the manual and the use of lab gowns; checking the data sheet to find out if students got the correct observation; teaching students the proper use of equipments

Faculty of LPU 4

Giving practical exam; giving a pre-lab quiz to determine whether they read the procedure before coming to class; not accepting students if they are not in their lab gown

Students of LPU 4

Not allowing students enter the lab room if not in lab gown; keep on saying “As long as you handle reagents and instruments properly, no accident will happen”

136

As observed by students, all the teachers of the four groups of students require their students to wear laboratory gowns. Three groups of students are taught by their teachers on the proper use of laboratory equipment. The teachers of two groups of students check their data while the teacher of one group of students check their skill in using equipment during practical exams. Most of the practices of faculty, as observed by students, indicate that the teacher is concerned with the safety of the students in doing experiment like wearing of laboratory gown because for them this is one of the skills they must learn in chemistry. This conforms with the belief of NSTA (2007) that laboratory investigation is well-designed if it gives opportunities to students develop safe and conscientious laboratory habits and procedures. The teaching practices observed among the faculty to develop the practical skills of students are shown in Table 13. As indicated in the table, the faculty always implement the practices with a mean of 3.00. These practices are requiring students to read and understand procedures before carrying them out and adapt them as required; checking whether students know how to operate laboratory equipment and understanding exactly how equipment works before physically approaching it; reminding the class about safety precautions and checking whether the students observe the precautions; helping students develop skills in using scientific equipment correctly and safely, making observations, taking measurements, and carrying out well-defined scientific procedures; requiring students make and record observations of their experiment; teaching the students to use measurement devices and to record data with correct precision; and checking the student response to in the lab report data table for correct accuracy and precision. Table 13. Practices Observed among the Faculty to Develop Practical Skills of Students Laboratory Practices

Requiring students to read and understand procedures before carrying them out and adapt them as required Providing the students with hints and suggestions on possible experimental design and encouraged students to try their own ideas Checking whether students know how to operate laboratory equipment and understanding exactly how equipment works before physically approaching it Reminding the class about safety precautions and checking whether the students observe the precautions Allowing students to visually describe the procedures of the experiment before actually doing it

Mean

Verbal Interpretation

3.00

Always

0.25

Never

3.00

Always

3.00

Always

2.25

Often

137

6. Helping students develop skills in using scientific equipment correctly and safely, making observations, taking measurements, and carrying out well-defined scientific procedures 7. Requiring students make and record observations of their experiments 8. Teaching the students to use measurement devices and to record data with correct precision 9. Providing opportunities for students to take readings from equipments 10. Checking the student response in the lab report data table for correct accuracy and precision 11. Encouraging students to deviate from given procedures if they know what they are doing 12. Encouraging students to consider alternative procedures and providing them with sufficient instructions to succeed 13. Checking whether students really acquired the necessary skills in the experiment OVERALL MEAN

3.00

Always

3.00

Always

3.00

Always

2.75

Always

3.00

Always

0.25

Never

2.75

Always

2.25

Often

Similarly, providing opportunities for students to take readings from equipment and checking whether students really acquired the necessary skills in the experiment, are also implemented always by the faculty because they got a mean of 2.75. Allowing students to visually describe the procedures of the experiment before actually doing it got a mean of 2.25 which means that this is often implemented by the faculty. However, practices with a mean of 0.25 and 0 are never implemented by the faculty. These practices are providing the students with hints and suggestions on possible experimental design and encouraged students to try their own ideas; encouraging students to deviate from given procedures if they know what they are doing; and encouraging students to consider alternative procedures and provide them with sufficient instructions to succeed. In general, the overall mean of 2.25 was an indication that the teachers often implement practices that develop the practical skills of their students. It reflects that they are doing these practices because they believe that with those teaching practices they can enhance the skills of students in employing a systematic and scientific methodology which in turn according to Salandanan (2002) will enable their students “to experience step-by-step procedure in finding answers to their endless questions.” This is supported by Moni‟s, et al. (2007) statement that “teachers must teach skills to students with the expectation that competencies in skills would support open-ended, student-driven explorations.” © 2014 The authors and IJLTER.ORG. All rights reserved.

138

Understanding the Nature of Science

Table 14 presents the practices of the faculty in developing students‟ understanding of the nature of science. Table 14. Practices of Faculty in Developing Students’ Understanding of the Nature of Science Group of Respondents Faculty of LPU 1

Students of LPU 1

Faculty of LPU 2

Students of LPU 2

Faculty of LPU 3

Laboratory Practices of Faculty Telling the students that their previous knowledge may or may not affect their conclusion; helping students to overcome errors by following procedures correctly; allowing students to interpret their data based on their own understanding asking student‟s own interpretation of the data, however at the end she will give the actual interpretation by relating it to the theory; explaining the source of error and then giving students another chance to correct their error; correcting students‟ misconceptions Asking students at the start of the experiment to give their knowledge about the concept and telling them that their knowledge may or may not affect their conclusion; giving tips to students on how to overcome errors Advising students to overcome error; asking students about their ideas, then compare our ideas and she will be the one to tell which is correct Gathering preconceptions of students before the experiment; making explanations if after the experiment the result is contradictory to their preconception; telling students that errors are normally encountered but it can be overcome

Students of LPU 3

Asking students to trace the cause of error so that next time they can avoid it; always tell the students that they cannot simply rely on what they previously know, they have to discover more

Faculty of LPU 4

Emphasizing to students that their ideas might be pure misconceptions; telling students that errors may be overcome only by correct techniques

Students of LPU 4

Often telling students that different persons have different interpretation of the result but whatever the interpretation is, it depend on their previous understanding of the concept

139

All the four groups of faculty advise their students to overcome their errors and also emphasizing to their students that each person‟s preconceptions or prior knowledge may or may not affect the final conclusion. Only one group of faculty allow their students to interpret their data based on their own understanding. The practices of the faculty indicate that the teachers are implementing practices that make students understand that science is a way of knowing aside from being a human endeavour. It is a way of knowing because the prior knowledge the students have may or may not be contrary to the result of their experiment. If the prior knowledge is a misconception then the teacher must find ways to correct it, however if the previous knowledge conforms with the result of the experiment, then the teacher must give opportunities to student to construct or build new ideas. This conforms well with Piaget‟s Theory of Constructivism (Muijs and Reynolds, 2011) which states that knowledge is always a construction by the learner where the student actively construct new concepts based upon prior knowledge and new information. On the same level, this reflects Singer‟s, et al. (2005) statement that “teachers must be challenged with the intuitive ideas of students by helping them move towards a more scientific understanding through change in and not merely an addition to what students notice and understand about the world.” To the students, the teachers of the three groups of students advised them to overcome error, to trace the cause or source of error and to correct their error. Two groups of students were asked by their teacher to make their own interpretation of the data or result of the experiment and telling that different persons have different interpretation. The teacher of one group of students corrects the previous knowledge of students if it is a misconception, another group of students were asked to discover more and not simply rely on their previous knowledge, while the teacher of another group of students tells the students that their interpretation depends on their previous knowledge. The data showed that the teachers design laboratory instruction that develop the students understanding of the nature of science because they are trying to emphasize that knowledge is subject to change. This supports Crowther‟s, et al. (2005) suggestion to teachers to design lessons around science topics or concepts that have changed over time and the instruction must be explicit on how knowledge has changed and why. Table 15 presents the practices observed among the faculty to develop the students‟ understanding of the nature of science. As shown in the table, the faculty always emphasize to the students that each person‟s preconceptions may or may not affect the final conclusion. It has a mean of 2.75. They often allow the students discover that different people may interpret the same data differently. This got a mean of 1.75. Advising the students to help them overcome errors, and discover that science is not as simple or as “black and white” as they may have thought got a mean of 1.75. This is also often implemented by the faculty. Table 15. Practices Observed Among the Faculty to Develop Students’ Understanding of the Nature of Science Laboratory Practices © 2014 The authors and IJLTER.ORG. All rights reserved.

Mean

Verbal

140

Interpretation Allowing the students discover that 1.75 different people may interpret the same data differently 2. Emphasizing to the students that each 2.75 person‟s preconceptions may or may not affect the final conclusion 3. Advising the students to help them 1.75 overcome their errors, and discover that science is not as simple or as “black and white” as they may have thought 1.

OVERALL MEAN

2.08

Often

Always

Often

As a general picture, the overall mean of 2.08 was an indication that the teachers often implement practices that develop the students‟ understanding of the nature of science. It could reflect that these teachers want to emphasize in their instruction that knowledge is subject to change because of the different types of investigations that provide different information and evidence concerning the natural world. Crowther, et al. (2005) reflect the same argument when he said that “scientific knowledge in and of itself is not static and that with new information, scientific theories can change.” F.

Interest in Science and Interest in Learning Science

The practices of the faculty in cultivating students‟ interest in science and interest in learning science are shown in Table 16. Table 16. Practices of Faculty in Cultivating Students’ Interest in Science and Interest in Learning Science Group of Respondents

Laboratory Practices of Faculty

Faculty of LPU 1

Making students realize how important their lesson is to their daily life situations; challenging them to find out for themselves the possible result if a certain situation may happen

Students of LPU 1

Asking students to relate the experiment to real life situation; at the end of our lesson, the teacher often leaves a question for the student to answer

Faculty of LPU 2

By emphasizing the relevance of what they are studying to their future job; by showing some magic in chemistry

141

during pre-lab discussion Students of LPU 2

Often telling us stories on how she apply the knowledge in chemistry in her on-the-job training; the teacher trigger studentsâ&#x20AC;&#x; interest in the topic at the start of the lesson

Faculty of LPU 3 Telling simple jokes about chemistry to motivate students; conducting plant visits to chemical industries Students of LPU 3

making the lesson not boring by telling funny stories about chemistry; taking students on field trips

Faculty of LPU 4

Stating the relevance of the lesson; connecting the lesson to real world experiences

Students of LPU 4

Asking students to give practical applications of what they learned in chemistry

All of the four groups of faculty provide practical and real life situations where the experiment is applicable by making students realize how important their lesson is to their daily life situation, emphasizing the relevance of what they are studying to their future job, conducting plant visits to chemical industries, stating the relevance of the lesson and connecting the lesson to real world experiences. Two out of the four groups of faculty motivate their students at the beginning of the lesson by showing some magic in chemistry and telling simple jokes about chemistry. Only one group of faculty said that they challenged their students to find out for themselves the possible result if a certain situation happens. The practices of the faculty proved that these teachers have a great desire to develop positive attitudes among their students towards chemistry and make them highly motivated to continue learning chemistry. Positive attitude is developed if a student is highly motivated and this can be done by the teacher through improving the teaching practices and by showing to the students the relevance of the topic to their everyday lives. This finding is similar with that of Movahedzadehâ&#x20AC;&#x;s (2011) findings that students lose interest in science when the teaching of its context seems irrelevant to their lives or even to their future jobs. As observed by students, the teachers of the four groups of students emphasize the application of their learning in chemistry by asking them to relate the experiment to real life situation, telling stories on its application to on-the-job training, going on field trips, asking them to give its practical applications. The teachers of two groups of students motivate them or trigger their interest in the topic at the beginning of the lesson and make their lesson not boring by telling funny stories about chemistry. The teacher of one group of students often leaves a question for the student to answer. ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

142

The table clearly shows that the teachers engage themselves in a more interesting approach that will make the students see the value of chemistry and will motivate them to develop a positive attitude towards the subject. The practices implemented by the teachers are in conformity to the teaching principle of Kanli and Yagbasan (2008) of exciting students by making a spark about the subject. The practices observed among the faculty that will cultivate students‟ interest in science and interest in learning science are presented in Table 17. Table 17. Practices Observed Among the Faculty to Cultivate Students’Interest in Science and Interest in Learning Science Laboratory Practices

Mean Verbal Interpretation

Providing avenue where interest of students are triggered making them more eager to find out the answer through experimentation 2. Illustrating how “alive” science can become if lab experiences are not limited to routine classroom laboratory 3. Providing practical and real life situations where the experimental set-up is applicable 4. Providing thought-provoking questions that compels students to find out things by themselves OVERALL MEAN

2.50

Always

1.75

Often

2.50

Always

2.00

Often

2.19

Often

As shown in the table, the teachers always implement practices such as providing avenue where interest of students are triggered making them more eager to find out the answer through experimentation and providing practical and real life situations where the experimental set-up is applicable. These practices got a mean of 2.50. Practices with a mean of 2.00 and 1.75 are often implemented by the faculty: providing thought-provoking questions that compel students to find out things by themselves; and illustrating how “alive” science can become if lab experiences are not limited to routine classroom laboratory. In general, the overall mean of 2.19 was an indication that the teachers often implement practices to cultivate the interest of students in science and their interest in learning science. It could be generalized that these teachers are committed to their desire of creating a learning environment that will encourage and inspire students to have a desire to learn and enjoy learning. These teachers have a vision in mind of making their students become future scientist who will contribute to the progress of the nation, similar to what Salandanan (2002) had said. According to her, with high motivations, students will decide to pursue a science profession in © 2014 The authors and IJLTER.ORG. All rights reserved.

143

the future and will develop a feeling of gratitude and appreciation for the advances in science and technology that continue to raise the present quality of life.

Teamwork Skills

The practices of the faculty in developing teamwork skills among students are presented in Table 18. Table 18. Practices of Faculty in Developing Teamwork Skills of Students Group of Respondents Laboratory Practices of Faculty Grouping students alphabetically; grouping is Faculty of LPU 1 permanent throughout the entire semester; assigning of leader in a group who is responsible for dividing the task among members; rotational leadership is implemented Students of LPU 1

Grouping the class into 5 members each; assigning a different group leader for every experiment; never regrouping students instead making them work with their group mates the whole semester

Faculty of LPU 2

Division of labor among the members of each group; there is rotational leadership; not allowing regrouping

Students of LPU 2

Encouraging students to participate with their group mates; giving chance to those who are willing to be the leader to lead the group; not regrouping the members of the group Giving tasks to each member of the; asking a member of the group to help his group mate if he finished earlier; making students work with their group mates all throughout the semester; assigning anyone to be the leader of the group

Faculty of LPU 3

Students of LPU 3

Requiring each member of the group to work on different procedure so as to finish the experiment at once; assigning leader by rotation

Faculty of LPU 4

Assigning leader for every experiment who monitors the performance of others; grouping is permanent in the whole semester; there is division of labor among members

Students of LPU 4

Asking the group leader to assign specific task for each

144

member; giving everybody the chance to be the leader; not allowing students to transfer to another group As shown from the table, all the four groups of faculty group their students into smaller groups. A leader is assign for every group and there is rotational leadership. However, these four groups of faculty never allow regrouping of students; instead the students work with their permanent group mates all throughout the semester. To compensate, all the four groups employ division of tasks among the members of the group. The table also indicates that the teachers implement practices that develop teamwork skills among students. They group students into smaller groups possibly to enable students collaborate effectively with others in carrying out complex tasks. They also divide the tasks among the members of the group maybe to make students contribute and respond to ideas of others. Leadership is on a project basis so that students will assume different roles at different times. They do not allow regrouping of students but make them work with their permanent group all throughout the semester possibly because they want their students to establish harmonious relationship with their group mates and such relationship will lead to the success of the experiment. Such findings are similar to those of Hallâ&#x20AC;&#x;s (2006) study where a system was implemented in a way that specific roles were assigned to students during laboratory and a grade was given based on their level of contribution to the group. As observed by students, the teachers of the four groups of students group the class into smaller groups and assign a group leader for every experiment performed. All of them were not allowed by their teacher to regroup or transfer to another group during the whole semester. The data proved that the teachers are implementing practices that develop the teamwork skills of students because these teachers want their students to interact with each other so that they can share their knowledge through performing specific tasks. This conforms with Kanli and Yagbasan (2008) laboratory principle of exchange where teachers prepare proper environment for students to discuss their ideas with their friends, observe and listen to students who are sharing their knowledge and ensure the interaction within student groups. The practices observed among the faculty that will develop teamwork skills of students are presented in Table 19. Table 19. Practices Observed Among Faculty to Develop Teamwork Skills Among Students Laboratory Practices

Mean

Making students collaborate effectively with 3.00 others in carrying out complex tasks, share the work of the task, assume different roles at different times, and contribute and respond to ideas 2. Making students work in the same group 2.75 ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

Verbal Interpretation Always

Always

145

throughout the entire semester 3. Allowing students to regroup during the 0.50 semester 4. Allowing students to take rotational and 3.00 specific active roles in the group OVERALL MEAN

2.31

Never Always

Always

As shown from the table, the teachers always implement those practices that got a mean of 3.00. Such practices are making students collaborate effectively with others in carrying out complex tasks, share the work of the tasks, assume different roles at different times, and contribute and respond to ideas; and allowing students to take rotational and specific active roles in the group. The practice making students work in the same group throughout the entire semester got a mean of 2.75 and is always implemented by the teachers. However, the teachers never allow students to regroup during the semester garnering a mean of 0.50. As a whole, the overall mean of 2.31 was an indication that the teachers always implement practices that will develop studentsâ&#x20AC;&#x; teamwork skill. It could mean that these teachers are familiar with small group instruction which according to Hidalgo (2000) is an effective strategy that enhances cooperation, teamwork, leadership and group motivation among students. II.

Extent by which Students Manifest the Attainment of the Goals of Science Laboratory Instruction in their: A.

Attitude and Motivation

The extent by which students manifest the attainment of their interest in chemistry is reflected in Table 20. It can be gleaned from the table that the 80 students have positive attitude towards chemistry as indicated by the overall mean of 4.14 and a standard deviation of 0.42. The mean of their attitudes toward chemistry ranged from 3.54 to 4.41. Table 20. Extent by which Students Manifest the Attainment of Interest in Chemistry Item

Mean

Learning chemistry requires a serious effort 4.41 and special talent 2. Reasoning skills that are taught in chemistry 4.14 can be helpful to me in my everyday life 3. For me doing well in chemistry courses depends on how well the teacher explains 4.39 things in class 4. Understanding chemistry gives me a sense of ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

Standard Verbal Deviation Interpretation 0.71

Positive

0.74

Positive

0.75

Positive

146

accomplishment 5. Theories and scientific laws in chemistry are difficult to understand 6. How well I do in chemistry exams depends on how well I can recall material in the way it was presented in class 7. Learning chemistry has helped me to understand situations in my everyday life

4.31

0.63

Positive

3.54

0.93

Positive

4.13

0.6

Positive

4.08

0.85

Positive

4.14

0.42

Positive

OVERALL MEAN

It could mean that these students view chemistry as a subject which requires serious efforts and special talents because the theories and scientific laws are not easy to be understand. Further, the student could harbor the feeling that their achievement in chemistry depends not only on how they can recall materials but also on how their teacher presented or explain it in class. However, they believe that chemistry can help them in their everyday life and understanding the subject gives them a sense of accomplishment. This finding is similar to the idea of Zulueta and Guimbatan (2002) that students enjoy goal-oriented activities and practical work where they can see the relevance of abstract concepts and principles and consequently become interested in sciences. Table 21 presents the extent by which students manifest the attainment of their interest in learning chemistry. Table 21. Extent by which Students Manifest the Attainment of Interest in Learning Chemistry Item

Mean

If I am having trouble learning chemistry, I 4.06 try to figure out why

Standard Verbal Deviation Interpretation 0.64

Highly motivated

4.08

0.71

Highly motivated

The subject has created a knowledge-base 4.19 which will help me in my career

0.86

Highly motivated

I put enough effort into learning chemistry

3.95

0.78

Highly motivated

I use my imagination and creativity in doing scientific investigations 4.08

0.58

Highly motivated

2. I have a real desire to learn chemistry

147

that I have the ability to pursue my study in 4.12 chemistry

0.64

Highly motivated

7. I am willing to master the knowledge and skills in chemistry course 3.74

0.85

Highly motivated

8. When learning chemistry, I prefer to put concepts/ideas in my own words

3.94

0.83

Highly motivated

4.02

0.48

Highly motivated

OVERALL MEAN

It appears from the table that 80 students are highly motivated by their teachers to learn chemistry as revealed by the overall mean of 4.02 and a standard deviation of 0.48. The mean of their motivations ranged from 3.74 to 4.19. The results mean that these students have willingness to learn chemistry because they believe that chemistry has created a knowledge-base which will help them in their career as a result of the encouragement their teachers have given them to pursue their study in the subject. Their great desire to learn chemistry is reflected in them using their imaginations and creativity in doing scientific investigations, figuring out why they are having trouble learning chemistry, putting enough effort to learn chemistry by putting concepts/ideas in their own words and willingness to master the knowledge and skills in chemistry course. The findings conform to Salandananâ&#x20AC;&#x;s (2002) statement saying that wholesome attitudes of students may be developed by awakening their interest and keeping them highly motivated to inquire about occurrence in the natural environment. She added that students must relentlessly pursue a scientific investigation and be responsible enough to complete an assigned task despite constraints. The extent by which the attainment of the understanding of the nature of science is manifested by students is presented in Table 22. The students have much understanding of the nature of science as indicated by the overall mean of 3.91. The mean of their understanding ranged from 3.54 to 4.08. No matter how difficult the theories and scientific laws in chemistry are, still these students can understand situations in everyday life. It suggests that they have the ability to interpret data from the material world because they put concepts/ideas in their own words and use their imaginations and creativity to do scientific investigations. Table 22. Extent by which Students Manifest their Understanding of the Nature of Science Item

Mean

Standard Verbal Deviation Interpretation

148

Theories and scientific laws in chemistry are 3.54 difficult to understand

0.93

Much understood

2. I use my imagination and creativity in doing 4.08 scientific investigations

0.58

Much understood

3. When learning chemistry, I prefer to put 3.93 concepts/ideas in my own words

0.83

Much understood

4. Learning chemistry has helped me understand situations in my everyday life

0.85

Much understood

OVERALL MEAN

to 4.07

3.91

Much understood

These findings are in consonance with the statement of Crowther, et al. (2005) that in teaching scientific laws, teachers must emphasize how these laws describe nature and how things act under certain conditions. It should be taught also that questions lead to investigation and experiments then lead to conclusions - but still there are many different pathways that scientists take. B.

Laboratory Skills

The extent by which the students manifest the attainment of practical skills is shown in Table 23. Table 23. Extent by which Students Manifest the Attainment of Practical Skills Skills Number Percent Verbal of Interpretation Groups of Students A. Handling Liquids and Measuring Volume Highly competent 1. Places the cover of the reagent bottle on the 18 100 table in an upside down position Highly competent 2. Uses a pipette correctly in getting liquid 16 89 chemicals from the reagent bottle 3. Reads the volume of liquids precisely using a 12 67 Competent graduated cylinder 4. Measures exact volume of liquids with a 13 72 Competent pipette Average B.

Highly competent

Moderately

149

1. Sets the scale to zero before starting to weigh 2. Places the object on the left pan and the set of masses on the right pan 3. Uses a dry spatula in getting solids from the reagent bottle 4. Avoids using bare hands in handling chemicals 5. Obtains accurate weight using platform balance 6. Uses a paper lining in introducing solids into test tube 7. Avoids returning unused reagents to the reagent bottle 8. Discards solid wastes into an appropriate waste container

6 18

33 100

competent Highly competent

Highly competent

Highly competent Moderately competent

Competent Competent Competent

Average C. Bunsen Burner Manipulation 1. Lights the Bunsen burner properly by closing the air inlet then lighting the burner from the side of the barrel going up 2. Regulates the amount or flow of gas properly so as to get an ideal height of the flame 3. Produces a non-luminous flame by opening the air inlet

Moderately competent

39 Highly competent

100

Highly competent

Competent

Average D. Heating Substances in a Test tube and Doing Evaporation 1. Heats the test tube in an inclined position (45Ë&#x161; angle) moving it back and forth while heating 2. Not pointing the mouth of the test tube to anybody while heating 3. Follows the proper set-up for evaporation Average E. Doing Filtration 1. Folds the filter paper correctly 2. Follows the proper set-up for filtration

Highly competent

100 Highly competent

89 Highly competent

18 16

100 89

Highly competent

Highly competent Highly competent Highly competent

95 Highly competent

150

1. Wears lab gown properly 2. Wears appropriate goggles all the time 3. Wears appropriate clothes and footwear Average

18 3 18

100 17 100

Not competent Highly competent Competent Highly competent

Overall Percent

It can be noted from the table, that as to handling liquids and measuring volume, 18 groups of students or 100 % placed the cover of the reagent bottle on the table in an upside down position; 16 groups or 89 % used a pipette correctly in getting liquid chemicals from the reagent bottle; 13 groups or 72 % measured exact volume of liquids with a pipette; and 12 groups or 67 % read the volume of the liquids precisely using a graduated cylinder. As a general view, about 15 groups or 82 % of the students are highly competent in handling liquids and measuring volume. It could be that these students had acquired skills in handling liquids and measuring volume even when they were still in high school because they have already performed similar laboratory activity before. This finding affirms Marineâ&#x20AC;&#x;s (2003) idea that if an experiment is repeated it can greatly help students to understand or improve their laboratory techniques. As to handling solids and weighing, 18 groups or 100% placed the object on the left pan and the set of masses on the right pan; 16 groups or 89 % avoid using their bare hands in handling chemicals; 14 groups or 79 % used a dry spatula in getting solids from the reagent bottle and obtained accurate weight using platform balance; 13 groups or 72 % avoid returning unused reagents to the reagent bottle; 11 groups or 61 % discarded solid wastes into an appropriate waste container; and 6 groups or 33 % only set the scale to zero before starting to weigh and used paper lining in introducing solids into test tube. In sum, only 67 % or 12 groups of students are competent in handling solids and weighing. It seems that not all students have acquired the necessary skills in handling solids and in weighing as justified by the number of students who were able to set the scale to zero before weighing and those who used paper lining in introducing solids into test tube. This can mean that these students are careless in following the correct techniques that they always work in a hurry for the purpose of finishing the experiment at once without considering the accuracy of what they are doing. They do not understand measurement error which may affect the result of their experiment. This finding conforms with Singerâ&#x20AC;&#x;s, et al. (2005) recommendation to teachers to help students learn to address the challenges inherent in directly observing and manipulating the material world including the understanding of measurement error. In terms of Bunsen burner manipulation, 18 groups or 100 % were able to produce a nonluminous flame by opening the air inlet; 15 groups or 83 % were able to regulate the amount or flow of gas properly so as to get an ideal height of the flame; while only 7 groups or 39 % light ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

151

the Bunsen burner properly by closing the air inlet then lighting it from the side of the barrel going up. In general, only 13 groups or 74 % of the students are competent in manipulating the Bunsen burner. This can be a clear indication that although students are familiar with the use of a Bunsen burner, still not all of them know how to light it properly. It could be that these students were not listening nor watching the demonstration made by their teacher during prelab discussion on the proper way of lighting the Bunsen burner. Demonstration method according to Garcia as cited by Acero, et al. (2000), is an imitative method where learning a skill is faster and more effective since students are shown how the job is done by using actual tools, machines and materials. As regards the skill heating substances in a test tube and doing evaporation, 18 groups or 100 % of students heated the test tube in an inclined position moving it back and forth while heating; 16 groups or 89 % were not pointing the mouth of the test tube to anybody while heating and followed the proper set-up for evaporation. Almost 17 groups or 93 % of the students are highly competent in heating substances in a test tube and doing evaporation. It appears that these students had encountered minor accidents on heating substances in a test tube and evaporation during their high school chemistry that they now developed the proper techniques from their previous mistakes. This affirms Jona‟s (2008) statement that mistakes encountered by students during experiments are not hindrances but they are opportunities for greater learning. On the other hand, doing filtration is easy for 18 groups or 100 % of the students who were able to fold the filter paper correctly while 16 groups or 89 % followed the proper set-up for filtration. Thus, 17 groups or 95 % of the students are highly competent in doing filtration. It suggests that the students have acquired the skills in filtration because they were taught about the principles of filtration and how to do filtration. These findings affirm Moni‟s, et al. (2007) idea that students must be taught of the differences among “knowing about” a topic, “knowing how” to complete a skill, “showing how” to complete a skill and “doing” the skill. This is done through integration of skills development with conceptual learning. In terms of safety considerations, 18 groups or 100% of the students wore laboratory gown properly and appropriate clothes and footwear while 3 groups or 17 % only wore appropriate goggles all the time. As a whole, only 13 groups or 72 % of the students are competent with regards to safety considerations. It could mean that those students found it inconvenient to wear goggles while doing experiments since they are not used to it aside from they were not given strict implementation on its use. This is in contrary to NSTA‟s (2007) suggestion to teachers of giving the students opportunities to develop safe and conscientious laboratory habits and procedures. In general, the students are highly competent in their practical skills as justified by an over-all percentage of 81 %. Table 24. Extent by which Students Manifest the Attainment of Teamwork Skills Number Skills of Percent Verbal Groups Interpretation of © 2014 The authors and IJLTER.ORG. All rights reserved.

152

Students 1. Tries to get other team members involved

Highly Competent

2. Presents ideas about how to work on the task

Moderately Competent

3. Enjoys working on the team

Highly Competent

4. Questions other‟s task ideas constructively

Moderately Competent

5. Tries to get other team members to voice their opinions about ideas on the table

Moderately Competent

6. Responds calmly to others

Highly Competent

7. Tries to raise alternatives that weren‟t on the table

Moderately Competent

8. Helps explain other ideas

Highly Competent

9. Integrates ideas of different members

Competent

Highly Competent

Competent

10. Responds appropriately to any questions presented in the group

Overall Percent

The extent by which the students manifest the attainment of teamwork skills is shown in Table 24. It can be gleaned from the table that out of the 18 groups of students, only 16 groups or 89 % tries to get other team members involved; enjoys working on the team; responds calmly to others; and responds appropriately to any questions presented in the group. Fourteen groups or 78 % help explain other ideas while twelve groups or 67 % integrate ideas of different members. Eight groups or 44 % present ideas about how to work on the task; question other‟s task ideas constructively; and try to get other team members to voice their opinions about ideas on the table. Only 6 groups or 33 % try to raise alternatives that weren‟t on the table. © 2014 The authors and IJLTER.ORG. All rights reserved.

153

To sum up, only 12 groups or 67 % of the students are competent in their teamwork skills. It seems that not all students attained teamwork skills because they did not demonstrate a true collaborative work. It reflects that they work in group for the purpose of dividing limited laboratory equipment and space among a large number of students. This is contrary to NRCâ&#x20AC;&#x;s (2005) idea of teamwork that requires high level of substantive conversation. There is high level of substantive conversation if there is considerable interaction about the ideas of a topic and if there is sharing of ideas. Table 25 presents the extent by which students manifest the attainment of the understanding of complexity and ambiguity of empirical work. It can be noted from the table that 18 groups or 100 % of students have knowledge on troubleshooting equipment. Seventeen groups or 94 % take notice of deviations from expected values. Fifteen groups or 83 % take notice of experimental errors due to equipment failure while only 7 groups or 39 % take notice of precision issues and accuracy issues. It appears that almost all of the 13 groups or 71% of the students understood the complexity and ambiguity of empirical work because they can find solutions to problems encountered while doing experiments as in troubleshooting equipment. It could be that they were given proper instructions on the proper use and maintenance of equipments. Most of them can take notice Table 25. Extent by which Students Manifest the Attainment of the Understanding of Complexity and Ambiguity of Empirical Work Number Skills of Percent Groups Verbal of Interpretation Students 1.

Has knowledge on troubleshooting equipment

100

Highly Understood

Takes notice of precision issues

Much Understood

Takes notice of accuracy issues

Much Understood

Takes notice of experimental errors due to equipment failure

Highly Understood

Understood

Takes notice of deviations from expected value

154

of deviations from expected result and can take notice of experimental errors due to equipment failure possibly because their teachers have emphasized to them that errors cannot be avoided when performing experiment. Instead students must know how to deal with these experimental errors. This conforms with Jona‟s, et al. (2008) statement that a well designed scientific investigation must allow students to understand measurement error. C. Achievement in Chemistry Table 26 shows the extent by which the students manifest the attainment of mastery of subject matter and of scientific reasoning. Table 26. Extent by which Students Manifest the Attainment of Mastery of Subject Matter and Scientific Reasoning Mean Percent

LPU 1

LPU 2

LPU 3

LPU 4

Average Mean

Enhancing Mastery of Subject Matter

11.91

11.84

12.54

14.23

12.63

60.14

Developing Scientific Reasoning

28.78

23.33

22.18

22.15

24.11

63.45

Goal of Instruction

It can be noted from the table that enhancing mastery of subject matter got an average mean of 12.63. Out of the 21 questions about mastery of subject matter, the highest score obtained by the students is 18 and the lowest score is 7. Students from LPU 4 got the highest mean of 14.23 while students from LPU 3, LPU 1 and LPU 2 got a mean of 12.54, 11.91 and 11.84 respectively. It appears that the students have attained an average level of mastery of subject matter as justified by the mean of 12.63 which is about 60.14 %. Developing scientific reasoning got a mean of 24.11. Out of the 38 questions about scientific reasoning, the highest score obtained by the students was 36 and the lowest score was 10. Students from LPU 1 got the highest mean of 28.78 while students from LPU 2, LPU 3 and LPU 4 got a mean of 23.33, 22.18 and 22.15 respectively. It appears that the students had attained an average level of scientific reasoning because the mean is 24.11 which is about 63.45 %. The students attained an average level of mastery of subject matter because they can readily understand and apply the concepts they have learned. It could be that their teachers taught content and process simultaneously. This affirms Jona‟s, et al. (2008) idea that mastery of subject matter could be attained if concept and processes are taught simultaneously so that in performing a process the student has clear understanding of the relation of that process to content.

155

The students attained an average level of scientific reasoning possibly because they were trained how to construct scientific arguments where they will use their reasoning skills. This is in consonance to Jona‟s, et al. (2008) statement that students should be taught of the various scientific processes and valid reasoning principles and at the same time must be given opportunities to practice those reasoning skills. III.

Model of Teaching Practices in Chemistry Laboratory to Attain the Goals of Science Laboratory Instruction

Laboratory experiences in chemistry are important for students to gain a deeper sense of understanding and a greater confidence in learning. With the acknowledged importance of a laboratory experience for all students, it is necessary for chemistry teachers to conceptualize clearly the elements that make up an effective and well-designed laboratory instruction. The chemistry faculty of the Lyceum of the Philippines University strive to provide their students with access to a more authentic laboratory experience by complying with the university‟s vision, mission, goal and objective (VMGO). The design of their instruction is tailored in accordance with what is stated in the purposes of the university. The university VMGO was simplified using the acronym S-E-R-V-E to make it more realistic for all stakeholders. Each letter in the acronym has a corresponding meaning on which the faculty patterned their teaching practices and strategies. Based from the findings of the present study, the LPU faculty, although did not fully implement the ideal practices as stated in the seven goals of science laboratory instruction, were still able to develop students‟ positive attitude, laboratory skills and high achievement in chemistry. This could be due to the uniqueness of their teaching practices that were anchored on the university‟s VMGO which are parallel to the seven goals of science laboratory instruction. A model of teaching practices in chemistry laboratory was proposed from the identified best teaching practices of chemistry faculty of LPU with the hope of contributing to the body of knowledge in science education. The identified best teaching practices of chemistry faculty of LPU are shown in Figure 3. The figure consists of four rectangular boxes. The first box shows the purposes of LPU which were embodied in its VMGO.

156

Letter S means to seek excellence in all the things that we do; E means to exert efforts to teach and motivate; R means to respect the opinion and efforts of others; V means to vigorously pursue the virtue of humility; and E means to enjoy the challenges that are in your hands. The second rectangular box contains the seven goals of science laboratory instruction. Each purpose in the first box is connected by a line to a goal or goals in the second box to indicate that the purposes of LPU are parallel with the seven goals of science laboratory instruction. The third rectangular box represents the studentsâ&#x20AC;&#x; manifestation of the attainment of the goals of science laboratory instruction in their achievement, attitude/ motivation, and laboratory skills. Arrows connect the goals in the second box to the third box to indicate that the goals of science laboratory instruction are attained if they are manifested in the high achievement, positive attitude, high motivation and competencies in laboratory skills of students. A fourth and final rectangular box contains the identified best teaching practices of the chemistry faculty of LPU which were anchored on the purposes of the university.

157

Twenty best teaching practices of LPU chemistry faculty were identified and were categorized under each purpose. Since each purpose is parallel to the goals of science laboratory instruction, this means that although the LPU faculty did not fully implement the ideal teaching practices stated in the goals of science laboratory instruction, they were still able to attain those goals as manifested in the achievement, attitude, motivation and laboratory skills of their students. In order to seek excellence in all the things being done, the LPU chemistry faculty, in designing their laboratory instruction, integrate teaching practices that will attain the mastery of subject matter and scientific reasoning and will lead to the high achievement of students in chemistry. Enhancement of mastery of subject matter may be attained by giving a pre-lab quiz to students before conducting a pre-lab discussion; requiring students to submit a flow diagram of the procedure of the experiment prior to its actual performance; outlining the procedure and drawing the experimental set-up on the board; and requiring students to make a logbook of the experiments. By simply giving a short pre-lab quiz at the start of the class where students will be asked to give the title of the experiment, the reagents and equipments to be used and even the objectives of the experiment, the teacher assesses if the students read the experiment and already have an idea about the experiment. Diagnostic, formative assessments when embedded into the instructional sequences can be used to gauge students‟ understanding. Requiring students to submit a flow diagram of the procedure prior to the actual performance of the experiment will enable the student to organize information that will increase the students‟ retention of concepts. A chemistry laboratory teacher must outline the procedure and draw the experimental set-up on the board in order to make students understand the process and make the learning outcomes clear to the learners. Students must be required to make a logbook of the experiment where they can record not only the results of their experiment but also the learning they got from the experiment. With this logbook, students can make multiple representations that will show the correlation between the results of the experiment and the concepts previously learned. Teaching practices implemented by LPU chemistry faculty that will attain the development of students‟ scientific reasoning are asking students the purpose of doing a certain procedure while performing an experiment; and making students reflect on their own learning by sharing their experiences while doing the experiment to the other groups of students during post-lab discussion. Students are trained to use their reasoning skills when they were asked on the purpose of doing such a procedure. The use of student reflection and discussion signify that the faculty supports metacognition and student self-regulation where they can control their own learning. In exerting efforts to teach and motivate, the LPU chemistry faculty implement teaching practices to attain the interest of students in science and their interest in learning science and also the students‟ understanding of the nature of science. This in turn will lead to a positive attitude and high motivation of students in chemistry. The students‟ interest in learning science can be developed if the faculty post the list of the top 10 or top performers on the door outside the laboratory room after each major examination or after the midterm grade had been released; and the teacher uses technical terms related or appropriate for the course of the student during pre-lab and post-lab discussion. Students will be inspired and motivated to strive harder in © 2014 The authors and IJLTER.ORG. All rights reserved.

158

learning chemistry if his name is included in the list of top performers. Using technical terms appropriate to the course of the students will enable the students see the relevance of the subject to their future job thereby motivating them to pursue their study. A teaching practice implemented by LPU chemistry faculty that will attain the students‟ understanding of the nature of science is the faculty accepts all the interpretations made by each group regarding the experimental data obtained without telling the students that they are wrong. In this way, the teacher can emphasize to the students that different people may interpret the same data differently depending on the steps they followed in making their scientific investigations. Respecting the opinion and efforts of others so that they will also respect yours is also very important in developing teamwork skills among students. The LPU chemistry faculty inculcate this virtue of respecting one another leading to the development of teamwork skills of students by implementing practices such as properly arranging the seats so that students can join their group mates not only during the actual performance of the experiment but also during pre-lab and post-lab discussion; asking the bright student in each group to guide his group mates and do peer tutoring; allowing students to discuss among themselves the result of the experiment before recording it on the data sheet; and assisting students in getting and returning materials from the stock room. If students are seated together with their group mates from the start until the end of the class, they will become close to each other so that there will be harmonious relationship in the group and they will enjoy working as a team. Peer tutoring is necessary to help explain other ideas. Students try to get other team members to voice their opinions and integrate ideas of different members when they are allowed to discuss among themselves the results of the experiment. The respect for the opinions of others can be observed when students respond calmly to their team mates and question other‟s opinions or ideas constructively. If the teacher assists the students in getting and returning materials from the stockroom, she demonstrates the value of cooperation which is very important in developing team work skills. To vigorously pursue the virtue of humility is a purpose of LPU chemistry faculty where humility is considered as the very foundation of leadership of a teacher so that a teacher can influence the behavior or attitude of students. In influencing the behavior of students, the LPU chemistry faculty employs teaching practices that lead to the interest or positive attitudes of students in science. These teaching practices of LPU chemistry faculty are assigning only one group instead of the whole class to get the reagents from the stockroom; and giving a borrower‟s slip to each group for them to list the needed equipment that will be borrowed from the stockroom. Only one group from the class was assigned to get the reagents and this group is responsible for distributing the reagents to the other groups. In this way, the students are taught the positive attitude of responsibility and willingness to share the resources to others. Proper enlisting of equipments in the borrower‟s slip is one way of training the students to be organized and systematic in what they are doing. Finally, LPU chemistry faculty also gear to make students enjoy the challenges that are in their hands because they will make them a better and complete person. In so doing, the LPU chemistry faculty implement teaching practices that lead to the development of practical skills of students and their understanding of the complexity and ambiguity of empirical work. The © 2014 The authors and IJLTER.ORG. All rights reserved.

159

teaching practices that lead to the development of practical skills include providing posters and signages on safety precautions in the laboratory room such as “No Lab gown, no entry”, “Place your bags in the shelves or tables provided for at the back of the room”, and “Check the gas supply before and after using”; placing all the needed reagents in the demonstration table for the teacher to see the amount of reagents the students will get; and giving practical test on the proper use of equipment such as Bunsen burner, pipette and platform balance. Posting signages on safety precautions will help students develop safe and conscientious laboratory habits which is one of the practical skills a student must acquire in science education classes. Another important practical skill is for the student to use the exact amount of reagents as indicated in the procedure to avoid wastage, contamination and inaccuracy of result. Practical test is given to check if students acquired the skill in using simple equipment such as proper lighting of Bunsen burner, measuring accurate volumes of liquids with a pipette and getting exact weight of reagents in a platform balance. These are fundamental skills which are necessary for the success of an experiment. Teaching practices of LPU chemistry faculty that lead to the students‟ understanding of the complexity and ambiguity of empirical work include making students trace the source of their error in the result of the experiment and requiring students to repeat the procedure if they got a result which is far beyond the actual value. By allowing students to trace the source of error, they will recheck data observations which will enable them to expect and understand experimental error in every scientific investigation. Requiring students to repeat the procedure makes them correct their own mistakes. In this way, a teacher can emphasize to students that experimental errors are not hindrances to learning but they are opportunities for greater learning. Figure 4 is a proposed model of constructivist teaching-learning approach based on the identified best teaching practices in chemistry laboratory. It consists of three rectangular boxes connected by arrows. The first box represents the constructivist teaching approach containing the four elements of constructivism such as interweaving, scaffolding, modeling and coaching. The second box shows the four elements of constructivism which are included in the constructivist learning environment such as collaboration, articulation, reflection and exploration. The seven goals of science laboratory instruction which include mastery of subject matter, scientific reasoning, understanding the complexity and ambiguity of empirical work, practical skills, understanding the nature of science, interest in science and interest in learning science and teamwork skills are contained in the third box.

160

An arrow linking the first box to the second box shows that a constructivist teaching approach creates a constructivist learning environment. Another arrow linking the second box to the third box shows that a constructivist learning environment manifests the seven goals of science laboratory instruction. It means that if a constructivist teacher implements teaching practices based on the seven goals of science laboratory instruction then the students will manifest the attainment of the seven goals of science laboratory through constructivist learning. Based on the findings of the study, the best teaching practices of the chemistry faculty of LPU which are based on the universityâ&#x20AC;&#x;s VMGO are parallel to the ideal practices which on the other hand are based on the seven goals of science laboratory instruction. This means that constructivist teaching could be attained if the ideal practices and the best teaching practices from LPU are both implemented. Both practices lead to the enhancement of mastery of subject matter, developing of scientific reasoning, interest in science and interest in learning science, understanding the nature of science, developing teamwork skills, practical skills and understanding the complexity and ambiguity of empirical work. Interweaving is connecting of new ideas to prior knowledge in order to make learning meaningful. The best practices of the LPU faculty such as giving of pre-lab quiz and submission of flow diagram by their students together with the ideal practices based on the seven goals such as making the topic in the lecture simultaneous with the laboratory and ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

161

conducting of pre-lab discussion, all of which lead to the enhancement of the mastery of subject matter. The practice of giving a pre-lab quiz will enable a teacher to diagnose what is already known by the student so that they can relate new knowledge (concepts and propositions). Submission of flow diagram of the procedure make students organizes information into a meaningful whole. When the topic in the lecture is simultaneous in the laboratory, the knowledge acquired is applied in the experiment so that concept and process are taught simultaneously. Conducting a pre-lab discussion will enable the teacher to find out what the pupils know about the topic before doing the experiment. Such practices lead to the enhancement of mastery of subject matter. Scaffolding is accomplished by giving assistance to students in achieving tasks that they cannot yet master on their own and then gradually withdraws the teacher‟s support. Best practices of LPU faculty such as assisting students in getting and returning materials and asking students of the purpose of doing such a procedure are examples by which scaffolding is done. Assisting students in getting and returning materials makes students develop teamwork skills while asking students of the purpose of doing such a procedure develop their scientific reasoning. Other best teaching practices of LPU faculty such as posting of signages about safety precautions, and placing needed reagents in the demo table are also implemented during scaffolding which lead to the development of practical skills of students. On the other hand, ideal practices based on the seven goals of science laboratory instruction were also implemented by LPU faculty, and these are checking and troubleshooting of equipment before the experiment, not skipping experiments simply because the materials are not available and supervising and guiding students in performing experiment. During modeling, the teacher performs a complex task to show the students the processes needed in carrying out the experiment. The best practice of LPU faculty of outlining the procedure and drawing of set-up of the experiment on the board make students learn large amounts of meaningful material from textual representations thus enhancing mastery of subject matter. Developing practical skills of students such as checking whether students know how to operate lab equipment and understand exactly how equipment works before physically approaching it, may also be implemented during modeling and these conform to the ideal practices based on the seven goals of science lab instruction. Coaching is a process of motivating learners, analyzing their performance, and providing feedback on their performance. The best practices of LPU faculty which are included under this process are posting of top performers, using technical terms related to the course of the students during discussion, accepting all interpretations made by each group about the result of the experiment, giving of practical test and requiring students repeat procedures if they got wrong result. Posting of top performers and using technical terms related to the course of the students during discussion will develop students‟ interest in science and interest in learning science. Ideal practices based on the seven goals of science laboratory instruction include emphasizing the relevance of the lesson to the students‟ future job, making the objective of the experiment clear to the students before doing the experiment, listing down keywords on the board, advising students to overcome their errors, emphasizing to students that each person‟s preconception may or may not affect conclusion and giving of post-lab quiz. A post – lab quiz © 2014 The authors and IJLTER.ORG. All rights reserved.

162

and practical test are given to analyze and provide feedback for the performance of the students. In collaboration, students learn from each other such as when their seats are arranged by group and when they are allowed to do peer tutoring. When students are grouped into smaller groups, a leader is assigned for every group, everyone is given specific active roles and not allowing students to regroup are examples of collaboration. In so doing, students develop teamwork skills. Assigning only one group of students to get the reagents and share with the rest of the class, and listing equipments needed in the borrowers slip may be implemented during collaboration. These practices lead to the development of practical skills. Among the best practices of LPU faculty are arranging the students by group, allowing them to do peer tutoring, assigning only one group of students to get the reagents, and making them list the equipments needed in the borrowers slip. On the other hand the ideal practices based on the seven goals of science lab instruction include grouping students into smaller groups, assigning a leader for every group, allowing students to do rotational and specific active roles, and not allowing students to regroup. During articulation, students are encouraged to articulate their ideas, thoughts and solutions. They are allowed to think about the method they use in doing the procedure and whether this method arrived to the correct result. Best practices of LPU faculty such as requiring students to submit logbook, allowing them trace error and repeat the procedure if they got wrong result may be implemented during articulation. Submission of logbook enhances mastery of subject matter, while tracing error and repeating procedure if students got wrong result makes them understand the complexity and ambiguity of empirical work. During reflection students compare their results with other students. Practices such as making students do reflection by sharing and making them discuss results may be implemented during reflection. Making reflection by sharing develops scientific reasoning. Developing studentsâ&#x20AC;&#x; understanding of the nature of science as in accepting all their interpretations of experimental result may also be implemented during reflection. During post-lab discussion students can compare results, give scientific explanations for their result, analyze and discuss the data and observations. Among the best practices of LPU faculty is making students do reflection, accepting all interpretations made by students about experimental result, and discussing among themselves experimental result. On the other hand the ideal practices based on the seven goals of science lab instruction include conducting post-lab discussion, allowing students compare result, making students give scientific explanations for their result and making students analyze and discuss the data and observation. In exploration, students develop novel combinations of ideas and thinking about hypothetical outcomes of imagined situations and events. In the logbook that students are required to submit, they must include the possible application of the experiment to other situations and they can propose investigatory studies related to the experiment. That is one of the best practices of LPU faculty which lead to the enhancement of mastery of subject matter. Ideal practices such as going on field trips, and connecting the lesson to real world experiences are also exploration. ÂŠ 2014 The authors and IJLTER.ORG. All rights reserved.

163

Conclusions Based on the findings of the study, the following conclusions are drawn: 1. The teaching practices employed by the faculty in teaching chemistry laboratory that attain the seven goals of science laboratory instruction are those practices where students engaged in: a.

experiential learning where experience is translated through reflection into concepts, which are used as guides for active experimentation

active learning or learning by doing where learners use their learning in realistic and useful ways, seeing its importance and relevance

c. meaningful learning where learners organize information through integrating new and previous knowledge and d. cooperative learning where students work as self-directed in small collaborative groups 2. Students enjoy goal-oriented activities and practical work, have willingness and great desire to learn chemistry and can understand situations in everyday life no matter how difficult the theories and scientific laws in chemistry are. Students can recognize the differences among “knowing about” a topic, “knowing how” to complete a skill, “showing how” to complete a skill and “doing” the skill. They demonstrate true collaborative work and interaction through sharing of ideas. They know how to deal with experimental errors and can find solutions to problems encountered while doing experiments Students can readily understand and apply the concepts they have learned. They are aware of valid reasoning principles and can practice those reasoning skills. Recommendations In the light of the findings of the study, the following recommendations are endorsed: 1. The model of teaching practices be used by chemistry faculty in designing their laboratory instruction to develop students‟ positive attitude towards chemistry, laboratory skills and high achievement in the subject. 2. The findings of this study could be an avenue for chemistry faculty in maximizing the active participation of students in the laboratory by incorporating the significant findings of the study in training and seminars of chemistry faculty. 3. An in-depth study can be conducted in other science subjects that will determine the best practices in its laboratory instruction. 4. The significant findings of this study may be integrated as a guide in developing instructional materials in chemistry. © 2014 The authors and IJLTER.ORG. All rights reserved.

164

5. Curriculum planners may include the salient findings of this study as a concrete basis in determining the objectives and methods in the design of chemistry laboratory instruction. 6. Policy makers may utilize the findings of this study as a guideline in considering the educational purposes that science education can best provide to students. References Acero, V., Javier, E. & Castro, H. (2000). Principles and strategies of teaching. Manila, Philippines: Rex Bookstore. Alasuutari, P. (2004). The globalization of qualitative research. Qualitative research practice. London: Sage. Aquino, L. (2003, March). Literature-based Science Instructions for Preschools. Educators Journal. Volume 22, No.10, p.7. Avilla, R. (2009). Students’ Images and Views of Chemistry; Its Implication to Student Achievement. (Masteral Thesis). Phil. Normal University. Manila, Phil. Bishop,C., Bishop M. & Whitten, K. (2000). Standard and microscale experiments in General Chemistry. U.S.A: Saunder‟s College Publishing. Brodie, M. (2006, July 26). Promoting science and motivating students in the 21st century. Scientix. Retrieved March 29, 2012 from http://scienceinschool.org/2006/issue2/rir Chang, R. (2009). Chemistry. New Jersey: Mc Graw Hill Inc. Claveria, G. (2002). The Utility Value of a Well-Managed Laboratory Activities in Students’ Learning of Science Concepts. (Masteral Thesis). Batangas State University, Batangas City, Phil. Cobb, P., Stephan,M., Clain K. & Gravemeijer, K. (2001). Participating in Classroom Mathematical Practices. Retrieved April 27, 2010 from www.nap.edu/catalog.php?record_id=11311#description. Corpuz, B., Rimas, A., Galangco, N.R. & Bautista, A. (2003). Student teachers’ manual for observation and participation. Quezon City, Philippines: Katha Publishing Co., Inc. Crowther, D., Lederman, N. & Lederman, J. (2005). Understanding the True Meaning of the Nature of Science. Retrieved March 12, 2012 from caee.naaee.org/...science.../katzenberger-methods-and-strategies_ ... Domin, D. (2009). Considering Laboratory Instruction through Kuhn‟s View of the Nature of Science. Retrieved April 27, 2010, from www.nap.edu/catalog.php?record_id=11311#description. Eastwell, P. & Rennie,L. (2002). Using Enrichment and Extracurricular Activities to Influence Secondary Student‟s Interest and Participation in Science. Retrieved November 29, 2009 from www.scienceeducationreview.com/open_access. Glynn, S. & Koballa, T. (2006). Science Motivation Questionnaire. Retrieved July 25, 2011 from http:// www. Coe.uga.edu/smq/ Hall, T. (2006). Encouraging Active Participation of Students during Science Laboratories. Retrieved January 8, 2010 from http://www.montana.edu/msse/capstone.2006.htm Hidalgo, F. (2000). Tips on how to teach effectively. Manila, Philippines: Rex Book Store. Isnian, S. (2009). Definition of Achievement. Retrieved May 30, 2012 from http://hmsofyanisnianspd.blogspot.com/2009/08/definition-of-achievement.html © 2014 The authors and IJLTER.ORG. All rights reserved.

165

Jona, K., Adsit, J. & Powell, A. (2008). Goals, Guidelines, and Standards for Student Scientific Investigations. Retrieved April 29, 2010 from www.inacol.org/research/docs/NACOL_ScienceStandards_hub-pdf. Kanli, U. & Yagbasan, R. (2008). The Effects of a Laboratory Approaches on the Development of University Students‟Science Process Skills and Conceptual Achievement. Essays in Education Special Edition. Retrieved April 15, 2012 from http://www.usca.edu/essays/specialedition/UKanl%c3%acandRYagbasan.pdf Leonor, J. (2007). The Effects of Scientific Inquiry Method on the Academic Performance in Chemistry of High School Students in the University of St. La Salle Integrated School. (Masteral Thesis). University of San Agustin, Iloilo City. Marine, S. (2003, April). Building Skills with Reiterative Laboratory Projects. Journal of Chemical Education. Vol. 80, No. 4. P. 366. Mentzer R. (2006). The Effect of Self-Reflective Journaling on Student Motivation. Retrieved January 8, 2010 from http://www.montana.edu/msse/capstone.2006.htm Minstrell, J. & Kraus, P. (2005). Best Practices of Science Teaching. Retrieved July 3, 2011 from http://www.phy.ilstu.edu/pte/311content/effective/best_practice.html Moni, R.W., Hryeiw, D.H., Poronnik, P., Lluka, L.J. & Moni, K.B. (2007, September). Assessing Core Manipulative Skills in a Large, First-year Laboratory. Advances in Physiology Education . vol. 31 no. 3 pp.266-269 .Movahedzadeh, F. (2011, Summer). Improving Students‟ Attitude Toward Science Through Blended Learning. Science Education and Civic Engagement Journal. Retrieved March 29, 2012 from http://seceij.net/seceij/summer11/movahedzadeh_im.html Morehead, L. (2006). Integrating a Science Club into an Alternative High School Setting in order to Enhance Students' Learning. Retrieved January 8, 2010 from http://www.montana.edu/msse/capstone.2006.html Muchmore, L.A. (2006). Does Cooperative Learning and Peer Assisted Learning Increase Middle School Student Achievement and Involvement? Retrieved January 8, 2010 from http://www.montana.edu/msse/capstone.2006.htm Muijs, D. & Reynolds, D. (2011). Effective Teaching – Evidence and Practice. 3rd edition. London. Sage Publications. Narayan, R. (2005). An Interpretative Study of the Discourse Practices. Retrieved January 9, 2010 from http://www.coe.uga.edu/prism/Ratna_Narayan_Dissertation_2005.pdf. National Research Council (2005). America’s Lab Report: Investigation in High school Science. Retrieved April 27, 2010 from www.nap.edu/catalog.php?record_id=11311#description. Nordick, T. (2006). Why do Students Fail to Turn in Work? Retrieved January 8, 2010 from http://www.montana.edu/msse/capstone.2006.htm National Science Teachers Association (2007). Laboratory Science Instruction Statement - NSTA Position Statement. Urban Science Education Leadership Academy. Retrieved April 29, 2010 from www.nsta.org/about/position/laboratory.aspx Ogle, R. (2005). Increasing Student Motivation. Retrieved April 5, 2010 from http;//www.uncw.edu/cte/events/spring 05/html Palada, G. (2002). Perception about Chemistry and Teachers’ Teaching Practices; their Influence on Student’s Performance. (Masteral Thesis). University of San Agustin, Iloilo City, Phil.

166

Petrucci, R., Herring, G., Madura, J. & Bissonnette, C. (2002). General Chemistry. Principles and Modern Application (8th ed). New Jersey: Prentice Hall Inc. Reyes, C. (2002). Effects of Task-Oriented Learning Approach on Selected Affective Variables and Chemistry Achievement among Self-Handicapping Students. (Dissertation). University of the Phil.Diliman, Quezon City, Phil. Reyes, P. (2005). Laboratory Competencies in General Chemistry I of College Students of the Lyceum of Batangas. (Masteral Thesis). Batangas State University. Batangas City, Phil. Ricardo, L. (2008). Chemistry Teaching in the Public Secondary Schools: Basis for Policy Formulation. (Dissertation). Mariano Marcos State University. Laoag City, Phil. Roble-Estrella, D.R. (2009). Predictors of Students’ Performance in Chemistry Laboratory of Selected Private Schools. (Masteral Thesis). Bataan Peninsula State University. Balanga City, Bataan, Phil. Rowan, B., Schilling, S., Ball, D. & Miller, R. (2011). Measuring teachers‟ pedagogical content knowledge in surveys: An exploratory study. Consortium for Policy Research in Education. Retrieved March 20, 2012 from http://www.sii.soe.umich.edu/documents/pck%20final%20report%20revised%20BR10 0901.pdf Salandanan, G. (2002). Teaching children science. Quezon City, Philippines: Katha Publishing Co. Inc. Singer, S., Hilton, M. & Schweingruber, H. (2005). America‟s Lab Report: Investigation in Highschool Science by Committee on Highschool Lab: Role and Vision. Retrieved April 27, 2010, www.nap.edu/catalog.php?record_id=11311#description. Straatman, C. (2006). Implementing Inquiry into Chemistry Classroom. Retrieved January 8, 2010 from http://www.montana.edu/msse/capstone.2006.htm Teachnology (2007). Current Trends in Education: Constructivism. Retrieved April 29, 2010 from www.inacol.org/research/docs/NACOL_ScienceStandards_hub-pdf Torres, M. (2009). Integrating Graphic Organizers in Facilitating Learning Chemistry. (Masteral Thesis). Phil. Normal University, Manila, Phil. Wachanga S. & Mwangi, J.G. (2004). Effects of the Cooperative Class Experiment Teaching Method on Secondary School Students‟Chemistry Achievement in Kenya‟s Nakuru District. International Education Journal Vol 5, No 1, 2004. Retrieved March 28, 2012 from http://iej.cjb.net Washtak, A. (2006). Evaluating Chemistry Students‟ Motivation and Learning with Technology. Retrieved January 8, 2010 from http://www.montana.edu/msse/capstone.2006.htm Wilson, J.D., Cordry, S. & Uline, C. (2004). Science Fairs: Promoting Positive Attitudes towards Science from Student Participation. College Student Journal, Vol. 38, 2004. Retrieved March 31, 2012 from http://www.questia.com/googleScholar.qst?docId=5006016959 Wink, D., Gislason, S. & Kuehn, J. (2000). Working with Chemistry (A Lab Inquiry Program). New York: W.H. Freeman and Company. .Zulueta, F. & Guimbatan, K. (2002). Teaching strategies and educational alternatives. Manila,Philippines: Academic Publishing Corporation. Zemelman, S., Daniels, H. & Hyde, A. (2005). Best practice. Today’s standards for teaching and learning in America’s schools. Third Edition: Portsmouth, New Hampshire HEINEMANN