Preparing a Teaching Dossier

Preparing a Teaching Dossier Christopher Knapper and Susan Wilcox

Centre for Teaching and Learning

2007 (revised)

Contents Introduction..........................................................................................................................

Teaching Dossiers at Queen’s.................................................................................

History and Rationale of Teaching Dossiers........................................................

1 1 2

Preparing a Teaching Dossier.............................................................................................

Components of Teaching........................................................................................

What Form Should a Dossier Take?......................................................................

Developing a Dossier -- The Steps ........................................................................

What to Include in a Dossier..................................................................................

Preparation Tips.......................................................................................................

Reviewing and Revising a Draft Dossier..............................................................

3 3 3 4 5 7 7

Data about Teaching: Some Guidelines and Cautions...................................................

Self-Evaluation.........................................................................................................

Peer Assessment.......................................................................................................

Letters from Students and Former Students........................................................

Evidence of Student Learning................................................................................

Service on Teaching-Related Committees............................................................

Student Ratings........................................................................................................

11 11 11 11 12 12 12

Questions and Concerns.....................................................................................................

14

Evidence for Effectiveness..................................................................................................

13

References.............................................................................................................................

15

Bibliography..........................................................................................................................

15

Appendix A: Statement on Effective Teaching at Queen’s............................................

17

Appendix B: Possible Items for a Teaching Dossier......................................................

21

Appendix C: Creating the Foundations for a Teaching Dossier....................................

27

Appendix D: Developing a Statement of your Teaching Philosophy..........................

31

Appendix E: Guidelines for the Interpretation of Student Evaluations.......................

37

Sample Dossiers available at: www.queensu.ca/ctl/resources/publications/preparing_dossier.html

-i-

- ii -

Introduction Teaching Dossiers at Queen’s Prior to the 1990s, teaching dossiers were not common at Queen’s, except in the case of faculty nominated for major teaching awards. In 1995 the Senate Committee on Academic Development (SCAD) subcommittee on teaching performance recommended that faculty at Queen’s be encouraged to prepare dossiers, especially for major career decisions such as tenure and promotion. And in 1997 the first collective agreement signed by the university and QUFA included a clause describing teaching dossiers and describing how they should be used. Article 29 in the most recent Collective Agreement applies to the assessment and evaluation of teaching for: • Annual Performance Review • Renewal • Tenure • Continuing Appointment • Reappointment • Promotion According to Article 29.1.6, “Any member whose teaching performance is being reviewed has the right to submit any information he/she believes to be relevant to the review, including a Teaching Dossier.” Clause 29.2 explains that: Teaching Dossiers are intended to provide a description of a Member’s major teaching accomplishments and strengths in a manner that conveys the scope and quality of the individual’s teaching. Responsibility for gathering and collecting the evidence for the dossier is the member’s. The contents of the Teaching Dossier may include, but should not be restricted to, such items as the following: (a) A statement of the faculty Member’s philosophy, objectives and methods of teaching, including reference to institutional and departmental teaching goals; (b) A list of undergraduate and graduate courses, including directed studies and thesis supervisions, taught by the Member; (c) Examples of course revision, curriculum development, and teaching methods such as evidenced by course outlines, assignments, final examinations and other materials the Member deems appropriate; (d) A record of the faculty Member’s role in curriculum and instructional developments such as administrative and committee service for the Department, Faculty, or Senate related to pedagogy, and including directing and coordinating programs, guest lectures, and other presentations; -1-

(e) Data from students including USAT per Article 29.3 and the Member’s Course Survey per Article 29.4, letters and testimonials; (f) A record of the faculty Member’s special contribution to teaching including teaching awards, publications and presentations, instructional development grants, participation in conferences and seminars on education/pedagogy, and other such evidence as the Member deems appropriate.

History and Rationale of Teaching Dossiers The concept of the teaching dossier was first introduced by the Canadian Association of University Teachers in 1980 and since that time dossiers (or portfolios as they are known in the USA) have been used in universities in many parts of the world. There is now a considerable literature about the philosophy and use of dossiers to document teaching performance. One seminal publication is the second edition of the CAUT Guide to The Teaching Dossier, which appeared in 1986, and which we recommend as an appropriate resource reference for use at Queen’s (Shore, Foster, Knapper, Nadeau, Neill, and Sim, 1991).

Teaching dossiers are intended to provide a summary of a professor’s major teaching accomplishments and strengths, in the form of selected short narrative descriptions of activities and achievements that convey the scope and quality of the individual’s teaching through multiple sources of evidence. Responsibility for gathering and collecting the evidence for a dossier is the professor’s. Responsibility for interpreting and judging the dossier lies with administrators (department heads, deans) and colleagues (e.g. members of a tenure and promotion committee).

Although teaching dossiers were originally developed as a means of documenting teaching for the purpose of performance review, they are also an excellent tool for developing and improving teaching through a process of documenting goals and achievements and reflecting on teaching activities and accomplishments. For this reason, faculty are encouraged to maintain a teaching dossier and update it each year even though there may be no immediate prospect of a tenure or promotion review.

-2-

Preparing a Teaching Dossier Components of Teaching Before beginning to compile your dossier it is helpful to consider that teaching is a lot more than lecturing, and covers many different activities in and out of the classroom.

A good dossier should reflect the full range of your teaching activities, with evidence that speaks to each of the following four components: Classroom teaching For example: • enthusiasm • clarity of explanations • communication skills • ability to handle questions

Course management For example: • assessment methods (exams, tests, assignments) • availability to students • feedback to students • liaison with TAs • organisation of student projects, field trips, etc.

Course content For example: • organization • level of complexity • coverage of field • use of examples

Teaching outside the course/classroom For example: • advising and counselling students • acting as mentor to colleagues • curriculum planning • writing texts and other teaching materials

What Form Should a Dossier Take?

A dossier has to be readable and manageable, otherwise it will not be taken seriously by busy department heads and committee members. Hence it is important to distinguish between the dossier to be presented for public scrutiny and the collection of evidence on which it is based. In this guide we use the term “dossier” to refer to the public document; the fuller collection of materials and raw data (e.g. complete sets of student evaluations or comprehensive course outlines) comprise a “portmanteau” of evidence that is kept by the faculty member as backup material. A rough analogy might be the shoe box full of financial material you collect and summarise each year on -3-

your income tax return. The contents of the portmanteau can be called on for further evidence if required by the review committee, just as published papers and research data back up listed publication on a vitae. Sometimes it may be appropriate to include parts of this material as an appendix to the shorter public dossier.

Hence the form of the dossier is a set of short narrative descriptions that convey the scope and quality of teaching, if possible with evidence for its effectiveness, and with more complete data kept on file to be called on if needed. There is no hard and fast rule on the length of a dossier. The CAUT Guide originally suggested a very short document for public consumption (three pages), but dossiers are nowadays typically 6-12 pages without appendices. Dossiers much longer than this run the risk of antagonising busy colleagues.

Developing a Dossier -- The Steps Criteria for effective teaching. If we are to present evidence for effective teaching, then it would be helpful to have some criteria that might guide us. In 1995 the SCAD subcommittee on teaching performance developed a statement on effective teaching at Queen’s. It comprises the following components: • • • • • • •

Knowledge, interest, enthusiasm, challenge Communication Clarity, organization, consistency, feedback, fairness, interaction, flexibility Curriculum development and knowledge of teaching methods Respect for student diversity Promotion of independent and collaborative learning Reflection and self-evaluation

A more detailed description of the SCAD criteria is included in Appendix A. Such criteria can provide a general framework for assembling a dossier. In addition there may be more specific criteria in particular departments or faculties: for example a professional school might emphasize problem-based learning or use of the case method. If so, evidence in the dossier should speak to these requirements. Depending on their personal philosophy of teaching, professors may choose to emphasize particular aspects of effectiveness over others.

Clarifying teaching responsibilities. Ideally, a professor’s teaching responsibilities should be clarified with the department head well before the start of the teaching year so that expectations are mutually acceptable (for example to teach three courses, one at introductory level and one at graduate level, to supervise three graduate theses, to develop one new course, to serve on the departmental curriculum committee, etc.).

Again this provides a general framework against which the accomplishments listed in a teaching dossier can be measured.

-4-

Selecting indicators of teaching activity and effectiveness. The next step is for the faculty member to select indicators of teaching activity consistent with the criteria and agreed responsibilities, and to begin collecting evidence that the activity not only took place, but was done effectively. Later sections describe in more detail the sorts of indicators and evidence that might be included in a dossier: the CAUT Guide lists 49 different indicators (see Appendix B), ranging from results of student evaluations to testimonials from employers. This evidence, including exemplary materials, is included in the master portfolio (portmanteau) and a summary series of statements is compiled for the public dossier.

When compiling a dossier for the first time it will quickly become apparent that evidence it would be highly desirable to have on hand is in fact not available, either because it was never collected in the first place or was not kept. This reinforces the point made earlier that compiling a dossier is an ongoing developmental process, like developing a research vitae. The first dossier is the hardest to prepare, but the process does become easier and more rewarding over time, as the portmanteau gets fuller and there is more evidence from which to select.

Creating an argument. A dossier is not only a compilation of teaching-related material.

It states a case for the approach you take to teaching. No matter the context in which you teach, the overall shape of the argument remains the same: in a dossier teachers must explain what and how they teach, why they teach that way, and whether or not it works. See Appendix C for an exercise designed to help teachers get started in creating the foundations for a teaching dossier by developing an argument for their approach to teaching.

Sections to Include in a Dossier While Appendix B lists 49 possible items to include in a dossier, most people include far fewer items, depending on their experience and range of teaching duties. The Queen’s collective agreement offers some guidance about the content of a dossier and the following suggestions are consistent with its recommendations.

• First should come a brief biographical background/overview: A few sentences that describe who you are and your current (overall) responsibilities at Queen’s so that readers can place the information in the dossier in a broader academic context.

The main body of your dossier must incorporate three essential components: • A statement of your teaching philosophy, or approach to teaching, preferably in relation to institutional or departmental criteria for effective teaching. This should be a reflective statement describing your goals for teaching and learning, as well as your preferred approach, practices, and methods. Further suggestions for developing such a statement are included in Appendix D. -5-

• A statement of teaching responsibilities/activities, to include courses taught, graduate students supervised, teaching committee responsibilities, etc. Describe instructional settings, the level and number of students, and student demographics.

Subheadings may include: course development and modification, development of teaching materials, teaching innovations, administrative and committee work related to teaching.

• An assessment of your teaching effectiveness, based on evidence you provide and refer to. This may take many forms, but nearly always will include some data from students (e.g., a summary of USAT results). Subheadings could include products of teaching (student learning); information/feedback from students; information/ feedback from colleagues; information from other sources (alumni, parents, employers); teaching awards.

Optional additional categories include: • Professional development, where you describe the steps you have taken to evaluate and/or improve your teaching (e.g., attending a workshop on teaching), and the goals you have for your development and improvement as a teacher.

• Teaching scholarship, such as conference presentations, classroom research, and publications on teaching.

• Educational leadership (e.g., presenting a workshop on teaching, mentoring TAs, coordinating a departmental series on teaching and learning, etc.).

For each of these general categories there is a range of sources of supporting evidence that might be collected and presented. The most comprehensive list is the one from the CAUT Guide that appears in Appendix B. Dalhousie University has been using dossiers for several years and the following is a list, in descending order of frequency, of the items their faculty most commonly include (the numbers refer to the CAUT list).

1. Student evaluations of teaching (#31) 2. Listing of courses taught (#12) 3. Description of course materials (#13) 4. Participation in workshops and seminars on teaching (#26) 5. Statements from colleagues on teaching effectiveness (#37) 6. Description of teaching innovations and their effectiveness (#24) 7. Student letters and comments after a course has been completed (#33) 8. Involvement in course/curriculum development (#27) 9. Supervision of theses and projects (#7) 10. Samples of exemplary student work (#2 and #3)

-6-

Preparation Tips 1. Aim for an integrated, cohesive dossier. Tie together the various components -- for example, connect the philosophy with student evaluations, peer review, and student products.

2. Aim for internal consistency: correlate assertions with evidence.

3. Remember that the teaching dossier is not just a record of your teaching, but provides evidence of the impact of your teaching.

4. Consider the full range of your teaching activities, including those beyond your home department.

5. Make a case in writing about any circumstances that have affected your teaching. For example, explain student evaluation data in context. There’s no need to be defensive, but it is helpful to explain the nature of particular teaching situations, especially when outcomes are unusual and/or disappointing.

6. Find positive ways to address and enhance weaker areas in the dossier. For example, what goals and plans do you have for future development?

Reviewing and Revising a Draft Dossier Once you have prepared a draft teaching dossier, you may want to ask a colleague, in your department or in the Centre for Teaching and Learning, to review the document and offer advice regarding improvement. As a guide to self- and peer-assessment of the dossier, we propose the following three questions: 1.

Is the dossier user-friendly?

Appearance, organization and presentation do matter. A logical and clear organization (e.g. a table of contents) helps readers find what they’re looking for in the dossier. We have emphasized that the dossier must be concise. At the same time, it must be complete, i.e., everything that is available and that adds to the reader’s understanding of your approach to teaching should be included (in the body of the dossier or an appendix), or there should be an explanation of why it is missing.

This category is easy for others to judge. After you have completed a first draft, ask a colleague for feedback on the way you have organized and presented the material -- your colleague may have good ideas for how it can be re-organized to make a better impression.

-7-

2.

Does the dossier present an accurate and honest portrait of your approach to teaching?

What is most important here is that readers get a pretty accurate feel for the kind of teacher you are. You need to be able to communicate the most significant things about your teaching. You can’t lie, but presumably your approach to teaching is a valid one, or you would take a different approach! So don’t be shy. Tell readers what you do and why you do it. That includes actions in and out of the classroom, it includes beliefs as well as actions (and a careful reader will be looking for connections between the two), and it includes what you do to develop your teaching.

This category is much more complicated to judge. You have an ethical responsibility to present yourself in an accurate (and positive) light. Readers of the dossier can say whether they’ve got a “good sense” of you. Only you (or someone else who is very familiar with your teaching) can say whether that sense is the right one. The challenge of creating a legitimate self-portrait is what makes the preparation of dossiers a great way to develop as a teacher. Again, ask a sympathetic colleague who knows something about your teaching style for feedback -- does the dossier give a full and accurate picture of your teaching?

3.

Is your approach to teaching a reasonable one?

Some ways of teaching are presumably better than others. In the final analysis colleagues need to be able to decide, based on your dossier, whether your approach to teaching is likely to promote student learning. One way readers of the dossier may judge this is by considering whether your teaching falls in line with generally accepted principles of teaching and learning. If yes, then you needn’t explain yourself since the reasonableness of your approach will be evident to an informed reader (although this begs the question of whether most dossier reviewers are informed, we suggest you write as if they are). If you are a maverick -- well, you’ll need to supply a good argument as to why your approach is a valid one (e.g. evidence of positive student learning outcomes).

At Queen’s, the SCAD statement on effective teaching (Appendix A) provides a useful set of generally accepted criteria against which teaching quality, as presented in a dossier, may be judged. Ask a colleague to review your draft dossier with these criteria in mind, and to give you feedback on whether you have highlighted both your strengths and your efforts to improve in weaker areas. When a friendly colleague points out inconsistencies between the criteria and your approach to teaching, it gives you an opportunity to reassess your teaching and the way you present your teaching in the dossier.

-8-

Data about Teaching: Some Guidelines and Cautions When compiling your dossier, bear in mind the impression it will create on those who must read it and make judgements about your teaching effectiveness (your department head, dean, and colleagues on the promotion and tenure committee). Dossiers provide a rich source of evidence about teaching. However, research on teaching evaluations has revealed that there are factors that can bias evidence gathered about teaching effectiveness. The following cautions should not be taken to mean that data from the listed sources is invalid, but rather to emphasize that no source of information about teaching is entirely definitive and objective, and that the context in which the data were gathered may affect the results. No one source of information about teaching tells the whole story, and a major strength of teaching dossiers is the opportunity they provide to compile information from different sources and perspectives. The following commentary suggests some important caveats relating to the major sources of evidence used in compiling a dossier.

Self-Evaluation. Although self-evaluations are by definition subjective and may be selfserving, data from the individual him/herself can provide evidence for critical reflection on teaching and may constitute an important gloss on or explanation for information derived from other sources, such as student evaluations.

Peer Assessment. Evaluations from colleagues have to be interpreted bearing in mind the relationship between the assessor and the person being evaluated (in other words a colleague may be unable to separate their friendship or antipathy for the individual from their views on his or her teaching). It is important to bear in mind what aspects of the person’s teaching are being evaluated. For example, classroom visits -- although useful for providing feedback that could lead to changes and improvements, are probably an unreliable source of information about teaching because (a) the visit itself may change the teaching being observed -- for example the instructor may find the situation intimidating (b) one or two classes are probably not a sufficient sample of teaching on which to make a valid judgement, (c) untrained observers may not be especially good judges of teaching, especially if the instructor’s style and philosophy differ from their own, and (c) such visits place an undue emphasis on one aspect of teaching -- classroom performance -- and ignore many other dimensions that might contribute to student learning. A better use of peer assessment might be to have colleagues review course materials, such as course outlines, course objectives, handouts, tests and exams, and samples of students’ work produced for the course (e.g. project reports, essays, etc.).

Letters from Students and Former Students. Letters from students can provide a richer picture of teaching than is often available from student evaluations. However, it is important to know whether the comments are representative of most students in -9-

the course and how the letters have been solicited. One way of avoiding the perception of pressure or bias might be to have a colleague solicit student opinion and summarise the results. Comments by former students and alumni offer a more distanced perspective on an individual’s teaching, but it is notoriously difficult to track down students who have graduated, and hence providing a representative sample is usually very difficult.

Evidence of Student Learning. In principle, evidence of learning is a wonderful indicator of teaching effectiveness, but in practice it is difficult to know how much of student achievement can be attributed directly to the work of a particular teacher, since so much depends on extraneous factors, such as the abilities of students, differences in learning styles, the amount of work they did for the course, etc. Basing evidence of learning on exam results may be especially problematic, since exams often measure only one type of learning (e.g. memorization), and exam marking is often unreliable.

Service on Teaching-Related Committees. Involvement on such committees would certainly seem to indicate commitment and interest, but it would be better to have information about the achievements of these committees and to know the contribution of the individual to the committee’s effectiveness.

Student Ratings. Student ratings are the most commonly used method of evaluating teaching and there is an extensive body of research on these instruments. While experts have generally concluded that student ratings tend to be reliable, valid, relatively unbiased, and useful, it is known that teaching context can influence results -- for example class size, course level, and whether a course is required or elective. A summary of relevant findings is contained in Appendix E. In presenting summaries of course ratings in your dossier, be sure to include a representative sample of courses you teach (graduate and undergraduate, lecture courses and seminar courses, etc.), cite ratings for a number of different years, and list results for several key questions (not just one summary item).

One great advantage of the teaching dossier is that it allows you to provide a brief commentary on your ratings, for example to explain special circumstances relating to a particular course (a very low response rate, an innovation you introduced, an unusual group of students, etc.).

- 10 -

Questions and Concerns Won’t it take too much time?

Documenting teaching properly will certainly take some time, especially if you have not collected relevant evidence over the years. But a good deal of material is probably already in your files (e.g. student evaluations, letters from former students). Once the first dossier has been prepared the process becomes much easier and can also have important benefits in helping you reflect on teaching and make improvements.

Is it fair to ask former students to comment?

Certainly, as long as they are not under your supervision (e.g. as graduate students) and you approach them honestly. To enhance the credibility of evidence collected from former students it may be advisable to ask a colleague to solicit, collect, and summarise the letters.

How can I document successful student learning?

Evidence might include exam scores (e.g. on independently marked professional exams), exemplary student work (e.g. project reports), student publications based on work done in a course or on a thesis you supervised, or student achievement in further courses. Be sure to get student permission for material you use.

What do I say about course innovations that backfired?

Documenting these efforts shows your concern for improving teaching and can provide useful contextual information for judging future changes. Documenting partial failures as well as successes gives evidence for a dossier that gives an honest depiction of your teaching accomplishments.

Should I include only information that is flattering to my teaching?

Colleagues will quickly spot obvious omissions (e.g. missing teaching evaluations) and a dossier should give a valid overall picture of your teaching while stressing the successes and achievements. (After all, a research vitae does not generally list papers rejected or negative comments of referees.)

Will maintaining a dossier improve student learning?

Drafting a dossier involves reflection on teaching and examination of course outcomes, which in turn can lead to improvement.

What resources are available to help?

Staff from the CTL can offer a variety of printed resources (as a start, see many of the items in the reference list at the end of this document) as well as personal consultation and occasional workshops on dossier preparation and interpretation. Once you have a draft dossier you can get useful feedback from a departmental colleague.

- 11 -

Will my head and colleagues read it?

Yes, if the dossier is carefully prepared, well written, and succinct. The use of dossiers is sanctioned by the collective agreement (article 29) and has been endorsed by the senior administration at Queen’s.

Should dossiers stress effort or accomplishment?

Ideally both. To assess accomplishment it is very helpful to have clear criteria for effective teaching and learning that are endorsed by the institution and the department.

I’m too modest to make a good case Baseless claims will not impress the chair or colleagues, but if you want your teaching efforts to be recognized, be prepared to put your best foot forward, as you would for research accomplishments.

Presentation will win out over substance Department heads, deans, and colleagues on review committees are better than you might think at interpreting documentation and assessing quality performance. They will likely spot misinformation and omissions just as they would with spurious research claims. On the other hand, a poorly organized or overly long dossier may undermine your case.

How can use of dossiers be reconciled with need for standardized evaluation procedures and criteria?

If the institution, faculty, or department has adopted teaching goals these can serve as general criteria against which to judge the evidence presented in a dossier. At the same time, individuals can differ in the ways they meet these criteria, just as they will do in the case of scholarly accomplishments.

In the end it’s all subjective All evaluation is a matter of judgement, but the better the evidence, the more reliable the decision.

- 12 -

Evidence for Effectiveness Dossiers (or portfolios as they are known elsewhere) have been used for several years in several hundreds of institutions in the USA. John Centra (1993), one of the foremost experts on university teaching evaluation, compared portfolios with other measures of teaching effectiveness (e.g. student ratings and classroom visits by the dean) and concluded that: • using a portfolio in summative decisions can help provide a more complete representation of performance; • evaluation of portfolios can undoubtedly benefit from discussion among evaluators about criteria and standards; • the portfolio should include not only what the individuals and others say about their teaching, but examples of what they actually do.

Pat Hutchings, of the American Association for Higher Education, conducted a major study of American universities using teaching dossiers. She comments that: It has become a truism of portfolio use that putting them together is easier than knowing ‘what to do with them once you’ve got them.’ But .. the accounts from the campuses I’ve visited do not bear out that observation.

Even . . where portfolios are being used to determine tenure and promotion by committees having little experience with them - judgments have been arrived at, committees have stood behind these decisions, faculty have not flocked to grieve the process. In fact .. the process of reading and reviewing portfolios has turned out to be illuminating and significant. I hear chairs talking about a better understanding of teaching and learning .. as a result of reading portfolios. (Hutchings, 1993, p. 3)

- 13 -

- 14 -

References Centra, J. A. (1993). Reflective faculty evaluation: Enhancing teaching and determining faculty effectiveness. San Francisco: Jossey-Bass.

Hutchings, P. (1993). Introduction. In E. Anderson (Ed.), Campus use of the teaching portfolio: Twenty-five profiles. Washington, DC: American Association for Higher Education.

Shore, B. M., Foster, S. F., Knapper, C. K., Nadeau, G. G., Neill, N., & Sim, V. (1986). The CAUT guide to the teaching dossier, revised edition. Ottawa: Canadian Association of University Teachers.

Bibliography Anderson, E. (Ed.). (1993). Campus use of the teaching portfolio: Twenty-five profiles.

Washington, DC: American Association for Higher Education.

Edgerton, R., Hutchings, P., & Quinlan, K. (1991). The teaching portfolio: Capturing the scholarship in teaching. Washington, DC: American Association for Higher Education.

Gibbs, G. (1988). Creating a teaching profile. Bristol, England: Teaching and Educational Services.

O’Neal, C., & Wright, A. (1992). Recording teaching accomplishment: A Dalhousie guide to the teaching dossier. Halifax, Nova Scotia: Dalhousie University, Office of Instructional Development and Technology.

Roe, E. (1987). How to compile a teaching portfolio. Kensington, New South Wales, Australia: Federation of Australian University Staff Associations.

Seldin, P. (1991). The teaching portfolio: A practical guide to improved performance and promotion/tenure decisions. Bolton, MA: Anker.

Seldin, P. & associates (1993). Successful use of teaching portfolios. Bolton, MA: Anker.

- 15 -

- 16 -

Appendix A: Statement on Effective Teaching at Queen’s From the Report of the SCAD Subcommittee on Teaching Performance, November 1995.

- 17 -

- 18 -

Statement on Effective Teaching at Queen’s Through discussion, consultation and reading, the sub-committee (on teaching performance) has developed the following statement on effective teaching and learning.

This statement is not intended to be exhaustive. Rather, it intends to signal some general criteria on what constitutes effective teaching and learning in the Queen’s context. The sub-committee takes a broad view of what constitutes teaching. Teaching includes not just in-class interaction between students and professors, but also, for example, such things as academic advising and graduate supervision.

(1) Effective teaching is a scholarly activity which is integral to the duties of all faculty members at Queen’s University, and to which they are expected to devote substantial proportions of their professional time. Effective teachers place high value on the teaching enterprise and the subject taught, manifest knowledge, interest and enthusiasm, and provide appropriate intellectual challenges to students. By inspiring and encouraging students, effective teachers draw students into the world of the disciplines, the university, and the habit for inquiry that guides the life-long search for understanding.

(2) • • • • •

Effective teachers have excellent communication skills, which include: clarity in the organization and presentation of ideas consistency and clarity on expected standards of student work timely, appropriate, and helpful assessment of student performance constructive feedback to students opportunities for interaction with individual students and among students

(3) Effective teaching employs appropriate curriculum design delivery, and attends to the development, evaluation, and revision of curricula. Effective teachers employ appropriate course design and instructional methods, and are consistent in their attention to the development, evaluation and revision of courses. Materials and teaching strategies should: • • • •

be academically challenging encourage critical thought and intellectual exchange take account of recent developments in scholarship reflect the diversity of student experience and issues, and the breadth and depth of their knowledge • be well-organized and coherent • be stimulating, responsive, flexible and open to modification in keeping with students’ needs - 19 -

•

take account of recent developments in the delivery of curriculum, such as innovative instructional technologies and alternative teaching strategies

(4) Effective teachers recognize and engage with the diversity of student experience and intellectual perspectives. Teaching is a highly complex interplay of relationships between teachers and students within which an effective teacher reveals and encourages respect for differences amongst students and seeks to draw on that diversity in a way that builds a constructive experience of learning for all. Effective teachers are therefore committed to the equitable treatment of all students and to understanding and removing barriers to learning that may have impeded the academic progress of those who are non-traditional students.

(5) Effective teachers promote both independent and collaborative learning on the part of students by fostering the talents, skills, abilities, and most important, the desire of students to take responsibility for continued learning.

(6) Effective teachers are reflective, self-critical and flexible. They consistently seek to learn from their students, from their own teaching, and from the teaching of others, and, in response, are willing to modify their instructional approaches. The committed teacher also serves as a role model and mentor to colleagues. What is effective teaching may vary with particular disciplines as teachers seek to address a wide variety of students, and approach diverse topics in diverse ways at different levels of expertise.

Effective teaching and learning occur through intensive interaction of teachers and students in a variety of places, both inside and outside the classroom, and as changing technologies offer new opportunities for expanding and diversifying the contexts of learning.

- 20 -

Appendix B: Possible Items for a Teaching Dossier

From CAUT Guide to the teaching dossier: Its preparation and use. Ottawa: Canadian Association of University Teachers, 1986.

- 21 -

- 22 -

Possible Items for a Teaching Dossier THE PRODUCTS OF GOOD TEACHING 1. Students’ scores on teacher-made or standardized tests, possibly before and after a course has been taken as evidence of learning.

2. Student laboratory workbooks and other kinds of workbooks or logs.

3. Student essays, creative work, and project or field-work reports.

4. Publications by students on course-related work.

5. A record of students who select and succeed in advanced courses of study in the field.

6. A record of students who elect another course with the same professor.

7. Evidence of effective supervision of Honors, Master’s or Ph.D. theses.

8. Setting up or running a successful internship program.

9. Documentary evidence of the effect of courses on student career choice.

10. Documentary evidence of help given by the professor to students in securing employment.

11. Evidence of help given to colleagues on teaching improvement.

MATERIAL FROM ONESELF Descriptive material on current and recent teaching responsibilities and practices.

12. List of course titles and numbers, unit values or credits, enrolments with brief elaboration.

13. List of course materials prepared for students.

14. Information of professor’s availability to students.

- 23 -

15. Report on identification of student difficulties and encouragement of student participation in courses or programs.

16. Description of how films, computers or other non-print materials were used in teaching.

17. Steps taken to emphasize the interrelatedness and relevance of different kinds of learning.

Description of steps taken to evaluate and improve one’s teaching.

18. Maintaining a record of the changes resulting from self-evaluation.

19. Reading journals on improving teaching and attempting to implement acquired ideas.

20. Reviewing new teaching materials for possible application.

21. Exchanging course materials with a colleague from another institution.

22. Conducting research on one’s own teaching or course.

23. Becoming involved in an association or society concerned with the improvement of teaching and learning.

24. Attempting instructional innovations and evaluating their effectiveness.

25. Using general support services such as the Education Resources Information Centre (ERIC) in improving one’s teaching.

26. Participating in seminars, workshops and professional meetings intended to improve teaching.

27. Participating in course or curriculum development.

28. Pursuing a line of research that contributes directly to teaching.

29. Preparing a textbook or other instructional materials.

30. Editing or contributing to a professional journal on teaching one’s subject.

- 24 -

INFORMATION FROM OTHERS Students 31. Student course and teaching evaluation data which suggest improvements or produce an overall rating of effectiveness or satisfaction.

32. Written comments from a student committee to evaluate courses and provide feedback.

33. Unstructured (and possibly unsolicited) written evaluations by students, including written comments on exams and letters received after a course has been completed.

34. Documented reports of satisfaction with out-of-class contacts.

35. Interview data collected from students after completion of a course.

36. Honors received from students, such as being elected “teacher of the year�.

Colleagues 37. Statements from colleagues who have observed teaching either as members of a teaching team or as independent observers of a particular course, or who teach other sections of the same course.

38. Written comments from those who teach courses for which a particular course is a prerequisite.

39. Evaluation of contributions to course development and improvement.

40. Statements from colleagues from other institutions on such matters as how well students have been prepared for graduate studies.

41. Honors or recognition such as a distinguished teacher award or election to a committee on teaching.

42. Requests for advice or acknowledgement of advice received by a committee on teaching or similar body.

Other Sources 43. Statements about teaching achievements from administrators at one’s own institution.

- 25 -

44. Alumni ratings for other graduate feedback.

45. Comments from parents of students.

46. Reports from employers of students (e.g. in a work-study or “cooperative� program).

47. Invitations to teach for outside agencies.

48. Invitations to contribute to the teaching literature.

49. Other kinds of invitations based on one’s reputation as a teacher (for example, a media interview on a successful teaching innovation).

- 26 -

Appendix C: Creating the Foundations for a Teaching Dossier

- 27 -

- 28 -

Creating the Foundations for a Teaching Dossier Think about the situation where you have taught most often.

Teaching situation: Note your initial reactions to three basic questions: 1. How do I teach?

I use the following types of instructional methods, materials, and techniques when teaching in this situation:

2. Why do I teach this way?

Central to my philosophy of teaching in this situation are my beliefs that...

3. Does it work?

The types of evidence that I currently have (or, could collect in the future) to demonstrate my effectiveness in teaching in this situation include:

Once you have explained the way you teach in a single situation, you can expand your thinking to examine and describe your teaching in other situations. The overall shape of the argument, in creating a teaching dossier, remains the same: teachers must explain what and how they teach, why they teach that way, and whether or not it works.

- 29 -

- 30 -

Appendix D: Developing a Statement of your Teaching Philosophy

Copyright 1997. Professional and Organizational Development Network in Higher Education, Valdosta, Georgia. Reprinted with permission. From Teaching Excellence, Vol. 9, No. 3.

- 31 -

- 32 -

Developing a Philosophy of Teaching Statement

by Nancy Van Note Chism, Ohio State University When asked to write a statement on their philosophy of teaching, many college teachers react in the same way as professionals, athletes, or artists might if asked to articulate their goals and how to achieve them: “Why should I spend time writing this down?

Why can’t I just do it?” For action-oriented individuals, the request to write down one’s philosophy is not only mildly irritating, but causes some anxiety about where to begin.

Just what is meant by a philosophy of teaching statement anyway?

In the current academic climate it is likely that most faculty will be asked for such a statement at some point during their careers. The emphasis on portfolios for personnel decision making, new commitment by institutions to the teaching mission, and the tight academic job market have stimulated more requests of college teachers to articulate their philosophies. At many colleges and universities the philosophy of teaching statement is becoming a regular part of the dossier for promotion and tenure and the faculty candidate application package. Such statements are often requested of nominees for teaching awards or applicants for funds for innovative educational projects.

Besides fulfilling requirements, statements of teaching philosophy can be used to stimulate reflection on teaching. The act of taking time to consider one’s goals, actions, and vision provides an opportunity for development that can be personally and professionally enriching. Reviewing and revising former statements of teaching philosophy can help teachers to reflect on their growth and renew their dedication to the goals and values that they hold.

The Format of the Statement One of the hallmarks of a philosophy of teaching statement is its individuality.

However, some general format guidelines can be suggested: • Most philosophy of teaching statements are brief, one or two pages long at most.

For some purposes, an extended description is appropriate, but length should suit the context.

• Most statements avoid technical terms and favor language and concepts that can be broadly appreciated. If the statement is for specialists, a more technical approach can be used. A general rule is that the statement should be written with the audience in mind.

• Narrative, first-person approaches are generally appropriate. In some fields, a more creative approach, such as a poem, might be appropriate and valued; but in most, a straightforward, well-organized statement is preferred.

- 33 -

• The statement should be reflective and personal. What brings a teaching philosophy to life is the extent to which it creates a vivid portrait of a person who is intentional about teaching practices and committed to career.

Components of the Statement The main components of philosophy of teaching statements are descriptions of how the teachers think learning occurs, how they think they can intervene in this process, what chief goals they have for students, and what actions they take to implement their intentions.

Conceptualization of learning. Interestingly, most college teachers agree that one of their main functions is to facilitate student learning; yet most draw a blank when asked how learning occurs. This is likely due to the fact that their ideas about this are intuitive and based on experiential learning, rather than on a consciously articulated theory. Most have not studied the literature on college student learning and development nor learned a vocabulary to describe their thinking. The task of articulating a conceptualization of learning is therefore difficult.

Many college teachers have approached the work of describing how they think student learning occurs through the use of metaphor. Drawing comparisons with known entities can stimulate thinking, whether or not the metaphor is actually used in the statement. For example, when asked to provide a metaphor, one teacher described student learning in terms of an amoeba. He detailed how the organism relates to its environment in terms of permeable membranes, movement, and the richness of the environment, translating these into the teaching-learning context by drawing comparisons with how students reach out and acquire knowledge and how teachers can provide a rich environment. Grasha (1996) has done extensive exploration of the metaphors that college students and teachers use to describe teaching and learning. An earlier classic that also contains an exploration of metaphors of teaching and learning is Israel Scheffler’s The Language of Education (1960). Reinsmith (1994) applies the idea of archetypes to teaching. Such works might be consulted for ideas.

A more direct approach is for teachers to describe what they think occurs during a learning episode, based on their observation and experience or based on current literature on teaching and learning. Some useful sources that summarize current notions of learning in a very accessible way are contained in Svinicki ( 1991), Weinstein & Meyer (1991), and Bruning (1994). Teachers can also summarize what they have observed in their own practice about the different learning styles that students display, the different tempos they exhibit, the way they react to failure, and the like. Such descriptions can display the richness of experience and the teacher’s sensitivity to student learning.

Conceptualization of teaching. Ideas on how teachers can facilitate the learning process follow from the model of student learning that has been described. If metaphors have - 34 -

been used, the teacher role can be an extension of the metaphor. For example, if student learning has been described as the information processing done by a computer, is the teacher the computer technician, the software, the database? If more direct descriptions of student learning have been articulated, what is the role of the teacher with respect to motivation? To content? To feedback and assessment? To challenge and support?

How can the teacher respond to different learning styles, help students who are frustrated, accommodate different abilities?

Goals for students. Describing the teacher role entails detailing how the teacher can help students learn, not only a given body of content, but also process skills, such as critical thinking, writing, and problem solving. It also includes one’s thoughts on lifelong learning -- how teachers can help students to value and nurture their intellectual curiosity, live ethical lives, and have productive careers. For most teachers, it is easier to begin with content goals, such as wanting students to understand certain aerodynamic design principles or the treatment of hypertension. The related process goals, such as engineering problem solving or medical diagnostic skills, might be described next. Finally, career and lifelong goals, such as team work, ethics, and social commitment, can be detailed.

Implementation of the philosophy. An extremely important part of a philosophy of teaching statement is the description of how one’s concepts about teaching and learning and goals for students are translated into action. For most readers, this part of the statement is the most revealing and the most memorable. It is also generally more pleasurable and less challenging to write. Here, college teachers describe how they conduct classes, mentor students, develop instructional resources, or grade performance. They provide details on what instructional strategies they use on a day-to-day basis. It is in this section that teachers can display their creativity, enthusiasm, and wisdom. They can describe how their No Fault Test System or videotaping technique for promoting group leadership skills implements their notions of how teachers can facilitate learning. They can portray what they want a student to experience in the classes they teach, the labs they oversee, the independent projects they supervise. They can describe their own energy level, the qualities they try to exhibit as a model and coach, the climate they try to establish in the settings in which they teach.

Personal growth plan. For some purposes, including a section on one’s personal growth as a teacher is also important in a statement of teaching philosophy. This reflective component can illustrate how one has grown in teaching over the years, what challenges exist at the present, and what long-term goals are projected. In writing this section, it helps to think about how one’s concepts as well as actions have changed over time. It might be stimulating to look at old syllabi or instructional resources one has created, asking about implicit assumptions behind these products. Dialogue with colleagues, comparison of practices with goals, and examination of student or peer feedback on teaching might help with the task of enumerating present questions, puzzles, and challenges. From these, a vision of the teacher one wants to become will emerge. Describing that teacher can be a very effective way to conclude a philosophy of teaching statement.

- 35 -

Examples of Statements By far, the best philosophy of teaching statement examples for most college teachers are those of peers who teach in similar settings or disciplines. Since statements tend to be tailored to specific contexts, peer examples are thus highly appropriate models.

Dialogue with colleagues on these statements can help to stimulate ideas for one’s own statement as well.

Other examples are contained in several recent books on teaching portfolios, such as Seldin (1993) and O’Neil & Wright (1993). Reflective books on effective college teaching often contain extensive descriptions of teaching philosophies, such as the chapter on “Developing a Personal Vision of Teaching” in Brookfield’s The Skillful Teacher (1990) and “Three Teaching Principles” in Louis Schmier’s Random Thoughts (1995).

References Brookfield, S. (1990). The skillful teacher. San Francisco: Jossey-Bass.

Bruning, R. (1994). The college classroom from the perspective of cognitive psychology. (pp. 3-22) In K. Pritchard & R. Sawyer (Eds.), Handbook of college teaching. Westport, CT: Greenwood Press.

Grasha, A. (1996). Teaching with style. Pittsburgh: Alliance Publishers. O’Neil, C., & Wright, A. (1993). Recording teaching accomplishment. (4th ed). Halifax, Nova Scotia: Dalhousie University.

Reinsmith, W. (1994). Archetypal forms in teaching. College Teaching, 42, 131-136.

Scheffler, I. (1960). The language of education. Springfield, IL: Charles Thomas.

Seldin, P. (1991). The teaching portfolio. Bolton, MA: Anker.

Seldin, P., & Associates (1993). Successful use of teaching portfolios. Bolton, MA: Anker.

Schmier, L. (1995). Random thoughts: The humanity of teaching. Madison, WI: Magda Publications.

Svinicki, M. (1991). Practical implications of cognitive theories. In R. Menges & M. Svinicki, (Eds.) College teaching: From theory to practice. New Directions for Teaching and Learning, 45, pp. 27-37. San Francisco: Jossey-Bass.

Weinstein, C., & Meyer, D. (1991). Cognitive learning strategies and college teaching.

In R. Menges & M. Svinicki, (Eds.) College teaching: From theory to practice. New Directions for Teaching and Learning, 45, pp. 15-26. San Francisco: Jossey-Bass.

- 36 -

Appendix E: Guidelines for the Interpretation of Student Evaluations

From the Report of the SCAD Subcommittee on Teaching Performance, November 1995.

- 37 -

- 38 -

Guidelines for the Interpretation of Student Evaluations Student ratings are the most commonly used method of evaluating teaching. Reviews of the extensive body of research on student evaluation of teaching generally conclude that student ratings tend to be reliable, valid, relatively unbiased, and useful. Murray (1994) selects from the research the following key findings: a) Student evaluations are generally consistent across raters, rating forms, courses and time periods for a given instructor. b) Student evaluations correlate moderately to highly with evaluations made of the same instructor by independent observers. c) Student evaluations correlate significantly with various objective indicators of student performance, such as student performance on standardized examinations. d) Student evaluations have generally low correlations with extraneous factors such as class size, severity of grading, etc.

Those conclusions are the basis for the implementation of a university-wide student rating procedure at Queen’s, using the USAT form. However, USAT results require careful interpretation if they are to be fairly and effectively used for the assessment and improvement of teaching.

The following guidelines are intended for anyone attempting to interpret the results of the USAT form or to use these results for administrative purposes. It is important to understand how student, course, or teacher characteristics can influence student evaluations; individually the effects of these characteristics on ratings are small, but the combination of several factors can have a significant effect. The purpose of the guidelines is to ensure that factors affecting student evaluations are seriously considered in interpreting the results. Some of these guidelines indicate ways to reduce the likelihood of bias, while others reflect the fact that one must always be sensitive to the potential for bias, and allow that sensitivity to inform one’s interpretation and use of the results.

1. Complement results from student evaluations with data from other sources.

Since no one source of evidence about teaching is definitive, multiple sources of data (peer observation, self-assessment, measures of student achievement, for example) should be used wherever possible, as is done in the compilation of a teaching dossier.

- 39 -

2. Use several sets of evaluation results.

• For decisions about the overall teaching effectiveness of an instructor, review evaluation results from different courses and different types of courses (lectures and seminars, graduate and undergraduate, for example) taught over an extended period of time. Results from at least 5 classes is best, including ratings from two or more courses from three or more terms.

• For course improvement, multiple evaluations of a single course over time and/or with different instructors can be useful.

3. Ensure that a sufficient number of students have responded -- the absolute number and the proportion of students responding are both important.

• The response rate from each class should be at least 65% to ensure that results are representative.

• Treat data from fewer than 10 students with caution; data from 10 to 15 students are acceptable; with 25 students, reliability estimates are excellent. When results are only available from very small classes, combine data from several classes to improve reliability.

4. Consider course, student, and teacher characteristics.

• Results of student evaluations need to be interpreted in context. While the effect of course, student, and teacher characteristics on results of student evaluations does not always represent bias, complete disregard for the impact of such factors may lead to a misleading assessment of teaching effectiveness. Marsh (1987) found that several factors combined could account for as much as 15% of the variance in student ratings he analyzed.

• Characteristics known to have an impact, and which may be a source of bias, include: Gender. There is some evidence that female instructors may receive lower ratings from male students and higher ratings from female students. The ratings of male instructors do not seem to be affected by the gender of the student.

Class size. There is a slight tendency for smaller classes to receive higher ratings.

Course level. Higher level courses, especially graduate courses, tend to be rated more positively.

- 40 -

Discipline. Mathematics, natural science and engineering courses tend to be rated more negatively. In male-dominated disciplines, female instructors tend to be rated more negatively than male instructors, especially by male students.

Required or elective. Elective courses (and the teachers of these courses), especially courses in the major field, tend to receive higher ratings than do required courses.

Difficulty level of course. Within disciplines, teachers of courses perceived to be more difficult tend to be rated more highly. Note that courses taught by women tend to be perceived as “easier” than courses taught by men.

Race. No studies have been reported that investigate systemic racial bias in student ratings. Centra (1993) proposes that results of gender studies suggest ways that race may affect ratings, for example a class of same-race teacher and students may result in a somewhat higher rating than one where race differs. Ng’s (1994) experience as a minority female faculty member suggests that minorities in positions of authority often have their knowledge and expertise questioned.

• Other contextual factors with important consequences for student ratings are: Course topics. Courses (and instructors) that explicitly challenge students’ assumptions about certain topics, especially sensitive or controversial topics such as gender, race, or sexual orientation, can generate resistance among students, which may be reflected in negative ratings.

Teaching style. The questions used on the rating form, and the students completing the evaluation, generally make assumptions about the nature of good teaching - although the USAT form attempts to avoid such limitations by having very general questions). The ratings of instructors with a teaching style that matches the assumptions of the students and the form are easier to interpret than ratings of teachers with a less conventional style. Instructors with unconventional assumptions/styles may wish to gather additional data, for example by use of optional questions. Persons interpreting results should be sure to review all data, and not restrict their view to overall effectiveness ratings.

An additional concern is that students’ assumptions about the nature of good teaching can be influenced by gender role expectations. For example, female teachers are generally rated more highly than males in terms of their interpersonal skills, yet women’s high interpersonal skills ratings do not translate into higher “overall” ratings and women with lower rated interpersonal skills may receive quite poor “overall” ratings.

Another caveat concerning teaching style is that course characteristics, especially class size, affect the appropriateness and effectiveness of an instructor’s preferred - 41 -

teaching style. Low ratings may indicate a mis-match rather than an absolute measure of ineffectiveness; while it is generally desirable that instructors adapt style to situation, there are limits to this principle in practice.

5. Use good, fair judgement and remain open to situational explanations for ratings results.

• Individual instructors may receive high student ratings in a situation that the research literature suggests should result in poor evaluations. For example, a female instructor teaching a large required course in a male dominated discipline may be rated very highly. It is tempting to conclude that such cases refute the literature and deny the usefulness of interpretive guidelines. Alternatively, one might conclude that the instructor has responded admirably to a challenging teaching situation. High student ratings may also reflect a high level of departmental support for the instructor and/or the course.

• Do not give undue weight to any single one of the above factors, or to other factors such as time of day when class was taught or leniency in grading -- which have no apparent effect on student ratings, unless an instructor provides evidence in his or her self-report for the influence of these variables; if you or others have such evidence, that evidence should be taken into consideration.

• Above all, remember that student ratings are only one source of data.

LITERATURE CITED Centra, J.A. (1993). Reflective faculty evaluation: Enhancing teaching and determining faculty effectiveness. San Francisco: Jossey-Bass.

Marsh, H.W. (1987) Student evaluations of university teaching: Research findings, methodological issues, and directions for future research. International Journal of Educational Research, 11, 253-288.

Murray, H. (1994). The impact/consequences of student ratings of teaching. Presentation at the CSSHE Annual Meeting, Calgary, June 16-18, 1994.

Ng, R. (1994). Sexism and racism in the university: Analyzing a personal experience.

Canadian Women Studies 14 (2), 41-46.

LITERATURE REVIEWED Basow, S. (1995). Student evaluations of college professors: When gender matters.

Unpublished manuscript. Lafayette College, Psychology Department.

- 42 -

Basow, S. and Silberg, N.T. (1987). Student evaluations of college professors: Are male and female professors rated differently? Journal of Educational Psychology, 79 (3), 308-314.

Cashin, W.E. (1990). Student ratings of teaching: Recommendations for use. IDEA Paper No. 22. Manhattan: Kansas State University, Center for Faculty Evaluation and Development.

Centra, J.A. (1993). Reflective faculty evaluation: Enhancing teaching and determining faculty effectiveness. San Francisco: Jossey-Bass.

Feldman, K.A. (1993). College students’ views of male and female college teachers: Part II--Evidence from students’ evaluations of their classroom teachers. Research in Higher Education, 34 (2), 151-191.

Franklin, J. and Theall, M. (1994). Student ratings of instruction and sex differences revisited. Tucson, Arizona: University of Arizona, Test Scoring and Evaluation Services.

Marsh, H.W. (1987) Student evaluations of university teaching: Research findings, methodological issues, and directions for future research. International Journal of Educational Research, 11, 253-288.

- 43 -

Centre for Teaching and Learning, Queen’s University, Kingston, Ontario K7L 3N6 Phone: 613-533-6428 Fax: 613-533-6735 Email: ctl@queensu.ca URL: www.queensu.ca/ctl