PGT Teaching Review Report 2011

Response on behalf of PGT students to Oxford’s Review of its Teaching Model1

!

Contents:

1. Introduction 2. Methodology 3. Critical Issues 4. Further Work

Annexes: A. Humanities B. Social Sciences – Professional Courses C. Social Sciences – Non-Professional Courses D. MPLS E. Medical Sciences F. Questions asked in focus groups G. Data on student attainment on PGT programs sorted by Division and whether a student has previously undertaken a degree at Oxford.

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

1

For more information on anything contained within this response please contact Jonny Medland, OUSU Graduate Academic Affairs Officer, gradacaff@ousu.org or Beth Evans, OUSU Vice-President (Graduates), graduates@ousu.org

Introduction Over the first fortnight of Trinity Term 2010 OUSU undertook a consultation with approximately 400 undergraduate students about their experience of teaching and learning at Oxford2. This consultation was based around 40 focus groups that ran in 10 colleges, organized by Division. Our findings were reported to Education Committee, the Conference of Colleges and the Divisions in the latter half of Trinity Term and have since been circulated broadly within the collegiate University.

At the time it was felt that “a similar exercise for PGT students” would have value (Education Committee, week 6 Trinity 2010). We have conducted such an exercise, organized along similar lines to our undergraduate consultation. Prior to focus groups beginning, questions were discussed and refined in conjunction with relevant University officers including the Pro Vice-Chancellor (Education), Head of Education Policy Support, Director of Student Administration & Services and Chair of Conference’s Graduate Committee. Questions were also developed in conjunction with the Oxford Learning Institute who advised on how best to phrase specific questions.

In addition to conducting focus groups for students on PGT courses in each of the Divisions we have reviewed existing survey data and data on student attainment on PGT courses prepared by Student Data Management & Analysis. Through this we have reviewed PGT student expectations and experiences of Oxford’s academic provision.

As stated last year, we are aware of the financial circumstances in which this Review of Teaching takes place. The circumstances around Oxford’s PGT provision have only become more complex over the last 12 months as a result of changes to the Tier 4 visa system, possibly declining numbers of graduate applications (albeit after a period of rapid growth), cuts in Research Council funding and increasing course fees arising from continuing student demand and reductions in the HEFCE T Grant. These events do, however, allow an opportunity to re-evaluate the provision that Oxford makes for PGT students. As became apparent through our focus groups, this provision needs to be looked at holistically – many of the issues identified are not exclusively academic but relate also to student expectations, induction to Oxford and the size & shape of the student body. In addition to these we found critical academic issues that vary at both course and Divisional level but include feedback provided to students, methods of assessment and the rationale for different PGT programs.

As the evidence-gathering phase of the Review of Teaching draws to a conclusion and Departments begin to examine how they might alter teaching provision we suggest that this report be seen in conjunction with our report on undergraduate teaching that was circulated last year. Many of the issues that we identify in both reports are similar; and can be addressed simultaneously. We continue to believe that there is potential for making cost-savings within teaching provision while continuing to provide world-class teaching at both undergraduate and postgraduate level. As our consultations with hundreds of students in the last two Trinity Terms demonstrate, students are engaged with this review. Departments making use of the data generated by the Review presents an opportunity for engaging students further on improving their teaching and we urge that such discussions take place at a departmental level as discussed in the ‘Further Work’ section of this report. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

2

This report can be found online at http://www.ousu.org/about/Teaching%20Review%20Report.pdf

Methodology In light of the approach we took last year we decided to repeat the running of focus groups for students on PGT programs. In recognition of the differences between graduate and undergraduate study these were primarily organized at course or departmental level with student representatives in Departments being asked to assist with the organization of focus groups. We did, however, feel that there would be a value to conducting focus groups in colleges and so all MCR Presidents were asked to assist in running a focus group for students on PGT programs in their colleges. Focus Groups ran for the following subjects/colleges.

Humanities All PGT programs in the Faculty of Classics All PGT programs in the Faculty of English All PGT programs in the Faculty of Music All PGT programs in the Faculty of Medieval and Modern Languages All PGT programs in the Faculty of Theology MSc Women’s Studies Social Sciences All PGT programs in the School of Archaeology MSc Global Governance & Diplomacy MSc Refugee & Migration Studies MPhil International Relations Master of Business Administration (MBA) Masters in Law & Finance (MLF) Postgraduate Certificate of Secondary Education (PGCE) Medical Sciences MSc Pharmacology MSc Psychological Research MSc Neuroscience Maths, Physical & Life Sciences MSc Applied Statistics MSc Zoology (by Research) Colleges Linacre St. Antony’s Focus groups lasted for between 1 hour and 1 hour 45 minutes depending on how long participants wished to continue discussions for. Focus groups were typically attended by around 7 or 8 current students on the course (some had former students who had continued to DPhil study at Oxford in attendance). All focus groups followed a structure annexed at F which had previously been discussed both with student representatives in Departments and relevant University officers. Every Department offering PGT programs was contacted for details of their student representatives at PGT level with a relatively high number of departments not responding to requests for this information. This, combined with the failure of student representatives on particular courses to respond to emails (and the understandable concerns of some student

representatives that students on their PGT programs would be unwilling to attend a focus group owing to proximity to dissertation deadlines and examinations) account for the number of focus groups that we ran. In total 21 focus groups were ran over the first 3 weeks of Trinity Term. All focus groups had extensive notes taken in them by members of the OUSU Executive and were run either by members of the OUSU Executive or course reps for the subject in which the group was taking place.

Since we were aware that we would be unable to run focus groups for as many PGT programs as we would have liked, we circulated a survey via OUSU’s all-PGT student mailing list, MCR mailing lists and certain Departmental mailing lists. This survey consisted solely of the focus group structure, enabling all students enrolled on PGT programs to participate in this consultation. It was not intended to provide a ‘statistically significant’ dataset in the manner of surveys such as the Student Barometer but was instead meant to provide a further opportunity for students to give input on the specific questions that we were asking. Responses from this survey are included in the analysis which we provide at a Divisional level later in this report. In addition to responses to the circulated survey we also received a number of detailed written responses to emails regarding our consultation. These were received both from course representatives and from other students enrolled on PGT programs and have been used in compiling this report. These included feedback collated by course representatives earlier in this academic year which was submitted to us.

As a result of the great deal of data on the PGT experience of Oxford already in existence we also chose to review existing survey data from the PTES survey from 2009 and various waves of PGT-specific Student Barometer survey data. Data on student progression onto PGR study from PGT programs and student attainment on PGT courses provided by Student Data Management & Analysis (SDMA) is also utilized.

The varied methodologies we employed were designed to ensure that we received feedback on PGT programs from as many students as possible with focus groups being particularly valuable in order to discover more about the experiences of students on these courses beyond headline figures typically generated by surveys. Possible lessons from this form of methodology for collecting future student feedback in departments will be discussed later in the ‘Further Work’ section of this report.

Critical Issues The following are what we see as the critical issues emerging from this consultation.

Detailed feedback provided by students across Divisions is detailed in Annexes A-E; the critical issues are included if they appeared in numerous focus groups across Divisions, if they had a significant detrimental impact on the experiences of students or if they could be rectified comparatively easily with significant benefit to students.

Provision of feedback for students While we asked a specific question about feedback in focus groups, students made specific reference to feedback in the context of induction to Oxford, departmental provision and – particularly – the relationship between assessment and the structure of their course. A desire for more feedback was felt across all four Divisions and was typically placed in the context of wanting more feedback so students could learn how they could do better in future work.

Given the striking frequency with which feedback was raised we pressed students further in focus groups about what form of feedback would be most useful to them. A consistent picture emerged across a broad range of courses. Students typically felt that marks on work were useful to them with this desire often being placed in the context that they were aware Oxford placed less of a focus on providing marks on work relative to other, competitor universities. They particularly wanted annotated essays/written work to be returned to them with comments broad enough to assist them in the development of their future work. Feedback provided on ‘cover sheets’ was felt to be less valuable as students had more difficulty seeing how this would interact with written work.

While Education Committee has recently consulted on feedback for assessed work for PGT students, our discussions with such students have indicated that the appetite among students for more feedback on their work is continuing. Although good relationships between supervisors and students were seen as being among the most positive aspects of the Oxford educational system a more consistent approach to feedback across courses would be highly beneficial to students on PGT courses.

Variability of provision between tutors As we found last year at the undergraduate level, students regularly voiced concern at a lack of consistency in teaching provision. Many tutors and Course Directors were mentioned by name in focus groups and in written submissions to us as providing highquality teaching and going above and beyond what could reasonably be expected of them in providing for students. This was, however, countered by the fact that many students were concerned about the quality of teaching they were receiving. This was expressed in a range of focus groups in terms of fairness for students; people felt that if some students did better than others as a result of the quality of supervision received then this was fundamentally unfair and did not reflect the competing merits of different students.

It was striking that students frequently made references in focus groups to burdens on academic staff as a result of heavy teaching loads. This was particularly pronounced in the Humanities where students referred to academics teaching outside of their

contracted hours for minimal additional salary out of passion for their subject and developing graduate students. It was, however, felt that such arrangements were both unsustainable and undesirable. Students reported feeling limited in their ability to approach academics as a result of these arrangements and stated that they felt less able to ask for provision to be made for them as a result of academics essentially working “for free” in teaching them. Students regularly made reference to their course fees in the context of expectations around feedback, teaching and contact time with tutors.

They also expressed confusion as to where their course fees were going if not towards the costs of providing their courses, reflecting disquiet among PGT students about the range of (arguably justified) cross-subsidies that exist within the University currently.

Diversity of Assessment A major finding of our consultations with undergraduate students was frustration at the “misalignment of the teaching model with the examination system”. This was less pronounced at the PGT level with students preferring diverse assessment practices where they existed. Students often drew distinctions between undergraduate and PGT programs in discussions on this point, feeling that the model of undergraduate assessment prevalent at Oxford would be unsuitable for PGT courses. Where students were heavily assessed on the basis of three-hour written exams they often indicated that they would prefer for coursework, dissertations and practical work to make up a larger part of their courses.

Students on multiple programs indicated that they would like assessment to include more practical assignments. This was heavily pronounced among students doing professional courses in the Social Sciences Division but was also found throughout a broad range of students in PGT courses. Students gave a range of rationales for these attitudes but among the most prevalent was a sense that students should be examined on the basis of skills which would be necessary for what they would be doing after Oxford. This could include – for example – skills at interacting within a group, at presenting work and (where relevant) for researching and writing policy briefings. While the exact form of suitable assessment would vary by Department and course, students nevertheless saw value in further diversification of assessment practices.

The objectives of PGT programs It was striking that students saw a range of purposes behind the programs on which they were studying. This is unsurprising given the extensive range of PGT programs within the University. A working party on PGT programs within the University in 2004/05 identified three broad sorts of PGT programs – those which typically led to a DPhil, those which were intended as a “stand alone” qualification and hybrids of the two. Many PGT courses at that point were felt to fall into this third category. It is this category that was felt to cause problems for a range of students. While courses in MPLS & Medical Sciences were typically seen as preparing students for DPhil study, those in the Humanities and Social Sciences were sometimes seen as having less clear objectives. American students frequently indicated that while their programs would enable them to do a DPhil in the United Kingdom, they would not feel prepared to apply to the United States as a result of the different focus of UK and US programs and the comparative lack of language provision at Oxford relative to in the United States.

In Social Sciences many students felt that the objectives of their courses were unclear. It was, again, particularly clear that students who had not done an undergraduate degree at Oxford were more likely to feel that their courses had ambiguous aims. Comparisons were regularly drawn by students to competitor universities outside of the United Kingdom, and particularly to courses in the United States which may offer a more “professional” focus than their most immediate counterparts at Oxford. This inclarity could usefully be clarified where it is felt to exist, both in terms of information made available to applicants and through greater information being provided to prospective students on alumni destinations from individual programs at Oxford. Students across a range of programs felt that greater contact with alumni would be valuable for purposes such as this, induction and future career development. Students across Divisions regularly made reference to the cost of their programs when explaining what they wished to gain from their time at Oxford, providing an indicator of student expectations being explicitly linked to the ‘value for money’ of Oxford’s provision for PGT students.

Repeated discussions about provision at other universities, particularly in the United States, provided reminders of the globally competitive field in which Oxford is competing to attract the best Taught Masters students.

Organisation of courses and competing areas of responsibility within the collegiate University Points were frequently made in discussions over induction, skills training and departmental provision about inclarity over aspects of course organization. This was felt to be detrimental to students – particularly those who had not previously studied at Oxford – in terms of them knowing who was responsible for providing specific aspects of their time at Oxford. It was highlighted that comparatively easy measures could improve organisation. These included a reduction in the number of emails students were receiving in favour of weekly e-newsletters and a timetable being provided to students both at the start of year and start of term outlining what courses would be happening and when. It was felt by students that this would enable them to plan their year more effectively and know what elements of their course were taking place and when. Data provided by SDMA is appended at G and provides information on student attainment on PGT programs controlled by whether or not students have done a previous program of study at Oxford. The disparities present across all Divisions are consistent, even when based upon a small sample size in MPLS & Medical Sciences and may reflect the difficulties existing with induction to Oxford at PGT level.

Related to this point was a desire from many PGT students to have assessed work spread over the course of the year to an extent that doesn’t currently exist. Students felt that the concentration of deadlines in Trinity Term often was not desirable and meant that students were not incentivized to work consistently over the course of the academic year. The issue was repeatedly raised of the difficulty of students making use of offered formative feedback given the concentration of deadlines. Multiple students made the case that the pace at which feedback was given meant that they could make no effective use of it on their courses. Students also wished to receive more information about opportunities including lectures and training taking place outside of their own Department in a more structured way than is currently the case. Many students felt that one of the great benefits of studying at Oxford was the range of opportunities which existed outside of individual courses and were keen for an organizational structure to support participation in a wide range of aspects of academic life.

Further Work At the time that we produced our report on undergraduate experiences of teaching the Review of Teaching was in its early stages with a great deal of work to be done on data collection and analysis. Over the last year much of this work has been undertaken with Departments and Divisions currently reviewing the data that has been collated on teaching.

The overall purpose of the Review of Teaching as endorsed by Education Committee was “to define and review the way that we teach in relation to our educational objectives, and consider how the academic priorities of the collegiate University can be accommodated in the light of the financial position prevailing�. Our consultations with undergraduate and PGT students have aimed to provide more detail on what objectives students have for their education at Oxford and how educational provision could be improved.

Our findings as discussed both here and in our report on undergraduate teaching reflect the consensus of student views from the consultations we have undertaken across a broad range of subject areas. It is nevertheless clear that differences on student views exist at departmental and course level; our collation of student views through Divisional structures means that detail has been lost which exists at course level. There will, of course, inevitably be differences of opinion between individual students on virtually all the areas that we discuss in this report.

As departments and colleges adjust their provision for students in light of the Review of Teaching there is as yet unfulfilled potential for further discussions to take place with students on the full range of issues that we have identified through our consultations.

When putting together focus groups with PGT students we were organizing a consultation through mechanisms that OUSU has historically had little interaction with.

This has largely been as a result of the necessity for OUSU officers to primarily engage with student representatives in colleges.

The evidence from this consultation is that PGT students see their colleges as bodies that serve a valuable social and pastoral function rather than an academic role. This provides an example of attitudes that OUSU and the University should take to developing postgraduate representation in departments. Many course representatives with whom we interacted asked us whether our consultation would be taken seriously within the University owing to difficulties that JCC reps have had in improving provision on their course. Others expressed uncertainty as to what their precise role should be, saying that while Departments were willing to make changes to certain elements of their courses, there appeared to be no clear mechanism by which students could substantively engage with how courses were delivered and improve the quality of teaching they received.

Improving PGT courses presents challenges for both the University and OUSU. The short period of time that many PGT students are at Oxford makes it difficult to get a sense of how PGT students find their time here and means that many students are reluctant to take time away from their degree to help improve their course. The course duration also means that many students simply do not have time to develop a coherent set of views on how their courses could be improved.

As the Review of Teaching moves on from the evidence-gathering phase there is a need to strengthen graduate academic representation in Departments to ensure that views of graduate students can shape their education to an extent that is not currently available. OUSU is currently taking steps to begin to address this issue – through increased provision for Divisional reps, interaction with Divisional structures, recognition in the 2009/10 report of its Strategic Review Group of the need for increased interaction with Departmental and Divisional reps and a new review of graduate representation within the University. We suggest that Departments and Divisions consider how best representation on educational issues in Divisions could be coordinated; the methods that MPLS uses brings student representatives together on a comparatively frequent basis on both course and Divisional level and is currently a model of good practice. This contrasts with difficulties which we found in certain subjects where we were simply unable to obtain details of course representatives from Departments & Faculties, making it impossible to meaningfully engage students on these courses with improving the quality of their education. We would highlight that our discussions with undergraduates from the 2009/10 academic year are currently being used to inform improving teaching provision across a range of subjects and suggest that both this report and our findings on undergraduate provision be used to inform discussions with current students at course and departmental level.

We would particularly commend the method of engaging students in group discussions on their academic experience at Oxford. Such discussions inevitably enable more detail to be developed than can be obtained through surveys that the University undertakes, immensely valuable though they are. This methodology is already widely used to collate student feedback at undergraduate level in colleges and is widely seen by students, student representatives and many colleges as being a valuable means of collating student views.

The key question related to improvement of existing teaching provision relates to the delivery and implementation of conclusions arising from the Teaching Review project.

Recent reviews of both the Humanities and Social Sciences Divisions have noted broad agreement among students, administrative and academic staff about the nature of the challenges facing the University in preserving and improving the quality of teaching in coming years. The shared interests of students and academic staff can be seen through repeated observations by students participating in focus groups of the burdens on academic time that exist at Oxford. The willingness of students to participate in representative networks and the consultations relating to teaching that we have undertaken indicates that Departments should engage in conversations with existing students as teaching provision is modified going forward. OUSU would be happy to support Departments and Divisions in any such projects.

Annexe A: Humanities Expectations of and induction to Oxford These questions were often elided together by participants in focus groups, with people feeling that their expectations of life at Oxford were heavily tied in to how they had been introduced to postgraduate study here. People’s expectations of the education they would be receiving were often heavily linked to the reputation of the ‘tutorial system’ used at the undergraduate level within Oxford’s Humanities provision with a sense that they were expecting intensive, weekly contact. Key points were: •

An expectation that work would be difficult, often expressed with a retrospective sense of trepidation. Participants made comments such as “From a different University you don’t really believe that you can do it” and “The jewel in the crown is obviously the tutorial system – I knew I wouldn’t get exactly an undergraduate course but I thought there’d be a big continuation from that system”.

•

In contrast to this people felt that they were getting less teaching than expected, a theme which was recurrent in subsequent discussion. Students indicated that “I did think a Taught Masters would mean more teaching – you can go to lectures but I’ve only had one lecture a week all year” and “I expected more work in terms of work being submitted to tutors rather than one 5,000 word essay per term”. Students felt that the sort of provision they were expecting included more seminars (led by academics rather than ‘discussions’) and thought that they might have the opportunity for more interdisciplinary work, particularly as referred to language teaching.. This was felt across a wide range of subjects with North American students in particular drawing attention to the entry requirements for many US DPhil programs that include knowledge of languages other than English.

With reference to induction, participants often felt that those students who had done a prior degree at Oxford had a large advantage when it came to being able to start work.

Comments such as “It’s really about knowing the system” and “You just need a linguistic list of Oxford to understand what to do” were frequent. As was the case in numerous other areas, comparisons were drawn to universities outside the UK, particularly in the USA with participants feeling that “In the North American system you’ve got people who are making sure that you know what’s going on”. Students felt that inclarity until well into their courses about the weighting of marks across different pieces of assessed work reduced their ability to know how to plan their time, with people reporting that such detailed guidance only emerged a limited period of time prior to assessment and, in some cases, after work had already been handed in. Students were unclear what the rationale for this was.

This was heavily related to a point that many people felt about the administration of their courses and the impacts that this could have on them. Students made comments such as “There’s too much work for one person to do” [with regard to course administration] and “The basic information is not there” with regard to details about course provision. When students were pressed on who they approached for information there was a sense that relevant knowledge was fragmented with people expressing views that “We don’t know who to go and speak to” and “If an induction

point glossed these sessions into one relevant document rather than splitting everything between my Department and the college then it would be a lot better”. Students often expressed views that no one person – including their tutors – seemed certain of agreed arrangements around the course and expressed frustration at this.

Academic elements to induction were also regularly mentioned with people indicating that more clarity in provision would be beneficial. This was expressed through comments such as “A checklist would be really useful for initial meetings with your supervisor – we don’t know what we’re meant to be asking and they don’t seem to know what they’re meant to be giving us”. People felt that more interaction with their Department over the summer would be useful to prevent arriving and feeling that “You’re just left to figure it out for yourself”. Students typically felt that issues relating to feedback and assessment criteria began from early on course, indicating that they believed “You’re really just left to figure it out from the start”.

Feedback In common with students in other Divisions feedback was a point that students saw as being hugely significant to their experience at Oxford. Many of the views across Divisions were being particularly strongly expressed in Humanities subjects. The key themes were: •

Variability of feedback between tutors. Students regularly assessed sentiments such as “In the specific subject it really depends on the tutor how good feedback is” and “The fundamental point is that there’s just been an enormous spectrum…a lack of consistency”. Students recognized that “it can be of really good quality, some of our supervisors have provided useful feedback” but that it “all depends on the tutor you have”. This was seen as potentially leading to unfair outcomes with people agreeing with views that “The biggest worry I have is that different levels of feedback will impact the quality of final work – if someone does better because of the quality of supervision they receive then that’s not fair. “

•

The relationship between feedback and assessment. As was also raised in other Divisions students felt that the lack of feedback they received created difficulties when preparing for assessed work. Comments including “The most appalling thing is ‘how do I learn?’ I can’t figure out how I’m meant to do better next time” represented a sentiment expressed by many participants. This was related to a sense that feedback should help to clarify assessment criteria. Students often expressed views that “what they’re looking for is so subjective” that detailed feedback would be a crucial way to help students understand what was expected of them. Students typically expressed that marks on submitted work would be useful in assisting them to understand the assessment of their work.

•

Students discussing the relationship between course fees, burdens on academics and feedback. Students across a range of courses explicitly discussed the intersection between these issues, often in terms that were sympathetic to academics. Comments included “I know that the program is really strained because advisors are essentially doing two jobs” and “People do graduate supervision because they’re passionate about it rather than because they’re being paid but where are my course fees going because if staff aren’t being paid to teach me then what is happening?”. Nevertheless, students repeatedly made

the point they felt the fees they were paying for their courses justified more feedback both in terms of quantity and detail. This mirrored themes present in the recent Review of the Humanities Division that discussed burdens on academic staff teaching over stint and the possibility that higher fees could be charged for PGT programs in the Humanities. While there is a clear trend across the University towards upwards fee harmonization this comes with increased student expectations which are felt ‘on the ground’ in terms of more expectations of academic staff.

•

When students were pressed on what feedback they would ideally like a comparatively clear picture emerged. Students wanted to receive marks on work prior to having to submit work for assessment as part of their final mark.

They felt that “more, informally assessed work would be useful”. In common with other participants in other Divisions, students were keen to get detailed comments on specific points on essays which could then be discussed orally with academics. Students in several focus groups raised interesting points about the possibility that “Our supervisors are responding to our draft essays as if they’d been submitted as an undergraduate weekly essay”. Participants in these focus groups felt that the increased levels of contact on undergraduate programs as opposed to PGT programs meant that such feedback on undergraduate courses would be more sustained and thus valuable, whereas similar levels of feedback were not sufficient on PGT courses. Possible lessons from these points include the fact that students had a clear view of differentiation between undergraduate and PGT programs. Students were keen to avoid feedback being provided in similar ways to that which existed at undergraduate level, believing that more advanced feedback should instead be offered. As was the case in other Divisions students also explicitly made the point that formative feedback was of limited utility if deadlines were concentrated in Trinity Term as it did not enable this feedback to be utilized in improving work handed in for assessment over the course of the academic year.

Skills & skills training Skills that people felt they were developing as a result of their course included skills relating to writing, independent learning and self-reliance. The crucial points on each these that emerged were: •

People felt that the writing skills they were developing related to being able to write rapidly and concisely. This was a similar skill-set to that which people doing undergraduate programs in the Humanities felt they were developing with sentiments on this point including “We’re definitely able to work through the night on things” and “I do write a lot quicker now which prepares you for the research type of lifestyle”. Development from undergraduate study (particularly for those students who hadn’t previously studied at Oxford) was noticeable and was expressed through comments such as “Doing a 10,000 word essay used to seem daunting, now writing a dissertation in 4 ! weeks doesn’t seem like an insurmountable obstacle”.

•

Participants made frequent comments about the self-reliance which they had to show while studying for their course. This related to points previously raised about feedback and included queries as to whether they were on a PGT

program or were – effectively – on a PGR program which was described as a “Taught Masters”. Comments on this point included “This isn’t a Taught Masters, it’s what I’d expect from doing a Phd; at this level you’re more unsure of things” and “Often our seminars more resemble guided discussion which is fine but I don’t feel like we’re necessarily being taught anything”. In several courses students reported working with PGR students who were effectively organizing their own seminars designed for PGT students with the perceived benefit of teaching experience for PGR students and more structured teaching for PGT students.

•

More general points relating to language provision emerged across a range of Humanities subjects. Although students disagreed about the level and quality of language provision many students expressed frustration about the difficulty of being able to learn languages other than English while studying at Oxford. This was particularly emphasized by PGT students from the USA who indicated that “You can’t avoid needing to know other languages in these subjects – in the US it’s just assumed that this is a part of the course but here it seems like you really need to try and find where you can get this training”. This point relates to issues around funding for the Language Centre which have been kept under active review by the Committee for the Language Centre in recent years. Students also emphasized that language provision was often necessary to support the research interests that had been mentioned on application forms. It was felt that these were therefore part of the rationale for students being admitted to different courses and should be supported through increased provision.

When questioned about skills training on PGT courses, students often felt that training could be more “interactive” with provision being less focused on lectures and more towards practical workshops. This mirrors the desire for interactive forms of teaching found both at undergraduate level and on PGT programs in different Divisions. Students on English PGT programs made comments such as “It would be useful to have more physical examples of what people are talking about a lot of the time – it’s hard to relate concepts to the course” with students on other programs echoing the sentiment for practical instances of applying research methods to the work they would be doing.

Students on several programs specifically drew a distinction between “guided discussions and teaching” within the research methods sections of courses (as they did on teaching more broadly), expressing a preference to see more interactive forms of teaching being used. Students typically said that training provided on making use of libraries and academic services within Oxford were valuable with North American students making particular reference to what was seen as a more complex system of library provision in Oxford relative to leading universities in the United States.

Assessment Students typically discussed the relationship between feedback and assessment, the balance of assessment between written exams and dissertations/coursework and clarity of assessment criteria when asked about the relationship between methods of assessment and the structure of their course. Key points were: •

The value of feedback in preparing students for assessment. Students understood the burden on academic time of providing feedback (as indicated above) but nevertheless felt that “They don’t give you guidance through the

course as to what essays should look like”. Participants regularly made comments about how most of them believed they would actually pass their PGT courses as a result of low rates of failure in programs and as “You just assume that they would not let you leave their office knowing that you were going to fail”. Focus groups often had to be steered away from the topic of feedback after this was initially discussed in response to this question.

Comments as “We don’t get definite marks so it’s hard to judge your progress” reiterated how central improved feedback was seen to improved preparation for assessment.

•

The balance of assessment between dissertations/coursework. As with other programs, students typically stated preferences for diversified methods of assessment although this was less pronounced than was the case in other Divisions. Surprising numbers of students indicated uncertainty over how they were assessed, making comments such as “I was positively surprised to find out that I didn’t have exams” and “Right now we don’t know how we’re assessed”.

When pressed it typically emerged that students were unsure of the precise weightings given to different elements of assessed work, often until after some work had already been submitted for assessment. This was seen as being unfair owing to students being unsure what bits of work to prioritise over the course of the year. Students typically preferred dissertations to be weighted more heavily than they were and reiterated concerns found elsewhere about workload being unevenly spread over the course of the year, with deadlines being concentrated in Trinity Term.

•

Clarity of marking criteria was – again – related to the issue of feedback on students’ work. People acknowledged that “It is very difficult to put in quantitative terms what a Distinction is” and thought that examiners frequently decided to award a Distinction/Good Pass/Pass and then determined which mark to award. Students typically saw the less clear marking criteria at Oxford relative to other universities as “Part of the Oxford way – it’s to do finals without really knowing what you’re doing”.

Teaching provision and expectations Many of the points made about feedback and essay-writing which students had already made were reiterated in this section of discussions and so are excluded here. Other key points regarding teaching provision included: •

Extended discussions about whether their programs were ‘Taught Masters’ courses in the sense that people imagined they were at the point of applying.

Some students saw this as a surprise relative to what they had expected but agreed with sentiments such as “I’m pretty comfortable with a hands-off approach but they just shouldn’t call it a taught degree” and “It’s not teaching, it’s correcting your own research”. Other students believed explicitly that they had applied to their courses in error, feeling that “I’d picked the Taught program to expand my knowledge of what I’d been taught at undergrad and that seems to have been a huge mistake”. There was a noticeable split in this area between students who had done an undergraduate degree at Oxford and those who had not. Those students who had studied at Oxford as undergraduates typically felt

that the teaching was similar to what they expected whereas this was not felt to be the case by students who had done first degrees elsewhere.

•

It was surprising how few points were made in response to this question on individual areas of teaching such as tutorials, lectures and seminars. Where people discussed elements of departmental provision they often spoke about individual tutors, highlighting what was felt to be large variation in the quality of supervisors. Students made comments such as “The range of experiences has been very diverse but I’ve been really lucky because [Name] just happened to be my supervisor but it’s really unfair that people have such a range of responses to their own work” and “It really does depend on who you’re taught by and sometimes you get really good provision and sometimes you just don’t”.

•

When discussing the Departments of which participants were members people often drew links between their PGT programs and research going on in the Department. Students indicated that “being here has taught me about being part of the community of academics” and expressed that they were impressed at the academic environment which existed in their Department and within the wider University. As in other Divisions students felt that they would benefit from having a broader relationship with students on PGR programs. A general point which we observed is that students in Social Science programs based in Manor Road, MPLS programs on South Parks and Medical Sciences programs often made reference to the value of meeting other students on similar courses to their own and the broad academic life within their disciplines. Bringing students from different courses within Divisions together was seen as beneficial by a wide range of participants, reiterating points we found last year regarding the future academic benefits of the development of the ROQ site.

Role of the College As with students in other Divisions the main function which PGT students in Humanities saw their colleges as providing was social. Although students were hugely varied in terms of the level of interaction which they had with their colleges (as expressed through comments such as “college is really hit and miss as a graduate”) the three main points which emerged from participants were: •

A recognition of the valuable role of colleges in providing financial assistance to graduates on course-related issues. This was shown through comments such as “It seems a bit hit and miss with funding for academic research trips” and “the discretionary budget which my college had was really useful in terms of assisting me with language provision”. Where students didn’t have access to financial assistance from their respective colleges they often contrasted this point with other colleges that made such provision.

•

Related to this issue was that of college choice at a graduate level. Students recognized that the choice of college could impact their experience as a graduate as shown by comments such as “I would have had a very different idea of Oxford if I’d gone to a different college” and “Compared to the guidance undergraduates get about applying to colleges – there’s not much – it would be really helpful if there was some sort of central list about which social facilities different colleges provide”.

•

A reiteration of the valuable social role which colleges played. While a minority of respondents said that their colleges had played a role in their academic study through programs such as college-based graduate seminars, most students referred more to the social element of college life through comments such as “My college is awesome because the MCR is very social” and “My college is entirely social – I think that’s OK as I didn’t expect or want anything beyond that”.

Sources of support for students The sources of support that PGT students in the Humanities felt that they would turn to were more varied than was the case for students in the other Divisions. While departmental tutors were often seen as a first port of call for student support, colleges were seen by many Humanities students as being a source of support that they might call upon at an early stage. Students felt that the “ad hoc system of provision” typically meant that either their departments or their colleges could be used to provide support with students noting that “you do get swamped with support networks – that’s a special part of the Oxford system”.

In spite of this many students referred to feelings of unease in making use of the support services available, reporting feelings that “in Oxford that doesn’t really feel like a road that is open to you, you do have to feel that it’s open to you before you do it”.

In common with our findings from our undergraduate consultation people spoke about the “fear factor” which made people reluctant to seek support. It was striking that all those people who said that they had made use of departmental or college welfare provision reported satisfaction with such provision through comments such as “if you ask for help you do receive it”. An overall sense was given that “The thing with welfare provision is that you can put all the structures in place and they are in place quite well but you need to find someone that you can identify with”. This mirrors many of the issues we have seen raised through the Student Union’s Advice Service, particularly the frequent problem that students (especially graduates) begin using student support services later than would ideally be the case in many instances.

Preparation for career post-Oxford Students felt that their programs were preparing them for DPhil study above all other options with several sub-themes emerging to this point. These included: •

Feeling that PGT programs in the Humanities might prepare students for DPhil study in the UK but not in the US. This perception existed across a range of subjects and was expressed through comments such as “I definitely feel that if I went into a Doctoral program in the US, I’d probably have to step it up a bit” and “If you’re doing a taught Masters at Oxford then you can’t go back to the US for a DPhil. They’re completely incompatible – the emphasis in the US is on teaching whereas in the UK it feels like research is what’s valued”. A secondary reason given for this differentiation between UK and US DPhil programs emerged in the context of language training with students outside of the Faculty of Medieval & Modern Languages indicating that the lack of language training available to them (for example from the Language Centre) would inhibit future DPhil applications owing to entry requirements of US universities.

•

The flip side of this point is that people generally did feel that their Taught Masters programs were preparing them for DPhil study in the UK. Comments on this point included “In terms of a DPhil I think it prepares you quite well, really in terms of you working by yourself” and “It has prepared me but it should have been called a Research Masters rather than a Taught Masters”.

•

The theme that courses were preparing students for DPhil programs as a result of students having to be self-directed learners was repeated throughout this question and was frequently not seen as a positive. Comments on this theme included “The world and other institutions have a perception that I’m getting really rigorous training here so I’m going to be OK – what I’m getting here is that I’m going to have to do this by myself” and “I will walk away wiser as a result of the reading that I did but not as a result as what I’ve been taught, as it were”.

Which one change would have improved your time as a postgraduate at Oxford?

The single most frequent point raised by students on Humanities PGT programs on this issue related to feedback. Other themes involved the difference between ‘taught’ and ‘research’ graduate programs and course organizations. Expanding on these points the key issues were seen to be: •

Improvements in the quality and quantity of feedback. Students reiterated that they valued feedback as “I need to know how I’m doing” and “You put a lot of work into things and it feels like it’s their last concern to give you feedback”.

Students felt it would be particularly helpful to have feedback provided early on the course with the example being given of “in the first few weeks a really short essay, around 2,000 words, you get detailed feedback on it so you know what they’re expecting in terms of bibliography, style so you have something to work on”.

•

Improvements in course organization and structure. Examples given by students included “They could communicate better about what the year’s looking like – the structure of how everything works has been unclear to me” and “The structure of deadlines leaves work really concentrated at certain points in the year and it’s not really clear that is”. The attitudes of students in areas such as MPLS and sections of Medical Sciences, who felt that course organization and structure were clearly explained to them at the start of their program and were subsequently adhered to possible provide examples of alternative ways of timetabling and organizing courses. The work already done within the Humanities Teaching Review project on consistent timetabling across undergraduate programs could also usefully be considered at PGT level if administratively feasible.

•

Throughout many focus groups for PGT Humanities subjects students expanded on a trend found elsewhere in certain Social Sciences programs; the distinction between “self-directed learning” and teaching. Many Humanities students felt that their courses had been “less taught than self-researched” with this point also being seen as related to the issue of a perceived lack of feedback that was mentioned by many students.

Annexe B: Social Sciences – Professional Courses 3 Expectations of and induction to Oxford Students on these courses frequently made points about expecting to enter an intensely academic environment. The balance between the academic and practical sides of these courses was felt to be unclear by many participants with students in the PGCE focus group making comments such as “I was slightly uncertain [about what to expect] in that it’s a professional course rather than a strictly academic course” and “I wasn’t too sure about the split between the academic and the professional and to what extent they’re complementary”. This distinction was also mirrored by students enrolled on the MLF who drew contrasts between the law and financial elements of their course throughout discussions. They agreed that “our course is more professional than academic in certain aspects…law is practice-based, economics is academic-based”.

MBA students drew out a contrast between the academic environment and the corporate environment to which many of them were accustomed, through comments such as “In a business environment we’re somewhere where feedback is consistently driven over a period of years and coming into an academic environment where there’s zero feedback…it’s expected by the institution but it’s not set out ahead of time”.

Although they felt prepared by “prior warning about the level of academic rigour” students enrolled on the MBA regularly expressed surprise – both in response to this question and throughout the focus group – about perceived tensions between the University and the Said Business School. This manifested itself through statements about the levels of salary paid to academics at the SBS; students felt that the University was “charging maximum level tuition but not paying Professors” and felt that they were being disadvantaged relative to their peers at US institutions who were paying similar fees but receiving higher standards of tuition. Students explicitly tied their expectations into levels of fee being paid (“if you’re going to spend 30k then you want to know how it’s spent”) and the large amount of comparative data available about MBA programs (“Business school programs are ranked internationally through a fairly transparent methodology – there are metrics available to business school students which aren’t available to others”). While students on these programs were particularly likely to refer to course fees in discussions about expectations students in other Divisions also repeatedly made similar points, meaning that such points should not be seen as exclusively held by students on professional courses.

Students on these programs were generally content with the induction they received relative to students on other courses. This may provide some lessons for how other courses could introduce PGT programs to students who are new to life at Oxford (of 30 students who attended these 3 focus groups only 1 had previously studied at Oxford with a clear majority of attendees having done their prior studies outside of the UK). The main points that arose from discussions on induction were: •

The value of having previous students on such programs playing a role in induction. MLF students acknowledged this would not have been possible for them owing to 2010/11 being the first intake of students to the program. MBA

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

3

‘Professional courses’ in this context refers to focus groups organized for students doing the MBA, PGCE, Masters in Law and Finance and feedback received online or in writing from students on these courses.

students reported that an initiative organized by students where “an ex-MBA student comes in and helps shape student expectations” was particularly valuable. This mirrors students wanting closer ties with alumni more generally and more interaction with former students on PGT programs specifically which arose in other focus groups.

•

The large quantity of information and restricted timeframe available for induction. This was heavily related to course administration with MLF students agreeing that “the main problem is that it’s [key information about the course] is simply not set out clearly enough”; although this was seen as likely being a function of the new nature of the program. Students nevertheless felt that a “full-term schedule” of the sort found in American universities would be useful.

Comments were also made about the different roles of colleges and departments in providing induction with students feeling that “Oxford is more complicated than other places and that warrants more effort”.

•

Students felt that the diverse nature of the intakes on PGT programs meant that more effort could be made to ensure people had opportunities for socializing early on the course with PGCE students in particular feeling that their early start-date meant that “you spent the first two days kicking around Oxford” and were unable to contact people. It was felt that circulating contact details for students (including some recent alumni) prior to their arrival at Oxford could help introduce people to Oxford and ensure that students were able to make contact with other new arrivals before they began their course.

Feedback As was the case with students in other Divisions, participants were dissatisfied with the feedback they received, although this varied by course level.

•

PGCE students felt that the quality of feedback they received on their teaching was “dependent on the school” but were generally satisfied with feedback they received on their “professional work”. There was agreement that reports received from ex-teachers mentoring them were valuable but that their academic assignments could usefully be returned with more feedback. Unlike other courses respondents were not particularly angry with this, feeling instead that their professional training was more valuable than the academic elements of the course.

•

The unique nature of the PGCE in terms of the split between academic & professional training led to a clear contrast between the attitudes of PGCE students towards feedback and those held by MBA and MLF students. MLF students were very satisfied with feedback provided by the law faculty, feeling that “finance is the problem as opposed to law”. They felt that “law tutors are willing to look at anything that you want”, noting that they particularly wanted feedback on the finance-related aspects of their course owing to the fact that they typically had legal backgrounds and wanted feedback “as its necessary for our next essays”.

•

Compared to students on the PGCE and MLF, MBA students were most dissatisfied with feedback received on their work with the immediate response

to questioning on it being “there is none”. Variability between tutors was seen to be a problem with people making comments such as “one tutor writes 2 pages on your assignment, others give you a ‘well done’”. Points were again raised about the link between course fees and expectations through comments such as “People are paying 35k to be here” and “To put comments in a ten page document takes ten minutes. We pay more than enough for that to be a requirement”. A recurrent theme throughout oral and written comments from MBA students was how their experiences related to expectations of Oxford with comments including “There needs to be a culture of feedback – we’re professionals and we’re either used to managing teams and giving feedback or getting it back”. People felt that the lack of feedback was demotivating, saying that “you shouldn’t under-estimate the importance of this – I know I’m not going to get feedback so it distorts the whole paper. Now I just do it quite halfheartedly…it distorts the whole learning process”. Students again raised their belief that attitudes to feedback within the SBS were those of the University rather than the Business School through comments such as “If the Business School had flexibility to make its own decisions then it’s something that they would have done years ago” and “If there’s one takeaway it’s that the University needs to respect the need for more independence of the SBS”.

Emergent trends from these points include the relationship between academic and professional training which exist on these courses – as discussed earlier students are expecting an academic environment at Oxford but nevertheless are often returning to University with different expectations to undergraduates and even other PGT students.

The theme of separation between ‘Law’ and ‘Finance’ which was recurrent throughout the MLF Focus Group reiterates many of the concerns that emerged in our discussions with undergraduates in 2010 about administration of joint courses. The attitudes expressed for students doing the MBA were very similar to those doing other Social Sciences courses and particularly reflect the professional experiences of students enrolled on the MBA as well as the fees that they pay. As is also stated with regard to the Humanities the recent trend towards increased fee levels in the Social Sciences is being felt “on the ground” with students making clear their expectations of academics and the fact that these expectations are explicitly linked to the fees that they are paying.

Skills & skills training The themes that emerged in discussion about skill development were varied but all ultimately related to the aims of the courses which students were taking. PGCE students noted that “academic skills were rushed” and felt that provision of examples of “good dissertations” would be useful so they could get a sense as to what work should look like. A desire for sessions on “how to improve your academic writing” stemmed from similar aspirations although the overall wish was for a re-evaluation of the balance between academic and professional work, particularly in the context of the intense workload that existed for students doing a PGCE. MLF students were particularly specific about what skills they were developing with a focus being on “economic understanding”. This was felt to be particularly useful since “as lawyers you do these transactions and you don’t get the whole picture”. Legal training was felt to be less challenging with a wish being expressed for a separation of lectures and seminars since students felt that the combination of these two different forms of teaching “were trying to fuse two different things”. MBA students expressed a belief that sections of their course were “very high-level, not at a practical level” which could “partially be blamed

on 8 weeks of instruction per class”. People felt that their course was possibly excessively academic with a consensus existing that more diverse forms of assessment related to skills felt to be necessary in the business world (e.g presentation skills) would be valuable. The general sentiment was similar to that found in the PGCE course; that “academic rigour and professional judgment aren’t perfectly aligned”.

Students across all three courses expressed a desire for more of a focus on professional training with PGCE students finding training in the Department “not that practical” and variable between different subject avenues to the PGCE. A belief was expressed that different subjects could learn from each other on the PGCE with there being more room than currently existed for good practice to be shared. MLF students felt that the heavy timetable for their course “squeezed out” possible skills training. It was felt that more assistance with mathematical elements of the course including use of Excel and accounting had been useful where they existed but that there would be scope to expand such training. Students on the MBA mentioned plagiarism without prompting (the only focus group where this was the case), saying that the training in avoiding plagiarism had been useful and that professional training in areas like financial modeling and marketing would be particularly useful. People agreed that “whatever they want to teach us, they’re doing a good job of it. But I’m not sure that they’re teaching us what we need to learn” Overall the focus on skills training and development for students on these courses was unsurprisingly very much towards skills necessary for their intended future careers and less on the skills training which exists on other PGT courses.

Assessment A general theme on this topic was similar to findings from other courses and our consultations with undergraduates; that people had a preference for diversified methods of assessment. PGCE students were generally satisfied with the means of assessment within the ‘professional’ section of their course, noting the variety of assessment practices existing for PGCE programs at other universities. They, in common with students on the MBA, raised the issue both of overall workload and of the timing of assessment. PGCE students expressed concern over having to carry out research for both their dissertation and assignments with people feeling it necessary to prioritise their dissertations and school workloads above their other assignments. Shifting assignments into terms where less assessment was taking place could address this problem. A similar point was made by MBA students who raised questions as to why the burden of handing in assessed work fell disproportionately in between weeks 5 and 9 of Trinity Term. The view that “there’s no magic reason why everything needs to be in on Monday of week 9” was raised with people feeling that means of assessment including presentations, case write-ups and class participation would both more accurately reflect the requirements of their intended future careers and promote more consistent work throughout the year. The point about the relationship between the University and SBS was again raised with regards to assessment with people feeling that the University set requirements for assessment with frustration being expressed that “we cannot even do a presentation as you can’t be judged on that basis”.

Students on the MBA and MLF both raised the question as to why they weren’t permitted to use computers during 3 hour examinations owing to the widespread use of computers post-University with it being felt that other universities made such provision. Students on both programs also explicitly raised the relationship between feedback and assessment, feeling that feedback on work assessed earlier in the

academic year would be academically valuable and enable students to improve as the year progressed.

Teaching provision and expectations Students on these courses made repeated references to employability when asked about whether “there is anything you feel that current academic provision isn’t providing”. Students on the PGCE felt that “it’s been covering the basics very well” and that the training they had been receiving would be helpful with job applications.

Students on the MLF specifically expressed a view that training on negotiating and debating would have been useful on the grounds that it would have related very clearly to the sorts of activities that graduates from the program would be doing on completion of study. They also made clear at this point that they were satisfied with the course in large part because of the level of ambition it required and the innovative nature of the program. MBA students reiterated the points that they made regarding means of assessment and skill development through teaching. They particularly expressed views on the variability of teaching that they had received, saying that ‘interactive’ teaching would be particularly valuable. The following points were particularly expressed on this: •

Expectations of Oxford – comments such as “The bar is set much higher for Oxford than for other universities” and “From an expectations perspective it’s one of the top 3 universities in the world. One of the areas where I’ve been surprised is the lack of quality of teaching” were representative of students taking the MBA and reflected their high expectations not only of Oxford but of the teaching they would receive. It was striking to us that students who saw themselves as ‘informed consumers’ and had metrics available to assist them in deciding which programs to apply to that are not available to other students nevertheless referred explicitly to the reputation of Oxford University (as distinct from the Said Business School) in explaining where their expectations of their time in Oxford originated from. This reflects views on the reputation of Oxford as a reason for applications which are also found through the Student Barometer Survey and in our consultation with undergraduates last year.

•

Engagement of students. As we found when consulting undergraduates last year students had a particular sense that interactive teaching was valuable to them.

We pressed students taking the MBA on this point owing to the strength of views they were expressing and it was clarified that what they were looking for included “the difference between facilitation and pontification”, “a front of the classroom experience which is the difference between teaching and relying on slides” and having “teachers who create the experience – the entire environment needs to revolve around that”. This theme was returned to in other focus groups but students taking the MBA made it particularly clear that they were looking for direct forms of teaching.

•

Expectations arising from the choice that students make to come to Oxford.

This has already been touched on but comments such as “This is a marketdriven business at the end of the day” and “We all choose to come here, we had choice and people really need to be mindful of that” emphasized the origin of some of the expectations that students had of their course.

A more general point that arose across the three programs related to course organization. It was felt that one-year PGT programs were inevitably heavily timepressured and that having more carefully timetabled teaching would prevent pressures arising unnecessarily. This theme emerged elsewhere in focus groups for nonprofessional programs within the Social Sciences Division and additionally in a range of other focus groups across the University.

Role of the College Most participants acknowledged that the relationship between students and their colleges was very personal and would not be identical between students. This notwithstanding, the overall relevance of colleges was seen by respondents to be primarily social, with this being seen as a positive function. Particular points that emerged were: •

PGCE students raised the possibility of further concentrating students on their course into a limited number of colleges. People expressed frustration at the fact that many of their colleges “shut down during holidays because all the undergrads are gone” and fixed charges for services they could rarely, if ever, use because of requirements to be at school. Students also felt that colleges providing more consistent financial assistance would be useful as the combination of funding cuts for PGCE students and the inflation of travel costs to and from school would compound difficulties in the cost of living.

•

Students on both the MLF and MBA mentioned the role that colleges have in promoting social interaction between students on different courses. Comments such as “It’s good to meet people outside of law – people spend a lot of time using the group social facilities”, “we spend so much time together that it’s nice to have a different group of people” and “for us college is purely a place to socialize if we choose to” were representative of these attitudes.

•

Students on the MBA made the point that a “business school graduation ceremony” would be preferable to having all graduations organized by college (a point made most recently by OUSU with regard to PGT students in the consultation over future arrangements for degree ceremonies in 2009/10) and also highlighted a belief that there were tensions between colleges and the SBS in fundraising. They made clear that “most of us consider the Business School where we’re linked with…for us it is our college” and discussed the college/University relationship in development activity.

Sources of support for students In contrast to many other focus groups which saw students mention their supervisor as a first point of contact, students on the MBA and MLF indicated that the administrative staff within their Departments would be their first point of contact if they ran into difficulties while studying at Oxford. PGCE students felt that “curriculum tutors” would be the first port of call for any support, mirroring the view of students on the MBA and MLF that their Department would be the first point of contact.

Preparation for career post-Oxford PGCE students were unanimously intending to go into teaching and there was a clear consensus that their course was preparing them “very well” for a career in teaching.

They also felt “very well” supported by the careers support available to them.

Students on the MLF were generally very satisfied with the course but indicated that there was a problem in communicating with employers about the program, saying “not all the employers know about the courses” although a relationship established with a City law firm which had established a mentoring scheme for current students was felt to be very useful. Students on both the MLF and MBA were very positive about the benefits of the Careers Service at the Said Business School although the single largest source of frustration for students on the MLF was the relationship with the Said Business School with students arguing that “the behaviour of the Business School has been atrocious” in terms of inclarity over the rights of students on the MLF to use the Business School with the same rights of students on the MBA. This feeling had extended to students on the MLF being unable to use the SBS Intranet and “administrative obstacles” to participating in the activities of the Business School. It was felt that greater integration between the Law Faculty and SBS in this area would lead to a better experience for students, particularly with regard to MLF students developing skills in finance & economics.

MBA students emphasized that the diversity of backgrounds of people on the course meant that it was difficult for all students to be taught in the same way and that to understand the benefits of studying at Oxford “you need to split the experience from the academic component”. They felt that the academic element of the MBA was only being one part of studying at Oxford. They also praised the SBS Careers Service, noting that “without it you’d slip down the ranking lists” although they argued that increased resource for the SBS Careers Service could enable increased contact with students before they start their course at Oxford and therefore a more personalized service.

Which one change would have improved your time as a postgraduate at Oxford?

The points that were repeatedly raised by participants to this question were: •

Course structure – PGCE students felt that the balance between academic and practical work, particularly given their travel obligations to and from school, meant that they were overstretched throughout the year and needed “a proper break at some point in the year”. It was felt that the PGCE prepared students well for life as a teacher but that the academic element of the course, and particularly the timing of assessment deadlines, meant that the work burden was excessive at points. MLF students also wished to have a “clearer course structure” with this concern being raised by MBA students also.

•

For MBA students the main changes related to revised course structure (taking into account more practical skill-based means of assessment), the possibility of a longer course (15 months was mooted as something which could be considered), use of the post-admission but pre-arrival period to introduce incoming MBA students to Oxford and addressing what were felt to be structural issues in the University/SBS relationship to enable higher salaries to be paid to attract the best potential Professors to the School.

Annexe C: Non-Professional Courses 4 Expectations of and induction to Oxford Students typically reported that they were expecting the following main features of an Oxford academic environment prior to arrival: •

A rigorous course which would be more challenging than undergraduate provision (this expectation existed regardless of whether participants had done an undergraduate degree either at Oxford, elsewhere in the UK or outside of Britain). People’s expectations of “academic rigour” were directly related to individual contact with academics with comments on this point including “I expected to get more from my teachers with not so much independent study”, “I was expecting a really rigorous engagement with Professors” and “this program is meant to be the best of its type in the world – I thought I’d be really closely co-operating with academics”.

•

A highly structured program that would build on their experiences as undergraduates. Typically students were expecting “a program very similar to how Oxford is always promoted – very close one on one interaction and clearly defined program goals”. They expressed beliefs that “I was expecting a more intense continuation of my undergrad but it felt a lot less intense and a lot more independent than I was expecting”. There was a common feeling among both those who had studied as an undergraduate at Oxford and those who had only arrived for a Masters program that the courses would be more “intense” than they ultimately turned out to be.

Students reported mixed views about the utility of the inductions that they received.

Generally speaking the following points were agreed upon by respondents: •

Departmentally provided course handbooks were useful in providing basic information about the course. People felt that “the course handbook was pretty useful as far as it went but there are lots of informal norms that go beyond it that no-one ever tells you”. These “norms” were made more confusing by what people felt was a “proliferation of sources of information”, some of which were seen to contradict each other. Students made references to the extent of printed information that they were given from both their department and college, indicating that the weight of information they were given in 0th week of Michaelmas could be excessive. Many students felt that better use of the summer vacation could be made to provide information about the course prior to arrival in Oxford.

•

Students felt that course handbooks often did not make clear either the structure of assessment or the criteria on which assessment was based. There was a sense that “we’re all coming from different backgrounds and British people are more likely to be familiar with the system but many people are not”.

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

4

In this context ‘non-professional courses’ refers to focus groups organized in programs in the Social Sciences Division laid out in the ‘methodology’ section except those discussed in Annexe B. In addition to focus groups conducted we received large quantities of written feedback which is also used in this section.

Other comments making this point included “My coursemates and I have had a lot of trouble trying to figure out the marking system for the course – which marks count, which don’t, how and when we are marked”. This confusion mirrors similar sentiments expressed by PGT students in the Humanities Division about structure of assessment and by undergraduates (also found in the NSS) about clarity of assessment criteria.

•

The supervisor relationship was often felt to be something which induction could focus on more heavily. Students expressed surprise that the supervisor often did not initiate interactions with new students saying “nobody tells you…you have to email the supervisor to set up a meeting. You’re given the name of the supervisor and that’s about it”. Other students indicated that “we could have used more guidance about what to expect in terms of dealings with supervisors” and “the expectations of what we could get from supervisors were never really made clear at all”.

•

Certain courses were seen as having unclear course objectives between professional training and progression to PGR study. This was particularly expressed by students through comments such as “There’s a confusion as to what the program is all about – is it designed for a DPhil, professional training or as an end in itself. It’s really not clear in what way the course is meant to help you shape your goals”. Comparisons were often drawn with comparable courses, particularly in the United States, with some students expressing surprise at how different they felt PGT courses at Oxford were to those at leading US institutions. Students on courses within the DPIR, Department of Social Policy & Intervention and Department of International Development raised questions about how their courses could interact with the Blavatnik School of Government, querying whether its establishment in providing “professional” courses might impact existing PGT provision within the Social Sciences Division.

Feedback As was the case on many other courses, students frequently expressed concern about the level of feedback they were receiving, believing that this was inhibiting their academic progress. Points that were widely agreed upon included: •

Variability between feedback received depending on which members of academic staff were supervising individual students. This was seen as being unfair with students saying “the amount of feedback I get varies hugely depends on who is marking my work” and “it’s so variable – some people get two Professors to read over their thesis multiple times, others have their supervisor talk about footnotes a little”. Students generally favoured more rigid statements as to what the obligations of supervisors were with less discretion being left to the individual supervisor.

•

The relationship between feedback on written work and assessed examinations.

Students felt that a lack of feedback was damaging their ability to prepare for exams with sentiments such as “None of my stuff has been given a mark so having never studied at Oxford I have no idea what kind of mark I would be given” being frequently expressed. Students were keen to receive marks on their work, feeling that “They make clear that grades don’t matter but it would

be nice to know how you’re doing…good work doesn’t tell me if it’s a 2:1, a good 2:1 or a 1st?”. Students indicated that they were keen to prepare for exams more fully with those who had sat mock exams saying they were valuable and other students making comments such as “I was surprised that no Masters collections were set even for the fixed options” and “I’ve only done one practice exam essay…but I think that was very useful and it would be encouraging if there were more”.

•

What sort of feedback students would like. When pressed on what sort of feedback would be useful, students typically indicated that a combination of written comments on essays that could also be discussed with the supervisor would be valuable. Students were generally against the idea of a standard form on which feedback could be written believing that the return of annotated written work that could then be used to improve future work would be particularly beneficial. During this section of discussions students often raised the point of academic workload, expressing various views about the viability of expanding the quantity of feedback provided. It was, however, generally felt that extending the amount of feedback was possible and would be desirable. This was similar to the attitudes expressed by students in other Divisions, particularly on PGT programs in the Humanities.

Skills & skills training The skills people felt they were developing on their courses were often very similar to those reported by undergraduates in the Humanities and Social Sciences. They often included: •

Essay-writing skills. People reported that they were “learning the ‘Oxford style’ of writing essays”, more broadly defined as “how to write essays really fast” and “having to churn out words on a regular basis”. This was heavily tied into time management with people saying “I’ve learned how to work incredibly independently” and “The main skill I’m cultivating is self-discipline; I’ve got no deadlines and lots of my work isn’t even assessed so learning that you just have to go to the library is a difficult skill which I’m cultivating”.

•

Analytical skills. Students indicated that “We’ve developed our analytical thinking a lot – in most of our undergraduate degrees we didn’t have many debates or discussions a lot of the time.” People also agreed that “Coming from my undergraduate program the course here is a lot less directed and you have to learn to do a lot of stuff off your own back”.

•

Students often differentiated between skills developed as a result of the teaching they were receiving and skills honed as a result of being in Oxford. People indicated that “One of Oxford’s best aspects is the ability to show up to a lecture, a course, anything like that and really step outside of your discipline”.

Students who felt that they were not having these opportunities often felt that they were missing out as a result. Students on one PGT program were unanimous that “the fragmentation in this place is extraordinary; you’ve got all these departments offering similar things which would just be ‘International Affairs’ in the US. There’s so much that we want to do and there’s no way of finding out where any of it is”.

Students often reported feelings of disappointment with the research methods training that was available. Specific points on which many students – across a range of departments – agreed included: •

The quality of provision in quantitative research methods. Comments included “There’s a real issue with the lack of quantitative teaching – we look at a lot of texts and theory but I would say we were ever really taught how to do statistical analysis”. People also indicated that “We feel that the way it’s taught is quite half-hearted; there’s a sense that if they’re not going to take teaching it seriously then what’s the point in us taking it seriously either”. People felt that a lack of background which many students had in statistics meant that there was a case for “some sort of bootcamp where all you’re doing is stats for a couple of weeks. At the moment everyone just acts like you can learn this doing it for 2 hours a week for 8 weeks and clearly you can’t”.

•

By contrast students generally indicated a higher level of satisfaction with qualitative research methods training. Few concerns were expressed and those students who did mention qualitative research methods indicated high levels of satisfaction through comments such as “You assume that it’s going to be quite basic but you do actually realize once you start just how useful the skills you’re being taught are” and “Stuff on elite interviewing and how to analyse qualitative data has been really valuable”.

•

The organization of research methods training was seen as being improvable through better administration. While many people felt that optional training sessions were valuable (few students who had attended such sessions expressed dissatisfaction) they indicated that the number and timing of emails they received from their Departments meant that opportunities were being missed out on. Comments on this point included” I hadn’t been to University for three years prior to coming here so it would have been great to have this information signposted for us” and “It doesn’t seem like there’s any clear timetable for this stuff – there’s no reason to get notices on a weekly basis given that all leading American universities manage to give you a timetable every semester, in advance, which tells you where you need to be and when”. Comments relating to course administration were recurrent across focus groups in a range of Divisions, often being framed in contrast to other universities.

Assessment In common with other programs at both undergraduate and PGT level students were in favour of diversified methods of assessment. Feedback was seen as being heavily related to assessment with many students repeating points that had previously been made about feedback in response to being questioned on the relationship between assessment and course structure. Specific points made on assessment included: •

The close relationship between better feedback and methods of assessment.

Students typically agreed with the view that they didn’t know “How you can improve if you don’t now how you did because you’ll probably just make the same mistakes from assignment to assignment if you don’t know what they like or dislike”. It was felt that feedback could be given on assignments handed in for

formal assessment early in the year with this possibly being done in writing to protect the separation between examiners and students within the assessment process. Other comments on this point included “You get feedback informally in certain ways but for a lot of students they’re left in the dark if their supervisors aren’t giving them feedback”. This relates to the point made earlier about variability of feedback provided by supervisors. Similar comments on this point include “As we don’t receive any grades until anything is over I’ve got no real idea of how I’m doing and won’t until I’ve handed in all my work.”

•

Support for diversified methods of assessment – students on courses with diverse assessment methods supported this practice while those who were assessed largely through timed examinations expressed concern over this form of assessment. Points made on this issue include “I don’t like the idea that my MPhil will be based on four exams and a dissertation – it should be about more than just that” and “As people who are largely used to American-style continuous assessment it’s by no means clear whether the attitude that you test everything at the end of the course is the best way of doing this”. Similarly to how students on professional programs within the Social Sciences Division supported means of assessment on the basis of practical skills, students on nonprofessional programs also saw the potential for continuing to diversify assessment techniques. Comments which many people agreed with included “Plenty of other universities carry out assessment partially on the basis of class participation and oral interaction in class – why can’t this happen here” and “Learning facts is important and that can be tested in exams but there’s lots of ways in which that could be assessed including through practical workshops”.

•

Related to the above point many students expressed a feeling that methods of assessment disincentivised students to prepare properly for teaching throughout the year. This was shown through comments such as “The incentive structure of the course doesn’t lend itself to put forth maximum effort each week to prepare for seminars since the marking is entirely based on exams at the end of the year. The quality of participation in class needs to be taken into account.”

Similar views were expressed through comments such as “I’ve heard people making fun of projects which aren’t assessed – it’s disrespectful but reflects that people don’t take them seriously”.

•

Students raised concerns over the anonymity of work handed in for assessment.

This was felt in regard to coursework that was used to assess research methods training and was raised particularly frequently by students in two Departments, across a range of PGT programs. They expressed concern that non-anonymous submission would mean that “People could be singled out on the basis of factors on the quality of their work” and indicated that “There’s no real reason why all work submitted can’t be anonymous”.

Teaching provision and expectations There was a diverse range of points made by participants about what academic provision wasn’t providing. These fell into a range of categories: •

Skills not being gained as a consequence of perceived confusion over what individual degree programs were for. These were particularly felt on courses

which advertised themselves as offering training in policy development where students felt that “The degree seems to be only useful for academia but the majority of people on the degree aren’t going into academia” and “For the most part academics are divorced from the practice of our discipline in the real world; they’re not there to provide a practical lens on the theoretical issues in class”.

Students who expressed this belief often felt that the problem came down to one of two issues. The first was that the purpose of certain courses was genuinely unclear and that by trying to be “all things to all people” they were failing to have clear objectives. Other students felt that courses had clear objectives but that these were not accurately defined through prospectuses, statements of provision and other public information available to applicants. This issue existed across a wide range of courses including several for which focus groups were not run. In these instances feedback was obtained through discussions with course representatives and in writing from current students.

•

Students expressed a wish to be put in touch with alumni from their courses (specifically) and alumni from the University who had gone into careers that could be relevant to them. Students expressed a view that “We have a lot of very interesting alumni but we don’t really get a chance to see who they are and what they’re doing”. This related to points about careers advice and sources of information where students agreed that “I was hoping for more careers advice from academics specifically – they don’t seem focused on what our goals might be and alumni might just be better placed to help us get to where we want to be heading”.

•

Students often expressed concern about the variability of teaching provision, including concerns that it was focused on what Professors wanted to be teaching rather than on what students wanted and – relatively frequently – what specific courses marketed themselves as offering. Comments about variability of teaching quality were raised by undergraduates in 2010 and were recurrent with students making comments such as “the role of the supervisor is to supervise me – I don’t know why I always have to try and find them, particularly when other supervisors do a really good job”. This feeling – as was the case in a range of other PGT programs – was tied explicitly into the costs of postgraduate study. Comments such as “My course is priced at a level comparable with programs in the US and I think that’s why they can justify charging so much” were used to explain the feelings of students that certain standards of teaching provision should be expected across the board. Dual supervision (currently under discussion in the Social Sciences Division for PGR students) was raised as a possible solution to problems relating to supervisors by a variety of students.

Comments such as “I think teaching was really good because I had 2 supervisors – one with hands on knowledge, one with ideas about being a better researcher” and “Having two supervisors per graduate would ensure that each graduate is fully supported” were used to explain these views.

•

Points were made about overall facilities provision including with regard to graduate workspace provision and overall IT facilities. Students whose Departments are outside of the main ‘Social Sciences area’ within the University (the Manor Road Building and Queen Elizabeth House) expressed views that “There’s no communal area where we can really interact either with people doing the range of courses in the Department or with students on other

Masters programs”. It was also felt that “There’s a shortage of departmental workspaces – this ties in with the issue of computer facilities where it’s extraordinary how many different passwords you need between colleges, departments and all the other systems that we use”. People felt occasionally that a “proliferation of research activity” meant that opportunities for PGT students to interact with one another and be part of the academic life of Departments was limited. It was felt that this could cause problems with integration into Oxford life, particularly given time constraints on PGT courses.

Overall the point that was recurrent in discussions about teaching provision for this subset of courses that we discussed with students related to the actual quality of teaching students were receiving. While students acknowledged that some tutors and supervisors were of the highest quality, they often felt that this compounded a sense of frustration that some tutors were perceived to be taking little interest in the academic development of some students on PGT courses. A theme that was also pronounced in our interactions with PGT students in the Humanities was that much of the ‘teaching’ students were receiving was not seen necessarily as teaching but rather as interactions with academics that were not necessarily focused on students.

Role of the College In common with students in other Divisions the primary role of the college was seen as creating a social environment for graduate students. Colleges were generally felt to be fulfilling this role well. Specific themes on this point were: •

The value of social opportunities provided in a college as shown through comments such as “My college has most been a place to socialize…it has been hugely useful in providing social diversion” and “My college has been very welcoming – there are always a lot of events going on which is really nice if you want to make use of them”.

•

People saw colleges as playing a particularly valuable role in providing accommodation with students generally appreciating the opportunity to live in college. Students thought that “especially for people who don’t live in college more should be done to encourage them into the community so that they can make the most of college membership”. Many students saw advantages to concentrating courses into certain colleges, feeling that “It would be good to have all our program in one college as we get on well and would benefit from the group dynamics”.

•

The role of the college advisor was typically felt as playing a mixed role (in other Divisions) with there being no clear consensus between competing views that “My college advisor could be more involved – maybe not supervising me but looking over my thesis and everything along those lines” and a view that college advisors were only necessary if “everything’s gone wrong – that’s where I think they could play a useful role”.

Sources of support for students There was general consensus from students that a supervisor would be the first port of call for support with any problems and, failing that, another member of the department

would be approached. In common with students in other Divisions respondents saw the role of the college advisor as being supplementary to departmental provision. There was a sense that they would be approached only if Departmental provision had clearly fallen short in some way and that students would prefer to talk to someone from outside the Department.

Preparation for career post-Oxford There was a general sense from participants that the courses they were studying were preparing students more for academic progression than a professional route and that this had not always been made clear prior to their arrival on course. Comments on this point included “With my other degree it was very different – professors would discuss your future whereas here tutors take a step back from discussing what you’ll be doing next.” Similar themes emerged across a variety of subjects through expressions such as “I want to do a DPhil and I feel prepared for that, but if I wanted to do anything else I don’t know how well this would be preparing me for it”.

Against that, however, many students thought that skills being developed on their courses were transferable, even if this was not immediately obvious. This manifested itself through comments such as “The high analytical level demanded is very helpful; you can’t get away with being vague in essays or in your comments in class” or “It’s not so much the teaching on my course – there’s just so much stuff going and lots of possibilities towards building to what you want without relying on your course”. This was similar to comments made in professional Social Sciences programs where people differentiated between skills being taught on their course and opportunities that presented themselves as a result of the Oxford environment. It is a possible conclusion that the reconciliation of these two themes outlined above may require reconsideration of how certain courses within the Social Sciences are presented to prospective applicants, owing to a perceived inclarity in what course objectives are.

Which one change would have improved your time as a postgraduate at Oxford?

Three main themes emerged which people believed could have played a key role in improving their time as postgraduates.

•

Many students felt that “silos” between and within courses limited opportunities for interdisciplinary thinking and, more broadly, meeting people on other courses. This was related to what some people felt was not enough teaching provision with some students expressing a view that “Just a once weekly seminar for everyone on the course would be great – the sheer number of options means no-one is doing the same course and that’s really isolating”.

Students talked about wanting to interact more with students doing DPhils and academic staff, feeling that “things like a broader curriculum or even just a departmental newsletter with details of relevant talks going on across Social Sciences would be really valuable”. The point about the possible impact of the establishment of the Blavatnik School of Government on existing courses was also raised along similar lines to these points.

•

Points about feedback and expectations of courses were also repeatedly raised.

Students reiterated previously expressed views on feedback through comments

such as “Knowing where I stand would be really useful” and “I wanted to know where I stood throughout the year but this hasn’t been possible”.

•

Discussions of feedback inevitably related to the quality of teaching that students were receiving with students making comments along the lines of “The course structure needs to be changed so we’re actually taught – you’re meant to have a real relationship with your supervisor but we barely see them at the moment”.

Students on several programs felt that “We don’t consistently see the same tutors at all so we don’t really have relationships with any of them. At the moment it feels like the primary focus of departments is research and actual teaching quality is hugely subordinated to that”.

Annexe D: Maths, Physical & Life Sciences Expectations of and induction to Oxford The clear majority of students within MPLS subjects who participated in our consultation in any way had not studied for a degree at Oxford prior to their Masters program. Their clearest expectation of what academic life at Oxford would be like related to its “intensity”, “difficulty” and the “depth in which we would be expected to study.” People were expecting a great deal of contact time within their course and – in contrast to the views of many students within the Humanities and Social Sciences – were generally satisfied with the contact they had with academic staff and the interest which they believed academics took in their work.

Participants were typically very satisfied with the quality of induction that they received, particularly in terms of academic expectations of them. They felt that “It was made clear at the very beginning what submissions we’d be expected to make and by when” with other students feeling that “They [the course administrators] explained what we were going to do practically every week. Everything was laid out at the beginning and what they said would happen is exactly what has happened”. Programs were felt to be very intense but students typically took the view that “They told us the courses would be intense and they are so it’s really what we signed up for”. Given the experiences of students in other Divisions with comparatively unclear inductions and course organization this manner of course organization could usefully be considered within other subjects.

Feedback In common with students in other Divisions, students on PGT programs within MPLS raised very similar concerns over feedback received on their work. People indicated that they “would love it if they can give us some marks” and mirrored comments made elsewhere about the variable quality of feedback provided between different tutors.

When asked what style of feedback would be most useful the following points were made most frequently: •

Feedback is most useful if it is provided both orally and in writing on individual components of essays. The practice of providing feedback on a separate sheet of paper to essays was felt to be less useful as it was unclear which bits of written work had attracted most feedback and why.

•

In common with points made at the undergraduate level in 2010 and by students on other PGT programs students felt that old essays/theses submitted by former students on PGT courses could be of particular value in helping current students understand what constituted work of varying standards.

Students felt it was useful to have access to old essays (as was the practice on certain courses) but that knowing what academic standard these essays were seen as being at would be particularly valuable.

•

As with other programs students drew a link between the timing of assignments and feedback, saying “it’s important that you can learn from previous experiences and if you hand everything in at once then you can’t do that”. This was felt to be particularly relevant given the intensity of short PGT programs

and the importance therefore of feedback being provided on assignments handed in early on the course.

Skills & skills training The skills that participants felt that they were most clearly developing included “writing skills”, “time management – how to work to deadlines” and “the ability to churn written work out really quickly”. These skills were reflected across a variety of subjects at both the undergraduate and PGT level. Beyond these skills people were broadly content both with the skills training which existed for their courses and the skills that they felt they were developing which were more bespoke to their courses. Students with a laboratory component within their courses typically reported that “we’ve developed research skills which will be useful to those of us going onto DPhils”. People also felt that there was a wide range of training opportunities available that meant people could choose what they wished to specialize in when they were working towards projects.

People typically felt that their departments were flexible both in terms of offering a wide range of options for projects (generally seen as one of the most valuable elements of PGT programs) and in offering training to support effective work on these projects.

Although some students felt that too much emphasis was placed on comparatively basic research methods and footnoting skills, others, particularly those from nonEuropean backgrounds, thought that these skills were particularly useful in assisting them in transition to Oxford’s style of writing and learning.

Assessment As was the case in other Divisions students preferred a model of assessment that placed less of an emphasis on written examinations at the end of students’ courses than is normal for an Oxford undergraduate degree. They typically felt that “science isn’t about taking an exam” and agreed with sentiments expressed by students such as “Taught Masters programs in science with an exam at the end are just like doing an undergraduate degree all over again”. The most frequent points raised within this category were: •

Support for the idea that assignments could be more evenly distributed throughout the year with students on some courses finding that lectures and other forms of teaching weren’t being attended at points in the year as a result of deadlines being bunched together at specific points of the course. This was felt to distort people’s work patterns over the course of their Taught Masters program.

•

A belief similar to that found in other Divisions were students wanted the basis for marks on certain assignments to be made clearer. This was particularly the case for students on courses who were being given marks in bands rather than told exactly how they did. They felt that “it’s really difficult to concentrate your efforts if you don’t know how you’re doing on specific pieces of work – if you know that you’re borderline obviously you can focus more on individual items than if you just don’t know how you’re doing”.

Teaching provision and expectations When asked what current academic provision isn’t providing, students within MPLS referred to guidance on future careers more than any other topic. Students studying Applied Statistics referred to “more specific careers guidance and recruiting for stats majors” and said that “provision was generally good but the careers”. People with Zoology felt that an excessive focus was placed on DPhil progression – while this was seen as being useful for people intending to do a DPhil they felt that the Careers Service “might not be specialized enough in our area to know about our needs” and said that engagement with recent alumni would be a good way of promoting possible careers outside of continued academic study. This finding also emerged in other focus groups including for students on professional courses in Social Sciences with regards to the Said Business School and a number of discussions in other Divisions. Some students reported feeling that the Careers Service was possibly more focused towards undergraduates than postgraduates and could not provide bespoke careers advice which would be relevant for PGT students. Across a range of courses students felt that more alumni engagement would be useful and that careers provision focused excessively narrowly on a number of “generic” options; either continued academic work or students entering careers within the financial services or management consultancy sectors. Along a similar vein, participants typically felt that more interaction with DPhil students would be useful; both in terms of helping current PGT students decide whether to progress to DPhil study, and also to assist with induction in helping new PGT students understand the “informal norms” which existed in Oxford.

MPLS students were generally more consistently satisfied with the quality of teaching that they had received than students in any other Division, saying that “overall the teaching here is either good or great” and that “compared to the undergrad programs we did it’s just been much better in terms of the quality of the teaching”. Students in MPLS were similar to students doing other courses in referring to the quality of the overall academic environment in Oxford, reporting that “there’s so many other lectures and talks going on here all the time – even outside of the Department there are people talking where you want to hear them speak and that sort of provision just doesn’t exist in many places”. Points were made about occasional bad experiences with tutors but it was typically felt that “while some are substandard you know they’re very intelligent” and that “the only bad lecturers are phenomenal scientists who just have a lot of difficulty bringing across what they do…they’re absolutely exceptional but they just can’t teach”.

More so than students in both the Humanities & Social Sciences, PGT students in MPLS reported that they felt heavily overworked at times and that – more than any other area – this was likely to have an impact on their experience of teaching. They felt occasionally that “sometimes I just want to spend more time to think about things” with students reporting that “at points you just have to stop attending lectures because you’re covering so much material”.

Role of the College The role of the college was felt by students in MPLS to be primarily social with students reporting variability and patchy provision in terms of the system of college advisors.

Typical comments made by participants included “I haven’t had much academic contact with them and they don’t support us on an academic basis but in a social way I have an

amazing college and it really helps you get out of the madness on your course”.

Students saw the financial support provided by certain colleges as being valuable but felt that college advisors were often of limited use as “they just don’t know enough about our subjects in most cases to be valuable”. Against this, however, some participants saw colleges as playing a role of a “backstop” within the pastoral system through comments such as “If you need them they are there, even though you normally don’t”.

Sources of support for students All participants felt that a point of contact within their department would be their first port of call for support with fewer students in MPLS than in the other Divisions saying that colleges would be a secondary source of support. In addition to supervisors and course co-ordinators MPLS students often referred to their friends on their PGT programs and DPhil students as potentially being valuable sources of support, regularly emphasizing the role which the Departmental community played in providing support to students who required it.

Preparation for career post-Oxford In addition to the comments about careers made earlier (under the heading of academic provision) participants typically felt that their programs of study were preparing them both for possible DPhil work. Students reported that the economic situation was putting this question to the forefront of their minds and that they had expected that “the pure reputation of Oxford would help with the initial screening”.

Broadly speaking students were content with how well their course prepared them for life after it, feeling particularly that it would be good to have more information about what a broader sense of career paths for them were.

Which one change would have improved your time as a postgraduate at Oxford?

The change which students in MPLS most consistently wanted to see was more time for the PGT programs within the Division. Numerous students on a range of programs raised the point that “This course has more than enough material to cover two years and I’m not really sure why it is a one year course” and “A two year program with a stronger link to the DPhil would be really good”. This issue appears to have been discussed within the most recent Review of the MPLS Division and could perhaps usefully be revisited given the attitudes of students on existing on year PGT programs within the Division. Although many students were supportive of the idea of introducing more two year PGT programs within the Division others indicated that distributing work more evenly over a one year PGT program could also address the problem, feeling that “When you have a week to do five pieces of work you become really good at writing things quickly and handling a large workload but less good at critical thinking”.

No other issues were consistently raised by PGT students within MPLS.

Annexe E: Medical Sciences Expectations of and induction to Oxford Participants who had done their previous degrees at Oxford or elsewhere agreed that they were expecting an intense academic experience from their Masters but that this hadn’t always materialized, particularly in their first term. Students who had done an undergraduate program at Oxford often expected “things would be pretty much the same as in my undergraduate” in terms of the intensity of experience they would have but found the teaching more “laid-back” than they ultimately received.

Students who had not previously studied at Oxford expressed similar beliefs through comments such as “I was expecting much more intensity all year long, the first semester there wasn’t very much on” and “You show up and then you get here and there’s not much on”. Such comments were, however, often specific to the first term with more deadlines and structured work being timetabled for the second and third terms within the course. Students who had not previously studied at Oxford were very dependent on University websites to find out details about the course and felt that such information as was available was useful.

Induction was felt to be relatively confusing for participants. This was largely a result of information being dispersed and distributed at various points by various sources.

Students felt that meeting students who had previously done an MSc at Oxford and had since moved on to doing a DPhil was very valuable when such opportunities existed. This was owing to the role that former PGT students could play in introducing new students both to Oxford academic life in general and to individual MScs in particular.

Departments were generally seen as providing accurate, timely information with students feeling that “emails started coming out about the course in April/May – while lots of things were unclear to me they were never related to my course”. Where students were less clear on introductions to Oxford it was typically because little prearrival contact had been made by their Departments. Students emphasized that pressures about accommodation pre-arrival could be particularly difficult through comments such as “some colleges aren’t very clear with the whole accommodation thing – I only found out a week before arrival where I’d be living”. Students also discussed the room for a more collaboratively produced induction, stressing that “sometimes there are millions of events you have to attend which don’t seem to be coordinated with each other at all”.

Feedback In common with students in other Divisions, participants generally felt that there was significant room for improvement with regards to feedback received on their work.

Students typically felt that feedback was “a bit scrappy” and often “not enough, too late”. The promptness of feedback was seen as being particularly important, particularly in context of fees being paid by students through comments such as “we are paying them to be our teachers; they should meet deadlines just as we’re expected to”. When pressed as to what form participants would like feedback the main recurrent points were:

•

Feedback that would be applicable both on the specific piece of work in question and to future work. This was expressed in terms such as “The feedback they give us is very specific to the essay, it would be better if it were more general”. This view was tied in with the idea of feedback being used as a means of helping students improve – “The whole point of getting feedback is so you can use it for essays afterwards”.

•

A combination of oral and written feedback where tutors mark essays and then provide feedback orally in a tutorial-like setting. Students who had studied at Oxford before found the tutorials they had received as an undergraduate useful for “getting to know the tutor and what they’re after”. It was, however, also felt that marks on work would be useful since in the absence of them “the correlation between the feedback and the grades you get in the end isn’t necessarily very good”. It was thought that students receiving marks on their work would assist them in identifying how they were doing relative to assessment criteria for their courses and assist in preparation for subsequent assessment. In such instances where it could be inappropriate to receive oral feedback on work (e.g for assessed work where the examiner could not discuss it with you) it was suggested that written feedback would nevertheless be useful.

•

Students who hadn’t studied in the UK as undergraduates were particularly keen to receive feedback on essay structure and how written work should be structured with participants emphasizing that “the system in the UK for undergraduates is just completely different to where I studied for mine”.

Skills & skills training The skills that students feel they are developing vary somewhat by program being particularly related to the predominant methods of teaching that exist on each course.

Project/practical skills are seen as being particularly desirable by students, in part because of the high rates of progression to PGR study which exist within PGT courses in the Medical Sciences Division, in part also because practical skills were seen by participants as being ‘new’ relative to skills which they gained through their undergraduate programs. Participants expressed a preference for practical classes over lectures with those students who hadn’t had the chance to do lab work in the first term of their Masters saying that they would have appreciated lab work being introduced earlier in the course.

In addition to practical skills students felt that they were developing their analytical skills through programs such as “Journal Clubs” where they were invited to critically assess articles printed in medical journals. Essay-writing was seen as being a skill that students on certain courses were developing with students from overseas reporting that “on some of our courses the focus was on lab reports – these aren’t anything like the essays we’re expected to do here so we need to know how to integrate things together to form an essay”. Interestingly, some students reported that they were developing media & communication skills through initiatives such as being taught to give presentations targeted at audiences with a non-scientific background. Although this does not yet appear to be widespread people felt that this was valuable for their future career development, possibly reflecting broader trends relating to the REF and future research funding.

There were comparatively high levels of satisfaction with levels of skills training with people indicating that training organized through the Medical Sciences Teaching Centre enjoyed variable rates of take-up but was seen as being useful when people opted into training sessions. Skills training was seen as “helping us appreciate the difficulty of actual experiments” although the MSTC was urged to run programs throughout the year and to centrally advertise skills training opportunities in a more structured way. In common with other subjects, students within the Medical Sciences Division reported that “the sheer number of emails we get from our Department” made it hard to sift through them to find training opportunities. There were few examples given of students not appreciating the value of skills training opportunities if they did enroll on them.

Assessment The diverse means of assessment within PGT programs were seen as positive by participants, mirroring the findings from our consultations with undergraduates that discovered that the uniformity of timed examinations was seen by many undergraduates as being excessively one-dimensional as a means of assessing work. The main points raised by participants were: •

Inclarity over assessment criteria and the relationship this had to feedback.

Students doing the MSc in Psychological Research commented that they were unclear on the marking structure for their course and how different assignments were weighted to produce a final mark. The relationship that assessment had to feedback was also indicated by students doing the MSc in Neuroscience who drew a distinction between PGT and PGR programs, saying “When we say we want more structure they say ‘it’s a graduate program, you should be independent learners’ but it is still a Taught Masters’.

•

Assessment being spread out over the course of the year. Participants highlighted a perceived lack of rationale behind – for example – three assignments being due in over the course of the Hilary vacation while only one was due in over the entirety of Michaelmas Term. Students studying Pharmacology explicitly drew a comparison with “other MSc programs which have very crammed timelines”, naming specific examples of such programs.

Students felt that having assessment more evenly spread over the course of a Masters would promote “constant work”, unlike the arrangements on certain programs where “you just find a topic to write on and then switch off…the methods of assessment encourage narrow areas of learning”.

•

The possibility of being assessed on the basis of work in groups including participation in class and interaction with other students. It was felt by participants that this could test problem-solving, research skills and incentivise more continuous work with other students.

Teaching provision and expectations When asked what current academic provision isn’t offering, there was a latent theme within all focus groups of progression after the Masters. Students on the MSc in Neuroscience emphasized the possibility of more information being provided on “nonscience Careers”, indicating that they had found an event run jointly between the Careers Service & Biology particularly useful, largely because of the opportunity which it

gave them to interact with Oxford alumni who had done PGT courses. By contrast, students on other courses wanted more guidance on how they could get published in academic journals and a greater overview of what life as an academic or DPhil student would involve. In spite of many participants having determined that they wished to continue to DPhil study there was nevertheless a sense that departments could do more to explain what a DPhil would involve and how it would differ from a PGT program. Participants had a clear sense that the courses they were undertaking were good preparation for a DPhil,. The rates of progression to PGR study were cited as a reason for this, reflecting the knowledge of students on PGT programs. Other than points about progression post-MSc, the other major point raised in response to this question again related to practical skills and the desire of most participants to spend more time doing labwork, believing that “it’s intense but the project is the most interesting part of the year”.

In terms of how teaching was comparing to expectations prior to starting on course the most common points were: •

A desire for more individual teaching. This was pronounced in requests for “more tutorials”; even when students were getting what they saw as more “personalized” teaching than they had received for their undergraduate programs. Students felt they wanted individual teaching although they were broadly satisfied with their teaching provision in most cases.

•

More consistency within teaching provision. Respondents indicated that “some tutors focus more on teaching you, others on telling you about their research.

The teaching isn’t brilliant, overall. It’s not bad, overall, but it’s not brilliant”.

Students recognized without prompting that some of their expectations could require an increase in resource available but felt that the variable quality of supervision was creating “unfair advantages” for some people enrolled on courses. Students who were dissatisfied with their supervision felt that “the fact that I’m getting zero support isn’t going to be taken into account” and, in the case of some students, to favour measures like a “Bill of Rights” so they could know what could be expected of a supervisor and a student. Support was expressed for ideas such as possible mandatory monthly meetings between a student, post-doctoral researcher and supervisor to ensure that student progress was being suitably monitored.

•

Related to this point was the perceived approachability of certain Professors.

While students were keen to express their view that they were generally satisfied with the quality of teaching, they raised the possibility that Professors could have a limited number of ‘open hours’ where they could be contacted by students within the Department owing to the difficulty that students felt they had in contacting certain academics. It was felt that this would assist in building a better community within the Department and foster closer links between faculty and PGT students.

•

We particularly asked participants about any variability between the treatment of PGT & PGR students by supervisors. This was asked in the context of work we did last year on the experience of PGR students with supervision but no such variability emerged in focus groups. Students typically felt that supervisors treated DPhil and MSc students with similar degrees of care. It was felt that the

variable quality of supervisors accounted for any discrepancies, rather than differences in attitudes to PGT & PGR students per se.

Role of the College As with other subject areas, the attitudes of participants towards colleges was split between those who had little interaction with their college and those who saw it as providing a social environment. When pressed on possible academic interactions with colleges there was an overwhelming response along the lines that “The masters is nothing to do with the college and it should be that way” or “From an academic point of view we don’t get anything from colleges and we don’t need it”. Students raised the point that for science students “the whole science area is part of our experience” thus minimising the need for college provision.

Students also felt that there was a valuable social function that colleges played in their experience (among those who made use of college facilities). They expressed views that “people are around all the time which is good” and “I go mainly for my social life – the colleges are in a way just a nice addition; it’s quite fun to have one but I don’t think they’re very crucial to a graduate student. If you’re with a good college then it’s quite nice to be able to see other people”.

Colleges were seen as being valuable for accommodation provision with some students expressing confusion prior to arrival in Oxford over how accommodation worked in terms of the obligations of colleges to provide accommodation or assistance in finding private sector accommodation. This tied in with a broader concern found within a significant minority of students who participated in our consultation and expressed sentiments such as “we’d really like to know what colleges actually offer specifically to postgraduates that isn’t open to all members of the college”.

Sources of support for students The first port of call for students would be either the Course Director or Course Administrator. Students placed an emphasis on “how approachable” the first port of call for pastoral support is as being the key variable in determining to whom they would turn for assistance. When pressed as to whom secondary sources of support would be a minority indicated that their College Advisor or College Academic Administrator would be a source of support. A majority signaled that another point of contact in their Department (such as their supervisor or another lecturer) would be a more likely source of assistance.

Preparation for career post-Oxford Students generally felt that their courses were preparing them well both for nonacademic careers but particularly for DPhil study. Comments about DPhil study included “You almost have to fight off DPhil offers after it”, “The research training really prepares you; not only for continued research on what you’re doing but thinking about what you could research more generally” and “It really puts you on a trajectory for the DPhil”. Although the majority of participants were intending to continue to DPhil study those who were not planning on such a route were similarly positive about the training provided by the course. Comments from those who were unsure about their intended route post-Oxford generally agreed with sentiments such as “It’s definitely kept my

options open and opened new options”, People felt that the reputation of Oxford was beneficial for their future career prospects in addition to the individual skills that they had acquired as shown through comments such as “I feel like I’ve got the upper hand as a result of Oxford as well as the skills that I’ve got”.

Which one change would have improved your time as a postgraduate at Oxford?

The biggest issues raised by participants in response to this question were: •

Concerns over feedback. In line with students on other PGT programs people felt that the lack of feedback they were receiving was hindering their understanding of what was expected of them. Comments such as “More feedback could help because they test too much and don’t teach you enough” and “The amount of feedback is way less than what I expected” were repeated across participants.

•

The course structure and time available. Some people felt that the brief nature of the course made it hard to apply for DPhil study as “Because it’s a one year course you have to apply for DPhil projects immediately”. Even those who did not raise DPhil progression explicitly, however, felt that the timing of assessment made courses problematic. A particular issue was what was felt to be a comparatively empty first term as opposed to a second term “where a massive load of work hits you” and a concentration of assessment from the beginning of the Hilary vacation onwards.

•

Support for diversified methods of assessment with students being supportive of more assessment on the basis of practical skills and “assessment being tailored to the content of the course”. This was seen as countering a perception that “the essay is the Oxford thing so you’re expected to be assessed on the basis of that even if it’s not the most relevant form of examination”.

Annexe F: Focus group structure (prompts were used if necessary to stimulate discussion) 1. What were your expectations of the academic environment you would enter before you started your course at Oxford?

2. How well do you feel you were introduced to what was expected of you at the start of your Masters (Prompts: course handbook, information on tutorials)?

3. Do you have any views about the quality and quantity of feedback which you receive on your work?

4. What skills do you feel you are developing as a result of the teaching on your course?

5. How content are you with the level of skills training which has been offered to you on your course (Prompts: research methods, footnoting, avoiding plagiarism).

6. How well do you feel the methods of assessment are suited to the structure of your course?

7. Thinking about what you wish to gain by the time you leave Oxford is there anything you feel that current academic provision isn’t providing?

8. How has the teaching you have received compared with your expectations prior to starting your course?

9. What role has your college played in your experience at Oxford (Prompts: accommodation, pastoral care, social environment, academic environment)?

10. Thinking about your relationship with your department, how satisfied have you been with the departmental provision you have received so far (Prompts: supervisor, lectures, seminars, classes)?

11. If you ran into difficulties while studying at Oxford which sources of support would you turn to (Prompts: supervisor, Disability Advisory Service, college advisor, counseling service, Student Union)?

12. How well do you feel your current course is preparing you for what you want to do after Oxford (Prompts: DPhil study, career, other academic study)?

13. If you could recommend one change which would have improved your time as a postgraduate in Oxford, what would it be?

Annexe G: Information on Attainment for Students on PGT programs 5 Medical Sciences Division – attainment data for students who have previously studied at Oxford

2005/06 2006/07 2007/08 2008/09 2009/10

% of%students of students attaining attaining a a Distinction Distinction 47 47 36 36 38 38 26 26 44 44

% of students % of students % of students % of students Total no. failing of attainingattaining a Pass a Pass failing their their course students in course category 53 53 N/A N/A 15 64 64 N/A N/A 14 63 63 N/A N/A 6 75 75 N/A N/A 8 56 56 N/A N/A 16

Medical Sciences Division – attainment data for students who have not previously studied at Oxford

2005/06 2006/07 2007/08 2008/09 2009/10

% of students attaining a Distinction 18 17 19 19 16

% of students attaining a Pass 81 83 81 80 82

% of students failing their course N/A N/A N/A N/A N/A

Total no. of students in category 95 114 121 103 107

Social Sciences Division – attainment data for students who have previously studied at Oxford

2005/06 2006/07 2007/08 2008/09 2009/10

% of students attaining a Distinction 26 25 29 26 24

% of students attaining a Pass 74 75 68 73 75

% of students failing their course N/A N/A N/A 1 1

Total no. of students in category 160 124 134 162 125

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

5

Data in this section has been prepared and provided by Student Data Management & Analysis. Where rows do not add up to ‘100’ it is because of rounding or the exclusion of students achieving a ‘Merit’ grade.

Social Sciences Division – attainment data for students who have not previously studied at Oxford

2005/06 2006/07 2007/08 2008/09 2009/10

% of students attaining a Distinction

% of students attaining a Pass

17 20 18 19 18

83 80 80 78 80

% of students failing their course 0 1 1 1 1

Total no. of students in category 1,103 1,234 1,302 1,359 1,050

Humanities Division – attainment data for students who have previously studied at Oxford

2005/06 2006/07 2007/08 2008/09 2009/10

% of students attaining a Distinction

% of students attaining a Pass

53 63 47 50 54

47 36 51 50 45

% of students failing their course N/A 1 N/A N/A N/A

Total no. of students in category 109 99 113 115 115

Humanities Division – attainment data for students who have not previously studied at Oxford

2005/06 2006/07 2007/08 2008/09 2009/10

% of students attaining a Distinction

% of students attaining a Pass

32 33 28 31 31

66 64 69 66 68

% of students failing their course 1 1 1 2 1

Total no. of students in category 273 270 293 306 280

MPLS Division – attainment data for students who have previously studied at Oxford6

2005/06 2006/07 2007/08 2008/09 2009/10

% of students attaining a Distinction

% of students attaining a Pass

45 22 47 60 53

55 72 53 40 33

% of students failing their course N/A 6 N/A N/A 13

Total no. of students in category 11 18 17 10 15

MPLS Division – attainment data for students who have not previously studied at Oxford

2005/06 2006/07 2007/08 2008/09 2009/10

% of students attaining a Distinction 26 23 29 26 30

% of students attaining a Pass 71 71 70 72 66

% of students failing their course 2 3 1 2 4

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

6

Data for the MPLS Division excludes programs in Software Engineering

Total no. of students in category 117 145 161 162 163