795finalreport

Proctored Testing On Campus for Online Classes at San Diego State University John Gangitano, EDTEC M.A. Candidate, SDSU Scott McAvoy, EDTEC M.A. Candidate, SDSU Paula Miranda, EDTEC M.A. Candidate, SDSU

1

Abstract This report examines the threat of academic dishonesty at San Diego State University, investigates strategies and technologies used to prevent cheating at other universities, and analyzes the effectiveness of a pilot exam delivery method sponsored by SDSU’s Instructional Technology Services (ITS) in the 2012 Fall semester. As classes are moving into online formats, instructors have expressed concern that online exam delivery systems are especially vulnerable to student cheating. The ITS, in partnership with the Love Library, began piloting a new method of assessment delivery of campus proctoring for a large group of students taking computerized exams in a monitored computer lab. A survey was distributed to participating students in order to assess the effectiveness of the pilot and the prevalence and frequency of academic dishonesty at SDSU. An analysis of results revealed that students perceived cheating to be widespread, though the pilot itself was effective. Recommendations for San Diego State University Policy and further research are offered.

2

Table of Contents The Problem Literature Review Do No Harm Cheating Prevention Technologies Cheating Prevention Strategies Survey Report Description of pilot Purpose of survey Population Method of Dissemination Discussion of Results Recommendations References Appendix

3

The Problem SDSU’s online programs have grown substantially over the past 5 years. There is immense pressure for some instructors to move their courses online, often without training in the nuances of the online course format. As class materials and assessment instruments are put on online, instructors are less and less confident that they can keep their students from cheating on exams. While traditional, face­to­face learning environments are certainly not immune to cheating, the exponential rise of online education over the last decade has brought renewed focus to the issue. The Sloan Consortium publishes an annual survey of 2500 institutions of higher education stating that over 6.7 million students took at least one online course during the fall 2011 term, an increase of 570,000 students over the previous year, or 8.5%.

SDSU is at the forefront of the move to online education, with annual growth rates far exceeding the national averages (Table 1.2). SDSU has experienced large and steady growth in both the number of and student enrollment into online courses. Larger course offerings during summer sessions are necessary for working and distance students, giving them the opportunity to continue their education in a season where face­to­face requirements prove inconvenient, and improving their times to degree completion.

Spring Summer Fall Total % Growth

(Table 1.1 Online Courses Offered at SDSU#) 2008 2009 2010 2011 3 6 17 27 12 31 42 62 4 11 16 17 19 48 75 106 n/a 153% 56% 41%

2012 19 89 34 142 30%

Spring Summer Fall Total % Growth

(Table 1.2 Online Student Enrollment at SDSU# ) 2008 2009 2010 2011 187 549 1608 2642 572 1483 2415 3751 426 1016 1675 2674 1185 3048 5698 9067 X 157% 87% 59%

2012 3051 5481 5745 14277 57%

data from https://sunspot.sdsu.edu/schedule/

Despite this worldwide growth of classes being offered on the Internet, there is still a perception that online classes are not equally rigorous or secure as traditional on­campus classes in educational quality. A joint project study from The Babson Research Group and Inside Higher Ed released in June 2012 stated, “The faculty report being more pessimistic than optimistic about online learning. Academic technology administrators, on the other hand, are extremely optimistic about the growth of online learning, with over 80 percent reporting that they view it with “more excitement than fear.” Professors, over all, cast a skeptical eye on the learning outcomes for online education. Nearly two­thirds say they believe that the learning outcomes for an online course are inferior or somewhat inferior to those for a comparable face 4

to­face course.”

As online learning continues to grow, an industry based on academic dishonesty, once hidden from instructors and school administrators now advertises in plain sight on the internet. WeTakeYourClass.com, UnemployedProfessors.com, and BoostMyGrades.com are just a few companies allowing students the option of buying their grades. Services include taking exams while impersonating the enrolled student, taking whole classes, writing term papers, or even entire theses­ all of which blatantly undermine the efforts made by faculty and institutions to mitigate the various threats to academic integrity in these environments.

Though blatant advertising has made academic dishonesty more apparent to instructors and school administrators, the problem is certainly not a new one. Paper mills and test taking services have been available to students since long before the advent of online learning. The A+ Review in San Diego offers students access and/or information on old (and sometimes current) test materials for a wide variety of undergraduate business courses at SDSU. Professional test taking services exist, capable of taking multiple Blackboard format tests at once when supplied with a student’s login information. Any seasoned instructor can share war stories about the ingenious and creative lengths students go to cheat on Scantron® exams. So while the issue of academic dishonesty has become more visible through an online medium, an analysis of scholarly sources shows that there is very little data proving that it is any more prevalent in online courses than it is in traditional, face­to­face classes, making efforts towards maintaining the integrity of online courses possibly more effective towards mitigating overall cheating, than they would be in face­to­face courses. SDSU’s need to ensure academic integrity is also motivated by the need to retain accreditation, and to abide by federal guidelines set by the Higher Education Opportunity Act (HOEA) of 2008 requiring that efforts be made to ensure the identities of students receiving federal funding. Instructional designers are then presented with an opportunity to leverage the visibility of cheating and regulatory necessity towards obtaining resources to mitigate the problem on the whole.

SDSU’s Instructional Technology Services, in partnership with the Love Library, is piloting an exam delivery method whereby large groups of students enrolled in an online course must come to campus and take their computerized exams in a proctored setting. The goal is to create secure tests by employing traditional test proctoring strategies, sound test design, and leveraging the capabilities of Blackboard’s assessment tools. The role of this paper is to evaluate the ITS’ exam delivery pilot through an analysis of a student survey, to investigate anti­cheating strategies and technologies currently in use at other institutions, and to provide recommendations towards the prevention of academic honesty at SDSU.

Literature Review No single strategy or tool can hope to eliminate the massively complicated issue of academic integrity. The problem is further complicated by a scarcity of campus monetary and physical resources, the novelty of technologies in this fast growing market, and the rarity of reliable data due to the secretive nature of academic dishonesty. Given this complexity, it is necessary to adopt a varied and pragmatic approach.

After reviewing available literature, researchers have arrived at the following three conclusions: 5

1) Doing no harm­ Online course delivery is by no means inherently inferior to traditional face­to­face formats in terms of the ability to prevent academic dishonesty.

2) Cheating Prevention Technologies­ There are a number of tools and services available which could present solutions to SDSU. These new technologies are to be investigated.

3) Cheating Prevention Strategies­ There are a number of universal pedagogical strategies which can mitigate the effects and potential for academic dishonesty at SDSU. These strategies are to be investigated.

Do No Harm Gaps between average class grades in online and traditional face­to­face course formats present a problem to the ITS. If scores are higher in an online section, critics of the online format are quick to suggest that cheating has occurred on a wider scale. If scores are lower in online sections, critics might claim that the decrease is indicative of inferior learning gains. It then becomes important to show that properly designed courses and assessments result in no significant difference in average course scores between the two formats. Gaps between class averages are then the product of poor design choices, rather than the method of course delivery itself.

A number of studies have been conducted in an effort to justify and support use of online exam delivery methods relative to traditional exams delivered on campus. The primary concern is the absence of an instructor during online exams, and what the subsequent the implications are for cheating and academic dishonesty. Some empirical findings show no significant difference in performance or scores between online or campus exams. In a 4­year study of exam grades between the online and campus sections of the same Earth Science course, Prof. Matthew Werhner found that his online students performed at the same level as those in his campus section (2010). However, are similar scores alone indicative of cheating, or lack thereof? In her dissertation about the validity of online testing, Jessica Street (2008) gathered qualitative data in the form of student opinion regarding cheating, and found that while a majority of students in her study acknowledged the increased opportunities to cheat during an online exam, there was no significant difference between the actual rate that cheating occurred between the two delivery modes.

When the stakes for a passing score are elevated, as they are in classes that are required for the completion of a degree or certification, both exam scores as well as student opinion still indicate that cheating isn’t any more prevalent in an online learning environment than they are in a traditional campus environment (Roh & Park, 2010). In some cases where test scores were lower in the online section of a course than in the campus section, researchers have suggested that academic rigor is actually improved in online exams, especially when good test design is used in combination with on­site proctoring (Prince, Fulton, & Garsombke, 2009).

Cheating Prevention Technologies Academic dishonesty, no matter the course delivery mode, is a major concern for any learning institution.

6

For online classes, as innovations in technology and pedagogy rise, so do the challenges to academic integrity and authentic assessments. As such, new threats and potential solutions are constantly being implemented, monitored, and reviewed. A combination of innovative software along with good test design and authentic assessment is recommended in order to create secure and meaningful exams in online courses (Howlett & Hewett, 2006). The methods listed below are by no means exhaustive; rather, they have been studied and proven to be effective at their respective institutions and, thus, included here for consideration.

Proctoring Centers and Independent Proctors A more traditional approach for delivering tests to distance education students has been to administer them at proctoring centers. The student schedule and pays a center to act as an intermediary, receiving test materials directly from an instructor, administer the test, and then send the results back. A presentation by SDSU’s own Dr. Stephen Schellenberg (2012) shows that testing centers are widespread, though underutilized, throughout the United States. Testing Centers are often associated with Universities, community colleges, military bases, and even some large­scale employers. Many libraries offer proctoring services, though many have stopped due to budget concerns. San Diego County Libraries have discontinued their proctoring services as of June 2012.

While testing centers may be plentiful, it is difficult to determine which organizations can be deemed trustworthy. There is little to no regulation regarding proper methods and facilities. Only test centers with contracts to administer high stakes standardized exams like the LSAT, MCAT, GRE and GMAT are required to adhere to regulations set by test creating companies.

Dr. Schellenberg also mentions a method employed on a smaller scale by SDSU’s Dr. Gary Girty. Dr. Girty allows his distance students to identify and organize test proctoring with any individual in their community boasting some sort of moral, lawful, or professional authority. These proctors could be professors, military officers, clergy members, or even medical doctors. These community proctors are then sent a letter/contract notifying them of their duties and restrictions, and thanking them for their participation. Students opting for this proctoring method are required to identify a community proctor and schedule test taking sessions for the whole year. From there on out, all handling of test materials is restricted to the class instructor and the proctor. Though a this method is open to a number of vulnerabilities, depending largely on an honor system, Dr. Girty has reported no indications of academic dishonesty.

Item Randomization One of the greatest benefits of a computerized test format is randomization. Randomizing the question and the answer order is another good tactic for ensuring that each student has a unique test that is essentially worthless to copy (Adkins, 2005). Option C on question 2 for one student, becomes option C on Question 25 for another. Students hoping to collaborate on tests are unable to do so when they cannot progress from question to question in sync, and when multiple choice answers are linked to varying options. This is a very difficult strategy to implement on traditional scantron and paper tests, but through use of a computerized system, like SDSU’s Blackboard, the method presents no additional effort for the instructor.

7

Proctoring Software Remote proctoring software offers students the ability to take a proctored exam from home at a cost of $10­$30 depending on the particular service and options. Proctoring software monitors in real time or records a student’s test taking session through a webcam. These softwares offer a range of features including: ○ ○ ○ ○ ○

Confirmation of user identity through matching photo IDs and faces Scanning the surrounding area for unauthorized materials and cheat sheets Confirming connectivity issues are legitimate, and not faked for additional test time Restricting internet access to only the online resources allowed by the Instructor Analysis of user movement and typing rhythms

Some MOOCs (Massive Open Online Course) have signed or are negotiating deals allowing course credit for MOOC students who opt to have their exams proctored. These services are becoming increasingly important in California as the passage of Senate Bill 520, which is still being deliberated, could open the window for MOOCs to provide college credit to waitlisted college students as a way to expedite their degree completion process (Gardner & Young, 2013). There are a number companies on the market at this time, but the following hold the largest market shares: ProctorU ­ Has signed an agreement with Coursera, one of the largest MOOC platforms.

The company has already proctored two, credit­bearing MOOCs (Eisenberg, 2013). While it is not yet clear what the specific requirements will be for delivering and securing online exams on such a massive scale, ProctorU­Coursera partnership makes a case for off­site online proctoring as a viable model for ensuring the delivery, security, and integrity of online exams.

SoftwareSecure ­ Provides a number of options for secure online assessments. Researchers at Arkansas Technical University tested SoftwareSecure’s Remote Proctor Pro, which utilizes the combination of a 360 degree camera, a fingerprint scanner to confirm student authentication, and a lockdown browser. Though some criticize such measures intrusive, students here found it a to be convenient solution for taking online exams (Cochran, Troboy, & Cole, 2010). This company also offers a less costly service which simply records a video of the test session using a regular webcam, and saves it for later review. The downside is a lack of preventive intervention. If a student is exhibiting strange behavior on camera, it cannot be addressed live, but rather noted and reported to the instructor to give a final ruling.

ProctorCam ­ is currently being used by the Metropolitan College at Boston University for their online Project Management graduate program (Leybourne, 2011), provides real­time, live proctoring via online webcam, where students check in and are monitored by proctors throughout the duration of the test. About three staff members are needed to administer a test for every eight students: one to greet them and two to serve as proctors (Moore, 2010). Both students and administrators of the PM program generally have a positive opinion of the service despite the cost,

8

since it is a strategy that they believe is indicative of a quality online program.

While these technologies provide effective deterrents to cheating, especially by way of confirming the identities of enrolled users, the services they offer are bound by an ever­changing technical infrastructure.

To create a more sustainable solution, it is very important to train faculty in cheating prevention strategies, which can transfer to any course design.

Cheating Prevention Strategies Many of the teaching strategies that can be used to prevent cheating in online courses are just better teaching strategies overall. Universal design has the potential to improve instruction and learning across the board. The following strategies have been identified as having the potential to be effective, and realistically implementable by SDSU. Honor Code and Honesty Contract Creating a culture of honesty in the classroom, as well as throughout a learning institution, is an important and effective cheating reduction strategy (Howlett & Hewett, 2006). From simply defining what constitutes cheating, to including an “honor question” in each exam, finding ways to discuss with and remind students about academic integrity helps to affirm its importance. An honesty contract takes a direct approach to the honors code by making students sign an agreement that states what is acceptable and what the consequences are for academic dishonesty. While it is not a solution in itself, a contract promotes student accountability for their work and establishes the importance of academic honesty early in the course.

Authentic assessment Whenever possible, using multiple methods of authentic assessment that require problem solving and written reflection rather than high­stakes objective exams is an ideal strategy for promoting academic integrity in the online learning environment. The lack of “evaluative threat” reduces the pressure for a student perform and, instead, allows them to engage in positive learning activities that might be more effective in promoting learning (Howlett & Hewett, 2006, p. 320). In situations where objective tests are needed­ as is the case in this pilot­ cheating prevention relies in ensuring that the test itself is designed to minimize the opportunity to cheat.

Allotted Assessment Times There are several settings on online tests that can deter cheaters. One common way is to minimize cheating is to restrict the allotted time to take a test as well as the window in which the test is available.

This makes it harder for students to look up the answers, as well as copy the questions and distribute them to others (WCET, 2009). Online courses are often designed for maximum student convenience, allowing wide periods of time in which a test can be taken. This is a vulnerability which is easily closed. The small numbers of students unable to comply with these time periods can usually be dealt with individually, by requiring a scheduled makeup exam.

Test Banks 9

Another way to ensure that tests are kept secure is to maintain update a test bank. Test banks, or question pools, are groupings of similar questions all made to assess the same concepts. This pool of varied questions can be called upon to create individual student tests which measure the same proficiencies, but use different test items. Test banks can either be created by an instructor, or bought from a publisher and modified. According to Howlett and Hewett (2006), “The larger the course and the more often it’s offered, the more often the question bank needs to be expanded” (p. 324) Doing so ensures that a variety of questions can be pulled, increasing the probability that two students receive a different set of questions.

This strategy can mitigate test material leaks by ensuring that each test has some degree of uniqueness.

Implicit in these strategies is that the faculty are at least trained on the issues regarding academic integrity in the online environment and that they are proactive in addressing them in their classes. Equally important is the support that learning institutions must provide by enforcing school­wide policies about cheating to students and online pedagogy to faculty. Ensuring online test security is a multi­dimensional effort that can be transferred between course delivery formats.

Survey Report Description of pilot In an effort to assess the effectiveness of combining available technology and sound teaching strategies, the SDSU Instructional Technology Services, in partnership with the Love Library, and Dr. McGuire’s Biology 480: Clinical Hematology course, is piloting a new exam delivery method.

The participating class required 4 exams, each ~11% of the course total, and a final exam worth ~18% of the total. Several students (5%) were unable to attend campus exams due to distance opted to pay for 3rd party proctoring services. Some students (numbering between 1 and 3 for each exam) made special arrangements to take paper forms of the exam in the instructor’s office.

Exams were delivered in a computer lab inside SDSU’s Love Library. Tests were created using Blackboard and delivered using Respondus LockDown Browser, which blocked the test taker’s access to online material and other computer functions, restricting the availability of information to which the student had access.

Blackboard Test Setup ● Displayed one question at a time ● Randomized question ordering ● Randomized answer ordering ● Instant score feedback ● Withheld feedback for individual question items Respondus LockDown Browser ● Disabled Access to online resources outside of SDSU BB ● Disabled OS functions (screen shots, copy/paste) 10

Purpose of survey A survey including 20 multiple choice and open answer items was offered to students participating in this pilot study for an additional 2 points of extra credit added to their entire course grade. This survey was designed to assess student opinions on the frequency and prevalence of academic dishonesty both within this pilot program, and in general. The instrument also listed several items pertaining to student opinions of online class formats.

Population The students who participated in this survey were enrolled in Biology 480: Clinical Hematology. Of a total enrollment of 123 students, 104 participated in this survey, resulting in a response rate of 85%. This rate decreased somewhat on the open ended answer section. The high response rate was likely the result of the extra credit offered by the professor. While the actual impact of 2 extra credit points represented less than one half of one percent of the class total, they were highly valued by the students. Perhaps this is a testament to the high stakes nature of this particular course.

BIO 480 is an elective class for the Biology undergraduate major (30% of respondents), a required class for San Diego State University’s pre­med program (24%), as well as a requirement for a California state licensure to perform blood work in a laboratory (38%). We can reasonably assume that the majority of students taking Bio 480 for state licensure were non­matriculated students, coming from the college of extended studies. The majority of students reported a current course load of more than 6 units (64%), while others (34%) were taking only this course. The majority of respondents (84%) reported an age range between 18 and 29 years old. The majority of students were female (65%).

Method of Dissemination The survey was disseminated through use of Blackboard’s anonymous survey tool and offered 2 extra credit points for their participation. The Blackboard survey tool tracks individual completions of the survey, but allows no connection between actual answers and participants. This instrument allowed surveyors to award extra credit points without jeopardizing anonymity.

Discussion of Results Course Quality Q: In general do you think online classes are easier than traditional on campus classes?(n=103) 82% of respondents do not believe that online classes are easier than traditional face to face classes. If online classes are thought of as “easier”, it could indicate that large numbers of students are cheating.

The large majority of students believe that online classes are not easier than on­campus classes. This has implications in both the perceived rigor of online classes themselves, but also gives a glimpse into the 11

motivation to cheat.

Q: Do you find it easier to communicate with other students while taking an online class?(n=104) 73% of respondents believe that communication with other students is not easier in an online class. 26% believed that communication difficulty was about the same.

We might find that students who have an easier time communicating online through discussion boards or through student driven social media groups might be more likely to engage in collaborative cheating, due to the many opportunities for anonymity.

Q:Do you find it easier to communicate with your professor while taking an online class?(n=104) 43% of respondents believed that it was not easier to communicate with their professors in online formats, 40% believed that it was about the same.

Perceived instructor presence has been identified as a factor in a student’s decision to cheat. If a student feels distanced from a professor, if that instructor exists as only a “talking head”, then a student is more likely to engage in academic dishonesty.

Test Quality Q: Would you prefer to take tests[online or on campus]: (n=104) 49% of respondents preferred to take their tests online, while 38% preferred to take their test on campus.

This question asks participants to identify the method of testing they most prefer. The majority of students prefer to take their tests in a fully online format, allowing for the highest convenience. A significant number of students prefer taking their courses on campus, possibly having had unpleasant past experiences with the technology required to take an online test.

Q: Are students are more likely to cheat on an online exam? (n=104) 60% of respondents believed students are more likely to cheat on an online exam than they are on a traditionally proctored face­to­face exam.

The majority of students believe their peers are more likely to cheat on an online exam, while a significant minority believes the format to be secure. This question is important for establishing norms. If a student’s perception is that their peers are cheating, then they may be likely to cheat themselves. Though students answering “no” could have possibly meant that they perceive the frequency of cheating to be equal between the two formats. This question should be modified with an “about the same” option.

Q: Are students less likely to cheat in a proctored environment? (n=104) 12

81% believed students are less likely to cheat in a proctored environment.

These answers conflict somewhat with the data from question #10. It is surprising that such a large number of students believe that their peers are so bold as to completely ignore the presence of a proctor when deciding whether or not t cheat on an exam. Again this question could benefit from an “about the same” option, as the “no” answers could be split between less likely/about the same.

Q: If an instructor indicated a test will be taken at home online but without the use of notes, textbook, or the internet do you think will most students honor those instructions?(n=104) 78% believed that most students would ignore instructions if they were held only to an honor code.

The majority of students believe that their peers will use whatever resources they have available when taking an unsupervised exam. It would be realistic for instructors to assume the same.

Q: In taking classes online at SDSU I have encountered opportunities to cheat (n=104) Never Rarely Sometimes Often Very Often

53% 22% 18% 6% 1%

There seems to be a gap between the perception of cheating frequency and the actual opportunity to cheat. Students find that their peers are exploiting opportunities they do not see themselves. It could be that fears about academic honesty are overblown, or it could be that students are less likely to admit to any personal practices of academic honesty.

Q: What are the top three (3) motivating factors that lead to cheating in an online exam?(n=104) To get a good grade Because there is a high pressure to perform Not enough time to study Because there is a low risk of being caught Successfully cheating in a previous exam Feeling disconnected from the teacher Other

57% 13% 11% 7% 2% 1% 9%

This question examines the main motivating factors in a student’s decision to cheat. The majority of students believe that students cheat simply to get a good grade, and not because of any particular circumstances surrounding a course. It should, however, be noted that this particular item was flawed in 13

that it did not allow the survey participant to choose more than one option.

Q: Does the instant test score feedback you receive upon completing these exams make the Blackboard test format more attractive to you than scantron or written exam formats?(n=104) 86% of respondents indicated that instant score feedback made the pilot’s exam delivery format preferable to Scantron exams.

One of the main benefits of utilizing the Blackboard exam delivery is the ability to give students instant feedback on their scores. The vast majority of students indicated that they value this feature, so much so that given a choice between methods, they would choose Blackboard over Scantrons.

Open ended questions Q: Have you witnessed any specific instances of academic dishonesty or cheating in this class?

Please note that this survey is absolutely anonymous and answers will in no way affect your grade or academic standing. (n=91)

No – 69 No, but have heard rumors­2 Yes (on assignments)­ 3 Yes (on Exam)­9 Yes (unclear where)­5

Yes (Specifics) ● Whispering­2 ● Sharing scratch paper­ 2 ● Communication between test groups­ 5 ● Looking at other screens­ 2 ● Access to previous course materials­ 4 ● Use of smartphone­ 1

The majority of students witnessed no cheating on exams. There may have been some confusion on what constituted “an exam”, as some confused other online course assignments with the proctored exams taken in this pilot. The instructor was fully aware of the potential to cheat and the availability of assignments from previous courses.

As far as instances of cheating, which can be directly linked to exams within this pilot, the majority of issues seem to stem from communication between the two testing groups. This vulnerability should be planned for in the future, either through the use of separate tests, or the simultaneous proctoring of the whole class.

Q: What is your opinion of this test taking experience, specifically taking a Blackboard exam on campus? (n=96) 20 expressed that they had liked the pilot method 14

19 expressed that they had found the method to be effective as a deterrent to cheating 12 expressed that they had no problem with it 11 disliked that the exams were on Saturdays 11 appreciated the instant score feedback 11 worried about technical issues with blackboard 10 found the setup to be inconvenient 7 found the environment to be distracting 6 described the process as smooth 2 stated that they had disliked the pilot and did not elaborate Many students appreciated the pilot, but those who didn’t cited reasons pertaining to the location and timing of the proctored exam. 11 students did express some anxiety over the possibility of technical issues.

Q: Did you encounter any logistical difficulties with this test taking format? If so, please specify (n=91) 73 had no issues 4 had trouble with scheduling or getting to campus 8 had issues with technology There seems to be a gap here between the number of people having trouble scheduling or getting to campus, and the number of people in the previous question who disliked the Saturday test dates and described the setup as “inconvenient”. Either students were not actually burdened by the required Saturday presence on campus, or it could possibly be that there was some confusion over the meaning of “logistical”.

Q: Did you encounter any technical difficulties while starting or taking the test? If so, please specify. (n=94) 80 reported no technical issues 5 had their test freeze 2 had problems with the mouse tracking on Macs 2 had issues with their passwords 1 had browser issues 2 said that the test was “slow” The majority of students cited no technical issues. It was interesting that so few reported issues. Proctors reported several crashes requiring exam rests over the course of the pilot, and short browser freezes were reportedly common. This could simply indicate that students had a high tolerance for technical issues, having experienced many of them in the past.

Q: Do you have any additional comments? (appendix only, due to varied responses) 15

While this program is still in its pilot phase, results show promise. A number of instructors have shown interest in the pilot, and it’s clear that SDSU faculty members are experiencing academic dishonesty in their courses, and are open to new solutions.

Recommendations There is no “magic bullet” that will stop academic dishonesty. As previously mentioned, the problem is widespread. There are, however, a number of effective tools and strategies being successfully implemented at other universities and from which SDSU could benefit. To best serve the faculty at SDSU in this endeavor, the following actions are recommended: Continuation and expansion of pilot program More data is needed to assess the effectiveness of this method. Students and instructors alike seem to appreciate the format, but there are some variable issues with test design, equipment, and scheduling to be worked out. Perhaps the continued evaluation and research of this pilot can be offered to ED 795A and ED795B in following years.

Expansion of Faculty Training and Support programs The most sustainable and transferable method to mitigate cheating is proper pedagogical training.

Many of the skills learned in practicing good test design can be adapted to work in any class or test delivery method. Faculty training and support should be a top priority.

Large scale adoption of test pools to mitigate old test material availability Question pooling is one of the most effective deterrents to collaborative cheating. The uniqueness of each test minimizes the threat of students gaining access to test materials from previous courses, or sharing answers amongst themselves during an exam.

Use of Blackboard Assessment settings as a means to deter cheating Many learning management systems offer a wide array of options and settings which can be used to minimize academic dishonesty. These settings, however, require faculty training and support.

Use of Academic Honesty policy contracts on Blackboard exams Academic honesty policies are excellent ways to inform students of what constitutes cheating, and what is expected of them within the course. These policies can be added to the instructions of online assessments, requiring that students agree to their conditions before proceeding to exam questions.

Cultivate an environment of fairness and buy in from students There will always be a game of “cat and mouse” with the prevention of cheating on exams therefore it is important to cultivate an environment of mutual trust between students and professor. This should be talked about and instilled on the first day of class and throughout the semester. Additionally, every exam’s first “question” of which they get credit for should be something along the lines of, “I promise to help maintain the integrity and fairness of this test and 16

class by not cheating.” Hopefully peer pressure will take over and influence all students in the class not to cheat.

Promotion of Multiple assessment methods/low­stakes assessments The use of multiple types and frequent assessment are good examples of universal design principles which have the potential to foster student communication, and decrease motivation and opportunity to engage in academic dishonesty.

Further research and piloting of remote proctoring technology Remote proctoring software is somewhat new and largely untested by independent studies. It is unclear, at this time, which companies and service types will emerge as market leaders, and it is therefore unwise to commit to any particular service at this time. It is recommended that the ITS continue to watch these services as they evolve, and pilot them on a small scale.

Further research into new user authentication methods User authentication must receive special attention as it is mandated by the HEOA of 2008. Much like remote proctor services, these tools are very new. Further research and small scale pilot testing is recommended as these tools evolve.

Minor changes in the process could further enhance test security and the test­taking experience according to the difficulties expressed by the students in their feedback. For example, computer stations could be tested for functionality immediately before the exam to account for and fix any technical issues. Also, finding ways to better sequester the exam group from environmental distractions present in the library, as well from one another in between sessions, could alleviate some of the difficulties that students experienced and curtail the collusion efforts of those intending to cheat using information from the earlier group. Others that weren’t predicated in the study but, rather, through observations after the fact, are to increase the size of the question pool and to inform students before the registration process about the requirement of proctored testing. These are relatively easily to remedy and can eliminate some of the test­taking vulnerabilities that were present in the BIO 480 pilot. Once a process of campus proctoring is streamlined, it can be used by other online classes at SDSU as another means of delivering secure tests to students.

Conclusion With the cooperation of the instructor, students, library and ITS staff, the pilot program proved overall to be a viable alternative to administering exams in an online class. This project proved a valuable learning experience and lent well to the expected learning outcomes of the 795a/b course. The analysis of academic dishonesty and of the ITS’ pilot exam delivery method cultivated the following, most notable, Edtec competencies: Data­based decision­making: Researchers used a variety of data to formulate their recommendations. These data types include: faculty interviews, client interviews,a student survey featuring both quantitative and open answer items, faculty survey, a literature review, and an 17

ever­expanding pool of newspaper and journal articles.

Technical: This project required independent mastery of a number of instructor­side Blackboard features, including an anonymous survey tool, grade center, and a number of settings and options governing the presentation of test items.

Interpersonal/Character: The research­heavy nature of this project required independent work, with researchers coming together to share their data sources. This work model mandated effective task distributions, acknowledgement of personal and group work styles, strengths and weaknesses, and task preferences.

Professional & Career: All three researchers are currently working in the field of higher education. This project gave deep insight into the inner workings of an institution of higher education, the flow of resources, and pragmatic approaches researching and implementing cutting edge trends and technologies.

Academic dishonesty is a complex issue brought into greater focus by the unchecked growth of online course delivery and the rise of MOOCS. While the problem is deeply rooted within the culture of academia, there are a great many things that can be done to lessen student motivation to cheat, and to hamper the effectiveness of dishonest activity. In the face of high controversy, media attention, increasing funds and resources for the development of online instruction, an opportunity has arisen for instructional designers to tackle this issue by promoting innovative technologies and solid universal course design. If properly approached, the solutions to academic dishonesty have the potential to change all higher education instruction for the better by allowing for more accurate evaluation of student knowledge and learning gains.

18

References Adkins, J., Kenkel, C., & Lo Lim, C., (2005). Deterrents to Online Academic Dishonesty. the Journal of Learning in Higher Education, 1(1), 17­22. Retrieved March 28, 2013 from http://jwpress.com/JLHE/Issues/v1i1/Deterrents%20to%20Online%20Academic%20Dishonesty.p df

Allen, I., & Seaman, J. (2011). Going the distance: Online education in the united states, 2011. Babson Survey Research Group and Quahog Research G. Bergstein, B. (2013). Online exams: Big brother is watching you. Technology Review, 116, 68­68. Retrieved from http://search.proquest.com/docview/1269702304?accountid=13758 Cochran, L., Troboy, L., & Cole, T. (2010). A Test of Integrity: Remote Proctoring in an Online Class. Journal of Business Administration Online, 9(2), 1­9.

Eisenberg, A. (2013, Mar 02). Keeping an Eye on Online Test­Takers. The New York Times.

Retrieved from http://www.nytimes.com/2013/03/03/technology/new­technologies­aim­to­foil­online­course­cheati ng.html Foster, A. (2008). New Systems Keep a Close Eye on Online Students at Home. Chronicle of Higher Education, 54(46), 45­47.

Gardner, L. & Young, J. (2013, Mar 14). California's Move Toward MOOCs Sends Shock Waves, but Key Questions Remain Unanswered. Retrieved from http://chronicle.com/article/A­Bold­Move­Toward­MOOCs­Sends/137903/ 19

Howlett, B., & Hewett, B. (2006). Securing and Proctoring Online Tests. In M. Hricko & S. Howell (eds.), Online Assessment and Measurement: Foundations and Challenges. Hershey, PA. Information Science Publishing.

Leybourne, S., Kanabar, V., & Warburton, R. (2011). Developing and teaching of a world­class online project management curriculum. Journal of Project, Program & Portfolio Management, (2)2, 1­19. Retrieved from http://adminsc.com/wp­content/uploads/2012/01/2231­10123­1­PB1­Article­1­Leybourne­et­al.pdf Moore, G. (2010, May 12). ProctorCam Betting On e­Learning Upswing. Retrieved from http://www.bizjournals.com/boston/blog/mass­high­tech/2010/05/proctorcam­betting­on­e­learning­ upswing.html Prince, D. , Fulton, R. , & Garsombke, T. (2009). Comparisons of proctored versus non­proctored testing strategies in graduate distance education curriculum. Journal of College Teaching & Learning, 6(7), 51­62.

Roh, K. , & Park, H. (2010). A meta­analysis on the effectiveness of computer­based education in nursing. Healthcare Informatics Research, 16(3), 149­157.

Schellenberg, S. (2012). Exams and test taking in the digital age. One Day in May. Lecture conducted from San Diego State University, San Diego.

Street, J. (2008). Examining the Validity of Testing in an Online Learning Environment. n.p.: ProQuest, UMI Dissertations Publishing.

WCET. (2009). Best Practice Strategies to Promote Academic Integrity in Online Education (Version 2.0).

Wehrner, M.J. 2010. A Comparison of the Performance of Online versus Traditional On­Campus Earth Science Students on Identical Exams. Journal of Geoscience Education 58:5, 310­312.

20

Appendix Complete survey https://docs.google.com/document/d/1A3E7YWkBWkjHfzVrRJHhFa_ZQHteofeG8WB1AqKL0 wg/edit?usp=sharing Complete survey data https://docs.google.com/spreadsheet/ccc?key=0Ak3BNygKa_IOdGRLaEVPSm9CMXFNbjlqWE lzMEVlanc&usp=sharing Qualitative data analysis https://docs.google.com/spreadsheet/ccc?key=0AuobwCQBOoiLdEw1dFVqLWRmV1U4RGc0U lJSLXVLQ3c&usp=sharing Participating class’ syllabus https://docs.google.com/document/d/1wCXEQIxlqyZX16fqkgT64FsFSIQnRfyEZ8_7SNnuoQM/e dit?usp=sharing University policy on academic dishonesty­ Center for Student Rights and Responsibilities http://csrr.sdsu.edu/cheating­plagiarism.html

21