Thwarting Misconduct By Item Pool Flooding
What if the person securing your bank’s computer systems cheated on their certification exam in order to get their job? What would you think if you found out that the person managing your company’s confidential employee data cheated on their certification exam? Or, how would you feel if you found out that you were declined a career opportunity because someone who is less qualified competed for the job and benefits that you deserved? These are very real scenarios that impact our everyday world globally. When we know that cheating occurs, and we know how it occurs, we must take steps to prevent it from resulting in granting valued credentials to unqualified candidates.
This article describes just a few of the test delivery approaches that Symantec takes in order to protect the credibility of its global IT certification program. In order to provide context for the approaches, this article will briefly define the activities that lead to exam piracy and candidate misconduct. The central point of the article focuses on hampering candidate misconduct as a result of exam piracy through item pool flooding. It should be known that it is very challenging, if not impossible, to accurately measure the success of item pool flooding. However, the time and cost to implement this approach and potential outcome outweighs knowing a precise metric of success.
Failure to Adequately Prepare
A Symantec customer certification survey in September 2011 sent to over 24,000 people indicated that 28.5 percent of candidates have used sample or practice exams from the Internet. The vast majority of those sample exams are in fact unauthorized by Symantec and are sold on infringement websites (a.k.a., braindumps). The braindump websites reproduce exam materials by any means, including reconstruction through memorization. Some candidates seek to “short cut” proper exam preparation and experience by attempting to memorize unauthorized content from braindumps. Symantec communicates that the only authorized preparation materials for Symantec exams are those made available by Symantec or an authorized Symantec partner. Nonetheless, there are candidates who are unaware of the proper Symantec exam preparation materials, truly believe the braindumps offer legitimate ways to “prepare” for exams, or simply don’t want to make the investment of time to study and gain experience.
Activities to Prevent Misconduct
There are several test delivery approaches to reduce cheating. This article assumes that each program takes actions to minimize the number of people who have access to item banks by limiting access to only those who must be involved in the exam development and delivery process. Other approaches to ensuring exams remain confidential include techniques such as publishing multiple forms with randomization to decrease memorization attempts for exam retakes. Some programs conduct regular exam maintenance and use statistical analysis to flag signs of misconduct through noticing anomalies in raw exam results.
Legal counsel is also used to work with payment processors to cease providing services to braindumps, as well as shut down infringement websites which have been proven to have stolen intellectual property from certification programs. The biggest challenge to addressing exam piracy is a legal challenge that typically crosses international borders. Some companies and their legal counsels work together with industry associations such as the Association of Test Publishers Security Committee (ATPSC) and Software & Information Industry Association (SIIA) to battle exam piracy. This type of cross-company and cross-legal collaboration strengthens efforts to combat the piracy of exam and testing materials offered through online infringement websites.
Working actively and directly with test delivery vendors is critical to identifying and implementing proper software, network, physical, application, and operational security of exams. Security should be an ongoing core competency for a program’s test delivery vendor(s). Still, it can be challenging to pinpoint where exam piracy occurs and identify the individual(s) responsible for exam exposure. What we do know is that exam source code published to a test delivery vendor is illegally hacked and stolen from a test center server through a single binary file that includes questions, answers, and specifications for how to deliver and score each item.
Item Pool Flooding
The purpose of item pool flooding is to hinder the candidate who wants to circumvent the process of studying and gaining real world experience by purchasing and memorizing unauthorized exam materials from braindumps. The process simply consists of publishing two or more legacy and retired decoy exams while only delivering, scoring and communicating out one exam - the intended and new exam. For example, if you intend to publish an exam testing the Administration of Software 2012, then you would also publish a completely different exam testing How to Sell Hardware 2009 (i.e., the decoy exam) under the exam name of Administration of Software 2012. However, you will instruct the test delivery vendor to only deliver questions on the exam testing Administration of Software 2012. Candidates taking the exam will only see questions testing Administration of Software 2012. But, candidates who purchase Administration of Software 2012 from a braindump will see double the amount of questions since the braindump file stolen from the test delivery vendor includes a binary file with questions from Administration of Software 2012 and How to Sell Hardware 2009. Therefore, candidates attempting to memorize a braindump will have to decipher which questions actually belong on the exam that they purchased (e.g., Administration of Software 2012 using this example). They will realize that they have purchased “bad exam data”, which will make it much more challenging to figure out which questions to memorize and which questions are completely irrelevant, or decoy questions. Some candidates might even attempt to get their money back from braindumps. Finally, Symantec has experienced that braindumps don’t actually review the stolen item banks in order to realize that they’ve also stolen a flooded exam with a decoy item bank.
Symantec advises to use a decoy item bank that’s from a prior version of an exam that has already been retired and translated. If a program localizes its exam content, program administrators must ensure the decoy item bank is also translated. If a program selects a decoy item bank that hasn’t been translated, program administrators will need to account for additional time and cost to localize the decoy items so that it’s not obvious to candidates who purchase braindumps that two or more different exams were published as one exam.
If a program wants to maintain the integrity of its brand and certification exam credentials, they must evolve test delivery approaches to hinder a candidate’s ability to cheat. As a result of test delivery approaches, programs should also consider actively investigating and taking corrective action against individuals and organizations who attempt to compromise the security of exams or engage in any form of exam misconduct. Implementing item pool flooding within exam delivery processes will force candidates to actually study and gain experience before attempting an exam. Hopefully, another positive side effect will be that fewer candidates receive credentials unless they are truly knowledgeable and competent.
Beverly van de Velde is a senior manager, global technical certification, at Symantec Corporation.
Written for TrainingIndustry.com