By Rick Gregory
for the IT Training Community
Microsoft is rolling out a series of new measures designed to improve security for certification exam development, delivery and administration. The most attention grabbing item is an across the board, lifetime ban from Microsoft certification programs for anyone caught cheating on an exam.
"We used to have a one year ban for some things and a lifetime ban for others," said Peggy Crowley, Anti-Piracy Program Manager at Microsoft Learning. "We decided that cheating is cheating across the board, so we decided to impose a lifetime ban for all forms of cheating," she said.
Microsoft is updating the Non Disclosure Agreement (NDA) that all candidates must sign before taking an exam. The NDA requires the candidate to keep any information about exam content confidential, and the new version will be much more informative than the existing agreement. "We felt that the old one might have been a little too vague, so we're going to be very explicit in the new NDA," said Crowley.
Microsoft plans to debut a new website this summer that will have information about the NDA and what you can and cannot reveal about an exam. Crowley pointed anyone with questions about what can be discussed to the Microsoft prep guides for the exams which have detailed information about what is included on the tests. "My overall guidance is that if you think it's questionable to talk about something you shouldn't," she said.
Microsoft has also revised several policies governing its certification programs, which they also plan to post on the new web site.
Microsoft's exam retake policy used to allow an unsuccessful candidate to retake the exam at any time. The exam retake policy has changed to impose a 24 hour delay before an unsuccessful candidate can retake the exam. If a candidate fails again, there is a two week waiting period before he can retake the test, and candidates are limited to no more than five attempts in a twelve month period. To take the exam more than five times over twelve months, a candidate must seek prior permission from Microsoft and provide an account of any extenuating circumstances that would justify an additional attempt.
The policy on testing center violations has also been toughened. If a test center is involved in fraud, they are banned for three years. They can reapply after three years, but the center is required to have a new security plan approved by the Microsoft Learning security team. The new feature here is that Microsoft reserves the right to close any center related to the offending center if it determines they may have problems as well. For example if the owner of an offending center owned multiple testing sites, all of them could be closed.
Microsoft is also implementing an out of country testing policy and a geo blocking policy.
Out of country testing is currently limited to India and China. Any non citizen must obtain prior permission to test there and must provide proper credentials and proof that they are in country to take the exam. Microsoft conducts a monthly audit of the program and if they can't determine an exam was properly documented, they contact the candidate and request proof he was in the country at the time of the exam and that he presented proper identification. If he can't he is subject to the lifetime ban and the testing center is closed.
Geo blocking covers test suspension at a local test center, an entire country or a larger region if Microsoft determines there is a security problem. They have currently suspended delivering beta exams to China, India and Pakistan because of security problems.
Microsoft uses a number of techniques to investigate suspected exam fraud, including lead tracking, suspect test purchases, secret shoppers, automated internet and auction monitoring, and data forensics.
Microsoft tracks leads through its tctips@microsoft.com alias. "We get leads from everybody you can think of," said Crowley, "It's one of the greatest tools that we have." Microsoft adds all the leads to a database with any supporting information like the type of lead, who sent it, how many leads for individual targets, etc. "We can track relationships between leads and actions within a lead. It helps us understand fraudulent activity," she said.
One major new change to the program is the extension of data forensics to enforcement. Microsoft has always used statistical analysis and data forensics to monitor the examination process but starting this summer, forensic results can be the sole basis of enforcement actions.
Data forensics is a sophisticated analysis of exam data to detect patterns that indicate test fraud, including cheating and piracy. "Any time you take a test, you leave data behind," Crowley said. "We can look at the time to take the test, any unusual response times or aberrants in the responses," she said. "One thing we can tell with data forensics is if people have been using brain dumps, so we will be able to enforce on that going forward."
Microsoft is implementing a policy of using forensic results as sole evidence for enforcement because they have determined that the data is scientifically sound to a one in one trillion chance that a certain result occurred by chance. "When we start the enforcement program, our baseline is going to be much higher," said Crowley. "It is going to be a trillion cubed that the results happened by chance."
Within the exams, Microsoft is moving to more performance based testing in which candidates perform real world tasks on simulated or live systems. The first generation of performance based exams were simulations in the 290 exam series. The new generation of performance based exams will be emulations performed on virtual labs. The latest generation of exams are in beta testing and will be released soon.
Microsoft is gearing up an educational outreach program along with the new security measures. "We want to educate people on what to be on the lookout for," said Crowley, "What kind of piracy is out there, what it is, how to stay away from it and the ramifications of what happens when people get caught up in it."
The new website will be the centerpiece of the education program. It will have all of the new NDA, updated policies, guidelines about brain dump sites, information on effective and allowable study practices and more. "We're going to be doing many more outreach efforts to get in touch with the community," Crowley said. "We feel that this is going to benefit our program and benefit the integrity of our certifications going forward."
© 2008 Bluestone Media, LLC
