When the Australian Securities and Investment Commission (ASIC) tested its in-house developed fraud detection software, it was so effective it nabbed a fraudster.
The software was jointly developed with the Capital Markets Co-operative Research Centre (CRC) at a cost of A$2 million (US$1.5 million). Full details of how the detection tool was developed will be disclosed Thursday at the South East Asian Regional Computer Confederation (SEARCC 05) by Professor John Patrick University of Sydney.
Dubbed Scamseek, the tool replaces a keyboard-intensive process ASIC previously used to collate and classify dodgy Web sites.
It is 300 times more likely to find suspicious Web sites than a human and replaces a manual, periodic search which 20 people conducted, each surfing online four hours a day.
Speaking at SEARCC 05 Wednesday, Keith Inman, ASIC electronic enforcement unit director, said they used to eyeball 1000 Web sites, 100 of which required scrutiny to find 10 that required "contravention".
For a Scamseek search of an hour, the commission scans a base of 40,000 dodgy Web sites.
"We halved the number of hours to batch process and now typically scan 40,000 documents; usually about 2000 have scam features and 257 will probably be of interest to us in some way," Inman said.
"It saves 1900 hours of work and finds sites 300 times more likely to be suspicious. In fact, the first time we turned it on we actually discovered an entity that we are now taking action against."
The project was conducted in two phases over a 15-month period and involved two computer linguists and three software engineers from Macquarie and Sydney universities.
The first phase, to produce a system that classifies and retrieves HTML pages, took six months.
The second phase took nine months to improve the accuracy and create trawling programs and classifiers for other delivery channels.
Inman said the benchmark for phase one was 50 percent accuracy, a big improvement on the manual process which had a success rate of 1 percent.
"In the first phase, we had an independent audit conducted using trial documents ... we ran the test system across that and found 75 percent accuracy," he said.
"Nine months later we tested again and accuracy results had improved again."
IDG is the official organizer and media sponsor of the SEARCC 05 conference.