Tony Thorne was one of the Apollo project officers, advising the team on the task of merging large amounts of data. Thorne, a former counter-terrorism officer with Wales’ extremism and counter-terrorism unit, said he was shocked by what he saw in Scotland. “We have left Scotland with a process that is by no means complete or adequate,” he said.
Key issues highlighted during the test run resurfaced quickly, according to emails and internal memos from 2014 and 2015 reviewed by BuzzFeed News.
Officials described the system as “routinely crashing” and “timeing out after 10 minutes,” adding that the glitches were so severe that they dramatically increased the “time required to perform a simple task.”
Even basic searches cause trouble. One officer described how he put in a search term and got a very broad set of results. He began sifting through the documents manually to find the ones he actually needed – but as he did so, the system crashed. When he logged back in, he typed the same search terms and found that “the search result was not the same.”
Officers using the new systems have reported serious difficulties with the problem the NCIA aims to solve: communication with other forces and agencies. After a suspicious person entered the UK by plane, an official reported that a vital intelligence report was received in an unreadable format by officials at the airport. Another told Apollo team members that the NCIA’s inability to share intelligence with other regions still using the old system was a critical risk “that could lead to an intelligence failure.”
The quality of intelligence produced on the system was generally poor. In some cases, the NCIA was overwhelmed by irrelevant information; Among others, the NCIA did not show important intelligence. One official complained that the system “auto ingests” documents that have nothing to do with terrorism. “This issue has always been a talking point,” the official wrote, “although not much has been done about it even though we are now live.”
The NCIA will be built on the template of an already existing system, the Home Office Large Major Inquiries System, four sources told BuzzFeed News. The problem is that HOLMES is used to investigate events that have already occurred but the NCIA intends to prevent attacks from happening. Another official told BuzzFeed News that building the NCIA on top of the HOLMES system created flaws that made it difficult to find large amounts of intelligence.
Officials have echoed these concerns in their emails and official reports. A key feature borrowed from the HOLMES system is a Google-like search tool that enables officials to quickly retrieve records containing a specific term, regardless of where the term appears in the record. If it works, it will be easy to find specific intelligence about potential terrorists from hundreds of thousands of files.
But the search tool is not working. Officials found that if they entered the same search term in multiple cases, they would get a different result each time. The search tool could not scan birth dates, making it difficult to identify the correct document.
This deficiency led to another major problem. Early on, it became clear that many duplicate documents would make their way to the NCIA – because it was compiling data from multiple forces that had a single file on a particular person. An internal report seen by BuzzFeed News acknowledges that there is a “knock on” effect that disrupts analysts. But the higher-ups finally decided that “no duplication would occur” until the whole of the UK used the NCIA.
A Manchester-based official who later began using the NCIA told BuzzFeed News that the duplicates find what you’re looking for “like trying to find a needle in a haystack” — such a struggle that “you can lose vital intelligence.”
Thorne, a terrorist detective who worked for the NCIA, was increasingly concerned. “Unfortunately,” he wrote to colleagues in a February 2014 email, “as we fully know the NCIA has struggled to deliver what it promised and is not fit for purpose.”
The rollout of NCIA is pressing ahead.
