Heartbleed bug crisis hit more government computers than previously disclosed

The infamous Heartbleed bug that forced the shutdown of Canada Revenue Agency computers last year had a wider impact on the security of government systems than previously disclosed, after a slow federal response gave hackers a two-day window to steal secrets.

The bug allowed hackers into vulnerable computers in federal departments dozens of times, leading to a series of information breaches, newly released documents show.

• What is Hearbleed? (2014)

The number of so-called "reconnaissance" events — in which attackers got into federal systems to assess their weaknesses — rose dramatically in the spring of 2014, to 91 detected intrusions. That was up from just seven in the fall of 2013.

And the number of actual information breaches rose sharply in the same period, increases that were "directly attributable to the OpenSSL 'Heartbleed' vulnerability," says a heavily censored document from the Public Safety Department.

A copy of the internal cyber-security newsletter, with the headline "Heartbleed event increases number of reconnaissance and breach events," was obtained by CBC News under the Access to Information Act following months of delay.

Previously, RCMP alleged only that a 19-year-old London, Ont., man had accessed hundreds of social insurance numbers by exploiting the bug in tax computers.

But the documents show Heartbleed-related attacks were much broader, involving other departments, though no information was provided on the number or identities of other attackers, or what data they may have stolen.

Shared Services Canada, which is in charge of most federal computer systems, declined to answer questions about the incidents. "We do not comment on breaches or potential threats," spokesman Ted Francis said.

Told to 'stay put'

A spokeswoman for Canada's privacy commissioner says the office has received no reports from departments about Heartbleed-related privacy breaches, apart from the single incident at Canada Revenue Agency.

Other internal documents suggest hackers took advantage of Ottawa's sluggish response.

The Canada Revenue Agency knew the bug had opened a "severe vulnerability" to its online tax systems by mid-afternoon on April 8, 2014, but didn't start shutting down its systems until three hours later.

That's because Shared Services Canada, which acts as the federal government's IT department, told the agency "to stay put, i.e., do not block portal," suggesting there would be more information the next day.

Nervous CRA officials, after consulting with private-sector experts, eventually ignored that advice, and unilaterally shut down all the agency's online tax systems in stages by 9:40 p.m., then began installing a patch.

Why is CRA the only one reacting?-–Tax official on the lack of action by other departments to the Heartbleed bug

But a directive ordering other Heartbleed-vulnerable government systems to shut down was sent only two days later.

"On 10 April, the chief information officer for the government of Canada issued a directive to all federal government departments to immediately disable public websites that are running unpatched OpenSSL (Heartbleed-vulnerable) software," says an internal chronology.

The slow response baffled revenue agency officials. "Trying to understand what other government departments are doing," said one of them on April 9, as other vulnerable systems remained exposed to hackers.

"Why is CRA the only one reacting?"

The broader effects of Heartbleed, and the questionable federal response, have been kept from public view for more than a year because of lengthy delays in the release of information requested under the Access to Information Act. CRA, for example, gave itself an 18-month extension for releasing key records.

The Heartbleed bug existed for up to two years before a public alert was issued on April 7, 2014, along with a patch to fix it. A limited number of people, firms and governments knew of the bug before that date, and some may have exploited it, but the alert itself triggered a wave of attacks on vulnerable computer systems in April 2014.

"This was like the proverbial dinner bell being rung all around the internet," said Patrick Malcolm, a computer-security consultant in Ottawa who sometimes trains RCMP officers.

Threat was real

Malcolm, founder of NetRunner Inc., applauded the Canada Revenue Agency for ignoring the bad advice it got from the recently created Shared Services Canada.

"It was very prudent of them to shut the whole system down," he said. "Shared Services Canada as a department is still on its fawn legs."

"The question I had as a technical security expert is: Why were more government departments allowed to stay open when this vulnerability was non-theoretical? The threat was real."

A government spokesman said federal officials made the right decisions.

"The government of Canada found out about the Heartbleed vulnerability at the same time as the rest of the world, and took quick and public measures," said Michael Gosselin of the Treasury Board Secretariat.

On April 9, "Departments performed risk assessments on their systems and evaluated their exposure to the vulnerability. There was a continuous sharing of mitigation advice … as departments and agencies accelerated their patching efforts," he said.

"When it was confirmed on April 10 that there had been a data breach at CRA, the Treasury Board Secretariat immediately asked departmental [chief information officers] to place all vulnerable public-facing sites offline."

Follow @DeanBeeby on Twitter