AI governance: Invisible risks in unchartered territory

Organisations are not ready for the challenges that AI brings but they can’t avoid them, according to Sir Jeremy Fleming, the former Director of GCHQ, the UK’s intelligence, security and cyber agency.  

“If the question is, are we ready, the answer is no,” Fleming told an audience at S&W’s invite-only event last week with AI Score, the AI governance and risk management platform. “With some honourable exceptions, organisations are really struggling to keep up with the pace of change.” 

He was speaking as part of a panel that also included Colin Bell, Chief Compliance Officer for HSBC Group, and Jonathan Kewley, Global Head of Technology Practice at Clifford Chance, one of the world’s biggest law firms.

The event was moderated by Josh Burch, another veteran of national security roles and now Cofounder of Gallos Tech, an early-stage funder of security, defence and resilience technology companies.  

Amid AI revolutions, companies needed to “re-earn the right to operate”, said Fleming. This required effective governance that, in turn, must start with a proper understanding of where AI is being used in organisations. And that’s the first problem, as Kewley explained. 

“You can have all the policies and legal documents you want, but if you don’t have proper control of this technology, there is an issue,” he said.  

One key “red flag” is the inability of those responsible to answer simple questions on how AI tools are applied and the data being accessed, Kewley added.  

As to who should be responsible, which is where Burch next moved the conversation, Bell thought the answer was straightforward.  

“If I’m a business leader, I own the risk of delivering that business to customers,” he said. That’s why it’s crucial that business leaders use AI in their personal lives, he added, so that they grow comfortable with and better understand the technology.  

The central tension those responsible for AI must manage is “between risk and speed of adoption”, he said. No company can afford to be left behind but must manage the risks in a fast-developing environment.  

“Often, you’re looking to loosen the organisation up to adopt technology at pace,” he said. “From a governance perspective, a culture of innovation, within defined guardrails, is the ultimate outcome you’re aiming for.” 

As they try to get there, the panel agreed there was limited help from governments.  

“For any other technology change you would expect government to be playing a much stronger role than it has with AI,” Fleming remarked.  

Despite that, regulation is beginning to come through. Many international groups are adopting compliance with the EU’s AI Act as a standard, observed Kewley, who noted that even in jurisdictions such as the US, that ostensibly leave the technology unregulated, it is rarely that simple.  

“There is a fracture between the EU and Europe, but it’s not as strong as you might think,” he said. “From a state perspective, they have some of the most vigorous AI legislation in the world. You shouldn’t think that if you’re in the US it’s a free for all.” 

Even in the UK, while there’s no specific AI legislation, there are plenty of existing data, privacy and other laws that touch on the technology.  

“Don’t think you’re not regulated here, because you are,” Kewley warned.  

But whatever regulation there is, it cannot answer the big questions AI raises about its use, risks and governance. It remains up to boards and business leaders to determine their approach, Fleming reiterated, and as the technology proliferates, the time to define this is running out.  

“The responsibility falls very firmly on you,” Fleming warned. “No one should be waiting to be told what to do.”