The unbearable unreliability of complexity

This is a text automatically translated from Italian. If you appreciate our work and if you like reading it in your language, consider a donation to allow us to continue doing it and improving it.

The articles of Cassandra Crossing I'm under license CC BY-SA 4.0 | Cassandra Crossing is a column created by Marco Calamari with the "nom de plume" of Cassandra, born in 2005.

Let's continue the saga of unsustainable stories. If you are interested there is a list of all of these stories also recently published by Cassandra.

This article was written on January 8, 2018 from Cassandra

Cassandra Crossing 424/ The unbearable unreliability of complexity

A well-documented “I told you so” by Cassandra; Excess complexity can catastrophically compromise our connected world.

The vastness of the problems revealed by the bug of the Intel and AMD CPUs (but not of the ARM7!) partly surprised even Cassandra, who had already expressed her opinion on the topic of the useless and dangerous complexity of IT here And here a decade ago.

Few have realized it, but the bug found in most existing silicon in the world is of a particularly benign type, given that it "only" compromises the confidentiality of the data processed by the majority of existing CPUs, and "only" if an attacker has achieved local access to the computer.

What takes the breath away of those who have ears to hear and imagination to extrapolate the future is the vastness and danger of this class of problems, no longer theoretical or hypothesized but demonstrated.

An example for everyone; to transform a global but limited problem such as Metdown in a nightmare, would it be enough to find a way not to read but to corrupt the kernel memory, thus crashing the computer?

What power would one have cyber weapon contain such a “silicon zero-day” if it could be exploited for crash all CPUs of an enemy country, or perhaps of the world?

It would be enough to use it using an existing botnet as a vector, or a rapidly spreading malware such as the now ancient one SQL-slammer; the end of the enemy's cyberspace, or all of cyberspace, in one swift move.

The underlying problem, however, is even more worrying; modern society does increasing reliance on technology, without worrying even theoretically about the risks of known catastrophes when they manifest themselves with enormous dimensions but low probability, events called in the literature black swans (see Nassim Nicholas “The Black Swan: The Impact Of The Highly Improbable” 2008).

Nor does it worry about unknown and unpredictable, as well as enormous, in jargon catastrophes double black swans (see Maurizio Barbeschi and Paolo Mastrolilli — “Dealing with the unknown. Governing uncertainty: sudden epidemics, natural disasters, terrorist attacks” — 2016).

Yet the problem of the "silicon bug", although so publicized by the media, could even be counterproductive for an increase in awareness of these problems; in a few weeks, when the news will be forgotten from the media point of view, and nothing serious will have happened, everything will seem calm, and people, from simple users to large hardware producers, will start living as usual and doing business again. as usual”.

The global danger to guard against is not simple poorly designed silicon, the real problem is the continuous increase in the complexity of any technological object, which takes it beyond the increasingly limited understanding of its own designers.

This is aggravated by technological homogenization which pushes, for mainly economic reasons, towards the adoption of increasingly similar hardware and software platforms which, as in the case of the homogeneity of the genome of organic crops, can be a harbinger of catastrophes caused by a single technical problem, as if from a single resistant parasite. The management and limitation of complexity, combined with the control of excessive technological homogenization of infrastructures, particularly critical ones, is the only thing that can avoid technological double black swans.

We shouldn't be allowed to use a robot with a big foot as a doorstop just because it's cheap, just as we shouldn't use a PC to display a single letter on an LCD screen; both systems will fail sooner or later because they are too complex for the simple function they must provide.

But the failure in primary function, even if it were catastrophic, it would be “only” a black swan.

An unexpected technological failure, such as that of a hypothetical “silicon bug that resets CPUs” would be a double black swan, with both unexpected and unpredictable consequences, practically unlimited in the damage it could cause, especially if used voluntarily as a cyber-weapon.

It would be extremely appropriate for those who handle public money or manage the research and development budgets of technology producers to start employing those who deal with theory of catastrophes, Of Complexity analysis and other currently marginal branches of physics and systems theory to design less vulnerable and unstable technologies, and analyze the possible consequences of those already existing and widespread today.

And that anyone, every now and then, would stop for a moment in the frenetic life that we all lead and ask themselves "What if what I am using stopped working or even disappeared"? Starting with tap water, continuing with the silicon that now permeates our life and continuing with the announcements of the new gifts that future technologies will bring us... gifts perhaps made of wood and in the shape of a horse.

Marco Calamari

Join communities