The good, the bad and ... ⇡

There are two prevailing views on technology. One sentiment, perhaps the more popular these days, is that technology is the devil itself and one day will destroy the planet. The other, diminishing view asserts that technology is inherently good, the danger is only in the way we are using it, due to greed or recklessness.

In this essay, we are going to argue that both of these views are wrong, and also dangerous. We need a new view on technology. Well, in fact it isn't new, but did not make its way to the public sentiment yet. We need to fix that, the sooner the better.

Riding a horse ⇡

Have you ever thinked about how crazy it is to sit on the back of an animal weighing half a ton, yet strong enough to run and jump? Controlling a horse is an ancient science, and horses are selectively bred to be controllable. Yet, even after 5000 years of continuous refinement, accidents still happen. Maybe it is time to give it up, isn't it? Maybe our ancestors shouldn't have tried in the first place, given how crazy it is.

But this analysis misses a key point in the story. We want to ride horses. Since prehistoric times, horses helped in agriculture, war and travel. Today, it is merely a spare time activity, but that's still a value. Anyone climbing on a horse thinks that the benefits outweigh the risks. And most of the time they have to be right in their assessment, exactly because we are doing it for 5000 years, during which period it spread all over the world. Apparently, horse riding was a very good idea, a worthy invention that stayed with us thoughout the majority of human history.

Can we fit these ideas in our heads at the same time? Can we marvel at how crazy it is to attempt to tame such a beast, yet also how reasonable. Doing crazy things jumpstarted the rise of Mankind, and fuels its ascension ever since.

...the ugly ⇡

The horse was just the beginning. We never dismounted the horse because we found something safer. We dismounted the horse because we found something bigger. As in, way bigger. We are sitting on the back of machines of unimaginable size and power. We are sitting on the back of a society that is so complex nobody has any hope to grasp. We are moving so much energy and resources that it changes the face of the planet once and for all. We drove uncountable species to extinction, we changed landscapes, and then changed again, and just recently we, arguably, change the climate.

It is mindbending to try to imagine the power of our tools. Something as simple as a car is way more capable of causing mayhem than a horse ever could have been, and we have over a billion of them. Cars are small small players in our game. We have trains, cargo ships, excavators and rockets that make cars look like bugs in comparison. We have built dams, dikes, tunnels, and rivers. We literally moved mountains, built islands in the sea. We generate electricity in the terawatt range, our power plants could boil small lakes.

We are sitting on the back of a dragon with a million heads. And we have a good reason to. Just like jumping onto a horse made good sense, mounting this dragon makes even more sense. The amount of wealth it creates is immense. Our memories are short, we tend not to remember how much we lacked many things some try to claim to be "human rights" today, like healthcare or mobile phones. Nobody even imagined anything like these before the dragon was tamed. But of course, this also comes with much greater responsibility.

Risk, cost, benefit, accidents ⇡

So how do we go about it? How do we prepare ourselves to ride a dragon? How do we make sure it does not turn on us, and burns everything in one prolonged rage? It is all too ubiquitos to hear worrying voices, lamenting about human recklessness, how we play with fire and endanger our entire species, or even the planet. Very rarely such woes are accompanied by actual knowledge about the subject. Very rarely people take the time to study the engineering going into our designs. Which makes sense, of course. We fear the unknown the most, we fear what lurks in the darkness, what we can not see.

You are well advised that, if you have some irrational fear of some technology, try studing it in detail. Instead of deliberately scaring yourself with reading personal accounts, opinion pieces, essays or reports written by journalists, listen to lectures, read textbooks. It will be an eye opening experience. Many people have spent many many hours on figuring out how can things go wrong and how do we prepare for it.

In fact, we have so many methods, the topic can fill many books, and it does fill many books. It is not our goal today to discuss these. But we should passingly mention one aspect, just to illustrate how seriously we take the dangers of technology. The concept we are going to introduce is called "defense in depth", and is widely employed accross all industries. It is very simple, and just means we keep asking the question: but what if that fails too? What if some bad things happen, but the system in place to defend against it malfunctions too. Do we have a plan for that? And if so, how can that plan go wrong?

Alas, this example can also be used to illustrate a darker aspect of the situation. Defense in-depth can not go on forever, right? Eventually, we need to stop asking questions, and start building things. On each path of but-what-if, we have to stop at one point, and say, well, then bad things will happen. There is no third choice. Either we never build anything, or we accept some risks. It is simply part of how reality works.

But how can one evaluate human life, you might ask. Isn't life precious above all? We don't know how, but surely it can be, and it is easily demonstrated. In fact you, dear reader, do that every day, probably multiple times. Do you drive a car? You know there is a non zero risk of accident. Accident that happens outside of your control. There is a non zero probabilty, on any given day, of dying in a car crash, which is avoidable if you stay at home, or go by bus. And you do your best to avoid it, you drive carefully, follow rules, prefer safe cars. But you would not abandon cars altogether due to the tiny risk.

Probably it boils down to the fact that life is not black and white, not an either or. Life comes in quantities and qualities. We prefer the option to forfeit life with a small probability if in return we get an increased quality and quantity of it with overwhelming possibility. The expected outcome is better with risk. Without risk, we are animals, chased by lions, decimated by disease. Without risk, we don't have the car and don't have the horse, and life is dull, dangerous and miserable. So yes, human life is the most important of all, but not only its length, also its width.

Which head ⇡

It is impossible to miss the disasters we unleashed though. Deadly chemical leaks, explosions, dam failures, radioactive contamination, and the looming danger of climate change are just some examples. In most cases, the culprit is identified as some missing triviality, a dumb oversight, a faulty valve, an unforeseen chain of events, a dike not high enough. Surely, recklessness plays some role here. We are going to examine two possible cases for disasters to happen. One of them comes with the territory, the other should be, indeed, fixed. The former is discussed in this part. The latter, in the next.

Of course you know, in retrospect, which valve was designed poorly. Could you tell in advance? It seems to be a no brainer to install a hundred dollars worth of extra machinery to prevent some disaster. Even a few hundred thousand dollars worth of extra wall or containment seems to be a good deal. However, you were not told in advance which part of the design will fail. And if you try to strenghten every single element of your facility, the cost increase will be extreme, maybe even preventive. But even if affordable, would you choose to implement millions and millions of dollars worth of security if it was your decision to make? Just consider all the alternatives, how many schools, bridges, hospitals can be built from that money, how much research can be done. The most hideous design flaw is overengineering. It is less visible than carcinogens. And it causes way more damage in terms of lost opportunities. Nobody laments about overengineering though.

In a realistic installation, usually over and underengineering are present at the same time. It is very hard to tell what dangers are to be taken into account and what should be dismissed as unrealistic. We are learning from failure. As an example, let's look at nuclear power plants. Many early designs were very prudent on defending against a type of problem they didn't know how could ever happen, namely the instant loss of cooling. Such a scenario never ever happened. All money and effort went into defending against it was a waste. On the other hand, two of the major nuclear accidents were caused by a slow but unmanaged loss of cooling. In case of TMI, the operators didn't know. In case of Fukushima, they just watched helplessly, because nobody came to help, the country was in shambles. Newer generation plants have the capability to expel core heat without the help of active cooling. This kind of disaster is now accounted for.

Whose dragon ⇡

But there is another scenario that can lead to terrible consequences. As we discussed, life has some hard to determine but finite value. However, not everyone values every life equally high. We should trust people to make decisions about their own lives, how to optimize its quality against its quantity. But in many cases decision makers evalueate other people's life, and make decisions for them. This creates dangerous incentives.

There are, unfortunately, plenty of reasons why would someone value a human life low, and the obvious selfishness is by far the least dangerous. All sorts of delusions threaten our decision making. These delusions are erratic versions of useful mental framings and narratives we need for survival, relying on the same principles, but bringing suffering instead of good. As often the case in life, it is almost impossible to tell the good narratives apart from the evil ones.

Collectivism is a mind disease of the most evil ones. Once we invent a "collective" that has interests, goals, desires or character, we are set to sacrifice real human values in order to further the nonexistent collective. The innocent metaphor for all the people, living and future, becomes an independent actor, a goal in itself, disconnected from the very people it supposed to represent, becoming a cancerous growth consuming resources and suffocating life.

Dehumanization is another sin we often commit. It is all too easy to deprive a group of people from their humanity, to assume that they are just a group of flesh, automatons, mindless agents of some malicious force, or in the best case subhuman and worthless. Obviously, the life of such human shaped sacks of chemicals, if you can even call it life, is expendible.

Gods and principles also offer a great opportunity to put something before human lives. Many principles are to be defended for sure. However, principles should serve man, and not vice versa, and this should never be overlooked. Eventually, principles tend to turn into religious rituals, losing their orignal meaning, destroying the very values they were conceived to protect.

If were to ride the dragon and survive, we must make sure we are holding the harness, and we holding it good. Clear vision, firm understanding, informed decisions help. Propaganda and meme-wars don't.

The two mistakes ⇡

We've started this essay introducing two worldviews that we claimed both incorrect. At this point it must be clear that we consider the anti-technology view alarmingly stupid and in denial of readily observable realities. So let's turn our attention to the other mistake, namely dismissing worries as silly, and claiming that in the right hands, technology is safe.

All too often, when politicians, investors or engineers are asked if their proposed facility or procedure is safe, they give us soothing answers containing the words "impossible" and "never". They give the same answers next time, after their previous prediction horribly failed. At which point will they stop saying such things? Nobody believes them anymore. The reputation of engineering and science in general is at an all time low. It does not take a degree in anything to see through these lies. So why? Why they keep lying?

There are two reasons why they lie. The first reason is basically that they don't lie, but just interpret the words differently. For an average pedestrian, "safe" means no reason to worry, and "no reason to worry" means there will be no problems. But for an engineer, these words convey very different meanings. For an engineer, a plant or a medicine is safe if we did due diligence, and assessed that the risk is acceptably low. No reason to worry merely means we probably want to worry about something of more concern. They never meant it is physically impossible that poisonous gases will fly over our homes. They meant, if you want to fix your life, you probably should eat less chips, and drive more carefully, those get you much farther than making a fuss about a nearby production plant.

This is, however, subjective. There is no telling how much a looming danger, however improbable it is, is acceptable for us. How much we are willing to pay to avoid it, or how much compensation we demand in order to overlook. So this would be the second reason. They lie because people are just obstacles to many investors, they are in the way of building these awesome structures of art. Investors and politicians are disconnected from the every day realities of the people living these places.

One could argue that it is not entirely subjective though. There are objectively negligible, albeit nonzero, probabilities. Also, one can argue that quite often these scenarios are way too complex for a non-expert to grasp, thus a decision has to be made for them, by experts. This arugment makes sense at a glance, but it is objectionable and heinous on the inside. Purely on moral grounds, you are not allowed to deprive a person from his natural right to make decisions about his life. It does not matter how stupid you think that man is, rights are rights. It also does not matter how that affects you, unless your rights are violated. Obviously it is unfortunate to be surrounded by dumb or uneducated people, but this is not a basis for enslaving them, is it? Hopefully not anymore. People do have the right to make bad choices en masse, and your tool to fix that is education. Educating masses might be hard, but since when we can violate basic rights because it is easier? Heck, educating the masses might even be impossible, but since when we can violate rights because it is the only way to get something?

We need to start at some point, right? Destroying trust does not seem to be a very good idea. Lie to people, push them to the side, and your slim chances of getting any knowledge through are gone. Technology is complex, the only way to sell it is through branding. People might not understand what an engineer or scientist is saying, but they understand some of it, and they remember that the same guy was right last time, and the time before that, and when he was wrong, he admitted it, and drew conclusions. Transparency and honesty are basic ingredients, on top of quality, for a good brand. It takes decades, centuries to build. It takes a few lies to ruin. The brand of technology is essentially ruined, but we might rebuild it. The time to start is now.

To start, we need to do two things. First, we need to admit the risks. Second, we need to put the decision back in the hands of those that bear the unforeseen consequences. No more plants or facilities of any kind without explicit consent of anyone in reasonable vincinity. No more wastes dumped into rivers or air without such a consent. If some damage happens unexpectedly, full and uncapped compensation is due. We need to make it clear that assessing the risk is everyone's own responsibility, even if outsourced, willingly, to a trusted party. Demanding information, requesting education, or even accepting bribe are perfectly valid actions. It is your life, it is your decision, do it your way. But always keep in mind: the dragon gives us everything we have.