Technical Disciplines as Ethical Disciplines
This post is inspired by mathematician and luminary Cathy O’Neil and her book, Weapons of Math Destruction, which is an eye-opening and disturbing survey of how society has constructed opaque and damaging systems on a mass scale, many of which are based on quantitative models. Having been hit deeply on an emotional level by this book, and seeing its significance to my own professional presence, I want to share some of what I have learned, as well as my own ethical principles surrounding the use of mathematics in society.
I am an applied mathematician not by training, per se, but by fiat (and self-education); though I have a degree in mathematics and now work in the space industry, little of my formal academic training directly prepared me to deploy mathematical tools outside of academia (You can read even the first page of my master’s thesis to see how utterly made up and far from reality — though rigourous — pure mathematics can be!). What brought about my fiat was the discovery that people who are able to serve as social/intellectual bridges by communicating effectively between groups of different perspectives can be enormously valuable. If nothing else in life, I am naturally positioned to serve as such a person because of the diversity of my interests and skills.
Leaving academia reminded me in a big way of something I already knew: that there is a lot of toxicity in the relationship between many people and quantitative reasoning, abstract mathematics, and ethical reasoning. Beyond sending useful objects into space and gaining information from space, what I really aspire to do in my career is to change how at least a small number of people think about these aspects of our world — to the extent that their decision making becomes more oriented toward human thriving and that they have more or better tools with which to pursue it.
One principle of mathematical models in society that has emerged in recent decades seems to be that the outputs of certain models, intended to represent reality, create reality rather than accurately reflect it. For example, when teacher performance or someone’s credit is evaluated by an algorithm (usually with no transparency or feedback to the algorithm), their score often becomes more predictive of their outcomes than their actual performance or trustworthiness; if the algorithm gives a trustworthy person a low credit score, limiting their ability to, say, buy a home or start a small business, maybe that person takes out a payday loan out of understandable desperation. A downward spiral might well ensue, reinforcing the original meaningless score.
I think that the fundamental problems with poor use of quantitative models by industry and governments are threefold: 1) They are sometimes using models to pursue goals that are irrelevant or detrimental to human thriving, 2) The scope of deployment of some models is way too large, 3) The measurements, monitoring, and maintenance going into some models are woefully inadequate. The complementary side of this problem is that there are huge sectors of the population vulnerable to the misuse of quantitative models for a wide variety of reasons: lack of information, tough financial situations, joblessness, low-quality work, illness, injury, etc. Pretty much anyone with a pain point detectable by advertisers will be targeted, often for harmful financial, “educational”, and physical products.
The above doesn’t even scratch the surface of what is going on. But it’s clear that there are severe and widespread problems. What am I personally going to do about this? Like other areas in my life, all I can really control is what goes on in my own backyard, so to speak. This means that I’m committed to using quantitative reasoning to sustain and improve the functioning of my own household and workplace, as well as the quality of my employer’s products and services. I can make measurements and perform calculations in all of these settings to inform decisions made by human beings for human beings. I can use this work to paint different pictures that represent different aspects of the underlying reality — which is existentially more complex than I could ever represent by any set of measurements or calculations. I can communicate effectively to others what these pictures say, what they don’t say, how they could be misinterpreted, and the possible consequences of making different decisions based on the measurements.
I am also committed to talking/writing to a broad audience about these issues and really taking the energy and time to understand audience perspectives. Very few people need the level of mathematical knowledge that experts possess, but society collectively needs better tools for mathematical-bullshit detection and protection. This protection is very different from a blanket dismissal of numbers, arithmetic, statistical data, or mathematics in general. Rather, it is its own branch of critical thinking, which should be applied less to academic experts and more to people who stand to gain by others, including you, making a poor quantitative decision — e.g. credit card agencies, banks, and for-profit schools.
This critical thinking requires knowledge of and the ability to use basic quantitative constructions and models, the stuff of which you might find in a financial literacy course: the principles of compounded interest and investing, creating and balancing budgets, spending tracking, and the fundamentals of different financial products available. To these I would also add solid grounding in statistical reasoning. By this I don’t mean the ability to perform t-tests to make everyday decisions as rigourously as possible. I think optimizing for rigour in the everyday is a recipe for burnout and losing track of what’s really important. I mean being comfortable using statistical heuristics to guide us toward decisions that will lead to better outcomes, in aggregate. We need to understand when we’re at risk of overgeneralizing from data that’s too limited; for example, most people would think it dumb to marry someone you’ve only interacted with a few times and in only one or two contexts. Most people would probably be right. But we make this same type of inference error all the time when drawing conclusions about how other people and the world work. And we lack the training to know how important it is to seek data that contradicts our conclusions and to make that into a habit.
The other main side of my personal commitment is to work to promote values that I believe, in turn, promote human thriving, so that mathematical tools are used as instruments of right purposes. I sincerely hope that my views about what these values should be changes and improves over time. Applying the principle from the last paragraph, surely at age 27 I haven’t seen enough of the world for my present views to be optimal — at least, that’s highly unlikely! Regardless, I make it a habit to be a member of my workplace who encourages and exemplifies both high standards and living a healthy, fulfilling life. This becomes more important the more influential I become. I also make a point to say in public talks that tech is not just about the cool/wow factor. The need for wonder in life is legitimate, but we need to assess the merit of that factor in context when deciding which tech endeavours to pursue and how. The ultimate goal of engineering is to fulfill human needs while optimizing for cost. Optimizing for the cool factor alone can ultimately fail those we’re supposed to serve in a number of ways: e.g. by burning out entire teams, needlessly destroying resources, or creating a costly solution to a problem that doesn’t matter enough to justify the cost.