2. Prediction Machines

The current wave of advances in artificial intelligence doesn’t actually bring us intelligence but instead a critical component of intelligence: prediction. ~ Prediction Machines (4)

2.1. Pre-Reading

TinyML Chapter 3: Intro and “What Machine Learning Actually Is.”

2.1.1. Objectives

• Understand that AI is fundamentally prediction.

• Understand that prediction combines with human judgement to result in a decision.

---

2.2. Our role as commanders

Our role as commanders is to exercise judgment and make risk decisions in an uncertain and adversarial environment

2.2.1. From Doctrine

It is a given in future conflicts that the joint force will be conducting operations in a contested environment. We must be prepared to execute in a degraded C2 environment where clearly delineated and forward thinking commander’s intent will be a requirement. It is imperative senior leaders provide our commanders with conditions-based authorities delegated to the lowest capable and competent level, and empower command by negation to accept the appropriate level of risk, all while working toward moments of clear C2. ~ General Charles Q. Brown, AFPD-3-30

AFPD 1 core principle of Mission Command:

Accept prudent risk. All military operations contain uncertain, complex, and ambiguous elements. Commanders must analyze risks in collaboration with their subordinates to balance the tension between protecting the force and accepting and managing risks that must be taken to accomplish the mission.

2.3. Making a Decision

The following is derived from Prediction Machines: The Simple Economics of Artificial Intelligence by Agrawal, Gans, and Goldfarb, 2018.

A decision includes prediction, judgement, action, outcome, and data.

2.3.1. Prediction

The machine learning algorithm builds a model of the system based on the data we provide, through a process we call training. The model is a type of computer program. We run data through this model to make predictions, in a process called inference. ~ TinyML

AI only brings us a single critical component of intelligence: prediction

Prediction takes information you have, data, and uses it to generate information you don’t have.

Machine learning reframes algorithmic problems as probabilistic prediction problems.

The Oakland A’s identified what problem they were trying to solve: the impact of a player on team performance and capitalized on that; see Moneyball.

Jeff Hawkins argued that “The human cortex is an organ of prediction.”

2.3.1.1. Accuracy of prediction

In contrast to machines, humans are sometimes extremely good at prediction with little data. This is largely because of our broad context of the world.

As dimensionality grows, humans’ ability to form accurate predictions diminishes.

Machines passed the human benchmark for image classification (5% error) in 2015.

Going from 85% to 90% accuracy means mistakes fall by one third. Going from 98% to 99.9% means mistakes fall by a factor of twenty. Better prediction leads to better outcomes. At some point, accuracy hits a threshold where the prediction can become transformational.

Prediction machines are valuable because:

1. they often offer better, faster, cheaper predictions than humans can

2. prediction is a key ingredient to decision-making under uncertainty

3. decision-making is ubiquitous throughout our economic and social lives

2.3.2. Judgement

Computers made arithmetic cheap. Internet made distribution, communication, and search cheap. AI has made prediction cheap.

A drop in price of one thing can increase the price of its complements. For example, cheaper coffee means more expensive cream and sugar.

As the price of prediction drops, the price of judgement will increase.

Having better prediction raises the value of judgement.

Only humans can express relative rewards of one action over another. The prediction may be handed to the machine but the human will still handle the judgment.

Objectives rarely have only a single dimension. Answering why you are doing something may be complicated.

Judging payoff requires time and effort. This slow decision-making has tradeoffs. Judgement, whether by deliberation or experimentation, is costly (especially in combat).

In some cases, the judgment may be that the reward function engineering is clear enough to hand off the decision to the machine based only on the prediction.

Prediction will likely shift to machine by default and human by exception.

To make the most of prediction machines you need to rethink reward functions throughout your organization to better align with your true goals. This task is not easy.

Humans are critical to decision making where the goals are subjective.

Contracting out judgement is risky because they do not have the full context to resolve uncertainty in a strategic dilemma; you should keep judgement-focused workers in house. (also, they are a big piece of your competitive advantage)

Judgement quality is hard to specify in a contract and difficult to monitor.

2.4. Risk

A key task for a business leaders is to anticipate and manage various risks.

Risk exposure matters when a decision can change ones exposure to risk.

In the case of weather, an automated, decent prediction right now is preferable; the human only gets involved in extreme events. Error rate increases for extended forecasts, so decision maker must decide when to decide.

Risk can be managed with insurance, where you take actions to reduce the cost associated with bad outcomes. Or risk can be managed with protection, where you take actions to reduce the probability of a bad outcome.

When you use insurance to minimize waste, the waste is visible; by contrast, the waste may be difficult to see with protection.

Enhanced prediction increases returns by reducing the waste from unneeded insurance or protection.

Air Canada was able to predict how to fill underutilized capacity because it had:

1. Measurable waste

2. Wealth of data about the problem

3. A well-contained problem

Prediction machines will provide new and better methods for managing risk.

2.4.1. Uncertainty

Perhaps the biggest weakness of prediction machines is the high confidence they can have in a wrong answer.

Uncertainty means you need judgement when the prediction turns out to be wrong, not just when it is correct. Uncertainty increases the cost of judging a payoff for a given decision.

2.4.1.1. Closing thought from AFPD-1

==Airmen execute mission command through centralized command, distributed control, and decentralized execution.==

This is the new way in which we will fight wars.