25. Emergence and Ethics of LLMs

25.1. Pre-Reading

• Falcon-RefineWeb Model Card

Objectives

• Describe how LLMs exhibit emergent behavior.

• Discuss ethical considerations with LLMs.

25.2. Emergence

Emergence is when you combine relatively simple sets of rules and at some threshold a mind-blowing result manifests.

This is one of my all time favorite books.

Emergence is the the opposite of reduction. The latter tries to move from the whole to the parts. It has been enormously successful. The former tries to generate the properties of the whole from an understanding of the parts. Both approaches can he mutually self-consistent.

\[ sand + glass funnel = ? \]

\[ sand + glass funnel = time keeping \]

The Mind as an Emergent Phenomena

There are a wide variety of understandings of what is meant by mind. The reductionist behaviorist tradition would argue that mind is an epiphenomenon of the activities of collections of neurons. They argue that minds do not in fact exist. At the opposite extreme, the idealist tradition going back to George Berkeley would argue that mind is all that exists, and matter is an epiphenomenon posited by minds for explanatory purposes. The Kantians would argue for the existence of both mind and matter, the latter being the ding an sich (thing in itself) that minds aspire to and cannot fully comprehend.

Our view is that all of the above is too simplistic. The universe, whatever its ultimate character, unfolds in a large number of emergences, all of which must he considered. The pruning rules of the emergences may go beyond the purely dynamic and exhibit a noetic character. It ultimately evolves into the mind, not as something that suddenly appears, but as a maturing character of an aging universe. This is something that we are just beginning to understand and, frustrating as it may be to admit such a degree of ignorance, we move ahead. That is our task as humans; some would call it knowing the mind of God and regard it as a vocation. ~ The Emergence of Everything, Harold J. Morowitz

LLMs and Emergence

A Socratic Journey to understanding Emergence in Large Language Models*

Q. What is the core of Natural Language Processing? A. Doing math with words.

Q. What is the key outcome of that math? A. Determining the relationship of words and how they are interconnected in context.

Q. How does understanding the relationship between words enable prediction? A. The system can predict the next word or phrase in a sequence.

Q. What happens when you scale the ability to predict the next word to billions of parameters? A. The shocking behavior of an LLM emerges: the ability to understand context, create coherent paragraphs, exhibit creativity.

25.3. Inflection Points

• In some ways this may be like the Industrial Revolution.

Only The Paranoid Survive by Andrew Grove, founder and CEO of Intel, discusses the inflection point that Intel faced when moving from the memory bussiness to the microprocessor bussiness. This ultimately was the right move, but an incredibly difficult one. Had they not made the move, you probably would never have heard of Intel. But as it is, the computer you are reading this on has Intel Inside.

The move was difficult in part because Intel had been so dominant in the memory bussiness that the identiy of the company and its employees was directly tied to making computer memory:

The company had a couple of beliefs that were as strong as religious dogmas. Both of them had to do with the importance of memories as the backbone of our manufacturing and sales activities. (90-1)

But strategic infleciton points also create opportunity:

Strategic infleciotn points, painful as they are for all participants, provide an opportunity to break out of a plateau and catapult to a higher level of achievement. Had we not changed our buisiness strategy, we would have been relegated to an immensely touch economic existence and, fore sure, a relatively insignifigant role in our industry. By making a forceful move, things turned out far better for us. (95)

Most critically,

Implicit in doing business every day is a mental map of the strucutre of ths industry. This map is composed of an unstated set of rules and relationships, ways and means of doing business, what’s “done” and how it is done and what’s “not done,” who matters and who doesn’t, whose opinion you can count on and whose opinion is usually wrong, and so on. If you’ve been in the industry for a long time, knowing these things has become second nature. You don’t even think about them; you just know that’s the way they are.

But when the structure of the industry changes, all of these elements change too. The mental map that you have been carrying with you for all these years and relied upon in charting your company’s course of actions suddenly loses its validity. However, you haven’t had a chance to replace it with a new mental map. You haven’t made the explicit substiution about how things are done now versus how they were done before, or who matters now versus who mattered then.

During a strategic inflection point, management continually has to refine its conception of the strategic map of the industry. We all automatically do this in our heads. But mental maps are awfully forgiving of ambiguity. You must force yourself to commit your thoughts on paper. (134-5)

More on Mental Models

Another one of my favorite books is Smarter, Faster, Better by Charles Duhigg.

Models help us choose where to direct our attentions, so we can make decisions, rather than just react (101).

However, once in a cognitive tunnel, we lose our ability to direct our foucs. Instead, we latch on to the easiest and most obvious stimulus, often at the cost of common sense. (77)

In particular, Chapter 3 - Focus: Cognitive Tunneling, Air France Flight 447, and the Power of Mental Models should be required reading for every Air Force Officer.

From that chapter:

What if, de Crespigny thought to himslef, I imagine this plane as a Cessna? What would I do then? (Duhigg, 98)

This is the exact same question that Grove asks Moore:

“If we got kicked out and the boad brought in a new CEO, what do you think he would do?” Gordon answered without hesitation, “He would get us out of memories.” I stared at him, numb, then said, “Why shouldn’t you and I walk out the door, come back and do it ourselves?” (Grove, 89)

So, ask yourself, how does the Emergence of generative AI change my mental model about the world?

25.4. Ethics

• LLMs are primarily Western because the Internet contains predominantly things written by the West.

• Bias?