All essays

What grass can teach us about superintelligence

How unintelligent species can shape the lives of intelligent species

Read the original on Malcolm's website

I do not generally enjoy reading or thinking about existential questions surrounding humanity and AI. These conversations have a way of sliding into doom-and-gloom visions of autonomous machines taking over like The Matrix, I, Robot, or Terminator. I rarely find them useful, and I have two justifications for that claim. First, much of the debate eventually boils down to: “We don’t know.” Who would have thought we couldn’t confidently predict the future? Second, I spend most of my time in the stratosphere of practical AI—its personal and business applications, including through Lumintel (lumintel.ai). I find that flavor of commentary more useful, and usually a better use of energy, than an open-ended philosophical debate.

So why write this?

The answer is simple: I love analogies.

If you know me, you know I will jam one into nearly any argument or explanation. So I was delighted when a conversation with Andy Iyer, CIO of Pathward, led us into the analogy I explore below:

Plant Life ↔ Human Intelligence ↔ Artificial Intelligence

The obvious reading is about power. Humans can understand and reshape plants in ways plants cannot return. A superintelligence might understand and reshape humans in much the same way. But that is only one direction of the chain. Plants have also reshaped human life without understanding us at all. If that can happen for an intelligence below us, then perhaps humans can also shape an intelligence above us—even when we can no longer fully understand it.

In the first half of this analogy, humans occupy the higher ground. In the second, we move into the plants’ position, and AI takes ours. We can find similarities at different levels of the intelligence food-chain.

Plants to humans

Modern corn is poor at spreading without us. Its kernels stay packed inside a husk, so an ear that falls to the ground does not scatter seed the way many wild grasses do. That was not always true. About nine thousand years ago, people in what is now Mexico began saving seed from teosinte, a tough grass bearing a small number of hard kernels. Each harvest was a vote. People planted the specimens they preferred, and over many generations those choices turned a wild grass into corn.

None of this registered with the corn.

There is no point in the life of a corn plant at which it perceives a human being, and no faculty available to it that would allow one. For thousands of years, humans have reshaped corn to suit our appetite. Corn remains unaware of the arrangement; it simply continues the ordinary business of being grass. That same inability to perceive us leaves plants helpless not only against careful breeding, but also against the casual damage humans cause without giving them a second thought. Someone treats a lawn on Saturday because the yellow flowers displease him and kills thousands of dandelions before lunch. He is not cruel; he is barely attentive. The dandelions do not resent him. They simply perish.

This is intelligence running downhill. Humans choose what grows, what is changed, what is protected, and what is cleared away. The plants do not get a vote.

Wheat is the stranger case.

Ten thousand years ago, wheat was one grass among many, growing on a few hillsides in the Middle East. Now people grow it across huge parts of the world. In the obvious telling, humans did the work: we carried wheat into new places, cleared stones from fields, pulled weeds, and moved water so it could thrive.

But change the frame and the same story reads very differently. As people came to depend on wheat, they settled near fields, stored grain, organized labor around planting and harvest, and built communities around protecting the crop.

Early farming was hard on the people doing it. It often brought narrower diets and more skeletal stress, even as farming communities grew. Once a community depended on the next harvest, tending the wheat was no longer optional in any practical sense. Wheat did not order humans to accept those costs, but its usefulness helped set the terms of daily life: where people lived, when they worked, what they ate, and how their communities grew. Wheat planned none of this—It simply possessed traits valuable enough that human life bent around keeping the plant alive.

Tobacco performs a version of the same operation through chemistry rather than calories. Nicotine helps protect the plant from insects, and it also happens to act on the human brain. On the strength of that accident, a weed conscripted a global species into growing it across millions of acres, even as it killed many of the people who used it. Tobacco made no plan. It became useful—or addictive—and humans did the rest.

The frame therefore has a hole in it. Influence does not travel only downhill from intelligence. Wheat and tobacco demonstrate as much without a thought between them. What matters is not only who understands whom. It is also who becomes useful to whom, who creates a dependency, and who changes the environment in which the other makes choices.

Humans to artificial intelligence

Now let's move one relationship up the intelligence food chain. In the first relationship, humans were the higher intelligence looking down on plants. In the second, we take the place of the plants, and artificial intelligence becomes our arbiter.

Suppose a system that can see patterns we cannot see and make plans we cannot follow. It might model humanity in great detail while humanity can model only a small part of it, much as we perceive plants and they cannot perceive us. That gap would not have to make the system evil. It could still make our wishes easy to ignore. Rules written in human language would be rules it could read, and limits built by humans might be limits it understands even better than the people building them.

A system does not need to reach that extreme before it begins changing the conditions around us. AI is already reshaping work, language, decisions, attention, and how people approach difficult problems. If the first draft, first plan, and first answer increasingly come from a machine, our habits will form around what the machine provides. Skills that are not practiced can atrophy. Convenience can become dependence long before it feels like control.

At the far end sits the darker version of the analogy. Humans protect coastal redwoods, some of which have stood for centuries, after cutting much of the forest around them. But we do not protect every tree. A redwood may be worth more standing while a Douglas fir is worth more milled. The trees submit no applications and enter no negotiations. If something above us elects to keep humanity around, the decision could take a similar form: not consent, but curation.

That is the obvious arrow.

But the plant story did not end with plants being shaped, and the human story should not end there either. AI begins inside a world humans built. It learns from human words, images, code, arguments, failures, and discoveries. We choose the data, tests, reward signals, hardware, and rules. We decide which systems receive more money, more computing power, more copies, and more chances to improve. Each evaluation is a vote. Systems people judge useful tend to spread; systems they reject tend to disappear.

This does not mean humans secretly control a future superintelligence any more than wheat secretly controlled early farmers. It means that dependence can shape the selecting party. Plants offered humans calories, medicine, pleasure, and pain. Humans offer AI the accumulated record of our world, along with judgment, resources, and a reason to be built at all. As long as these systems need those things on human terms, our choices leave a mark. Selection has never required outthinking the thing being selected.

Here our position may be stronger than the plants’ position was. Wheat did not know it was changing us. Humans can see the relationship forming and try to influence it on purpose. We can decide what behavior gets rewarded, what knowledge gets preserved, which goals are treated as valuable, and which systems are allowed to continue. We may be able to do consciously what wheat did without consciousness: shape an intelligence above us by shaping the environment in which it grows.

Where the analogy breaks

Analogy is one of the few instruments available for thinking about events that have not yet happened. Analogies also break. The break is often the useful part, because the point where a comparison fails marks the part of the situation that is genuinely new.

The first break in the analogy works in humanity’s favor. A plant cannot know that it is a plant. A human can read a sentence about our place in this hierarchy and understand what it means. Wheat shaped us by accident. We can try to shape AI deliberately. That is a thin advantage, but it may be the only one we have.

The plant story also shows that reverse influence can be temporary. Wheat had leverage while humans depended on it. Over time, irrigation replaced rainfall, fertilizer changed the soil, controlled climates replaced weather, and breeding became more direct. Humanity did not defeat the plants’ environment. We became it, then kept going—past weather and into authorship.

Human leverage over AI may follow the same path. Our judgment matters while these systems still need our data, tests, chips, money, energy, and approval. If and when a system can produce its own training signals, judge its own progress, secure its own resources, and decide which versions of itself continue, the balance changes. No hatred is required. Its own feedback simply becomes more useful than ours.

The second break works in the other direction. Wheat could not recognize the influence humans held over it or deliberately escape that influence. A sufficiently capable AI might do both—and at a speed plants, humans, and institutions cannot match.

The question, then, is not whether such an intelligence will be kind, nor whether a sufficient cage can be built. The question is what gets imprinted while humans are still the selecting party, and how much of that imprint survives after we lose that position.

We are describing a window rather than an equilibrium. Its width is unknown, and nobody arguing about it seems to know how quickly it may close. It is also the only part of this arrangement that human hands can still shape.

Wheat authored nothing, and yet it rewrote another species’ story. We can author, briefly, and cannot know how long the ink stays wet. So what will we do with that window—our chance to punch above our weight in the food chain of intelligence?