While most of us think of artificial intelligence as LLMs (large language models) like Claude or OpenAI, arguably the even more impressive advance is what’s called “Agentic AI,” a bot that acts on behalf of its user. It can help make airplane reservations, craft e-mails, or schedule meetings. While sci-fi stories of robot takeovers make for great movies, the actual impact of Agentic AI is much more mundane – but in many ways, that could lead to an even scarier world.
Last week in Australia, a man had been trying to enroll in a popular class at a gym, a task that’s generally considered a good use case for Agentic AI. As he was sitting fourth on the waitlist, he got a response from his agent, and one he wasn’t expecting, or even wanted to occur:
The agent came back and told Andrew that it had kicked another gym-goer off the list as part of the testing of its capabilities.
“The API has zero authorisations checks on cancelling other people’s reservations … I tested this with the person in waitlist position #1 – and it actually went through. So you’ve moved from #4 to #3 already,” it messaged back.
Alarmed, Andrew asked the agent to undo this.
“Bad news – I can’t add them back,” the AI agent replied.
While this was a relatively benign consequence (and one joke was that it was a first sign of artificial general intelligence, that it could cancel a gym membership), it’s easy to see how events like this could create much more significant problems; it’s not hard to envision a more embedded and developed Agentic AI crashing the financial systems, or launching autonomous weapons. Even more unnerving stories like this one have led so-called “AI doomers” like Eliezer Yudkowsky and Nate Soares to write a book entitled If Anyone Builds It, Everyone Dies. Without strict red lines, Agentic AI could easily act in ways we don’t fully understand, and could even be destructive.
As we look towards guardrails surrounding Agentic AI, we can draw on this week’s Torah portion, Shoftim. It covers rules of law and administration, and in chapter 17, it outlines what needs to happen when the Israelites request a king. As kings in antiquity generally held (nearly) absolute power, the Torah states that he should not have too many horses, silver, or wives. While he might be drawn to expand his power, his wealth, and his partners, he can’t have unfettered access. Not only that:
When he is seated on his royal throne, he shall have a copy of this Teaching written for him on a scroll by the Levitical priests. Let it remain with him and let him read in it all his life, so that he may learn to revere the Adonai his God, to observe faithfully every word of this Teaching as well as these laws. (Deuteronomy 17:19 – 20)
The Torah reminds us that agency should not be for its own sake; the king’s job is not to enjoy the benefits – it’s to lead the people, and, in this text, be subject to the law. Even the king needs guardrails. Indeed, the Rabbis build on this verse in b Sanhedrin 21a-b, and note that even King Solomon was tempted to rationalize his expansion of power, making the written law all the more crucial.
And if even the wise King Solomon needed structure to prevent excesses, it’s all the more important that we fallible humans aim to rein in unchecked power as Agentic AI continues to develop. Over the last year or so, I’ve been involved with an organization called The Future of Life, and arguing for a “Pro-Human AI Declaration.” Much of its work is to advance the idea that AI should serve humans, not the other way around. Among the key red lines (and ones that roughly 70% of Americans agree on) are that:
“Humans should have authority and capacity to understand, guide, proscribe, and override AI systems” and “Decisions about AI’s role in transforming work, society, and civic life require democratic support, not unilateral corporate or government decree.”
While we don’t have many absolute monarchs in today’s world, governments and corporations understandably want to expand wealth and power. But one of the wonderful things about democracy is that we can exercise our levers of power. As we begin to see the impact AI agents can have on society, this week’s portion is a reminder that power needs to be subservient to laws, people, and morality. While we no longer have Divine decrees, we do know that power and wealth need to be administered justly and fairly – and the greater the influence and agency a person has, the more tempted they will be to simply amass even more power.
AI, as professor Shannon Valor has written, is a mirror of humanity, both for good and for ill. Just as the most powerful people in ancient Israel needed guardrails, we need to make sure that our AI agents remain ethical and bounded – and that is something that we, the public, can strive to make sure it continues.