Suppose you walked into a busy office, laptops everywhere, people tapping at keyboards, chatting in little groups, filling out forms, making things work. Now, imagine that instead of humans, this office is filled with little computer brains—AI agents. They can read, write code, send emails, do spreadsheets, even gossip in the hallway (well, sort of).
Sounds exciting, doesn’t it? But here’s a puzzle: if we let these digital workers loose in a real office, how much work would they actually get done?
The Experiment: Digital Office Olympics
To answer this, a group of very methodical folks built something called TheAgentCompany. Think of it as a laboratory for digital workers. They recreated a small software company—not just any boring simulation, but one with real tools: source code repositories, internal chat, spreadsheets, project managers, finance departments, the whole deal.
They invited today’s brightest AI agents (those built on large language models) to try their luck. The challenge? Get through a stack of actual workplace tasks. Some easy, some nasty. Everything from setting up servers, to collaborating with “colleagues” (other simulated agents), to managing a budget or running a project meeting. And, just to keep it honest, these tasks were checked step-by-step, like a meticulous science experiment.
How Did the Agents Do?
Now, here’s where it gets interesting. The top agents—let’s call them the “Einsteins” of the AI world—could fully finish about one in four of the tasks. If you gave them partial credit for getting some of the steps right, they scored about a third. Not bad, but far from perfect.
But here’s the catch: the tasks they completed weren’t always the ones you’d expect. Give them a tricky programming challenge, and they’ll often do just fine. Ask them to fill in a financial form, or read a document, or politely follow up after a conversation—now the wheels start to come off.
Why? Because, as it turns out, computers don’t see the world the way we do. You give a human a file called something.docx, they know it’s a Word document. For a digital agent, that’s not so obvious. Humans breeze through web pop-ups and annoying forms, while AIs sometimes get hopelessly stuck.
The Office Grows More Complicated
Let’s imagine a typical day: the agent is asked to summarize the team’s progress in a project management tool, notify everyone in chat, clone a code repository, run a script, and upload a report. For a person, this is a little dance—they know whom to ask, how to interpret unclear instructions, and what to do when they hit a snag.
The agents? They march straight ahead—sometimes missing a key step, sometimes inventing a “shortcut” that skips the hard bit. If the path isn’t perfectly clear, they stumble. If they need to talk to a coworker, sometimes they forget to follow up. They’re clever, but not wise.
Why Should We Care?
Now, you might wonder: what’s the big deal? Isn’t it enough that these digital agents can do something? It is. But in the world of work, something isn’t always enough. We want to know: Where do these systems shine? Where do they fall flat? And what does that mean for the rest of us?
This experiment tells us several important things:
- AI agents are quite good at jobs with clear rules, especially coding.
- They’re not so great with messy, “human” tasks—things involving judgment, conversation, or real-world context.
- The more complicated and interconnected the tools (like company chats, spreadsheets, and management dashboards), the more the agents struggle.
It’s not that these agents are dumb—they’re actually astonishing, compared to just a few years ago! But they lack that mysterious, flexible glue that lets people “just figure it out.”
So, What Now?
TheAgentCompany is an open invitation to everyone—developers, business owners, dreamers—to see what happens when we turn AI loose in our offices. If you want a sense of the future, this is a good place to start looking.
Here’s what we learn: today’s AI agents can be helpful assistants, automating little pieces of the puzzle. But the big picture—the stuff that makes work feel “human”—that’s still our territory, at least for now.
Maybe one day, digital workers will learn the whole dance. Until then, they’ll keep practicing, and we’ll keep inventing better experiments.
If you’re building digital tools, running a business, or just curious about the next wave of work, try to keep that spirit of questioning. Don’t just ask, “What can AI do?” Ask, “Where does it get confused, and why?” That’s how you find out what’s really going on.
Want to see these agents in action? Check out the-agent-company.com and try your own experiments. The world is full of puzzles, and sometimes the best way to solve them is to look—really look—at what happens when you try.
