THE LONG CONVERSATION
2010 SPEAKTOIT2016 API.AI JOINS GOOGLE2021 INWORLD IS FOUNDED

People / Ilya Gelfenbeyn / Conversational AI

Ilya Gelfenbeyn and the Computer That Could Take a Hint

Before Inworld’s virtual characters, there was a phone assistant with millions of users. Ilya Gelfenbeyn has spent his career working on the awkward distance between what people say and what computers understand.

Advertisement

A person tells a phone, “I’m bored.” Three words, no technical vocabulary, and already the machine has a problem. Should it recommend a nightclub? Tell a joke? Suggest calling a friend? In 2012, Ilya Gelfenbeyn used those alternatives to explain the difficulty facing the voice assistant his company was building. The sentence was easy to recognize. Knowing what to do about it required knowing something about the person.

It is a modest example for a career that would eventually pass through Google and into the business of giving virtual characters something to say. Nobody is asking the phone to solve philosophy. A satisfactory Friday evening would do. Yet the example exposes an awkward bargain in conversational software: people supply ordinary language and expect the computer to supply the missing context.

Gelfenbeyn has returned to that bargain in several jobs. He co-founded Speaktoit, led the conversational development platform API.AI before Google acquired it, helped establish Google Assistant Investments, and co-founded Inworld AI. He now serves as Inworld’s executive chairman. The companies and titles have changed. Read together, they form a career spent trying to make the exchange feel less like operating machinery.

The trouble with “I’m bored”

Speaktoit was founded in 2010. Its consumer assistant put natural-language interaction on smartphones, including Android, and eventually reached more than 40 million users. A person could ask a question or request a task through a conversational interface. The attraction was straightforward: an assistant promised to spare its owner some of the business of hunting through a phone.

By 2012, Gelfenbeyn was describing a field still near its beginning. He wanted an assistant to account for location, earlier dialogue and user preferences. The same words could mean different things to different people. Even a perfectly transcribed sentence would leave work to do. In his account, useful assistance depended on the surrounding circumstances.

There was restraint in that argument. He distinguished an assistant’s practical duties from the open-ended conversations people have with friends. He also resisted the idea that progress required a single secret discovery. Asked about a breakthrough, he offered a less theatrical explanation.

“It just really requires hard work.”

Ilya Gelfenbeyn, 2012

The answer is almost offensively unsuitable for a launch event. It does, however, leave room for the years of product development that followed. An assistant needs to work across many small situations. The user has little reason to admire the underlying engineering when the appointment still has the wrong time.

The toolbox inside the assistant

The first business contained the beginnings of another. Companies were interested in using the technology behind Speaktoit. In 2014, the team launched API.AI, offering developers a platform for building their own conversational agents. The consumer product had created a need for tools; those tools could serve people making entirely different products.

The change widened Gelfenbeyn’s audience. Instead of convincing every user to adopt one assistant, the company could help developers build conversations into their own services. An interface of that kind has to accommodate the vocabulary and expectations of its particular setting. The machinery becomes useful precisely because its customers can put it somewhere the original team did not build.

On September 19, 2016, Google announced that API.AI was joining the company. More than 60,000 developers were already using the platform. Google’s announcement named Slack, Facebook Messenger and Kik among the environments where those developers were creating conversational experiences. That number measured adoption of a toolbox, rather than downloads of a single assistant.

60,000+developers using API.AI when Google announced the acquisition in September 2016

API.AI later became Dialogflow. Gelfenbeyn joined Google as a product lead, moving from running an independent company into an organization with its own assistant ambitions. The acquisition supplied a clear milestone. It also left him working in the same general territory: helping other people build software that could handle a conversation.

ONE QUESTION, DIFFERENT WORKBENCHES
  1. 2010SpeaktoitA consumer assistant
  2. 2014API.AITools for developers
  3. 2017Google Assistant InvestmentsBacking the ecosystem
  4. 2021InworldInteractive characters

From writing the software to backing its makers

At Google, his work expanded into investing. He became the founding head of Google Assistant Investments, a program publicly announced in 2018. It supported startups building in the digital assistant ecosystem. The task now included helping other founders find a use for conversational technology and bring their products to more people.

A February 2019 update under his name gives that work some texture. AskPorter applied an assistant to property management. StyleHacks offered clothing recommendations that considered preferences and the weather. StatMuse delivered sports information through recognizable sporting personalities. These were different businesses with a shared interface problem: a spoken request had to lead somewhere useful.

The portfolio made conversation look less like a standalone destination and more like a way into an existing activity. People needed to manage a property, choose clothes or check a score. Voice could shorten the journey. The appeal depended on whether it actually did. A new interface still owes the user a result.

His own education continued alongside that work. After studying mathematics and computer science at Novosibirsk State University and earning an MBA at the University of Brighton, he completed Stanford’s LEAD executive program in 2020. His public profile also lists a Chevening Scholarship. The combination places business training beside the technical subject that had occupied his earlier career.

He left Google in 2020 and helped establish The.AI Ventures. For a time, the conversation problem occupied the investor’s chair. Another company would soon give it a different setting.

A world needs somebody to talk to

Inworld was founded in 2021 by Gelfenbeyn, Michael Ermolenko and Kylan Gibbs. There were existing connections inside the team. Ermolenko had led AI work at API.AI. Gibbs had worked on conversational AI products at DeepMind. The new company brought those experiences to a question about the inhabitants of virtual environments.

The premise was that an immersive world needed a population capable of interacting with its visitors. A character might have a personality, a backstory and knowledge of its surroundings. Players could speak to it and receive an answer shaped by those details. The ambition extended beyond generating isolated lines of dialogue.

A fictional world imposes peculiar social obligations. A character should know things appropriate to its role, respond to the situation and maintain some continuity across the exchange. An impressive landscape can survive being admired in silence. Its inhabitants become less convincing when every question sends them back to the same sentence.

Ilya Gelfenbeyn, second from right, with the Inworld team at a booth displaying virtual character technology
Brains on the stand. Gelfenbeyn, second from right, with the Inworld team and a very literal invitation to try the technology. Photo: Intel Capital / Inworld.

Intel Capital’s involvement joined the new venture to the old one. It had backed API.AI and worked with Gelfenbeyn and Ermolenko before Google’s acquisition. In August 2022, it co-led Inworld’s Series A with Section 32. The investor was returning to people whose earlier work it had already watched develop.

Inworld let developers describe a character in natural language and test it in a studio environment or an Oculus companion app. Game-engine integrations connected those characters to virtual settings. The workflow carried a pleasing symmetry: language helped specify the character, and language became one way to meet it.

Giving the trickster a specification

One small document makes the design problem unusually concrete. Gelfenbeyn is a named co-inventor, with Ermolenko, Gibbs and Aleksey Tikhonov, on a US patent for archetype-based generation of AI characters. Granted in November 2024, it describes selecting an archetype from a description and using it to assign characteristics to an AI character model.

The possible archetypes include a hero, a mentor, a villain and a trickster. These familiar figures provide a starting point for character construction. A creator supplies a description; the system uses a recognizable pattern to help determine the model’s parameters. The proposal gives technical form to something storytellers do when they establish what sort of person has entered a scene.

A patent describes an invention, rather than proving every proposed experience works in a finished product. Even with that distinction, its subject is revealing. The practical work involves translating a human description into settings that can govern behavior. Calling somebody a trickster is quick. Specifying how the trickster should answer the next question takes considerably more thought.

The second question

Gelfenbeyn’s present title marks another change in responsibilities. Inworld identifies him as co-founder and executive chairman, with Gibbs as CEO. His public professional profile also lists mentoring at StartX from February 2024 and a corporate director education program completed the following month. The career now includes governance and work with other founders alongside his founding role.

Inworld itself has broadened its description beyond game characters. Its current business includes realtime voice models and infrastructure for developers building interactive applications. That brings the company back toward the practical machinery beneath a conversation: producing speech, handling the exchange and making the pieces usable by somebody else.

The progression also changes what a builder must pay attention to. A consumer assistant puts one experience in front of its owner. A developer platform has to help someone else create an experience, make choices and discover what went wrong. An interactive character adds the expectations of a story. Across those settings, ordinary language keeps doing two jobs: it expresses the user’s request, and it exposes everything the software still needs to know.

Across Gelfenbeyn’s career, a working assistant has repeatedly opened the door to a larger set of tools. The pattern gives his story its continuity. A person says something ordinary. Software must decide what it means, what matters from before, and what should happen next. The first answer earns attention. The second question reveals how much of the conversation the computer understood.

Continue the conversation