Pete Florence and the trouble with ordinary things

A robot can know a great deal about a shoe and still pick it up by the wrong end. Pete Florence has spent years working on the distance between recognizing something and knowing what to do with it. For a person, the distinction barely deserves a thought. For a machine, it can consume a doctorate.

At MIT, Florence and his collaborators taught robots to find particular points on objects: the tongue of a shoe, for instance, even when the shoe moved. Today, as co-founder and CEO of Generalist, he is pursuing a larger version of that problem. He wants machines to acquire physical skills, carry those skills into unfamiliar situations, and keep working when the world declines to cooperate.

The story runs through chemistry, flying robots, children's engineering workshops and the peculiar difficulty of inserting a block into a slot. Florence's career makes a persuasive case that intelligence has a practical examination. Eventually, something has to move.

The chemistry student who kept teaching

Florence studied chemistry at Princeton, graduating in 2012, and went on to a one-year physics master's at Cambridge. Before beginning his MIT doctorate, he also worked for NASA and spent a year at a venture-backed Silicon Valley startup. His route into robotics passed through several different ways of asking how the physical world works.

Teaching occupied a surprising amount of the route. He tutored writing at Princeton, taught first-year physics as a Cambridge teaching assistant, and spent a summer teaching English in China. He coached children's basketball during college and youth football afterward. For three years, he led backpacking trips for incoming Princeton students. Long before the robots, there were plenty of learners who needed a demonstration, another attempt, or a different explanation.

With his friend Andrew Barry, Florence co-founded Stage One Education. The workshops put engineering into students' hands; his personal account records more than 10,000 participants. The organization offers experiences in fields including robotics, mechanical engineering and AI, taught by engineers and graduate students. A research career can easily become a conversation among specialists. Here was a reason to explain the machinery to someone encountering it for the first time.

First, admit you might be lost

At MIT, working with Russ Tedrake, Florence investigated autonomous flight. A drone moving quickly through a cluttered environment cannot afford a leisurely debate about the furniture. It needs a useful account of the space around it, and it needs that account quickly.

NanoMap, developed with John Carter, Jake Ware and Tedrake, treated uncertainty about a robot's position as part of the planning problem. Rather than insisting on a single perfectly reconciled map, the system kept a history of depth measurements and the uncertain relationships between them. It could consult the relevant view when deciding where to fly.

The paper demonstrated obstacle avoidance on a quadrotor at speeds up to 10 meters per second. The engineering lesson was wonderfully unpretentious: a machine can navigate more reliably when it accounts for what it does not know. Confidence, left unsupervised, is perfectly capable of steering into a tree.

“Overly confident maps won’t help you”

Pete Florence, on drone navigation, 2018

A shoe is a collection of possibilities

Then came the problem of hands. Dense Object Nets, Florence's work with Lucas Manuelli and Tedrake, gave objects a learned visual description at the level of individual points. A robot could use those correspondences to locate a particular part across different views or configurations.

The distinction matters. Picking up any available piece of a shoe is useful if the job is clearing a table. Picking up the tongue is useful if the job requires the tongue. Objects have internal geography, and practical tasks depend on it. A robot that understands only the outline has rather limited travel plans.

The method learned without people laboriously labeling each correspondence. The research demonstrated targeted grasps on deformable objects and transfers between objects in a category. Dense Object Nets won the 2018 Conference on Robot Learning best paper award. Florence completed his MIT computer science PhD in 2019, with a thesis titled Dense Visual Learning for Robot Manipulation. The title was also a fair description of the question he was carrying forward.

Small tasks, awkward physics

At Google, where he became a senior research scientist at Google DeepMind, Florence continued working on robot learning. The papers offer a record of patient attention to tasks that sound almost comically ordinary until a machine has to perform them.

Transporter Networks, published with Andy Zeng and other collaborators in 2020, learned relationships between where an object appeared and where it should go. The demonstrations included assembling kits, stacking blocks, moving ropes and pushing small objects. The architecture used spatial structure to learn manipulation efficiently. The project also released code, environments and datasets, letting others examine and build on the work.

Implicit Behavioral Cloning followed in 2021. Florence and the team studied a way of learning from demonstrations that could accommodate multiple valid actions and abrupt changes in behavior. One real-world task required sliding a block and inserting it with one-millimeter precision. Another involved sorting colored blocks in different possible orders. A successful action can have several acceptable beginnings and one very unforgiving ending.

These projects help explain his concentration on dexterity. The difficulty lives inside the apparently modest instruction. “Put it there” leaves a machine to settle questions of orientation, timing, contact and recovery that a person may never bother to verbalize.

When words acquired somewhere to go

By 2023, Florence was a co-author of PaLM-E, an embodied multimodal language model. The project fed visual observations and other continuous information into a language model alongside text. Instructions could be considered together with what a robot was actually seeing.

In one demonstration, the system planned the steps needed to bring rice chips from a drawer. In others, it arranged blocks or worked with objects beyond a robot's direct training experience. The research explored whether a single model could benefit from learning across language, images and different embodiments. A request about a drawer now had a connection to an actual drawer.

RT-2, also co-authored by Florence, pushed the connection toward control. It represented robot actions as tokens, allowing action data to be trained together with vision and language. The experiments examined whether knowledge learned outside robotics could help a robot interpret unfamiliar objects and instructions.

One example asked for an extinct animal; another explored choosing an object suitable for an improvised hammer. Those tests made an old boundary less tidy. Knowledge about the world could become useful in deciding how to act within it. The robot still needed training and physical competence. But language and movement were beginning to share a working vocabulary.

The ambition gets a company

Generalist brought Florence together with Zeng and Barry, collaborators with whom he already had substantial history. The company describes its purpose as building general intelligence for the physical world, beginning with dexterity. Its team works in the Bay Area and Boston.

In November 2025, it introduced GEN-0 and reported predictable improvements as physical interaction data and compute increased. Its training corpus then contained more than 270,000 hours of manipulation experience. The bet was that robotics could develop its own large-scale pretraining engine, with physical experience as a central ingredient.

GEN-1 arrived in April 2026. Generalist reported 99% average success on its evaluated simple tasks, using about an hour of robot data per task. Its base training used a much larger collection of human physical interaction data. The distinction between broad prior experience and the smaller amount needed for a particular job is central to the company's approach.

By June, Generalist had announced $400 million in new funding, taking its total raised above half a billion dollars. Radical Ventures led the financing; participants included 8VC, Union Square Ventures and existing investors such as NVIDIA. The money supported an expensive proposition: build the data collection, training infrastructure and models required to make physical learning repeatable.

A lesson measured in seconds

With GEN-1.5, released on August 19, 2026, Generalist reported learning new short tasks from a single demonstration lasting three to twelve seconds. Across ten evaluated tasks, the company measured 59% average success using that demonstration in context. With five minutes of task data and ten gradient steps, the reported average rose to 83%.

Those are different learning conditions. Neither percentage means that a robot can now be shown any job and perform it reliably. What makes the result interesting is the possibility of giving a trained robot an example and having it infer a new task without another round of model training. The everyday act of showing someone how becomes a technical interface.

The July release on different robot hands supplied another part of the argument. A shared model worked across end effectors, from five-finger hands to specialized tools. Generalist's aim is to carry useful knowledge across different physical equipment. A skill should have a life beyond the particular gripper that first learned it.

The teacher's question

Florence has written that concrete goals should guide research choices. In an April essay with the Generalist team, he described reducing the amount of task-specific data while increasing performance as a practical roadmap. The method can change; the measurable job remains.

His ambitions include accelerating experimental science: machines able to do more of the physical work that determines how quickly experiments can run. There is a pleasing connection to the chemistry student, the physics teaching assistant and the engineering instructor. Each role involves making knowledge usable by someone else.

Florence still has to answer the question that follows every robot demonstration: can it do the job again, under different conditions? It is an exacting question, and a useful one. A machine earns its place in the world one completed task at a time.