BREAKING PROF. TOM YEH TEACHES AI ONE HAND-DRAWN CALCULATION AT A TIME 62,000+ SUBSTACK SUBSCRIBERS - RANKED #66 IN TECHNOLOGY 200,000+ SOCIAL MEDIA FOLLOWERS AND COUNTING MIT PHD • CU BOULDER ASSOCIATE PROFESSOR • CREATOR OF AI BY HAND NSF CAREER AWARD WINNER • H-INDEX 41 • 7,488 CITATIONS SIKULI: THE SCREENSHOT AUTOMATION TOOL WITH 200,000+ DOWNLOADS FEYNMAN METHOD MEETS DEEP LEARNING: "WHAT I CANNOT CREATE, I DO NOT UNDERSTAND" PROF. TOM YEH TEACHES AI ONE HAND-DRAWN CALCULATION AT A TIME 62,000+ SUBSTACK SUBSCRIBERS - RANKED #66 IN TECHNOLOGY 200,000+ SOCIAL MEDIA FOLLOWERS AND COUNTING MIT PHD • CU BOULDER ASSOCIATE PROFESSOR • CREATOR OF AI BY HAND NSF CAREER AWARD WINNER • H-INDEX 41 • 7,488 CITATIONS SIKULI: THE SCREENSHOT AUTOMATION TOOL WITH 200,000+ DOWNLOADS FEYNMAN METHOD MEETS DEEP LEARNING: "WHAT I CANNOT CREATE, I DO NOT UNDERSTAND"
Professor Tom Yeh - AI Educator and Creator of AI by Hand
AI Educator • Professor • Newsletter Creator

Tom
Yeh

The man who taught AI to go slow - and the world to keep up.

Associate Professor at the University of Colorado Boulder. Creator of AI by Hand, the newsletter where 62,000+ subscribers learn transformers, LLMs, and deep learning the old-fashioned way - pen, paper, and patient arithmetic. No Python required. No shortcuts taken.

62K+ Substack Subscribers
200K+ Social Followers
41 H-Index
150+ Papers Published

The Professor Who Put AI Back in Your Hands

- a first-principles education story

In early 2024, Tom Yeh posted a hand-drawn exercise on LinkedIn showing how a transformer works - not with code, not with diagrams borrowed from a research paper, but with actual numbers written in colored pen on graph paper. He expected mild interest. What he got was a movement.

Within weeks, 25,000 people were following along. Within months, the number was 200,000 across LinkedIn and X. The AI by Hand newsletter on Substack crossed 62,000 subscribers - ranking in the top 100 technology publications on the entire platform. None of it was planned. All of it was earned, one calculation at a time.

Yeh is an Associate Professor of Computer Science at the University of Colorado Boulder, where he has been teaching and researching since 2012. He holds a PhD from MIT, where he built some of the early foundations of what we now call computer vision-driven automation. His Sikuli project - which let computers automate GUIs by looking at screenshots the way humans do - was downloaded over 200,000 times and spawned one of the most-cited HCI papers of the 2010s.

But the part of Tom Yeh that matters most to most people right now sits at the intersection of a Feynman quote and a box of colored markers. His guiding philosophy: "What I cannot create, I do not understand." In an era where AI has become a vending machine - put in a prompt, get out a result - Yeh insists on understanding the mechanism. The math. The actual numbers flowing through an attention head, a feed-forward layer, a normalization step.

His AI by Hand series does not treat the reader as a passive consumer. You work through it. You fill in the cells. You multiply the matrices by hand - or in an Excel spreadsheet if you prefer, since Yeh also built an entire GitHub repository of Excel-based deep learning exercises that garnered 1,300+ stars almost immediately. The exercises span everything from basic matrix multiplication and gradient descent to full transformer architectures, diffusion models, and the Mamba state-space model.

What makes this genuinely unusual is that Yeh drew every single one of those exercises himself - by hand, in color, on printed paper. Three hundred individual exercises across twelve workbooks. He did not outsource the aesthetics. He did not generate them. He sat down and created them, the same way he asks his students to engage with the material.

"What I cannot create, I do not understand."
- Richard Feynman, the guiding principle behind AI by Hand

The response has surprised even Yeh. When students find errors in his solutions - and occasionally they do - he is reportedly delighted. It means they understood the problem well enough to catch the mistake. That is the point. Not passive absorption, but active engagement. Not trust in the black box, but mastery of what's inside it.

In 2026, Yeh launched AI by Hand Academy: a structured curriculum of video-based courses covering introductory and advanced AI topics, starting with a module on Agentic AI that walks through LLMs, RAG, agents, and tool use. The Feynman method, now with video. Over 130 founding members - recognized personally by their initials on the Academy page - joined in the early months.

Beyond the newsletter, Yeh leads the Imagine AI Lab at CU Boulder and co-directs the Center for the Brain, A.I., and Child (BAIC). His research sits at the junction of human-centered AI and interpretability - studying how people build, inspect, and reason about AI systems. His academic citation count sits at 7,488, with an h-index of 41, and funding from NSF, NIH, DARPA, and two private foundations.

He did not arrive at AI education through a single clean narrative arc. He grew up in Taiwan, completed his BSc at Simon Fraser University in Canada, earned both his MS and PhD at MIT, spent three years in postdoctoral training at the University of Maryland, and eventually landed at CU Boulder. The range of his research - computer vision, 3D printing, assistive technology, brain imaging, citizen science, AI ethics - makes more sense when you understand that he has always been more interested in what AI can do for people than in AI for its own sake.

The viral newsletter is not a pivot. It is the same instinct - toward legibility, toward inclusion, toward making the inaccessible accessible - that drove VizWiz in 2010, when he built a system that let visually impaired users photograph an object and receive a real-time spoken answer. Years before GPT-4V. Years before anyone used the phrase "multimodal AI" in casual conversation.

Tom Yeh is what happens when deep expertise meets a stubborn commitment to clarity. He is not simplifying AI. He is insisting that it can be understood - and then proving it, cell by spreadsheet cell.


The Scorecard

What 20+ Years of Serious Work Looks Like

150+
Research Papers

Spanning AI, HCI, computer vision, assistive tech, 3D printing, brain imaging, and ethics. Funded by NSF, NIH, DARPA, Knight Foundation, and the Piton Foundation.

41
H-Index

One of the cleaner measures of sustained research impact. Yeh's citation record reflects over two decades of work that other researchers actually use.

1,216
Citations: VizWiz

His 2010 paper on visual Q&A for blind users - years before modern AI vision - is still one of the most cited works in accessible AI.

200K
Sikuli Downloads

The screenshot-based GUI automation tool he built at MIT. Open-sourced in 2010. Downloaded 200,000 times. Still referenced in test automation frameworks today.


Origin Story

The Long Road to a Viral Spreadsheet

2001

Graduated from Simon Fraser University, Canada with a BSc in Computer Science. Left Taiwan for academia, armed with a question: how do computers learn to see?

2004

Received his Master of Science from MIT, deepening his focus on vision-based user interfaces and human-computer interaction.

2009

PhD from MIT. Co-created Sikuli - a system that lets computers automate tasks by recognizing GUI elements from screenshots. One of the most cited HCI papers of the decade.

2010

Published VizWiz: nearly real-time answers to visual questions for visually impaired users. Over 1,200 citations. Released Sikuli as open-source; 200,000+ downloads follow.

2009-2012

Postdoctoral fellowship at the University of Maryland Institute for Advanced Computer Studies (UMIACS), bridging HCI, computer vision, and software engineering.

2012

Joined University of Colorado Boulder as Assistant Professor of Computer Science. Founded the Sikuli Lab (later the Imagine AI Lab).

2014

Received CU Boulder Student Affairs Faculty Member of the Year Award - an early signal that teaching, not just research, was central to his identity.

2015

Awarded the NSF CAREER Award - the National Science Foundation's most prestigious grant for early-career faculty.

2024

Launched AI by Hand series on LinkedIn as a one-month experiment. Gained 25,000 followers in weeks. Launched the Substack newsletter and Excel GitHub repository (1,300+ stars).

2025

AI by Hand Substack grows to 62,000+ subscribers; ranked #66 in Technology on Substack. 200,000+ combined social media followers.

2026

Launched AI by Hand Academy with structured video courses including Introduction to Agentic AI. Over 130 founding members joined in the opening months.


Academic Foundation

Where the Fundamentals Were Built

Simon Fraser University B.Sc. Computer Science Graduated 2001 • Vancouver, Canada
MIT M.S. Computer Science 2004 • Cambridge, Massachusetts
MIT Ph.D. Computer Science - Vision-Based User Interfaces 2009 • Advisor: Robert C. Miller
University of Maryland Postdoctoral Fellow, UMIACS - Human-Centered Computing, Computer Vision, Software Engineering 2009-2012

Recognition

Milestones Worth Noting

NSF

NSF CAREER Award (2015) - the National Science Foundation's premier grant for early-career researchers who exemplify both research excellence and educational impact.

CU Boulder Student Affairs Faculty Member of the Year (2014) - voted by students. An unusual honor for a computer scientist to receive.

CHI

Best paper awards and honorable mentions from CHI, UIST, SIGCSE, and MobileHCI - the top venues for HCI and computer science education research.

DARPA

Research funding from NSF, NIH, DARPA, the Knight Foundation, and the Piton Foundation - a portfolio that spans national security, public health, journalism, and civic tech.

#66

AI by Hand ranked #66 in Technology on Substack - placing it alongside some of the most widely read tech publications in the world, built without a media company behind it.

200K

Sikuli downloaded 200,000+ times after open-source release in 2010. One of the most influential GUI automation tools in the pre-Selenium era, and still widely referenced.


Behind the Work

The Stories That Don't Fit in a CV

The Colored Pens

Tom Yeh drew all 300 exercises in the AI by Hand workbooks himself - by hand, in colored pen, on printed paper. Each of the twelve workbooks contains twenty-five exercises. None of it was generated. He had a collaborator, Mohsena Ashraf, film tutorial videos by doing the exact same thing: printing out the workbook, picking up the pens, and writing through each page. The aesthetic of AI by Hand is not a choice. It's the point.

The Happy Mistake

Occasionally, students catch errors in Tom Yeh's AI by Hand solutions. His response? He's reportedly glad about it. It means the student understood the problem deeply enough to check the answer rather than just copy it. For a professor who has spent his career on interpretability and human understanding of AI, a student catching an error is not embarrassing - it's the whole idea working exactly as intended.

The One-Month Experiment

The AI by Hand series was never supposed to be a movement. Yeh started posting hand-drawn AI exercises on LinkedIn as a one-month experiment in early 2024. He expected moderate engagement from fellow academics. What he got was 25,000 followers in weeks and a community that now spans 200,000+ people across platforms, a Substack newsletter with 62,000+ paid and free subscribers, and an Academy with founding members from around the world.

VizWiz: A Decade Ahead

In 2010, Yeh published VizWiz - a system that let visually impaired users photograph an object with their phone and receive a real-time spoken answer to questions like "What does this label say?" or "What color is this?" The paper now has over 1,200 citations. It predated modern AI vision assistants by nearly a decade and established the research agenda that AI camera features on smartphones would eventually follow.

Computers Learning to See

Yeh's Sikuli project at MIT taught computers to interact with GUIs by recognizing interface elements from screenshots - the same way a human identifies a button by looking at it. Released as open-source in 2010, it was downloaded 200,000 times and became foundational to the field of visual GUI testing. It is essentially the intellectual ancestor of modern screenshot-based test automation.

The Excel Transformer

Yeh built a full implementation of the Transformer architecture - the foundation of every modern LLM - inside a Microsoft Excel spreadsheet. No Python. No GPU. Just columns, rows, and formulas. His description of the core math: "Combine columns (attention), combine rows (feed forward), and repeat." The repo gained 1,300+ GitHub stars shortly after launch, suggesting that more than a few engineers wanted exactly this explanation all along.


In His Own Words

Principles That Drive the Work

"What I cannot create, I do not understand." - Richard Feynman - the philosophy that turned a quiet professor into a viral AI educator.

"The core math of a Transformer model is to combine columns (attention), combine rows (feed forward), and repeat." - The clearest single-sentence description of the most consequential architecture in modern AI.

"AI by Hand disseminates first-principles, pen-and-paper approaches for understanding AI systems, broadening participation and improving public and professional literacy in artificial intelligence."


Quick Fire

Things You Might Not Know About Tom Yeh


Profile

Tom Yeh at a Glance

Full Name Tom Yeh (Pei Hsiu Yeh)
Origin Taiwan
Current Base Boulder, Colorado, USA
Occupation Professor, AI Educator, Newsletter Creator
Institution University of Colorado Boulder
Department Computer Science (+ ECEE, ATLAS, ICS)
Newsletter AI by Hand (byhand.ai)
Research Lab Imagine AI Lab, CU Boulder
Substack Rank #66 in Technology

Share This Profile LinkedIn X / Twitter Facebook Copied!