An AI-powered data engineering platform Fortune 50 banks and pharma giants quietly bet on - turning drag-and-drop into production Spark and SQL.
Not a slide. Not a roadmap. A real pipeline - reading raw transaction data from an S3 bucket, joining it against three reference tables in Snowflake, scoring it for fraud, and writing results to a dashboard the compliance team will check in the morning. The person who built it isn't a Spark expert. She drew it on a canvas, asked an AI to fill in the messy parts, and pressed run. The code that shipped is plain Spark, sitting in Git, reviewable by anyone on the team.
That is Prophecy in 2026. Not theory. Not a demo. The Palo Alto company has spent eight years turning data engineering into something that resembles design - and the data engineers, mostly, have stopped fighting it.
For most of the last decade, every company that wanted to use its data well hit the same wall. The business asked for an answer. An analyst wrote some SQL. A data engineer translated that SQL into Spark, scheduled it, monitored it, and re-translated it the next time anyone changed their mind. The pipeline became a private dialect spoken between two people, fragile and undocumented, and when one of them left the company, so did the dialect.
The industry's answer was either to hire more engineers - expensive, slow - or to buy a no-code tool that hid the code entirely. The no-code tools were faster but produced output the engineering team couldn't review, version, or own. The choice was between speed without governance, and governance without speed.
Prophecy looked at this and decided the framing was wrong. The fight was never visual versus code. It was visual or code, when it should have been both, simultaneously, edited by the same team, on the same artifact.
Raj Bains is not a typical data-infrastructure founder. He spent years at Nvidia writing compilers for CUDA - the tooling that taught GPUs to do general-purpose math - and then product-managed Apache Hive at Hortonworks through its IPO. The first job teaches you that abstractions either lower correctly to fast machine code, or they don't. The second teaches you what enterprise data teams will and won't tolerate.
The bet he made with co-founders Maciej Szpakowski and Vikas Marwaha was simple and slightly heretical: a visual editor and hand-written code can be the same file. Move a box on the canvas, the Scala or SQL underneath updates. Edit the Scala by hand, the canvas redraws. No lock-in. No proprietary runtime. Open code, in your Git, on your cloud.
Raj has said in interviews that he wanted to build "the Apple of data engineering" - opinionated design taste on top of open standards. The data world generally finds the Apple comparison faintly amusing. The customers seem to find it persuasive.
What Prophecy actually sells is a development environment that compiles in two directions at once. There is a copilot for natural language, a visual canvas for assembly, a code editor for control, and a deployment layer that ships everything to the customer's cloud. The four parts of the product line look like this in catalogue form:
Describe the goal in English. The AI proposes a workflow on the canvas - and the underlying Spark or SQL. Editable, reviewable, not magical.
Self-hosted for big data teams. Git, CI/CD, lineage, governance, role-based access. The grown-up edition.
The cloud SaaS tier. Smaller teams skip the infrastructure setup and ship pipelines the same afternoon.
Business users describe outcomes. Agents propose pipelines. Engineers approve them. Everyone keeps their job.
You can tell a lot about a data infrastructure company by the logos that bother to put their name on the marketing page. Hype tools attract early-stage startups. Real tools attract regulated industries that can't afford to be wrong. Prophecy's published list, as of 2026, tilts heavily toward the second group.
The Series B1, closed January 2025, was led by Smith Point Capital - the firm founded by former JPMorgan board director Jeff Smith. Insight Partners, SignalFire, JPMorgan, and Berkeley SkyDeck all wrote follow-on checks. HSBC, already a customer, decided it would also like to be a shareholder. That kind of order of operations - customer first, investor second - tends to be a tell.
The official line out of Prophecy is that the company wants to make data engineering accessible to every business. The unofficial line, repeated across podcasts and product pages, is more pointed. The data tooling stack is too hostile to the people who actually need it. Analysts wait on engineers. Engineers maintain code nobody else can read. Business users get reports that arrive a week late. The result is companies that paid a fortune for cloud data warehouses and use them at a fraction of their capacity.
Prophecy's answer is that the cure is not removing engineers, but removing translation. Put the analyst, engineer, and AI agent on the same canvas. Let each work in their preferred medium. Compile to one shared artifact. Ship to one shared cloud. The thesis is that productivity is what you get when you stop forcing your teammates to speak each other's language - because the tool already does.
The size of the bet is worth naming. Cloud data warehouses - Databricks, Snowflake, BigQuery - have collectively raised tens of billions of dollars on the premise that storage and compute would get cheap and abundant. They were right. They got cheap and abundant. What did not get cheap and abundant was the human labor required to actually move data through them. That gap, between cheap infrastructure and expensive expertise, is the gap Prophecy is selling into.
AI does not close that gap on its own. A copilot that hallucinates a join condition is worse than no copilot at all, especially when the join is between a customer table and a regulatory filing. Prophecy's wager is that AI is useful only when it produces output a human can read, review, and version. Visual canvas plus open code plus AI suggestion - all three, or the whole thing breaks.
If that wager pays, the next decade of data engineering looks different. More analysts shipping pipelines on their own. Engineers reviewing instead of writing. AI agents handling the parts everyone hated anyway. Whether Prophecy is the company that owns this shift, or simply the company that defined it early, will depend on whether the Fortune 50 logos keep renewing and the smaller teams keep adopting.
The pipeline she shipped before lunch is now running on a schedule. It writes its output into a table, the dashboard updates, and the compliance team in London reviews it without ever needing to know whether a human or an AI agent wrote the underlying Spark. The canvas is still open in her browser. The code is in Git. If she leaves the company tomorrow, the next person will be able to read it.
That is the small, unflashy promise Prophecy makes - and the reason the logos keep adding up.