The data engineering problem nobody solved
There is a moment in every Raj Bains interview where he says something so blunt it stops the conversation. The version that keeps surfacing: "Data transformation has been identified as a bottleneck for two decades now." Not one decade. Two. And the tools, in his words, were from the Stone Age.
Bains grew up north of Delhi, in a family that treated career choices the way a good engineer treats a design decision: choose the approach that works best right now, and stay ready to adapt. That instinct - call it pragmatism, or just good pattern recognition - carried him through three continents, an internship in France, and a relocation to the United States before he landed at 3Dlabs in 2004 writing compiler optimizations at the JIT level.
What followed was a graduate program in applied obscurity: Microsoft Research, NVIDIA's founding CUDA team (where he developed multiple patentable algorithms and watched GPU computing become the backbone of everything that would later be called AI), then a pivot into products at Clustrix, then a spell managing Apache Hive at Hortonworks through its IPO. By the time he founded Prophecy in 2017, Bains had spent 13 years watching the same data engineering problem from every angle - systems, infrastructure, product, and enterprise sales - and concluded that none of the existing tools were going to solve it.
Before AI was a buzzword, Raj Bains was writing compilers for the chips that now run it. That history is not a footnote - it is the foundation of how Prophecy thinks about data infrastructure.
Prophecy's core insight is deceptively simple: make data transformation visual without faking it. Every drag-and-drop action in Prophecy's interface generates real, production-quality Apache Spark code, committed to Git, reviewable and versioned like any software project. No hidden DSL, no lock-in - just the code a senior engineer would have written, produced in a fraction of the time by someone who has never touched Scala in their life. That is what "low-code" means here: not less engineering, but engineering made accessible.
The company's AI layer extends this further. Prophecy's data transformation copilot lets a data analyst describe a pipeline in plain language - "build me a pipeline that does this" - and the system builds it. The output is still Spark. The difference is the floor just dropped, and ten thousand analysts who couldn't get a data engineering ticket prioritized can now ship their own pipelines.
"A data analyst using a visual drag and drop and some generative AI can just say: 'Build me a pipeline that does this,' and we build it for them."- Raj Bains, CEO of Prophecy • SiliconANGLE interview, January 2025
Twenty years of watching the problem
Raj Bains's career reads like a deliberate tour of the exact infrastructure that enterprise data engineering would eventually need to transform. He started as a compiler writer - the kind of engineer who cares about what happens at the level of registers and instruction scheduling, not just APIs. That training at 3Dlabs, and later at Microsoft Research working on the Visual Studio compiler, gave him an intuition for how abstraction layers either fail or succeed in practice.
NVIDIA was a turning point. Bains joined as a founding engineer on the CUDA compiler team - the group that built the software layer for GPU computing before anyone outside of a small research community knew why it would matter. He developed multiple patentable algorithms during that stint, and he was there before GPU computing was called AI infrastructure. That vantage point - being present at the creation of something fundamental - shaped how he thinks about timing and opportunity.
The pivot to products was not a retreat from engineering. At Clustrix he ran engineering alongside product and marketing simultaneously. At Hortonworks he managed Apache Hive, the SQL layer on Hadoop, through the company's IPO - getting a front-row view of how large enterprises actually consume data tools, what slows them down, and what they're willing to pay to fix. The answer: they'll pay a lot to fix data transformation, if someone builds a tool that doesn't require a PhD in Spark to operate.
One detour worth noting: at some point in his career, Bains filed a patent for designing a language for insurance contracts. A compiler specialist who pivots to domain-specific languages for financial contracts, then builds a visual programming interface for data engineers - the thread is consistent. He is attracted to the problem of making complex, formal systems legible to people who shouldn't need to be experts.
What Prophecy actually does
Prophecy is a low-code data engineering platform built for enterprise scale. The core product is a visual interface for building data pipelines on Apache Spark and Apache Airflow - but unlike most "visual" tools, everything in Prophecy compiles to open-source code that goes into Git. There is no proprietary runtime, no vendor lock-in, no hidden execution engine. The visual layer is a development environment, not a replacement for real engineering.
The platform came at the problem from a specific angle: the gap between data analysts who know the business logic and data engineers who know Spark. Prophecy's argument is that you don't need to close that gap by training analysts to write Scala - you close it by building a visual environment that generates Scala automatically, lets analysts build pipelines by describing what they want, and puts the output through the same Git-based review process as any other codebase.
The AI expansion followed naturally. Prophecy launched its generative AI copilot in 2023, letting users describe a transformation in plain language and get a working pipeline back. Prophecy v4, launched in early 2025, went further with AI agents capable of visual data prep and analysis. The platform also includes a transpiler that converts legacy ETL workflows from tools like Informatica, Ab Initio, and SSIS into modern Spark code - which is relevant to every enterprise sitting on years of Informatica investment and staring at a cloud migration that nobody wants to rewrite by hand.
Prophecy's transpiler converts legacy Informatica, Ab Initio, SSIS, and Alteryx workflows to Apache Spark - automating the grunt work that has blocked cloud migrations for a decade.
The customer base reflects the ambition: HSBC, JP Morgan, Microsoft, Toyota, Amgen, SAP, Ralph Lauren, Deutsche Telekom, Marks & Spencer. These are not pilot customers. They are enterprises with massive data engineering footprints using Prophecy to move those footprints into the cloud faster than manual rewriting would allow. The 2025 HSBC investment was notable precisely because the bank was already a paying customer - they liked the product enough to put money behind it.
In January 2025, Prophecy raised a $47M Series B extension led by Smith Point Capital, with HSBC joining as a strategic investor alongside JPMorgan, Berkeley SkyDeck, Dallas VC, Insight Partners, and SignalFire. Total funding crossed $114M. Revenue was doubling every two quarters as of late 2023, and FY2024 delivered 3.5X revenue growth with 160% net revenue retention - meaning existing customers are spending significantly more year over year.
What Raj Bains actually says
"Data transformation has been identified as a bottleneck for two decades now."
"The current tools were from the Stone Age."
"Enterprise spend has held, and Prophecy is doubling revenue every two quarters."
"We can just take a big company with a massive data engineering footprint and move the entire operation through the cloud."
"A data analyst using a visual drag and drop and some generative AI can just say: 'Build me a pipeline that does this,' and we build it for them."
"Whatever is best to do."
(his family's guiding philosophy - and, evidently, his own)
"Most importantly, data analysts and other non-coders can no longer serve themselves."- Raj Bains • On the real cost of inaccessible data tooling
Built different, not born different
What separates Raj Bains from most enterprise software CEOs is the depth of the technical layer he operates from. He is not a former banker who hired engineers, or a product manager who outsourced the hard parts. He wrote compilers. He debugged GPU instruction pipelines. He managed the Apache Hive codebase. When he tells an enterprise customer that Prophecy generates real Spark code and commits it to Git, he understands every word of that sentence at a level most founders do not.
That credibility matters in a market full of low-code promises that don't survive contact with production. Prophecy's early adopters were data engineers - the exact people most likely to reject visual tools as toys. Winning them over required a product that could survive their code review. Bains built one.
The co-founders bring complementary depth: Maciej Szpakowski as CTO, Vikas Marwaha, and Rohit Bakhshi. The board now includes Herb Cunitz, the former president of Hortonworks - Bains's old employer, and one of the people who watched him develop his product instincts up close. Elena Zislin from J.P. Morgan joined as a board observer in 2023, the same year JPMorgan participated in the Series B as a strategic investor and customer.
Bains's personal story does not fit a single geography or a single industry. He has lived on three continents, held positions in hardware compilers, cloud databases, open-source data infrastructure, and enterprise SaaS. The family philosophy - do whatever is best to do - turns out to be reasonable preparation for founding a company in a market that keeps changing its name (big data, Hadoop, Spark, cloud data lakes, now AI pipelines) while staying, at its core, the same unsolved problem.
- Founding engineer, NVIDIA CUDA compiler team
- Multiple CUDA-related patents
- Patent: domain-specific language for insurance contracts
- Led Hortonworks Apache Hive through IPO
- Raised $114M+ across 6 funding rounds
- 3.5X revenue growth in FY2024
- 160% net revenue retention
- Fortune 50 customers in banking, pharma, auto, tech
Raj Bains on record
The goal is not a better Informatica. The goal is to make the concept of "data engineering as a specialist skill" obsolete.- The Prophecy thesis, as Raj Bains tells it
Bains sees Prophecy as infrastructure, not software. The long-term play is that every enterprise running AI - which is to say, eventually every enterprise - needs a data transformation layer that can keep up with the rate of change in AI tooling. Informatica was built for a world of batch ETL and on-premise warehouses. Prophecy is built for a world where the question of where your data lives, and how you transform it, changes every two years.
The AI copilot and agent layer announced in 2025 point at the next version of the argument: not just visual pipelines for engineers, but pipelines that can be specified by anyone who can describe what they want. That is a much bigger market than data engineering. It is, in effect, a claim that Prophecy can serve as the execution layer between business intent and data infrastructure - a position that would make it foundational in the same way that compilers, once mastered, become invisible and indispensable.
The man who helped build CUDA before GPU computing had a use case beyond gaming has a habit of arriving early and staying patient. Prophecy is seven years old. The enterprise cloud migration - the core driver of Prophecy's market - is perhaps a third done. The next wave is AI-native pipelines. Bains has been in the right place before.