The Waterline Development Story: How a Startup Built Its Own AI Model Filtering System to Avoid Costly Mistakes

Water companies have a reputation for being rapacious rent-seekers, but they can also be unwittingly generous due to their penchant for prioritizing popularity over quality. This legacy of commercial haste has led to the development of a fix by Waterline Development, a water desalination startup.

Avoiding Costly Mistakes with AI

Derek Bednarski, founder and CEO of Waterline Development, shared with The Register that when his company tried to use large language models for materials science research, they were confidently wrong in ways that cost them months. The company was trying to build a desalination product that was essentially a water battery – charging the cell would remove ions like salt from the water.

Bednarski explained that they were debating between carbon cloth and cast carbon electrodes. Not being PhDs in the space, they read relevant academic papers and used LLMs like Grok and ChatGPT to validate their findings. They chose carbon cloth, which is heavily used in academic papers like the Stanford dissertation they based their initial prototypes on, due to commercial availability.

However, this material turned out to have issues that didn’t exist for cast carbon electrodes, including poor conductivity, water retention that affected ion removal, and poor durability. Bednarski stated that while they were not solely relying on LLMs, they did influence their research meaningfully. LLMs chose statistics from various papers and fields and put them together in ways that were plausible enough.

The Problem with Commercial AI Models

The problem Waterline Development encountered is that commercial AI models are ill-suited to multidisciplinary research, which requires synthesizing expertise from a variety of fields. No single AI model does this reliably, and frontier language models hallucinate under extended multi-step reasoning. They produce plausible answers that silently break when a problem crosses domain boundaries.

At best, this wastes time; at worst, it poisons critical decision-making. Time to build a better solution. Rather than trying to integrate domain-specific tools or to make the work of human expert teams more efficient, Waterline created Rozum, a multi-model reasoning system that operates various AI models in parallel and synthesizes their answers through a verification layer.

Rozum: A Model Orchestration System

Rozum is a model orchestration system that operates at inference time. It relies on an ensemble of commercial models, open weight models, and domain-specialized models. These models each process the queries they receive using tools that perform verifiable operations and provide deterministic results that serve to ground answers.

The tool passes answers through a verification layer designed to detect and correct errors and hallucinations, errant claims, miscalculations, and phony citations. Rozum uses a deterministic verification process to advance a final answer based on the evidence and reasoning from the ensemble of models it employs.

The Benefits of Rozum

According to the white paper, the system can come up with correct answers from a set of partial truths, even if no individual model has the complete, correct answer. Bednarski stated that Rozum is not focused on correcting LLMs to the extent they can be used for, say, critical engineering work like bridge construction. Rather, the goal is to empower researchers, engineers, and scientists so they can do their jobs better.

Rozum can spend minutes or even hours working on its responses, much more time than commercial AI models like Gemini 3.1 Pro or GPT 5.4 require with native tools. So it’s not well-suited for real-time conversations, high-volume commodity queries, or tasks where current frontier models perform adequately.

However, the cost probably isn’t consequential for the kind of projects to which Rozum is best suited. Bednarski stated that it does cost more than running a single frontier model. However, Rozum is being used by early customers for high-stakes questions and decision-making, like a $3M dollar solar investment or allocating months of engineering time towards one R&D priority or another.

Results and Conclusion

Rozum outscored GPT-4, Grok 4, and Gemini 3.1 Pro on the Humanity’s Last Exam benchmark by several percentage points or more in every category but one. When they ran 1,000 PhD-level benchmark questions through the pipeline, the verification layer flagged unsupported claims in 76.2 percent of frontier model responses and couldn’t confirm cited sources in 21.3 percent. Only 5.5 percent of questions produced clean consensus across all models.

That consensus rate – 5.5 percent – underscores how variable AI model responses can be and why AI alone is not enough. Rozum debuted last week and is currently offered through a wait list. The Waterline Development story serves as a reminder that while AI can be a powerful tool, it’s not a replacement for human expertise and judgment. By building its own AI model filtering system, Waterline Development has taken a significant step towards avoiding costly mistakes and achieving better outcomes in its research and development efforts.