Featured

Forged in Rust, Powered by Thought: Embedding LLMs into Our Custom Database

jeanepaul

07 Jul 2025 • 3 min read

In the age of AI-first architectures, the line between structured and unstructured data is dissolving fast. At our company in Tokyo, we asked a radical question:
What if your database didn’t just store data—but understood it?

So I built one. An intelligent database engine written in Rust, with PostgreSQL as its rock-solid backend. But the twist? It behaves like a NoSQL system—powered by a mind of its own.

Welcome to our journey building a database that thinks.

🛠 Why Build a New Database at All?

PostgreSQL is excellent—but rigid schemas, verbose queries, and shallow text search can feel like relics in an AI-native world. Modern teams demand:

Schema-less flexibility and natural interfaces
Hybrid structured + semantic retrieval
Zero-boilerplate RAG capabilities
Local and secure AI inference in production

Rather than layering complexity on top of an existing RDBMS, I reimagined the interface—preserving the robustness of Postgres but augmenting it with language-native intelligence and a NoSQL-like façade.

🚀 The Tech Stack: Rust, PostgreSQL, k3s, and Local LLMs

Engine Core: Rust delivers memory safety and blazing performance. I use it to build parsers, optimizers, and LLM pipelines natively.
Storage Layer: PostgreSQL powers durable persistence and indexing, including structured joins and embedding lookups.
LLM Integration: I embed models like Ollama or Gemma 3B, running them in containers managed by our lightweight k3s cluster.
Query Layer: Our LLM generates SQL—even JOINs and nested conditions—from natural language prompts.

🧠 Use Case #1: Natural Language Queries (Even With Complex Joins)

Our users don’t need to write SQL. They talk to the database in plain language—and our LLM translates it into optimized internal queries.

“Show me all customer records who bought from Tokyo last month and haven’t purchased since.”
→ The engine translates this into a SQL query using joins and date filters under the hood.

Whether it’s filtering by metadata, joining across user behavior logs, or surfacing edge-case anomalies—our LLM knows how to ask the right questions for you.

🧩 Use Case #2: Native RAG Support

Out of the box, our engine supports Retrieval-Augmented Generation:

Upload documents as vectorized embeddings
Filter them semantically with contextual prompts
Pass the top results into your LLM-powered app

We use k3s-native Jobs to periodically update embeddings and metadata, and we support hybrid filters (e.g. “only return docs written by JeanePaul after 2024”).

Perfect for chatbots, knowledge bases, agent workflows, or search-driven apps—without standing up a separate semantic stack.

🧪 Use Case #3: AI-Powered Data Generation

Need sample data for testing? Missing fields in a scraped dataset? Our engine can synthesize clean, realistic entries using local AI models.

It works like this:

Detect missing or incomplete data patterns
Send context into the embedded LLM
Receive enriched or synthetic data rows to populate your tables

This makes onboarding, testing, and simulation radically faster—without depending on external generators or handcrafted logic.

💡 Developer Experience

You’re free to use it like a document store—just POST a JSON and query by meaning. But peek under the hood, and you're backed by full SQL compatibility. That means:

Seamless joins
Transactions
Postgres extensions
Even full-text + vector hybrid queries

// Rust-like pseudocode for natural language interface
let result = db.query("Find all Tokyo-based users with active subscriptions", {
   limit: 50,
   embeddings: true
});

// TypeScript - Natural Language Query via REST
async function queryDatabase(nlQuery: string) {
  const response = await fetch('http://your-db-api.local/query', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ prompt: nlQuery })
  });

  const result = await response.json();
  console.log('Query Result:', result);
}

// Example usage
queryDatabase("List all active users in Tokyo who purchased in the past 30 days");

// JavaScript - Upload document for semantic indexing
async function uploadDocument(doc) {
  const response = await fetch('http://your-db-api.local/documents', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      content: doc,
      metadata: { source: "website", category: "product_manual" }
    })
  });

  const data = await response.json();
  console.log('Upload Successful:', data);
}

// Example call
uploadDocument("The QuickStart VX-200 device requires an input voltage of 12V...");

It’s the best of both worlds—schema-aware when you want it, fluid when you don’t.

🔭 Looking Ahead

I am currently exploring:

Native WebAssembly for in-query inference
CLI copilots for debugging embedded logic
Open-sourcing key components, with docs and DX in mind

🧬 Final Thoughts

This isn’t a bolt-on AI plugin. It’s a database that thinks—shaped by the mindset of natural language, data synthesis, and machine reasoning.

Rust gave us the safety and speed to build boldly. PostgreSQL gave us consistency. And LLMs made it truly intelligent.

It’s not just a database.
It’s an interface between your data—and your thoughts.