What “private” has to mean before it means anything
Almost every AI product now claims to be private, and almost all of them mean the same thing by it: that a company has promised to look after your data properly. That promise asks you to trust their staff, their contracts, their breach response, their future owners, and whichever government can eventually compel them.
There is a stronger version available, and it is worth naming clearly because it is the whole point of building your own. A system where the data never arrives cannot leak it, sell it, train on it, or be forced to produce it. Not because anyone behaved well, but because there is nothing held to produce.
The question is not whether a company will look after your documents. It is whether they ever receive them.
That distinction is the one thing to hold on to while reading the rest of this. Every technical choice below is downstream of it.
The three parts you actually need
A private assistant that can answer questions about your own material is not one piece of software. It is three, and understanding the split is what makes the privacy question answerable instead of a matter of faith.
1. A model that runs where you choose
The model is the part that writes the answer. What matters here is that it is open weight: published so that anyone can download it, inspect it, and run it without asking permission. The Qwen3 family is the usual recommendation and the one we use, released under Apache 2.0. Llama and other families work too.
Open weight matters for a practical reason rather than an ideological one. A model you hold as a file on your disk can be run offline, checked against the public release, and kept working after whoever published it loses interest. A model behind an API cannot be any of those things. The usual way to run one locally is Ollama, which is open source and handles fetching and serving the model for you.
2. An index of your own documents
This is the part people are surprised by, and it is where the privacy is actually won or lost. Your assistant does not learn your documents, and it is not trained on them. Instead the documents are split into passages, each passage is converted into a numerical representation by a small embedding model, and those representations go into a searchable store, often called a vector database.
The important property: that index is a set of files in a folder. It can live entirely on your own disk, and on a properly built system it never leaves, on any tier. Because nothing is trained, there is no point at which your documents get absorbed into a model that someone else holds.
3. A retrieval step, so only fragments travel
When you ask a question, the system searches your local index, pulls out the few passages that look relevant, and hands the model your question plus those passages. The model answers from what it was handed. This pattern is called retrieval-augmented generation, and it is why a small local model can outperform a far larger cloud model on your own material: it is reading your actual documents rather than trying to remember them.
It is also the part that determines what leaves your machine. If the model runs locally, nothing does. If the model is hosted, the question and those few passages are what travel, and nothing else.
Building it yourself
This is entirely achievable with open-source parts, and if you want the version with no account, no subscription and no company involved at all, this is the honest route. Roughly:
- Install Ollama and pull an open-weight model sized to your machine.
- Pick an embedding model for turning your documents into searchable representations, and run it locally too.
- Stand up a local vector store to hold the index, and write the ingestion step that reads your PDFs, Word files and notes, splits them sensibly, and fills it.
- Write the retrieval and prompting layer that searches the index and passes the results to the model.
- Put an interface on it, and keep every one of those parts updated as versions move underneath you.
The part most guides skip. Steps one and two are an afternoon. Steps three to five are a project, and then an ongoing one. The difficulty is not any single component; it is that document parsing is genuinely messy, sensible chunking changes answer quality more than model choice does, and the whole stack moves fast enough that a working setup drifts if you leave it alone.
That is worth saying plainly rather than implying it takes ten minutes. If you enjoy this kind of work, it is a good project and you will understand your own system completely. If you do not, that is the gap a packaged product is for.
What actually travels, in both modes
This table is the whole privacy question, stated without adjectives. It describes a correctly built system, ours included.
| What | Model runs locally | Model is hosted |
|---|---|---|
| Your documents | Never leave your disk | Never leave your disk |
| The index built from them | Never leaves your disk | Never leaves your disk |
| Your conversations | Written to your disk | Written to your disk |
| Your question | Stays on your machine | Sent to the model, processed, not stored |
| Retrieved passages | Stay on your machine | Sent with the question, processed, not stored |
The uncomfortable row is the last one, and it deserves saying out loud: on a hosted model, excerpts from your documents genuinely do travel over the internet every time you ask something. They are processed and discarded rather than filed, but “not retained” is not the same as “never sent”, and anyone telling you otherwise is being careless with your expectations. If your situation does not tolerate transit at all, run the model locally. That option sends nothing.
What it costs you
Two real costs, neither of them money.
Hardware. Machines with 8 to 16GB of RAM run a smaller open-weight model comfortably, which is plenty for answering questions about your own documents. 16 to 32GB opens up the larger models and bigger collections. Below 8GB, a local model will be frustrating and a hosted one is the sensible choice.
Capability, at the top end. For general-purpose reasoning, the largest cloud models still have an edge, and it would be silly to pretend otherwise. For the specific job of answering questions grounded in a defined set of documents, open-weight models are genuinely excellent, and your assistant has an advantage none of the cloud ones do: it already knows your material, persistently, without you pasting it in again every session.
Where Ngaro fits
Everything above is true whether or not you ever use our product, which is why it is written that way. Ngaro is the packaged version of exactly that architecture: you describe what you want in plain English, add your PDFs, Word files, notes or links, and get a desktop app for Windows or macOS. No command line, no config file, and it fetches the model for you.
The parts we are honest about: it requires sign-in and a current subscription, and there is no offline mode. If a self-hosted Ngaro is running on your machine and we ever cease trading, it keeps working, free, permanently, because the model and your index are already on your disk. We cannot promise that for the hosted tiers, since those answers come from rented hardware that stops when the bills do.
We are in private preview and not yet open to the public. If you want to know precisely how each claim here is enforced, the security page goes through the mechanism behind each one and says where you simply have to trust us.
The short version
Build it out of three parts, keep the index on your own disk, and be clear-eyed about the one case where fragments travel. Do that and “private” stops being a promise somebody made you and becomes a property of the thing you are running.