Private AI for business: how to choose, GDPR, 10 questions
In short. Private AI for business can run locally or use hosting under an appropriate agreement. This guide focuses on local processing, compares three deployment options and gives ten questions for a supplier. Local operation alone is not proof of security or GDPR compliance: inspect access, external connections, storage and the actual use.
Private AI describes a controlled way to use AI, which may be locally operated or hosted under an appropriate agreement. This guide focuses on local deployment for internal documents. Its suitability depends on the data, access controls and the entire processing chain, not the label. Sensitive or regulated uses require a specific assessment. If you are weighing a model on your own machine, our guide to a local LLM for small business covers model sizes and seven questions to ask first.
It draws on the CNIL page on deploying generative AI (published in July 2024), on the text of the GDPR and on the llama.cpp documentation, all read on 28 October 2026. We install local AI in our studio in Nice, and the question grid below comes from that work.
What are the three ways to use AI in a business?
| Consumer online chatbot | Professional cloud or API offer | Local AI (computer or server) | |
|---|---|---|---|
| Where your documents go | to the provider’s servers | to the provider’s servers, under contract | stay on your machine or approved storage |
| Legal framework | the provider’s standard terms | a processor contract to read and sign | The controller duties remain; assess any outside processor |
| Shape of the cost | free or a per-person subscription | per user or per token | hardware and setup, no per-token charge |
| Suits | non-sensitive content | varied uses without strong confidentiality needs | confidential documents, repetitive tasks |
| Weak point | you control neither the processing nor the reuse | dependence on the provider and its sub-processors | hardware, setup and upkeep are on you |
None of the three is “the right one”. What matters is the kind of document you put into it.
What do GDPR and the CNIL say?
First establish the roles for the particular processing. A supplier processing personal data on your behalf may be a processor; it is not automatically one for every use. Where that relationship applies, assess the following requirements with your data protection adviser:
-
Sufficient guarantees. Article 28 says to use only processors providing sufficient guarantees of appropriate technical and organisational measures.
-
A contract. The processing is governed by a contract: the processor acts only on documented instructions from the controller, its staff are bound by confidentiality, and it deletes or returns the data at the controller’s choice.
-
Transfers. Under Article 44, a transfer of data to a third country is allowed only if the conditions of the Chapter on transfers are met. So ask where the data is processed.
The CNIL, in its July 2024 page, recommends starting from a concrete need rather than deploying generative AI with no precise aim, defining authorised and forbidden uses (for example not giving personal data to the system), training users, and involving the data protection officer, the head of information systems and the business managers early. It favours local, secure and specialised systems. Failing that, you need to assess whether the provider can reuse the data given to the system and adapt the use accordingly. It also reminds readers that such systems can produce answers that are inaccurate but plausible.
When does a local AI make sense?
A local AI is mostly justified in three cases:
-
Confidential documents, such as contracts, client files or patient records, that have no reason to leave your computer.
-
Repetitive tasks, such as transcribing recordings, extracting tables from PDFs or answering frequent questions from your own documents. See for example our article on PDF invoices to Excel.
-
A cost that must stay predictable, with no billing per token or per user.
It makes less sense for occasional drafting of non-sensitive text, or if nobody can look after installation and updates. In that case a well-contracted professional offer may be more sensible.
What do you need to run AI locally?
The llama.cpp project gives a good reference. Its aim is to run language models without a heavy setup on very varied hardware: processors, Apple silicon, Nvidia and AMD graphics cards, among others. It offers quantisation from 1.5 to 8 bits, which speeds up execution and reduces the memory used, and shares work between the processor and the graphics card for models larger than the video memory. It also provides a server compatible with the OpenAI interface, so existing tools can connect to it.
The takeaway: the bigger the model, the more memory it needs, and the right hardware depends on the use. We do not give a universal figure: you find it by testing your own documents.
A practical acceptance test before using real documents
Start with fictional or authorised non-sensitive files. For a document assistant, ask a question whose answer is in a named source, then another whose answer is absent. The first answer should point to the correct passage; the second should say that the information was not found. A citation marker alone does not prove the answer is correct: open the source and check it. For a chatbot on your own website, a 20-question pre-launch test goes further.
Before handover, agree how to test user permissions, the deletion of an indexed document, retention of logs and a backup restore. If offline operation is required, test with external connectivity disabled after dependencies have been installed. Check OCR, embeddings, browser tools, telemetry, backups and fallback models: a locally running model does not make these components local.
Record the configuration, expected result, observed result and unresolved issues. These are proposed acceptance checks, not tests we ran on a customer’s installation. Keep human approval before sending a message, changing a business record or taking another consequential action.
Local inference may avoid a provider’s per-token bill, but hardware, power, maintenance, software licences and optional cloud services can still cost money. Check model and software licences, permitted business uses, concurrency and response times before choosing equipment. A regulated or sensitive use needs its own assessment, not a general-purpose setup promise.
Which questions should you ask an AI provider?
This is the grid we use, for a cloud offer as much as for a local installation. A vague answer to any of these questions is already information.
| Question | Healthy answer | Warning sign |
|---|---|---|
| Where are my documents processed? | a precise place, in writing | “in the cloud” and nothing more |
| Where and for how long are they kept? | a fixed period, deletion possible | unlimited retention |
| Is my data used to train or improve the model? | no, in writing | silence or “maybe” |
| Is there a processor contract (Art. 28)? | yes, with documented instructions, confidentiality, deletion or return | general terms only |
| Which sub-processors are involved? | a known list | an unknown or refused list |
| Are there transfers outside the European Union (Art. 44)? | no, or framed and explained | no answer |
| Who can access my data remotely? | nobody without your agreement, access logged | permanent access not described |
| Are accesses and actions logged? | yes, and you can read the logs | no log |
| Can I retrieve and delete my data? | yes, with a written procedure | export impossible |
| What is guaranteed? | the work and the acceptance tests | “GDPR compliant” as a guarantee |
The last line matters: nobody can guarantee the compliance of your processing for you. A serious provider will tell you so.
What are the limits of a private AI?
-
It is not a compliance certificate. You remain the controller, and the setup replaces neither your data protection officer nor a lawyer.
-
Errors remain possible. A local model can also produce answers that are inaccurate but plausible, as the CNIL points out. Keep a human check.
-
Quality depends on the model and the hardware. For some tasks a model run on one computer can perform below the largest hosted models. You check this on your documents before committing.
-
There is upkeep: updates, backups and access rules, to plan for from the start.
-
“Offline” is a configuration option, not an audited security standard.
How does Castel Studio work on private AI?
We specify, install and teach the use of a private AI workstation: a written specification and fixed quote, a hardware list you approve before purchase, a local setup for documents, search and drafting, and training at handover. The starting price is 2,900 € before hardware, with no per-token or per-user charge. An acceptance test is agreed in writing before the build. For narrower needs, the local AI page offers among other things an assistant on your own documents from 790 € and transcription without the cloud from 90 € per batch.
If you are unsure between a cloud offer and a local installation, describe your documents and uses in a few lines, without sending sensitive files through the form. We answer in writing within 48 hours, including a plain “local AI is not the right answer” when that is the case.
No per-token billing means no such charge for the selected local inference engine, not zero running cost. Hardware, electricity, maintenance, licences, support and optional remote components must be specified separately in the quote.
Questions
Is a local AI automatically GDPR compliant?
No. A fully local configuration can avoid sending documents to an AI service, but connected tools, remote support and backups must also be checked. Legal basis, transparency, security and retention remain relevant. Local deployment is not a compliance certificate; involve your data protection officer or legal adviser where needed.
What is the difference between private, local and on-premise AI?
They are different concepts. Local describes where processing runs; on-premise means in your own premises. Private describes access or contractual arrangements and can include hosted infrastructure. Ask who operates the system, where data goes and who can access it.
Can I use an online chatbot with client data?
Start with the data, purpose, legal basis and the actual roles of each party. If a supplier processes personal data on your behalf, assess the Article 28 processor agreement; also check any applicable international-transfer requirements. A standard consumer chatbot is not automatically suitable for confidential client files.
Do I need a big server to run AI locally?
Not necessarily, but memory matters. Projects such as llama.cpp run on processors, Apple silicon and graphics cards, with quantisation that reduces the memory needed, and can share work between processor and graphics card for models larger than the video memory. The right hardware depends on the model and the use, and you find it by testing your own documents.
How much does a private AI setup cost?
At Castel Studio a PC ready for local AI starts from 390 € before hardware, an assistant on your own documents from 790 €, and a private AI workstation from 2,900 € before hardware, with no per-token or per-user charge. A team server or a full system is priced by written quote.