Using credgate with local LLMs (Ollama, vLLM)

credgate removes personal data from text before it reaches a language model. This vignette shows the workflow with models served locally by Ollama or vLLM, using ellmer. All data is synthetic.

Why redact when the model is local?

“Local” does not by itself make a workflow compliant:

a vLLM server is often shared by several teams;
prompts end up in server logs and request traces;
a RAG index keeps every chunk it is given, PII included;
fine-tuning sets built from chat logs carry PII forward;
model answers copy PII into reports and tickets;
the DPDP Act’s data minimisation and purpose limitation apply wherever the model runs.

1. Preview, then redact

Always look at what will be sent first:

library(credgate)

note <- paste(
  "Customer name: Priya Nair. Called about her home-loan EMI. PAN ABCPN1234K,",
  "mobile +91 98765 43210, UPI priya.n@oksbi. She wants the EMI date moved",
  "to the 10th because her salary is credited late."
)
preview(note, min_confidence = 0.5)
#> 4 items would be redacted:
#>   Customer name: [PERSON_NAME]. Called about her home-loan EMI. PAN [PAN], mobile [MOBILE_IN], UPI [UPI_ID]. She wants the EMI date moved to the 10th because her salary is credited late.
#> 
#>  placeholder   value           confidence
#>  [PERSON_NAME] XXXXX XXXX      0.80      
#>  [PAN]         XXXXXX234K      0.97      
#>  [MOBILE_IN]   +XX XXXXX X3210 0.97      
#>  [UPI_ID]      XXXXX.X@Xksbi   0.99      
#>  reasons                                                    
#>  matched Person name pattern                                
#>  matched PAN pattern; context 'PAN'; holder type: individual
#>  matched Indian mobile number pattern; context 'mob'        
#>  matched UPI ID pattern; context 'UPI'; known UPI handle

redact() returns the text that is safe to send; identifiers become placeholders naming their type:

safe <- redact(note, min_confidence = 0.5)
safe
#> [1] "Customer name: [PERSON_NAME]. Called about her home-loan EMI. PAN [PAN], mobile [MOBILE_IN], UPI [UPI_ID]. She wants the EMI date moved to the 10th because her salary is credited late."

min_confidence = 0.5 leaves low-confidence matches alone (for example a 12-digit order number that happens to pass the Aadhaar checksum, scored 0.42 because of the word “order”). Use 0 to redact every match.

Check the preview. Names are found next to an honorific, a relation or a field label (Shri, S/o, Name:, नाम:). A name in running text without such a cue - “Customer Priya Nair called” - is not detected yet, and would reach the model.

2. Ollama

library(ellmer)

chat <- chat_ollama(model = "gemma3:1b")
chat$chat(paste("Summarise this call note in one sentence:", safe, sep = "\n"))

The model only ever sees [PERSON_NAME], [PAN], [MOBILE_IN] and [UPI_ID].

3. vLLM (or any OpenAI-compatible server)

vLLM serves the OpenAI-compatible API. ellmer’s chat_vllm() talks to it; set VLLM_API_KEY if your server checks keys (local servers usually do not, but ellmer requires a value).

if (!nzchar(Sys.getenv("VLLM_API_KEY"))) Sys.setenv(VLLM_API_KEY = "unused")

tickets <- c(
  "Shri Ramesh Kumar (Aadhaar 5422 2937 1182) reports a failed UPI payment to ramesh@okaxis.",
  "Account 50100987654321, IFSC HDFC0004567: customer asks to update email to [email protected]."
)
chat <- chat_vllm(
  base_url = "http://localhost:8000/v1",
  model = "meta-llama/Llama-3.1-8B-Instruct",
  system_prompt = "Classify the ticket as PAYMENTS, PROFILE or GENERAL. Answer with one word."
)
for (ticket in tickets) {
  print(chat$clone()$chat(redact(ticket, min_confidence = 0.5)))
}

The same code works with Ollama’s OpenAI-compatible endpoint (base_url = "http://localhost:11434/v1"), LM Studio, a llama.cpp server or an internal gateway.

4. Getting the real values back: tokenise and restore

Redaction is one-way: the model’s answer can only say [PAN]. When the answer should name the customer - a drafted reply, a summary for a case file - tokenise instead. Each identifier becomes a numbered token, the vault remembers what it stands for, and restore() puts the values back:

vault <- cg_vault()
safe <- tokenise(note, vault, min_confidence = 0.5)
safe
#> [1] "Customer name: [PERSON_NAME_1]. Called about her home-loan EMI. PAN [PAN_1], mobile [MOBILE_IN_1], UPI [UPI_ID_1]. She wants the EMI date moved to the 10th because her salary is credited late."

answer <- chat_ollama(model = "gemma3:1b")$chat(
  paste("Draft a two-line reply to this customer:", safe)
)
restore(answer, vault)

Without a model at hand, this is what restore() does with a typical answer - including tokens the model copied loosely:

restore("Dear [PERSON_NAME_1], we have noted PAN [ pan_1 ] and will call MOBILE_IN_1.", vault)
#> [1] "Dear Priya Nair, we have noted PAN ABCPN1234K and will call +91 98765 43210."

The same identifier always gets the same token, so the model can still tell two customers apart. restore(strict = TRUE) stops if the answer contains a token the vault does not know. The vault prints with values masked:

vault
#> Vault with 4 tokens (values masked; use `print(x, reveal = TRUE)` to see them):
#>            token           value
#>  [PERSON_NAME_1]      XXXXX XXXX
#>          [PAN_1]      XXXXXX234K
#>    [MOBILE_IN_1] +XX XXXXX X3210
#>       [UPI_ID_1]   XXXXX.X@Xksbi

5. Data frames: find the PII, send only what is needed

complaints <- data.frame(
  cust_pan = c("ABCPE1234F", "AAACR5055K", "XYZPQ9876R"),
  mobile = c(9876543210, 8826781145, 9123456780),
  complaint = c(
    "EMI debited twice from A/c 50100123456789; please refund.",
    "Shri Mohan Lal wants the statement sent to [email protected].",
    "App crashes when opening the cards tab."
  )
)
scan_pii(complaints)
#> Scanned 3 columns; PII found in 3: cust_pan, mobile, and complaint
#>     column         type rate matched whole confidence               example
#>   cust_pan          pan 100%  3 of 3     3       0.97            XXXXXX234F
#>     mobile    mobile_in 100%  3 of 3     3       0.93            XXXXXX3210
#>  complaint bank_account  33%  1 of 3     0       0.97        XXXXXXXXXX6789
#>  complaint        email  33%  1 of 3     0       0.95 [email protected]
#>  complaint  person_name  33%  1 of 3     0       0.80             XXXXX XXX

Only the complaint text is needed to categorise complaints, so send only that column, redacted:

prompts <- paste("Categorise this complaint:", redact(complaints$complaint, min_confidence = 0.5))
prompts
#> [1] "Categorise this complaint: EMI debited twice from A/c [BANK_ACCOUNT]; please refund."
#> [2] "Categorise this complaint: Shri [PERSON_NAME] wants the statement sent to [EMAIL]."  
#> [3] "Categorise this complaint: App crashes when opening the cards tab."

6. Measure it on your own data

If you have labelled examples of your documents, cg_evaluate() reports precision, recall and F1 per type, so you can decide on a threshold with evidence:

docs <- list(
  list(text = "PAN: ABCPE1234F", spans = list(list(start = 5, end = 15, type = "pan"))),
  list(text = "order no 878245671885", spans = list())
)
cg_evaluate(docs, min_confidence = 0.5)
#> Evaluated 2 documents, exact matching:
#>     type tp fp fn precision recall f1
#>      pan  1  0  0         1      1  1
#>  overall  1  0  0         1      1  1

Limits and what comes next

The vault behind tokenise() lives in memory until you save it with vault_save(); the file is encrypted with a passphrase, and vault_open() (or Python’s Vault.open()) reads it back. Keep *.cgvault files out of version control, and the passphrase out of code.
You call redact() or tokenise() yourself. A planned proxy mode - a local OpenAI-compatible server in front of Ollama or vLLM - will redact, restore and audit automatically, so tools you cannot change (Open WebUI, Continue, Positron, Jupyter AI) are covered by pointing their base URL at it.

Runnable versions of these examples, in R, Python and the shell, are in the examples/ folder of the repository.