The Challenge: Intelligence vs. Privacy

As we move from simple chatbots to Agentic AI (using frameworks like LangGraph), our agents are increasingly tasked with executing real-world transactions—banking transfers, updating HR records, or processing insurance claims.

These tasks require Personally Identifiable Information (PII). However, sending raw PII (like SSNs, account numbers, or private addresses) to a Large Language Model (LLM) introduces significant risks:

1. Data Leakage: Sensitive data may be stored in provider logs.

2. Training Risk: Potential for data to be ingested into future model training sets.

3. Compliance: Violations of GDPR, HIPAA, or internal SOC2 mandates.

How do we give an agent the context it needs to call a tool without actually giving it the data?

The Solution: The "Collect, Mask, Rehydrate" Pattern

In an agentic design, we treat the LLM as a reasoning engine, not a data store. We use a three-step architectural loop to keep PII within our secure trust boundary.

1. The Masking Layer (Pre-LLM)

Before the user’s prompt reaches the LLM, we pass it through a local interceptor. Using deterministic patterns (Regex) or local Named Entity Recognition (NER) models like Microsoft Presidio, we identify sensitive entities.

• User Input: "Transfer $500 to account 123-456-789."

• Action: The system vaults the account number and replaces it with a stable token.

• Result: "Transfer $500 to [ACC_VAL_1]."

2. Reasoning with Tokens

The LLM receives the masked string. Because the token [ACC_VAL_1] acts as a consistent variable, the LLM can still perform its logic. When it decides to call a tool, it populates the JSON schema using that token.

Generated Tool Call:

{
  "action": "execute_transfer",
  "params": {
    "amount": 500,
    "recipient_id": "[ACC_VAL_1]"
  }
}

3. Rehydration (Pre-Execution)

This is the critical step. Within your Tool Node (in LangGraph) or your Tool Executor, the system intercepts the outgoing JSON. It looks up [ACC_VAL_1] in your secure, short-lived session vault and swaps it back for the real value 123-456-789.

The transaction is then executed against your secure API with the real data, which never left your infrastructure.

Why This Matters for the Enterprise

By implementing this pattern, you achieve Referential Integrity without compromising security.

• Security: PII is never sent to the LLM.

• Auditability: Your logs show the reasoning path via tokens, while your secure vault handles the sensitive execution.

• Efficiency: Local masking (like spaCy or Presidio) adds negligible latency (20-50ms) compared to the safety it provides.

Next Steps for Practitioners

If you are building agents on IBM watsonx.ai or using LangGraph, consider building a custom StateGraph node specifically for PII detection. This ensures that every message in your InMemoryStore or checkpointer is masked "at rest."

Have you implemented PII masking in your agentic workflows? I’d love to hear your approach in the comments below!