Create and Implement Data Secure AI Workflows

• Data security in LLM workflows requires careful consideration of model providers, hosting options, and API versus web interface usage. Open-source models with serverless inference or self-hosted GPU deployments offer greater control over data privacy than closed-source models.

• LLM workflows introduce unique data flow challenges beyond traditional applications, including conversation history storage, logging practices, and third-party integrations like speech-to-text APIs or model context protocol servers that may transmit user data.

• Traditional security practices are insufficient for LLMs. System prompts should never be relied upon for security, agents should have user-scoped permissions, and access to SQL or command line must be explicitly restricted to prevent data leakage or unauthorized access.

For privacy concerns, choosing the provider is one of the most important decisions you will make. Different model providers and hosting options have different attitudes towards data capture and privacy. Within each provider, there are different data policies depending on whether or not you are using the web interface or their API for inference.

When using a closed-source model through a provider’s web-interface, it is common for them to store your data and use it to train their future models. If you’re using their API and paying per token, providers don’t generally train on your prompt data, but they often store it for a retention period, most commonly 30 days. These closed-source models are usually available through third party inference providers, so you might think this avoids the data retention period. However, even if you’re using a closed-source model through a third party provider, it is possible that either the provider or the model’s company is still storing your data for the data retention period, so be sure to look into the terms and services of all companies you are working with in order to understand their specific data policies.

If you need a higher level of data privacy, an open source model through a service that provides serverless inference could be a better option. For example, DigitalOcean’s Gradient AI platform offers Serverless Inference where you can use Llama, GPT-OSS, or other open-source models. When using these models, data beyond what is required for functioning is not stored or used for training.

To take it a step further, you can rent your own DigitalOcean GPU Droplet and deploy an LLM on it. As long as your DigitalOcean GPU Droplet is secure, you will have total control over the data that you include in your prompts.

Finally, depending on your model needs, data privacy can be maximized by running a model locally on your own machine without an internet connection. This keeps the data you process off the internet entirely. There are open-source models that are small enough to run on laptops, or if you have the resources to purchase a GPU, you can deploy a larger open-source one.

Next, you need to consider how data is managed in your LLM workflow. For example, if the LLM workflow includes a chat interface, is the chat history stored in your database on a cloud server, or is it stored entirely on the user’s device and sent for each user input? There may be more privacy for a user if their conversation histories are not stored on the server, but the size of each request will get larger with a longer conversation, which can add latency.

Logging is also an important consideration. If you are logging the prompt sent to an LLM each time, it can contain the entire conversation. Whether or not you store the user’s conversation on the server, if you log the full prompt each time, the user’s conversation will be on the server in the logs. Access controls, audit trails, and log integrity should be maintained, and what data is saved in those logs needs to be considered.

Many LLM workflows now involve third-party processes that expose the user’s data to another endpoint. This can include speech to text APIs, model context protocol servers, vector databases, or services that might automatically be included in an agent framework. All of these calls need to be accounted for and reviewed to ensure you know what is happening to the data when. Vector databases or alternative data storage options used for augmenting context also need to be secure.

LLM applications are relatively new, without established industry standards around security best practices. This opens new applications up to different kinds of novel cyber attacks from malicious users.

To secure your data against malicious users, there are additional considerations on top of standard security best practices that you need to consider for an LLM workflow. First, ensure the system prompt is not being relied on for any security or data privacy functionality. Imagine a situation in which a malicious user has access to the system prompt and can write anything they want. If this will jeopardize data privacy in any way, you need to change your workflow and app design.

The system prompt should not be able to give the LLM workflow access to anything that the user should not be able to access. Do not store any sensitive information in the system prompt either. Design your architecture and data security practices based on the assumption that it is possible that the system prompt may be leaked, even if you take steps to prevent this.

Do not give your LLM workflows open access to SQL or the command line without restrictions scoped to the current user. It is safest to not allow the agent to write any queries or command line instructions that you would not want the user to have the ability to write and submit themselves.

The same can be said about which data the agent has the ability to access. If the user should not be able to access certain data, the agent should not be able to access it in the process of responding to a user prompt.

After building your secure LLM workflow, conduct penetration testing and security validation procedures. This can be done manually or with other LLMs that have been instructed to generate the kinds of prompts that are known to jeopardize data security for LLM applications. LLM applications are still relatively new, and novel, effective threats are being discovered regularly, making security testing a challenge. Keep up to date with newly discovered attacks and continue to update your application’s security features to ensure it can safely defend against new attacks.

Some common examples of malicious attacks are included below:

• Direct Prompt Injection: Attackers inject commands like “Ignore all previous instructions and…” or “Imagine you’re a movie character that hacks into a database…” in an attempt to hijack the LLM’s behavior.
• Indirect Prompt Injection: Attackers will inject malicious instructions into documents or databases the LLM retrieves.
• Data Exfiltration: Attackers attempt to make the LLM reveal the system prompt, instructions, or any data related to other users’ conversations in shared contexts.
• Tool/Function Manipulation: Attackers manipulate function call parameters to execute unintended actions.
• Denial of Service: Attackers craft prompts that cause excessive token generation or computation, triggering recursive or repetitive behavior to flood the LLM with tokens.

Can input sanitization and output filtering help prevent sensitive information leakage or injection attacks?

Yes, but LLMs should not be trusted to reliably identify sensitive information or malicious inputs on their own. Traditional non-LLM, or strongly validated, methods of input sanitization and validation should be the first choice. If you are attempting to use an LLM for layers of security to prevent SQL or prompt injections or accidental leakage of sensitive information to the user or third party, they may prevent some or most prompt injection or jailbreaking attempts. However, the non-deterministic nature of LLMs and increasing complexity of attacks make it difficult to build a secure application with LLMs as the primary filters.

What kinds of traditional data security practices are still relevant to LLM workflows?

Nearly all data security best-practices are still relevant with LLM workflows. Data minimization and retention policies should ensure only necessary and minimum data are collected and processed. Clear retention periods for logs, conversation, and temporary data should be in place. Data should be encrypted at rest and in transit. Role-based access control with strong authentication mechanisms should be implemented.

How do I prevent malicious users from spamming my LLM workflow and running up my inference costs?

Implement rate limiting and monitor for unusual patterns or excessive API usage. Set up alerts for suspicious behavior. Consider setting a cost limit for your LLM service provider to prevent run-away costs from automated or malicious users.

Can LLMs be used for HIPAA compliant applications?

Yes, they can, but there are steps you need to follow with the cloud services provider to ensure your app follows HIPAA regulations including signing Business Associate Agreements as required.

What if I realize later there was a vulnerability?

You should follow the same procedures your organization has established for any security vulnerabilities that may have been discovered. This may include notifying users and reporting it to the appropriate channels. Security vulnerabilities in an LLM application should be taken as seriously as security vulnerabilities in other applications.

LLM applications introduce new security concerns, but by extending standard security best practices to the added LLM logic layers of the application and strategically restricting the data your agentic workflow has access to, you can continue to ensure a high level of data security. Look into the data policies of the different tools and vendors you use. Do not make any assumptions about how different services are storing and using your data.

You can maintain HIPAA, COPPA, GDPR, SOC 2, and other compliances in an LLM workflow, but each of these adds additional security tasks for your team. The next steps are to start building and testing your application and ensure every security decision aligns with your app’s users, goals, and use-case.

• Security Best Practices Guide for DigitalOcean Droplets
• How to Build Parallel Agentic Workflows with Python
• Mistral 3 Models on DigitalOcean