Building an AI demo takes a few days; running the same solution in production with thousands of documents, real users and audit requirements is a completely different job. What closes the gap is architecture. In this article I summarise the key decisions I make as an AI architect and project specialist on enterprise AI projects.
1. The right use case
A good first use case has three properties: it happens often, its rules are clear and its errors are measurable. Reading supplier invoices, classifying customer requests or question-answering over internal documents fit this description. Broad goals such as "an assistant that understands all company data" are not suitable for a first project; the success criteria stay vague.
2. The data layer and RAG
The model does not know your company. To produce correct answers it must be given the relevant information at the moment of the question; this is called RAG (Retrieval-Augmented Generation). Documents are split into chunks, added to a searchable index, and for each question the most relevant chunks are passed to the model as context. Quality depends on document hygiene, the chunking strategy and preserving access rights. A user should never get an answer from a document they are not authorised to see.
3. Structured output and validation
In a solution that writes data to the ERP, you ask the model for structured output with a defined schema, not free text. The output is validated automatically for required fields, date and amount formats, and matches against master data (does this supplier code actually exist?). Records that fail validation go to human review.
4. ERP integration
Writes to systems such as SAP happen only through standard interfaces (BAPIs, OData), so the system's business rules and authorisation checks stay in force. The AI layer connects to the ERP not directly but through an intermediate service that handles approval and logging.
5. Security, data protection and cost
- Data minimisation: only the necessary fields are sent to the model, and personal data is masked.
- Provider choice: enterprise APIs that do not retain data, or on-premise models, are evaluated.
- Key management: API keys and system instructions are kept on the server only.
- Cost control: rate limits, caching and a model sized for the task.
6. Provider-independent design
AI models change fast. Tightly coupling application code to one provider's API means redevelopment every time the model changes. Keeping prompts, schemas and model calls in a separate layer reduces a model switch to a configuration change.
7. Criteria for moving from pilot to production
- Is accuracy above the threshold defined in advance?
- Are time and cost per document measured and on target?
- Are human approval and a rollback path defined for failure cases?
- Are the logs sufficient for audit?
- Do users find the results reliable, and do they actually use them?
When the answers are "yes", scope and automation can be increased step by step. Success in AI projects comes not from picking the most advanced model but from getting data, integration and measurement right.
For the building blocks of this architecture, read about RAG, measuring document-processing accuracy and data protection, or see AI solution architecture.
