A chat demo on a language model takes about a week to build. The trouble starts afterwards, when that same feature has to run on production data, under real traffic, with a predictable cost and without hallucinations that embarrass the brand. For a CTO, Head of Product or Head of AI, the real question is not whether to add AI, but where, how and with whose hands to do it so the feature survives contact with users. That is the point where AI features in a product stop being a demo and become a piece of architecture you have to maintain for years.
In our experience, the hardest part is not the integration with the model but the engineering around it: choosing the data, evaluating quality, controlling cost and handling privacy. Within the RITS Group we build real AI/Data competence, including our AI/LLM/data teams, and we have delivered GenAI, RAG and LLM projects, including an AI-based knowledge assistant for a client in the insurance sector. We carry that experience into tech products, filling the roles that are usually missing on the client’s side.
Where does AI actually add value in a product?
Before the question about the model comes the question about the place. AI pays off where a product has a lot of unstructured text or repetitive cognitive work, not everywhere. The real value usually shows up in a few patterns:
- search and question answering over documentation, a knowledge base or ticket history,
- summarisation and extraction from long documents, emails and transcripts,
- decision support for an operator: suggestions, classification, first drafts of replies,
- automation of tedious steps in a process that a human does manually today.
A well-chosen first feature has one thing in common: you can measure it and you can roll it back if it fails. That is where we start, because it limits risk and earns the organisation’s trust for the next steps.
How do LLM and RAG features differ in a product?
This distinction drives both the architecture and the cost. A plain LLM works on what the model “knows” from training and what you put in the prompt, so it is great at generating, rephrasing and classifying, but it does not know your data. RAG (retrieval-augmented generation) adds a retrieval layer: it first finds the right fragments of your knowledge, and only then does the model answer based on them, with a reference to the source.
In practice, most product features that genuinely help a user are RAG, not a bare LLM. The reason is mundane: a company wants the answer grounded in its own documents and verifiable, not in the model’s general knowledge. That is exactly the pattern we used in the knowledge assistant for the insurance client, where the model had to answer from a specific, company-owned base rather than guess.
How do you integrate and evaluate AI models?
Integration is the easier part. The harder one is evaluation, the answer to whether a feature is good enough to ship. Without it, the team works blind and reacts only once users complain.
- A test set of real questions and expected answers, built together with domain experts.
- Quality metrics matched to the task: answer relevance, source citation, hallucination rate.
- A comparison of models and configurations (different models, prompts, parameters) on the same set, instead of picking one by gut feel.
- Regression before every release, so a prompt change does not break what already worked.
We treat the model as a component that is tested and versioned, not as a magic box. That way the decision to ship rests on numbers, not on a single lucky demo.
How do you fit AI agents into delivery processes and the product?
An AI agent is a model that does not just answer but takes steps: reaches for tools, calls APIs, chains several actions into one flow. In a product it works where the task has a clear goal and a measurable result, for example handling a ticket or preparing a first draft of a document.
We also use agents on our own side, in the delivery process, to speed up routine parts of engineering work. The principle is the same in both cases: an agent gets a narrowly defined scope and hard permission boundaries, not a free hand. The more autonomous the step, the stronger the controls around it, because the cost of a mistake grows faster than the convenience of automation.
How do you control the quality of prompts and outputs?
A prompt is not a note; it is a product artefact you maintain like code. So we keep it in a repository, version it and test it, instead of leaving it in a few people’s heads. Around the outputs we build a safety layer that catches what the model should not return.
- Prompt versioning, tied to evaluation results.
- Output format validation (e.g. enforced JSON), so the result can be processed safely.
- Content-safety rules and filters for sensitive data on both input and output.
- Production monitoring: logging of requests, costs and low-quality-answer signals.
This layer decides whether an AI feature is repeatable rather than dependent on luck with a particular query.
What does it cost, and how do you handle data security and privacy?
The cost of AI in a product is not one invoice but the sum of model calls, vector storage and maintenance. We can make it realistic already at the design stage, by matching the model to the task (smaller and cheaper where that is enough), shortening prompts, caching repeatable answers and deciding what even needs to reach a model at all.
We treat security and privacy as an entry condition, not an add-on. When working with a client’s data, we establish early where the data is physically processed, whether it goes to a public model or a private variant, and how it is anonymised and retained. In regulated projects, such as insurance, it is these decisions, not the choice of model, that determine whether a feature may be shipped at all.
Why do AI features need a human in the loop?
Because a model can be confident even when it is wrong. For features that touch money, law, health or brand reputation, we design a human in the loop as a deliberate choice, not a safeguard bolted on at the end. In practice that means that for low-confidence answers the product proposes rather than acts, leaving the final step to a person. This setup lets you ship AI faster and more safely, because the organisation does not have to trust the model one hundred percent overnight.
Which AI skills do we provide for the project?
Even if we have not built your exact type of product, we supply people who have done it before on other GenAI/RAG/LLM projects. We shape the team around the specific task, not “just in case”:
- AI/ML and LLM engineers building the features, prompts and the RAG layer,
- data engineers cleaning and preparing the knowledge for retrieval,
- architects joining the model, backend and infrastructure into a maintainable whole,
- DevOps/MLOps owning deployments, monitoring and model costs.
We plug this team into your product as team augmentation or a dedicated team, under your management, adding skills exactly where they are missing.
How do we do it at RITS?
We do it as a nearshore partner from Poland, with real AI/Data competence inside the RITS Group (including our AI/LLM/data teams). We start from one measurable feature, not a whole AI platform at once. We build it on RAG and the client’s own data, with evaluation and monitoring from the start, not only after the first complaints. We set privacy and data security before the first model call, because in regulated projects they decide whether a feature ships. We have delivered GenAI/RAG/LLM projects, including an AI-based knowledge assistant for a client in the insurance sector, and we carry that experience into tech products, filling the missing roles with proven people.
Where do you start adding AI features to a product?
Start from one concrete user problem you can measure, and from the question of what knowledge AI needs to answer sensibly. That decides whether you need a plain LLM or RAG, what data you must prepare and which roles to fill. If you are weighing up which engagement model to start in, we covered the differences in our article on IT engagement models.
Adding AI features (LLM/RAG) to your product? Let’s talk - we’ll match the skills and architecture to your case.