---
title: How to design an AI proof of concept and decide whether to move forward
description: How to design an AI proof of concept that produces useful learning,
supports fair comparisons and informs the decision to move forward.
author:
name: Walter Jerusalinsky
title: AI strategy and implementation
url: https://www.linkedin.com/in/walter-jerusalinsky/
canonical: https://www.idoneos.com/en/applied-ai/proof-of-concept/
image: ../resources/testing.png
source: applied-ai/proof-of-concept/index.md
---
A well-written reply immediately suggests capability. To understand what we can do with it, we need to examine where its information came from, how much work it took to obtain and what happens with a different enquiry.
A **proof of concept** studies a defined technical capability. Its value lies in the question it helps answer and the evidence it provides for the next decision.
For our distributor, that question could be whether assistance can produce correct replies about one product family with less total working time. We have already [prepared the sources and review criteria](../prepare-your-business/). Now we can organise the comparison.
## Define what we expect to observe
In the first article, we worked through a hypothetical change from 18 to 8 minutes per enquiry. We can treat that difference as an initial expectation to be tested.
The hypothesis must also define when the task is complete. Here, that means a reviewed reply with the correct part, a price from the authorised source and checked commercial terms. An incomplete enquiry should lead to a request for enough information to proceed.
Before testing, agree on the time improvement that would justify moving forward, the quality criteria and the errors that would require changes. These decisions express what the business needs from the capability.
## Compare equivalent work
First, observe the current process. Measure from opening an enquiry to having an acceptable reply ready, including searches, questions and corrections. This is the **baseline**.
Then gather enquiries covering the variety that matters: complete requests, ambiguous descriptions, unavailable parts and documents requiring interpretation. Selection should reflect actual work. The number of cases depends on that variety and the confidence needed for the decision.
Use some examples to develop and adjust the solution. Reserve others for evaluation after fixing the version to be tested. This lets us observe cases that did not guide those adjustments.
The comparison must also account for learning. Someone solving the same enquiry twice already knows part of the answer on the second attempt. Equivalent cases can be distributed between methods, or order varied between participants, with the test arrangement recorded.
The calculation includes review and corrections. This is a hypothetical example: time, quality and cost need measurement in real work. View larger diagram.
## Count the work of review
Suppose a draft appears after one minute. The salesperson then spends four minutes checking sources and three correcting and completing it. The assisted process takes eight minutes. That is the time to compare with the eighteen-minute reference.
> 1 minute of preparation + 4 of review + 3 of correction = 8 minutes to a usable reply.
These figures are illustrative and still need measurement. They show where the possible benefit arises and which effort to record. Alongside the average, observe variation between cases and which require more intervention.
Quality needs its own record. An incorrect price and an unclear sentence have different consequences. Classify problems, examine their causes and check whether the intended review allows people to detect them.
When a solution consults documents, separate two questions: did it find the relevant information, and did it build an adequate reply from it? The [Amazon Bedrock evaluation documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/evaluation.html) addresses these capabilities. The distinction helps locate problems even when a solution uses other tools.
## Decide within the limits of the evidence
At the end, compare total time, quality, review effort and operating cost. Retain results by case, the sources used and the evaluated version to understand differences.
The evidence may support moving forward, adjusting part of the solution or stopping this line of work. If review absorbs the expected improvement, investigate why. Favourable results can support a pilot with people and controlled conditions of use.
Record the limits too. An evaluation on one product family provides evidence about that scope. Observing few errors in selected cases leaves questions about situations we have not tested.
## From a demonstrated capability to a possible practice
A pilot studies how the solution behaves within everyday work: whether people use it, which interruptions arise and what effort is needed to sustain it. Its scope should grow in line with what is being verified.
A proof of concept serves its purpose when we can explain what worked, under which conditions and what we need to learn next. That understanding is the resource we take to the next stage: developing and integrating a capability the business can use routinely.
Next step
Build on what has shown value
The test provides a firmer basis for deciding. We can now examine what it takes to turn a validated capability into something people can use every day.
