AI Insight Validator (AIV) – a NEW feature in AI-Concerge for content management for the best answers!
Ladies and Gentlemen
Today you will find another exciting innovation in your AI concierge!
Wir fragten uns:
Do we give the right answers and do we find the potential in the basics (website, FAQ, PDF,…) to provide better answers to the questions asked?
Our solution:
- AI Insight Validator (AIV) – An exclusively AI-supported process to evaluate AI-generated answers.
A specially used AI model, which serves as a control authority, evaluates the answers generated by another AI based on specified criteria. This structured assessment (3 quality criteria) ensures that answers meet quality standards and are both informative and accurate. In addition, we create a short report on the given response quality (analysis)
- Column: Quality – “1, 2 or 3 bars” indicate whether sufficient sources were found for the answer
- Column: Answer – “No star, blank or filled star” indicate the overall quality of the answer in relation to the question asked.
- Notice
This means you can simply select those questions where the existing knowledge base was not sufficient to answer them (i.e. no star or an unfilled star for Answer!)
- Result
For each question you can then take a close look at the AIV rating, evaluate it and act accordingly! From today on you will find the AIV results for every question/answer in the AI Concierge!
- This is how the AIV process works:
Control AI follows this systematic approach:
- Contextual analysis: First, the evaluator contextualizes the response by reviewing the information provided and assessing how effectively it was used in formulating the response.
- Evaluation criteria
The assessment includes the following key parameters:
- Overall quality of the response (overall_quality)
- Relevance to the question (relevance_to_question)
- Accuracy of information (accuracy_of_information)
- Explanation of parameters
Each parameter is designed to address specific aspects of the
Evaluate the AI’s response:
Overall quality of the answer
- Perfect: The answer addresses the core question comprehensively and precisely.
- Average: The answer covers the question but may lack detail or contain minor errors.
- Bad: The answer fails to effectively address the main question.
Relevance to the question (relevance_to_question)
- Highly relevant: The content directly relates to the query and provides precise and focused insights.
- Somewhat Relevant: Although the answer contains relevant information, it may also contain non-essential details or digressions.
- Not relevant: The content does not address the query and focuses on unrelated or off-topic elements.
Accuracy of information (accuracy_of_information)
- Completely Accurate: All facts and data in the answer are verified and true.
- Mostly Accurate: The majority of content is accurate; however, there may be slight inaccuracies.
- Imprecise: The answer contains factual errors or misleading data.
