Confidence Scores

Understanding Confidence Scores in Iris

Iris uses confidence scores to evaluate the quality and relevance of answers generated for security questionnaires, RFPs, and other documents. Each answer is assigned a confidence score of High, Medium, Low, or Contradictory, based on the information provided by the uploaded sources and the match to the question's context.

How Confidence Scores Work

1. High Confidence:

   • Assigned when the sources provide direct, relevant, and unambiguous information to answer the question.

   • Example: A clearly defined password policy from a company document.

2. Medium Confidence:

   • Assigned when the sources provide partially relevant information, but there is a lack of additional context or supporting data.

   • Example: Information is relevant but doesn't fully address all aspects of the question.

3. Low Confidence:

   • Assigned when the sources lack relevant data or the answer does not align with the question's intent.

   • Example: The question asks about cassette tape backups, but the sources only discuss cloud backups.

4. Contradictory:

   • Assigned when two or more sources provide conflicting information for the same question.

   • Example: One source specifies a maximum password age of 60 days, while another lists 90 days.

   • Solution: Iris highlights these contradictions, allowing you to identify discrepancies and make informed decisions. You can exclude specific sources and regenerate answers for more consistent results, or navigate to the Approved Answer Library to remove any erroneous or outdated information.

When you hover over a confidence score in Iris, a tooltip explains why the score was assigned, helping you understand the decision-making process.

Filtering by Confidence Score

Iris allows you to filter answers by their confidence scores to streamline your workflow:

• High Confidence: Focus on reviewing and approving these answers first to save time.

• Medium Confidence: Identify areas that might require additional clarification or sourcing.

• Low Confidence: Prioritize these answers for rework or deeper investigation.

• Contradictory: Quickly identify conflicting sources and take steps to resolve inconsistencies.

To use this feature, select the Confidence Score Filter at the top of your project view and choose the desired score level.

Why Confidence Scores Are Useful

Confidence scores help you:

• Streamline your review process: Quickly focus on answers that need the most attention.

• Prioritize work effectively: Allocate time to address low-confidence or contradictory answers first.

• Improve accuracy: Understand why an answer might lack confidence and address gaps or conflicts in your sources.

• Resolve inconsistencies: Use the contradictory score to identify and address conflicting data sources efficiently.

This feature ensures that your team spends less time reviewing irrelevant or incomplete answers and more time finalizing accurate and comprehensive responses.