Microsoft.Extensions.AI.Evaluation

These packages have been designed to work together to support building processes for evaluating the quality of AI software.

• Microsoft.Extensions.AI.Evaluation - Defines core abstractions and types for supporting evaluation.
• Microsoft.Extensions.AI.Evaluation.Quality - Contains evaluators that can be used to evaluate the quality of AI responses in your projects including Relevance, Truth, Completeness, Fluency, Coherence, Equivalence and Groundedness.
• Microsoft.Extensions.AI.Evaluation.Reporting - Contains support for caching LLM responses, storing the results of evaluations and generating reports from that data.
• Microsoft.Extensions.AI.Evaluation.Console - A command line dotnet tool for generating reports and managing evaluation data.

Install the packages

From the command-line:

dotnet add package Microsoft.Extensions.AI.Evaluation
dotnet add package Microsoft.Extensions.AI.Evaluation.Quality
dotnet add package Microsoft.Extensions.AI.Evaluation.Reporting

Or directly in the C# project file:

<ItemGroup>
  <PackageReference Include="Microsoft.Extensions.AI.Evaluation" Version="[CURRENTVERSION]" />
  <PackageReference Include="Microsoft.Extensions.AI.Evaluation.Quality" Version="[CURRENTVERSION]" />
  <PackageReference Include="Microsoft.Extensions.AI.Evaluation.Reporting" Version="[CURRENTVERSION]" />
</ItemGroup>

Install the command line tool

dotnet tool install Microsoft.Extensions.AI.Evaluation.Console --create-manifest-if-needed