​
Overview

The evaluation API allows you to score and evaluate your queries and highlights. Providing scores for results improves the quality of the results over time and makes them more relevant to your users.

​
Usage

Every query returned through the search method contains a query_id field. Similarly, each highlight returned contains a highlight_id. You can use these IDs to score and evaluate the results: 
{
  "query_id": "<string>",
  "score": <number>,
  "documents": [
    {
      ...
      "highlights": [
        {
          "highlight_id": "<string>",
          "score": <number>,
          "comments": ["<string>", "<string>", ...]
        }
      ]
    }
  ]
}
The score field is the average score of the query based on all evaluations sent. The comments field is a list of comments given for a particular highlight using the methods described below. You can also use the query_id to retrieve an existing query at a later point in time.

​
Scoring queries

You can score a queries with the Evaluation API: 
client.evaluate.score_query(
    query_id="<string>",
    score=1.0
)
The score must be between -1 and 1. The higher the score, the more relevant the query results will be. If you are receiving user feedback as a simple thumbs-down or thumbs-up, you should interpred this as a score of -1 or 1 respectively.

​
Scoring highlights

Similarly,you can score individual highlights with the Evaluation API. This also lets you provide a comment for the evaluation. 
client.evaluate.score_highlight(
    highlight_id="<string>",
    score=-1.0,
    comment="This needs to be improved"
)
As above, the score must be between -1 and 1. The comment field is a free-form text field that you can use to provide additional context for the evaluation. If you only want to provide a comment, you may omit the score field.