LangSmith/Evaluator

• LangSmith Evaluator

  • 
    • Key Idea: a specific input (Input=x1) is shared between a LangSmith/Dataset/Example and a langsmith/Run, and that run is over a specific version of your application (v1). The input is tied to a LangSmith Reference Output. The Input, Reference Output, and output from the Run over a specific version are the input to a specific Evaluator, whose job it is to calculate metrics.