The Compare feature enables you to experiment with different task configurations and continuously improve performance. You can access the Compare view by selecting two different versions from the Task view. If you only select one, it will automatically allow you to Compare against your Active task.

What Can I Compare?

In the Compare view, you see two task configurations side-by-side. You can configure any of the core components that make up a task, such as:

• User Prompt: Defines the task for the LLM.

• System Prompt: Provides additional context and constraints.

• Temperature: Adjusts the variation in responses.

• Model: Choose from leading proprietary or open-source models.

• Output Format: Specify structured formats that integrate seamlessly with your database.

After you’ve decided on the two versions you’d like to compare, simply provide the input data you’d like to run both on. Your tasks will run simultaneously and produce two distinct responses.

Testing a New Prompt and Outputs

Let’s try comparing a new prompt that provides more detail for the LLM and specifying exactly how we’d like the outputs formatted. You can instantly see the difference in the response.

Evaluating a New Model

Curious about testing a different model? You can instantly compare models—for example, Google Flash 2.0 versus Claude 3.7 Sonnet. This allows you to continuously evaluate and deploy new models, uncovering opportunities for both performance and efficiency gains.

Task Revisions

Every time you update your task configuration, you have the option to save it as a new revision. If you run a new version prior to saving, it will create a test revision.

These revisions can be:

• Tested side by side with other versions.

• Promoted directly into your active production pipeline.

Each revision is assigned a unique ID, which you can use for version logging and A/B testing.

If your task is currently running in a staging or production environment and you’d like to promote a new revision, simply click “Promote.” This action immediately replaces the existing version in your pipeline. In other words, the API endpoint for running the task will now use the new active version.