Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

View judge scores and comments side by side when comparing #1378

Open
Philipp-Userlike opened this issue Jan 6, 2025 · 0 comments
Open

View judge scores and comments side by side when comparing #1378

Philipp-Userlike opened this issue Jan 6, 2025 · 0 comments

Comments

@Philipp-Userlike
Copy link

Philipp-Userlike commented Jan 6, 2025

Feature request

When comparing multiple experiments, allow seeing scores and comments side by side (ideally with only the score in a row and then only the comments in a row).

Right now it is hard to compare single scores visually since you have to match them to the right place visually.
And judge comments are only visible on hower, so I can never see them both at once.

This is the view I am referring to:
image

The only way I found to see judge comments was hovering like this:
image

Motivation

We found scoring, especially with LLM-as-a-judge to not be that relyable.

I often find that I want to dig into the judgements and see how they came to be, when comparing two experiment results.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant