Tag: evals
-
Simon Willison’s Weblog: yet-another-applied-llm-benchmark
Source URL: https://simonwillison.net/2024/Nov/6/yet-another-applied-llm-benchmark/#atom-everything Source: Simon Willison’s Weblog Title: yet-another-applied-llm-benchmark Feedly Summary: yet-another-applied-llm-benchmark Nicholas Carlini introduced this personal LLM benchmark suite back in February as a collection of over 100 automated tests he runs against new LLM models to evaluate their performance against the kinds of tasks he uses them for. There are two defining features…
-
Simon Willison’s Weblog: Creating a LLM-as-a-Judge that drives business results
Source URL: https://simonwillison.net/2024/Oct/30/llm-as-a-judge/#atom-everything Source: Simon Willison’s Weblog Title: Creating a LLM-as-a-Judge that drives business results Feedly Summary: Creating a LLM-as-a-Judge that drives business results Hamel Husain’s sequel to Your AI product needs evals. This is packed with hard-won actionable advice. Hamel warns against using scores on a 1-5 scale, instead promoting an alternative he calls…