Generative-model evaluation
A minimal, personal reimplementation of a generative-model evaluation method, built against public model APIs, carrying no prompt, code or figure from any employer.
- Period
- 2026 to today
- Role
- Sole developer
- Status
- in progress
- Stack
- Python · public image and video APIs
Choosing a generative model by running it repeatedly and comparing results by eye does not scale, and it is not reproducible: nobody can say afterwards what was tested. The method I use instead scores a model across a matrix of reference images and prompts, combining classic image and video metrics with an LLM that reads the output against the prompt and the reference and judges the match.
The repository is not public yet, and this page carries no link until it is. The method turns adopting a model into a measurement, not an opinion.