github.com
optillm
A proxy for experimenting with techniques that change how language model responses are produced.
https://github.com/algorithmicsuperintelligence/optillmAbout This Resource
A proxy for experimenting with techniques that change how language model responses are produced. It offers approaches such as additional reasoning steps and response selection behind a model API interface.
Saved Notes
Original notes saved with this link. They may describe an earlier version.
GitHub - codelion/optillm: Optimizing inference proxy for LLMs
Optimizing inference proxy for LLMs
optillm is an OpenAI API compatible optimizing inference proxy which implements several state-of-the-art techniques that can improve the accuracy and performance of LLMs. The current focus is on implementing techniques that improve reasoning over coding, logical and mathematical queries. It is possible to beat the frontier models using these techniques across diverse tasks by doing additional compute at inference time.