optillm

项目介绍

🚀 2-10x accuracy improvements on reasoning tasks with zero training

OptiLLM is an OpenAI API-compatible optimizing inference proxy that implements 20+ state-of-the-art techniques to dramatically improve LLM accuracy and performance on reasoning tasks - without requiring any model training or fine-tuning.

It is possible to beat the frontier models using these techniques across diverse tasks by doing additional compute at inference time. A good example of how to combine such techniques together is the CePO approach from Cerebras.

Get powerful reasoning improvements in 3 simple steps:

Before OptiLLM: "x = 1" ❌ After OptiLLM: "Let me work through this step by step: 2x + 3 = 7, so 2x = 4,…