Aiven plans customer gateway after cutting AI costs by 80 per cent
Aiven says its internal gateway has cut AI spending on heavy users by about 80 per cent, by making it easier for developers to use lower-cost open-weight models. The Finland-based open-source database provider plans to offer the gateway to customers, as organisations look for ways to manage rising costs from large language models.
The gateway is built around the open-source LiteLLM framework and adds budgeting and access controls. Aiven runs models on rented GPUs and also hosts them itself; it still uses more powerful frontier models to plan work, while open-weight models handle much of the execution. The company says teams should also compare cost per task, since model performance varies.
In short
- Aiven reports an 80 per cent spending reduction for heavy users.
- Its gateway helps developers switch between different AI models.
- The company plans to offer the service to customers.
Steelman: both sides, in good faith
The strongest fair case each way — we don't pick a winner.
The case for
Offering the gateway could help organisations control unpredictable AI bills by directing routine work to lower-cost open-weight models while reserving frontier systems for tasks that need them. Budgeting and access controls may also make adoption easier to govern, and comparing cost per task gives teams a way to weigh savings against performance.
The case against
An 80 per cent saving reported by one company may not transfer to customers whose workloads, infrastructure and quality requirements differ. Relying more on lower-cost models could require extra checking or retries, and running models on rented or in-house GPUs brings its own costs and operational demands; organisations may therefore prefer to pay for frontier models when reliability or capability matters more.
Explained simply
Large language models are AI systems that generate text and help with tasks such as writing or coding. Some are offered by major providers, while open-weight models make their underlying model files available for others to run and adapt; they can cost less, though their performance varies.
Aiven is a Finland-based company that provides open-source database services. Its gateway uses the LiteLLM framework to route requests to different models and set budgets and access rules, while Aiven runs models on rented or company-hosted computer chips.
AI costs matter to organisations using these tools at scale, but the cheapest model may not complete a task as well. Comparing the cost and performance for each task can help businesses decide which models to use.
Read the full article at the source →
Originally published by The Register as “Euro open source specialist proposes LLM gateway to ease token bills”.