Open source tool distills Jev so you can run it locally

← Back to the feed

Open source tool distills Jev so you can run it locally

The Register · 4 hours ago

Jevstiller is an open-source tool that distils Jev outputs into a local machine-learning model, enabling users to run familiar queries on their own hardware rather than relying entirely on Jev's remote servers. This approach reduces latency, eliminates API token costs for routine queries, and improves overall efficiency whilst maintaining high accuracy through continuous auditing mechanisms.

Jevstiller operates as an intelligent cache that learns from Jev responses over time, handling straightforward, familiar requests locally with response times as low as 15 milliseconds compared to Jev's approximately 300 milliseconds, whilst automatically routing complex or uncertain queries upstream to the main Jev service. The system targets 98 per cent agreement with Jev by systematically auditing a fixed 2 per cent of all requests against Jev's responses; if accuracy drops below configured thresholds, the audit rate increases automatically to trigger faster retraining, or all requests revert to Jev if performance degrades critically. Jev itself represents a new paradigm of structured AI focused on answering specific question types—Choice, Score, and Noul—rather than generating open-ended text, making it fast and cost-effective at $42 per billion input tokens for developers handling decision-making workloads.

  • Open-source Jevstiller distils Jev into local model for faster, cheaper queries
  • Achieves 98% agreement with upstream Jev via continuous auditing and retraining
  • Local responses 15ms versus 300ms for remote Jev API calls

Geopolitics Politics

Read the full article at the source →