In a move that catches attention across the AI and defense sectors, VigilSAR, a defense-focused ISR software company, has released a public leaderboard ranking various language models based on their ability to handle intelligence, surveillance, and reconnaissance tasks. This leaderboard is designed to evaluate models on trustworthy reasoning, reporting, and restraint—the qualities an analyst needs—rather than simple trivia accuracy.
The evaluation covers 14 models and 300 tasks, with scores collected as of July 17, 2026. Importantly, the task set remains secret to prevent models from training specifically on it. Instead, there is a private, held-out set that enables the company to assess each model’s true capabilities by comparing public scores to the hidden results. This approach helps identify models that might rely on memorization rather than genuine reasoning.
Current results show that Claude-Fable-5 leads with a stable score of 67.77, earning it the top position in the pinned Band A. Notably, a new contender, Kimi K3 from Moonshot, has made a remarkable debut, claiming third place with a score of 64.65. This entry falls into Band B, surpassing every GPT and Gemini model on the leaderboard. Such a debut signals a significant shift, highlighting the emergence of models that can compete with established AI giants in specialized tasks.
In terms of model categories, the GPT-5.x family occupies Bands C and D, while Gemini models are positioned in Bands E and F. Additionally, the leaderboard features a model that is locally deployable and considered ‘sovereign-deployable’, indicating that deployment realities are factored into the scoring. This transparency helps users understand not just the model’s capability but its practical usability in real-world scenarios.
VigilSAR emphasizes that vendor claims are not considered evidence. Their evaluation is purposefully designed to determine which models can truly match their own products and operational standards, with the aim to provide an objective measure free from vendor influence. The site’s philosophy is summarized as: “we would rather be measured than believed,” underscoring their commitment to honesty.
To promote transparency, the leaderboard includes various honesty features: confidence intervals, held-out gaps, a pinned reference row, and cost-per-correct-answer metrics. For those interested, the full results and detailed scoring are available on the public leaderboard. As the AI landscape evolves, this ranking offers a rare glimpse into which models are trusted for high-stakes intelligence work—while keeping the specific test questions under wraps to preserve fairness and integrity.
Local LLM Inference Optimization: A Comprehensive Guide to Quantization, Hardware Acceleration, and Efficient Private AI Deployment
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Trustworthy AI – Integrating Learning, Optimization and Reasoning: First International Workshop, TAILOR 2020, Virtual Event, September 4–5, 2020, … (Lecture Notes in Artificial Intelligence)
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
aosu T2 Pro 3K Dual Cam Solar Security Camera, AI Tracking, 2-Cam Kit
- All-in-One Security System: Build with 2 or 4 wireless cameras
- Comprehensive Property Coverage: Covers entry, driveway, yard, perimeter
- Dual-Lens Outdoor Cameras: Front-to-back and corner coverage
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Generative AI for Developers: Integrating Open-Source LLMs into Your Applications: Build Private, Scalable, and Cost-Effective AI Solutions with Llama 3, Mistral, and RAG
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
