We’re Ranking AI Models All Wrong And Why Human Capability Should Drive The Benchmark

At a moment when the AI industry is obsessed with bigger models and higher scores, Professor Ganna Pogrebna opened the ...

MemRL outperforms RAG on complex agent benchmarks without fine-tuning

MemRL separates stable reasoning from dynamic memory, giving AI agents continual learning abilities without model fine-tuning ...
ZDNet

'Humanity's Last Exam' benchmark is stumping top AI models - can you do any better?

On Thursday, Scale AI and the Center for AI Safety (CAIS) released Humanity's Last Exam (HLE), a new academic benchmark aiming to "test the limits of AI knowledge at the frontiers of human expertise," ...
Hosted on MSN

AI benchmarks are a bad joke – and LLM makers are the ones laughing

AI companies regularly tout their models' performance on benchmark tests as a sign of technological and intellectual superiority. But those results, widely used in marketing, may not be meaningful.… A ...
Semiconductor Engineering

Benchmark For AI-Aided Chip Design That Evaluates LLMs Across 3 Critical Tasks (UCSD, Columbia)

Researchers at UCSD and Columbia University published “ChipBench: A Next-Step Benchmark for Evaluating LLM Performance in AI-Aided Chip Design.” Abstract “While Large Language Models (LLMs) show ...
Business Wire

New MLPerf Storage v1.0 Benchmark Results Show Storage Systems Play a Critical Role in AI Model Training Performance

SAN FRANCISCO--(BUSINESS WIRE)--Today, MLCommons ® announced results for its industry-standard MLPerf ® Storage v1.0 benchmark suite, which is designed to measure the performance of storage systems ...
Ars Technica

New secret math benchmark stumps AI models and PhDs alike

On Friday, research organization Epoch AI released FrontierMath, a new mathematics benchmark that has been turning heads in the AI world because it contains hundreds of expert-level problems that ...