The Standard Performance Evaluation Corporation (SPEC), which tasks itself with maintaining standardized benchmarks for computing components, has invalidated more than 2,600 results for Intel Xeon chips.
The non-profit consortium revealed that it would no longer publish SPEC CPU 2017 results for Intel PCs running a specific version of the Intel compiler.
The decision stems from concerns over what the consortium describes as a targeted optimization for a particular workload. In other words, SPEC is accusing Intel of tweaking its products to obtain better benchmark results.
2,600 Intel benchmarks invalidated
SPEC CPU 2017 is a widely used benchmark for high-end servers, data centers and workstations, testing performance across a total of 43 benchmarks. The controversy relates to the Intel oneAPI DPC++/C++ Compiler, which has been allegedly optimized for the 523.xalancbmk_r / 623.xalancbmk_s benchmarks.
A disclaimer is now attached to more than 2,600 results: “The compiler used for this result was performing a compilation that specifically improves the performance of the 523.xalancbmk_r / 623.xalancbmk_s benchmarks using a priori knowledge,” thus invalidating the results.
Although benchmark figures are generally non-reflective of real-world conditions, testing is designed to be as life-like as possible, so optimizations designed to cater to specific benchmarks can result in skewed (and unrealistic) results.
According to a Phoronix report, 2022.0 to 2023.0 models are affected, leaving models either side of this date range unaffected by the disclaimer notices. The article continues: “The SPEC CPU specific optimization could result in a 9% overall SPECint speed uplift and around 4% for the SPECint rate.”
Intel told us in a statement: "We respect the SPEC committee decision and are focused on delivering real world workload performance for our customers."
TechRadar Pro has reached out to SPEC for any further information, but we did not receive an immediate response.
More from TechRadar Pro
- Check out the best processors for your PC
- We’ve rounded up a list of the best small business servers
- Will Intel’s cheapest 14th-gen CPU be worth buying? Price leak gives us a hint
