AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |
Back to Blog
![]() ![]() The OnePlus 5, on the other hand, is an entirely different beast - it resorts to the kind of obvious, calculated cheating mechanisms we saw in flagships in the early days of Android, an approach that is clearly intended to maximize scores in the most misleading fashion. In short, cheating behavior was clear and demonstrable by both looking at score variance, and by monitoring CPU frequencies throughout the benchmark, which showed a frequency floor that – for the most part – allowed the device to consistently score closer to its full potential. This minimum frequency reduced the effective frequency range, which in turn reduced the number of step frequencies in benchmarks, this resulted in slightly lower variance and, as we showed, higher sustained performance as the higher minimum frequency could not be overridden by thermal throttling. Then, the ROM would alter the frequency in relation to an adjusted CPU load - our tools showed CPU load would drop to 0% regardless of obvious activity within the application, and the CPU would see a near-minimum frequency of 1.29GHz in the big cores and 0.98GHz in the little cores. Such application names were explicitly listed by their package IDs within the ROM in a manifest that specified the targets. So how does it work, and what’s the difference? Last time around, OnePlus introduced changes to the behavior of their ROM whenever it detected a benchmark application was opened. What is worse is that, this time around, the cheating mechanism is blatant and aimed at maximizing performance, unlike last time which did not increase scores by much on average, but did reduce variance and thermal throttling, as we found. As a result, every OnePlus 5 review citing benchmark scores as an accolade of the phone’s success is misleading both writers and readers, and performance analyses based on synthetic benchmarks are invalidated. This is an inexcusable move, because it is ultimately an attempt to mislead not just customers, but taint the work of reviewers and journalists with misleading data that most are not able to vet or verify. Unfortunately, it is almost certain that every single review of the OnePlus 5 that contains a benchmark is using misleading results, as OnePlus provided reviewers a device that cheats on benchmarks. While no customers have a device in their hands (it just launched after all), we have learned about OnePlus’ new benchmark cheating mechanism through our review unit, which we received about ten days ago before the day the embargo breaks and reviewers are allowed to report on the device. Today, we sadly must follow-up on our accusations as the company has once more been inappropriately manipulating benchmark scores in the OnePlus 5. Earlier this year, we published a report that denounced OnePlus (and other companies) for their improper behavior in regards to benchmark manipulation on newer builds of OxygenOS.
0 Comments
Read More
Leave a Reply. |