I did leave a comment, but I'll reiterate here.
The framerate benchmarks unfortunately don't really show anything of much value, and the minimum framerates are kind of worthless because variables like the shader cache(*) isn't controlled for.
This is why most techtubers will use 1% and 0.1% minimum framerates rather than the absolute minimum, so that they don't have to control for that stuff.
I get the impression he ran the benchmarks with denuvo, then non-denuvo; as he responded to me, he did the denuvo ones, rebooted, then non-denuvo.
The cancer known as Denuvo has no impact on framerates,
provided that ample CPU power is available.
It performs no work on the GPU, so if we're looking at GPU bound benchmarks, we will see no difference.
Denuvo, at least for the moment, is a strictly CPU based affair, so benchmark wise, he'd be far better off locking the framerate at moderate settings and comparing the CPU usage of the before and after.
Its known impacts include
- Massively bloating executable file sizes
This is bad for cache coherency; meaning it causes thrashing of L1/L2/L3 caches, which can severely impact performance.
Windows also needs to map the entire executable into RAM, which add a few seconds of overhead and an unnecessary 200MB or so to RAM usage.
- Blocking synchronous checks
This can cause stalls, in particular causes a ~20s or so stall during first run before the game will even initialise.
As you can see later on his screen, his CPU didn't even clock up from 800MHz until denuvo validated and the game finally commenced loading.
- General overhead from validation checks
From what I understand, denuvo games need to do regular validation checks, these are often done in loading screens, or in the input handler.
The more well behaved games do about 5 a second or so maximum.
Ergo, his benchmarks were completely focused on the wrong thing.
The loading times are still useful statistics, as that's almost pure CPU, and shows improvements between 10-30%.
But the frametimes are useless without the corresponding CPU Usage to put it into perspective, as Denuvo has no impact on GPU load.
It's all well and good to know that framerate went up 10%, but what's the CPU usage?
Does it achieve that 10% at the same CPU Usage? Meaning it's just a straight out 10% improvement.
Does it perhaps achieve the 10% while using 20% fewer CPU cycles? Suddenly that 10% becomes a far more impressive 37%.
(*) Assuming the only difference between the executables is the presence of Denuvo, then he could've given the non-denuvo run an unfair advantage.
As they're still the same game, they'll most likely use identical shaders; for each shader during compile time, the driver follows this simplified procedure.
- Calculate checksum of shader
- Check cache for match, go to step 5 if there's a hit
- Compile Shader
- Store in cache
- Return compiled shader
This cache is persistent, and is only invalidated when the
driver is upgraded to a different version.
This means that the denuvo runs possibly paid the full the shader compilation costs, whereas the non-denuvo run might've gotten away without having to compile a single shader.
And by having to pay the full compilation, the denuvo run had terrible minimum framerates - I believe the 59 vs 60 runs to be more indicative of what it would be in reality if this was controlled for.
This is also why games tend to stutter a bit when you play them for the first time, or right after a driver upgrade.
And why proper benchmarks consist of multiple runs (minus outliers) - and techtubers utilise .1% and 1% minimums, rather than absolute minimums.