For example, if I wanted to benchmark Warhammer 3 with different hardware or graphic settings. I know there’s a built-in benchmark, but I don’t think that’s valid as it’s scripted and guessing doesn’t stress the cpu as much as actual gameplay would.
The same goes for any other game. In modern games, it seems like if you tried to setup the same sequence there would be so much variance it would invalidate the results. Along the same lines, what app do people use? Long time ago I used to use FRAPS, but I’m guessing there are much better software applications available that could do the job with less overhead?
It’s going to be hard to compare benchmarks with Warhammer 3 online since most folks use the built-in tool. Even if you believe it’s not accurate, it’s what everyone would be using, so your numbers won’t mean much. Benchmarks aren’t really that useful other than getting a sense of satisfaction in my opinion - it’s more about how smooth the game feels at the settings you prefer. If you feel like 40fps feels pretty smooth at Ultra settings, for example, knowing the average FPS doesn’t really change anything.
Those sequences are scripted but fully in engine and meant to represent typical to higher than average stress than your normal gameplay experience. I personally see that as a plus, not a disadvantage.
Surely the best thing to do is to use the built in benchmark to get baseline settings at, say, a consistent 60fps. Then tweak the settings as you play if things get choppy.
I find that there is no real added value to me trying to find a good location to reliably benchmark a game.
For big releases, my second port of call is to let DF do the analysis and post recommended settings I can use as a basis: https://youtube.com/@DigitalFoundry
I must say I haven’t reinstalled FRAPS after my last PC upgrade.
This isn’t to compare against anyone else, but to try and quantify the changes I make to have things run better, from hardware to settings etc.
I start overthinking things when I try to “feel” if things seem better or worse. Without numbers I won’t be able to stop wondering, “Am I just experiencing the placebo effect”? And then I drive myself crazy :)
See again to me, it seems like all you need is a baseline, which is what you get from the basic built-in benchmark tool. Just get a baseline from that, and go from there. I guess I’m not really following the purpose of the question itself?
I think I have an idea. Going to do the Warhammer 3 tests, and found Anno 1800 has one as well. But I found NVidia has a benchmarking tool. I have an old save from X4 Foundations where it was running abysmally, like 15 fps on a busy sector. So I’m going to position my camera over that sector and just run the benchmark for 5 minutes. Should give me low, average, high fps etc. I can make changes, then do it all over and see how it is. That game is really important to me and getting it to run at greater than potato speed is one of my goals.
I’m still not sure what your aversion is to using the built in benchmark, other than you seem to be fixated on finding the most demanding test possible. But that’s not the point, the point is to compare configurations to see what improves things and by how much. As long as the stress isn’t trivial, it doesn’t need to push your system to the absolute limit. The built in benchmarks are useful for that because they are consistent, so you know it’s the changes you made that are responsible for different results*. Hopefully they are also designed to include a range of different scenarios so you’re stressing different parts of the system (CPU vs GPU, high VRAM usage, RT, etc). There’s no guarantee anything bespoke you come up with will do that, especially if you just leave the camera in one position.
All that said, I understand the desire to see how performance improves in a particular game you really want to perform well. I’m just not sure doing manual benchmarking in that game is particularly useful for that. You’d probably be better off figuring out in a general sense where your bottlenecks are and improving those, and if the internet tells you anything is particularly demanding in X4 (I imagine that’s more CPU and RAM heavy than most but I don’t know), focusing on that.
I mean, not 100%, there will be variation, there could be things outside the game affecting performance etc.
Agreed. To me, that’s by far the most important result obtained from those benchmarks. If I disable feature X, I gain 20% performance without much loss. If I disable feature Y, I gain 5% performance but the game looks worse. Some will be worth it, some not. but the point is that you get a pretty good idea of what really affects your performance.
It does not guarantee that the game will run flawlessly in that one spot. But that’s kinda irrelevant. It’s running as best as it can on average with the visual fidelity you are happy to compromise on.
It’s not an aversion, but a challenge. A game like X4 and it’s insane cpu workload seems like it would have very different demands for performance vs. the built in benchmark for the game Control (just an example).
Or you could just pare it down to me being a bit OCD.
