Comments on: How Slow Are Virtual Functions Really?

By: Mark’s Testblog » Blog Archive » Data oriented design links - …for these are testing times, indeed.

Wed, 21 Jul 2010 18:22:16 +0000

[...] catch-all declarations of “virtual functions are slow!” For the general case, this can be proved as a nonsense as virtual function calls are blatantly not slow! They’re very, very fast. However, if [...]

By: On _purecall and the Overhead(s) of Virtual Functions « Ofek’s Visual C++ stuff

Thu, 03 Jun 2010 20:03:40 +0000

[...] calls are known to be more costly than calls that are resolved at compile time. Elan Ruskin measured ~50% difference – I measured a bit less, but the difference is certainly there. For functions [...]

By: Virtual functions – an experiment | .mischief.mayhem.soap.

Virtual functions – an experiment | .mischief.mayhem.soap. — Sun, 14 Jun 2009 13:17:19 +0000

[...] months ago I’ve read this article by Ellan Ruskin, when he measures the overhead of virtual functions. There’s also a follow-up [...]

By: Allan

Allan — Thu, 30 Apr 2009 13:12:17 +0000

If you have a tight loop, the first time you hit the virtual call, you are highly likely to have an L2 cache miss on the vtable lookup, which on 360 is ~610 cycles.
Then you are very likely to have a branch mispredict, which can be another 24 cycles while you fetch instructions from the *right* place, then carry on.
As Steven points out, the additional cost can be inconsequential and obviously many other criteria (ease of expression, how else can I do virtual style indirection without an L2 miss, overall engineering simplicity) come into play.
But you do get small accessors appearing in performance profiles a lot, and often the author may not have given it any thought and assumed it would be inlined.Also on 360, we often spot overzealous use of virtual simply via spotting bctr penalties with the same target address 100% of the time. Thats a penalty you quite likely can just drop (all else equal).
I’m not sure I understand Steven’s point on the compiler being smart enough to directly call virtuals on a leaf class – do you mean virtual calls within the class? This would imply the compiler is able to tell that there are no other derived classes declared anywhere else in the project…?
Calling a virtual externally via a base class pointer, I’m not sure how the compiler can know to call directly. To my knowledge, the compiler can only inline when it definitely knows the type of the object..
Object& object; object->Virtual(); // Inline away
Object* pObject = ptr; pObject->virtual(); // Can’t inline, surely

By: Mark James

Mark James — Thu, 12 Feb 2009 22:54:28 +0000

“I can tell you for sure that the problem is not the cost of looking up the indirect function pointer from the vtable”

That’s true if you make sure everything is in the cache. If not, the vtable lookup is an extra cache miss, and you can get much worse performance.

By: Steven

Steven — Tue, 20 Jan 2009 23:29:42 +0000

The problems with indirect jumps (such as virtual) is that they are not entirely mapped as code, and I’m not sure how the various CPUs deal with that. For example, for an Intel processor, functions[i](args) is considered normal data until the call is executed. The functions[i] part is a normal data read from a data segment. If you somehow can make sure the values from functions[i] are read sequentially, the MMU should manage to bring functions[i+1] in cache (and i+2, i+3,…). As for the actual value of functions[i] which is used to perform the call, yes, I guess you are right: it messes the jump prediction good, because it’s data that is read from memory that is turned into code (so to speak).

The ~2.4x just shows that the Xenon has a very different architecture than AMD and Intel x86/x86_64 chips.

By: Elan

Elan — Tue, 20 Jan 2009 04:35:29 +0000

It's absolutely true that the more work the function itself does, the less relative weight the virtual function call overhead has compared to the work done inside the function. Still, it's not zero, and I was just curious to directly measure how much that overhead was. Unfortunately the first version of this post had some incorrect numbers in the table for 50000 iterations: what I didn't realize until I looked my code over this morning was that the CPU-cycle counter I was using for a fast timer only had 32 bits of resolution, so it would alias for anything longer than about 1.5 seconds. It did seem odd that somehow the relative difference between virtual and inline function calls would get smaller the more times the test was run. Anyway I've corrected the numbers above and it shows that the virtual overhead is the same for the larger test run as it is for the smaller test run, and a virtual call takes 2.37x as long as a direct call. I think we're talking about the same thing when I say branch prediction and you say the MMU. In the Xenon, the instruction-fetcher is considered to be part of the CPU pipeline, not the MMU, and the address snooping is performed by the branch prediction stage of the CPU pipe. The MMU is responsible for hauling the instruction stream from main RAM into the cache, but in all of my test cases the code is small enough that it just stays in the icache after being loaded for the first time, and so cache misses don't figure into the calculation. The problem with branch-predicting virtuals on the Xenon (and the Cell's PPC, which is nearly the same) is that the indirect branch op, bctrl can't snoop the branch address to report to the fetch stage of the pipeline and so it continues to fetch instructions from the wrong address until the bctrl executes. Static branches on the other hand are predicted quite early, and since the fetcher actually runs faster than the rest of the pipeline it can often fill the bubble before any CPU time is lost. This sketch of the CellPPC pipeline may make it slightly clearer: in that illustration, a static branch can be predicted by the eighth stage of the pipeline, meaning that only eight instructions need to be annulled when the fetcher goes to the new address; an indirect branch on the other hand doesn't start fetching from the new address until it hits the EX3 stage. I'll try to post my test code up soon, and write a followup to explain the pipeline issues more clearly.

By: Steven

Steven — Mon, 19 Jan 2009 21:20:16 +0000

In the experiment you point at, the overhead corresponds to about 50% of a normal, direct call. That is, a virtual call, due to all of the compiler optimization, is about 150% the time of a direct call, so it’s very litte additional cost for the added flexibility. In an experiment like yours (and mine, for that matter) the function itself does about nothing; barely enough to be not optimized away by the compiler, and that makes the call dominate the timing. But suppose each function know takes 1 microsecond to execute (instead of a few nanoseconds), and your 7 nanosecond overhead is now simply irrelevant.

Also, compilers are smart enough to call directly the virtual function when you’re calling a virtual on a (pointer as a) leaf-class. This may account for an important optimizations, since there’s simply no look-up. You still have to pass the this pointer, which adds instructions, even if very little. Additionnally, the compiler may inline the call as well.

Branch prediction isn’t the whole story either. CPUs (I don’t know about the Xenon, but I would guess it does too) have MMU (memory management units) that snoops on the addresses generated and try to prefetch the next read value in cache. The MMU can guess addresses when you scan sequentially, or by incremental, known steps (like, very 16 bytes) but it cannot do anything special if it can’t figure out the address generation pattern, and that’s what happens when you scan a collection of objects with virtual methods: it cannot figure what to load next until the address is actually read, yielding to read faults/time penalties.