Speed of property setting vs typed method arguments vs un-typed objects as arguments

We just had a small discussion on the Papervision3D list about using vague objects with untyped properties passed to methods as a single argument, rather than setting multiple arguments strongly typed and how that compares in terms of speed. It was suggested that it was faster to use untyped objects in a constructor call ( or method call for that matter ) because as3 constructors are not JIT compiled, but in fact, it isn’t faster from these basic tests I’ve run.

The first set of tests were just purely calling methods in an FLA – no new constructors for new objects at this point. The tests ranged from setting properties, then calling a method to calculate, to passing objects with untyped properties and then calculating. I ran 4 tests, as you can see below, and the fastest by FAR was setting properties first, then calling a method to calculate. The slowest, was passing a vague object with untyped properties:

    Test Zero: set member variables, then call method to calculate – 308ms
    Test One: pass typed arguments to a method and have it calculate based on the args passed – 330ms
    Test Two: pass untyped properties in object as a single argument, then assign to local variables, then calculate – 1253ms
    Test Three: Pass untyped properties in object as single argument, then calculate without assigning to local variables – 1310ms

[as]
import flash.utils.getTimer;

var time:Number = getTimer();

var dingZero:Number;
var dongZero:Number;

runZero();
runOne();
runTwo();
runThree();

function runZero():void
{
time = getTimer();
for (var i:int=0; i<1000000; i++)
{
dingZero = Math.random()*5;
dongZero = Math.random()*5;

zero();
}
trace(“zero: “, (getTimer()-time));
}

function runOne():void
{
time = getTimer();
for (var i:int=0; i<1000000; i++)
{
var r1:Number = Math.random()*5;
var r2:Number = Math.random()*5;

one(r1, r2);
}
trace(“one: “, (getTimer()-time));
}

function runTwo():void
{
time = getTimer();
for (var i:int=0; i<1000000; i++)
{
var r1:Number = Math.random()*5;
var r2:Number = Math.random()*5;

two({ding:r1, dong:r2});
}
trace(“two: “, (getTimer()-time));
}

function runThree():void
{
time = getTimer();
for (var i:int=0; i<1000000; i++)
{
var r1:Number = Math.random()*5;
var r2:Number = Math.random()*5;

three({ding:r1, dong:r2});
}
trace(“three: “, (getTimer()-time));
}

function zero():void
{
var r:Number = dingZero + dongZero;
}

function one(ding:Number, dong:Number):void
{
var r:Number = ding + dong;
}

function two(obj:Object):void
{
var ding:Number = obj.ding;
var dong:Number = obj.dong;

var r:Number = ding + dong;
}

function three(obj:Object):void
{
var r:Number = obj.ding + obj.dong;
}[/as]

Now, after that, I wanted to test what it meant in terms of creating new objects and setting properties through constructors as well as doing a calculation. The results were that setting properties outside of the constructor and then calling a calculate method was the overall fastest method. Setting properties though the constructor with a vague object with untyped properties was 2x’s slower.

    Test Zero: send 2 arguments in the constructor, then calculate based on those 2 args – 939ms
    Test One: create new object (no arguments passed), set 2 properties, call another method to calculate – 930ms
    Test Two: send 1 object with 2 untyped properties, then calculate in the constructor – 1849ms

[as]
import flash.utils.getTimer;

var time:Number = getTimer();

runZero();
runOne();
runTwo();

function runZero():void
{
time = getTimer();
for (var i:int=0; i<1000000; i++)
{
var st:StrongType = new StrongType(Math.random()*5, Math.random()*5);
}
trace(“zero: “, (getTimer()-time));
}

function runOne():void
{
time = getTimer();
for (var i:int=0; i<1000000; i++)
{
var stsp:StrongTypeSetProperty = new StrongTypeSetProperty()
stsp.ding = Math.random()*5;
stsp.dong = Math.random()*5;
stsp.calculate();
}
trace(“one: “, (getTimer()-time));
}

function runTwo():void
{
time = getTimer();
for (var i:int=0; i<1000000; i++)
{
var ut:UnTyped = new UnTyped({ding:Math.random()*5, dong:Math.random()*5});
}
trace(“two: “, (getTimer()-time));
}[/as]

Now, in terms of creating new objects, this REALLY plays a part in parsing 3D models where many objects are created at one time for new DisplayObject3D, materialsLists and Materials themselves. In the regular running of the engine, this are most likely fine since constructor calls are left out of the render loop. However, that’s why I did BOTH tests – to show that using objects with untyped properties in method calls is much slower than using typed arguments, and that DOES affect the render loop performance of a 3D engine.

I’ll also say, I don’t know if these tests are the “best”, but they’re consistent across the board. I’ve posted the files so people can mess with it and let me know if I’ve messed up somewhere, which is entirely possible. I’m just trying to find the truth of the matter here, so give er’ a go and let me know.

I’ve said this many times before, but API’s that use these types of objects in methods and constructors create a real issue with learning the api through auto-complete tools. Yes, I can learn alot from a sample app, and I can read help files, but I’m 10x’s faster when auto-complete works properly. Also, having arguments in a constructor is a way of telling a developer “these are the essentials to getting this particular object up and running”. It’s not just a convenient way of setting properties, it’s a way of communicating what has to be done to and with the object itself. It’s a real strong indication of what the object is all about. Without that, you leave alot of questions unanswered, and in many cases, you end up wasting time trying to track down what IS necessary for the objects you’re creating. Its such a time waster in my opinion, and you never really learn an engine past the few methods and properties you have memorized.

My 2 cents on the matter.

  1. With typed vars and constructors in mind I’d like to see the addiition of skins for primitives to replace MaterialList. This would make it easier to know what was required of the materials (i.e. the constructor of a CubeSkin would require 6 materials), instead of you having to look up whats required, and the class itself would have 6 public properties.

  2. I 1000% agree – thanks for making my point exactly!

  3. There is one possible flaw in your methodology:

    var x : Number = 0.3;

    method(x);

    and

    var x : Number = 0.3;

    method({value: x});

    differs by more than the first being statically typed and the second not. In the second example there is also object creation involved. I’m not sure how expensive object creation is in AVM, but I bet it’s not insignificant. The difference in execution time you see may very well be all down to that anonymous object being created.

    I’m not sure whether this is what you wanted to test, and if you did then just ignore this (or consider it an explanation as to why the second example is slower). If you, on the other hand, wanted to test whether it matters if constructor arguments are typed or untyped you have to take that object creation out of the calculation.

    Even putting efficiency aside, I agree that using anonymous objects for passing method arguments is just bad form, it render the type checker useless and makes the compiler unable to help you.

  4. @Theo: yes, that’s my point exactly – not only does it have to deal with object creation, but also casting. When the properties are not strongly typed, then the avm has to cast them anyway to be able to do the calculations – and that all takes time. Whether the object is created outside the method call or not, it still take time to create that object and you still have the same situation. Plus, if you have another layer on top of that to do your own type casting, it’s even crazier. It’s like trying to do the avm’s job, when the avm is perfectly capable of doing it – and doing it faster

  5. It’s interesting seeing the API design debates in the AS3 community. They’ve got a very different slant due to the amount of folks coming from a purely dynamic background (AS2) — where there was less API design in general — and those coming from traditional C++/Java/C# backgrounds (strongly typed).

    At the end of the day, if efficiency is the goal, strong, static typing will (should*) always be faster. It’s always more information for the compiler and the JIT, which is a good thing. And, with modern IDE’s, it’s more information for things like auto-complete and refactoring tools. More efficient for the machine, more efficient for the programmer.

  6. @Troy: that’s a great point. What’s funny is, I came from AS1/AS2 days – if anyone should love dynamic, it should be me. But I’ve grown to sincerely appreciate AS3, and I just cringe when I have to look back at what we used to live with.

  7. I’m from the AS 1.0 days too!

    Those were the days when you spent hours looking for a bug, hours later finding a typo. doh!

  8. @Tink: hence why I invented Xray – to save me from myself in as1/2 days🙂

  9. “It was suggested that it was faster to use untyped objects in a constructor call ( or method call for that matter ) because as3 constructors are not JIT compiled […]”

    To be completely fair, the claim wasn’t that it was faster, it was that passing an anonymous object as an argument for initialization wouldn’t incur a penalty severe enough “to matter”, since the constructor wouldn’t be JIT compiled (and the interpreter’s “slowness” would eclipse any coversion costs).

    Anywho, the code completion issues and runtime problems associated with the initObject philosophy are what really frustrated me, personally, when I was taking Away for a test spin, so I think the arguments about it being a more ‘user friendly’ method are a bit of a stretch… but that may vary by individual. Personally, if the initObject *were* faster, I’d still probably go with the traditional constructor–explicit, statically typed parameters. The code completion and absent runtime errors are enough to make that hypothetical situation “worth it” to me. I also do C++ stuff, so I’m used to the compiler being my BFF, and I like it that way. I’ve had runtime issues in C++ before. *stares off into the distance with a haunted look in his eyes*

    Theo – From what I can tell, empty object construction is cheaper than argument-less class construction, as you might expect (though it’s only about 40% cheaper at best!). The thing is, twice as many arguments need to be stacked for objects–the index into the string pool for the property ‘name’ + the actual value… classes require only the arguments be pushed. Because of this (I guess!), the instantiation time for Objects will increase much faster than it will for classes as the number of supplied arguments increases. I’ve posted the .as file I used to come up with this crazy jibber jabber, as well as a decomopiled bytecode assembly listing here: http://transcendentaxis.com/pub/douthomp/flash/pv3d/initObject/
    There doesn’t seem to be any coercion muddling up the test, either (the convert_i’s are for the timer stuff only, so you can ignore those, I think). Let me know if you have a different interpretation.

    – If you factor out the anonymous object’s instantiation as you mentioned, and just change its values in the loop, you’ll reduce the runtime difference by around one half. Doing this isn’t TOO unreasonable, as I understand it, because the initObject just acts as proxy to map the values into the actual class, and a reference to it won’t be saved by the class for future use or anything (so it can be reused to instantiate another object), but it probably wouldn’t be ideal in practice anyway. It’ll do for testing how much impact the conversions alone have, though (which is still maybe around 50% for this particular test, I believe).

    I do hate the initObject stuff with an undying passion, but as far as performance goes, I’m going to have to sit in the “doesn’t really matter in this case” camp. I know how implicit conversion can suck the life out of a program, but I just don’t see that happening with the constructors. If you’re really going to instantiate a million objects, does it really matter if you have to wait 1.3 seconds vs 0.9 (factored out Object taken here)? If you have a million objects floating around, you’ve got a lot more to worry about performance wise. Scale the numbers as necessary.

    Also, this may be better said on the mailing list, but JIT compilation isn’t free. Having constructors not JIT seems like a good idea, imo. It takes time to translate bytecode to native code at runtime, so why spend time translating stuff that will be for sure done only one time? The idea of extracting code into an ‘init’ method to trick the JIT compilation process should be saved for heavy loops and/or heavy calculations which need to be done during construction, not for “setting properties”. I guess I haven’t done any testing in this area, so I could be wrong… but I think this stuff is worth considering (and if someone knows otherwise, let me know!).

    Anyway, I’m unsure about a lot of this stuff, so if I’m ridiculously wrong about anything, you guys can feel free to correct me… ALL NIGHT LONG

  10. I think that static typing and less object creation always creates more overhead. But there is always a tradeoff. In flash static typing should be more focused but it is also great to be able to create an object off the cuff if needed.

    This same debate will rage for eternity. It is about ease of use and performance balance. The same thing is happening with the java/c#/C++/C world vs the dynamic python/ruby/etc crowd. The latter are much slower but development is faster.

    In as3 I think that the cruft and bloat is still very young and tight and doesn’t succumb to the problems that C++ or java typing have gotten people into but I see some bloat happening in Flex and other frameworks. I come from a static background C++/C/C#/Java but have been more into python in the last year or two in hobby. It is truly fun to bust things out fast even if it is a prototype.

    It is important to the project I guess. Here where there is a focus on each byte for speed in a 3d engine these things should be highly important. Anytime the threshold hits, or like using static over dynamic for high traffic web apps.

    For things like Tweener when you are animating a property that might not be defined these types of dynamic methods are needed (such as all of a sudden in pv3d the z property emerges, Tweener just worked). But in the core of a 3d engine really it should be tight.

  11. I meant to say: “I think that dynamic typing and less object creation always creates more overhead. “

  12. I really really meant to say: “I meant to say: “I think that dynamic typing and more object creation always creates more overhead. “

    Sry been writing a paper for a few hours head is spinning…

  13. @Douglas: Well, I think there *is* a significant impact when parsing collada objects. Sure, I had to use 1 million iterations to see the separation of time, but when you consider the polys, materials, materials lists, DO3DS, texturing that can occur for a 9000 poly scene, I think the difference *could be* very noticeable. If each of those is expecting a vague object in the constructor, bam, there’s an exponential hit to the entire scenario. Am I right?

  14. From a newb perspective, when I started with Papervision experimentation a year ago, one of the first things I had challenges with was grapsing the materials list object. My Flash Develop code hinting did not allow me to easily grasp the properties and methods of the materials list object and so it made the general learning curve more challenging.

  15. @Douglas Thompson
    The code you have provided is not completly correct.
    Yes, it shows the expensive of construct parameter less construction, but only with NO parameter.
    For an exact comparision you have to init those constructor variables in the class as {x:12} does.
    private var a:int, b:int, …
    this.a = a;
    this.b = b; …

    After completing this code the only reasonable differenze will be the type inference and flash internal hashmap generation.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s