2.42.7b runs much slower!

[Testlund]

I did a comparison between 2.42.7 and 2.42.7b and where the first runs my sim at 5.2 cycles/sec the other one runs the same sim at 1.4 cycles/sec. I wonder what you have added that cause it to run slower.

[Numsgil]

How many bots and shots are in the sim?  That's going to be important in finding out if the problem is in a very small percentage of functions or if it's some sort of timing problem or...

[EricL]

I would be interested in knowing whether the same slow down is observed in 2.42.7a.

I did make a change between 2.42.7a and 2.42.7b in the management of the shots array size.  It now starts out larger (5000) and grows in jumps of 5000.  I did this because I was hitting some strange crashing bugs when lots and lots of shots get created in a single cycle - more then that previously handled by a single increase in the array size - like when a whole bunch of bots die and their poffs get displayed.  VB was doing something strange that I still don;t understand where the shots array was getting locked.  My thinking is that it will be very rare that more than 5000 shots get created in a single cycle.

The perf impact of this change should be quite minimal, primarily only impacting smaller sims with few or no shots (they will be uncessarily loopoing through a larger shots array).  The impact on larger sims with a "typical" number of shots should be essentially nil - unless I screwed it up of course.

So, the main questions that could help me:

Do you see the slowdown in 2.42.7a?

Does the sim have any shots?  If so, how many typically?

If you turn off poffs, does it change things?

Posting the sim would also be very helpful.

There are some things that have a surprising and unexpected perf impact beyond just total number of shots, bots, ties, shapes, etc..  For example, the exact same sim with the exact same number of bots and shots will run considerably slower if all the bots are close together on the field then if they are spread out.  This is becuase the collision detection routines have to do more work if the bots (or bots and shots) are close enough that a collision is possible that cycle.  If they are far apart, a very fast and simple distance test can rule out a collision.  If they are close, the code has to do a lot more math.  (Thus, if you want null evo sims to run faster, use a big field and turn off planet eaters!)

[Testlund]

I haven't tested version 2.42.7a because I was thinking that the only thing you added there was something with day/night cycles that wasn't interesting for me. But it's probably better if you compare it yourself, so here's a sim wich was saved in 2.42.7.

[EricL]

I think I may have found this.  Allow me to expound.

Over the course of the last few releases, I have slowly changed the way the shots array works.  Before I came on board, the code tried to keep the shots array small and relatively packed.  That is, if there were 900 shots in the simn, the code would keep the array size just above that, say size 1000, and would try to always use and re-use the lower slots to keep from having to increase the array size.  When a slot empitied becuase a shot got deleted (when it impacts or expires) then the code tried hard to re-use that slot instead of using, say, slot 901.  Each time a new shot was created, it walked the array from the beginning and used the first open slot.  This had the advantage of keeping the array size smaller and thus keeping the shot loop size smaller than it otherwise might be, but had the huge disadvantage that new insertions were very VERY expensive, essentially O(n/2) on average.

The above strategy is okay when insertions and deletions are rare but since shots generally have short lifespans and come and go so frequently, the insertion cost of the above algorithm completely and utterly swamped the benifits of maintaining a tightly packed array.  What I have done is to change the insertion algorhithm so that the probability of using any slot is the same.  When a new shot is created, the code starts looking at a location where the last shot was created.  This has the effect of dramatically lowering insertion costs with a reasonable increase in array size.

In 2.42.7b, I made a bunch of additional changes to allow for > 32000 shots in a sim.  I think I screwed up the code that made the change above as part of that and was setting the pointer back to 1 for each new insertion, essentially going back to the old algorithm, but now with a larger shots array, at least for smaller sims with few shots.  Ooops.  

Try this one.
2.42.7c Buddy Drop Download

[EricL]

But it's probably better if you compare it yourself, so here's a sim wich was saved in 2.42.7.

Your sim is a null evo sim with no shots, so my blurb above isn't what you are seeing.
You are probably seeing the loop cost of the difference in the initial size of the shots array (5000 v. 300 previously).

I'd be curious to know what your machine is, CPU, how much RAM, etc.  I did not expect this.

Try this buddy drop.  The initial size of the shots array is only 50.  This should make a difference for sims such as null evo sims where there are few or no shots and only occasional deaths (which gneerate poff shots).  As long as the total shots remains below 50, the shots aray will stay small and the loop size will similarly be small (even smaller than it used to be, so maybe this will be even faster then previous versions).


2.42.7d Buddy Drop Download

[Testlund]

I just tried it. It made no difference, and I'm not surprised, because when you load up a sim there are no shots at all. So I think the problem lies somewhere else. Sorry.  

Ok, wait. we posted at the same time here. I'll try 7d. just a moment.

[Testlund]

Ok, tried 7d with no difference either. My computer specs are AMD Athlon 1.53 GHz, 512 RAM.

[EricL]

Ok, tried 7d with no difference either. My computer specs are AMD Athlon 1.53 GHz, 512 RAM.

Ok, I'll keep working on it...  

The good news is that the changes above are still good ones.

[EricL]

Okay, try this one.

2.42.7e Buddy Drop Download

[Testlund]

No difference with that either. Maybe you have just reached a limit how much more you can add to the program without slowing it down. There has to be a limit sometime.

[EricL]

No, if the change is as dramatic as you say, I screwed something up.  While you are correct that some new features such as shapes add an additional, incremental, unavoidable performance cost if and when they are used, nothing I have done or plan to do should slow down a sim that does not use the new features.  In fact, many changes I have made such as the changes to the shots array indicated above should speed up such sims.

No, if you are really, truly seeing what you say, that you can load the same sim in two different versions and see such a dramtaic perf difference between the two versions over the same cycle interval, then something is very very wrong.

For my sanity, can you please re-verify that this is indeed what you see and that it isn't just that you were installing a new windows patch in the background or doing background file cleanup or something when you saw the slower perf....  Thanks.

I will be doing my own tests later tonight or tomorrow PST when I get a few minutes to rub together.

[Testlund]

I just compared by first just loading the sim in 2.42.7. Didn't add any shapes or anything, and checked the speed. It was above 5 cycles/sec. Then I just loaded that same sim in these later body drops and the speed showed a pending between 1.3 and 1.4 cycles/sec. If I whould have made anything else not related to DB then the speed should slow down in 2.42.7 too I think. I don't run anything in the background other than my firewall when I run DB.

[Testlund]

I have also found something else. The button where you can toggle to fast mode. In 2.42.7 when clicking that button I can increase the speed with +2 cycles/sec, but in 2.42.7e it has no effect.

[Henk]

New issue;
When using teleporters intrasimulation and disabling video output, lines from teleporter to destination are still drawn.