Darwinbots Forum
Code center => Bugs and fixes => Bug reports => Topic started by: EricL on March 29, 2006, 09:42:21 PM
-
I've been working towards a new drop of the VB version. Will probably show up this weekend or next week. Here's what I've fixed so far:
Saving and loading settings with field sizes > 8 was broken. DB saved a long but was trying to load it into an int. Overflow crash.
Loading a sim from a file now changes all the settings to match the values in the sim. Allows for easy tweaking without having to manually set everything.
Lots of misc. fit and finish changes in charting. They update immediatly upon opening, the lines fit in the chart space correctly, resizing charts works correctly, etc.
Chart dialogs now have an "Update "Now" button which will refresh that chart (and that chart only) immediatly without having to wait the built-in 200 cycles for another data point and refresh. You can ust click away if you want a bunch of data points right now. (I'm not planning on adding many features on the 2.4 VB fork, but this was easy and I kind of wanted it so I did it.)
Various overflow and stability fixes.
A few other non crashing fit and finish things: The Mutation rate slider works better, the Edit menu items invoke the options dialog at the right tab setting and so on.
error.sim outlput enablement in the compiled version.
[you]Things I'm still working on that will probably be in the next drop:[/you]
A couple crashing bugs I'm still working on plus any others reported that I can find/fix.
The issue with mutation being off after saving.
A configurable chart update interval. This is easy enough that I'm goign to just do it.
Internet Organism Sharing (I want this!)
More fit and finish stuff. If it pisses me off sufficiently, I will spend time to fix it.
[you]Things I'm working on that probably will NOT be in the next drop:[/you]
Leagues
Locality-based mutation experimentations. (personal project)
If you have other priorities, let me know. My philosophy is to drop a release once there is enough in it to justify people moving to it.
-E
-
By charts you mean graphs I assume?
I'm excited. Graphs have always been something I wanted to spend time working on but could never justify it over other projects.
-
on the charts/graphs, Eric ...
it seems the colors don't follow the colors assigned to the bots in the
species window, but rather the default VB colors ...
ie ... yellow, red, blue, purple ... and so on in that order.
so if you want you bots to match the chart, you have to ensure
that you assign yellow to the first species, red to the 2nd, and so on.
I haven't been able to find a way to have the charts can use anything
other than those defaults ...
but seeing as you have been playing with them ...
thought I'd throw the idea out.
great work my man ...
I had sorta lost interest ...
as it's no fun when your toys keep breaking. ;)
thanks for all the effort ...
although I don't guess you'd be doing it if you didn't
love playing at this kind of thing.
I'd like to be able to see the code you changed as well ...
so I can go and see what changes you made and how
the fixes work. that's the only way I have of learning this stuff.
so keep good notes, eh? ;)
thanks again.
-
Kind of a suggestion than anything else but anyways...
Do you think it'd be possible to add the ability to concatenate and de-concatenate numbers? It'd be possible to transmit and store simply massive amounts of info with these relativly simple code additions. I was thinking something based on the number of digits in front of a decon command would be best. Ie:
20300 2 decon
Would leave:
20
300
on the stack.
-
Yes, I mean graphs. They are called charts in the source some places and something spanish in others. One of these days I'll have to ask who Carlo is (or was). :)
I'm not really adding much new functionality to graphing, besides the update interval and update now button. I'm really just making them look pretty and work correctly though it would be trivial to add new graph types. If anyone has something specific they want graphed that the program is already maintaining, I could easily add it. But custom graphs, well, that will have to wait. Perhaps one day I'll write a perfmon plugin and export tons of counters. That would give us some cool capabilities on Windows at least. Custom graphs, graph logging to file, alerts (including email) on thresholds, etc.
Another thing I may do dig in and speed up the data point calculations. They look to be always performed whether there are graphs up or not and the calculation for every data point is at least O(nm) where n is the number of bots and M is the number of data points in the (oldest) graph! The number of decendents chart actually calculates this recursivly for every bot for every datapoint for every cycle insetad of just keeping counters! All this is done even when no graph is up, every 200 cycles by default. Do this stuff every time the graph gets updated and its at least O(n^3) to draw a full graph! Update the graph every 10 cycles and the speed imapct is noticable. Gack!
Yea, having graphs use the bot color has been bugging me too and I've changed it so graph colors match bot colors. A little more involved then it seems, but not hard. Another idea I've toyed with is persisting a bot's color and skin in the DNA file... That way when you load a bot, it always uses a specific color/skin unless you change it. Could be the beginnig of more sophsicated persistant visual stuff on day... (I won't do this without seeking input on file formats, etc.)
Griz, some of the things I'm doing have several changes in multiple places in the code. Somr are form tweaks, some property tweaks, some brand new code and routines. Everything is commented in the source and I'll try my best, but I probably won't take the time to write up every detail line by line of everything I've done in a readme.
FYI, one good way to teach yourself is just to find a piece of code your interested in and then just step through it. Set a break point at the top of the routine and then just F8 through it line by line watching what it does.
Cheers,
-E
-
Do you think it'd be possible to add the ability to concatenate and de-concatenate numbers?
Yes, certainly. BUT.....
Right now the stuff I'm doing isn't core. It's UI and bug fixes and trivial little additions and other superficial stuff I can do with the right brain while the left brain watches Stargate. To be honest, I don't feel, um... entitled to add new sysvars or impact the core design in any meaningful way just yet.
So, while it is within my skill set to write the code to do what you suggest, I am going to shy away from doing anything core like this for a few more weeks at which time if all goes well, everything that was supposed to work in the VB version will be working (reliabily) there will be no reason for anyone to run a 3.X version and Nums will be in a position to tell me whether it's worth my while adding new features to the VB fork or cutting over to the C++ code base... :)
-
Shame on you! Carlo Was, Carlo Is and Carlo Will Be forever.
He's the Supreme Being (look in Members) and all who run DBs must worship him (yes, it is in the user agreement).
(http://www.mraxt.de/forum/images/smiles/danke.gif)(http://www.mraxt.de/forum/images/smiles/danke.gif)(http://www.mraxt.de/forum/images/smiles/danke.gif)(http://www.mraxt.de/forum/images/smiles/danke.gif)(http://www.mraxt.de/forum/images/smiles/danke.gif)(http://www.mraxt.de/forum/images/smiles/danke.gif)
-
Well, what can I say, Eric? Awesome work! All those things have been bugging me too. I can't wait to try the new version! :boing:
The thing you mension about data point calculation, maybe fixing that will speed up the program!
When it comes to the color graphs I've been using a workaround; the colors are allways assigned to the bots in a certain order, so I've set the colors of the bots close to the colors on the graphs. Works best if you load the graph before you load the sim. Hehe.
This program is starting to rock! :Headbang:
-
Carlo was the original programmer. He was also italian and liked to code in italian, which caused me no small mess of headaches (and PY before me even more so I'm sure).
He had(has) a fairly good grasp on basic computer science it seems, from various algorithms he's implemented, but I have a feeling based on the original code architecture that it might have been his first large program (it's my first too, so I guess it takes one to know one) as it was incredibly scalable-unfriendly when I first found it (all the code was pretty much in Form1.) and generally had some program flow issues (incredibly dense and long functions for one).
2.4 was as much a code upgrade as a feature upgrade, as I tried to untangle many of the various functions that had been accumulating randomly from various updates.
-
Well, I stand in awe of the Supreme Being and all others who have come before me on this code base. Some of the work here is amazing and the amount of hard core math in the code is quite phenominal. Wish there were more comments and fewer italian varible names, but all and all the code quality is quite high. That's saying something for a VB project of this size and I think I can perhaps claim some authority on the subject. I'm making 2.4 work, but really, I'm only completing the last 1% of a job that was 99% complete already.
That said, 2.42 is going to rock. Besides the fit and finish stuff and small feature tweaks, I've fixed of couple of really nasty core stabilty bugs, one of which was in the core species data structure and was the root of much evil and I hope, most of the overflows people hit in 2.4. Anyway, I'm 250,000 cycles into a huge sim on my latest build: largest field size possible, lots of mutation, complex bots using posion and venom and shots and ties. It routinly knocks up against the 5000 bot limit. I'll drop the release when I get to 1 million cycles and finish one or two more things. There will no doubt still be some paths that end in overflows, but I hope those will be fringe - canceling file load dialogs and that sort of thing. I think, I hope 2.42 is going to be very very stable at the core....
-E
-
How'd you manage 5K? I was having some really oddball effects at about 2000 :sad2: not to mention it begins running in real time.
-
As a school project I'm implementing an algorithm in the C++ source (which I started in the VB version, which I called "buckets") that should change the core program from an O(n^2) complexity to an O(n) complexity. Those huge sims should run incredibly faster.
Most of the really complicated math is mine. I'm a math major you see. Like the shot decay log function, etc.
Feel free to add comments to the code around any code block you had to spend time figuring out. I try to add mucho comments but I'm not always as forthright as I should be.
-
How'd you manage 5K? I was having some really oddball effects at about 2000
I recently bought a new machine. 3Ghz P4 with a Gig of ram (and a totally wicked 24" screen!). Even on the biggest, slowest sims it seems to stay above 1 cycle per second and just flys with the bot count below 1000. Haven't seen any really oddball stuff, but I have fixed a lot of things in my fork and also, with the crazy mutation rates I'm using (mostly to test stuff) I may not notice weird stuff anyway... :)
As a school project I'm implementing an algorithm in the C++ source (which I started in the VB version, which I called "buckets") that should change the core program from an O(n^2) complexity to an O(n) complexity.
I've seen some of that in the VB source. Most execellent sir. Hashtables are your friend! :clap:
-
How'd you manage 5K?
Also, I don't like hitting (or having) the artifical bot limit. This is a problem a lot of programs like this have I.e. how to keep numbers reasonable when there is an abundnace of energy in the sim. Microsft's Terrarium had the same problem and used "sickness" to randomly kill off bots when populations got too high for the machine to handle simulating in reasonable time. Of course, when there is no darwinian selection invovled in which bots live or die, it makes evolution difficult...
I prefer using environmental conditions that increase selective pressures as a mechanism for keeping populations under control and not having any built in limits. Someday I'll do some work here. (The current limit of 5000 is a pretty fundemental artifact of the way the VB code is currently written using arrays instead of linked lists or other more dynamic structures. Changing this would be a major piece of work and this is exactly what Nums is doing in the C++ fork, though the thigns you do for speed, such as using hashtables are not always compatable with the thigns you do for having no upper bounds...)
Anyway, I limit the energy in the system by using a small number (like 4) of really big, high energy veggys that rarely die (start with one in each corner with it's own little starting bot species and you get isolated populations evolving separatly for a time that evlove to stay near their food source but eventually start to venture and come into contact with the other populations once their populations get large enough). I also raiseg costs on bots so that they die off if they don't eat fairly regularly. Adding even a small cost to Flow commands will accomplish this well...
-
I recently bought a new machine. 3Ghz P4 with a Gig of ram
How did you manage to find a P4 that fast? :blink:
It is my understanding that P4s only go about as high as 2Ghz
Anything faster than that is a newer generation of chipsets such as Sempron or Athlon.
Semprons run from about 2400 up to 3200(nominal speed) or so and Athlons run up to about 4 Ghz
-
Actually I take that back to some degree.
You can get a P4 that fast (if you are running a Dell at least) I am in actual fact running a 3.2 Ghz P4 myself right now. :redface:
I just checked.
It is just that most motherboard manufacturers don't support P4s any more.
The Sempron and Athlon processors are designed to run on a motherboard with two memory channels which allow the processor to work in "dual" mode so that it actually processes at twice the nominal speed.
-
Yup, mine's a Dell. I just bought at the knee in the price curve a few weeks ago. I covetted one of the dual core machines, DB will parallel wonderfully I imagine, if architected right, but such machines are still on the other side of the knee for me...
-
I just bought a Compaq Pressario with a Sempron 3200++ processor for my son at Christmas.
It was the cheapest PC anywhere in town. $438 at walmart complete with a 17 inch flat screen monitor.
It only came with 256Mb of RAM but the chipset and motherboard are set up so that if you put two memory cards of equal size into two different memory slots then it will go into dual processing mode.
The strange thing is that when we look in the system settings it only shows the processor running at 1.8 Ghz. need to get the memory upgraded and see what happens. It can hold 4 1gig memory cards
-
DB should be a program that could parallel quite well, but I have no idea if the code is set up right for it. Would be interesting to find out what you might need to do. Maybe add some threads in the engine.
-
The strange thing is that when we look in the system settings it only shows the processor running at 1.8 Ghz
That's because in the last couple of years both AMD and Intel switched to the scoring system which measures "performance". The 3200 refers to that performance measurement, not to GHz. Your processor is supposed to run at 1.8 GHz. What they are saying is that if you took the processors as they were a couple of years ago and run them at 3.2 GHz, they would be as good as today's processors running at 1.8 GHz, because modern processors have a better architecture.
The way I see it is that this is a load of bullshit designed to lure customers to buy newer systems.
-
I'm a compuer science person and that just bleew my mind :D
Maybe we should measure our computers in FLOPS.
-
DB should be a program that could parallel quite well, but I have no idea if the code is set up right for it.
It's not. There are only two threads in the process and one does all the work. This is easy to see. Run perfmon.exe from a command line and add the % processor counters for DB from the Thread object.
Multi-threading it will take some work to make it thread safe. Non trivial, but not rocket science.
-E
-
The C++ code is multithreaded, but only between the GUI and the Engine.
-
I bet we could get a lot of bang for the buck by doing corse grain threading in just a few key places in the engine, like anywhere there is a loop that cycles through the rob array. Spin 10 threads, do 10 robots at a time in parallel. Just given the nature of the beast(s) (pun intended) I think a lot of the loops like this are already relatively parallizable, with few or no critical sections needing protection though we would want to take a very careful walk through anything we thread to make sure. Trying to find multi-threaded timing bugs can drive you crazy - they tend not to repro when you want them to. Much better to not introduce them in the first place...
Even the simplest profile run will tell us which routines are the key ones to focus on for both threading and algorithm improvement. Sigh. One more thing to add to the to do list....
-E
-
I usually do profile the code from time to time. I like to use Intel's VTune even if it is way overpowered for my needs.
I think you're right about setting up different threads for the robot loops. I'll start working on it when I get passed the implementation and debugging stage.
-
I have to say the only improvements I can see in this version is the graph. It looks a lot better and the function where you can set how often it will update. The rest looks the same except for the draw boxes on the species tab, wich has become a complete mess. Lots of things in the GUI doesn't get saved with the sim as promised and some values doesn't stay as set. They keep reset themselves to other values sometimes. :sad2:
-
I like the graphs ... much easier to use now.
like Testlund says ...
something happens to the placement windows in the select Species tab.
my veggies all end up just 'outside' the field in the upper left corner ...
massed together.
the window placement boxes are all scrunched to zero height
at the bottom or top ... difficult to even find them to reposition them.
and even when once done, when going back in to edit settings ...
they all dissapear again.
otherwise ... looks good!!! ;)
will play with it some and see.
-
VB crashed on me once and messed up a bunch of stuff on the options dialog. Thought I had put it all back, but probably missed the placement window default. Does the reset button let you address this?
Can you tell me which settings specifically don't round trip through saved sims?
Thanks,
-E
-
Does the reset button let you address this?
reset?
where?
here's what I get:
start a new sim ...
go to Species and click on say, the Veggie.
the new 'use this color' thing comes up.
fine.
I click on that and the 'borders' of the placement window
and they do one of two things ...
they either default to the full field ...
which one can see by simply clicking in the center ...
the 'borders' then showing ...
or ... they end up having zero height and/or width .
sometimes reduced to 0x0 and somewhere up in the upper left
corner. clicking around and dragging, I can sometimes find it
and expand/move it back where I want.
same with the other species.
ok ... I get them where I want and start the sim. all is well.
now .... going back to edit settings or alter anything ...
I click on the Species and the same thing happens.
the borders reset or dissappear and I have to retrieve them again.
haven't had time to play with save/resave to see what settings
survive or not.
-
and ...
it doesn't appear the bots are placed where I reset the windows ...
but where I found them when clicking on species ...
all squished up against a border. sometimes. ;)
-
The reset button is that button that says, um, "Reset" right above the initial position window on the Species tab. It should reset the position box to be the entire field and let you find it. I think the problem is that the defaut size and Z-order on the dialog got messed up when VB crashed on me. When you come back to it after setting the position, the size is probably still correct, you just can't see it because of the Z-order - its underneath what shoudl be the background...
Are you saying you get the Custom color selection box coming up when you click on a newly added species? I do not see that. The default color shoudl be Green and you should only get the custom color selection duialog if you select 'Custom' in the color dropdown.
-E
Edit: I'm in the code . The inital postion thing is move involved than just Z-order. Will post more tonight...
-
The custom color selection box only comes up if custom colors has been selected for that species before, just as in previous version. The boxes on the other hand... I can't select any postition where I want the species to appear. The only workaround is to click on the reset button, then click once in the box without moving it, then the species will appear spread out all over the field. But it's not possible to chose any other place or size where you want the species to appear. When it comes to the rest of the GUI... If you make settings on all the tabs, start a sim, save the sim, make new settings, start a new sim, then load the old sim, the old sim will only load some settings back. I found the following settings doesn't get saved:
General tab:
Waste treshold, Graphing update interval.
Physics and costs tab, Advanced costs:
All values have been reset to 0, except for the shell cost that displayed this value: 1,401298E-45. Clicking ok and then clicking on the Advanced costs button again will change the shell cost to 1.
Mutations tab:
Current multiplier doesn't get saved.
Restart and league tab:
Restart sim when all robots are dead doesn't work at all.
Recording tab:
Settings here doesn't get saved, and you can't have different values in the fields here.
There might be more settings that doesn't get saved, but these are the onces I've tested.
-
Are you saying you get the Custom color selection box coming up when you click on a newly added species? I do not see that. The default color shoudl be Green and you should only get the custom color selection duialog if you select 'Custom' in the color dropdown.
the very first time I click on the Veggie species in a new sim, it does. after that no.
just that one. surprised me as well. ;)
and yes, the reset does return to the default ... same size/aspect ratio as the field.
however, if you don't do this anytime before you click 'new' ...
the placement windows end up in those bizarre positions ...
my veggies all in a 0x0 field up in the upper lift corner ...
and other species smashed up against a wall or the top/bottom.
so, if you ever want to modify settings ...
you have to come back to species and place them where you want again.
and even then it doesn't always work.
I don't have control over their placement windows anymore ...
sometimes they're close ... other times ... they go wherever.
-
Okay, I've found and fixed all the issues Testlund mentions above with the exeception of the last two.
The inital position box thing was my fault, I broke it with a different fix that impacted the fractional starting positions of saved species. It's solid now.
The round trip issues through saves sims and settings files were a number of different things. I fixed a couple of big things in 2.42.0 that I thought were to blame for all the issues, but didn't do sufficient testing and hadn't caught these.
In the graph update interval and waste threshold cases, the code was just never written to save those settings to sim or settings files (I just added the graph interval code in 2.42.0 and hadn't handled saves to sim files yet I assume the waste threshold was pretty new also). The code to handle them for both sim and settings files has now been added.
In the costs case, an off-by-one error in two different loops was to blame, one in the sim save routine and another in the load routine. These caused positional problems reading/saving sim files which meant that after a certain point in the file, what was read back in was essentailly random and the costs array near the end of the file took the brunt of the damage. The entire array of all sim costs was messed up by this and was essentially random when read from any saved sim file. It all works now.
The mutations slider thing was a case of aggressive initialization. It was getting set to 1 in the sim initialize start routine no matter what got saved or loaded from the sim/settings files.
I'll work on the rest tonight or tomorrow and drop a 2.42.1 tomorrow.
-E
-
Nice work. Large projects like this are always 10% more bug fixing after you think you've finished, everytime you finish. Very annoying.