Large sim won't load

[Sprotiel]

I have a (very) large sim which I can't load, no matter what I do. I've tried to strip the mutation details, successfully I believe since it works for other files, but the resulting file won't load either, with DB giving the same error message it gives for the full file: "blablabla Invalid property value DarwinBots 380 0".

I'm adding both files, in case my procedure is incorrect.

PS: Seeing the trouble I had to upload the small one, I won't upload the large file here.

[EricL]

You sim file is corrupt.  Why, I'm not sure.

The invalid property errror is beacase the field size is 0x0.  But the root problem happened long before that when reading in the sim file.  A whole bunch of stuff is zero including a bunch of the bots (there are 2508 bots in the file.  The first few look okay but by the time it gets to the last, they are all zero.)

What's strange is that the file goes all 0's.  It's not truncated, it's 0'ed out.  You didn't load it into an editior or anything did you?  

Have you had any file systems issues?  Did you defrag overnight or anything?  Are you running FAT or NTFS?

I'm not sure how far I will get with the sim file.  There could be a bug on writing sim files, reading sim files or it could be something outside DB that messed with the file.  I'll do some more digging and let you know if I find anything...

[EricL]

Hey, is it possible that a bot in your sim has >32000 mutations?

[EricL]

The problem is with the mutation details in bot 2319.  The length of the mutation details is negative, which is an old old bug back when the program didn't allow more than 16k worth of mutation strings (about 300 mutations).

Did this sim originate in an older version of DB like 2.37.6?  Was it ever saved with an older version?

[Sprotiel]

The problem is with the mutation details in bot 2319.  The length of the mutation details is negative, which is an old old bug back when the program didn't allow more than 16k worth of mutation strings (about 300 mutations).

Did this sim originate in an older version of DB like 2.37.6?  Was it ever saved with an older version?

The sim was always run with 2.42.9t. I edited it to remove the mutations details, making sure that DB would read it correctly. I'm quite sure I didn't introduce a negative length.

[EricL]

You hand edited it using an editor, correct?  Sim files are super sensitive to position.  The slightest change can throw off the read frame.  The files arn't text, their binary, so they don't even break on byte boundaries.  Unless you edited the mutation details string length exactrlky right for every bot record in addition to simply deleting the mutation details strings, you will introduce read frame problems which could very well result in the program reading in a byte sequence that when cast results in a negative length.

I will see if I can resurrect the sim by overriding things in the debugger as the file is read in, but no gaurentees.  In general, you void the warrenty when you edit a sim file by hand.

[EricL]

Okay, I tried, but there are other read frame issues in the file.  It's too far gone.  Sorry.

[Sprotiel]

You hand edited it using an editor, correct?

No, I used a script, parsing for mutation details and copying everything else, and also modifying the mutation details length bytefield according to the actual algorithm used in VB code. Yesterday, it seemed to work for other sim files, but not today. I guess I should send you the unedited files.

[EricL]

Okay.  If you have looked at the code, you know there are two mutatiuon details length fields in the file.  One is the old integer one, the other is the new Long valued one.  Your script needs to set the old one to length 1 and set the new one appropriatly.  Or, you could just use the DB program itself to save without mutation details....  

Does the unedited sim file not load?  If it does not, then yes please, compress and send that to me and I will debug that one with vigor.

[Sprotiel]

It appears that I can load a previous autosave, albeit with some mutation details removed. Which is no big deal since I'm forced to remove them anyway. So case closed, I guess.
Edit : the "Fix bot radii" checkbox doesn't seem to be working properly, and I don't recall having trouble with it before the save and reload.

Okay.  If you have looked at the code, you know there are two mutatiuon details length fields in the file.  One is the old integer one, the other is the new Long valued one.  Your script needs to set the old one to length 1 and set the new one appropriatly.  Or, you could just use the DB program itself to save without mutation details....

Actually, I set the short int value, not the long one. But, for some reason, it doesn't work, which is too bad because I'm quite fond of the sort of stuff I get with my phylogenetic tree script (see attachment) and I hoped to be able one day to strip mutation details without losing the information in them.

[EricL]

I think Nums has the right idea.  Mutation details should be stored in sim files by the program as a vector of small structs instead of a giant string and a look up table should be used by the program to display the strings .  This would make them tiny and greatly reduce the need to strip them.  This is on my list, but has not been urgent.

As far as phylogenic trees, etc. go, why don't you add the code the program itself and make it a feature?  I'd be happy to hand merge any source code changes you make into the mainline or help some other way.  Doing it at runtime in VB is a lot easier than using a script to parse sim files! I think we arn't too far away from putting the VB source into a source management system anyway that would allow mutliple authors.  You see, the rest of us want to play with your cool stuff too!

I'll take a look at the bot radii thing.  It's possible the setting got messed up in the sim file by the script...