Darwinbots Forum

Code center => Suggestions => Topic started by: bacillus on January 25, 2010, 04:02:16 AM

Title: DB ate my Hard Drive!
Post by: bacillus on January 25, 2010, 04:02:16 AM

In a spring cleaning of my system, I found out that the DB folder, which I had previously assumed to be of negligible size, had eaten up 2GB of my hard drive! Deleting and zipping simulations cut this down to about 35MB, but still raises an issue. Is there any way to save simulations so they don't take up a huge amount of space? I ask as it strikes me as some part of the original spaghetti code that nobody's been willing to touch for years. (And I won't blame you for avoiding it   )

Title: DB ate my Hard Drive!
Post by: Numsgil on January 25, 2010, 12:30:37 PM

Yes, we could include something that zipped the simulations before saving them to disk.  It would probably make saving faster as well, since usually zipping is faster bytes/sec-wise than disk access.  But we'd need to find some way to hook our VB6 code in to a compression library, which isn't trivial.

Title: DB ate my Hard Drive!
Post by: bacillus on January 25, 2010, 04:51:50 PM

I was thinking more in the direction of saving simulation parameters seperately as they usually stay the same between saves and things like that, but sure, whatever works...

Title: DB ate my Hard Drive!
Post by: Numsgil on January 25, 2010, 07:17:05 PM

It's the DNA and mutation details that take up most of the space.  The simulation details are like 512 byes, if that.

Title: DB ate my Hard Drive!
Post by: bacillus on January 25, 2010, 09:04:19 PM

Quote from: Numsgil

It's the DNA and mutation details that take up most of the space.  The simulation details are like 512 byes, if that.

Maybe making the mutation details optional, then? After a few dozen generations, they are usually too hard to keep track of anyway.

Title: DB ate my Hard Drive!
Post by: Numsgil on January 26, 2010, 01:54:29 AM

They are currently, actually.  "Save without mutation details".

Title: DB ate my Hard Drive!
Post by: bacillus on January 26, 2010, 02:30:06 PM

Oh...  

Title: DB ate my Hard Drive!
Post by: Houshalter on January 27, 2010, 07:00:41 AM

Why do the mutation details take up so much space? It seems like each one is kind of small like, on cycle w mutation x changed bp y to bp z. Thats four variables.

Title: DB ate my Hard Drive!
Post by: ikke on January 27, 2010, 07:24:58 AM

Save last 10 only ?

Title: DB ate my Hard Drive!
Post by: Numsgil on January 27, 2010, 12:41:26 PM

Quote from: Houshalter

Why do the mutation details take up so much space? It seems like each one is kind of small like, on cycle w mutation x changed bp y to bp z. Thats four variables.

They're stored as full strings.  They don't have to be, just no one ever did them differently.

Title: DB ate my Hard Drive!
Post by: bacillus on January 27, 2010, 11:06:12 PM

Quote from: ikke

Save last 10 only ?

I always turn this option on but apparently not. My guess is that new sims don't overwrite other sims.

Title: DB ate my Hard Drive!
Post by: ikke on January 30, 2010, 02:34:29 AM

Quote from: bacillus

[My guess is that new sims don't overwrite other sims.

The chek would be to save a sim every minute, start a new one every hour and see what happens