Author Topic: Evolving Reproduction  (Read 11424 times)

Offline EricL

  • Administrator
  • Bot God
  • *****
  • Posts: 2266
    • View Profile
Evolving Reproduction
« on: December 07, 2006, 01:53:55 PM »
Right now, a bot must not only evolve a .repro sysvar followed by a store in order to reproduce, but must also store a positive number into that location.  That is

407 .repro store

will result in reproduction but

-407 .repro store

will not.  The code currently ignores reproduction requests where a negative number is stored into .repro.

Additionally, the current code treats any number above 99 as 99.  Thus

407 .repro store

is equivalent to

99 .repro store

This is one reason the phenomem of bots giving 99% of their nrg to their offspring is such a common occurance in evo sims.  Evolving it is 320 times more probable then evolving reproduction using some percentage other than 99%.

I would like to change both of these such that stores of negative values to .repro result in reproduction and stores of values >99 are MODed 100.  The result would be that storing any non-zero value into .repro would result in reproduction and there would be an equal evolution probability for all reproduction percentages.

Comments?
« Last Edit: December 07, 2006, 01:55:00 PM by EricL »
Many beers....

Offline shvarz

  • Bot God
  • *****
  • Posts: 1341
    • View Profile
Evolving Reproduction
« Reply #1 on: December 07, 2006, 02:47:39 PM »
How about making it circular?  So that 101 .repro store is the same as 1 .repro store  Seems more fair to me...

As for negatives we can either ignore the sign completely, or set them to 0.  Or (here's an idea!) set off a timer during which the bot cannot reproduce even if a positive number is stored into .repro  So that -20 .repro store would mean that bot cannot reproduce for 20 cycles no matter what.  This kind of thing may be useful for something like multibots where a delay in reproduction is needed.
"Never underestimate the power of stupid things in big numbers" - Serious Sam

Offline Numsgil

  • Administrator
  • Bot God
  • *****
  • Posts: 7742
    • View Profile
Evolving Reproduction
« Reply #2 on: December 07, 2006, 02:48:00 PM »
Eric, I have a slightly alternative suggestion, but it breaks backwards compatibility.  Most sysvars have a domain or range in the [-1000, 1000] range.  I would bump the reproduction values from 1-99 to 1-999, and then mod the values.

But again, this breaks backwards compatibility, so it's probably not something to add at the moment.
« Last Edit: December 07, 2006, 02:49:10 PM by Numsgil »

Offline EricL

  • Administrator
  • Bot God
  • *****
  • Posts: 2266
    • View Profile
Evolving Reproduction
« Reply #3 on: December 07, 2006, 02:57:04 PM »
Quote from: shvarz
How about making it circular?  So that 101 .repro store is the same as 1 .repro store  Seems more fair to me...
This is precisely what MODing means and is precisely what I am suggesting...

Quote from: shvarz
As for negatives we can either ignore the sign completely, or set them to 0.
I like equally out the probabilities, so I prefer to ignore the sign.


Quote from: shvarz
Or (here's an idea!) set off a timer during which the bot cannot reproduce even if a positive number is stored into .repro  So that -20 .repro store would mean that bot cannot reproduce for 20 cycles no matter what.  This kind of thing may be useful for something like multibots where a delay in reproduction is needed.
Assinging a waiting interval between reproducing or a cost to reproducing too soon has been discussed recently elsewhere and I view it as a separate issue to this question, which is laser focused on the domain of mutations which can result in valid reproduction DNA.  I like where you are going, but I see it as a separate discussion.
Many beers....

Offline EricL

  • Administrator
  • Bot God
  • *****
  • Posts: 2266
    • View Profile
Evolving Reproduction
« Reply #4 on: December 07, 2006, 03:13:52 PM »
Quote from: Numsgil
Eric, I have a slightly alternative suggestion, but it breaks backwards compatibility.  Most sysvars have a domain or range in the [-1000, 1000] range.  I would bump the reproduction values from 1-99 to 1-999, and then mod the values.

But again, this breaks backwards compatibility, so it's probably not something to add at the moment.

The domain of sysvars is [-32000, 32000].  Mutations to DNA values which get stored to sysvar values will range in this range.   I understand what you are saying in that the domain of vaues that actually do something interesting is often much more limited depending upon the sysvar.  My goal is to address that for this sysvar and in the future for others.

I'd prefer to treat the question of adding more granularity to sysvar operations and the question of what percentage of mutations result in valid sysvar actions (and the probability distribution of mutating those values) as two separate issues.

Your suggestion of increasing the granulatiry for the .repro (and .mrepro) sysvar(s) so that bots can give offspring 50.8% of their nrg for example where today it must be 50 or 51 may be desirable.  Certainly I would not be opposed to it but for the backward compatability issues but to be honet, I don't really see it as that critical.  Yes, what you suggest has the side effect of increasing the percentage of mutations that result in DNA that does something, but I view the Abs and the MOD portions as much more important that the increased granularity and I think would prefer to do those without changing the granularity and breaking backward compatability.
Many beers....

Offline EricL

  • Administrator
  • Bot God
  • *****
  • Posts: 2266
    • View Profile
Evolving Reproduction
« Reply #5 on: December 07, 2006, 03:21:15 PM »
FYI, I am deblibertly suggesting that multiples of 100 are in fact equivalent to 0.  Thus 641 out of the 64001 possible values that could be stored into .repro would not result in reproduction.
Many beers....

Offline Numsgil

  • Administrator
  • Bot God
  • *****
  • Posts: 7742
    • View Profile
Evolving Reproduction
« Reply #6 on: December 08, 2006, 12:22:52 AM »
Quote from: EricL
The domain of sysvars is [-32000, 32000].  Mutations to DNA values which get stored to sysvar values will range in this range.   I understand what you are saying in that the domain of vaues that actually do something interesting is often much more limited depending upon the sysvar.  My goal is to address that for this sysvar and in the future for others.

I'd prefer to treat the question of adding more granularity to sysvar operations and the question of what percentage of mutations result in valid sysvar actions (and the probability distribution of mutating those values) as two separate issues.

Your suggestion of increasing the granulatiry for the .repro (and .mrepro) sysvar(s) so that bots can give offspring 50.8% of their nrg for example where today it must be 50 or 51 may be desirable.  Certainly I would not be opposed to it but for the backward compatability issues but to be honet, I don't really see it as that critical.  Yes, what you suggest has the side effect of increasing the percentage of mutations that result in DNA that does something, but I view the Abs and the MOD portions as much more important that the increased granularity and I think would prefer to do those without changing the granularity and breaking backward compatability.

I like modding, absoluting, and clamping values to ensure that as many things as possible do something.  What's I'm proposing is for a next generation DNA specification, with old DNA either being deprecated or mutually supported.  At present, here are the sysvars that operate in the -1000, 1000 range:

Movement (.up, .down, etc.)
Turning (actually it's 1256, but I'd like to move it to 1080 as was suggested a while ago) and angles (and there are alot of sysvars in this category)
Positive Shots
Memory locations
Tie Length

And other sysvars, such as eyes, etc. return and use values in a smaller range in the 100s, so I'm toying with the idea of "Upgrading" and redefining alot of commands, sysvars, etc. to work in the range -1000, 1000.

The physical memory would still record -32000 to 32000, but I think standardizing the domains and ranges of sysvars into a smaller and well defined range would make things a little more interesting.

In the end, I think this is one of the fundamental limitations about evolution, with very real implications to real world biology.  The numerical systems ALife uses has all sorts of problems, as we've found out.  Evolution is terrible at finding exact values.  Real biology probably figured this out a while ago, and doesn't work in a way that's isomorphic to the system we or most other alife systems work, specifically because the exactness of numbers is too difficult to figure out.  Real evolution probably works as a balancing act between competing genetic forces.

Offline EricL

  • Administrator
  • Bot God
  • *****
  • Posts: 2266
    • View Profile
Evolving Reproduction
« Reply #7 on: December 08, 2006, 12:38:27 AM »
Cool.  I agree with everything you say here, especially the comments about the descrete nature of DB and its relationship to more 'analog' biological systems.  When we make the big break from the past and break compatability, I'll all for adding additional sysvar granularity.

I think for this imminent drop however, as you say, we should not make such a break.  I will however, add the mod and abs().  In fact, I already have.  
Many beers....

Offline Jez

  • Bot Overlord
  • ****
  • Posts: 788
    • View Profile
Evolving Reproduction
« Reply #8 on: December 08, 2006, 08:21:33 AM »
Equal probability of different repro values being evolved sounds cool to me!

I also like the sound of the direction the ideas for DBv3? are heading, even if it is going to cause enormous problems for old bots.
If you try and take a cat apart to see how it works, the first thing you have in your hands is a non-working cat.
Douglas Adams

Offline Numsgil

  • Administrator
  • Bot God
  • *****
  • Posts: 7742
    • View Profile
Evolving Reproduction
« Reply #9 on: December 08, 2006, 03:22:38 PM »
Thanks.  

Offline Zinc Avenger

  • Bot Builder
  • **
  • Posts: 56
    • View Profile
Evolving Reproduction
« Reply #10 on: December 09, 2006, 02:07:43 PM »
Hang on a sec!

Evolving bots to do something conditionally (as opposed to constantly) nearly always ends up on storing invalid values into locations until they hit a valid one (as the example I gave in Mutation Sims, the evolved bot that stores *.robage - 1365 into .repro so it doesn't reproduce for the first 1365 cycles of its life). I agree 100% on the "wrapping" of positive numbers, I think the abs idea would make not constantly reproducing vastly harder.

Offline EricL

  • Administrator
  • Bot God
  • *****
  • Posts: 2266
    • View Profile
Evolving Reproduction
« Reply #11 on: December 09, 2006, 03:20:06 PM »
Hmmm.   I think this comes down to a general philosophical question of genome design having to do with sysvar space sparsity and how conditionals are expressed/evolved in the DNA.

In general, if the space of valid sysvar values is sparse, then as you say, bots can evolve code where values are stored each cycle and the result of some calculation governing the stored value is essentiallly the conditional.  Nothing happens until they hit upon the right magic value, or a value that is in the small range relative to the space of all possible values.

If on the other hand, storing just about any value into a sysvar reuslts in some morphological action, then bots would need to evolve conditional logic which dictates whether the store occurs or not if they did not want it to happen every cycle.

Using your example, it's the difference between

cond
start
*.robage 1365 sub .repro store
stop

AND

cond
*.robage 1365 >
start
(any number) .repro store
stop

Personally, I prefer the latter design approach for several reasons:

1) I think it results in faster evolution.  When the space of possible sysvar values that result in morphological action is not sparse, a given string of DNA has a higher probability of doing something which selection can then favor or not favor.  Action evolves first, then conditional action.

2) I think it results in a milder slope in the fitness landscape leading towards stable traits and strategies.  For example, in the first approach, incremenenting or decrementing through sysvar values as in the example above as a means of hitting upon a value which does something would probably be a common approach bots would evolve.  Makes sense for evolution to just walk through every value to find the ones that work, right?  But because the space is spares and the range of values that do something contigious, the result is that nothing happens for a while and then suddenly a lot happens.  In the example above, it reproduces every cycle after beginning its 1365th cycle of life, giving each offspring a differnet nrg percentage.  I would claim (without evidence) that it is harder (I.e. that the genetic distance is further, requiring more subsequent mutations) to go from this genome to one where a specific nrg percentage is used (or a number of other 'stable' strategies) than it would be were the sysvar space non-sparse and the conditional evolved as a conditional for performing the store, not the value.

Try the mental excercise of mutating different single base pairs in both genes above I.e. change a number, change a sign, change a operator, etc.  Which gene is 1 base pair away from a larger number of interesting reproduction strategies?  I would claim the second.

Granted I am arguing from opinion without data, but there's my argument.  I just prefer a dense sysvar value space.
Many beers....

Offline Zinc Avenger

  • Bot Builder
  • **
  • Posts: 56
    • View Profile
Evolving Reproduction
« Reply #12 on: December 09, 2006, 05:47:44 PM »
Cond-start-stop is, as far as I am aware, exclusively found in hand-coded bots. After many hundreds of millions of cycles of evolution in countless sims, I have not managed to find a single instance of this structure evolving in a workable state. I normally end up with flow control commands all over the place with no rhyme or reason.

It is not only unlikely (what's the chance of getting a cond, then the condition, then a start, then the action, then a stop from a purely evolved genome?) but it is fragile - it is far too easy for mutations to break it. Change any of those instructions or flow commands and the entire thing stops working.

I'm all for wrapping values in, say, .repro so that any positive integer is accepted with meaningful result. I agree with what you say about allowing the maximum possible interesting mutations from any given starting point. But in your example, the only way to evolve not reproducing is through the useful, powerful but fantastically unlikely cond-start-stop structure.

I also agree yes, an invalid inputs method of behaviour control is a long way away in "genetic distance" from the "ideal", but the "ideal" is further removed still from a bot without any behaviour control.

It seems very hard in the current scheme of things for a bot to evolve to do something not-constantly. So far I've only seen storing numbers in random locations until it happens to hit the right sysvar (X Y rnd store, where X is a valid input and Y > the memory location it really wants) and invalid inputs which has already been discussed.

I'd like to hear if anyone has actually seen any other form of behaviour self-control evolve.

Offline EricL

  • Administrator
  • Bot God
  • *****
  • Posts: 2266
    • View Profile
Evolving Reproduction
« Reply #13 on: December 09, 2006, 07:28:36 PM »
You have a very good point regarding the rarity of cond start stop in evolved DNA.  I havn't done enough (any) reverse engineering analysis of evolved DNA to know whether bots are evolving their own conditional flow using some sort of analogious mechanism or whether they are all just big single genes or if we just need more cycles for behaviour complex enough to need conditional logic to evolve.  Perhaps all of the above.  

Your point regarding fragility is particulary apt and DNA fragility, particularly as we woudl like to evolve stuff much faster than nature with higher mutation rates, worries me .  I have long wanted to add loci-specific mutation probability coding so that bots can evolve to have low mutation rates for important structures or sequences in thier DNA, but of course that does not exist yet.

I will point out that in my suggestion, there is a 1 in 100 chance of a stored value in .repro not resulting in reproduction (any multiple of 100) but that does not address your main point, which I will concede.  I'll change it back and have all negative values do nothing and MOD positive values.  That leaves a little less than 50/50 chance of a value stored to .repro resulting in reproduction and the probability of each reproduction percentage equal.    

I fully agree think that both the subjects of evolved flow control and sysvar sparsity are critical disscussions that should be continued.
Many beers....

Offline Greven

  • Bot Destroyer
  • ***
  • Posts: 345
    • View Profile
Evolving Reproduction
« Reply #14 on: December 15, 2006, 10:16:55 AM »
Zinc Avenger well said. Introducing artificial constraints on DNA/genome level aint doing any good, "Life takes it own path" or whatever Dr. Ian Malcolm said in Jurassic Park.

It would indeed be funny to create a DNA parser, which could turn evolved DNA into a more readable form.
10010011000001110111110100111011001101100100000110110111000011101011110010110000
011000011000001100010110010111101001110100110010111100101000001000001111001011101
001101001110011011010011100011110100111000011101100100000100110011010011100110110
010110000011100111101001110110111101011101100110000111101001101001110111111011101
01100100000111010011010001100001110111010000010001001000010100001