General > Off Topic

Personal Project -- Biogenesis and QLearning

(1/2) > >>

rwill128:
Hey all,

So I hope there's nothing wrong with me posting about another evolution sim program on here. If there is, I'll be okay with someone letting me know.

But anyway, I wanted to share the results of the project I've been working on for the last few days. It finally reached a milestone where I have a working "proof-of-concept" program, and now I want to tweak it so it looks prettier and has lots of interesting features. I've been using Biogenesis (Specifically, a mod of it called "Biogenesis Color Mod" : https://sourceforge.net/projects/biogenesiscolor/) and a QLearning framework I found at http://elsy.gdan.pl/

Check out some of the samples on the Elsy website. He's actually got a nifty little library there. He implemented a type of reinforcement learning that uses neural networks, and the "Wanderbot" or "Apollo Lander" examples both show what kind of tasks it's effective for.

Anyway -- I decided I wanted to combine Biogenesis, which I've always found to be a fascinating program because of its relative simplicity (but cool results), with this other guy's QLearning framework.

In Biogenesis, each creature's lines have different functions based on their color, and each creature's "genes" just store the overall shape of the creature. So more effective shapes are selected for. But movement patterns are largely unintelligent. But now, as of about 5PM today, I've got a working prototype of a Biogenesis mod where each creature is connected to its own (surprisingly effective) neural net, and can make movement decisions based on the information passed to that net.

---

With all the evolution sim enthusiasts around here, I thought I'd come and ask if anyone's experimented with either of these programs before, and if they have, do they have any input on this concept? Any ideas of how you'd like to see it implemented? How should creature's brains be given reward feedback? Should their "preferences" be stored as hereditary information?

Numsgil:
Sounds cool; I haven't seen either project before.  Figuring out a good reward metric is always the tricky part with any sort of directed learning optimization (whether using a neural net or genetic algorithms or anything along those lines).

rwill128:
I'm thinking a reward strongly weighted toward "Did you gain more health than you would have through photosynthesis over this last click?" would work well.

But I was also thinking about passing a bunch of different parameters to the learning framework, along with a short genetic code that describes which ones to use as a reward (and with what weights). Then just let the evolution of the simulation figure it out. :)

Could end up with some cool effects... for example... altruistic behavior: a creature that "wants" the health of the organism it sees if that organism has similar color segments. Could be selected for naturally, too, I'd think.

Botsareus:
In DB2 the findbest algorithm searches for all children and all childrens children etc. total energy and selects for the best result.
I am thinking you can modify that and select for 'better then average' or 'worse then average'.
If you need a floating point you can do current / average. Better should be > 1 Worse should be < 1

Botsareus:
Anyway, that is just an idea for one of the parameters. Let us know when you put it all together. ok?

Navigation

[0] Message Index

[#] Next page