Author Topic: DB3 progress and networking  (Read 14347 times)

Offline Peter

  • Bot God
  • *****
  • Posts: 1177
    • View Profile
Re: DB3 progress
« Reply #30 on: April 12, 2015, 03:55:51 PM »
DB3 will be multithreaded, but what that gives I'm going to suck up with an even more aggressive simulation.  You could run it on some $10k "supercomputer" and a simulation might only run twice as fast as a dinky $500 machine.  If we want the sorts of bot-cycles that make a good ALife sim we're going to need to think wide (clusters) instead of tall (servers).
Kinda open question, but what amount of parallelization can be achieved in DB3? What's the maximum amount of processors it could fully use extra. 4, 12, 30, no clue yet? Is there a module that can hardly be parallelized?
Oh my god, who the hell cares.

Offline Numsgil

  • Administrator
  • Bot God
  • *****
  • Posts: 7742
    • View Profile
Re: DB3 progress
« Reply #31 on: April 12, 2015, 04:33:32 PM »
DB3 will be multithreaded, but what that gives I'm going to suck up with an even more aggressive simulation.  You could run it on some $10k "supercomputer" and a simulation might only run twice as fast as a dinky $500 machine.  If we want the sorts of bot-cycles that make a good ALife sim we're going to need to think wide (clusters) instead of tall (servers).
Kinda open question, but what amount of parallelization can be achieved in DB3? What's the maximum amount of processors it could fully use extra. 4, 12, 30, no clue yet? Is there a module that can hardly be parallelized?

I think the majority of CPU cycles are going to be going to optimization problems, which tend to parallelize pretty well up to maybe half the size of the problem?  So say the problem size scales with the number of panels in the world.  Say each bot has 16 panels on average and there are 100 bots in the world.  That works out to around 800 CPU cores I could probably utilize pretty well.  It's a good candidate for GPGPU work, for instance.  But the work scales up faster than the cores.  By which I mean, double the size of the problem, and you ~8x the amount of work while you can only really 2x the number of cores you have working on it.

Also, most of these optimization problems are sparse, so in the long run I expect to use some sparse algorithms instead, and get maybe a factor of 100x speed up there, but utilize far fewer CPU cores simultaneously.  Then maybe it scales to just dozens of CPUs regardless of the problem size?  Hard to say.

What are these optimization problems?  Things like fluid and bot deformation and physics to a certain limited extent.  For fluid especially, enough small things working in concert can create some fairly global effects.  Like if you had enough bots pushing the fluid leftwards you could get basically a river current going.  I'd like to allow that sort of thing, and the math for that ramps up pretty quick.

Offline spike43884

  • Bot Overlord
  • ****
  • Posts: 656
    • View Profile
Re: DB3 progress
« Reply #32 on: April 13, 2015, 08:14:35 AM »
DB3 will be multithreaded, but what that gives I'm going to suck up with an even more aggressive simulation.  You could run it on some $10k "supercomputer" and a simulation might only run twice as fast as a dinky $500 machine.  If we want the sorts of bot-cycles that make a good ALife sim we're going to need to think wide (clusters) instead of tall (servers).
Kinda open question, but what amount of parallelization can be achieved in DB3? What's the maximum amount of processors it could fully use extra. 4, 12, 30, no clue yet? Is there a module that can hardly be parallelized?

I think the majority of CPU cycles are going to be going to optimization problems, which tend to parallelize pretty well up to maybe half the size of the problem?  So say the problem size scales with the number of panels in the world.  Say each bot has 16 panels on average and there are 100 bots in the world.  That works out to around 800 CPU cores I could probably utilize pretty well.  It's a good candidate for GPGPU work, for instance.  But the work scales up faster than the cores.  By which I mean, double the size of the problem, and you ~8x the amount of work while you can only really 2x the number of cores you have working on it.

Also, most of these optimization problems are sparse, so in the long run I expect to use some sparse algorithms instead, and get maybe a factor of 100x speed up there, but utilize far fewer CPU cores simultaneously.  Then maybe it scales to just dozens of CPUs regardless of the problem size?  Hard to say.

What are these optimization problems?  Things like fluid and bot deformation and physics to a certain limited extent.  For fluid especially, enough small things working in concert can create some fairly global effects.  Like if you had enough bots pushing the fluid leftwards you could get basically a river current going.  I'd like to allow that sort of thing, and the math for that ramps up pretty quick.

About the river thing...Then you need particles, because to create a fluid with momentum, god that'd be really intensive nums. really intensive. I think it'd be better spent to be adding resources which the bots would utilise, such as oxygen, carbon, hydrogen, nitrogren, amino acids, glucose, fructose and starch. Also, an interesting thing to see would be an Z axis of some sort, maybe 2 layers on the Z axis which bots can go into, to drift over things. It'd create a new challenge of eyes but it'd surely be fun.
Autism can allow so much joy, and at the same time sadness to be seen. Our world is weird, and full of contradiction everywhere, yet somehow at moments seems to come together, and make near perfect sense.

Offline Numsgil

  • Administrator
  • Bot God
  • *****
  • Posts: 7742
    • View Profile
Re: DB3 progress
« Reply #33 on: April 13, 2015, 01:19:13 PM »
About the river thing...Then you need particles, because to create a fluid with momentum, god that'd be really intensive nums. really intensive.

You mean simulate the fluid as individual particles bouncing around?  No, that's not a good way to do it.  Instead I'm simulating the fluid using "panel methods".  Specifically the fluid sim is a simplification of an incompressible potential flow, commonly used for simulating things like airfoils.  Basically each panel in a bot can pull or push the fluid so that the fluid won't pass through it.  This pulling or pushing takes the form of hills and valleys in the "potential" of the flow, and the velocity of the fluid at any point in space can be calculated as the sum of the slopes (gradients) of these hills and valleys at that point.  Which is a nicely linear problem.  Expensive to solve, but not that expensive.  Not entirely computationally dissimilar to if you connected each bot together using a spring, or played around with "planet eaters" mode in DB2.  But faster since I know what I'm doing :)

Have you seen the fluid demo I put up around two years ago? 

Quote
I think it'd be better spent to be adding resources which the bots would utilise, such as oxygen, carbon, hydrogen, nitrogren, amino acids, glucose, fructose and starch.

There'll be something along those lines certainly.  From what research I can find it seems that biodiversity (which is one of those holy grail goals I want for DB3) arise in nature from energy rich/resource poor areas.  So I need to simulate limiting nutrients, I think, and that implies I need to simulate nutrients.

Almost certainly it won't be things like oxygen or carbon dioxide.  That particular feed back loop is only really interesting on a global scale, so I don't think it's worth the effort for us.  That is, it's not like rain forests are oxygen rich because of all the plants.  The atmosphere is too well mixed for that.

But something analogous to phosphorus or potassium seems like a good idea.  Something necessary to build panels or other organelles probably.  Then bots can fight each other not just for energy to sustain themselves but essential nutrients they need to grow larger and reproduce.  Animals have an easy time of it, since they can rob from the plants. The plants have a harder time of it, since they can't go looking for more.

Quote
Also, an interesting thing to see would be an Z axis of some sort, maybe 2 layers on the Z axis which bots can go into, to drift over things. It'd create a new challenge of eyes but it'd surely be fun.

I'm thinking through having "leaves" in a third dimension for plants.  So plants can compete for space and access to light without preventing animals from running between them.  The other option for that I've been thinking about is having something like shapes emitting light from their surface.  It doesn't map well to how our universe works, but then it would encourage plants to crowd around the light giving shapes and the spaces between shapes would be mostly empty.  But even here I maybe want plants to be able to connect to each other using "roots", and I wouldn't want the roots to block animals from running around.

So while I like the simplicity of a 2D universe I might end up going with an "under" and/or "over" dimension to allow very specific sorts of things to not compete for space with the main 2D layer.  Or put another way, maybe physics between bots and some things they can build are disabled.

Offline spike43884

  • Bot Overlord
  • ****
  • Posts: 656
    • View Profile
Re: DB3 progress
« Reply #34 on: April 14, 2015, 07:00:39 AM »
About the river thing...Then you need particles, because to create a fluid with momentum, god that'd be really intensive nums. really intensive.

You mean simulate the fluid as individual particles bouncing around?  No, that's not a good way to do it.  Instead I'm simulating the fluid using "panel methods".  Specifically the fluid sim is a simplification of an incompressible potential flow, commonly used for simulating things like airfoils.  Basically each panel in a bot can pull or push the fluid so that the fluid won't pass through it.  This pulling or pushing takes the form of hills and valleys in the "potential" of the flow, and the velocity of the fluid at any point in space can be calculated as the sum of the slopes (gradients) of these hills and valleys at that point.  Which is a nicely linear problem.  Expensive to solve, but not that expensive.  Not entirely computationally dissimilar to if you connected each bot together using a spring, or played around with "planet eaters" mode in DB2.  But faster since I know what I'm doing :)

Have you seen the fluid demo I put up around two years ago? 

Quote
I think it'd be better spent to be adding resources which the bots would utilise, such as oxygen, carbon, hydrogen, nitrogren, amino acids, glucose, fructose and starch.

There'll be something along those lines certainly.  From what research I can find it seems that biodiversity (which is one of those holy grail goals I want for DB3) arise in nature from energy rich/resource poor areas.  So I need to simulate limiting nutrients, I think, and that implies I need to simulate nutrients.

Almost certainly it won't be things like oxygen or carbon dioxide.  That particular feed back loop is only really interesting on a global scale, so I don't think it's worth the effort for us.  That is, it's not like rain forests are oxygen rich because of all the plants.  The atmosphere is too well mixed for that.

But something analogous to phosphorus or potassium seems like a good idea.  Something necessary to build panels or other organelles probably.  Then bots can fight each other not just for energy to sustain themselves but essential nutrients they need to grow larger and reproduce.  Animals have an easy time of it, since they can rob from the plants. The plants have a harder time of it, since they can't go looking for more.

Quote
Also, an interesting thing to see would be an Z axis of some sort, maybe 2 layers on the Z axis which bots can go into, to drift over things. It'd create a new challenge of eyes but it'd surely be fun.

I'm thinking through having "leaves" in a third dimension for plants.  So plants can compete for space and access to light without preventing animals from running between them.  The other option for that I've been thinking about is having something like shapes emitting light from their surface.  It doesn't map well to how our universe works, but then it would encourage plants to crowd around the light giving shapes and the spaces between shapes would be mostly empty.  But even here I maybe want plants to be able to connect to each other using "roots", and I wouldn't want the roots to block animals from running around.

So while I like the simplicity of a 2D universe I might end up going with an "under" and/or "over" dimension to allow very specific sorts of things to not compete for space with the main 2D layer.  Or put another way, maybe physics between bots and some things they can build are disabled.

Do remember I wasn't here 2 years ago :P (also I just quickly scimmed over your fluid sim thing just in the past 45 seconds really) anyway, if your simulating panels for fluids, im presuming your working it more off surface tension. If you were to include surface tension via the panels oxygen could be more key than I thought, I wasn't originally thinking about oxygen for rich/poor area's and was more thinking of the bots spending some energy to take in oxygen, to then perform respiration etc. which would make them struggle in a CO2 rich sim. But if you included surface tension, you could create air bubbles and such...
Also, could you make something to help bots 'attach' to non-bot things more. I mean water droplets or shapes would be nice to be interacted with, I've always wanted to use shapes or such to make an amoeba with its little dwelling.
Autism can allow so much joy, and at the same time sadness to be seen. Our world is weird, and full of contradiction everywhere, yet somehow at moments seems to come together, and make near perfect sense.

Offline Numsgil

  • Administrator
  • Bot God
  • *****
  • Posts: 7742
    • View Profile
Re: DB3 progress
« Reply #35 on: April 14, 2015, 01:24:58 PM »
im presuming your working it more off surface tension.

Surface tension is a different phenomenon.  If you want the closest analog that makes physical sense, I'm simulating pressure (only that's not really exactly true).

Quote
If you were to include surface tension via the panels oxygen could be more key than I thought, I wasn't originally thinking about oxygen for rich/poor area's and was more thinking of the bots spending some energy to take in oxygen, to then perform respiration etc. which would make them struggle in a CO2 rich sim. But if you included surface tension, you could create air bubbles and such...

It'll have a primitive form of respiration, but it'll work more like the "speed" of metabolism (how fast you can break things apart for nrg, or how fast you can build sugars from light using photosynthesis) is controlled by how many panels you have.  I really don't want to simulate anything that's supposed to be well mixed.

Quote
Also, could you make something to help bots 'attach' to non-bot things more. I mean water droplets or shapes would be nice to be interacted with, I've always wanted to use shapes or such to make an amoeba with its little dwelling.

Shapes will be made up of panels, too.  By which I mean the surface of shapes will also be composed of equilateral line segments.  Panels can get "sticky" and attach to other panels.  So in that way bots can connect to each other or to their surroundings.

Shapes will also be destructible.  So bots can remove material from a shape and carve out a little hole.  Bot corpses will tend to "clump" over time and form new shapes (partly to keep the environment interesting, partly to make the sim run faster).

Offline spike43884

  • Bot Overlord
  • ****
  • Posts: 656
    • View Profile
Re: DB3 progress
« Reply #36 on: April 15, 2015, 07:29:03 AM »
im presuming your working it more off surface tension.

Surface tension is a different phenomenon.  If you want the closest analog that makes physical sense, I'm simulating pressure (only that's not really exactly true).

Quote
If you were to include surface tension via the panels oxygen could be more key than I thought, I wasn't originally thinking about oxygen for rich/poor area's and was more thinking of the bots spending some energy to take in oxygen, to then perform respiration etc. which would make them struggle in a CO2 rich sim. But if you included surface tension, you could create air bubbles and such...

It'll have a primitive form of respiration, but it'll work more like the "speed" of metabolism (how fast you can break things apart for nrg, or how fast you can build sugars from light using photosynthesis) is controlled by how many panels you have.  I really don't want to simulate anything that's supposed to be well mixed.

Quote
Also, could you make something to help bots 'attach' to non-bot things more. I mean water droplets or shapes would be nice to be interacted with, I've always wanted to use shapes or such to make an amoeba with its little dwelling.

Shapes will be made up of panels, too.  By which I mean the surface of shapes will also be composed of equilateral line segments.  Panels can get "sticky" and attach to other panels.  So in that way bots can connect to each other or to their surroundings.

Shapes will also be destructible.  So bots can remove material from a shape and carve out a little hole.  Bot corpses will tend to "clump" over time and form new shapes (partly to keep the environment interesting, partly to make the sim run faster).
Can you make the interiors from the bots leak out so the materials can be eaten easily. Everyone likes some guts spilling out.
Autism can allow so much joy, and at the same time sadness to be seen. Our world is weird, and full of contradiction everywhere, yet somehow at moments seems to come together, and make near perfect sense.

Offline Numsgil

  • Administrator
  • Bot God
  • *****
  • Posts: 7742
    • View Profile
Re: DB3 progress
« Reply #37 on: April 15, 2015, 12:10:22 PM »
Can you make the interiors from the bots leak out so the materials can be eaten easily. Everyone likes some guts spilling out.

Yes, that's on my feature list :)

Offline spike43884

  • Bot Overlord
  • ****
  • Posts: 656
    • View Profile
Re: DB3 progress
« Reply #38 on: April 16, 2015, 08:13:47 AM »
Can you make the interiors from the bots leak out so the materials can be eaten easily. Everyone likes some guts spilling out.

Yes, that's on my feature list :)
It'd be nice if we had the protein patterns on the surface of bots...Plus some sort of overhaul of the virus system...
Autism can allow so much joy, and at the same time sadness to be seen. Our world is weird, and full of contradiction everywhere, yet somehow at moments seems to come together, and make near perfect sense.