Author Topic: DB3 progress and networking  (Read 2239 times)

Offline Numsgil

  • Administrator
  • Bot God
  • *****
  • Posts: 7709
    • View Profile
Re: DB3 progress
« Reply #15 on: March 30, 2015, 01:25:04 PM »
The peer-to-peer part gave me an idea, it mentions how its primarily now only used for RTS games (or turnbased games) Darwinbots, is a turnbased program. Each cycle representing one turn. The main problem they state is desyncronisation but thats mainly stated because they're sharing the same view, as in darwinbots we use different views, connected via a teleporter. Of course in darwinbots the cycles in the currrent IM rarely match, so we don't have to match speeds, we just need to hold whatever is being sent in limbo until it can be transported to the other end. The limbo also allows it to switch ports whichever its going through, though if it includes a limbo that'd start to transfer it more into a client-server topology (though it'd still function relatively close to peer-to-peer).

If we just want to use teleporters like in DB2, a peer-to-peer architecture is probably overkill.  It would only make sense to me if we were trying to do something like patch the two simulations together (like, if bots go off the left of my screen they come in the right of yours), which is a neat idea but beyond what I think I know how to do easily, because once bots start forming larger multibot structures we'd have to either teleport the entire thing once it reached the edge of the screen or try to simulate it on two computers at once, and neither is very tractable.  If we're just sending bot files to each other we might as well make things easy and push to a central server.

Quote
This benefically means that completely slow computers, won't damage the sim in any way, as they're signal is sent out, then when its received its directly fired into the simulation, and both sims have no need to be in-sync as its one-way and they're different ''area's''

If the sims on the slower computers are outoging portals only, wouldn't it make more sense to just skip the simulation all together and just allow users to submit bots directly via something like a webpage?  But then you're not really talking about a distributed "internet mode" type thing anymore.  You're talking about a single computer running a simulation that others can add bots to.

Offline Peter

  • Bot God
  • *****
  • Posts: 1177
    • View Profile
Re: DB3 progress
« Reply #16 on: March 30, 2015, 05:24:45 PM »
Without some knowledge of the other parts of the system. I'm not sure if you can get do much more than simple bot transmission.

It would be nice to have one universal system.
When one bots moves to another; how to integrate multibots between 2 sims, different sim speeds, different physics.
How feasible would it be to have a computer calculating a bot in a different simulation?
Extending the former, maybe not any simulation at all on a computer itself and just give computing power to a server, which gives a stream of the simulation back, or hosts a stream(svg animation?). Is that possible without overhauling the whole system?

A P2P system using DHT, through grid computing effectively creating a supercomputer which can't be turned off breeding super AI's sounds fun! Now we only need someone who has the knowhow to implement it properly.
Oh my god, who the hell cares.

Offline spike43884

  • Bot Overlord
  • ****
  • Posts: 656
    • View Profile
    • OurIllogicalWorld Blog
Re: DB3 progress
« Reply #17 on: March 31, 2015, 12:13:53 PM »
Without some knowledge of the other parts of the system. I'm not sure if you can get do much more than simple bot transmission.

It would be nice to have one universal system.
When one bots moves to another; how to integrate multibots between 2 sims, different sim speeds, different physics.
How feasible would it be to have a computer calculating a bot in a different simulation?
Extending the former, maybe not any simulation at all on a computer itself and just give computing power to a server, which gives a stream of the simulation back, or hosts a stream(svg animation?). Is that possible without overhauling the whole system?

A P2P system using DHT, through grid computing effectively creating a supercomputer which can't be turned off breeding super AI's sounds fun! Now we only need someone who has the knowhow to implement it properly.
With a central hub computer which we can access to the visualise everything...And of course things can monitor your CPU so it could integrate to not over-use slower computers...
Anyway, Without knowledge of other parts of the system, I could still get a simple bot transmission. The bot is a packet, packet transmissions are easy.

Actually, we probably should start treating bots as packets more, rather than a complex object...It'd be easier in theorywork. I mean, that'd solve the MB assembly problem, as you'd just recompile it like the packets in a message?
I think we certainly need to find what will be the central hub first. Something that doesn't go off (just incase) and can carry reasonable amounts of data (as it needs to cope with transmissions). Benefit of this system were devising, each computer could select whichever port it wishes to connect via? Then the hub just needs to have all ports open :)
Autism can allow so much joy, and at the same time sadness to be seen. Our world is weird, and full of contradiction everywhere, yet somehow at moments seems to come together, and make near perfect sense.

Offline Peter

  • Bot God
  • *****
  • Posts: 1177
    • View Profile
Re: DB3 progress
« Reply #18 on: March 31, 2015, 12:27:57 PM »
(For the record, I doubt the core engine for Darwinbots will be deterministic just because I haven't put any effort towards that.)
Why wouldn't it be deterministic?
Oh my god, who the hell cares.

Offline Peter

  • Bot God
  • *****
  • Posts: 1177
    • View Profile
Re: DB3 progress
« Reply #19 on: March 31, 2015, 12:40:28 PM »
Actually, we probably should start treating bots as packets more, rather than a complex object...It'd be easier in theorywork. I mean, that'd solve the MB assembly problem, as you'd just recompile it like the packets in a message?
Nope, the issue is that the MB is in two simulations at one. Transmitting a MB has been possible in DB2.

I think we certainly need to find what will be the central hub first. Something that doesn't go off (just incase) and can carry reasonable amounts of data (as it needs to cope with transmissions). Benefit of this system were devising, each computer could select whichever port it wishes to connect via? Then the hub just needs to have all ports open :)
Most server software got only one port open, why would DB need all ports?
« Last Edit: March 31, 2015, 12:42:36 PM by Peter »
Oh my god, who the hell cares.

Offline Numsgil

  • Administrator
  • Bot God
  • *****
  • Posts: 7709
    • View Profile
Re: DB3 progress
« Reply #20 on: March 31, 2015, 01:59:40 PM »
(For the record, I doubt the core engine for Darwinbots will be deterministic just because I haven't put any effort towards that.)
Why wouldn't it be deterministic?

Determinism is one of those hard things that doesn't just happen.  There's no such thing as mostly deterministic.  If there's even one thing that's not bit-for-bit identical across two different machines, the whole thing is spoiled.  I'm not doing anything that shouldn't be deterministic in principle, I'm just not putting any effort in to it, which means almost certainly there's going to be something that is going to differ across machine architectures (maybe there's an acos call somewhere.  Those trig functions are notorious).  Not to say it couldn't be deterministic, but it's not a priority.

Offline Peter

  • Bot God
  • *****
  • Posts: 1177
    • View Profile
Re: DB3 progress
« Reply #21 on: March 31, 2015, 03:14:24 PM »
Gotcha, I forgot about the floating points of hell. :blush:
Oh my god, who the hell cares.

Offline spike43884

  • Bot Overlord
  • ****
  • Posts: 656
    • View Profile
    • OurIllogicalWorld Blog
Re: DB3 progress
« Reply #22 on: April 01, 2015, 11:35:43 AM »
Hence why a central machine running one sim with visual accessible by the users, with then bots input via a one way teleporter would be good.
All ports being open (or most) is simply to relieve "port strain"
Its rare that port strain happens, but its best to avoid it. Its basically a port being overloaded, plus it increases compatibility (Everyone loves compatibility <3)
The floating points of hell wont be a problem with the central machine running it.
Autism can allow so much joy, and at the same time sadness to be seen. Our world is weird, and full of contradiction everywhere, yet somehow at moments seems to come together, and make near perfect sense.

Offline Numsgil

  • Administrator
  • Bot God
  • *****
  • Posts: 7709
    • View Profile
Re: DB3 progress
« Reply #23 on: April 01, 2015, 12:24:43 PM »
Hence why a central machine running one sim with visual accessible by the users, with then bots input via a one way teleporter would be good.

Hmm, you know I wonder if you have some misconceptions that are driving how you see the program.  Do you think DB2 runs real time (30FPS+) for everyone but you, and you're stuck with .1 cyc/sec sims for some reason?  Because that's not true.  I was running simulations circa 2005 with a P4.  My phone has at least 4 times the horsepower I did back then.  The simulation just doesn't scale well.  That means as you double the size of the sim you need something like 4 times more resources.  Or worse, since it's not multithreaded and you can't really find single threaded performance gains anymore in hardware.

DB3 will be multithreaded, but what that gives I'm going to suck up with an even more aggressive simulation.  You could run it on some $10k "supercomputer" and a simulation might only run twice as fast as a dinky $500 machine.  If we want the sorts of bot-cycles that make a good ALife sim we're going to need to think wide (clusters) instead of tall (servers).

Quote
All ports being open (or most) is simply to relieve "port strain"
Its rare that port strain happens, but its best to avoid it. Its basically a port being overloaded, plus it increases compatibility (Everyone loves compatibility <3)
The floating points of hell wont be a problem with the central machine running it.

Hmm, I'm calling BS on this.  That's not really a thing.  Linux can handle tens of thousands of connections from a single source on a single port.  Most webservers for popular websites have something like a "routing" server that accepts all the incoming http connections on port 80 for the entire world (or at least a region) and forwards them on to actual servers to send pages.

I don't understand why ports are such a hobby horse for you.  Is there something I'm missing?

Offline spike43884

  • Bot Overlord
  • ****
  • Posts: 656
    • View Profile
    • OurIllogicalWorld Blog
Re: DB3 progress
« Reply #24 on: April 02, 2015, 01:32:33 PM »
Hence why a central machine running one sim with visual accessible by the users, with then bots input via a one way teleporter would be good.

Hmm, you know I wonder if you have some misconceptions that are driving how you see the program.  Do you think DB2 runs real time (30FPS+) for everyone but you, and you're stuck with .1 cyc/sec sims for some reason?  Because that's not true.  I was running simulations circa 2005 with a P4.  My phone has at least 4 times the horsepower I did back then.  The simulation just doesn't scale well.  That means as you double the size of the sim you need something like 4 times more resources.  Or worse, since it's not multithreaded and you can't really find single threaded performance gains anymore in hardware.

DB3 will be multithreaded, but what that gives I'm going to suck up with an even more aggressive simulation.  You could run it on some $10k "supercomputer" and a simulation might only run twice as fast as a dinky $500 machine.  If we want the sorts of bot-cycles that make a good ALife sim we're going to need to think wide (clusters) instead of tall (servers).
I don't have a misconception about this such things like the change in cycles per second. The thing is, we need a way around "floating points of hell" and the differentiating between the computers. In real life when a cell moves an inch, time doesn't slow down 10 times for it, then another inch speed up 5 times. What about thinking wide and tall. Running the simulation on the central machine giving visual feedback, but using the participating terminals (or personal computers) to calculate the bots. Remember, the simulation is split into 3 segments primarily, the area of the simulation, the bots then finally the projectiles etc. If you spread the load of the bots across the computer terminals then it can calculate the DNA triggers, then thats returned to the server which then treats the simulations (and maybe projectiles) then once a projectile has hit a bot it lets the computer terminals handle it as a 'bot event'. Thus because the actual simulation is on one centralised machine it doesn't suffer from differentiation, but the load is spread by the terminals to calculate the bots. Tall and wide. Then a simple little visual output is given to every terminal.
Autism can allow so much joy, and at the same time sadness to be seen. Our world is weird, and full of contradiction everywhere, yet somehow at moments seems to come together, and make near perfect sense.

Offline Numsgil

  • Administrator
  • Bot God
  • *****
  • Posts: 7709
    • View Profile
Re: DB3 progress
« Reply #25 on: April 02, 2015, 11:10:48 PM »
Problem is that maybe 70% of the simulation cost is doing physics and vision and the like, 20% is the DNA execution, and 10% is random stuff.  Even if you offloaded everything that wasn't physics to other machines, you're only maybe going to make things 40% faster.  Not exactly compelling.

Even if you offloaded physics too, there's something like a 100ms latency between computers on the internet (more like 20-200ms, but on that order).  Even if there was a computer that could instantly give you the answer, if you have to wait 200 ms (for the round trip) for it, best case the simulation runs at 5 cycles/sec.  Again, not exactly compelling.

Finally, you have to consider the bandwidth of the server.  Let's say conservatively that a simulation has about 100K of state that you have to send and receive every cycle.  At a guess of $10/Mbps, that works out to a cost of about $10/month just for the bandwidth.  Which isn't a lot, but that's not really scaling the way I want either.

...

On point, having different parts of the "universe" for a bot run at different time rates isn't really a problem as I see it.  In fact, that's how are universe works anyway.  Even ignoring things like relativity, life in the tropics is more rapid than life in the arctic.  The more energy in an ecosystem the faster life there tends to evolve (Evolution and latinitudinal diversity).  In a virtual landscape, faster simulations become valuable real estate, which adds texture.

I would say the biggest problem (or at least, area that could do with some thought) with the current method for IM in DB2 is the teleporters.  Not to say that the way they work now is bad, but the exact mechanism of transfer has a huge effect on what sort of simulation IM would be. 

For instance, it makes me think of Eve Online's wormhole space.  Basically there's a part of Eve Online where different star systems are connected with non-stable wormholes (so two different systems might be connected one hour and not connected the next).  The different star systems are tiered, with difficult but rewarding systems deeper in, and easier systems closer to "normal" space.  Generally the wormholes are two way, so if A connects to B, B connects to A, and the entrances and exits are next to each other in either system. 

A few months back they made a fairly innocuous change to how entering a system through a wormhole works which made heavier ships spawn further from the entrance back to their original system.  Which made it harder for people to scout out systems for "content" (ie: other people to fight) because once a heavier ship had jumped in they were pretty much committed to fighting ("slow boating" it back to the wormhole taking so long that you'd be dead before you got there if things went sour).  The small change made a fairly large change to the competitive landscape: it changed the playing field to encourage fewer, larger corporations, since they were the ones that could afford to lose expensive ships.

We might want to consider what sort of competitive landscape we want DB3 to look like, and then decide on a serving architecture that supports that goal.  For instance, do we want a single dominant species across all machines? (ala Pando?).  A fairly well connected universe with similar settings on machines would probably look like that.  I think that's what the current DB2 model encourages.  Or do we want regions of computers rules by the same species, but different species on the whole (ala Eve Online nullspace)?  Sort of a chunky ownership map basically.  Machines connected so that there are a few choke points here and there would encourage that, I think.  Or do we want each machine to almost be an island with different species holding different machines?  Maybe something like the wormhole connectivity from Eve online would work for that.

Then there's issues of what can teleport out and what can teleport in and where.  If large multibot structures can be teleported in anywhere on the incoming sim, a species would be well advised to only allow its large multibots to teleport in to sims it's sure it can dominate.  If there's a single place to teleport in, then species can camp the entrance to their local sim to prevent incursion.  Or if only single bots can teleport, it might be more difficult for a species to invade another's machine, because they can't bring as much to the fight.

Offline spike43884

  • Bot Overlord
  • ****
  • Posts: 656
    • View Profile
    • OurIllogicalWorld Blog
Re: DB3 progress
« Reply #26 on: April 03, 2015, 08:54:08 AM »
Weve also got to decide how we want life to progress in it. I mean your making a point a lot of these systems might run at 5 cycles/sec but if its dedicated then it can run for a long time slowly.
Remember, not everyone runs at 100 cycles/sec or wants to have a complete mad evolution in 1 and a half minutes. Plus, your point of tropics having faster evolution, they still don't have faster time. The mutation rates check how fast our evolution goes...Not the speed of time.

Second, heres a possible solution to the multibot transportation problem. Make the simulations 'overlap'
The simulations have a 1st border, then 2nd border. The 1st border gives you visual output, its the edge of the overlap really...Its a region around the edge, then just after that the 2nd border. The 2nd border is the other ends 1st border, its more transfering infomation of what is going on there to the 1st border on your simulation. Thus you might not nessisarily transport the entire multibot at once, but it'll appear seamless still.

Finally, Compatibility is still something you need to prioritise, And latency you need to think of along with floating points. Irregularities in area's are fine, except for the most extreme ones. Plus everyone wants to equally enjoy IM, and if someone has a quantum computer running in their house, and end up creating a super-monsterous-cancerous-bot it might run fine on their computer, but on a computer that struggles it would obliterate. Now, I can understand your points that your coming from, but compatibility and everyones equal enjoyment are key factors to consider, which in DB2 weren't nessisarily as prioritised and should be one of the highest things on the IM agenda.
Autism can allow so much joy, and at the same time sadness to be seen. Our world is weird, and full of contradiction everywhere, yet somehow at moments seems to come together, and make near perfect sense.

Offline Peter

  • Bot God
  • *****
  • Posts: 1177
    • View Profile
Re: DB3 progress
« Reply #27 on: April 03, 2015, 03:24:39 PM »
We might want to consider what sort of competitive landscape we want DB3 to look like, and then decide on a serving architecture that supports that goal.  For instance, do we want a single dominant species across all machines? (ala Pando?).  A fairly well connected universe with similar settings on machines would probably look like that.  I think that's what the current DB2 model encourages.  Or do we want regions of computers rules by the same species, but different species on the whole (ala Eve Online nullspace)?  Sort of a chunky ownership map basically.  Machines connected so that there are a few choke points here and there would encourage that, I think.  Or do we want each machine to almost be an island with different species holding different machines?  Maybe something like the wormhole connectivity from Eve online would work for that.
What would the wormhole connectivity do? Make it so only a minimal amount of bots arrive, weaken the bots before they can attack the native bots?

Added a image for DB2 and 'choke' points.
Assuming that DB2 random sends from any sim to any sim to any location. And choke points to sims 'nearby' at stuck locations.
« Last Edit: April 03, 2015, 03:27:03 PM by Peter »
Oh my god, who the hell cares.

Offline Numsgil

  • Administrator
  • Bot God
  • *****
  • Posts: 7709
    • View Profile
Re: DB3 progress
« Reply #28 on: April 03, 2015, 04:16:20 PM »
What would the wormhole connectivity do? Make it so only a minimal amount of bots arrive, weaken the bots before they can attack the native bots?

Well in Eve the wormholes collapse after a certain amount of mass goes through them.  So in that scenario you'd sever the outbound connection after some total number of bots, or their total mass, exceeds some threshold.  Not sure if that's a good idea or not, but something to consider and think through the implications.

Offline spike43884

  • Bot Overlord
  • ****
  • Posts: 656
    • View Profile
    • OurIllogicalWorld Blog
Re: DB3 progress and networking
« Reply #29 on: April 04, 2015, 07:49:19 AM »
If you used the overlaps instead of wormholes, shapes could drift as well, because its not a teleporter and more a solid connection...the shapes could move across, or if arranged solidly in  a simulation form a chokepoint, which would be more realistic as in real life objects get in the way ;3
Autism can allow so much joy, and at the same time sadness to be seen. Our world is weird, and full of contradiction everywhere, yet somehow at moments seems to come together, and make near perfect sense.