Code center > Suggestions

League stats tests

<< < (3/5) > >>

Griz:

--- Quote from: EricL ---I'm going to stay out of the statistics debate, having built few combat bots myself, other to say that I personally have no issues with the fact that a bot may have to employ multiple strategies to defeat multiple bots ahead of it on the ladder - strategies which the higher ranked bots perhaps never required given their earlier genesis.
--- End quote ---


--- Quote ---One thing I have noticed however that can greatly impact the results of a contest are inequities in the random layout of veggies and contestants.  Since there are so few starting bots, thier starting positions relative to each other and to the veggies in the sim can have a huge impact on their ability to utilize veggy energy to gain nrg or numbers and thus how well they perform in the contest.  One bot may actually be demonstably better than the other, but the margin may be slim enough that a bad start will inevitably cost it the round.  It is not statisticly improbable to get 5 bad starts in a row, which can obviously lead to bogus rankings.
--- End quote ---
yes. I have noticed this as well ... part of my concern about actually having
better control over veggie re-population, esp max pop.
some bots go into sort of a holding pattern and don't move about much ...
and if the repopulation/reproduction of veggies doesn't happen to occur
near them, they eventually starve.
I forget which bots ... but whether or not a veggie appeared near them or not ...
was the deciding factor in whether they died out or went on to win.

--- Quote ---I might suggest we hard code certain restrictions such as starting contestants in very specific locations on opposite sides of the field and exactly placing the initial veggies equa-distant from the competitors, therby giving neither contestant a positional advantage at the beginning of a round.
--- End quote ---
yes. that's something to think about ...
otherwise the 'random' element can easily eclipse our best efforts at
making the rounds statistically valid.
that random' thing can be a very large elephant loose in the room.
I wondered before if we shouldn't also use a given 'seed' when running leagues ...
as an effort to get as much repeatability into it as possible.

btw Eric ...

I've been noticing large numbers of certain bots and veggies that are tied
together ... suddenly 'leaping' in mass from one place to another ...
a big gob of them say in the lower half of the screen ...
suddenly being transported, as a whole organism, to the top ...
or similarly from right to left. sometimes they go back and forth
a number of times.
it is as if one of the tied bots strays below the bottom of the screen ...
and then brings the whole gob with it when it is relocated [wrapped]
to the top, rather than just that bot being wrapped. must have something
to do with ties.

Numsgil:

--- Quote from: Griz ---yes. I have noticed this as well ... part of my concern about actually having
better control over veggie re-population, esp max pop.
some bots go into sort of a holding pattern and don't move about much ...
and if the repopulation/reproduction of veggies doesn't happen to occur
near them, they eventually starve.
I forget which bots ... but whether or not a veggie appeared near them or not ...
was the deciding factor in whether they died out or went on to win.
--- End quote ---

I'm not so keen to immediately dismiss this element of randomness.  It's true that simpler bots tend to starve to death at times, but more advanced bots usually have an active search system in place, that either makes the bots move around looking for food, or reproduces and spreads out as many bots as possible to find some veggies.  Why should we coddle bots that can hardly survive?

You should only see bots have an unfair selection of veggies near them in all 5 rounds 1 in every 32 matches, which is about 3% of the time.  I think that's reasonable, but if we want to I would extend the minimal number of rounds instead of setting things up nice and easy for the bots.


--- Quote ---I've been noticing large numbers of certain bots and veggies that are tied
together ... suddenly 'leaping' in mass from one place to another ...
a big gob of them say in the lower half of the screen ...
suddenly being transported, as a whole organism, to the top ...
or similarly from right to left. sometimes they go back and forth
a number of times.
it is as if one of the tied bots strays below the bottom of the screen ...
and then brings the whole gob with it when it is relocated [wrapped]
to the top, rather than just that bot being wrapped. must have something
to do with ties.
--- End quote ---

This is a "feature".  Basically, the "wrapping" from left to right or up to down is really just a teleporter.  And to prevent problems from having multibots straddle these borders, the entire multibot is teleported.

It wouldn't take that much effort to fix to be done better, it's just never been done.

Jez:
I'm also not keen to change the randomness of bots start positions. As PY said when the idea was suggested before; "I'll just design bots that don't have such a random 'starting' search then"
If you think it is a problem it might be better to look for other solutions, such as increasing the number of veg while giving them less energy, something that has already been done once, successfully, to help some of the bots that had problems with the relative size difference of the sim now that bots change size.

EricL:
It's not really a big issue for me, but just to make my position clear, I have no objection to the starting positions of bots or veggies being random and different each round.  My objection is with them being inequitable.  I will make the unsubstantiated claim that in closely matched contests, starting position inequities can dwarf combatant competence in determining contest winners, dramtacily increasing the number of rounds needed to determine a statistical winner.  Even a few cycles advantage in reaching a veggy or a positional advantage that provides for an initial 2 on 1 enguagement will influence the outcome given how few starting bots we are talking about.  He who strikes first may not need to strike again.  My main point is that were we to remove these inequities, determining a statisticly valid winner would requre fewer rounds in general and contest results would likly be more repeatable.

It would not be too difficult to modify the current league code to position combatants and veggies in such a way that positional advantages were eliminated yet insured no contest had the same starting conditions.  Failing this, increasing the starting number of combatants would help reduce (but not illiminate) inequities since a single bot would represent a smaller percentage of your total force.

Light:
Would eliminating the bots ability to repro reduce inequalities? would make the fights based more on bot strength than numerical superiority, would be interesting to see the results from a league based on this idea, I like Ericl's main point it sounds a good solution to the problem if it can be implemented without too much trouble

Navigation

[0] Message Index

[#] Next page

[*] Previous page