Code center > Suggestions

Making the league reproducable

<< < (3/3)

bacillus:
More along the lines of 'can I prove Bot A won if everybody expects Bot B to win?' Anyway, it could always be made optional.
For the formula itself, I don't know the restrictions of the random seed (32bit, 16bit, 32000 etc.?), but if a formula could be linked between some variable that usually is different, like DNA length of each bot or something to that effect, then combined with the contest number, it could be made reproducable.

Numsgil:
The statistical reasoning is a bit fudged, because you're not supposed to vary the number of data points in a statistical analysis until you get some result you want.  You're supposed to take N data points and then do the math.  So the way it works now doesn't necessarily mean that the bot that one would win 95 out of 100 matches.  I'm not even entirely sure it means anything at all, other than that the winning bot is two standard deviations from a tie.  But in practice it's worked very well.  I don't think there's ever been a case where rerunning the league resulted in different results.

So two two of the same species... one will eventually be the winner.  It will take a long time but one will eventually win just through random chance.

If anyone knows a better test than the current one (1/2 rounds + sqrt(rounds) < wins), from a statistical point of view, I'm all ears

Peter:
Pretty simple statistial formula. But it seems to do the trick, very very well.

If I calculate it my bot needed atleast 198,97 wins and it had 199 by the way

Well I have a few suggestions but not directly mensioned at the formula, it works fine. I don't doubt the formula. Ofcource it can be made more strinct, but I think the program will make extra useless rounds then.

Suggestions

Pointing at my first fight. (I still can't really believe one just won in 5-0)
1. A number of minimum rounds. Say like ten or something. Well 6 or 7 would probably be enough to kill most strange results.
1b. A minimal change between ''(1/2 rounds + sqrt(rounds)'' and wins of one round could do this trick too. (you get calculated 4 rounds won when you had won five) So wins is real wins -1.
If one in certainly better it wouldn't cost much more time, I believe.

Second draw.
2. A number of maximum rounds where a draw is called. One of the fights I had as a example had 360 fighs. I don't have a proper number to call for. But to stop useless fighs there is need for a draw. If both bots have won 100 rounds I would believe they're both as good.

Numsgil:
We can extend the minimum number of rounds, probably 7 is a good number.

I dunno if I like the idea of ties.  360 rounds is a lot, sure, but in the end you have a winner and a loser, even if it's just by random fluctuation.

bacillus:
Minimum number of rounds is easy enough, it can be manually changed, but I'm not too sure about maximum number of rounds either. Maybe rather have a threshold of number of rounds:difference in wins ratio after which a draw is called.

Navigation

[0] Message Index

[*] Previous page