Making the league reproducable

[bacillus]

This might be a stupid idea, but what if every match had a random seed equal to the round number? That way, everyone could reproduce the result of a certain match.

[abyaly]

That is an awesome idea.

[Peter]

This might be a stupid idea, but what if every match had a random seed equal to the round number? That way, everyone could reproduce the result of a certain match.

And for what reason. Do you mean for reproducing errors?
Normally everyone should get the same results. Well with 95% sure. 95% sounds pretty statistical so it is probably coded that way.

Do you have a good example that shows your problem?
Like a league fight that can turn both ways.

What do you mean by a random seed equal to the round number?
Random <-> equal. Is what I don't understand.


Setting something like that steady sounds wrong to me. Every fight should be completely different. Not every first fight the same or something like that.

[abyaly]

It has to do with how computers get their random numbers. Bacillus is proposing that DB saves the seed it uses to generate the random number sequence for a particular league match so that someone could later use that seed to see an exact replay of the match.

[Peter]

I couldn't get that from it. But a automated save couldn't be wrong anyway. And then I mean a standard save of the seed-number of any sim. I believe that could get some info for sims in case for errors and no save.

[bacillus]

Yes, I probably expressed my idea clumsily (as I usually do   ). What I meant is to create a sort of formula that allows the seed to be the same evry match. This is mostly for matches where the end result is something like 24:26 or something like that.

[Numsgil]

The program's set up right now for you to seed the random number generator.  So in theory you should be able to do this already, unless F1 mode overrides the seed with a new one.

For actual official league runs, I think the randomness is important.  Otherwise the bot starting positions would always be the same for any given match, so a bot would know that it's round 3 and they're the challenger, for instance.  Which strikes me as bad.

[bacillus]

Which is exactly why I hesitated posting this thread in the first place. The generation would have to be random enough to prevent bots from knowing what will happen, yet reliable enough to be reproducible.

[Peter]

Yes, I probably expressed my idea clumsily (as I usually do   ). What I meant is to create a sort of formula that allows the seed to be the same evry match. This is mostly for matches where the end result is something like 24:26 or something like that.

Well that is becouse they're close to each other. Every match should be different(point)

I agree to the fact that the last fight see number could be remembered. I could maybe help out against league-bugs.
I disagree with a league match being exactly the same every time. I completely digagree if your stand for it is shorter fighting times, it sounds like that.
If you could post a fight between some bots that get results that can go eather way. Bot A or bot B wins in the end. I could get what your point is. Altrough I doubt that happens.
What does happen if you put two exact same bots in a league, forwarding on that?
I didn't test it, but will there be called a draw or will there win one in the complete end?

When is something called a draw? By what number of fights. Or isn't there one. Or is there some special formula.

[Peter]

So how do the formulas work.

I just had two fights run.
With two versions of multiply, fights are rather fast so good for testing.
First one : 0-5 Multiply (the second) wins without doubt.
Second one :199-161 I gues the first multiply wins this one.

For my personal feeling there is something not completely right. They're not supposed to win.
Or I was just lucky.

So what is the accepted error anyway. I thought 95% but with 5 fights you get 50 :25 :12,5 :6,25 :3,125: , so a 3,125% percent change one bot wins if the both bots are equal. So a 6,25% percent change eather one of the bots wins if they are the same.
My reasoning could be wrong, I'm just still suprised that both times a winner was ellected.  When I just ran two of the same bots, both times one of them won a fight.

Formula, code snippet. Any thoughts.

[bacillus]

More along the lines of 'can I prove Bot A won if everybody expects Bot B to win?' Anyway, it could always be made optional.
For the formula itself, I don't know the restrictions of the random seed (32bit, 16bit, 32000 etc.?), but if a formula could be linked between some variable that usually is different, like DNA length of each bot or something to that effect, then combined with the contest number, it could be made reproducable.

[Numsgil]

The statistical reasoning is a bit fudged, because you're not supposed to vary the number of data points in a statistical analysis until you get some result you want.  You're supposed to take N data points and then do the math.  So the way it works now doesn't necessarily mean that the bot that one would win 95 out of 100 matches.  I'm not even entirely sure it means anything at all, other than that the winning bot is two standard deviations from a tie.  But in practice it's worked very well.  I don't think there's ever been a case where rerunning the league resulted in different results.

So two two of the same species... one will eventually be the winner.  It will take a long time but one will eventually win just through random chance.

If anyone knows a better test than the current one (1/2 rounds + sqrt(rounds) < wins), from a statistical point of view, I'm all ears

[Peter]

Pretty simple statistial formula. But it seems to do the trick, very very well.

If I calculate it my bot needed atleast 198,97 wins and it had 199 by the way

Well I have a few suggestions but not directly mensioned at the formula, it works fine. I don't doubt the formula. Ofcource it can be made more strinct, but I think the program will make extra useless rounds then.

Suggestions

Pointing at my first fight. (I still can't really believe one just won in 5-0)
1. A number of minimum rounds. Say like ten or something. Well 6 or 7 would probably be enough to kill most strange results.
1b. A minimal change between ''(1/2 rounds + sqrt(rounds)'' and wins of one round could do this trick too. (you get calculated 4 rounds won when you had won five) So wins is real wins -1.
If one in certainly better it wouldn't cost much more time, I believe.

Second draw.
2. A number of maximum rounds where a draw is called. One of the fights I had as a example had 360 fighs. I don't have a proper number to call for. But to stop useless fighs there is need for a draw. If both bots have won 100 rounds I would believe they're both as good.

[Numsgil]

We can extend the minimum number of rounds, probably 7 is a good number.

I dunno if I like the idea of ties.  360 rounds is a lot, sure, but in the end you have a winner and a loser, even if it's just by random fluctuation.

[bacillus]

Minimum number of rounds is easy enough, it can be manually changed, but I'm not too sure about maximum number of rounds either. Maybe rather have a threshold of number of rounds:difference in wins ratio after which a draw is called.