Code center > Bugs and fixes
League Problems
Jez:
--- Quote from: Griz ---this may have been a valid ranking at one time ...
from the days of 2.37???? or did someone rerun leagues
since 2.4X? ...
--- End quote ---
To the best of my knowledge the (F1) league has never had a rerun, even when PY programmed the league function into DB the existing list on the forum was used as the starter list for the league. (which wasn't rerun, bots have just been added to it since.) There may have been a rerun done, for some if not all of the bots, when the statistical analysis of rounds was introduced. ( I pressed for and researched the formula because one of my bots didn't gain the league position I knew it should get.)
It might be better to consider it as wrong of me to rerun the league at all, especially without reentering all the existing bots in the bestiary in date order...
--- Quote ---so you do a trade off ...
trading speed for accuracy ...
it's a balancing act ...
which I don't have a problem with, you do what you have to ...
but lets not pretend it is then as correct as it could be ...
or can justify using all that time for statistical calculation
re # of rounds required to win ...
because the error introduced in the initial placement ...
is far greater than that, and will nullify all those great
efforts at being precise.
--- End quote ---
Indeed, you are calling into question, for me, if rerunning the league will result in more accuracy than we have already, perhaps better to leave it as it is and wait for a new method of running the leagues, allowing for re-entry of all the bots and better analysis of the results.
The league as it stands is, to use your analogy, 4+ years of stacking dominoes. Is it right to change that because some dominoes are looking old?
--- Quote ---and I am saying they, and your premise, are not valid.
so I guess we can agree we disagree.
let me know what they are up to in another week. lol
--- End quote ---
Funnier than you think but I'll tell you about that in a min...
--- Quote ---see ... here we are at the place I don't get ...
it is a statistical draw!!!!
why keep screwing around with it?
why not give it to Animal S ... he's got more rounds ...
and move on.
if you are concerned about the time to run leagues ...
I suggest you take a look at what is consuming much
of that time ... and determine if it is worth it or not.
I don't happen to think it is ... but ...
--- End quote ---
It's the VB debugger that's taking the time, consider that's making it 10x slower than it would be otherwise, it's only taken ~20 hrs to get this far.
--- Quote ---hey ...
made a mistake before for # of matches required
for all bots to go up against each other ...
had a + there instead of a -
it isn't n(n+1)/2 but n(n-1)/2
3 bots = 3 matches: ie A-B, A-C, B-C
--- End quote ---
Reminds me of the 'gypsie rose' problem (same thing really) they gave the maths class once at school, took me 90 seconds to figure it out and two weeks to fail to make the answer into a formula
Anyway; at 258 rounds, 114 for Animal S - 143 for Spanish C the bots disappeared and I'm left with a bunch of veg firing viruses at each other!
Numsgil:
--- Quote from: Griz ---so you are going to run into the very problem I've been
attempting to draw attention to ...
I've already run into them ... and showed you one example
being that of D Scarab 3 being prevented from moving up.
[which would have happened to him in a challenge as well, btw]
--- End quote ---
This is the difference in our opinions. You view this is a problem, I do not. It is mostly impossible to rank bots in a linear fashion anymore, because the program is getting to a rock-paper-scissors strategy. It's okay if stronger bots get stuck in the bottom of the ladder. Thems the breaks.
--- Quote ---...which I don't have a problem with, you do what you have to ...
but lets not pretend it is then as correct as it could be ...
or can justify using all that time for statistical calculation
re # of rounds required to win ...
because the error introduced in the initial placement ...
is far greater than that, and will nullify all those great
efforts at being precise.
--- End quote ---
This is a bias, not an error. Run the same initial ordering of bots 10 times. If there's any deviation in the placings, that's error. There should be very little error. If you run 10 different initial orderings and get 10 different results, that's bias.
Bias is inherant in all ladders. But that's okay, because we have a fair way of ordering the initial robots. It's okay if a strong bot is stuck at position #25. Bots at the top of the list should not only be strong, but capable of defefating some of the tricky bots (I believe there was some Umbra bot that stops alot of bots from proceeding in the leagues).
I would use either bots' ages or present ranking in the league to setup the initial ranking. Ages would be best, since most older bots rank low in the league.
..........................
If you want a fair way of running the leagues, what about something like this:
take some bots, randomly order them into an initial order. Run the league. Reverse the new order of the bots (so 1st place becomes last place, etc.) and rerun the league. Keep doing this until the rankings stop fluctuating every time you run the leagues. I'm guessing they'll never stop totally fluctuating, but you'll definately have the case that bots at the top of the league are strong, which is what you want.
--- Quote ---see ... here we are at the place I don't get ...
it is a statistical draw!!!!
why keep screwing around with it?
why not give it to Animal S ... he's got more rounds ...
and move on.
if you are concerned about the time to run leagues ...
I suggest you take a look at what is consuming much
of that time ... and determine if it is worth it or not.
I don't happen to think it is ... but ...
whatever.
--- End quote ---
There's no such thing as a statistical draw when you can run an arbitrary number of rounds. Supposing for a moment that you managed to find 2 bots that are identically matched (exactly 50/50), there isn't a test for that. So maybe we should add one. But I warn the number of matches you need to run to determine a true statistical draw is probably in the thousands.
Try this simple experiment. Hack into the league code, and set up a league match with a "fair" coin (assign a random winner based on a 50/50 probability). Run the league. It should be an enlightening experience either way.
I imagine a league winner will eventually be declared, which, come to think of it, isn't good. We should add a catch for when the results are indicative of a true statistical draw. But again, we're talking possibly thousands of rounds.
Just declaring a winner after 200 rounds based on who has the most isn't proper. Imagine flipping a coin 200 times. It's not going to end up 100/100. Would you declare the coin unfair if it was 130/70? That's where stats comes in. Stats assures us that we have control over arbitrarily picking winners.
Light:
--- Quote ---Supposing for a moment that you managed to find 2 bots that are identically matched (exactly 50/50), there isn't a test for that.
--- End quote ---
Entering the same bot under 2 different names would give you an exact 50/50 scenario your after.
Griz:
--- Quote from: Numsgil ---n(n+1) / 2 approaches n^2 as n gets very large (actually, it approaches 1/2 n^2, but big O ignores those coefficients), which is the point I was making. You're going to be running alot of rounds, no matter how you slice it. Suppose you run 1000 bots in a round robin tournament. That's going to be something on the order of half a million matches. That's a lot of matches.
--- End quote ---
we aren't talking 1000 bots ... but 30 ... 435 matches.
--- Quote ---It's okay that initial order matters in the ranking, because we have a prechosen method for the preranking: age. Initial ranking is always going to matter, there's no getting around that in a ladder. But using seniority for the initial ranking makes the most sense, for reasons Jez outlined above. In your own personal ladder, feel free to chose whatever initial ordering you like. Initial ordering does matter, but so long as your choice is arbitrary most of the league is going to be ordered correctly.
--- End quote ---
I'm sorry Nums ... is is not.
you are missing what I am talking about ...
as you already think you know what I'm saying, and you don't ...
so there's no room left there for you to take a look at what
I'm pointing at.
you're not only not on the same page ...
but not even in the same book.
ok. I'm tired of beating my head against the wall.
forgettabout it then.
--- Quote ---The idea isn't that the ladder is a perfect representation of the strength of the bots. The ladder is simply a quick way to relatively sort them. That's why they're used in sports, people aren't patient.
I'll post in suggestions forum with my stats findings.
--- End quote ---
Griz:
--- Quote from: Numsgil ---This is the difference in our opinions. You view this is a problem, I do not. It is mostly impossible to rank bots in a linear fashion anymore, because the program is getting to a rock-paper-scissors strategy. It's okay if stronger bots get stuck in the bottom of the ladder. Thems the breaks.
This is a bias, not an error. Run the same initial ordering of bots 10 times. If there's any deviation in the placings, that's error. There should be very little error. If you run 10 different initial orderings and get 10 different results, that's bias.
Bias is inherant in all ladders. But that's okay, because we have a fair way of ordering the initial robots. It's okay if a strong bot is stuck at position #25. Bots at the top of the list should not only be strong, but capable of defefating some of the tricky bots (I believe there was some Umbra bot that stops alot of bots from proceeding in the leagues).
I would use either bots' ages or present ranking in the league to setup the initial ranking. Ages would be best, since most older bots rank low in the league.
..........................
If you want a fair way of running the leagues, what about something like this:
take some bots, randomly order them into an initial order. Run the league. Reverse the new order of the bots (so 1st place becomes last place, etc.) and rerun the league. Keep doing this until the rankings stop fluctuating every time you run the leagues. I'm guessing they'll never stop totally fluctuating, but you'll definately have the case that bots at the top of the league are strong, which is what you want.
There's no such thing as a statistical draw when you can run an arbitrary number of rounds. Supposing for a moment that you managed to find 2 bots that are identically matched (exactly 50/50), there isn't a test for that. So maybe we should add one. But I warn the number of matches you need to run to determine a true statistical draw is probably in the thousands.
Try this simple experiment. Hack into the league code, and set up a league match with a "fair" coin (assign a random winner based on a 50/50 probability). Run the league. It should be an enlightening experience either way.
I imagine a league winner will eventually be declared, which, come to think of it, isn't good. We should add a catch for when the results are indicative of a true statistical draw. But again, we're talking possibly thousands of rounds.
Just declaring a winner after 200 rounds based on who has the most isn't proper. Imagine flipping a coin 200 times. It's not going to end up 100/100. Would you declare the coin unfair if it was 130/70? That's where stats comes in. Stats assures us that we have control over arbitrarily picking winners.
--- End quote ---
bullshit.
you're talking theory ...
and I'm talking practical application.
but you already know it all .......... as always so ...
fuck it.
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version