|
As a fairly new member of RGD back in May of 2001, I joined a thread -- no doubt one of many -- discussing how powers are assigned on the judges. My complaint was that preference lists can't really be used as intended, because if I submit this preference list: FEAGRIT I am much less likely to get assigned France than if I submitted the preference list: FEGRTIA This argument is based on the way the judge attempts to "maximize the happiness" of all players in the game in Assigning Powers. (See Tarzan's excellent article on power assignment methods in the F1998M issue of the Pouch). I'm certainly not the first person to make (or rebut) this argument, but it was pointed out by Jody McCullough that the complaint is based on a number of assumptions. Most notably, it assumes there is a bias that against Austria among players, that this bias is strong, and that it is significant enough to affect power assignment in actual judge games. After some discussion, we decided to try and determine whether the perceived anti-Austria bias among players is real or assumed, and to determine what the impact of any such bias is. To attempt to discern the actual level of bias against Austria, I gathered preference lists that had been submitted by players in 251 Standard Diplomacy games and analyzed the power distributions in each list, looking for patterns. In a later article, Jody or I may further take the information gleaned here and investigate how these trends would affect actual power assignment on a judge, and how different power assignment algorithms might be more or less effective at removing any bias (assuming one exists) against Austria. The data that follows is based on reviewing 1505 preference lists from 251 standard Diplomacy games played on DEDO and USCA prior to December 2001. 11 games in which each player submitted a unique, one- power preference list and one game where all players submitted the same "[AEFGIRT]" list have been excluded from this analysis on the assumption that they represented "off judge" agreements about who would play what powers. At the time these games were run, no judges had implemented the "random power assignment" feature (set pref *), so this feature won't have any odd effect on the stats gathered so far. To rank the power's popularity, each power was given a point value corresponding to where it was placed on each preference list. That is, a power placed first on a preference list was given 1 point, and a power listed last a 7. Powers placed at the same preference level (in []'s) are each given the same score, which is the average of what they would have scored had they not been in brackets. As an example:
Powers not listed in a list were treated as being tied for last place on the preference list. Some people may object that trying to measure rankings is inappropriate, since there may be a bigger subjective difference between your first and second choice than there is between your second and third choices – e.g., “I really want France, but failing that, I’d kind of prefer England over Germany”. However, since the judge does assign scores to each power in a preference list for its distribution algorithm, it seems appropriate to use its scoring system to weight the powers in a list. Note that only preferences of powers that actually submitted lists are included in the statistics I gathered. The alternative is to treat empty lists as "[AEFGIRT]" and to give each power 4 points. Obviously in this scoring system a lower number means a more favored country. So looking at all 1505 preference lists, here is the average score each country received based on the scoring outlined above:
Without worrying too much about standard deviations and chi-squared plots, it seems reasonable to assume from this data that the "average" player will submit a preference list that looks like: FE[RT]GIA So it is pretty clear, at least among past players on USCA and DEDO, that there is substantial bias towards France and against Austria. Another way to look at these numbers is to review the number of times a power was placed in a specific position on a preference list. Note that because the scoring system I used allows a power to end up with a fractional placement (1.5 if tied for first), I rounded all scores down to compile this chart. So two powers tied for first would all show up on this list as being in "first position" (1.5 points each), but three or four powers tied for first would all show up as "second" position (2-2.5 points each).
Frequency in percentage (%) of power placement on a position on a preference list.
On this list France's popularity is very clear. France is listed first on over 30% of all preference lists, and is listed in the top two on over 50%. Austria is a virtual mirror image of France's popularity, being listed last on about 30% of lists and in the bottom two on over 50% of all lists. England and Italy are less extreme versions of the same phenomenon, with England being listed in the top three 60% of the time and Italy in the bottom three with the same frequency. Going back to the argument that started this analysis, are these biases strong enough to mean that players willing to play Austria (e.g., that list them in the top 3 or 4 spaces) are disproportionately likely to receive Austria, regardless of what powers (other than Italy) might be listed above them? It would certainly seem so. If 85% of players are listing Austria at the bottom of their preference lists, it is a good bet that putting it above the middle point in your own list is likely to get that power assigned to you. But of course, that analysis is slated for a later article! However, another interesting note is as noted above, the 251 games reviewed had a total of 1505 preference lists. That actually translates to 6 preference lists submitted each game. So while the players submitting preference lists are certainly putting FE at the top and IA near the bottom, you can also expect there is a good chance that at least one player has not submitted any preference at all. (The original data gathered did not take into account game skill levels, so I can’t say whether this is more or less common in newbie games than others). Another way of looking at the numbers, which may or may not be useful, is the comparative preference of each power on each preference lists:
This shows that, for instance, Austria is listed above England on 284 preference lists, whereas England is listed above Austria on 1117 lists. I'm not sure what this data shows directly, but it allows one to infer things like: when France is NOT listed first it's usually because England was listed above it -- or when Austria is not listed last it's usually because Italy was listed below it. There's more number crunching that could be done, but seeing as I've sat on this data since February of 2002 it’s probably time to publish it, huh?
If you wish to e-mail feedback on this article to the author, and clicking
on the mail address above |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
