I read your experiment article and liked it a lot! I found it most interesting that the expert players were most affected by the unit positions. And -- what do you know? -- the most expert players are those who attend World DipCon and other face-to-face conventions, and who probably have the most experience with FTF games. I would imagine that exclusive PBEM'ers (the bulk of respondents, perhaps?), have much less experience with seeing units oriented differently (or even differently orientABLE) on the board.
An interesting point. If you assume that the original premise that orientation would subconsciously affect a person's assessment of the board is correct (something that has by no means been firmly established as being true), the next question would be whether people with FTF experience are more likely to be biased by orientation than others. Had I asked a question to establish the level of FTF experience, I'd be able to compare people with much FTF experience vs. those with little to none to see if there's a difference in the level of bias. Alas, I didn't ask that, so the data is not there.
Hello Simon:
I was intrigued by the thought underlying your deceptive little survey. On a gut level, I would tend to agree with you that one can affect person's perceptions by orienting units on the physical board. I think there are a number of reasons, though, why your online experiment proved inconclusive.
Some of those reasons you covered yourself: a too-small sample size; a potentially imbalanced scenario; and a post hoc realization that the answers didn't give you the data that you really wanted. I would add a couple more issues. First, that you just can't achieve the same effect with an electronic map that you can with the physical board; second, that the "cover story" might have affected results; and, third, that it's extremely difficult to account for and filter out preconceived notions about the game from players.
I agree with all your points. It would certainly be interesting to have the resources (time, subjects) to conduct a better-designed, non-electronic experiment under more controlled conditions.
On the first point, I can remember back to my old, casual FTF days, when players would snipe at the GM as he adjudicated the board. My younger brother, especially, was big on physically "re-centering" pieces in provinces and accusing the GM or other players of trying to manipulate pieces to create precisely the kinds of perceptions you are talking about. Manus' 3D artwork just isn't the same thing as fiddling with the pieces so that they hang (ever so slightly) over borders into the other guy's provinces and letting that perception sink in. The effect was far more subtle and, I'm guessing, less perceptable.
The sub-optimal artwork was of my doing, so Manus is not responsible. But I agree. The other problem for many people is the fact that on small monitors, you don't see the entire board at once without scrolling, and the biasing effect probably loses something when you aren't seeing the big picture laid out at once.
On the second point, I'm one who didn't answer the survey at all. Basically, it was because I didn't agree that this was the "right" way to measure risk-taking behavior. [Elaboration deleted] Since I couldn't bring myself to view the questions in those terms, I passed on replying to them. I wonder how many people thought like I did (more or less), but answered the survey anyway. [Of course, I have no idea which map I saw--heh.]
Yeah, the entire scenario was a bit contrived. Had I had more time to invest, I would have tried to come up with something better. (Better maps, a better fake-risk-taking-experiment and better questions for getting good data for the real experiment.
On the final point, I would remark that, in addition to a certain bias in the position towards the potential for IT conflict, you were also (silently) inviting the respondents to indulge their preconcieved notions about Italy's relations with it's neighbors. If you conducted another survey and the questions was "Italy and Turkey make good allies, in your experience--true or false?", I daresay you would get a flood of "False" answers, especially from players who classify themselves as less than expert. [More elaboration deleted]
Another good point.
I think that someone could do the world a favor by writing a Dip Pouch article that addressed some of the "conventional wisdom" out there regarding who should ally with who and why. It's really quite maddening to, time and again, run accross players who seem to be adopting somebody's "how to" article as their play by play manual for playing a power in a game.
I'll talk to Manus and see if he can get that Cassia guy to write such an article. :-)
That's enough carping for one letter. I did like your anecdote and will repeat that I think you are right about the ability to sneak this kind of perception into FTF play. Your survey was a valiant attempt to define the truth of that, but I think it defined other--more persistent--truths about "perceptions" and this game. Still, I think *that* was a valuable exercise, too.
Thanks for the kind words, and the overall feedback as well.
I wasn't involved in your original survey (nor did I even get around to looking at it until the article), but a possible hypothesis on your "interpert" statistics would be this:
It takes real brass to declare oneself a Diplomacy "expert", and nine times out of ten I would say doing so is a result of overinflated self-opinion rather than actual mastery. It could be said that true masters don't feel the need to broadcast the fact, and/or are always striving to better themselves and recognize that they don't know everything there is to know. [Elaboration deleted]
Your survey was anonymous, but I think the idea holds. You added two new levels, "novediate" and "interpert." Now if a true expert is adhering to Sun Tzu's ideals of humility, he still won't declare himself an expert. [More elaboration deleted]
Why's that so big an issue? The man with ego enough to declare "Expert" is quite possibly self-centered and thus handicaps himself and his nation with not looking at the whole board... to his mind only the immediate surroundings concern him. The Interperts (if they are indeed the true "expert" category as I theorize) are savvy enough to discount surface obfuscations and look at the big picture and the long term goal. They will play as if their opponents are the same way, until and unless the opponents prove themselves irredeemably incompetent.[More elaboration deleted]
Overall, a valid point. Of course, the issue you bring up only makes a difference if the biasing effect I hypothesized varies with different skill levels. Originally, I hadn't expected it to necessarily be so. And given the point you make, even given the differences across skill levels, it's hard to know if the differences are meaningful because it's hard to know whether the skill levels people answered with are truly their real skill levels. I was aware of that difficulty ahead of time, but I had no way to assess true skill levels, so the best I could do was to ask and go with the given answers. But I don't disagree with you at all.
This person did not ask to remain anonymous, but I'd hate to impact his performance in tournaments by publishing his name along with his admission. I will say that it's a name you'd recognize if you read rec.games.diplomacy or the Pouch Zine regularly.
I always make sure I orient the pieces certain ways in tournament play...
Aha! Another believer.
Dear Simon,
As a statistician, I was very interested in your survey analysis.
Oh oh. I'm in trouble.
Yet in my opinion, it has not been done in very protocolar ways. Whatever the way you ask the question or reorganise the data, the main thing to do is test (statistically) for any difference between groups. [Elaboration deleted]
Then, you can also test for association (I wouldn't use correlations with such category data; even the scale responses are not clearly measurable, as you noticed yourself.) You tested correlation between the two latest groupings, but it is their association with Map that would prove the most interesting.
One important aspect of your statistical analysis, which you seem to totally overlook, is the fact that people were asked to enter their own expertise and playstyle levels. As such, these are ALSO perceptions. [More elaboration deleted]
Yes, as I said in one of my above responses, that was the best I could do given the resources I had. But I agree.
It may also be that the particular way in which map 1 and map 2 differ is not the one that yields more insight. I, for one, when looking at your two maps, was not sure which one was intended to be felt as "pointed towards France." Definitely, I am more sensitive to the shape the units form on a larger scale, like you said with the "arrow." Maybe other people are influenced by other things. There is ground for further analysis. I am not saying this is hopeless, on the contrary.
Would it be possible to look on the raw data? Would you want me to run some basic tests?
I'd gladly send you the raw data. If anyone else is interested, just let me know. I'm the first to admit that I only know enough about statistical analysis to hurt myself.
Dear Simon,
I enjoyed your article concerning the unit orientations, especially since I was one of your victims. Your perfect setting up of two test groups gives your experiment a lot of credibility, even in the face of your reluctance to make definitive conclusions based on your results.
It seems pretty obvious from both your method and your email address that you have a good bit of experience in research methodology
Well, my method was far from scientific, and my experiment far from controlled, so don't put too much weight on it. My email address, doesn't imply that I know anything about statistics. I assume I know much more about statistics than the average person, but far less than most people who have a strong background or expertise in the area of statistics.
I am curious if you ran any Chi-squares or t-tests on your data. There is a computer program out that i recently used for one of my grad classes that takes care of all that type of work for you, it is called the SPSS( i think it is an acronym for "statistical program for social sciences" but dont hold me to that), and would take a while to key the info in, but once you have that done, you could run alot of stuff with it, despite not having that many variables(questions asked).
Nope. I am quite aware of the fact that much more could be done with data than what I did, but I haven't done anything more sophisticated than what I described.
Anyways, I just wanted to express my appreciation for your type of article as it gives credence to those of us who must defend to our wives, family and friends that we are doing more than "just playing a game" and are exercising our minds and imaginations for the challenges and obstacles that the world will place before us in the future. Even as a 36 year-old father and husband, I cannot abandon the 'training missions' of my youth that have given me edges in today's business and academic climate. Thanks again, I hope that enough interest is generated for a follow up article...
You've got an understanding wife. When I try to defend Diplomacy to my wife as being more than just a game, she just laughs. It's a real hobby, I say. Look, I help maintain a big web site, I say. Look, I actually write about the game, to gain and share knowledge. She doesn't buy it. :-)
Simon --
I'd like to see the reprocessed data -- but only if you have the time to do it, of course! Very interesting study. Proud to have been a part of it. I live in a studio apartment (space at a premium) and so made a wall-mounted map for myself (velcro is a wonderful thing). When I drew the map, I redistrubted Supply Center DOTS (for instance, my Sevastopol dot is in the center of Sev, instead of right on the Black Sea). I did this because I thought the current placement of the dots themselves would influence me when I was looking at map positions -- after all, a unit in Moscow is just as big a threat to Sev as one in the Black Sea, but it doesn't feel that way because the dot is so much closer to the Black than to Moscow!). Of course, I wasn't smart enough to test my theory! :)
Aha, another believer!
Greetings!
I did just now read your article on the survey you did. And one thing that I had to mention to you is that I do not think the two maps are different enough. To me the 3-D pieces do not look they point in any particular direction. Unless it is towards the onlooker. :-)
Yes, as I said above, an electronic board is not a perfect substitute for a real one, and my artwork was not great. On the other hand, the idea was not to for the orientation of units to be obvious, but to see if it influenced peoples' perceptions even if they didn't specifically take notice of how they pointed.
Simon,
It is articles (and experiments) like the one you ran that keeps me so intrerested in what really boils down to a simple game. The results although inconclusive will keep me thinking for weeks. Thanks.
And thank you for the comments. It's positive feedback like yours that keeps me interested in writing about Diplomacy.
[A bit deleted]
I did, however, learn something in the local Axis and Allies club. You can send subliminal messages with playing pieces. However, in A&A, the teams are set. There is no discussion on allegiances or anything. So you can hint to your opponent that you're attacking in a certain direction, even a certain province (and then you head the opposite direction, or if he's really onto you, you do the obvious). If you can do that with A&A, why can't you hint in Dip who's attacking who?
Okay, fair dues, in Axis and Allies the units actually "point" in a certain direction. If you see a bunch of tanks pointing in one direction, you take a hint. Not the same in Dip, but you can be more subtle in dip.
Enjoy that tidbit if it helps.
Thanks. The idea is the same, I think. Since Diplomacy pieces have one dimension longer than the others, they can still "point."
No really. I'm not angry. I'm thrilled. Good idea.
Good idea, bad execution. Sorry, just had say that. You limit yourself and your capacity. Why aim for a Yes-No answer (whether median or zero). Take the answers and weight the on a scale -4 to 4. Qustion 1 minus Q2 or whatever. More flexible and more sensitive.
Yes, the execution could have been better. As I said above, I'm happy to give the raw data to anyone who is interested in doing more.
BUT I don't think the maps differ that much. I took a long look and only found the French F MAO shifted significantly. In such a test the thesis that's being tested should catch your eye. I don't think that I'd react if someone showed me map1, asked me to turn away, and then showed map 2. I'd say, "well, what?"
People were not necessarily supposed to take notice of the difference. The issue of orientation was supposed to have been subtle enough that it would affect perceptions subconsciously, even if one didn't notice the orientations.
I'm not an expert, but think that many players could have been much more affected by the general surrounding story and general board situation than by unit positioning.
Definitely. I expect that the context of a game (the current situation and the past history leading up to it) is going to affect people much more than something on the level of unit orientation. Obviously somebody who just stabbed you is not going to convince you that he's your friend just because his units aren't pointed at you. But everyone got the same general story and general board situation. Since that was a common factor to both groups (those who saw map 1, and those who saw map 2, any statistically significant difference between the answers from one group to another should not be because of the story or the setup. They should be due to other factors that differ from one group to the next... in this case, the unit orientations.
Hi Simon,
Congratulations on successfully misleading me with your survey. :-(
As for the analysis, I don't think it has any meaning that the "experts" for map2 deviated so much from the trend. From your numbers I deduce that only seven (7) "experts" replied that saw map2. That seems far from being significant.
Two or three extreme opinions can easily change the average. (You have the individual numbers and should be able to see whether all seven hat a +2 rating or whether a few extreme +4 values changed the outcome.)
I agree. On my graphs that gave results vs. style of play, I didn't plot the results for the small sample sizes, but marked those areas as having insufficient data. I don't know why I didn't do the same for the data you refer to (results vs. level of expertise, for experts seeing map 2). I should have.
Somehow I don't subscribe to your theory... :-)
I'm not sure if that refers to that particular data point with the small sample, or overall. Despite the lack of conclusive data, I still think the original hypothesis is true and that a better experiment could be constructed to show it.
Back to the updated results article.
|
Simon Szykman (simon@diplom.org) |
If you wish to e-mail feedback on this article to the author, click on the letter above. If that does not work, feel free to use the "Dear DP..." mail interface.