Games and Their Outcomes: Qualitative Analysis Coding



I’m currently upscaling my knowledge about different qualitative methods approaches, and one of the approaches that I read extensively about and have now tried out is qualitative data analysis using codes applied in a software designed for such coding (in my case, Dedoose…largely because of the free trial month).

What seemed clear just from reading about this kind of methodology, and what became clear from my analysis is that the scope of the work is potentially quite large. I could have kept coding and working with the data for weeks longer, but because the data in question was only gathered for the purpose of this exercise, I decided to make sure that I dedicated an appropriate amount of time to the work and got what I could out of the exercise in that reasonable amount of time.

A few notes about the data: I asked my Monday Night RPG group if I could record one of our playsessions, code the data, and then write a blog post about it. With their permission, I recorded a 2.5 hour session and then chose a 25 minute section for transcription. I anonymized the data by using the character names rather than the player names in the transcript.

What I learned can probably be summarized thusly (no, not actually):
1. Transcription is awful and I wish I could pay someone else to do it.
2. Coding is a rabbit-hole from which one must plan a careful return. There are so many lenses I could have held up to the data.
3. Writing memos connected to the data helps to clarify just what lens you were using – both for yourself and others.

As I’ve noted elsewhere, our Monday night roleplaying group, as it exists now, contains an equal amount of male-identified and female-identified core members (when I say core, I mean those who attend weekly), with currently no “core” non-binary members, but with occasional guests. I understand that this is fairly unusual. With that knowledge as a starting point (and unsure where else to start, or how one ought to begin coding), I started to code the speakers as male-identified or female-identified. I also coded for the topic of conversation. I did a comparative analysis across codes and excerpts to see what patterns seemed to be emerging.

The section that I chose for analysis was a problem-solving activity involving decoding a journal entry. Not accounting for the lengths of contributions, the female-identified players spoke up 185 distinct times, where as the male-identified players spoke up a total of 99 times. Overall, the group usually spoke up to think aloud through the problem together, comparing notes and helping the group through the problem. Some players may have been more silent than usual as they tried to work on the problem separately. Female players were more likely to speak up in affirmation or support of their fellow players and their achievements towards solving the problem. Male players were more likely to question where the female players were drawing their conclusions from (perhaps because they were working the problem separately and not necessarily following the conversation).

Perhaps due to the problem-solving task that was put before them, in this excerpt the players did not act clearly “in-character” at any point. The only references to the setting and characters in this transcript were jokes related to anachronisms and game rules, as well as the contents of the journal itself.

Overall, I think I’m beginning to get the hang of this technique, but, as is also evident from the literature, my skills will evolve the more that I use this technique. In terms of my own game making practice, I could see this kind of analysis potentially being useful for analyzing focus group-type and other similarly-sustained conversations about my games. I don’t think this is something that I would use on shorter questionnaires or on shorter comments about my games. It is definitely something that works well when comparing different interviews together.

I want to thank my RPG group for being such good sports and letting me record them, given that my last post was about “accidentally” doing research while running Fate Accelerated for them last week. Thanks, folk!

Speculative Play: Deep Time and the Onkalo RPG

adventures in gaming, game jams, research


After spending the weekend immersed in thoughts about Deep Time at the Speculative Play Deep Time jam this weekend, it turned out that my Monday night RPG/board game group didn’t have anything to play that night. During the weekend, we had watched “Into Eternity” ( and thought about Onkalo (a waste-storage facility being built 4 or 5 kilometers deep in the Finnish bedrock), as well as nuclear waste more generally. Our discussions about deep time had talked about problem of designing for someone who might or might not share the same physical attributes, sensibilities, and senses. We talked about how difficult it was for the human brain to conceptualize a 100 000-year time-span, given that our own recorded history is so short and yet older events still feel so remote. We talked about intergenerational communication and responsibility, the durability of different materials and how to communicate broad strokes in imprecise mediums – perhaps things like massively-scaled stones, or “universal” symbols like thorns or other things that might represent danger to some unknown beings. We also thought about whether such warnings would only spur on treasure-seekers, who, unconvinced of the altruism of the people sending such a message (well, altruism except in the sense of assuaging our own guilt, perhaps), might think that something valuable was being hidden from them. And, given that nuclear waste materials can be reprocessed, and that a relatively small amount of their energy is used before the material is considered waste, it might be considered valuable indeed.

Given that I am moving to Alberta fairly soon and that our membership is already becoming increasingly scattered (Guelph, NYC, Regina…), the RPG group is working on strategies for being able to continue playing when we’re apart. So far, we have had mixed results with digital play, and of course it comes with a whole host of potential challenges with regards to tech, lag, internet issues, etc. Meeting for a casual board game wouldn’t further that cause at all, and I had been itching to run a game of my own for some time. I used to run a Star Wars expanded universe campaign, but it became too much for me to manage, and so I hadn’t actually “GMed” in years — there just seemed to never be enough time. Fresh off of discussions from the weekend, I decided that, given a simple enough system (Fate Accelerated, in our case), I could indeed run a one-shot campaign on-the-fly that evening.

I decided that I would give the group very little context, asking them only to give me information about who they were as a people (human, genetically-modified/differently-evolved humans, aliens). Their constraint was that they had to be of a similar size to humans (somewhere between human-sized and elephant-sized). My primary goal was to balance feasibility and fun, and so I did have to invent and alter certain details that may not be within the realm of possibility. Admittedly, although the results of this campaign were an interesting enough way into this design problem that I am now writing about it for you here, my primary motivation was running the game in a way that would be compelling for the players. Having dedicated so much thought and consideration to Deep Time and Onkalo over the weekend made them convenient subjects for exploration, and I thought that the ideas would work well in a one-shot campaign rather than something more sustained.

The players were experienced roleplayers from different backgrounds, although all were Canadians from the East Coast (Ontario and Quebec), including a biochemist, a store manager, a researcher working with Montreal’s itinerant population, and a bank worker. Although the group usually has an even gender split, the players this time were three male-identified players and one female-identified player.

Here is what they decided about themselves, their society and their context:
The game was to taking place 90 000 years in the future. The group was part of a race of genetically-modified humans that eventually evolved further to become quite sea-mammal like — specifically, they decided that they were the Otterfolken and had large lung capacity, webbed hands and feet, oily fur to protect themselves from cold in the water. They also decided that they would have bronze-age technology (and were quite insistent that this should include Archimedes’ death ray). Their characters were part of a caravan traveling across the land, seeking trade goods. One of them was the caravan chef and mixer-of-medicines, one of them was a religious elder/prophet who had visions, one was the caravan funder, a rich otterperson who was seeking adventure, and the other was a youngling who was in charge of caring for the caravan’s animals (these pack animals were known as “Finless”). Additionally, I seeded the adventure by giving them each one piece of information that none of the other players knew: the rich caravan funder knew that there were areas on this landmass that had not yet been scavenged by other caravans, the animal-tender knew that the area they were entering had very hard bedrock and was considered very stable (not prone to natural disasters, volcanoes, flooding, etc.), the caravan cook knew that food sources were getting more scarce and the land less hospitable as they ventured onwards, and the religious leader knew that there were legends/stories told in his religion about “places that you are supposed to forget, places that no one should ever go, deep places, sacred places” and that most of these were on land.

(That tiny track and even tinier truck represent the entrance to Onkalo).

Over the course of the weekend, Rilla Khaled and I explored questions around what we ended up calling “communicative geographies” — what kinds of human-made geographies could be used to primally communicate, beyond language, that Onkalo was a place to be feared. Using plasticine (reusable modeling clay), tin foil, and plastic cups, we built a structure that was designed to surround Onkalo. We were inspired by the shape of the Hoover dam — smooth, and descending at a terrifying angle — and by the idea, brought up in “Into Eternity,” that thorns were a threatening shape, one that might potentially still be understood in 100 000 years. So, Rilla and I surrounded the entrance to Onkalo with spikes on two sides and Hoover Dam-like curves of self-healing concrete (using bacteria) (knowing that such concrete is probably not infinitely self-repairing, we still decided to imagine it as such in a speculative future), all of this on a massive scale designed to inspire feelings of the sublime in the viewer.

For the RPG, I thought about Onkalo as more of a fortress – the huge thorny spikes on the outside, and smooth, Hoover-dam inspired bowl on the inside. To make it possible for the game to proceed, I decided that at some point since their creation, one small section of the spikes had fallen or been sheared off, allowing a climbable surface in one spot, should the adventurers decide to undertake such a climb.

Additionally, I surrounded Onkalo with other safe guards, attempts at communication: obelisk-like structures (some which had collapsed) with information in every known language, and a field of flowers, genetically-engineered to recoil away from other varieties to help them grow in set patterns (and also poisonous), forming the shape of a giant pictorial radiation warning as seen in the Onkalo film. However, the warning was designed to be seen from a birds-eye view, and they could not completely discern the pattern, although that they knew there was one (until, of course, they reached the top of the ominous structure, looked back and said “Oh, no!” — but their characters didn’t understand the symbols anyhow).


As the game played out, it became clear that the players, with no context, were playing out scenarios and thinking in ways that were consistent with our discussions over the weekend. When faced with a mystery, and in the context of the RPG, their solution was to go further and solve it. When presented with ominous symbols and danger, they decided that there must be something worth protecting hidden beyond — and, in the case of one character, their primary motivation was adventure-seeking, and this definitely looked like adventure.

The fact that this all took place in the context of an RPG night can’t be overlooked. This is the metagame — the tension between player knowledge (such as knowing the symbol for radioactivity) and character knowledge. The players knew, of course, that if I was leading them towards a certain place, there would be danger. This place wouldn’t just contain a pile of treasure for them to find. And although they discussed turning back many a time, they never did. The context of the game (and perhaps the lack of real-world stakes) encouraged them to move forward rather than turn back. But is the curiosity that drove the Otterfolken to Onkalo only human?

As I slowly pulled back the curtain and they discovered maps of the space within the Onkalo archives as well as more obelisks with writing and symbols, the group seemed driven by two motivations: uncover the rest of the mystery, and act according to the characters that they had set out for themselves. Afterwards, I gave them context for their adventure, telling them about “Into Eternity,” Onkalo and the weekend’s projects and adventures.

After this foray into using RPGs to explore a design problem, I’m convinced of their potential value as a design probe, especially for the Speculative Play project. Given time and space to do so, all humans are capable of speculation.

Crossposted here and here.

Games and Their Outcomes: Ethics Paperwork


This semester, I’m doing a directed reading course called “Games and Their Outcomes” and it is largely centered around this question:
“How can we make claims about player experience?”

Or, on a more basic level, what are qualitative research techniques that can be said to be “rigorous” and allow us to back up our methods and say with some degree of confidence that we know what we know. The upshot of taking on this kind of course is that I’m upscaling my research methods (or in a lot of cases, learning that what I was already doing intuitively is a thing that qualitative researchers who want to be rigorous do).

The first assignment that I’ve completed is one that, fittingly, also comes at the beginning of many research projects: the ethics portion. I’ve completed drafts of ethics paperwork for the Speculative Play team which I’ve just joined (featuring Rilla Khaled, Pippin Barr, Christopher Moore, Brian Greenspan, Liane Decary-Chen, Agustina Isidori and, now, me!)

There isn’t too much to say about the paperwork itself (it was fairly straightforward), except that Ethics within a research framework, especially at a University, especially for the arts, is not perfectly designed to fit research-creation work. Having co-designed a game about consent, I know my way around the topic fairly well.

But proposing ethics around design and art projects where what might develop is unknown and might potentially spiral into something completely different is a particularly strange experience. In research-creation, you don’t necessarily know what you’re going to do before you do it – it’s the nature of the beast. Ideally, we would like to not have to submit ethics paperwork for every tiny project that we plan to do — that would waste both our time and the Office of Research Ethics’ people’s time, too. We want a flexible framework that respects people and our ethical responsibilities that isn’t too bogged down in the bureaucracy of the thing. It turns out that, that’s complicated.

I think that what was important about this assignment was learning to complete ethics paperwork of a similar kind to what I will actually need for my research. It was…demystifying. But I also kept bumping up against the limits of my knowledge — and the limits of what I could decide on my own from common sense. But, if we wait to submit the paperwork until we know exactly what the project will be, we may end up waiting on the ethics paperwork to actually be able to do the research, which may delay the research. Definitely not an ideal solution!

At time of writing, we are waiting to talk to the OOR ethics folk over the phone to see what we can do with our application that will allow for proper ethics but also won’t require us to submit an application for every small-scale project.