Playtest Update
It’s another play test update from Mike in this Legends and Lore post. The majority of the post details what’s going right and what’s going wrong, based on the latest round of surveys. The details are interesting if you’re trying to design the next D&D, but I’m going to focus on the broader design process.
Mike’s entire post is about what people have said via the playtest surveys. The key trends that he mentions:
- eldritch blast is too powerful
- Sorcerers aren’t Sorcerer-y
- Wizard HP is too low
- Wizard isn’t flexible enough
- Rogues aren’t focused enough on skills
- Skill mastery is boring
- Sneak Attack is too good
- Issues with health and healing
- Monster accuracy is too low
- Monster HP is too low
- DMing is too hard
- Fighter expertise is good
- turn undead shouldn’t be a spell
- Advantage/Disadvantage is good
For any of these, feel free to mentally rephrase them as “the ____ is too damn _____.” “The monsters are too damn easy.” “The eldritch blast’s damage is too damn high.” &etc.
Anyway, the interesting thing about aggregate results like this is that they’re entirely non-personal. Depending on the exact distribution of the data, it may be pretty unlikely that any one person thinks all of these things are true. “Fixing” all of them is likely to cause new problems since (for example) some people who thought skill mastery was too boring will now think that rogues are too weak, etc.
Using these aggregate results is like designing for a mythical Everygamer. It’s easy to look at huge splats of data and come to conclusions on what will make the data trend different ways, but that doesn’t always correlate very well with individual people.
Even when it does correlate well, what people complain about and what they actually want can be very different. The testing process for the video game Borderlands is an interesting illustration of this point. The article is well worth a read, but in case you’re strapped for time I’ll pick out a few key points.
During testing Borderlands received some feedback like:
- Testers wanted to build their own guns
- Sprint speed is too low
- Guns reloaded too slow
- Skag Gully has too many clusters of enemies
This is the kind of feedback Mike and the D&D team are getting right now. It’s easy to take that at face value and do things like make weapons reload faster, make movement faster, add gun building, and remove some enemies from Skag Gully.
What the Borderlands developers found, though, was what people say they want doesn’t always line up with what they actually want.
Instead of increasing the sprint speed, the developers added more detail to the ground. The testers didn’t really want to move faster, they wanted to feel like they were moving fast. Having more references to judge speed by solved it.
Instead of making guns reload faster they made the reloads more interesting to watch. Reloads felt faster while still being balanced within the game.
Skag Gully didn’t have too many Skags, it had too few. Testers thought of it as a travel area with too many enemies, but it was actually a combat area with too few enemies.
Gun creation is the most interesting point. Borderlands is a game about wanting: wanting a faster gun, a more damaging gun, a bigger gun. You play the game to find more guns. Once you don’t have a burning want for more guns the game falls apart. Gun creation would actually make the game less fun.
Since reading that article I’ve always wondered about how similar RPG feedback is. When people say the wizard has too few HP, is that really what they mean? Or do they mean that the wizard needs more ways to stay out of combat, or that the Fighter needs ways to defend them, or that there needs to be more small healing spells (which make a lot of difference to the Wizard but not much to the Fighter)?
Testers are fundamentally honest and they report what they feel about gameplay, but they aren’t designers. In fact, that’s the point of these kind of testers. When a tester reports an experience the best thing to do is consider that experience, the tester’s response to it, and how those all fit into what the game does.
With Mike’s summary of playtest feedback I don’t know where those items fall. Is monster accuracy too low because monsters actually need to be more threatening, or is it too low because the system makes misses boring? I don’t know, but I’d lean towards the latter, in my own designs.
This is a possible pitfall of design by survey. I don’t know that D&D Next has fallen into it, as I haven’t played the next version that addresses some of these issues. It’ll be interesting to see how the game progresses as they gather more and more feedback.
Very interesting analysis. Thank you.