I started playing Dwarf Fortress again recently, partly inspired by the videos produced by Kruggsmash. He does a wonderful job of extracting the story from a fortress, illustrating it in a cartoonish way, and pulling out a narrative. But a lot of that narrative comes from the game systems, influenced by player actions, through procedural generation. That also provides some motivation for me to explore this in my own games, not least so I can experience surprise like the players do!
In “classic” D&D, such as Mentzer BECMI in 1983, the Dungeon Masters Rulebook (note that it is not a “Guide”), the last chapter is devoted to “Creating Dungeons”. Here, the book warns against too much randomness.
A good dungeon is reasonable. Its design is carefully thought out, and the monsters and treasures are placed for a reason…
Even a random dungeon could be a good dungeon, if the monsters within it were selected and placed carefully. Randomly drawn caves would give a disorganized design of rooms and corridors, but could be a good dungeon if occupied only by cave dwellers…page 46
That said, it goes on to use several “random” (procedural) techniques within the “making a good dungeon” checklist. For example, when talking about monster placement:
You should select (and not roll at random) some special monsters, based on the scenario. You may create new monsters if desired. For example, if the scenario is “Recovering Ruins” in a “Ruined Town” setting, you might place a few hobgoblin lairs (15-20 creatures in each), plus their pets and friends, as Special Monsters. The rest of the ruins could be filled randomly. The entire “dungeon” could be used for several adventures.page 47 (emphasis original)
After the dungeon is designed, there is a section on random stocking using a 2d6 table plus a random treasure table. The first d6 determines the basic content of the room, while the second determines whether treasure is present (T).
Combine this with a random encounter table for the monsters and some basic traps, and you can stock an existing dungeon directly. After 35 years, game design has iterated on this pretty significantly, the core idea remains the same and has resulted in highly-tuned blockbusters like Diablo.
For what it’s worth, I used exactly this table when restocking the levels of Dyson’s Delve in my 5th Edition conversion for the Relic Hunters Guild a few years ago, and that’s why I wanted to summarize it here. 5e works remarkably well in these classic modes of play! Mike Mearls uses Tarokka cards and other techniques to do this sort of thing as well, specifically “to keep the game interesting” for himself as DM:
Outside the dungeon
That gives us a start for the dungeon itself, but what about getting us there? Again, think about classic D&D tropes like “you start in a tavern.” It wouldn’t take much to generate some rumors, Mad Libs style, plus a few patrons – perhaps including a quest giver or two. Donjon does exactly this in the Random Inn Generator (and in fact this drove big chunks of the early levels of Dwarves & Dweomers). So I’d start with that list, plus a generated town (recently updated, thanks Drow!), and see what happens.
You could go a step further and generate a villain from lists of types and motivations, then maybe determine their general approach (suborning someone in authority, hiring bandits, etc.) The 5e DMG has quite a few pages on this, some of which were released in the previews. Tarn Adams, best known for Dwarf Fortress, recently gave a talk on his thoughts in some detail:
You could use this process to generate a whole world. (I should write an update to my previous post on map generation in DF, as the tools have improved tremendously.) But these days, I think starting small and working from the bottom up would work better. That gives you the opportunity to interpret the random results and thus to shape the narrative based on your improvisation and collaboration with your group. As the BECMI Rulebook pointed out, random generation can only go so far.