Z-A Challenge 2016 Index

A-Z 2016 "Z"Buffering made the challenge much easier this year. I had time to prepare three or four posts before April 1, and while I never got that far ahead again, it was enough that I was able to stay on time. I think almost always I had at least one post for the day up by 9:00 PM the night before (Pacific time; midnight Eastern time). This was mostly because I get up early and wanted to make sure they were live before I went to bed.

Not that I often get to bed by 9:00 PM, but that was the thought.

Slightly more posts than previous years (34 this year, 28 last year and 32 the year before). About 36,583 words, a little lower than a couple years ago (about 38,000), somewhat more than last year.

The most common topic in this challenge was ‘graded items’, magic items with multiple powers assigned at different grades (measures of power). I suspect that after review and revision the topic will form the core of another book. Like Polyhedral Pantheons there’s a good chance a small part of the book will explain the processes and much more of the book will be spent on gameable examples. Somewhat to my surprise I didn’t actually create many such examples during the challenge — they’re quick and easy, but I had bigger thoughts to write about.

Another major topic this time around was some cartography techniques regarding how to draw mountains. Rounding the list out were a few posts about encounter design and development, and the data capture and encoding process I use in creating the Echelon Reference Series (and how I think I’ll be doing it over, better… again).

A pretty satisfying run this year. Buffering made a huge difference, even being a single day ahead gave me some grace when I was pressed for time. Having two evenings a week consumed by judo makes this pretty important.

Okay, enough A-Z Challenge for this year, time to get back to work.

Date Letter Words Title
2016-04-30 Z 552 Z-A Challenge 2016 Index
2016-04-29 Y 1,153 Yet More Reorganization Notes
Y 1,288 Yet Another Grand Reorganization
2016-04-28 X 680 Exotic… No, Esoteric Draconic Bloodlines
X 389 XML Workflow, A New Direction
2016-04-27 W 2,133 Weapons and Armor made of ‘Special Materials’
2016-04-26 V 1,099 Variation: Graded Item Sets
2016-04-25 U 1,100 Umbral Mail
U 963 Unchained Skill Unlocks for Echelon
2016-04-23 T 1,333 Touching up the Mountain Colors
T 348 Thinking Again About the Price of Graded Spell Trigger Items
2016-04-22 S 1,120 Several Mountain Ranges Together
2016-04-21 R 1,195 Revisiting the Mountain Tutorial, Drawing the Initial Landform
2016-04-20 Q 1,396 Quirk, Flaw, Curse: What’s the Difference?
2016-04-19 P 1,159 Pushing Your Luck: Enchantment Gone Wrong
2016-04-18 posted midday Sunday because it was part of a conversation online
2016-04-17 O 3,002 On Mapping Mountains, Using a Few Simple Tricks
2016-04-16 N 841 New Uses for Unchained Item Qualities
2016-04-15 M 296 Metamagic Feats in Graded Wands and Staves
M 296 Midpoint Check-In
2016-04-14 L 457 Legendary/Spontaneous Graded Items, Made Simpler
L 2,096 Legendary/Spontaneous Graded Items
2016-04-13 K 767 “Kill Everything”: Making That Plan B
2016-04-12 J 2,416 JRPG-Inspired Encounter Design
2016-04-11 I 2,055 Improving Encounter Economy and Design
2016-04-09 H 1,157 Hammer Time! Polyhedral Graded Weapons and Armor
2016-04-08 G 940 Graded Weapons and Armor
2016-04-07 F 1,317 Forging Graded Items
F 62 Fey Bloodlines
2016-04-06 E 1,789 Exploring Multiple Charge Casting for Graded Items
2016-04-05 D 845 Determining Market Price of Graded Staves and Wands
2016-04-04 C 907 Crafting Graded Staves
C 718 Crafted Graded Wands
2016-04-02 B 198 Blood of Dragons? In My Veins? It’s More Common Than You Think
2016-04-01 A 466 Assigning Graded Abilities
Total 36,583

Yet More Reorganization Notes

A-Z 2016 "Y"I had time to think about this some more on my bus ride home. And even take notes, though that’s a bit of a challenge (I hate writing in a moving vehicle).

It appears I was only mostly right in what I had planned. I was having difficulty reconciling ‘data type’ and ‘data object’ in some cases. For instance, ‘rage power’ is a class feature, cool… but it can also be a data type. There are literally hundreds of rage powers. I know. I counted.

Turns out a solution is pretty simple. Time for some data modeling.

Data Dictionary

Good data projects really should have a data dictionary. I’ve worked (been stuck with) too many that don’t, so I’m starting one now.

Some of the names are subject to change, but I describe the purpose of the various types below.

Data Object

This is the core element of the entire system. Almost everything I care about is a data object: feats, spells, skills, monsters, everything. A data object has:

  • attributes, meta-information about the object that describe how the object is to be treated. Most data objects have default values for the attributes.
    • one of the common attributes is ‘parent’, indicating the data object this is an example of (“Rage Power” has an parent of “Class Feature”). This will often be set more or less implicitly by a new object being subordinate to a data object or data type. “Rage Power” is defined inside “Class Feature”; I can set it explicitly but don’t need to.
    • another attribute indicates that the data object can be used as a data type. For instance, “Class Feature” might be a data object (that probably never gets printed, oh well) marked “data-type=’class-feature'”. “Rage Power” is a class feature that might be marked “data-type=’rage-power'”. When I want, I can use a ‘rage power’ data type marker and all data objects under it are marked as being rage powers.
    • another attribute indicates ‘index type’, how to mark objects of this type when they get indexed. “Rage Power” is a class feature, and is indexed as “Rage Power (class feature)”, while “Animal Fury” gets indexed as “Animal Fury (rage power)”.
  • stat block (often empty): spells and monsters are examples here, but even feats can be said to have a stat block (type, prerequisites, type, and minimum level). May or may not be rendered as an actual stat block.
  • content: block text, tables, lists, etc., and may have sections and subobjects.

Data Type

Data type is actually much smaller and simpler than I’d anticipated. It’s basically a marker to show that the objects ‘inside it’ are of a particular type. Importantly, though:

  • the data type markers are never printed, they exist only in data.
  • there is a hierarchy to the data type Word styles, but only so they can be nested within other things (a bloodline has bloodline powers; structurally the data file might have
  • Bloodline [data type]
    • Draconic [‘bloodline’ data object]
      • statblock (bloodline skill, list of bloodline feats, list of bloodline spells, etc.)
      • content
      • Bloodline Powers [section heading]
        • Bloodline Power [data type]
          • Claws [‘bloodline power’ data object]
            • statblock
            • content
          • Dragon Resistances [‘bloodline power’ data object]
            • statblock
            • content
          • Breath Weapon [‘bloodline power’ data object]
            • statblock (save DC)
            • content
          • Wings [‘bloodline power’ data object]
            • statblock
            • content
          • Power of Wyrms [‘bloodline power’ data object]
            • statblock
            • content
  • while there is a hierarchy to the Word styles used, it does not reflect on the data hierarchy. “Rage Power” and “Bloodline Power” are both data types, even though “Bloodline Power” is typically used inside a “Bloodline” (as shown above). There is really no fixed hierarchy to the objects, though there is to the data types. If it turns out convenient to have a “Magic Item” inside a “Monster” (not common, but not unheard of) you can do it.
    • Word style hierarchy probably goes
      • data type 1 >
      • data object 1 >
      • data section 1 >
      • data type 2 >
      • data object 2 >
      • data section 2 >
      • data type 3 >
      • data object 3 >
      • data section 3
    • If I somehow really need more levels (class -> class feature -> class subfeature -> class subfeature power: sorcerer -> bloodline -> draconic bloodline -> claws) I’m probably making it difficult for myself. The ‘bloodline class feature’ can stay here (it identifies that the class has that feature), but the bloodline definitions can themselves be higher-level objects.
    • It occurs to me that this even lets me have objects that are children of the same kind of object. For instance, ‘monster group’ and “magic item group” can now be nested. “Dragon” > “Chromatic Dragon” can now work, as can “Rod” and “Metamagic Rod” or “Wondrous Item” > “Figurine of Wondrous Power”.


I have a practice of indexing the game content pretty aggressively. I can include indexing rules in the data objects so that when they are referenced as types the objects can be indexed the way I want. This definitely includes the type markers I have in the index, but can also include hierarchy displayed in the index.

I expect that I will be able to create indexes something like:

  • Affliction
    • Curse, see Curse (affliction)
    • Disease, see Disease (affliction)
    • Poison, see Poison (affliction)
  • Bloodline (class feature)
    • Draconic
    • Elemental
    • Fey
    • Undead
  • Claws (bloodline power)
  • Domain (class feature)
    • Air
    • Death
    • Earth
    • Fire
  • Draconic (bloodline)
  • Earth (domain)
  • Electricity Resistance (domain power)
  • Elemental (bloodline)
  • Fey (bloodline)
  • Lightning Arc (domain power)
  • Undead (bloodline)

Parts of this will be pretty easy, a few might be a bit tricky… but I’ve solved harder problems.


The semi-arbitrary/semi-abstract nature of the data structure also makes it so I can give each object a unique identifier, as long as the object is defined in basically the same way twice.

Data objects have two identifiers. The first is the group-id and consists of the type and name (modified). The “Rage Power” class feature has group-id=”class-feature.rage-power”. Some (parent) data types can be marked to be appended to the end of their children. For instance, while “Rage Power” has group-id=”class-feature.rage-power”, when described in the barbarian class (“Class” data type says “append to children’s IDs”) that object has id=”class-feature.rage-power.barbarian”.

The dual IDs let me group the objects by group-id (they are the same type and same name, and therefore equivalent for search purposes) while keeping the individual instances separate. This is important as there is more overlap, because I can then refine the data: I can have a single master definition of “Uncanny Dodge” that explains what it does, and the class-specific entry is reduced to a note saying when that class gains access to it.

Closing Comments

Didn’t plan to write this tonight, already had a “Y Day” post (and we were supposed to go judo but I’m having vehicle problems… again). I had some notes scribbled on the bus and I thought I’d try to get them into more readable form.

Strangely enough, I didn’t actually look at them. It seems the act of writing them was sufficient to cement them in my mind for later. Which is good, because I can’t read the damn things.

Yet Another Grand Reorganization

A-Z 2016 "Y"One of the hazards of being a data geek, and good at it, is that over time you become better at it. You find better ways to do things.

Also, the Pathfinder Roleplaying Game has become progressively more complex, both from player perspective (hence the Echelon Reference Series) and particularly from a data modeling perspective.

It’s time for me to fall back and regroup, reorganize how I’m capturing the data. The workflow changes somewhat, but more importantly the data and file management changes.

Existing Data Models

Today, each source is captured more or less exactly as published. Each Word file represents (usually) one source, complete with document structure (book, part, chapter, section, subsection, etc.) and game element structure (each game element is a major element, minor element, subelement, etc., and there are divisions within them such as for a bloodline’s bloodline powers).

Word produces relatively flat, unhierarchical files. Whether converted to HTML or converted to XML, the document structure basically lacks hierarchy. For instance, conceptually a book’s structure has chapters, and a chapter may have sections. That is, there is an implied hierarchy. In the document files, though, rather than

  • chapter
    • paragraph
    • paragraph
    • section
      • paragraph
      • table
        • table rows
      • paragraph
    • section
      • paragraph

you’ll see

  • chapter
  • paragraph
  • paragraph
  • section
  • paragraph
  • table
    • table rows
  • paragraph
  • section
  • paragraph

There are tricks, tools, and techniques for dealing with these, and I’ve gotten good at them. However, the process ultimately generates some eighteen levels of grouping (7 for document structure and 11 for data). In many cases I need to infer from element ancestry what I’m looking at. That is, I might have a “class-feature” called “domain”. Each “class-subfeature” is an instance of a domain (“Air”, “War”, etc.). The “class-subsubfeature” in that is a domain power… but it’s up to me, in my code, to recognize that.

Do you have any idea how many class features exhibit complicated internal structure like this?

New Data Models

I’m splitting the files. Data will all go into one set of files, and ‘document content’ will go into another set of files. I have found in my preliminary experiments that the ‘documents’ change remarkably little after initial capture, but the data elements get tweaked and massaged quite a bit.

Sometimes the purely document content is not Open Game Content (OGC), or is outright Product Identity (PI). I capture it mostly so I can reproduce the original document formatted to my taste (easier for me to read; so many publishers use hard to read fonts, for example), partly so I can view the game elements in situ so I have better context for examining them later, and frankly because, well, I’m a data geek and I like things to be complete. The PI and other non-OGC never gets republished.

Document Files

The document files are pretty straightforward. They follow normal document structure conventions (chapters, sections, etc.). They also can have “include commands” that identify game elements to be added to the document at that point when rendered.

When I captured the text of Pathfinder® Roleplaying Game: Ultimate Combat™ I originally reproduced the document structure, marking up the feats so they could be automatically extracted. Now I would create a data file for the feats, and the “Feats chapter” is reduced to introductory text and a series of “include these feats” instructions.

This gives me much more control than I had before. It also significantly reduces the size of the files I work with, which makes my job much easier.

Data Files

The data files are even simpler. Most game elements have a very similar structure, and the major difference is how they are applied.

  • Element name
    • Summary/statblock information (includes prereqs)
    • Descriptive text
    • Subelements (repeatable… and have the same structure as the parent)

Right now I define new Word styles for the various element types… but structurally they almost all boil down to the same basic structure. There are exceptions, but probably 80% or more of the game elements I deal with fall into this structure.

Instead, I’m going to rely more on metadata to identify what a particular element is. The metadata type will have a definition that is shared when needed, but otherwise will be used just by name. That is,

  • “Feat” [data type marker]
    • “Dodge” [data element]
      • prerequisites Dex 13
      • benefit lorem ipsum
    • “Mobility” [data element]
      • prerequisites Dex 13, Dodge feat
      • benefit lorem ipsum

(I spelled out ‘Dodge feat’, but given just a name the parser can usually find what it’s after… but ‘Dodge feat’ is explicit and resolves ambiguous cases).

Ultimately I end up with data objects something like

<d20:object class="feat" name="Dodge">
    <d20:prereq refid="score.dexterity" refclass="score" value="13" />
  <!-- content elided -->

<d20:object class="feat" name="Mobility">
    <d20:prereq refid="score.dexterity" refclass="score" value="13" />
    <d20:prereq refid="feat.dodge" refclass="feat" />
  <!-- content elided -->

Feats, most class features, and so on are generally presented in pretty much the same way. There are exceptions, of course, but I can now focus on handling them differently at need rather than having to lay out each data type explicitly.

It can be more complicated, but in many ways it isn’t. For Polyhedral Pantheons I might have something like

  • Deity [base data type]
  • Shu-shi [parent=Deity]
  • Jixiang Shen [parent=Shu-Shi Deity]
  • Zhengchang Shen [parent=Shu-Shi Deity]
  • Bukeishiyi Shen [parent=Shu-Shi Deity]
  • Goblin [parent=Deity]
  • Vorubec [parent=Goblin Deity]
  • Jhesiri [parent=Goblin Deity]
  • Kouzelnik [parent=Goblin Deity]


  • Jixiang Shen [data type]
    • Huanghou
    • Xingyun
    • Xiao Ling
    • Chengshi
    • Zhongli
    • Jingcai

Because I defined the data types as I did, I can traverse the relationships a couple ways. If I need to, I can determine that Huanghou (empress of heaven) is a Jixiang Shen (auspicious deity), a Shu-shi Deity, and a Deity. This gives me quite a bit of control over the formatting (default ‘game object’ formatting? more specific ‘deity’ formatting?), and even the indexing. The index might include

  • Deity
    •  Shu-shi
      • Jixiang Shen
        • Chengshi
        • Huanghou
        • Jingcai
        • Xiao Ling
        • Xingyun
        • Zhongli
  • Huanghou (shu-shi deity)

(because I decided I only wanted to go as far as the pantheon, not the subpantheon, here… custom indexing rule)

Closing Comments

My existing file and data structure has evolved over time to the point it has become hard to use. Splitting the files into “document content” and “game data content” lets me offload a lot of the more static content (document) and focus on the more often edited content (game data). It makes it easier to exclude the bits I mostly don’t care about most of the time (I don’t have to load the document content into my data store, where it gets repeatedly loaded and processed later) while keeping them available for later if I find I want them. This should speed up capture, editing, and processing.

Abstracting the data lets me rely more on the common aspects of the data. Feats, spells, and deities can all be structurally quite similar: name, statblock, text, done. I can start from there and refine as needed, rather than the current model that requires that I get detailed early, and find that I have many objects that are structurally the same.

This will let me do the RAF (Rough and Fast) versions of new data… well, rougher and faster. It will also let me focus my effort on the more complex cases where I want to know more about the game object. Spells can be structurally similar to other blockish game objects, but I can gain quite a bit by parsing further. Similarly, I know the “Domains” field of a deity definition will contain references to domains (object of type ‘domain’), so if I put just a little more effort into it I can parse and extract that information… let me both index the domain reference, and even update the domain object by adding a “Deities” line identifying the deities who have that domain.

I… did say I’m a data geek, right?

Exotic… No, Esoteric Draconic Bloodlines

A-Z 2016 "X"‘X Day’ is always difficult in the A-Z Challenge. So few words actually start with ‘X’.

I was going to write about how I was going to add esoteric draconic bloodlines to Draconic Bloodlines, since I deliberately excluded them when I released the book. I even started drafting that post, but when I started examining things more closely I realized my original reasons for excluding them stand.

The esoteric dragons are too different from the other dragon families, and creating bloodlines for them the same way would involve me picking things without enough knowledge to make good decisions. The esoteric dragons often have many spells (in a couple of cases seven or eight), and trimming that down requires me to choose from among them. I’ll have to come back after I’ve had a chance to read more about occult magic and internalize it somewhat.

That said, I now see four ways forward, where previously I’d seen one.

Ways to Include Esoteric Draconic Bloodlines

The primary stumbling block here is that almost all dragons cast spells as sorcerers, except the esoteric dragons. Esoteric dragons cast occult spells as psychics. I present them below, in approximate order of deviation from plan.

Steal From the Psychic Bloodline

The psychic bloodline’s bloodline arcana says

Your sorcerer spells and spell-like abilities count as psychic instead of arcane. You use thought and emotion components instead of verbal and somatic components when casting your spells.

In principle that arcana pretty much solves the “sorcerer magic vs psychic magic” disconnect, but it seems incomplete. I’d probably want to incorporate some of the bonus spells, bonus feats, and bloodline powers (at least one) to make the bloodline ‘more psychic’. I thought I might find sufficient overlap in the dragon special abilities to handle some of the ‘psychic bloodline powers’, but after reading them I don’t think so. There are some nifty abilities, though.

Create a Sorcerer Archetype

This honestly was my first intended approach, and went beyond what I was prepared to do when I considered it. Changing the sorcerer’s spell casting is, after all, a pretty fundamental shift. This would give me the opportunity to tweak more of the class features so things would fit properly. This is probably my favored approach right now, but again I’ll need to do some more research.

Create a Draconic Sorcerer Prestige Class

In discussion online it was suggested that a prestige class could solve the problem. I usually wouldn’t agree, I’m not a big fan of prestige classes to solve mechanical problems (I do like them as campaign elements — actual “prestige” classes), but in this case it could work. If a sorcerer starts with the core draconic bloodline, a prestige class could allow the sorcerer to shift to a more specific draconic bloodline, or even add it altogether. There would probably have to be some kind of additional cost or change to class features… a prestige class like this is something like a late-added archetype, I suppose.

Backup plan, definitely behind the archetype and possibly even behind the psychic bloodline.

Make This a Psychic Discipline

Perhaps I’m going about this the wrong way entirely. Esoteric dragons, unlike the other dragons, cast as psychics rather than sorcerers. Maybe there should instead be a ‘draconic’ psychic discipline that can be specialized.

Right now I’m leaning toward this as my favored approach. I’m not sure if it would be a separate product or not, though.

Closing Comments

I think a large part of my difficulty with esoteric dragons is that they are a poor fit for the sorcerer class altogether. The minimalist approach (pull from the psychic bloodline) seems woefully inadequate. Creating a prestige class to fix it smacks of a band-aid fix, and I don’t like those. A sorcerer archetype is my favored solution from a design standpoint, tweak the class first before adding the psychic dragon stuff.

Ultimately, though, I think keeping them as a psychic class thing and making them psychic disciplines (possibly with a generic ‘draconic’ discipline) is the right way to go. I’ve got some more reading to do.


XML Workflow, A New Direction

A-Z 2016 "X"A couple years ago, or just slightly more, I wrote about my workflow for extracting game information captured in Word. It’s kind of long:

  • Type (or copy and paste) into Word;
  • Convert Word files to ‘Filtered HTML’;
  • Fix character encoding;
  • Convert to XHTML;
  • Convert to XML closer and closer to the problem domain (game elements) using a series of XSLT scripts.

Once I’ve got the information encoded I can do other transformations to get my actual goal products:

  • Machine-generated diagrams:
    • Build a hierarchical model for each game element that has or is a prerequisite;
    • Convert that hierarchical model into DOT format (GraphViz input file; I’ve written about visualization using GraphViz before);
    • Render the DOT files into PNG and SVG format, giving me diagrams I can redraw (GraphViz is powerful, the output isn’t always suitable for inclusion in my books) showing the relationships between game elements.
  • PDFs:
    • Convert XML files (created as above, but using ‘book markup’) to LaTeX;
    • Convert LaTeX to PDF (this can incorporate diagrams redrawn as described above.
  • Index and analysis files; I sometimes create spreadsheets containing…
    • spell summary information;
    • master spell lists for all classes (and domains and bloodlines and patrons…);
    • summary monster stats;
  • I also sometimes create new Word files containing aggregated or reformatted content.

This has proven fairly effective over the last few years, but I think it’s time for a change. Word has a ‘WordProcessingML’ that represents, to a fairly large degree, the internal memory representation of a document. There is a great deal of information there that can be discarded, and some ‘internal Wordisms’ I’ll need to work around, but I think this can get me past some niggling translation and encoding difficulties I’ve been having.

The new workflow will probably look much like:

  • Type (or copy and paste) into Word;
  • Convert Word files to WordProcessingML;
  • Convert to XML closer and closer to the problem domain (game elements) using a series of XSLT scripts.

This doesn’t seem like it saves me a lot of steps, but in reality it does. The “Fix character encoding”, “Word -> Filtered HTML”, and “Filtered HTML -> XHTML” do useful work, but all three stages introduce some annoying data artifacts I need to work around. The new workflow should not only reduce the number of stages (the initial bullet points), but should make the processing after that much simpler.