Pyrel dev log, part 4

[Kinematics]

Have rewritten the nameProc function to eliminate hard-coding the affix type values. It now constructs them based on the nameOrdering field in the affix types: first two are for adjectives before the first noun, second two are for adjectives before the second noun, and everything else is ordered after that for suffixes.

Still fairly heavyweight in code, so added a cachedDescription field to item that can store the results. The name can then just be rebuilt from that plus the quantity. Need to consider how to handle mutating the 'state' adjective (modAdj1, affixTypes[0]) in a callback. Anything that goes beyond that (eg: experience for weapons changing their powers) should just clear the cachedDescription to force a recalculation.

Finally got the list working for AFFIX_TYPES. Figured out that the reason you didn't need to use the 'global' keyword in the original code was that you were just mutating collections, not setting them, whereas my list comprehension was setting the variable directly. Changed code around and now it works.

[Kinematics]

Added in handling of a callback parameter that can change any of the item description elements.

Tidied up the code a bit, and tried to get things back down to ~80-90 columns long.

[Kinematics]

BTW, pushed all my latest code (aside from some testing stuff). Still has a number of todo's in it, and definitely needs review, but figured it would be easier to see the current progress that way.

Went through a couple more revisions from the comments above, cleaning up and making it as generically adaptable as possible.

[Kinematics]

Continuing thoughts:

The core of the naming construction code seems to be coming along fairly well, though there's still a few uncertainties left. Primary issues right now are how it connects to other code: how and which procs get attached to each item, what persistent info should be stored on each item object, and how it all interacts with the template nameInfo/Category information.


nameInfo: a dictionary collection of information per template. Stored per template? Copied to each item? How is it accessed?

Will start with assuming it works like a lot of other item fields, where the core info is stored on the template, and each instance of an item that's created from that template gets most pertinent information copied over to it.

What does nameInfo need to store?

proc: proc name -- usually the default code
baseNoun: the noun that's the building block for the rest of the item's name
modifierNoun: a modifier noun or partial noun phrase. eg: "set of"
flavorTypes: type of flavor to use for items of this type (eg: 'wood' for staves, 'colors' and 'liquids' for potions, etc)
properName: proper noun that names an item, or attributes its origin (eg: Galadriel for "Phial of Galadriel", 'Elessar' for "the Amulet 'Elessar'")


Rereading Derakon's post, my code may be a little too monolithic. I can see where I could approach things slightly differently with a few lightweight proc classes, and then an overall processing function. EG:

# pass itemRecord.nameInfo in params

# Default namer class for most items
class DefaultItemNamer(proc.Proc):
    def __init__(self, triggerCondition, params, procLevel):
        proc.Proc.__init__(self, triggerCondition, params, procLevel)
        # Store parts of the item noun
        baseNoun = params['baseNoun']
        modifierNoun = params['modifierNoun']
        # And store the 'name/origin' of the item, if provided.
        properName = params['properName']

    def trigger(self, item, alterNameCallbacks = None):
        return getItemName(self, item, alterNameCallbacks)


# Namer class for unique (ie: artifact) items
class UniqueItemNamer(DefaultItemNamer):
    def __init__(self, triggerCondition, params, procLevel):
        DefaultItemNamer.__init__(self, triggerCondition, params, procLevel)

    def trigger(self, item, alterNameCallbacks = None):
        return getItemName(self, item, alterNameCallbacks, unique = True)


# Namer class for flavored items
class FlavoredItemNamer(DefaultItemNamer):
    def __init__(self, triggerCondition, params, procLevel):
        DefaultItemNamer.__init__(self, triggerCondition, params, procLevel)
        # Also store flavor from provided parameters
        flavor = flavors.chooseFlavor(params['flavorTypes'])

    def trigger(self, item, alterNameCallbacks = None):
        return getItemName(self, item, alterNameCallbacks, flavor = self.flavor)


# Function to handle actually creating an item's name.
# Called via one of the above procs.
def getItemName(proc, item, alterNameCallbacks = None, unique = False, flavor = None):
    baseNoun = proc.baseNoun
    modifierNoun = proc.modifierNoun
    itemName = proc.properName
    pass

The flavor can end up being encapsulated in the proc attached to the item, then, and set when the proc is constructed.

A 'unique' namer proc means you can specify names for items that aren't unique, since the unique check is no longer tied to the existance of a proper name.
EG: 'fake' artifacts -- "the Dagger 'Angrist'", or "a Dagger 'Angrisste'"


I'm not entirely sure on the logic flow for the alterNameCallbacks. It depends on where they might be created (fixed at proc creation time, or dynamic during program flow?), how they're referenced, and whether code will want to access them, or if they're separate procs, etc. That can be tweaked later.


Still not sure how we intend to handle subtypes for flavored items. Derakon's example put the subtype in the category section, but since the category section contains many different things (none of them labelled), how do you determine which one is relevant?

Categories: ["potion", "cure light wounds", "glass bottle"] -- how do you turn that into "a green potion of cure light wounds"? We can get the flavor from the nameInfo and/or the proc, but we can't yet pin down the subtype.

Aside: I might just be missing it, but I can't seem to find where any of the code handles applying the affix of a subtype (eg: a ring of strength). If the affix -is- being created for it, the naming code needs to be adjusted to pull that one first when constructing the item name, and not list it twice.

If each proc instance is created for each item, and the proc instance is defined by the factory that made it, the proc can maintain all of the item description information, rather than looking up the data as a field on the item itself.... (rewrote the above code with this idea in mind)

[Kinematics]

More thoughts on object data...

Where/how to put in nameInfo data

current:

{"type": "staff:staves", "templateName": "staff", 
"display": {"ascii": {"color": "flavor", "symbol": "_"}}, 
"stats": {"weight": 5.0}, 
"flags": ["EASY_KNOW", "FLAVORED"]},

revised?:

{"type": "staff", 
"nameInfo": {"baseNoun": "staff:staves", "flavorTypes": "Woods"},
"display": {"ascii": {"color": "flavor", "symbol": "_"}}, 
"stats": {"weight": 5.0}, 
"flags": ["EASY_KNOW", "FLAVORED"]},

Aside: I had been thinking of flavor only affecting the description, even though I was noting that it affected the display color. The above reminds me that perhaps setting the flavor inside the naming trigger proc may not be the best choice, since it's a little more difficult to access. Plus, the proc isn't directly aware of the item it's attached to (unless we pass in the item in the params), so it can't return that value to the item. The item would need to check its naming procs to figure out its flavor before it could be used by the UI color system. Is there a better approach to that?


From what I can see, the template name always matches the type name, aside from pluralisation decoration. In that sense, there is no difference between a template and a type.

So instead of the redundant values, can just have 'type' and 'templates', where templates refers to the type the current object data is based on.


current:

{"type": "prayer book", "templateName": "prayer book", 
"display": {"ascii": {"color": [0, 255, 0], "symbol": "?"}}, 
"stats": {"cost": 100, "weight": 3.0}, 
"flags": ["EASY_KNOW"]},

{"index": 464, "subtype": "[Beginners Handbook]", 
"templates": "prayer book", 
"allocatorRules": [{"commonness": 50, "minDepth": 5, "maxDepth": -1}]},

revised?:

{"type": "book",
"display": {"ascii": {"symbol": "?"}}, 
"stats": {"cost": 100, "weight": 3.0}, 
"flags": ["EASY_KNOW"]},


{"type": "prayer book",
"templates": "book", 
"nameInfo": {"baseNoun": "Prayer Book~"},
"display": {"ascii": {"color": [0, 255, 0]}}},


{"index": 464,
"templates": "prayer book",
"nameInfo": {"properName": "[Beginners Handbook]"},
"allocatorRules": [{"commonness": 50, "minDepth": 5, "maxDepth": -1}]},

Which pushes us into categories. Either "book" or "prayer book" could be considered a category. However moving them to a general list or set like that removes some of their semantic value. For example, how do you apply the proper template? You'd have to do a search of every category list, which seems troublesome.

"type" and "templates" are clearly for mapping one object record to another. Although.. I went with "type" rather than "templateName" since type is used in various places to reference things. However the purpose of type in those other areas lends itself better to categories, while what I have there is more for templates. So....

revised #2:

{"templateName": "book",
"categories": ["book"],
"display": {"ascii": {"symbol": "?"}}, 
"stats": {"cost": 100, "weight": 3.0}, 
"flags": ["EASY_KNOW"]},


{"templateName": "prayer book",
"templates": "book", 
"categories": ["prayer book"],
"nameInfo": {"baseNoun": "Prayer Book~"},
"display": {"ascii": {"color": [0, 255, 0]}}},


{"index": 464,
"templates": "prayer book",
"categories": ["book1"],
"nameInfo": {"properName": "[Beginner's Handbook]"},
"allocatorRules": [{"commonness": 50, "minDepth": 5, "maxDepth": -1}]},

"templateName" and "templates" are specifically for pointing between records. "categories" is for describing the object in question. "type" (and "subtype") is removed entirely, but I also end up with more duplication to provide other info (ie: category "book" for handling burnable or whatever else affects all books, category "prayer book" for determining spell set, and category "book1" for specific spell selections (though I'm not sure on how that aspect needs to be handled, so it could change).)

I wanted to get rid of needing two separate fields with the same info, but all I did was move it around. Perhaps it would instead be possible to automatically add the templateName to the categories? But no, that ties you into knowing the structure of the dat files.

[Kinematics]

Ok, two different systems need to access the flavor -- text output and GUI graphics. So, you can't tie it solely to a proc, since it's not easy to access. The easiest common access point both systems have is still just the item itself, so can leave things as they are.

Two separate procs for normal vs unique is still fine, though, since it removes the dependency of definite article determination from the proper name.


Should nameInfo be stored on the item, or only passed in as params for the proc and then discarded?


Categories should be sets, not lists. Duplicate values are meaningless, and a set can be searched faster as well.

record.applyValues needs to be updated to account for sets as well.

Rewrote the templates file with the above revisions in mind: http://pastebin.com/XQnD6Uf8


However.. applying the proc is actually a little tricky. If I set up a base template that sets the default name proc that's applied to all items, and then set another template that applies a proc for unique items, won't both of those procs end up on the item? Then there's no certainty about which one will fire at any given time.

The unique name proc can't overwrite the default one. Its name defines which proc to load, which means it must necessarily have a different name than the default proc in order to load the correct one, which means it can't overwrite the default proc, which means both procs end up on all artifacts.

Setting up a template that simply applies a "unique" category to items seems much more sensible. Have to access that info in the naming code. However that means we're once again down to a single naming proc.

An alternative, with respect to the dat files, is to not apply any naming proc at all; let the default one get applied internally, and only provide a proc in the dats if you want to override the default. That's still a little clumsy, though, in determining whether to use the default or not. Probably a better approach, though.




Current fields of item (including some that I've added during this patch) that may change:

        ## Broad typing for the object (e.g. potion, polearm, missile)
        self.type = None
        ## Decorated form of the main type, that includes plural indicators
        self.decoratedType = None
        ## Base name of the object
        self.subtype = None
        ## Decorated form of the subtype, that includes plural indicators
        self.decoratedSubtype = None
        # cache description values
        self.cachedDescription = dict()
        ## Flavor of the object, if applicable.
        self.flavor = None
        ## Display metadata
        self.display = dict()
        ## List of procs we can trigger.
        self.procs = []
        ## List of affixes on the item (will store name, type and level).
        self.affixes = []
        ## The item's theme name
        self.theme = None
        ## Amount of the item.
        self.quantity = None
        ## Number of charges on the item.
        self.charges = None
        ## Stats instance describing our abilities.
        self.stats = things.stats.Stats()
        ## List of equip slots we can be wielded to.
        self.equipSlots = []
        ## For items that are containers, whether or not the item's contents
        # are on display.
        self.isContainerOpen = True
        ## Prose description
        self.description = ''
        ## Whether to use a definite or indefinite article
        self.useDefiniteArticle = False
        ## Prefix to attach to the name, like "Heavy", "Spiked", "Holy", "Green Dragon Scale" or "Dwarven".
        ## Used for quality/material/make affixes and themes.
        self.namePrefix = None
        ## Suffix to attach to the name, like "of Slay Evil" or "of Cure Light Wounds"
        self.nameSuffix = None

A number of those can be removed. Reduced form:

        ## Display metadata
        self.display = dict()
        ## List of procs we can trigger.
        self.procs = []
        ## List of affixes on the item (will store name, type and level).
        self.affixes = []
        ## Stats instance describing our abilities.
        self.stats = things.stats.Stats()
        ## Categories that describe the item
        self.categories = set()
        ## List of equip slots we can be wielded to.
        self.equipSlots = []
        ## Amount of the item.
        self.quantity = None
        ## Number of charges on the item.
        self.charges = None
        ## For items that are containers, whether or not the item's contents
        # are on display.
        self.isContainerOpen = True
        ## Flavor of the object, if applicable.
        self.flavor = None
        ## The item's theme name/position
        self.theme = None
        ## Prose description
        self.description = ''

        ## Subtyping for the object (ring of -strength-, potion of -cure light wounds-, etc)
        self.subtype = None

        # cache description values (maybe?)
        self.cachedDescription = dict()

Still not sure what to do with subtype.

[Kinematics]

Re: default/unique procs:
Just realized I'd actually written it slightly differently in my earlier notes, but changed it based on finding a proc entry in the object templates.

My first thought was to put the naming proc in the nameInfo list.

"nameInfo": "proc": "default name", "baseNoun": "ring~"
"nameInfo": "proc": "unique name", "baseNoun": "One Ring"

In that case, the "proc" entry can be overwritten by later templates, and only the final one needs to be created when the item is made. That gets rid of the duplicate proc issue and simplifies the overall handling of it.



Re: subtype, perhaps the addition of the following:

"tempaltes": ["ring"]
"affixes": ["Charisma"]
"categories": ["Charisma"]

"tempaltes": ["ring"]
"affixes": ["Strength"]
"categories": ["Strength"]

Basically, the template can specify a specific affix (or affixes) to apply to any given item, by name. This affix can be applied whenever the factory makes a new instance of the item. If that affix is also kept in the item's categories, when you list affixes you can pull any that match the categories and put them first.

Affixes already exist as specifiable in themes, so adding them to items shouldn't be hard. Don't need all the weighting details, though, since they'd be guaranteed.

Edit: though that only works for rings and amulets; still need to deal with potions, scrolls, etc.

[Kinematics]

So, can convert ring/amulet subtypes to affixes. Potions/scrolls/wands/rods/staves can have their subtype converted to Effects. On use, the system can check that value to see what effect should be generated.

That should finish the separation of the overloaded item fields (mainly type and subtype) into fields that each have very specific meanings. Also clears up the handling of naming. Effects can be shown if the item doesn't have a name, and if there isn't a name or effect, show affixes.

Revision N: effect can be a field in nameInfo, since the effect is triggered via proc, and thus doesn't need to handle the effect indirectly.

[Kinematics]

Gotten almost all the changes in, but finding that subtype is still being a pain. For example, Amulet of Adornment, the subtype has no bearing on any sort of affix or effect. An Amulet of Devotion has a number of mods, rather than an affix, per se.

So, need more thought given in how subtype can be better handled.

[Kinematics]

Scrolls:
Scrolls have a flavor of 'titled "xxx yyy"'. I suppose books could potentially be considered the same, but that's more a necessary distinction than a flavor.

In game now, I have 'a Scroll titled "nin xuxucre" of Light'
Might be better as: 'a Scroll of Light titled "nin xuxucre"'?

Basically, current structure puts the flavor in the first suffix spot, before the item's effect name. It seems more natural to have it after the effect name, though I can also see the appeal of focusing on the title before the effect. If the title is first, it implies a given title could have multiple effects; if the title is last, it implies a given effect could have multiple titles.

Regardless, though, the title doesn't go in the same spot as other object flavors, which were basically just materials, and went in the same spot as material affixes.

We can also speculate on flavors of similar type: inscribed "xx", named "xx", painted "xx", etched "xx", etc. All of them have the same basic property: a postfix position.

The random characters on scrolls don't quite hold the same intuitive meaning, so let's try an imaginary variant.

Suppose we add a new item called a "whompstick", and have it flavored with names. A Whompstick named "Bob", a Whompstick named "George", a Whompstick named "Eddie", etc. Suppose one of these whompsticks had an effect of *Destruction*. We could then have either a Whompstick named "Eddie" of *Destruction*, or a Whompstick of *Destruction* named "Eddie".

Again, it really seems better to have the flavor text as the last item in the suffix list.

So, adding another (optional) nameInfo field for flavorPosition (prefix, suffix). If none is listed, it will default to prefix.

[Derakon]

I just wanted to pop in and say that while I haven't been exactly keeping up with all of the posts you've been making, I do appreciate all the thought you're putting into this! Feel free to keep making the posts -- I know how helpful it can be to have a place where you "think out loud", and every once in awhile someone may have some useful insight.

[Kinematics]

More on subtype.

Amulet of Adornment -- does nothing
Amulet of Charisma -- adds mods
Ring of Resist Cold -- adds resist cold flag
Ring of Extra Attacks -- adds extra stats
Ring of Acid -- adds flags, stats, and mods
Potion of Healing -- probably should add a proc for effect

Essentially, there's a ton of possible aspects of the item that the subtype might refer to, but the subtype itself is purely descriptive. There's also no single 'thing' that the subtype refers to; it's just the name of the item, essentially. One might even consider it a sort of 'flavor', except that it relates specifically to what the ring does.

So, proceding from there, the subtype descriptor does not have to be associated with an affix, nor should it be expected that an affix will be created for it. In fact, an affix will almost certainly -not- be created for it (at least, not a named one), as all its workings are defined in the object creation.

That means reverting the expectation of a category value to use as the first suffix. Instead, something like a 'descriptive' value should be added to nameInfo.

I had previously added an itemEffect value, though I'd expected to pull it from some sort of 'effect' attribute. As that really wouldn't be the case (and in some cases there's no effect at all), it can absorb the idea of the 'descriptive' value. The only real issue would then be giving it a proper name.

In a sense, 'subtype' does describe these things well -- they're subtypes of rings, potions, scrolls, etc. Various words that could be used:

type
subtype
descriptive
characteristic
class
particular
distinct
form

'type' would refer to the type of the specific template/category. IE: the type of ring, not the type of object. It really does seem to be the most succinct term to use.

However, what identifies a 'type'? For rings and scrolls, it fits easily -- a Scroll of Light, a Ring of Searching. However it could also apply to non-flavored items -- Boots of Levitation, Gloves of Free Action, etc.

However there's a somewhat distinct difference in the intent -- all rings must have a type, but most boots won't have a type. And the ones above aren't defined objects, but rather equipment with affixes, anyway. However the basic premise -- that you can define a very specific type of a given item and put it in the object lists -- that seems reasonable.


However that still leaves us with referencing a given item. Currently, item factories are hashed by (type, subtype). Type is being removed, putting it instead in the item categories. Subtype being removed and placed as nameInfo['type'] leaves us with no direct fields that we can reference to hash to.

To make it hashable, it has to be an immutable type -- string, tuple, number, etc -- and can't be a list, set or dictionary. Since these new groupings are dictionaries and sets, we can't use them directly, and have to extract the values out again.

We also want them to be relatively easy to reference. A chain mail being in categories: (armor, body armor, hard armor, metal armor, chain mail) may be useful for determining how other things interact with it, but it would be a pain to have to define it that way just to create a new one. And what is the minimum necessary to be sure we get exactly that item?

While this pretty much mimics the existing structure, I think we can work with:

properName -- if it exists
(templates, type) -- if type exists
(templates) -- if templates exists
(templateName) -- if no templates exist

Of course templateName should only ever exist on objects defined in object_templates.txt (and that's the only place the program looks for them). Everything else must necessarily have at least one template to draw from, and only that primary template (as opposed to ancestor templates) is needed to narrow down the object of interest.

Type as an optional second field should be sufficient to narrow down objects with multiple types.

And of course any named object should have a unique name.

That lends itself to using a function in the itemFactory to get the name of the factory, rather than explicitly pulling factory.type and factory.subtype.


~~~~~

Derakon -- With lack of feedback, I'm just pushing forward on whatever ideas I come up with. I'm trying to keep things clean, and in line with earlier notes and comments, but I'm not always sure some of the changes I make are how you'd want to handle things. Still, I hope it'll work out.

Also, with apologies, this is becoming a massive patch. Granted some of that is just a full rewrite of the data files, but verification is still going to be a lot of work. Spiraled a bit out of control from just wanting to pluralize nouns properly.

[Kinematics]

Another major update to the pull request.

Cumulative changes:

Type/subtype have been completely subsumed in the naming system, but are still used in other systems. Transition in progress.

An item's naming details are specified in the nameInfo structure, including: modifier noun, primary noun, flavor types, flavor position, 'type' of item, item name (eg: artifact), and 'proc' -- the string that maps to the type of proc to use when trying to get the item's name.

The nameInfo struct (and the modifierNoun and baseNoun entries, in particular) is the -only- place where plural-decorated strings belong. Plural decorations should not be needed in any reference to an object. Ongoing cleanup.

A proc is created per item factory that gets attached to any items created. There are currently two procs: one for normal items, and one for unique items.

The proc code handles determining what bits belong where when describing the item, including dealing with 'type's, flavors, affixes, themes and names.

The grammar code handles assembling all the parts of the name together, with commas, 'of's, 'and's, articles, etc. All(?) such extraneous text has been removed from the object files.

The grammar code also has a subtantial unit test written for it.

object.txt, object_templates.txt, and artifact.txt have all been updated to match the new revisions, though they still retain type/subtype for now.

Templates can be based on other templates (to any depth). This allows a little extra flexibility in item construction, though I doubt it will be used much. Its primary value is neatly accumulating categories that an item belongs to.

Misc code tweaks throughout to make sure everything works nicely together.



Still to be done:

Clean up flavor-generating code, particularly for procedurally-created flavors.

Clean up affix/theme creation code to be sure no extra work is done trying to name items.

Find all uses of type/subtype and adjust as needed.

Remove obsolete class fields, primarily from item and itemFactory.

Verify and decide on effect of capitalization within object data files.

Outside review of the changes to see how well they fit in with expectations and intent.


Consider unit tests for other aspects of the code.

Major future work: handling of player knowledge, and how that affects what's displayed.

[Kinematics]

Type/Subtype are still used in the creature code. Is its purpose there sufficient to leave it as is?

The factory name needs to be easily identified, in order to provide a lookup mechanism that can be used elsewhere (eg: when generating loot drops).

item.categories (combined with parent template) is too burdensome for creating a unique name. Rather than (hard armor, chain mail) if using type/subtype, you'd have (hard armor + hard armor, body armor, armor, chain mail), plus any others that may get added.

Can it be done using sets instead? Except you can't keep a mutable object in a set, so working from that direction doesn't work either.

In what cases is this used?

1) Basic init/genTown for debugging makes items with a direct call that requires knowing the exact factory hash. Not a standard case.
2) Wizard Mode, same thing.
3) The only call that explicitly gets a specific factory is when creating an artifact. Artifact identification (and thus hashing) is trivial, depending only on the artifact's name.
4) getTypes, getSubTypes and getArtifacts are only accessed through Wizard Mode. (getArtifacts needs to be rewritten, as well)
5) itemAllocator does a loop through all the items in the ITEM_FACTORY_MAP, so knowing specific hashes doesn't matter.

It seems that having a simple, predictable hash is only a requirement for debugging and wizard mode. Otherwise a simple combination of categories and type is sufficient to make a unique key. Unfortunately that's not predictable across sessions, since the categories aren't provided in any particular order (though they could be forced alphabetical).


After some thought, we could leave the subtype field, and treat it as such: the subtype is the category added by the current object instance.

The hash for an object would thus be: templates, subtype, nameInfo['type']

This makes it reasonable to use in Wizard/Debug mode, but has minimal effect on the main play code.

[Kinematics]

More subtype revision:

Since I'd decided that subtype can logically be considered the same as the category added at the object's level, I can actually set the value based on the category being there, rather than duplicating values in the object.txt file.

So, if the subtype is not specified in the object's record (not including ancestor templates), and a category is provided, the subtype is assigned the first category value.

Since the factory name already accounts for nameInfo['type'], I won't bother duplicating that.

This will generate subtypes on templates as well, but since templates are ID'd by the template name, it will have no effect.

So next level of revision is elimination of explicit subtypes, and only using categories.