Idea: YAMLband

[Therem Harth]

Monster, item, spell, etc. properties in YAML instead of the current custom format. More verbosity, better readability, and possibly more detailed specs. Each monster/item/whatever would be a separate "document" (within the same file as related ones).

Example:

---
name: Ancient Red Dragon
level: 41
rarity: 1
speed: 10
hitpoints: 880
armor: 90
experience: 2500
sight: 200
description: >
  A huge draconic form. Wisps of smoke steam from its nostrils and the extreme heat
  surrounding it makes you gasp for breath. 

flags: [evil, dragon, bashDoor, pushMonster, spawnAsleep]
resists: [fire, light, rockRemover, sleep, confusion]
breaths:
  - effect: fire
    damage: 300
    frequency: 0.2
spells:
  - effect: blind
    frequency: 0.1
  - effect: scare
    frequency: 0.1
  - effect: confuse
    frequency: 0.2
attacks:
  - type: claw
    damage: 4d9
  - type: claw
    damage: 4d9
  - type: bite
    damage: 7d9
    effect: fire

IMO having a better info format would go a long, long way towards making the game more hackable. The way things currently are, most of my Angband related projects have hit a brick wall of "Ugh, I don't want to edit any more info files."

... And yes, for the record I am volunteering my time for this, if anyone thinks it's a good idea.

Actually, I'm currently free on weekends, and itching for a project; adding a libyaml dependency to Angband, and moving stuff from C and text files into YAML, seems like a good possibility.

What do you people think? Any advice on what code base I should be using as a basis, etc. before I take the plunge?

[Derakon]

You might want to look at Pyrel, which uses JSON as its record storage system. For example, here's creature.txt. I'm not saying that you should necessarily use JSON instead of YAML (I went with JSON pretty much simply because Python has a built-in JSON library), just that it's bound to be a similar conversion process.

My initial approach involved writing small Python scripts to parse the info files and rewrite them in the new format. That got me bootstrapped; however, invariably I had to make file-wide tweaks later (e.g. when I converted monster resistances from a binary flag to a percentile stat modifier). I did this by having each file loader include a "dump this file to disk" method, which I could call after doing some modification to the in-memory version of the file. In pseudocode:

monster info = load("creature.txt");
for record in monster info:
    modify record;
write(monster info, "creature_modified.txt");

EDIT: while I support the idea of making the monster spellcasting behaviors more detailed, I suggest your first pass focus solely on replicating the existing data within the new format, and that you save tweaks and changes to game logic for later.

[Nick]

So here's the same monster as currently in the restruct branch:

name:444:Ancient red dragon
base:ancient dragon
color:r
info:120:880:20:108:80
power:41:1:1593840:226:2500
blow:CLAW:HURT:4d9
blow:CLAW:HURT:4d9
blow:BITE:FIRE:7d9
flags:HAS_LIGHT
flags:BASH_DOOR
flags:IM_FIRE
spell-freq:6
spells:BLIND | CONF | SCARE
spells:BR_FIRE
desc:A huge draconic form. Wisps of smoke steam from its nostrils and the
desc: extreme heat surrounding it makes you gasp for breath.

In terms of readability, the main difference is that yours explodes the info and power lines into individual speed, hitpoints, etc; I think we're heading in the same direction there.

I would be hesitant about adding another dependency unless there's a very good reason. Many, many of the bugs and difficulties we come up against are related to current dependencies (I've always thought the curses library is very well named). Every new dependency means checking it works for every platform we currently want to build on, or may want to in the future. Moreover, we currently have a very nicely written (ie not by me) parser which can handle everything we throw at it.

One of the big reasons for the current restructure is to make the game more hackable, so again we're going in the same direction.

If you're looking for a project, we've got a fairly big one going on right now. Feel free to have a look around the current state of things - any help and suggestions are welcome

[AnonymousHero]

Just to second Nick (I think?): YAML sounds great on the surface, but it is in fact hideously complex when you get into the nitty-gritty of it because of layout and encoding rules (IIRC). As it turns out YAML just appears great initially because us humans don't tend to think of all the ways parsing can go wrong because we think whitespace is just fab. Nowadays, every time I'm forced to write YAML I cringe because I can never be sure exactly what the computer is going to think of my... scribblings. And it scares me.

[Therem Harth]

... And as it happens I've been looking into Ansible for configuration management at work. Guess what Ansible uses for manifest files?

Puts a whole new spin on things, that does.

Thanks.

[AnonymousHero]

... And as it happens I've been looking into Ansible for configuration management at work. Guess what Ansible uses for manifest files?

Puts a whole new spin on things, that does.

Thanks.

Heh! We're using Ansible too. Of course there are a myriad ways you can break things using Ansible that have nothing to do with syntax, so I'm always careful to review diffs extremely carefully before running changed playbooks

[Therem Harth]

Awesome. I thought part of the point of having declarative languages for config management was making it harder to break things.

(Right now we're actually using shell scripts and SSH, like in the olden days. I think it's possible to do this in a sensible way, but we definitely aren't.)

[AnonymousHero]

Awesome. I thought part of the point of having declarative languages for config management was making it harder to break things.

(Right now we're actually using shell scripts and SSH, like in the olden days. I think it's possible to do this in a sensible way, but we definitely aren't.)

Don't get me wrong, Ansible is way better than manual configuration management. The "danger" part just comes with the territory once you start to automate stuff like updating machines, installing/uninstalling packages, etc. Rolling out configuration changes automatically just means it's hard to notice if you've done anything stupid before the change hits every machine . (Ansible can do "partial" rollouts to make this kind of thing safer, but what you only have a handful of non-identical machines, it's not really possible to do partial rollouts sensibly.)

EDIT: ...aaaaanyway, this is probably getting too far off-topic.

[Therem Harth]

Ah, gotcha. I was thinking more of provisioning and initial configuration, rather than maintaining a baseline.

No problem re going OT, configuring servers is fun.

Edit: anyway what other formats do you think would be more... robust than YAML in terms of parsing, with decent parsing libraries available?

[Therem Harth]

I was thinking that XML with extensive use of attributes would also be acceptable (as opposed to just tags, which would be far far too verbose). e.g.

<monster name="Ancient Red Dragon"
  level="41"
  rarity="1"
  speed="10"
  hitpoints="880"
  armor="90"
  experience="2500"
  sight="200">
  <description>
A huge draconic form. Wisps of smoke steam from its nostrils and the extreme heat surrounding it
makes you gasp for breath. 
  </description>
  <flags>evil dragon bash-door push-monster spawn-asleep</flags>
  <resists>fire light rock-remover confusion sleep</resists>
  <spells frequency="0.16">
    blind
    scare
    confuse
    breath-fire
  </spells>
  <attacks>
    <claw damage="4d9"/>
    <claw damage="4d9"/>
    <bite damage="7d9" effect="burn"/>
  </attacks>
</monster>

[debo]

EDIT: while I support the idea of making the monster spellcasting behaviors more detailed, I suggest your first pass focus solely on replicating the existing data within the new format, and that you save tweaks and changes to game logic for later.

I'm sure we don't have any maintainers in attendance who would be tempted to make gameplay tweaks while undergoing major restructures....

[Derakon]

I was thinking that XML with extensive use of attributes would also be acceptable (as opposed to just tags, which would be far far too verbose).

Honestly, if you're going to go down that route, then I would recommend using JSON instead. In exchange for every key having to be a string (and thus enclosed in quotation marks), every value can be considerably more polymorphic, and in particular you can have numbers in the attributes instead of having to enclose each number in a tag.

XML is the Java of markup languages -- it can get the job done, but it's painfully verbose and there's almost certainly a better option that's aimed more specifically at your niche. And I'm saying this as someone whose job currently involves a lot of Java development.

[Therem Harth]

Hmm, thanks. I'm not a programmer by profession so I don't know this stuff very well.

JSON looks readable enough... The use of both brackets and braces is a bit confusing, not too bad though.

There seem to be quite a lot of JSON libraries for C. AH used Jansson for ToME 2... I'm thinking I could do the same, but might want to embed it.

Re Nick's new format, something CSV-ish like that would also be good. I feel like it would be nice to remove monster indexes entirely though, and use the name string as the unique reference.

(IMO if you have multiple instances of the same monster name, that's a design problem!)

[Tarrant]

In the context of Angband design, that's probably true. I think that monster index number is an example of general good practice programming, though. Ask anybody who has ever done database work- do you REALLY want the person's name to be the unique identifier, or are you worried that another Ben Jones is going to come along and overwrite the entry accidentally?

Having a numerical unique index also makes selecting a random entry less complicated, although I haven't done enough Angband code-diving to know if they take advantage of that.

[takkaria]

Re Nick's new format, something CSV-ish like that would also be good. I feel like it would be nice to remove monster indexes entirely though, and use the name string as the unique reference.

(IMO if you have multiple instances of the same monster name, that's a design problem!)

I think we're getting close to that, actually. The only reason there is an index still is because we need something to save in the savefile, and saving a string for each monster is wasteful. (Or at least, that's what my 1990s sense of storage capacity is telling me...)

The real advantage of the current format is its compactness. It's nice to be able to fit multiple monster entries on the screen at once.