4E Item Parser

[primarch]

If your playing 4E I built in a tag (mm3mode) for NPC powers which instructs the ruleset how to parse the Powers of a NPC's stat block.

If a power has the mm3mode tag and its set to 1; the ruleset will parse as per MM3 monster formats( i.e. Traits, Hit:/Attack:/Effect: lines etc. etc.), if no tag is present the ruleset will treat the power stat block as per MM1/MM2 format.

Hi!

DrZeuss, what utility contains this tag you built? Was it an addition to the Tenian parser?

On a side question the new 5e parser will it parse 4e data (scrapping as well) or will someone have to step up and add that?

Thanks!

Primarch

[Trenloe]

Also, the item parser has only been downloaded around 50 times, so there does not seem to be that much demand for it.

50 times! I think that's pretty good.

[Zeus]

Hi!

DrZeuss, what utility contains this tag you built? Was it an addition to the Tenian parser?

On a side question the new 5e parser will it parse 4e data (scrapping as well) or will someone have to step up and add that?

Thanks!

Primarch

Its not a utility, it was an enhancement I delivered originally via the 4E Enhanced Personalities extension. Moon_wizard then incorporated the extension into the base 4E and 3.5E FGII 2.8.x rulesets along with Enhanced Items, Partysheet, Calendar and Tables etc. etc.

As for PAR5E, yes it will be able to support 4E parsing (in fact it will support ANY rpg system) but first someone will have to create the appropriate library for the target FGII ruleset and routines for handling the rpg system specific data content and format. PAR5E will initially release with a 5E library and an example template library. As for scraping, if Wizards release an online Compendium for 5E, I will look to add support to PAR5E at a future date.

Once PAR5E and 5E are finished and released and if time is available (and there's strong demand for it) I'll see if I can release a library for 4E.

However, I have to ask, why would you want 4E to be supported by PAR5E? The existing 4EParser and Valeros's tool both seem to cope well enough with 4E content. The issues relating to embedded HTML tags is an impossible situation for the developers of these tools to permanently fix. The issue lies with Wizards and their haphazard approach to entering data into the Compendium with consistency. For a company that relies on the quality of the information published through the various mediums they employ, they do a very poor job of managing the content. I mean who stores data in HTML, seriously? Whoever made that decision at Wizards should be shot.

If I were in charge, I would have insisted that the data be stored as XML and then employ XSL/CSS to reformat to HTML for presentation.

[Zeus]

50 times! I think that's pretty good.

Also, remember its typically only DMs that will use the Parsers, so 50 x 4 average players per group, suddenly thats 200 users benefiting ...

Keep up the effort Valeros. In fact, would you be interested in developing the 4E library for PAR5E? I can make the PAR5E source script and modules (Perl) as well as the current WiP 5E library I am developing available as a reference. I would do it my myself but I have a list of projects as long as my arm not to mention the new 5E ruleset, so it would have to wait a bit before I could focus on it.

[Zeus]

Valeros - Another thought.

The 4EParser by Tenian appears to be no longer under active development. However functionally it still works well with current 4E data (aside from Item scraping which is slightly broken).

Instead of developing and supporting a tool that scrapes the items and then produces FGII XML, have you considered outputting the data to flat file text format.

If you use the format used by the 4EParser, the 4EParser will take care of producing the XML. The parsing functions still work, its just the scraping functions that have been impacted by DDI changes.

Benefits of approach.

- you don't have to worry about maintaining a tool to output FGII 4E specific XML
- you will be able to leverage the Item Tables (for example) with figuring out the XML structure
- users can easily produce 4E modules with all content inclusive

Just a thought. If on the other hand you want to add Table support to your parser, here's a breakdown of the XML structure you require.

- 4E uses the following nodes under the modules root XML to hold data:

   <reference4EAA>         - where 4EAA is the moduleid for the module 
        <weapondata>       - listnode containing nodes of all weapon DATA i.e stats
              <weapon1>    - where weapon1 is the name of a weapon
              ...
        </weapondata> 
        <armordata>        - listnode containing nodes of all armor DATA i.e stats
              <armor1>     - where armor1 is the name of an armor
              ...
        </armordata>    
        <equipmentdata>    - listnode containing nodes of all equipment DATA i.e stats
              <equipment1> - where equipment is the name of a weapon
              ...
        </equipmentdata>
   </reference4EAA>

   The second root node holds the data for the window list (or Table):

   <equipmentlists4eAV>
	<core4eAV>
	   <description type="string">Equipment Table</description>
	   <groups>
	     <section001>
		<description type="string">Item Category/description>
		<subdescription type="string">Item SubCategory</subdescription>
		<equipments>
		   <equipment1>
		     <link type="windowreference">
		       <class>referenceequipment</class>
		       <recordname>reference4eAV.equipment.equipment1@4e Adventurers Vault</recordname>
		     </link>
		    <name type="string">Equipment Item 1</name>
		    <weight type="number">0</weight>
		    <cost type="number">0</cost>
	          </equipment1>
              </section001>
              <section002>
              ...

[primarch]

50 times! I think that's pretty good.

Hi!

I agree. A community that represents a niche within a niche (as in people whom play RPG's AND play them online), 50 is pretty good.

I for one have greatly enjoyed Valeros contribution and would hope he will be enthusiastic in creating more things such as his item parser.

Primarch

[primarch]

Valeros - Another thought.

The 4EParser by Tenian appears to be no longer under active development. However functionally it still works well with current 4E data (aside from Item scraping which is slightly broken).

Instead of developing and supporting a tool that scrapes the items and then produces FGII XML, have you considered outputting the data to flat file text format.

If you use the format used by the 4EParser, the 4EParser will take care of producing the XML. The parsing functions still work, its just the scraping functions that have been impacted by DDI changes.

Benefits of approach.

- you don't have to worry about maintaining a tool to output FGII 4E specific XML
- you will be able to leverage the Item Tables (for example) with figuring out the XML structure
- users can easily produce 4E modules with all content inclusive

Just a thought. If on the other hand you want to add Table support to your parser, here's a breakdown of the XML structure you require.

- 4E uses the following nodes under the modules root XML to hold data:

   <reference4EAA>         - where 4EAA is the moduleid for the module 
        <weapondata>       - listnode containing nodes of all weapon DATA i.e stats
              <weapon1>    - where weapon1 is the name of a weapon
              ...
        </weapondata> 
        <armordata>        - listnode containing nodes of all armor DATA i.e stats
              <armor1>     - where armor1 is the name of an armor
              ...
        </armordata>    
        <equipmentdata>    - listnode containing nodes of all equipment DATA i.e stats
              <equipment1> - where equipment is the name of a weapon
              ...
        </equipmentdata>
   </reference4EAA>

   The second root node holds the data for the window list (or Table):

   <equipmentlists4eAV>
	<core4eAV>
	   <description type="string">Equipment Table</description>
	   <groups>
	     <section001>
		<description type="string">Item Category/description>
		<subdescription type="string">Item SubCategory</subdescription>
		<equipments>
		   <equipment1>
		     <link type="windowreference">
		       <class>referenceequipment</class>
		       <recordname>reference4eAV.equipment.equipment1@4e Adventurers Vault</recordname>
		     </link>
		    <name type="string">Equipment Item 1</name>
		    <weight type="number">0</weight>
		    <cost type="number">0</cost>
	          </equipment1>
              </section001>
              <section002>
              ...

Hi!

No particular reason DrZeuss. Perhaps having it all in one program instead of two.

Primarch

[valeros]

Wow, a lot to which to reply.

1) I realized after my post that I believe Tenian’s parser could put all monsters in one module. Since the parser can take a batchfile, if someone developed the batch file that appends all the different sources and only parses NPCs, then that should work. Probably someone did that already and I am just unaware of it.

2) I will look into trying to parse the “old style”monsters as life permits. But do people still really use the old monsters? And if so, don’t you have to upgrade them manually? Is being able to get just the "new" monsters good enough?

3) I did consider having mine output in a form thatTenian’s could use. In hindsight, maybe I should have done that originally. But since Tenian’s still uses tags on his output, I would still have to parse, just not as much. Also, I originally made mine specifically to make it easier to uset han Tenian’s (not saying I succeeded), so using mine followed by using his would be even worse. (No bash intended. It is a great tool. Just not super easy to use at first and especially if you want ALL the entries for a particular category.) Also, I have already created the code to output to FG, so it does not save me that.

4) As far as the XML portion:
a. Although my XML output for items is not identical to what you showed (and thank you very much for providing that!), I believe I do use the enhanced tables for equipment, armor, weapons, and magic items. Griogre helped me with that.
b. As far as combining with other modules, I intentionally make all the data (descriptions & indexes) under one XML element so that using a editor like Notepad++ you can copy that one element into an existing module. Then you copy the library entry as well and you have the data in whatever module you wanted. Not automated but fairly easy. Griogre even made a videoon how to do it.
c. Good to know about the mm3mode tag. I figured it was to distinguish “new style” from “old style”, but I was unsure for what it actually was used by FG. But it does not change the damage output to the new standards, correct?
d. I agree 100% about storing the data in HTML. So many exceptions in parsing. What a pain.

5) One thing I did not mention when referring to the demand is my own personal assumption that 50 downloads translates to about five people who actually use it. Also I did not mention the group with which I play is moving to Pathfinder and I will not be the DM. Both affect my motivation since I will not personally benefit. I did not mean anything negative by that comment. Decent point about it being a DM tool, so it benefits more than the people that use it. So again, I will look into parsing old style monsters as time permits.

6) And just to shamelessly plug myself to primarch (per "creating more things such as his item parser"), I created other tools for FG. The main ones:

a. A tool that downloads images of miniatures as tokens (now with optional border!) https://www.fantasygrounds.com/forum...ad.php?t=17628

b. A tool that converts monster files from the Monster Builder tools (the old one and the new Silverlight one.) Not perfect, but gets most info. https://www.fantasygrounds.com/forums/showthread.php?t=17679

c. A tool that downloads Fortune Cards to go with the fortune card extension DrZuess created. At the end of this thread: https://www.fantasygrounds.com/forum...ad.php?t=15352

Thank you

[Griogre]

On the old 4E monsters, I use them. Practically speaking, the differences between the stats on old and new monsters are small enough to be meaningless up until high heroic tier - the exceptions are brutes starting about 8th level.

Its so fast to update old monster stats to MM3 ones, that for me it takes more time to figure out whether the stats are old or not. The double the mod number for all non brutes and triple the mod for brutes works very well after 6-7th level.

[valeros]

2013/03/03 Updated parser:
1) Added ability to parse monsters/creatures
2) Should now build the ".mod" file automatically
3) If output directory already exists, output overrides existing files instead of deleting entire directory
4) Default file now "db.xml" instead of "client.xml" (due to monsters)

If you parse a large numbers of monsters, keep the resulting entries text file (e.g. "Monsters 5170 Entries 2013_0303.txt".) This is in case there are issues with the output of the parser, you can use that file for input when the parser is patched instead of downloading all the monsters again.

Notes about monsters:
1) Had to use "db.xml" instead of "client.xml" because otherwise FG would not show saving throws & action points line. FG would also generate an error each time a monster record was opened.

2) If a monster has a saving throw like "+2, +4 while bloodied", FG does not seem to show the line correctly. So if you see a monster and the saving throws line seems to be cut-off, that is FG, not the parser. (Examples: Querelian or Sharn)

3) If you add a "tokens" directory with a subdirectory named the same as the module to the resulting directory, those should be automatically mapped to the monster. The token name must match exactly the name of the monster, be in lower case, be a PNG file, and end with ".png". So for example, if the module was named "Monsters", then the token for "Shuffling Zombie" must be "Monsters\tokens\Monsters\shuffling zombie.png"

4) If a monster has disease, description, or revisions data as part of the entry, that data is added to the "Other" tab.

5) "Published in" data is added to the "Other" tab.

6) Indication of whether "New Style Monster" or "Original Style Monster" added to the "Other" tab.

7) Where obvious I fixed a few monsters/powers (like adding missing keywords or aura size) and converted text to common wording (e.g., variations like "while bloodied", "only while bloodied", "usable only while bloodied", "only usable while bloodied" are all converted to the same text.)

8) Right now, monsters are only listed by name or level (so not not by group, level/role, or role/level like Tenian's). If parsing all monsters, the lists are so long you need to use the search function or collapse the sections since scrolling does not work well (at least for me.)

9) There are so many variations in the data I am sure I did not parse something correctly. Please let me know what is not parsed correctly.

Some stats just for fun:
1) Currently parses 5170 monsters of which 2135 are new style and 3035 old style monsters
2) Output XML file for FG with all 5170 monsters is 497,162 lines long
3) Parser takes my computer 34 seconds to parse all monsters
4) For the old style monsters, there are approx 12,318 powers with approx 1,848 unique combinations of just the Recharge and Action fields (e.g., "Standard; At-Will", "Minor; Recharge 6", etc.)
5) The file containing the raw monster data is 129,621 lines long