Awesome!
Thank you very much for this :)
Printable View
Awesome!
Thank you very much for this :)
absolutely brilliant! fast, easy to use and well crafted... worthy of all thanks and admiration!
Hi!
Kudos to you Valeros!!
I was wondering you ever thought adding functionality that would parse themes and related powers for use? Currently we can't do that. I think its the only thing left no one has tried.
Thanks for your excellent work!
Primarch
How would that work (themes that is.) There is no theme template. The description of the theme could work like backgrounds (just a formatted text in a reference text window.) But how would the powers be associated to the theme? I do not know a add a link for the associate powers to something like that.
Quote:
Originally Posted by valeros
You would need to write an extension which adds a new windowclass for 4E Themes. As you suggest you could use the Backgrounds class as a template and use the formattedtext field to link to associated powers.
You can use the <listlink> tag within formatted text fields to reference other objects in the module. e.g.
Where in the above example 4eAP and 4e Arcane Power are the module id and name of the module.Code:<listlink>
<link class="reference_classpowerlist" recordname="refpowerlists4eAP.bardpowers@4e Arcane Power">Bard Powers</link>
<link class="powerdesc" recordname="powerdesc4eAP.powerJinxShot@4e Arcane Power">Jinx Shot</link>
</listlink>
4E uses reference_classpowerlist and powerdesc as the windowclasses for Class Power Lists and Power sheets.
Hi!
Thank you DrZeuss for giving valeros that tidbit of code!
Valeros, as it stands, themes is the only thing missing to be able to have a complete scraped and parsed 4e database.
As of now, I basically use just your parser for monsters and items. Getting themes would be the proverbial cherry on top of an already great desert.
Heck, have you ever thought of making utilities for just downloading feat and powers like you did for monsters and items?
At least for me, having these "master" lists is proving to be leaps and bounds more convenient than separate in their own books.
In any even, I'd like to reiterate my thanks to DrZeuss and you for all the great stuff you've put out. Personally, I will be staying with 4e, so all these great tools are of immense long term value.
THANK YOU!
Primarch
Thank you for the code DrZeuss.
So I took a quick look at powers. There are a lot (9356 to be exact), but I glanced through some of them.
I got a little confused about how the XML would look for powers, so I used Tenian's parser to parse Arcane Power. I found some oddities looking at the XML output from Tenian's parser.
Example oddities:
1) Tenians parser outputs tags like special, hit, miss, effect, etc. which translate into the text with the tag name bolded at beginning of the line. (So "<effect type="string">Some effect text here</effect>" translates on screen to "Effect: Some effect text here" with the word "Effect" bolded.
2) Tenians parser outputs other special tags like firstfailedsavingthrow, hit_fortitude_, etc. that translate to headers other than the name of the tag. When I look if FG, they are translated into meaningful headers like "First Failed Saving Throw" or "Hit (Fortitude)"
3) sustain is another tag like the others but instead of the tag name being bolded, everything before the first ":" is bolded.
4) When trying to link a power, Tenian's parser output a "<linkedpowers>" tag as opposed to the "<linkedlist>" mentioned. It seemed to work.
5) I noticed what seems to me to be the same case to be parsed differently at different times. For example, Grease, Evard's Dreadful Mist, and Shard of the Endless Avalanche all parse the second power differently. Grease tries to link the power, Dreadful Mist has the second power text under the sustain line, and the second power of Endless Avalanche looks like a continuation of the main power.
Especially for the first four, do you know where this is documented or how was I supposed to know this?
Yes. The powerdesc window class defines the expected fields for a power. When Jamie first wrote the parser, powers only used formattedtext fields. At the time, access the data in a formattedtext field was programatically constrained (no getValue and setValue methods) and so over time Jaime added individual string fields in the module output to make accessing data a little easier from within the ruleset.Quote:
Originally Posted by valeros
Each power's flavour text is parsed (in its entirety) and added as the flavor tag using formatted text. In addition, the same text (minus mark-up tags) is added as a string flavour field (stringflavor). Similarly each power's descriptive text is parsed (in its entirety) and added as the shortdescription tag using formatted text. In addition, the same text (minus mark-up tags) is added as a string description field. Finally any special tags are also captured e.g. <effects> <sustain> etc. etc.
The formattedtext fields are used for displaying the power as its prettier. The string based tag data is used by the ruleset e.g. power list filtering and the keyword fields are used by the ruleset to manage the various format changes that can occur when dealing with class, item, feat powers etc. They are also used for parsing attack/damage/effect strings on the charsheet and CT.
Hmm, not sure. Probably a bug. I am guessing that the Parser traps lines containing a : character and bolds all text prior to the : assumming its a name. If a line contains multiple : characters I am guessing this breaks the capture he has setup.Quote:
Originally Posted by valeros
Ah yes, each power sheet contains a small subpower list under <linkedpowers> for handling ***GRANTED*** powers. I guess you could use this to store any linked power information however the association here would be to each power and not to a theme (which I thought was what you were asking for).Quote:
Originally Posted by valeros
Linking powers from within formattedtextfields will only work if you use the <linkedlist> tag I mentioned.
They all parse identically for me? Jaime's parser uses the ZPOWER tag to link to sub powers and attacks. I am guessing the input power text is not marked up correctly between all three powers, check to ensure you have the ZPOWER tag added.Quote:
Originally Posted by valeros
Aside from the ParserDocument.txt file which is included in the Parser folder which details the expected syntax for the various different content types, I am not aware of any other documentation. You could try PMing either Tenian or EugeneZ for more information.
Thank you very much for the info.
So I if understand:
1) The "description" tag/field is the one used to display the data in FG. (That makes sense and matches teh fields specified in "ref_power.xml".)
2) The other special tags/fields (like effect, firstfailedsavingthrow, etc.) were used by the 4E ruleset. Do you know if they are still used by the ruleset (and therefore I would need to include them in addition to the data being in the "description" field)?
3) Same thing with the "shortdescription" field. That only seems to be used by custom powers. Do you know if I would need to include that tag/field.
Feats seem to work like powers. That is, they have special tags/fields that are duplicate of the text in the "description" tag/field. Same question as above, do you know if those duplicate fields are still needed?
Thank you
2013/04/07 Added Feats & Powers
The ~374 powers that have an additional power in the compendium entry (e.g., Evard's Dreadful Mist) have the additional power currently listed as part of the main power as opposed to a second separate power.
Also, for powers, there is now a required file called "Power Classes.txt". It is used to categorize the powers based on the class of the power. It is in the form "Category: Class". You can change it to make categories more to your liking. If you make a better one, please post it.
Thank you
Holy crap! Thanks a million Valeros!
Hi!
I second the "holy crap"! Man, Valeros your stuff rocks the house! ;)
Does this mean the only thing left for you to add are the character themes? If you achieve that then you have truly done it all! Thumbs up Valeros! :)
Edit: As I parsed and went over the book, it seems you DID get the theme powers done! Is there anything left to do? Awesome!
Primarch
Oh wow.... nicely done. You are awesome!
First off, valeros; you are the coolest dude ever. Thanks a ton for your work. Last night your parser pretty much talked me into jettisoning all the individual rulebooks I'd parsed with Tenian's Parser and just use the parses from yours. It's so much cleaner to just have like 8 books in my Library instead of 20, and it's sooooo much easier to find Powers when it's in one Library entry named "4E Powers", or Monsters in my "4E Monsters" book. Man is that sweet!
A couple things:
- I parsed the Powers last night, but when I add them to a character sheet, it doesn't update the actual sheet with (a) the description, and (b) the preformatted conditions and effects for a given power (ie the little sword icon clicky on the sheet). I can click the button on the character sheet and it'll pop up the actual Power Card, but the character sheet fields themselves aren't being populated. Did I do something wrong when I parsed?
- Any chance you'll support Traps in a future revision?
Thanks again, and keep up the awesome work! Tools like this are so insanely helpful for all the 4E DM's out there.
Thanks for the feedback.
I will have to check on the powers. My guess is that what you are seeing is related to the questions I asked above about which tags really need to be in the output. I did not output as many tags as Tenian's parser because I did not think they were used anymore. But it sounds like they are. I should be able to add them, hopefully this weekend. (So hopefully you kept the file(s) with the downloaded data so you do not have to download the entries again when I update the parser.)
Also, you can combine entries if you want into one output module. You can either specify multiple types when downloading or specify multiple files to read. That way you can have less than 8 files if you want.
I will look at traps as time permits, but I will try to fix powers first.
Thanks
Looks like I forgot to include the shortdescription in the powers. I fixed and the powers seem to work fine for me on the character sheet.
Torgaard, when you get a chance, could you please try the updated parser and see if this resolves the issue for you. Thank you
Sweet!
And I'm stoked to try your idea of just making one big module. That could be cool as hell. Ohhh, I know what I'll do: I'll create like a "4E Players Compendium" with like Powers, Rituals, Backgrounds, stuff like that, and then make a "4E DM's Codex" with Monsters, Terrain, etc. Awww ya.
Works! Well done. And thanks again, val.
Off to make my giant sooper modules!
Hi!Quote:
Originally Posted by Torgaard
I'm gonna try this too! :D
Valeros, as I am going through the I noticed that racial and class features don't appear in the powers or feat book.
For example the deva astral resistance and astral majesty or the Battle Cleric's lore for example. I had them functional in the original books, but they don't seem to pop up on your parser. I know from previous experience that these can be "finicky" so you may not have gotten to those yet.
Also do you plan on doing conditions and skills? They are a short list, but I realized that Tenian's latest version does not parser the "nature" skill. Its real odd and I have tried a bunch of methods but it seems to "ignore" it when parsing. Earlier versions of the parser do it just fine. Also that parser doesn't really parse the skill text and tables correctly. I have done my own patch job, but its not a great solution.
Besides, skills and powers, I guess the paragon paths and epic destinies could be done as well. All these are pretty short lists and hopefully won't burden you too much.
I must tell you how grateful and thankful for all these parsers. It has made my DM'ing infinitely easier as well as cutting down the amount of books to almost nothing.
Again many thanks!
Primarch
Totally awesome Valeros. Thank you, truly
:D
Added Glossary entries
Skills show up under "Category: Skills"
Conditions show up under "Type: Rules Condition"
Lots of other goodness like "Type: Rules Table"
Hi!Quote:
Originally Posted by valeros
I bow to your coding magnificence!
I have not been this enthused with parsing for quite some time. I spent most of the afternoon putting some custom books together using your parser. It's never been this easy and good! :)
Primarch
Awesome parser and super easy to use not fiddly at all like the old one!
I do have one question/issue though. How can I change the .xml type created in the module from db.xml to common.xml or even client.xml?
As it stands I seem to be unable to do this which means I can't push the module for the players to see. They all have DDI accounts but its a pain to have to look stuff up in books or on the DDI when I have all the data parsed already but they can't see it.
Am I missing some option on the parser? I know Tenians has an option for the output format, does the new one?
Either way, thanks for all the work you have put into this tool!
There's a super easy way to do this. Change the .mod to a .zip file.
Open up the .zip file.
Rename the db.xml to whichever kind you want.
Rename the .zip and .mod.
I *think* that works.
Oh really? Haha I had thought about that but assumed the xml was different in some way aside from the name.
I'll give it a try and report back :)
Yes, This does work. I recently did it and tested it.Quote:
Originally Posted by mattcolville
Hi, what kind of offline file does this parser except? What can you specify by add files?
The parser only accepts files created by the parser when you download entries.
When you dowload Compendium entries, it leaves a file name "<module name> <number of entries> Entries <date>.txt". (For example, "Monsters 5170 Entries 2013_0421.txt") That file has the downloaded entries. You can select multiple of these files if you want.
It accepts these files so that you do not have to download again from the website when you want to combine entries in different ways or if the parser changes to fix a bug or add an enhancement.
Any way in creating files like these? I have a lot of the compendium backed p on my machine from when i had a subscription, actually php + mysql db scrape
Im trying to get that stuff int either your or the old parser... atm i am actually trying to write a c# prog to convert the stuff into the old format.. but maybe it would be better to convert it to your format. dont know if i could post an example of what i get from my database as it is actually wotc content.
Easiest would be to renew my sub and parse it.. but strangely paypal does not work anymore for it
But at the moment i would like to get my data into fg somehow.
Thanks for this great tool!
And I'll remove my original question as I found the answer earlier in the thread.
Valeros could you give me a template for those files? I managed to put together a small program with the help of the htmlagilitypack that converts my data to the format of the old parser txt files.. Sadly it doesn't like monk powers with[] in them (I simply changed all [] to() ) but it still doesn't like monk powers because they are basically 2 powers, movement and attack technique.
I think it would be much better to rewrite my program so it outputs files in the format you use for saving scrapes.
I know its much to ask, and probably not really useful for anyone but me. S I understand if you don't have the time.
I managed to get quite far with my power parsing, my problem is all powers linked to another power as a granted power dont look right.. if i go into the xml and manually give the power a recharge value(either at-will, encounter or daily) it looks right but without it all the fields overlap. This makes all linked powers unreadable... btw are the monk full disciplines supposed to parse in this way (attack technique as power with the second technique added as a granted power) ?
So I went and redownloaded the items. I got an error to pop up though in the box after everything was complete:
Quote:
Mold King's Crown: Not Parsed: <H1 class=atwillpower><SPAN class=level>Level 10 Disease</SPAN>Blistering Corruption</H1>
Mold King's Crown: Not Parsed: <P class=powerstat><B>Stage 0</B>: The target recovers from the disease.<BR><B>Stage 1</B>: While affected by this stage, the target takes a -2 penalty to Fortitude.<BR><B>Stage 2</B>: While affected by this stage, the target takes a -2 penalty to Fortitude. In addition, the target regains only half the normal number of hit points from spending a healing surge, and the target’s hit point total cannot exceed its bloodied value.<BR><B>Stage 3</B>: While affected by this stage, the target suffers the effects of stage 2 and has an aura 2 (poison) that cannot be deactivated. Any creature that ends its turn in the aura loses a healing surge.<BR><B>Check</B>: At the end of each extended rest, the target makes an Endurance check if it is at stage 1 or 2.<BR><I>12 or lower</I>: The stage of the disease increases by one.<BR><I>13-17</I>: No Change<BR><I>18 or higher</I>: The stage of the disease decreases by one.<BR><BR>
Thank you for reporting. I add debug like that in case I miss something or something new appears in the data.
In this particular case, the data not parsed should not be a problem. It did not parse the disease associated with the item. The item should have parsed fine. (The disease can be parsed separately under diseases.) So you can ignore these particular errors for that specific item.
Added traps.
One issue I did not resolve yet is that traps sometimes have the same name for multiple entries. FG will only link to one even though all are parsed. For example, Glyph of Warding, Whirling Blades, and Pendulum Scythes. (I do not remember any other type having the same name for more than one entry.)
As usual, please let me know if you see anything odd.
Thanks
I would have a suggestion for the traps issue:
I assume the problem is that the same entry name results in duplicate database node names.
The simplest thing you could do is, have a integer counter that counts up after each parsed item, and when the parser creates the database node name simply append the counter.
Let's say you have two traps called Spike Pit after appending the counter you have database nodes called SpikeTrap94 and SpikeTrap95 (I made up the number, but counting up for every parsed item ensures that its a number unique to each entry.)
I managed to parse everything except the disease of the mold kings crown and the augmentable powers from phb3 (these do parse but loose the augment x tags)
Is there a way to show category and level for rituals? and maybe sort them by level (its one of my players big complaint since the beginning)
And for monsters the old parser had a neat function to automatically pair tokens to the monsters if a token with the same name was present in the tokens folder .. maybe with the improvement to select a standard letter token if no token is available.
Another question is it intended that the alchemical rituals don't have the items they create linked to them or , is it an error on my part?
A way to automatically pair tokens would be amazing.
I find that aside from locating appropriate maps, assigning tokens to each creature takes the longest as far as setting up an encounter goes.
For the resulting output directory (not mod file), if you add a "tokens" directory with a subdirectory named the same as the module to the resulting directory, those should be automatically mapped to the monster. The token name must match exactly the name of the monster, be in lower case, be a PNG file, and end with ".png". So for example, if the module was named "Monsters", then the token for "Shuffling Zombie" must be "Monsters\tokens\Monsters\shuffling zombie.png"
After adding the tokens directory, use the FGPEMUtility.exe to convert resulting directory to an actual .mod file (or run the parser again which should form the .mod file if you use the same module name.)
This was to match the same way as Tenian's parser did it. That way everyone who had already created token directories could just use their existing directories (or copy the tokens.)
Correct, alchemical rituals don't have the items they create linked to them. Right now, nothing has a link to a different entry. The items do show up under items if you parse items.
(Closest things are powers that have a secondary power in the same compendium entry are combined into one power and diseases on monster entries are included in the Other tab of the monster.)
Added a new version that has level and category lists for rituals.
It seems like rituals have some sort of filtering option, but I do not know how to create the list to use it. If someone has an example of a good ritual list, I could try to use that format instead. For now, just a list of levels and categories shows up when selecting rituals (and an "All Rituals" option).
Noticed something strange. In the equipment list if you scroll over a price it alters the price even if mouse wheel editing is disabled or set to ctrl only. The ritual list is fine this way totally what I was looking for great work!