I would like some help with parsing DDI content

[Sergei-san]

Hello!

Yesterday I purchased FG Ultimate and one month subscription to DDI.
I am having trouble with the parser programs, however.

I tried using the item parser from 4E forum. It managed to create me a text file with loads of entries (this was dungeon master entries, so it contained monsters, traps and such - I don't remember exact figure now, but it was in between 5000 and 10000). However, after a VERY long parsing process (the download took less than an hour, parsing almost half a day), I was greeted with index out of bounds error message from NET framework. Having no source code around, I just closed the window, frustrated.

I had a different experience with parsing player content (powers, class things, items and such). Fetching data from DDI (was something in between 15000 and 20000 entries) took relatively short time, but my problems began when it went to parsing mode. I noticed a huge amount of rows containing "Not Parsed" messages on parser log window. They seemed to be magic items with different level of items inside them. Since obviously something was wrong, I canceled the process after watching those Not Parsed entries running in the log window for a few hours.

Am I doing something wrong? Why I am getting these errors and why does the parsing process take ages? Download with, say, 10 browsers was quick.

I'd like to have at least the three core books (DMG, PH, MM) parsed so I could run a campaign for my players. However it would be nice to get all of the content DDI has to offer, because I paid for that and the forum topics indicated this could be done. Especially so for more monsters, to give more ideas to plan around.
I'm not a completitionist though, I would be more than happy with some libraries that produced correct results for the most of the DDI entries, I'm not fussed about missing some.

Could someone help me out with this, please? Alternatively, if someone has the modules correctly done, I'd love to have those. Obviously that would require me to be DDI subscriber, as well so that I would have a legal access to those entries as well. How would I go and prove that I am a subscriber?

Overall I am happy with my FG ultimate purchase, and it seemed my players liked it too, even though I had my doubts about them complaining about missing live games.

Thank you!

[Sergei-san]

Hello!

I managed to debug this a bit myself. I noticed some people mentioning browsers and possibly OS version might have something to do with this...

I ran 4E parser on four different virtual machines at work. Win7 on them all, different VMs having IE8, IE9, IE10 and IE11 on each of them. My home setup is Win8.1 + IE11.

I had to run Parser on 1 browser only since it seemed to crash on opening too many - however what I noticed is that after download was done, the module was created in a few minutes and I had only one warning row in all DM entries thing. All of the VMs produced mod file with same Parser log.
Poison extra line: Granny's Grief: <<P class=publishedIn>Published in .</P>>
All 6725 entries were successfully parsed.

I did also a quick diff on the text files each of the VMs produced.
IE9 and 10 files were exactly the same. 9/10 compared to 8 had some entries being placed in a different sequence, no data was different or missing. I assume the Parser does not mind the different order.
9/10 compared to 11 produced results that had syntactically different form, though. One such example:
<IMG align=top src="https://www.wizards.com/dnd/images/symbol/aura.png">
<IMG src="https://www.wizards.com/dnd/images/symbol/aura.png" align=top>
I went through several of these and all of them were the same syntax difference. I'd say this probably does not throw Parser off either.

This is weird. Might have something to do with Win7 vs Win8(.1) then. NET framework version doesn't seem to have anything to do with it either, some VMs had 4.0 and some 4.5.

I'll try Player Entries now.

[Sergei-san]

Looks like the forum ate my reply for some reason - but to let you know, DM part of the Parser finished successfully on Win7 virtual machines (IE8, 9, 10 and 11 worked all). My home setup is Win8.1 IE11.

[gmkieran]

Sergei-san, does that mean you got everything working you wanted to? If not, have you gone through this thread on using the parser?

https://www.fantasygrounds.com/forum...reate-a-Module

There may be some answers to the issues you were having in some of the later posts there. Many of the problems folks have is with inconsistencies in the formatting of the data from WotC. They've changed how their stat blocks are set up multiple times and are often inconsistent even within the same source. The last I had heard, the parser had been updated to handle pretty much everything except items properly, but that may have changed.

Cheers!
GMK

[Sergei-san]

I managed to get everything with that 4E item parser (not the one mentioned in the guide, but on 4E forum first page topic). Apparently it doesn't like Win8.1 + IE11 or multiple browser option. Thanks for help though!

[Griogre]

Sergei-san I'm glad you were able to get the full data from DDI. If you have any other questions, just ask.