Just Updated Fantasy Grounds - Campaign now has Errors

[TMO]

I launched Fantasy Grounds for the first time since our game session last Sunday (which worked just fine and closed gracefully.)

Today I launched Fantasy Grounds and noticed an update was available. I loaded all updates.

When I went to open my campaign I was notified of several XML parsing errors for various modules and the campaign. When the campaign finally loaded I was able to see my modules but all of my characters disappeared. I think I have a backup of the campaign from before Sunday's game but I am a bit worried what this could mean and how careful I will need to be making sure I backup after every game session.

Any known issues with the current update or is this mere coincidence?

[Zacchaeus]

The update was simply the release of the SRD (https://www.fantasygrounds.com/forums/showthread.php?27823-5E-OGL-Released) so it shouldn't be affecting anything.

[TMO]

Thanks. I didn't think anything major was updated since the version hadn't changed either. Plus, this update took at lot less time than normal. So, I will consider this a coincidence but I'm still worried about the cause of the formatting problems. I know that improper closing of FG will leave a campaign database open (and thus a backup is made), but that has never caused any XML parsing errors. Any ideas of the likely causes of this?

[TMO]

Just a quick follow-up, I examined my offending db.xml file and I can see why it generated parsing errors. After line 10705 (which is where the console started to report errors), the XML is all fubar. All kinds of garbage shows up and then a bunch of Char(0) characters. I am clueless as to how any of this got into the XML. Could this be the result of someone copying information from an external source and pasting into FG? The last opening tag that I see before the corruption is <treasureparcelcoinlist>. I do recall that the last thing I did before closing the campaign was update the Party Sheet inventory. This was our first game in a new campaign so this was the first time I opened and added anything to this window. I'm just wondering if some damaging content in the clipboard made its way into FG.

[Zacchaeus]

You've got me, I don't know anything about xml. However I seriously doubt that anyone - other then you - has access to the db.xml. Certainly not the players.

[Nylanfs]

It's not residing in a shared location that happened to try syncing is it?

[TMO]

You've got me, I don't know anything about xml. However I seriously doubt that anyone - other then you - has access to the db.xml. Certainly not the players.

Oh, I quite agree. I didn't mean anybody copied garbage directly into the XML itself. I meant the garbage got copied into FG. For instance, I know some of my players are using the Notes system. I'm not sure where these get stored. My point is that there are a variety of places where the DM (me) and my players can modify data in the game itself and since FG supports copy/paste, I was just speculating that perhaps somebody pasted content that FG didn't like and it corrupted the XML file.

[Andraax]

Just a quick follow-up, I examined my offending db.xml file and I can see why it generated parsing errors. After line 10705 (which is where the console started to report errors), the XML is all fubar. All kinds of garbage shows up and then a bunch of Char(0) characters. I am clueless as to how any of this got into the XML. Could this be the result of someone copying information from an external source and pasting into FG? The last opening tag that I see before the corruption is <treasureparcelcoinlist>. I do recall that the last thing I did before closing the campaign was update the Party Sheet inventory. This was our first game in a new campaign so this was the first time I opened and added anything to this window. I'm just wondering if some damaging content in the clipboard made its way into FG.

Sounds like the symptoms of a) a hard drive that is in the process of failing or b) your directory is synced to another location (or cloud) and the file was corrupted during an update.

[TMO]

It's not residing in a shared location that happened to try syncing is it?

That was my fear at first, but then I remembered having trouble keeping campaign data in folders like Dropbox and OneDrive so several months ago I moved the Data folder outside of those and I just copy this folder into my sync folders after each game session as a backup. So, the answer is no, there should not be any interference as a result of this.

[TMO]

Sounds like the symptoms of a) a hard drive that is in the process of failing or b) your directory is synced to another location (or cloud) and the file was corrupted during an update.

Thanks. I just addressed the sync/cloud issue so I'm certain that isn't it. Hard drive failure is always a possibility (yikes!), but this is a relatively new SSD drive. Anything can fail at any time, of course, so I'll chase that thread to see if it yields any possibilities.

[Zacchaeus]

The other possible culprit(s) being extensions?

[TMO]

The other possible culprit(s) being extensions?

Hmmm...I might need a refresher course on what qualifies as an extension versus a module. I do have a custom GM icon, which exists in my Data > extensions > GMIcon folder. I just copied the one that damned (?) previous posted and modified to suit my needs. I've had this in place for a while so I am doubtful this is the problem.

The only other custom addition to this campaign (which is PotA) is your own custom maps that you provided. I have some images I added to the images folder, but that didn't require any XML mods.

[Nylanfs]

What about the NSA agents spying on your game? Did you check with them?

[TMO]

What about the NSA agents spying on your game? Did you check with them?

LOL. They probably didn't take to kindly to my wifi being named "NSA Surveillance Node #55".

[Zacchaeus]

Hmmm...I might need a refresher course on what qualifies as an extension versus a module. I do have a custom GM icon, which exists in my Data > extensions > GMIcon folder. I just copied the one that damned (?) previous posted and modified to suit my needs. I've had this in place for a while so I am doubtful this is the problem.

The only other custom addition to this campaign (which is PotA) is your own custom maps that you provided. I have some images I added to the images folder, but that didn't require any XML mods.

Extensions are the things you select at the 'Load Campaign' screen; they are listed below the 'Select Campaign' dialog. The GMIcon shouldn't be any problem.

[TMO]

Extensions are the things you select at the 'Load Campaign' screen; they are listed below the 'Select Campaign' dialog. The GMIcon shouldn't be any problem.

I appreciate the tip. Nothing extra ordinary in this list: DM Icon, D&D Elemental Evil Desktop Decals, 5E Theme - Wizards, and 5E - Language Fonts (FR) - Wizards.

I should have mentioned that while last Sunday was our first official game, we had a warm up session the week before and I've been setting up the campaign for weeks. Nothing has changed other than the addition of the custom maps and the fine tuning of each character's sheet.

[Andraax]

Thanks. I just addressed the sync/cloud issue so I'm certain that isn't it. Hard drive failure is always a possibility (yikes!), but this is a relatively new SSD drive. Anything can fail at any time, of course, so I'll chase that thread to see if it yields any possibilities.

This brings up another wrinkle; SSD drives write to a cache, and later on the cache is written to solid state. There could have been a problem between writing to the cache and the later update...

[TMO]

This brings up another wrinkle; SSD drives write to a cache, and later on the cache is written to solid state. There could have been a problem between writing to the cache and the later update...

Well, I realize many possibilities exist and I do not want to be dismissive, but this seems unlikely to me. Many storage systems use cache first and then commit to disk. I've used SSD drives for a number of years - this is just my most recent device - and I develop database software and have never seen anything like this. Granted, FG doesn't work like a tradition database, so I have less confidence in its data management capabilities (translation: I am merely expressing my lack of knowledge, not suggesting it is flawed), and therefore am open to most ideas. I'm still looking into the possibility of an SSD failure.

[Andraax]

"Next, there might be a cache in the disk drive controller; this is particularly common on RAID controller cards. Some of these caches are write-through, meaning writes are sent to the drive as soon as they arrive. Others are write-back, meaning data is sent to the drive at some later time. Such caches can be a reliability hazard because the memory in the disk controller cache is volatile, and will lose its contents in a power failure. Better controller cards have battery-backup units (BBUs), meaning the card has a battery that maintains power to the cache in case of system power loss. After power is restored the data will be written to the disk drives.

"And finally, most disk drives have caches. Some are write-through while some are write-back, and the same concerns about data loss exist for write-back drive caches as for disk controller caches. Consumer-grade IDE and SATA drives are particularly likely to have write-back caches that will not survive a power failure. Many solid-state drives (SSD) also have volatile write-back caches.

"These caches can typically be disabled; however, the method for doing this varies by operating system and drive type:"
More Info (https://www.postgresql.org/docs/9.1/static/wal-reliability.html)

[dulux-oz]

Hmmm...I might need a refresher course on what qualifies as an extension versus a module.

Check out this (https://www.fantasygrounds.com/wiki/index.php/Development_Overview) FG Wiki Article :)

Cheers

[damned]

Hi TMO.
As mentioned in the start of the thread the update was for the distribution of the SRD only.
SSD drives tend to be all good or all bad - they dont do much in between (usually).
SSD cache drives have a bit of both - they can have standard hard disk bad sectors etc and there can be issues with data writing to cache and from cache to disk but its not a common occurrence either.

if you have a backup of your campaign i would definitely have a look at the backup and see if you have lost much - if not I would prolly just go ahead and use that. if there is a lot missing you might have to dig into the XML to see what can be salvaged...

and its always a good idea to backup, and backup regularly...

[TMO]

Hi TMO.
As mentioned in the start of the thread the update was for the distribution of the SRD only.
SSD drives tend to be all good or all bad - they dont do much in between (usually).
SSD cache drives have a bit of both - they can have standard hard disk bad sectors etc and there can be issues with data writing to cache and from cache to disk but its not a common occurrence either.

if you have a backup of your campaign i would definitely have a look at the backup and see if you have lost much - if not I would prolly just go ahead and use that. if there is a lot missing you might have to dig into the XML to see what can be salvaged...

and its always a good idea to backup, and backup regularly...

I'm probably borderline obsessive when it comes to backing up, both my computer in general and my FG stuff. As I had noted, this was really our first legit game session and I had a backup from the night before. So, I ended up restoring that backup and updating the party sheet with the few encounter XP and treasure that had been earned. I told the players what had happened and suggested they log in to update their character sheets and game notes with anything they might have add.

So, I will now make it a habit of backing up my game immediately after each game session. I would like to just backup the minimal files since I have multiple backups already. Up till now, I've been backing up my entire Data folder, but I know that is way overkill since I only have one or two campaigns going on. Do I only need to worry about backing up db.xml or should I just plan on backing up the entire campaign sub-folder?

[Zacchaeus]

I'd suggest the entire folder since that includes stuff like portraits and map masks as well as settings and a few other things that would be a pain to have to reset up.

[Andraax]

I use rsync to backup - it only backs up files which have changed.