FGU Memory Leak

[jharp]

I did some analysis last week and found that there was a fair amount of overhead in FoW data. I have the save speed bottleneck fix for FoW in the next release (soon).

Also, @cpinder is currently out of office; but I plan to work with him to see if we can streamline the FoW data, which can get quite large. Small LoS sections such as per bar portcullis or lots of small pillars can exacerbate the situation.

The save impact will occur even for closed maps, since the data is still stored and reloaded; so any edited map with tokens would need to be reverted or tokens removed. The save fix I have in progress will help mitigate the save time but the data can still get large.

Note that the performance impact will remain until FoW is cleared for each token, and closed/reopened. (Or reverted and closed/reopened). This is because the FoW data is preserved in undo states while the image is open as well. (And cleared on close)

More than likely, any changes we make to optimize the FoW will probably reduce the fidelity of the edges slightly to simplify the shape and number of points.

On a related note, extensions that attempt to preserve fog of war between maps may make the situation worse, since the large data sets will be getting duplicated. I recommend turning these off for now.

Regards,
JPG

I will also say that I found the following in my FoWEnhanced testing. Maybe the FoW lines should be shorter lines with more lines. Again, Moon, I'm open to suggestions on improvements to FoWEnhanced.


Interesting Tidbits

If you save a single DB entry of 84k characters the save is several 5+ seconds but with 84 1k entries its less than 1/2 second to save.
A 15k character entry saves 10 times faster per character than does a 85k character entry.

[jharp]

On a related note, extensions that attempt to preserve fog of war between maps may make the situation worse, since the large data sets will be getting duplicated. I recommend turning these off for now.

Regards,
JPG

On this single point of Moon's, FoWEnhanced does not store the FoW data in the DB so the data is not duplicated in the DB. Obviously strings are used in the process of getting the FoW data, and these will remain around until collected. I see no reason for turning off FoWEnhanced at this time. If Moon has additional data to share, I'm happy to evaluate if FoWEnhanced is a significant contributor to the issue.

Jason

[Moon Wizard]

If someone is running their FG folder off of a network or external drive, the file load/save times could add potentially significant delays to any archiving and restoring process. So, while your extension may not save the data in the FG database, it is storing the data in a file somewhere on the disk.

Additionally, if there are large FoW data sets that have built up, and your extension is restoring those large data sets; then it propagates the large data set issue. While it is something that I want to look at, it is not something that we will be able to immediately address.

Between those two items, that is why I suggest not running with the extension. It's not that the concept is bad or unwanted; but the system will not handle it well, especially in certain situations.

Regards,
JPG

[jharp]

Well I guess the choice is to not have the feature for quite some time or to have the feature now with some slow down (if any). Please if anyone is having slowdown with the save and load of FoW data let me know directly on the FoWEnhanced support thread.

We can only dream of a time where FG actually works better out of the box and doesn't need a bunch of extensions to fill out the functionality. Personally, I don't think anyone should have to pay for the functionality provided by FoWEnhanced. It should have been built into FG to start with.

Jason

[BushViper]

Disclaimer - I'm not a programmer so the language I use may not be correct, but the hope is that the idea is generally understood.

@Moon Wizard.

So, I've been doing a lot practical testing to see what I can learn (and possibly do) about the performance issues, myself. While I haven't discovered anything new, my testing has raised some questions, for me.

Line of sight, lighting, shadows, etc. all have massive performance costs. As do extensions that need to consider the positions of tokens, occluders, etc., on a map.

So, my question is this -- Why does the FGU engine constantly calculate the data for everything, all the time? Instead of....say....just the token(s) that are selected/targeted?

Admittedly, I don't know exactly how the data is handled, but I have been tinkering with the platform for long enough that, at least to me, it seems that there is a bunch of redundant, or unnecessary processes/calculations happening, which absolutely crush performance.

I don't know. I could be completely off the mark, but every time I log in to FGU I'm left scratching my head because I literally cannot begin to understand how, for what amounts to, a relatively rudimentary GUI has its performance brought to its knees by a few tokens, some extremely basic lighting, and shadows.

Like everyone else, I get plenty of the pre-described "lag" where mod files take 3-10 seconds to open and, every session, I get my share of whiteouts that end up as "not responding" hang-ups before FGU raises itself from its temporary death. While I hate that crap, I can live with it. However, the absolute garbage performance on maps is what's going to completely poison the well for me.

Performance issues are becoming an increasingly common topic of discussion in the numerous FGU Discord servers I'm a member of and, as far as I can tell, a lot of people are getting completely fed up with having to make feature concessions, spend days Googling for work-arounds, and then having to devote even more time to figuring out how to implement those 'fixes.' All just to get a half-baked session out of the platform. Unless, of course, the user is running a client with a tiny campaign folder, not using LoS, no lighting, no shadows, no extensions, keeping map/token size small, and only load the mod files that are absolutely necessary for your campaign. Do all that and your client will hum right along. Sounds amazing...to uninstall.

FGU has a steep learning curve, but it's rewarding once you get ahead of that curve. However, many people will never get there because they're not going to invest the time and effort it takes to master FGU when the UI looks and runs like it's on a Nokia cell phone from 2002.

I've said it before and I'll say it again. I truly like FGU and I want it to succeed, but people have limits to what they're willing to tolerate and I don't think that expecting the platform to run without constant, debilitating interface lag is too much to ask. I also understand that SW is a small company and you only have so many resources to allocate to any particular problem, but from where I'm standing, it seems like addressing fundamental issues like performance trumps nearly everything else because no-one cares how many bells and whistles it has if it doesn't run well. However, according to many of your staff, "performance issues only impact a small minority of FGUs users," "are greatly exaggerated," or due to some other excuse.

I've had a number of sessions after accidently deleting the data I was going to turn over but, after a lot of thought, I decided I wasn't going to bother packing up my campaign folder so some forum mod could examine it and then instruct me how I could shuffle a few files around to squeeze out a negligible increase in performance. That does absolutely nothing to fix the fundamental problems and for those of us that can grasp what's really happening and have spent literal weeks trying to fix it ourselves, the hubris is infuriating.

[Moon Wizard]

Any time that a token, occluder, point light, or token light is moved; all calculations (what to draw on screen, intra-token visibility, LoS, FoW, etc.) have to be redone. Under the hood, all of that information is just points, lines and radii. (i.e. there is no implicit "regioning" of data like your eyes and brain do automatically) Additionally, the way that the graphics work for LoS/FoW/lighting/FX/etc is that they use GPU shaders to determine the value of a pixel on every frame. So, lots of data slows down per frame performance.

The particular issue that I think we're running into right now is that the fidelity of the fog-of-war is too high. It is mirroring LoS exactly down to the pixel; which creates very large data sets. That's why I mentioned we're looking at ways to simplify the data (similar to how other games use more blocky fog-of-war).

In no way would I describe the image systems implemented in FGU as "rudimentary GUI". If you are a developer, you are welcome to explore building out image mapping/exploration GUI systems yourself to understand the considerations. As for the other UI elements, this is a subjective matter, and personally I prefer theme-ability and customizability over cookie cutter UI.

As for performance in general, I am constantly mentioning to people to keep things simple, to keep maps under certain sizes, etc. These are guidelines to improve performance.

For specific performance on specific machines, you can always try setting the "/vsync 0" to force 60 fps, because I believe the ever increasing monitor frequency rates and the default tying of frame rate to monitor refresh rates to be slowly increasing the per frame rate calculation burden. After doing some recent monitor shopping, I'm considering whether we should force the fixed 60 fps frame rate by default; with perhaps an additional setting for fixed 30 fps for older machines.

As for providing the data and steps you followed, this is a very important part of the feedback process to improve the product. We are not a gigantic company with 100s of people developing this software; but just a small company with only 2 client developers and 2 ruleset developers (out of 12 people total). We are looking to build up, but that takes time to find the exact right people.

By providing data sets and reproduction steps, this allows us to more easily pinpoint specific scenarios where the current systems are not performing well, and may even allow us to pinpoint the exact code we can adjust, fix, or analyze. Without data/steps, we are essentially guessing which one of a gigantic number of moving parts "might" be the issue. Then, we spend all our time on investigation, instead of fixing issues.

If you are having issues with particular scenarios, you need to raise those issues. That being said, sometimes the answer will be to pare down the data (whether it's number of custom modules, extensions, etc.)

Regards,
JPG

[bmos]

The particular issue that I think we're running into right now is that the fidelity of the fog-of-war is too high. It is mirroring LoS exactly down to the pixel; which creates very large data sets. That's why I mentioned we're looking at ways to simplify the data (similar to how other games use more blocky fog-of-war).

I'd be happy with fog of war that was 1/4 of the grid size, personally. Pathfinder rules seem like they'd be fine with full grid squares even, but 4-8x that resolution would be much prettier.
Right now it's possible for LOS to be calculated at over 100x the grid size on a high-res map which is (as you point out) quite excessive.

[Moon Wizard]

The problem with doing chunky blocks of fog-of-war is that it could inadvertently expose "more" than expected, and isn't necessarily what most people expect to see. I was thinking more about smoothing the curves, so that that small angles are combined into straight lines that reveal just "slightly" less, but significantly reduce data points. As I mentioned, this is something I need to explore with @cpinder, who is currently out of office, as well as on another project.

Regards,
JPG

[BushViper]

As for performance in general, I am constantly mentioning to people to keep things simple, to keep maps under certain sizes, etc. These are guidelines to improve performance.

Regards,
JPG

Heh. Christ, it's a lost cause, but I suppose I should thank you for reiterating one of the fundamental points of my post.

[LordEntrails]

Moon, just in case you and Carl have not thought about it already, but what about periodic refactoring the FoW to consolidate the data into polygons? You could perhaps set/control the resolution of the polygons to some factor of grid size. And maybe do it based upon action (open/close image?) or periodicallly.

No need to answer either way, just a thought that might have been worth your time.