Testing Update: Lighting still causing performance issues

[Neovirtus]

My group has been patiently testing Lighting features with me during our weekly Curse of Strahd campaign and during our most recent session, despite improvements, we were forced to turn off dynamic lighting due to performance issues again. My group got into an effect, lighting, and NPC dense encounter, and the whole client slowed down to the point of unplayability until I turned Lighting off.

The map:
Castle Ravenloft (07 - 10) - Spires and Tower Peaks (Players Map) from the third party map pack "Castle Ravenloft Player Maps" from DMsGuild with custom LoS and Lighting definitions

With:
Mist FX layer on
Dynamic Lighting on for all lights on the map (they are probably off in the state of the campaign I will share, turn them on to replicate the environment of the issue)
Ambient Lighting on, masking used to remove light from interior locations.
Combat Tracker at one point included the Party, several friendly NPC's, Strahd, all 10 animated swords from the map encounter, and two additional vampire spawn

System:
v4.1.0 ULTIMATE (2021-04-13)
D&D 5E
Windows 10- 16 Gb RAM
Ryzen 5 1600 processor
Nvidia GTX 1070 - 8 Gb dedicated RAM
2560 x 1440 Resolution

I am prepared to send a zipped copy of the campaign (and logs) to support if needed, just let me know. According to my recollection and referencing the logs, the issues would have surfaced sometime after sharing the above mentioned map, so probably in the 10:49-11:05 window of the logs. If there are other files I should include let me know.

[Zacchaeus]

How big is the map?

[Neovirtus]

576 kb, 2000 x 1440 px

[Zarestia]

Not wanting to 'advertise' my thread, but this is probably partially due to the high GPU load:

https://www.fantasygrounds.com/forums/showthread.php?67775-Heavy-GPU-load-(Build-04-13)

Nevertheless good findings and reporting.

Some questions for clarification:
- Did everyone have problems, you included?
- Did those problems occur when the party enountered the NPCs or even before? This might give some hints to what exactly is driving the software/hardware to its knees.

[Moon Wizard]

Can you please reference this thread, and send a copy of your campaign to [email protected]?

Thanks,
JPG

[Neovirtus]

@Zarestia - Yes, I do think this is a general GPU load issue, and not necessarily a memory leak or the like. I think that all of FGU is causing high load, and the lighting on top of everything else was too much for my (and my players) PC's.

1) Yes, everyone had issues, including me.
2) The problems became impossible to overlook once we added the 10 animated weapon NPC's. Whether that was because the whole thing was bogged down due to high GPU load which would have happened even without the NPC's or not, I don't know. All I know is I turned off the FX layer and the lighting, and the software was able to keep up again.

[Neovirtus]

@MoonWizard - Done.

One other note, it looks like in my haste to stop the issue during play, I deleted the FX layer, rather than just turning it off. So to replicate my case just add the FX layer - Myst (speed low ~53) to the map I describe in my post, as well as the 10 Animated Halberds which were on the field during the combat but my players have now bested :) They were in the heart of sorrow (North West tower on the map, where the PC's currently are)

[Neovirtus]

Further playing with this issue. I wanted to see if the issue persisted with no players connected, so I recreated the scenario as described in my previous post.

As I suspected, I don't see a huge load on my computer just having the map with effects and all tokens present on the map. GPU utilization hovers around 40%, GPU memory usage is 1.4 GB, temp is 67 degrees. Very stable. What I did notice in the resource monitor panel was that when I tried to move tokens on the map, there were many graphical hitches and slowdowns in the map image (freezing effect animation and the pulsing lights lag), and during this process the GPU utilization - rather than spiking as I would suspect - actually goes to zero. Very strange. It almost appears that the animations pause when tokens move, which could cause usage to go to 0%, but during long drags of tokens eventually they do stutter back to life.

update: removing the 10 animated halberd NPC's does drop the utilization more than I would expect, by half down to 20%. Moving tokens in this reduced load situation still tanks GPU utilization to 0%.

https://www.fantasygrounds.com/forums/attachment.php?attachmentid=45889&stc=1&d=1618864765

[Weissrolf]

Watch your CPU load when those GPU dips happen, you will likely see it increases to max out one of your CPU cores. This in turn leads to the GPU load drops, because FGU is not feeding new data to the GPU. This would then be a CPU bottleneck then, not a GPU one.

[Neovirtus]

Watch your CPU load when those GPU dips happen, you will likely see it increases to max out one of your CPU cores. This in turn leads to the GPU load drops, because FGU is not feeding new data to the GPU. This would then be a CPU bottleneck then, not a GPU one.

It does not. CPU usage is constant throughout.

[Weissrolf]

This is only a single CPU core maxing out and Windows may keep moving the load around. If you don't see a change then you likely are already close to maxing out to begin with and once you move a token it overloads.

https://i.imgur.com/wO0DjmE.png

[EllivasKram]

Try moving NPC’s via cursor keys.

[Neovirtus]

Try moving NPC’s via cursor keys.

I did, it's the same effect.

[Weissrolf]

Hold shift while moving NPCs, this forgoes the LOS/Light calculations during movement.

[Neovirtus]

Let's please keep the speculation, suggestions, and work arounds out of this thread. This is for reporting the bug, and for the devs to request additional information from us if needed. While I appreciate your efforts to help, that's not what this forum is for.

[Weissrolf]

No problem. Keep in mind, though, that "The Laboratory" also is "A place to discuss...". ;)

[Moon Wizard]

@lavoiejh,

I've downloaded the campaign; and taking a look at it now. However, I can't figure out which map image record you are using from your picture. It doesn't map to any of the CoS module maps.

Thanks,
JPG

[Moon Wizard]

I sent e-mail. I think it's a custom module I need.

JPG

[Neovirtus]

Sent the module for testing. Thanks.

[Moon Wizard]

Thanks for your patience and detail on this one. I've recreated the issue locally; and packaged up all my notes and your notes for Carl to review.

Regards,
JPG

[Neovirtus]

Hey @MoonWizard, I see the patch notes released today. I don't see any generic "performance improvement" note, which I assume this would be under. Safe to say a fix for this issue is still outstanding? Just curious! Thanks.

[Moon Wizard]

Yes, I was able to recreate with @cpinder on his local machine yesterday with your data; but we didn't want to hold up getting the other changes tested.

He believes it has to do with the number of occluders; and how they are laid out. Because there are a lot of circles, it's more complex to cull occluders, especially when sharing big open spaces like tower sections. So, it has a lot to do with that specific map. I've asked him to look at this to see if there is anything he can think of to change or help with the performance.

Regards,
JPG

[Neovirtus]

Glad to hear it was a useful test case. I may just need a new processor, the Ryzen 1600 single thread performance isn't what it used to be! But at the same time I'm sure many people are playing on much less performant systems.

[kevininrussia]

Just want to chime in on the Circle occluder. I used the circle tool to make terrain occluder for trees. Those circles had a bunch of vertices and our test game slowed down to unplayable. Last game I deleted them all and manually added occluders with much less vertices. Performance was way better. So maybe the Circle tool should add less verticies? I dont think it needs to be a smooth circle.

[Moon Wizard]

I thought we did a pass to reduce points in the circle occluder before, but I'll have to ask Carl and review again. The problem is that depending on the size of the circle, you don't want to make it too blocky. Also, the map @lavoiejh provided has circle in circle, which is doubling up.

Another wrinkle is that we can't assume how people design occluders; and there's no magic algorithm to tell us when occluders will cause performance issues (since they are just lists of points). So, @cpinder is doing a look to see what he can do performance-wise first; then we'll probably revisit circle occluder tools too.

Regards,
JPG

[Zacchaeus]

The number of points created by the ellipse tool was indeed reduced somewhat; but that was back in Alpha if I remember correctly - which is a while ago now.

[Neovirtus]

I don't think I defined the LoS occluders for these, so I'll have to double check how many points there are.

[Griogre]

For what it's worth, when I was playing around with terrain for brushes and occluders for trees I found small circles had a ton of points I didn't need and seemed needlessly complex. So instead of using a circle, I ended up using hexes with just 6 points for large bushes and I went down to squares/diamonds for small trees. Changing the number of points based on size might be helpful.

[Neovirtus]

I updated to the new patch and tested briefly and it seemed that the issue was much improved. I didn't have any players though, and will update again next week after more testing.

[Neovirtus]

Glad to report that we just had another session (our FINAL session of Curse of Strahd!) and lighting worked great, without any game breaking performance issues. We never went back to the map that started this thread, but overall wanted to share that it is performing much better a across the board.

Thanks for your work on this, dynamic lighting was a real treat in Castle Ravenloft!

[EllivasKram]

Did you use at all the “/vsync and /imagequality” chat window commands.

[Neovirtus]

Did you use at all the “/vsync and /imagequality” chat window commands.

Nope, set to 1 for both which should be the default I believe.

[EllivasKram]

Nope, set to 1 for both which should be the default I believe.

Zero is the default.

But I suspect for most people with 4K monitors and like to use maps in background mode. (Like me) we will have to use /VSYNC 3 or 4 [0-4] and /imagequality 2 or 3 [0-3]

I’m using maps in window mode now for testing and on live and it’s ok. Just annoying how you end with a million windows behind it. Maybe a always lowest window depth option would be nice.

[Kelrugem]

1 is default for vsynch, and 0 for imagequality :)

[Neovirtus]

When I was testing I changed image quality and intended to return to default, but apparently did not lol. Either way, I didn't see any issues when testing any of the settings, but my gameplay testing I guess was at 75% quality FWIW.

[Zarestia]

Didn't want to open an extra thread, so I keep my experience from my first TEST campaign (~4h, 6 PC) short. I hope it's fine.

- No performance issues on my or my player's side (we played with adjusted vsync & imagequality settings).
- Lighting and Vision worked fine as far as I could tell (they didn't explore really much).
- There were some minor issues with one door (they couldn't seet through), don't know if it was a bug or my bad & quick LoS points, all other doors worked fine. Deleted the door LoS, was fine after.
- Player View through GM was a bit quirky sometimes, wasn't updating perfectly and sometimes mixed the views (I think?). Maybe an extension. If I can replicate this in a fresh campaign I will start a thread for it.

All in all it was a pleasent experience (despite one PC dying again [1h after being revived]) with two minor hiccups. :)

EDIT:
- Player View works after the newest update way better. There is still an extension causing issues if I desselect. Need to find that one...

[Sterno]

- There were some minor issues with one door (they couldn't seet through), don't know if it was a bug or my bad & quick LoS points, all other doors worked fine. Deleted the door LoS, was fine after.


I've seen an issue like this on live and filed a bug report about it a few weeks ago (though obviously it's possible the root cause of what you were seeing is different... as you said, maybe it was bad LOS points). I've seen some others report it too. I'd heard it was supposed to be fixed on Test though so the fact that you seem to be having it (or something like it) caught my eye. Any chance you have a version of your campaign somewhere from before you deleted the problem area? I'd be real curious if it was the issue we're seeing in Live still present on Test, or if indeed it was just some bad LOS on the door.

Related threads (maybe):
https://www.fantasygrounds.com/forums/showthread.php?67986-GM-Can-t-See-through-Open-Door
https://www.fantasygrounds.com/forums/showthread.php?67733-Suddenly-having-LOS-issues-with-doorways

[Zarestia]

I've seen an issue like this on live and filed a bug report about it a few weeks ago (though obviously it's possible the root cause of what you were seeing is different... as you said, maybe it was bad LOS points). I've seen some others report it too. I'd heard it was supposed to be fixed on Test though so the fact that you seem to be having it (or something like it) caught my eye. Any chance you have a version of your campaign somewhere from before you deleted the problem area? I'd be real curious if it was the issue we're seeing in Live still present on Test, or if indeed it was just some bad LOS on the door.

Related threads (maybe):
https://www.fantasygrounds.com/forums/showthread.php?67986-GM-Can-t-See-through-Open-Door
https://www.fantasygrounds.com/forums/showthread.php?67733-Suddenly-having-LOS-issues-with-doorways

A player standing right in front of the door couldn't see in. I created the LoS in the session, so I got no old state.
Just recreated the door, with player as localhost everyhting seems fine. Probably badly drawn LoS by me on this particular door.

[Griogre]

Were you using a map pack and stamping down tiles? I think there maybe an issue with doorways but if feels like it needs some specific preconditions.

[Neovirtus]

If you have an issue to report you should start a new thread. This appears to be a completely separate issue from the one this thread is discussing.

[rathen45]

Has there been any thoughts on having a slider or a drop down menu to have the DM choose the amount of nodes that go into circles?