[2.8]Network issues

[DM_BK]

Last Monday was a 1st game on 2.8.0, everyone took an hour to download everything from me (I didn't actually time it but it was around an hour). This is 6 clients with an upstream of around 800k. We just rolled with it because I was hoping everything needed re-downloaded due to the new patch. There were no other issues after everyone completed download....we played fine from there.

Now we're on the 2nd week and everything is re-downloading again. Taking around another hour to do so. As a test I had one of the clients that I was no longer transmitting to disconnect, close out of FG, and re-connect. Upon re-connection not only are they re-downloading everything but it appears everyone else is as well! What the heck?

Other details that may or may not be of use:
-I'm using the Windows 7 Resource Monitor to watch the network activity.
-I didn't do anything to the campaign files from one week to the next.
-There's to much lag to play when everyone is downloading. Once it tappers down to only a few clients still downloading its playable.
-I wouldn't say networking/caching was optimal in past releases of FG but this seems much worse.

I still long for some way to see what's going on behind the scenes with networking. Wasn't there something added recently to get more info? I know /console tells more now but it just seems to be errors (which I haven't seen many).

Thanks!
DM_BK

[Griogre]

Typically the longest download is that of the ruleset. You should not have to re-download the ruleset everytime unless it is being changed between sessions. What ruleset are you using? And have you made any changes to it *ever*? If you have made changes to a ruleset and not changed the name and your players play in games that use a ruleset with the same name then you would have to download the ruleset every time.

[Moon Wizard]

I haven't heard of such extensive download times before. The 2.8 code actually increases the number of downloads at once from 1 to 5 for each client, as well as caching token data on every client instead of downloading each time.

* Which ruleset are you using?
* Which extensions are you using?
* Which modules are you using? How did you build them?
* How large are the files in your images directory?
* How many tokens do you have in the tokens directory?

Any information from the console would potentially be helpful. You can save off the console.log file between starting up FG instances, and send to [email protected].

As an example, my players connect in under 30 seconds normally, and under 5 minutes for a full download with a clean install. My campaign has ~50 images in the 100-200K range, and 1700 tokens (~50MB) in the tokens folder.

Another idea is to try running the /flushdb command which will remove all sharing of campaign information other than the data that each person owns (i.e. characters, notes).

Thanks,
JPG

[DM_BK]

Griogre-

Pretty sure the ruleset is coming down upon connection...might even be part of that progress bar you see. At any rate, I don't think you could pick your character if you didn't have the ruleset down. All my issues are around background downloading after the character selection is made.

I'm using the base 4e set with no changes at all.

Moon-

Im using all the Dr Z extensions plus the wound conditions extension. All of which have been updated to latest for 2.8.

I have a lot of module files for 4e, most have been built with client.xml in them. Could be something here....though all of the clients have these files as well.

38 images averaging 800k size. 686 tokens.

I do not think you will find any joy in the console.log. A couple of warnings about not being able to find /common.xml in one of the module files but otherwise all just connections/disconnection notices.

I will try /flushdb. That's handy to know.

I think the biggest question to have answered is what all is transmitting to the other clients when a client connects. Because once all of the background transfers ended I had one of the clients completely exit and re-connect. This seemed to re-trigger all that background downloading over again for ALL of the clients.

In that test I was hoping to see that once they had everything down once they wouldn't be re-background downloading again when re-connecting. Worse expected case was for them to have to download it again...however it was much much worse then expected as EVERYONE started background downloading again....everyone else didn't disconnect, they did nothing but sit there waiting....only that single client disconnected.

I hope I being clear enough because its really odd behavior. If you trace the network calls from where a new client connects you might see something here. At least I hope so.

I should also note: I played my other campaign on Saturday. There were 5 clients connected and we did not detect this issue at all. Its a far newer game (only its 2nd session played), where as my Monday game as been long running. Token counts, extensions, modules, etc are all the same. Image count is less (25, same average sizes). The biggest difference I can see is the Saturday games db.xml is about 1 meg and the Monday game(the problem game) is over 4 megs. We also notice that a players character comes up fairly quickly in the selection screen (not instant by any means) on the Saturday game but the Monday people have to wait many minutes for their character to appear.

I also wonder if upping the downloads to 5 per client might not be causing some issues previously undetected. 30 uploads (6 clients x 5 streams) is likely hammering my upsteam bandwidth harder then 1 steam/client used to. Because of this its causing all of the clients to be unplayable because due to the lag they experience. As I said before, once the upload activity finishes we can play with little to no lag experienced. Perhaps pushing upload count back might re-leave this issue some (2 maybe?).

Thanks for the help!
BK

[Griogre]

Do you have any common modules? A very large common module might cause a download on connection.

[Moon Wizard]

Here are the items that get downloaded:

When showing loading bar:
* Ruleset files

After loading bar:
* Any shared/owned database information (including images, masks, etc.)
* Token names
* Portraits
* Pointers
* Module names/permissions
* Logged in identities
* Available identities for character selection

The characters will not display in the character selection window until the other data loads. So, long delays in seeing the characters on the selection screen are probably linked to lots of shared records in the database. The /flushdb command should help with that.

I don't think that the increase to downloads is affecting the situation, since we're still dealing with packets of 8K or less.

I'm curious why you think that the downloads start over when someone connects. There are no indicators of database downloads happening in the background on the clients, so I'm not sure how anyone could tell.

The only actions that trigger on a player logging in other than the standard downloads list above is that the 4E ruleset will add the user as a viewer to the CT, campaign effects, campaign modifiers, and options. This would only trigger those portions of the database to be sent to the new viewer, not all clients.

All in all, I would try the /flushdb command, and report back. I think it will help considerably.

Cheers,
JPG

[DM_BK]

I'll come back next Monday night with a report on my findings with /flushdb. I have a strong feeling your right about that.

As to why I think it's re-downloading to everyone upon client connection.....

I have attached a picture of Windows 7 resource monitor (address column I blurred). This is a standard tool with Windows 7 (I think all editions). When clients are connected you will see multiple entries (one for each client) in the Image column called FantasyGrounds.exe (not pictured in the attachment) and you can watch the Send column to see how much data is being transmitted to each client.

After all of the downloads finished to all of the clients (reading near 0 Send to everyone) I had ONE client disconnect, exit FG, re-connect. Upon re-connection of selected tester ALL clients showed Send activity in roughly the same quantity and duration that occurred just before we started the test. Thus leading to the conclusion that they are getting, what ever they got the first time, all over again.

As to the download counts....

While this download process is occurring the lag back to the clients is so serious that the game is unplayable. This is caused by my entire upload channel being completely maxed out, causing a delay to all client actions.

I do not believe this is entirely FG's fault as I don't think I'm getting my entire upload bandwidth(I'm guessing I'm getting half of what I should have). Likely caused by line noise. However, the bandwidth problem has become more apparent when the upload count increased as its taxing my reduced upstream more then it used to.

Clearly I need to talk to my ISP....but I think you might find anyone with less then 256k upsteam (not uncommon in the US) may be impacted by this upload increase. Though I wouldn't suggest worrying about it until a lot more people complain then just me.

BK

[Moon Wizard]

Another item to check is the size of the portraits uploaded by the players. You should be able to look at the file sizes in the campaign portraits directory.

Regards,
JPG

[DM_BK]

About 3 meg in the campaign I'm having issues with vs 200k in the campaign I'm not having issues with. That's something...I wiped them all out of the bad campaign....not to say they aren't going to come right back next session but maybe less now.

Does /flushdb work at any time? Can I execute it with no clients connected and expect it to work or does it work better when everyone is connected?

[Moon Wizard]

Yes, it works at any time. It just clears all the holder data in the database, except for ownership tags.

Regards,
JPG

[DM_BK]

We missed a week and I just now remembered that I was going to follow up on this thread.

Everything worked wonderfully. Actually better then it has in many months as there was no lag when a new client connected. Even on a good day we used to have a few seconds of lag for everyone when a new person connected.

I monitored outbound network activity and saw very little sustained uploads going on now, /flushdb I'd say did trick. Mystery as to the cause but I'm just happy everything is working well now.

Thanks for the help,
BK