Trouble with FGU opening port, but not FGC.

[Sarteck]

When I have FGC running trying to do a LAN game, and I have Port 1802 forwarded, various port checkers can see it open just fine.
FGU, however, refuses to show as open when it's running in LAN mode.

I have Windows Firewall turned off for this test, so I know it's not the issue. What else could cause one application to be unable to listen on the port but not the other?

[LordEntrails]

FGC uses TCP protocol on port 1802. FGU when using a LAN game uses UDP protocol on port 1802.

Most of the online port checkers only check ports using TCP protocols, therefore they will not show FGU availability.

[Sarteck]

Ah, I see, I didn't even think that the online tools could only check one protocol. All right, thanks, that makes sense. Glad I'm not going nuts.

[Ludi]

Some online tools can check UDP (we found ipvoid.com/udp-port-scan but I am sure there are others). However, you might still be going nuts, because my group did.

Yesterday we had serious issues with matchmaking services and we could not use the cloud setting, so we tried to get local mode running. We have been facing the same issue, with me hosting on a Mac with a provider that has no IP v4 ranges anymore. It took us the whole evening to try to find a proper configuration. We were not experienced in IPv6 and had to do a lot of trial and error. In the end, we had found the correct IP and opened port 1802 for both TCP and UDP, and checked with third party tooling.

We had the same final result: the tools (command line and online) saw an open UDP protocol port 1802 for the correct IP v6 address (and TCP as well, because who knows), but Fantasy Grounds Unity in yesterday's and the June 4th version (prev channel) refused to work. We do not know whether older versions would have worked because we never bothered. The messages in the logs seemed to be inconclusive. We were stuck. And this was from a group of IT professionals (with some serious administration skills, albeit not mine, my developer days are long past). We first suspected my new router, or our Internet provider (who assigns only IP v6 and tunnels IP v4), or my Mac settings, but in the end we fixed it for everything except Fantasy Grounds.

Has someone managed to get this (IPv6 local mode) running properly in FGU?

[Zacchaeus]

No, I can''t get anyone to connect in LAN mode either in Unity although Classic works just fine.

I can see my port is open using netstat but something is still preventing connection. I am on IPv4 network. The only thing I haven't tried is phoning up my ISP to see if there is something on their router software which prevents UDP port forwarding from working properly. I have seen some reports on Discord where people have successfully connected using Port 1803 or Port 1804; but that kinda baffles me as to why that would work and 1802 doesn't.

[mlbrown]

Have you tried with Hamachi? That should restrict the issues to the local machines. If that works, you know it is something with the ISP or the router. On my router forwarding defaults to TCP only.

[greytiger10]

I was able to open mine up by opening the port in my Windows Firewall and on my router (Netgear Orbi). I'm also running my router in an advanced DMZ feature that my ISP (Bell-Aliant) has in their modems. I would assume running a router behind a regular DMZ on an ISP's modem/router (unless you can run their modem in bridge mode). The setup works very well for me; it even allows multiple game consoles to run with open NAT's.

[damned]

My Unity Port Forward works.

[Zacchaeus]

My Unity Port Forward works.

:bandit:

[Ludi]

Meanwhile, we tried a bit harder. Different ports - did not matter, and tools report that the host's UDP ports are open and accessible. However, we found no evidence (e.g. via netstat) that FGU even tries to listen to an IPv6 port.
It opens a tcp6 connection for https, and listens with udp4 protocol... only. Is IPv6 even supported? Is there a hidden flag somewhere? The port IS definitively open in the router (it's a premium router, and not that difficult).
We did not try Hamachi yet, but thanks for the tip. This is only a workaround, however, and might require some serious effort from our not-so-technical players to setup.
We will dig deeper if time permits, and in the meantime hope for patches, or pray that the matchmaking will work next Saturday.

[damned]

Make sure your server is set to LAN

[greytiger10]

Ludi, how does your router connect to the internet? Is it through an ISP modem/router and if so is it in a DMZ or in bridge mode? It is possible that if your router is just connected to an ISP modem that does routing that its getting a double NAT and being stopped there.

[Ludi]

My cable router has no bridge mode, the FAQ claims this is to be able to provide telephony functions. It runs in DS Lite (Dual Stack), so the primary connection is via IPv6 and all IPv4 are tunneled, and there would be one NAT instance for outgoing IPv4.
If you get an IPv6 request, there is no NAT. NAT is rarely if ever used in IPv6. In IPv6, you do not forward ports, you open them (firewall-alike).

We verified with external scanners which is tricky for UDP ports (ipvoid.com/udp-port-scan was one of the few with proper support; it was not sure and replied with open|filtered). I can emulate the proper behavior with netcat (nc -v6ulk <IPv6> 1802 which means verbose, IPv6, UDP, listen, keep-alive) and check that port: nc -v6unz returns "succeeded" or netstat -av | grep udp6 shows the process id of the first netcat and confirms that concomp1 is open ("concomp1" is the application registered for port 1802, but that is a NASA graphical processing tool; essentially, 1802 is not a bad choice, the port should be free).
Now FGU comes along - I tried with the latest update, just a few minutes ago; I start in LAN mode (doh), the IPv6 address and port it is going to use match my opening rule. According to netstat, FGU does not listen for udp6. The port scanner diagnoses "closed" (noone is replying), and netstat shows that the FGU process uses tcp6 (for https) and udp4 (for who knows what). There is no udp6 protocol in use from FGU (the only uses are from mDNSResponder).

For me, it looks like FGU does not bother with UDP via IPv6. I am glad for any ideas what to try next, I am not really a networking professional and IPv6 is also a bit unusual, 25 years after its definition. Of course, a VPN might work, or maybe the matchmaking service (Cloud mode) will work coming Saturday. But we are 5 IT professionals (not all network experts, though; I am not), and we believe that this problem has stats and hence can be killed. I am also ready to talk to my ISP but it will not be easy to get to a technician who knows enough.

[damned]

I loaded FGU just now and ran:

netstat -p udpv6 -b -a

and got

Active Connections

Proto Local Address Foreign Address State
UDP [::]:123 *:*
W32Time
[svchost.exe]
UDP [::]:500 *:*
IKEEXT
[svchost.exe]
UDP [::]:1802 *:*
[FantasyGrounds.exe]

Disclaimer - I have never tried IPv6 connections but this looks like it is listening...

[Ludi]

Uh, the Macos version of netstat is a bit limited, it does not provide nice port and application names. However, "lsof -P -iUDP" does something useful.
There was only an IPv4 version open. However, I saw something interesting: Port 1802 was used in IPv6 by udpdump. This is apparently part of wireshark which I installed just yesterday (getting desperate... not that I can properly use that tool).

I killed the two udpdump processes and now I get something better (cut out my username):
Fantasy 78814 [...] 73u IPv4 0x87ba989b4d06c715 0t0 UDP *:1802
Fantasy 78814 [...] 74u IPv6 0x87ba989b4d06c9fd 0t0 UDP *:1802

And there it is - IPv6 UDP from FGU. My hypothesis was wrong (which is good).

I am going to verify where this udpdump came from (it did not work before I installed wireshark either - weird), and whether it is now working after a reboot and for my players. I have to wait till evening, we are CET.

Thank you for your assistance, anyway.

[Ludi]

Half a step further by now, but still no cigar.

I have now made my computer reachable from the outside via UDP portscan (ipvoid.com/udp-port-scan can check it in IPv6), via ping (www.ipv6now.com.au/pingme.php can check it in IPv6), and find it with traceroute (ultratools.com/tools/traceRoute6). The port is open (or at least, open-or-filtered, which changes to closed when I quite FGU, so there is at least an observable state).

However, my players still cannot connect. I tried contacting my ISP to see whether they do some strange filtering. All my players share the same provider, it's the most prominent cable provider in our region. So far I got no useful reply. I will try get on the nerves of my ISP a bit more, maybe I can get some competent people on the line.
I played around with a VPN but Hamachi is rather invasive in Mac (or I did not get how to use it) - I could not get it to release the IP address it highjacked without deinstalling it.

[Trenloe]

I have now made my computer reachable from the outside via UDP portscan (ipvoid.com/udp-port-scan can check it in IPv6), via ping (www.ipv6now.com.au/pingme.php can check it in IPv6), and find it with traceroute (ultratools.com/tools/traceRoute6). The port is open (or at least, open-or-filtered, which changes to closed when I quite FGU, so there is at least an observable state).
How were your players trying to connect?

If you'd like me to try to connect, PM me your IPv6 address and have your FG server running in LAN mode ready for connections.

[Ludi]

With the demo clients, also IPv6, same provider.
I just reached a tech guy, and since I already had a pricey option they will provide me with a cost-free dual stack solution now. This might take up to two hours.

Thanks a lot for your offer, I can also ask one of my players when he's back from office to try to connect. Luckily, I have a couple of days off, so I have some time to deal with that config hell.

[Ludi]

So, I now own a proper dual stack connection. My provider (Vodafone) removed the DS-Lite and I now have an IPv4 as well.
And, see: everything works just fine, people can connect.
With IPv4. And only with IPv4. Not with IPv6. We tried. The 1802 port for UDP is open and accessible. One of my players installed a tool to send UDP packages and we tried. His messages arrived in my console (running a nc -v6ulk <MyIP6> 1802) and I saw his messages. Fantasy Ground Unity does NOT connect. There is nothing in my log, or in his log (he did activate the debug log mode). It simply does not connect.
We did not try with more sophisticated tooling to see whether his FGU client (free version, I have ultimate license) does send anything.

Client was Windows (not sure which one, I would assume Windows 10), I as host was using Macos (10.15.5), with the latest version of FGU (updated today).

[Tuleen Donai]

So, I now own a proper dual stack connection. My provider (Vodafone) removed the DS-Lite and I now have an IPv4 as well.
And, see: everything works just fine, people can connect.

Awesome! Glad you were able to get THAT fixed!! ISPs can sometimes be "problematic" when resolving these kinds of issues.



With IPv4. And only with IPv4. Not with IPv6. We tried. The 1802 port for UDP is open and accessible. One of my players installed a tool to send UDP packages and we tried. His messages arrived in my console (running a nc -v6ulk <MyIP6> 1802) and I saw his messages. Fantasy Ground Unity does NOT connect. There is nothing in my log, or in his log (he did activate the debug log mode). It simply does not connect.

This sounds like the packets are getting blocked somewhere still. Could be a port/protocol/IP combination or something. There might be an authentication protocol before the data protocol that's getting blocked. I had that happen years ago with a Cisco VPN, and trying to bore holes in my firewall. I had to bore 3 - I can't remember the details, but the last one was the authentication protocol.

(VPNs are so much easier to get working today than then.)

[Sulimo]

So, I now own a proper dual stack connection. My provider (Vodafone) removed the DS-Lite and I now have an IPv4 as well.
And, see: everything works just fine, people can connect.
With IPv4. And only with IPv4. Not with IPv6. We tried. The 1802 port for UDP is open and accessible. One of my players installed a tool to send UDP packages and we tried. His messages arrived in my console (running a nc -v6ulk <MyIP6> 1802) and I saw his messages. Fantasy Ground Unity does NOT connect. There is nothing in my log, or in his log (he did activate the debug log mode). It simply does not connect.
We did not try with more sophisticated tooling to see whether his FGU client (free version, I have ultimate license) does send anything.

Client was Windows (not sure which one, I would assume Windows 10), I as host was using Macos (10.15.5), with the latest version of FGU (updated today).

Quick question for you, since you are using a Mac to host, and I recall someone else having issues previously. Does you Mac have both LAN and WLAN, and if so are both connected at the same time?

As I recall, that caused problems because for whatever reason macOS saw traffic coming in on one connection, but was responding on the other (something to that effect anyway).

[Ludi]

This sounds like the packets are getting blocked somewhere.

Maybe. They did not from command line via router to internet via router to command line. However FGU via router to internet via router to FGU did not do anything.
The base software should be identical between Demo and Ultimate license, right?

Maybe we could install some network traffic monitoring tool to see whether FGU is sending something on the client machine. But not sure which one - would need to monitor an outgoing UDP stream via IPv6 for Windows.

[Ludi]

Does you Mac have both LAN and WLAN, and if so are both connected at the same time?

My Mac gets several IPv6 addresses, local ones and global ones, and so does the router (well, actually it seems to have only one). There is no need for NAT in IPv6, but selection of which address is a topic in itself (there is an algorithm to select the default address). Luckily, FGU tells you which one it wants. That's the one I opened (actually, you use only the latter parts - "interface", the first parts "site" and "subnet" are provided by the router, or something like that).

Also, with my traceroute experiments from a service in Australia (can't get farther away than that, mate) I made sure that this IPv6 is found (and the hop before it was the router). Extra allowances in the router config were needed that my machine would be present, but then it worked.

Also, it does work with command line from a different subnet (albeit same ISP), using a low level UDP tool, Windows-to-Mac. If there was some schizophrenia effect or misrouting in place, it would also affect that connection, wouldn't it? Hence, I would bet that something is going on inside the Unity code. Unfortunately, the log level does not provide enough information to find out what is going wrong; maybe with info or debug levels we could find something?

[Tuleen Donai]

The base software should be identical between Demo and Ultimate license, right?

I would imagine they are identical. can you imagine what a nightmare it would be maintaining two separate versions/code bases, etc?



Maybe we could install some network traffic monitoring tool to see whether FGU is sending something on the client machine. But not sure which one - would need to monitor an outgoing UDP stream via IPv6 for Windows.

The problem is two-fold that I can see:

1) you really don't know all the network transactions FGU needs to set up a valid connection. You could try doing this with two local machines, with your monitors and see what you can see.
2) you also don't really have any way of identifying what packets were exactly dropped by your ISP (or Internet connection.) It's tough, because you have to be able to see all the packets of a successful connection, and compare those to an unsuccessful one.

You could potentially take the machines physically somewhere else (like Starbucks or something) and see if you can establish a successful connection and record the "conversation." Then return "home" and compare that against the SAME setup, but examining the unsuccessful converation.

[Ludi]

can you imagine what a nightmare it would be maintaining two separate versions/code bases, etc
I can. And they already have that: FGC / FGU. Windows / Linux / MacOS. Stable / Live / Test / Dev. Different branches. Software development is hard, and developing a game is one of the hardest undertakings ever. These users have highest expectations (it should not only work, it needs to be FUN), will try the hardest to break your stuff (cheaters), and are on very very different skill levels (noobs to nerds), in a very competitive field.

I have experience in IT for automotive (embedded stuff), finance (server security), logistics (com-pli-ca-ted algorithms) and I believe gaming industry might be the most demanding of them all.


The problem is two-fold that I can see:
I believe you are correct, that would work. We'll see, I still have a few days off, so I might dust off my old Macbook (the slim, slow one) and try something out.
However, it sounds like ... work?

[Tuleen Donai]

I can. And they already have that: FGC / FGU. Windows / Linux / MacOS. Stable / Live / Test / Dev. Different branches. Software development is hard, and developing a game is one of the hardest undertakings ever. These users have highest expectations (it should not only work, it needs to be FUN), will try the hardest to break your stuff (cheaters), and are on very very different skill levels (noobs to nerds), in a very competitive field.

I have experience in IT for automotive (embedded stuff), finance (server security), logistics (com-pli-ca-ted algorithms) and I believe gaming industry might be the most demanding of them all.


You are ABSOLUTELY correct sir! I tried my hand at coding for gaming yarns ago (1990 timeframe,) and frankly, it overwhelmed me. I've poked at SWGEmu code, but that's server side not client side, and it's still ... Whew!!!

I used to use PC games to evaluate embedded PC Motherboards because they pushed ALL the aspects of an embedded PC so hard.

The last game I actually finished coding was a Hang the Ayatollah game on a Commodore Color PC back in 1979. My best friend at the time did Baby Seal Killer and the Henny Youngman game. Consequently, we got A's in Computer in High School. :o

[Ludi]

Now I can reproduce the connectivity issue on my own. I got my old Macbook running.

First, I tried connectivity within the local network - fine. Then, via internet: tethered my notebook to my phone, deactivated WLAN there to force LTE, which is a different provider than my ISP. Also fine.
I made sure that connectivity is there in both directions with a (echo "Welcome"; cat) | nc -v6ulk <MyIPv6> 1802 - works in both directions, I get messages via that port through my router.

Then, I started FGU with a new user and the demo client on my Macbook, joining my own game that I set up on my iMac.
FGU tries to connect, and never stops trying. There is nothing in the client logs, whatsoever.

Then, I closed the host and ran the command line listener again. The client sends a base64 message to my listener, identifying itself (decoded it reads "Fantasy GroundsSmiteWorks2019.2.15f11.2.8"). So, the client can, and does, initiate a handshake - which the host version apparently does not answer. (Corona I guess? No more handshaking?)

In the server log, I got a series of "Client connected. Waiting for authorization." in the console.log. I did not see that one before (maybe I was not looking hard enough, or it never had connectivity before). Finally, something. There are 4 attempts, each of them about 1.5 seconds in between. This is not enough. Please try harder, longer (Hystrix, Polly, whatever floats your boat).

[6/26/2020 10:30:11 AM] Client connected. Waiting for authorization. [1]
[6/26/2020 10:30:12 AM] Client connected. Waiting for authorization. [1]
[6/26/2020 10:30:14 AM] Client connected. Waiting for authorization. [1]
[6/26/2020 10:30:16 AM] Client connected. Waiting for authorization. [1]
[6/26/2020 10:32:45 AM] NETWORK COMMAND STATS [1]: 1 (1) / 0 (0)
[6/26/2020 10:34:33 AM] Campaign saved.

One minor side concern: please trim the IP input field; it is very easy to copy an additional line feed or blank with the address, and then FGU complains about now knowing that IP.

Finally, I used my newly obtained IPv4. It requested the password and let me in. Just like that.
[6/26/2020 10:36:43 AM] Client connected. Waiting for authorization. [2]
[6/26/2020 10:36:43 AM] Authorization failed: Incorrect password.
[6/26/2020 10:36:59 AM] Client connected. Waiting for authorization. [3]
[6/26/2020 10:37:00 AM] '<MyTestUser>' connected

So, maybe the password? I had one set for the cloud, but now I restarted the host, removing the password, trying again with IPv6.

[6/26/2020 10:41:29 AM] Client connected. Waiting for authorization. [1]
[6/26/2020 10:41:30 AM] Client connected. Waiting for authorization. [1]
[6/26/2020 10:41:32 AM] Client connected. Waiting for authorization. [1]
[6/26/2020 10:41:33 AM] Client connected. Waiting for authorization. [1]
[6/26/2020 10:43:11 AM] NETWORK COMMAND STATS [1]: 1 (1) / 0 (0)

Well. It's not the password.

I think this concludes the part where I can find out things. I would need source code now.

[Ludi]

I think this concludes the part where I can find out things. I would need source code now.
Well, wrong. I found out a bit more, daring to run a tcpdump. Astonishingly enough I managed to find something after guessing the interface and switches and learning how the expressions work.
I saw my notebook (client) sending an UDP package to the desktop (host) using the IPv6 as selected by FGU. The reply package sent to the same IP / port that the client used was sent using a different IPv6. This IPv6 was higher in the device list of my router.

I am far out of my league here, so this is me wildly guessing (again): FGU selects one address according to the IPv6 address selection algorithm. The router / OS uses a different one. UDP is connection-less, so it does not know who is sending here. So the host replies under a different IP, but the client does not know where this is from and does not answer correctly. In IPv4 you would now probably start with NAT, or assign static addresses. I found no such mechanism in my router, and no way to get rid of extra IPv6 addresses or to modify priorities. Not even sure that this is indeed the core issue.

Anyway, this is just for sports. I have an IPv4 now and it works fine. Still bothering me I cannot get it to work with IPv6.

[Tuleen Donai]

I'm trying to picture all of this. Can you perhaps draw a diagram of all the connections, from your client to the cloud, from the cloud to your router, from your router to the server? Label all links with the IPv6 addresses used, or maybe some facsimiles to protect the innocent, but still representing what you are seeing. I'm trying to determine where the IPv6 address changed and why?

Can you tell me what machine your server is? Windows 10 desktop? If so, can you do a IPCONFIG /ALL and again slightly obfuscate the IPs to protect the innocent?

What router do you have? and how much access to you have to its configuration? When you say that the IPv6 address was higher in the device list of your router, what exactly do you mean? Are you talking DHCP?


(Networking issues can be such a pain-in-the...)

[Ludi]

internet (ISP: vodafone; since yesterday full dual stack) --> router (FRITZ!Box 6591 Cable; IPv6 ends with :9939) --> device (iMac, Catalina 10.15.5)
According to my router, the iMac has gotten 4 IPv6: fe80::xxx:xxxx:xxxx:eb6e, 2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:518a, 2a02:xxxx:xxxx:xxxx:xxxxx:xxxxx:xxxx:96fc, 2a02:xxxx:xxxx:xxxx:xx:xxxxx:xxxxx:bfe4. My notebook (client) currently has 8 IPv6 listed, my phone has 5 at the moment.
Fantasyground selected the :bfe4 as external IP. I configured the router to open port 1802 for that one.
According to tcpdump (tcpdump -i en1 udp and port 1802 and ip6 -n -v), the notebook (:a7c9) sends UDP packages to the specified host (:bfe4). The host sends a package back, but under :96fc (which at that time was #1 after the fe80 address).

As far as I understand, devices get more than IPv6 address all the time, for different prefixes and so on. There is supposed to be an algorithm to select one (rfc 6724) to be used as default. That's what FGU selects, I would assume. This is part of DHCPv6.

ifconfig tells me (xxxx inserted):


lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
options=1203<RXCSUM,TXCSUM,TXSTATUS,SW_TIMESTAMP>
inet 127.0.0.1 netmask 0xff000000
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
nd6 options=201<PERFORMNUD,DAD>

gif0: flags=8010<POINTOPOINT,MULTICAST> mtu 1280

stf0: flags=0<> mtu 1280

en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
options=50b<RXCSUM,TXCSUM,VLAN_HWTAGGING,AV,CHANNEL_IO>
ether 98:10:e8:f2:ce:22
inet6 fe80::c6a:f57c:ac0c:bf06%en0 prefixlen 64 secured scopeid 0x4
inet 169.254.64.90 netmask 0xffff0000 broadcast 169.254.255.255
nd6 options=201<PERFORMNUD,DAD>
media: autoselect (1000baseT <full-duplex,flow-control>)
status: active

en1: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
options=400<CHANNEL_IO>
ether 98:9e:63:39:1e:9e
inet6 fe80::83d:e152:d22f:eb6e%en1 prefixlen 64 secured scopeid 0x5
inet 192.168.178.20 netmask 0xffffff00 broadcast 192.168.178.255
inet6 2a02:xxxx:xxxx:xxxx:xx:xxxx:xxxx:bfe4 prefixlen 64 autoconf secured
inet6 2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:96fc prefixlen 64 deprecated autoconf temporary
inet6 2a02:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:518a prefixlen 64 autoconf temporary
nd6 options=201<PERFORMNUD,DAD>
media: autoselect
status: active

en2: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICA ST> mtu 1500
options=460<TSO4,TSO6,CHANNEL_IO>
ether 82:1f:e6:e4:28:00
media: autoselect <full-duplex>
status: inactive

en3: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICA ST> mtu 1500
options=460<TSO4,TSO6,CHANNEL_IO>
ether 82:1f:e6:e4:28:01
media: autoselect <full-duplex>
status: inactive

bridge0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
options=63<RXCSUM,TXCSUM,TSO4,TSO6>
ether 82:1f:e6:e4:28:00
Configuration:
id 0:0:0:0:0:0 priority 0 hellotime 0 fwddelay 0
maxage 0 holdcnt 0 proto stp maxaddr 100 timeout 1200
root id 0:0:0:0:0:0 priority 0 ifcost 0 port 0
ipfilter disabled flags 0x0
member: en2 flags=3<LEARNING,DISCOVER>
ifmaxaddr 0 port 6 priority 0 path cost 0
member: en3 flags=3<LEARNING,DISCOVER>
ifmaxaddr 0 port 7 priority 0 path cost 0
nd6 options=201<PERFORMNUD,DAD>
media: <unknown type>
status: inactive

p2p0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 2304
options=400<CHANNEL_IO>
ether 0a:9e:63:39:1e:9e
media: autoselect
status: inactive

awdl0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1484
options=400<CHANNEL_IO>
ether d2:b9:33:b8:35:38
inet6 fe80::d0b9:33ff:feb8:3538%awdl0 prefixlen 64 scopeid 0xb
nd6 options=201<PERFORMNUD,DAD>
media: autoselect
status: active

llw0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
options=400<CHANNEL_IO>
ether d2:b9:33:b8:35:38
inet6 fe80::d0b9:33ff:feb8:3538%llw0 prefixlen 64 scopeid 0xc
nd6 options=201<PERFORMNUD,DAD>
media: autoselect
status: active

utun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1380
inet6 fe80::4359:9633:ddc3:3782%utun0 prefixlen 64 scopeid 0xd
nd6 options=201<PERFORMNUD,DAD>

utun1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 2000
inet6 fe80::26b6:a0c8:f079:726%utun1 prefixlen 64 scopeid 0xe
nd6 options=201<PERFORMNUD,DAD>

[Ludi]

37185

[Imagix]

I dunno, the description made sense to me :) Unless Unity is doing some funky stuff under the covers (like sticking a bunch of connection IDs or something in the payload), I would have expected the returning traffic to originate from :bfe4. That FG/Unity is choosing to use a different source IP (which is also both temporary and deprecated...) is concerning (to me).

[Ludi]

Not sure who is doing the chosing. FGU is probably using the Unity implementation (rUDP), which has to call OS network primitives sooner or later.
Apple apparently had some issues with IPv6 and UDP in iOS back in 2017. Not sure who or what to blame. I would look forward to a solution that is not "fall back to IPv4".

Damnit, the definition of IPv6 is 25 years old, and still the infrastructure is not bullet-proof and easy to use. Us IT guys suck.

[Tuleen Donai]

As far as I understand, devices get more than IPv6 address all the time, for different prefixes and so on. There is supposed to be an algorithm to select one (rfc 6724) to be used as default. That's what FGU selects, I would assume. This is part of DHCPv6.

I think that your close, but the way I understand IP, whether it's IPv4 or IPv6, is that an IP address is assigned to an interface - either a hard interface like a wired ethernet port or a wireless ethernet radio, or a soft interface like the endpoints to a tunnel, or virtual interfaces that are created by hypervisors like VMWare or VirtualBox. An interface can have BOTH an IPv4 address and an IPv6 address, and in the case of IPv6, an interface can have multiple IPv6 addresses, but I DO think they need have a quality of uniqueness as far as scope or weighting (preferred vs. deprecated as described in the RFC.)

IF there are multiple connections to the Internet from a machine in IPv6 land, then I read that rfc 6724 describes the DEFAULT behavior of any IPv6 implementation (this is typically in the OS of the machine - not the application) for selecting source and destination addresses for a communications channel. The RFC goes on to say that Applications can choose to override these defaults, and so can network administrators. The problem I can see with all this is STATE. This is an IP condition that is very relevant for firewalls, routers and switches. These entities make decisions about allowing traffic in/out of interfaces.

Typically, if a connection is established from an end-point to an outside server, then the STATE of that connection is saved by the firewall, and when the server responds, that packet or series of packets is passed through to the client. If someone outside the firewall, tries to establish a NEW connection to the client, typically, for safety reasons, that packet is BLOCKED.

Do you see what I'm getting at?

If the FGU Server is trying to establish a connection to the client using a NEW IPv6 address, why wouldn't that exchange get blocked? To me, it makes no sense that the Server would try to communicate to the client with more than one IPv6 address. It CANNOT do that with IPv4 - IMHO.

[Tuleen Donai]

Have you tried connecting to your FGU Host from outside your ISP and Router??

The diagram you provided shows both the client and the server on one side of the Router/Firewall.

[Ludi]

Have you tried connecting to your FGU Host from outside your ISP and Router??
Yes, tethered my Notebook to my phone and turned off the phone's WLAN, forcing it to go LTE via T-Mobile. Worked for IPv4, did not work for IPv6.
Also IPv6 does not work for my players, but IPv4 does.
I can do more experiments with my own hardware, however, so I documented my current setup.

[Tuleen Donai]

Yes, tethered my Notebook to my phone and turned off the phone's WLAN, forcing it to go LTE via T-Mobile. Worked for IPv4, did not work for IPv6. Also IPv6 does not work for my players, but IPv4 does.

Yes, as I understand it, IPv4 works for you, as I would expect. I think the issue is something subtle with IPv6, your ISP, and/or your Router, and/or FGU.

Can you try reversing the roles? i.e. connect your FGU Server to the phone's WLAN, and connect your client MacBook to your Router/ISP.

See what happens then? (Then we know if it's a client role or server role issue.)

[Ludi]

Do you see what I'm getting at?
I believe I do. Still, there is only so much I can influence. The response package might be blocked as you assume (I cannot see any indication for that, but also I cannot be sure it that it goes through), or it might simply be not delivered (because the rUDP might not expect a response from that IP?).
I have no idea how to prevent that. I just tried to add manual port allowances for all alternative IPs of my host, but it did not do anything.

[Ludi]

Can you try reversing the roles? i.e. connect your FGU Server to the phone's WLAN, and connect your client MacBook to your Router/ISP.
That was such a crazy idea, I had to try it. It was a bit slow, but finally FGU host started via LTE. Unfortunately, the IPv6 connection (address ...:bbc6 in case we want to refer to it) also failed. There is no message in the console.log this time, so I assume the port is blocked, and iOS does not really support all that much in terms of port forwarding / opening.

[Imagix]

Say: here's another thing to try. Run in a terminal:

sudo sysctl -w net.inet6.ip6.prefer_tempaddr=0

And then try to relaunch FGU and try to connect and see if the responses now prefer the permanent address.

Note: this does appear to be a system-wide change and thus will change the behaviour of all programs and may have privacy/security implications.

[Ludi]

Imagix, my man - that was it. "Acquiring file list."

Damn. That was some serious magic there.

So, now, what did I destroy with the little kernel setting? (previous value was 1, so I can probably set it back)

[Imagix]

This article will save me some typing (and how to make it permanent... it will go away if you reboot right now):

https://slaptijack.com/networking/osx-disable-ipv6-address-privacy.html

TL;DR: Your "permanent" address is fairly permanent and predictable. This may allow sites that you connect to to possibly correlate your visits using that predictable IP. The temporary addresses are somewhat randomized which would make that harder.

Given that the FGU server needs to have its address "published" somewhere... it sorta needs to be predictable, so that concern doesn't apply. The issue here is that the sysctl is a systemwide change and thus your browser will also be affected.

Treat this solution as a workaround until Smiteworks can adjust the code to have FGU respond from the correct IP. I wouldn't want to keep it permanently.

[Tuleen Donai]

Guys, so this is AWESOME, but I don't quite follow what that command did??

Can you elaborate?

Edit: I read a bit of that article:


OS X: Disable IPv6 Address Privacy
November 13, 2015 by Scott Hebert

For those that are really into privacy, the Privacy Extensions defined in RFC 4941 are a really good thing. This extension circumvents SLAAC and has the result of randomizing your IPv6 address. Like I said, if privacy is a big deal for you, this is definitely something you want enabled. Fortunately for you, this feature is enabled by default in OS X.

Unfortunately, I ran into a situation recently where my ever-changing IPv6 address was causing a bit of a problem for debugging. My primary interface had more than half a dozen temporary IPv6 addresses assigned.


Holy obfuscation Batman!! I had NO idea that IPv6 would do this!! Change IP addresses for every connection attempt?? Crazy!!!

[Ludi]

Imagix is the expert, but from what I understand: this is a kernel setting that (de)activates a privacy feature that creates new IPv6 addresses so that they are rotated around a bit and you are a harder target for attackers.
The ambiguity of the addresses is a security feature according to the link that Imagix provided. Deactivating it prevents the schizophrenia effect I have observed, but it also reduces security a bit.

[Imagix]

OK, though take some of this as speculation/suspicions as I don't work for Smiteworks, nor have I seen their code, nor have I programmed in Unity. (Though I have been programming in the networking world for nearly 30 years though).

As noted elsewhere, the computer will have multiple IPv6 addresses, and FGU/Unity uses UDP to communicate. Since UDP is connectionless, every packet gets treated separately. Speculation: FGU/Unity is opening its socket to listen for UDP traffic using "::" as the IP (which means, listen to all). Where this is more important, is when FGU needs to send a UDP packet somewhere. That packet gets handed off to the kernel to send to a destination. Since the socket wasn't bound to a particular IP, the kernel needs to choose a source IP to use. Each of the IPs (at least the routable ones.. and in the example there are 3) are equally good to use. With the kernel setting set to 1 (the default), the kernel will choose to use a temporary address because it is "safer" (from a privacy point of view at least). Changing that kernel setting to 0 instructs the kernel to choose the permanent address.

[Tuleen Donai]

OK, though take some of this as speculation/suspicions as I don't work for Smiteworks, nor have I seen their code, nor have I programmed in Unity. (Though I have been programming in the networking world for nearly 30 years though).

As noted elsewhere, the computer will have multiple IPv6 addresses, and FGU/Unity uses UDP to communicate. Since UDP is connectionless, every packet gets treated separately. Speculation: FGU/Unity is opening its socket to listen for UDP traffic using "::" as the IP (which means, listen to all). Where this is more important, is when FGU needs to send a UDP packet somewhere. That packet gets handed off to the kernel to send to a destination. Since the socket wasn't bound to a particular IP, the kernel needs to choose a source IP to use. Each of the IPs (at least the routable ones.. and in the example there are 3) are equally good to use. With the kernel setting set to 1 (the default), the kernel will choose to use a temporary address because it is "safer" (from a privacy point of view at least). Changing that kernel setting to 0 instructs the kernel to choose the permanent address.

I'm hoping one of the devs is reading this, and this REALLY should get addressed in FGU as a "bug."

Doesn't affect me, because my network, router, and ISP connections are all IPv4, but I imagine more and more folks with more mainstream ISPs like Vodaphone, will run into this issue. And especially those folks running OS X on Macs.

It would be interesting to find out if this could happen under Windows 10.

[damned]

FGU does listen on ::
The behavior you describe is probably what is happening.

[Ludi]

It would be interesting to find out if this could happen under Windows 10.

At least in theory, the problem could occur there as well. But I do not have a windows system at hand, so I cannot reproduce actual behavior.

Temp IPv6 Addresses are available since Windows 7. You can disable them (Windows 7-10) with

netsh interface ipv6 set global randomizeidentifiers=disabled

netsh interface ipv6 set privacy state=disabled

[Moon Wizard]

I'm not sure I'm following the issue completely. Some questions:
* Are you saying that an adapter using temporary IPv6 addresses may not be able to connect; but disabling the temporary IPv6 address system for the adapter allows it to start working?
* Also, is this specific to GM server, player connect, or both? Which one did you disable above to get it working (GM or player)?
* Is this for Cloud server mode, LAN server mode, or both? Which one were you using above when you got it working?

We don't have any control over which adapters listen (all for server); and which adapter requests (request connection through library; which goes through multiple layers that ultimately request socket from OS). However, if that's the case, it may give me a pointer that I can share with network library developer to help investigate.

I've seen an occasional pause on connection resolution for the Mac; but not on Windows. I'm wondering if it's something specific to the Mac IPv6 temporary addressing scheme that is different from Windows.

Thanks,
JPG

[Imagix]

Gm side. I’d be happy to discuss through other mechanisms (Discord DMs, or some other mechanism)

[Moon Wizard]

Yeah, you can PM me here or on Discord; and I can provide my direct e-mail if needed there too.

JPG

[Ludi]

I can provide more details to reproduce the issue if you need further information or tests. But not this evening, we actually planned to play.

[Moon Wizard]

I just need to understand the exact scenario that was problematic, and which the IPv6 temporary address shutoff fixes. (Which machines? Which OS? Which server mode?)

Regards,
JPG

[Ludi]

Post #31 contains a partially helpful diagram I drew which contains the technical data you need. The issue can also be reproduced with the standard setup (clients external to the router), but the core issue happens with UDP pairing on IPv6 at the host's side when using LAN mode.

[Imagix]

So my distilled understanding: MacOS, LAN mode. Ludi's machine has 3 routable IPv6 addresses on their interface. One non-temporary, and two temporary. FGU seems to have the presence of mind to see the non-temporary IP and shows that in the UI. Clients attempt to connect to that IP. Ludi saw the traffic arrive to that IP address, but the returned data (from server to client) originated from one of the temporary IPs. This will, of course, likely confuse whatever NAT gateway the client is behind since traffic originating from that IP wouldn't be associated with any current connection.

My theory is that FGU is binding to all interfaces, and as a result FGU isn't controlling the source IP, but is leaving it up to the OS to pick. (remember, this is UDP, and thus is connectionless). By default, the OS is choosing to use a temporary address. When we told the OS to prefer to use the non-temporary address, this resolved the problem.

[Moon Wizard]

I'm not following the flow on Ludi's picture exactly. But, based on Imagix's comments, it sounds like this is specific to LAN mode when hosting on a Mac, correct?

I would think that if the IP addresses both point to the same adapter that it shouldn't matter which one gets the packets. Unless Imagix's theory that return packets from a different IP address are being ignored/blocked.

The code to pull the IP address for the UI is actually independent of the network library at this point, since it was written before we switched libraries. So, I would need to look into how the network library chooses which adapter to use. At this point, I'm not sure if the network library offers a way to know the correct IP address to give the users (original or temporary, and which temporary) before the host is started; or to specify that only certain adapters should be used (I believe it is handed to OS). One of the concerns with overriding the default behavior is that some systems have adapters which do not have Internet access, so it's not a trivial "just pick one" scenario.

Regards,
JPG

[Ludi]

My first attempt was more like a napkin sketch. I took some more time now.
I hope this explains the observations better.

37198

[Ludi]

this is specific to LAN mode when hosting on a Mac
Plus IPv6. Catalina might have different behavior than previous MacOS. In theory, this might also occur under Windows and Linux, but I cannot reproduce this myself.


[...] it shouldn't matter which one gets the packets.
Speculation: the package might arrive at the client, but since the base UDP is connection-less, you need to pair them; if it comes from an unexpected source and there is no other transaction / connection ID involved, this might be impossible to match to a connection (rUDP level).


original or temporary, and which temporary
There is in fact, a standard algorithm for that (rfc number in the diagram). It should be deterministic, so all compliant implementations should produce the same results. As far as I can see the selection of the external IPv6 is legit.

[Tuleen Donai]

So my distilled understanding: MacOS, LAN mode. Ludi's machine has 3 routable IPv6 addresses on their interface. One non-temporary, and two temporary. FGU seems to have the presence of mind to see the non-temporary IP and shows that in the UI. Clients attempt to connect to that IP. Ludi saw the traffic arrive to that IP address, but the returned data (from server to client) originated from one of the temporary IPs. This will, of course, likely confuse whatever NAT gateway the client is behind since traffic originating from that IP wouldn't be associated with any current connection.

My theory is that FGU is binding to all interfaces, and as a result FGU isn't controlling the source IP, but is leaving it up to the OS to pick. (remember, this is UDP, and thus is connectionless). By default, the OS is choosing to use a temporary address. When we told the OS to prefer to use the non-temporary address, this resolved the problem.


From all the behaviors we've seen, I think Imagix is the best theory we have to go on. It makes sense to me. If FGU receives on it's permanent IPv6 address, it should respond from that address, non?

Otherwise, clients would need to respond to ANY IPv6 addressed messages they were sent, and that just "feels" like a security risk.

[Imagix]

Yep. There's a couple of ways this could be solved (from a programmatic standpoint): either the network socket needs to explicitly bind to the IP address that's shown on the initial screen (one shouldn't need to bind to the interface per se, but just to the IP address), or FGU needs to remember what IP the client was originally sending to and the network sending code needs to set the source IP directly on the outgoing packets.

I (and/or your networking developer) would need to know the specific networking library in use in order to look at the API to see how one might specify that information.

Oh, and I misspoke a little earlier. It's not a NAT gateway that would get confused (well, I suppose it could, but NAT is more commonly an IPv4 thing), I meant Stateful Firewall.

[Ludi]

Imagix is correct (as usual), it is hard to speculate without knowing the actual implementation and the full requirements.

According to stackoverflow, there are some traps with binding to any interface when you try to use dual stack sockets (how-to-support-both-ipv4-and-ipv6-connections), which FG/U might have good reasons to attempt. Since FG/U is also a multi-player application, you have to be able to juggle multiple virtual connections (in RUDP lingo), potentially using multiple threads? Depending on what the RUDP library in question is, this can be a fairly trivial or a rather frustrating exercise, I would guess. Better left to the professionals anyway.