Will FGU ever be updated to use Lua 5.4?

[houseofdexter]

Will FGU ever be updated to use Lua 5.4? Lua 5.1 was last updated in 2012.

[TheDarkMantle]

My suspicion would be probably not, or at least not any time soon.

Every game system, extension, module, and everything else that has lua scripts in it would probably need to be completely rewritten for 5.4. That is a lot of work.

[damned]

Moon Wizard did a small test to see the impact and it was considerable.
At this time there are many items that provide more value on their to do list.
So... maybe... but dont hold your breath.

[Moon Wizard]

There was a huge change between Lua 5.1 and 5.2 that breaks the entire way that function environments are defined in Lua. A couple years ago, I tried migrating the engine to Lua 5.3, but the function environment changes blocked that move. Because each object and script is expected to have its own environment in FG (as well as handler registrations), the function environments are required for the FG API to work. Changing that assumption breaks every ruleset and extension ever written. I tried the suggested workarounds I found in various forums and coding sites, but none of them were able to replicate that behavior.

I may re-assess at some point; but there are not really any pressing must-have enhancements I have seen in the Lua 5.2/5.3/5.4 change notes.

Regards,
JPG

[Weissrolf]

I would mainly wonder about performance improvements?

From https://lwn.net/Articles/826134/:


Version 5.4 also brings a new generational garbage collection (GC) mode, which performs better for certain kinds of programs where objects usually have a short lifetime.


One of the unsung features in 5.4, is a significantly faster interpreter, though the release notes have overlooked it. In a test I did on my 64-bit macOS machine using Gabriel de Quadros Ligneul's Lua Benchmarks suite, I found that version 5.4 was an average of 40% faster than version 5.3 across 11 benchmarks included in the suite:
https://static.lwn.net/images/2020/lua54-benchmark.png

From https://eklausmeier.wordpress.com/2020/05/14/performance-comparison-pallene-vs-lua-5-1-5-2-5-3-5-4-vs-c/



C (no optimization flags): 0.53s
C: 0.2s
Lua 5.1: 6.15s
Lua 5.2: 7.2s
Lua 5.3: 5.83s
Lua 5.4: 3.92s
LuaJIT: 0.58s
Pallene: 0.32s
Pallene with compiler optimization flags: 0.31s


Those who need much higher performance can use Mike Pall's LuaJIT, a just-in-time compiler for Lua 5.1 that is significantly faster than the stock Lua interpreter. However, LuaJIT hasn't added any of Lua's new features since version 5.1 (which was released in 2006).
At first I assumed that FG would use a JIT compiler and was rather surprised that it uses interpreted language only. So maybe have a look at this: https://luajit.org/

[Weissrolf]

LuaJIT also claims to speed up the interpreted parts:


For LuaJIT 2.0, the whole VM has been rewritten from the ground up and relentlessly optimized for performance. It combines a high-speed interpreter, written in assembler, with a state-of-the-art JIT compiler.

On the following site you can compare both the LuaJIT JIT compiler and interpreter with the standard Lua 5.1.5 interpreter:

https://luajit.org/performance_x86.html

For example: MD5 of LuaJIT compiler is 112 (!) times faster than Lua 5.1.5 and LuaJIT interpreter's MD5 is still 5 (!) times faster than Lua 5.1.5

[Weissrolf]

Last, but not least, LuaJIT uses a MIT license, which seems very permissive on a quick glance.

[Jiminimonka]

Last, but not least, LuaJIT uses a MIT license, which seems very permissive on a quick glance.

I feel this subject was answered quite succinctly by Moon Wizard a few posts up.

[Surge]

I think Weissrolf's concern (and mine too tbh) is the missing out of the potential performance gains that an update would benefit from. As it currently stands, FGU's biggest weakness is its poor performance, even before comparing to other VTTs. When you do start comparing it - especially to the newer ones using the newer technologies - FGU is going to end up in a worse and worse place.

Edit: and my comments on poor performance is regarding the state of things before all of the lighting updates.

Edit2: One more thing I would like to highlight - I can't see myself ever changing VTT. I love FG, warts and all!

[Weissrolf]

LuaJIT seems to be Lua version 5.1 on steroids. If that's not worthwhile to look at then nothing ever will.

[Weissrolf]

Out of curiosity I tried to simply replace FGU's lua5.1.dll with a compiled LuaJIT dll. This is said to work with Civilization, so it they could have been compatible with each other. Unfortunately FGU then claims to not being able to find said dll, so no dice.

No idea if some kind of version check is involved or if FGU's Lua dll is differently structured?! But it was worth a try.

[Weissrolf]

I tested campaign load times (modules + extensions) between an Intel 9900K and an AMD 5900X, they are virtually the same. Similar results for load times in WoW (many Lua addons).

Checking Lua benchmarks on the web also seems to suggest that Lua does not benefit much from CPU instructions-per-clock improvements, but only from higher clock-rates. So throwing more money and new CPUs at the problem does not help anymore. Lua 5.1 LuaJIT (just in time compiler) might be the best (if not only) way of improving Lua performance then, so I would like to know if this is a possibility that is being looked at?

[bmos]

There is a fork of Lua 5.1 called Luau that might be worth looking into.
It claims full backwards compatibility (https://luau-lang.org/compatibility), performance gains (https://luau-lang.org/performance) (supposedly almost as fast as JIT), and improved features (like if-then-else expressions (https://luau-lang.org/syntax#if-then-else-expressions)!)
https://luau-lang.org/

[jharp]

@Weissrolf
I routinely change the FGU lua.dll so I will check out these two options. I make it so I have access to _G for development.

Jason

[jharp]

WARNING: No one should use this!

@Weissrolf
Attached is a LuaJIT dll that will work (it hiccups on some items it interprets but it will launch a LuaJIT).

No promises it simply is what it is, a "compatible" replacement for the standard Lua. Don't complain to SW about any errors you get. It is purely for your enjoyment/testing/????

Moderators - If you have issues with this feel free to delete the attachment and I won't reupload.

Jason

[jharp]

If you are using 5E you must update the following in campaign/scripts/npc_skill.lua at line 33

local nStarts, nEnds, sLabel, sSign, sMod = string.find(aClauses[i], "([%w%s%(%)]*[%w%(%)]+)%s*([%+%-–]?)(%d*)");

[Moon Wizard]

The problem with LuaJIT is that it is not 100% compatible with Lua 5.1. There are immediate incompatibilities with Lua regular expression support as noted, and there may be other issues that won’t be discovered until later. Additionally, simple timing of ruleset loading and some basic actions was not significantly impacted by the replacement.

Given the immediate incompatibilities and risks of unknown issues cropping up with replacing libraries not meant to be replaced, we do not support replacement of the DLL, and ask that you do not provide any steps for modifying the executable or the libraries in this public forum.

Please take onto PM for further discussion.

Regards,
JPG

[bmos]

I just want to point out that Weissrolf's post was almost a year old and I only added my Luau suggestion to this thread because this thread seemed most relevant.
IIRC Moon Wizard actually tried LuaJIT last year but ran into script errors due to sub-optimal backwards compatibility.
But thanks jharp for an interesting toy to play with and the SmiteWorks team in general for engaging with community concerns.
Sometimes it's hard to tell because development timelines are long, but I can absolutely see FG and its rulesets improving constantly, usually based on specific requests and concerns the community has been asking for. The new collapsible sidebar, for instance, is really great.

[Trenloe]

I just want to point out that Weissrolf's post was almost a year old and I only added my Luau suggestion to this thread because this thread seemed most relevant.
Yeah, it was probably worth starting a new thread rather than post in this. Vaguely similar, maybe - but we don't have an issue with people starting new threads, in fact we prefer it to resurrecting old vaguely related threads - mush better to start a new discussion in a new thread, it allows your post to be more easily seen and discussed and found in future searches.