PDA

View Full Version : Extension load order



Ikael
June 3rd, 2011, 14:38
After updating FG to latest release version I noticed that extension load order has changed, which has caused me troubles in my custom made ruleset. As for short description of the issue I have made custom ruleset which separates basic logic from graphics and basic logic from RPG specific aspects. The core ruleset only provides basic ways how it works but doesn't offer any graphics, nor any RPG setting based stuff. To use the ruleset I need to load at least two extensions: Theme extension (gives me the graphics) and rules extension (gives me RPG specific aspects). The problem with the newest release version is that the extension load order has been changed. Before the non-capital starting names were loaded first (for examples 'rules Unisystem' were loaded before 'Theme outbreak' because of the non-capital starting name). But for some reason the loading order has changed that capital starting name is loaded before the non-capital starting. This has caused lots of issues where XML-specific sources are not loaded from RPG specific extenstions and it break the whole mechanism. [EDIT: Ok, I don't remember if non-captial starting names were loaded sooner or after capital starting ones, but still the order has changed in the latest release, and it breaks the thing]

Of course I could simply rename RPG setting extension names to start with capital letter that after 'T' (as for 'Theme') but I feel there could be better built-in way to handle extension loading order.

My suggestion/request is that there should be way to set extension load order "manually", for example there could be <loadpriority> tag in extension's extension.xml which would contain number value to define the priority to load the extension (the smaller number the sooner the extension is loaded in). Of course the already built-in extension dependency feature would be handeled before this, but all other "wild card" extensions (which doesn't have any specific load order) would use this tag to determine in which order they are loaded in (by default the tag's value could be zero which would be the same as the already existing load order).

This load order issue has caused troubles also in different rulesets. For example in Savage Worlds (before the newest release) there were the patch extension and awsome powers extension but they would not work together because of the load order (I needed to rename the other extension to make it work). The extension dependency system works very fine as-is, but it doesn't provide enough flexible way to set load order for these "wild card" extensions, and in my case I cannot use it as I have several theme extensions and several RPG specific extension which should no be tied to each other (I want to have ability to use one RPG rules in all themes).

Moon_Wizard, I believe (hope) this is not galatic-scope request and possibly doable, but I would like to hear your opinion. If it's not doable then I just need to rename all the extensions and hope that the loading order won't change again in the future.

Moon Wizard
June 3rd, 2011, 22:24
As far as I can tell, the processing order for the files should not have changed (i.e. same Windows OS function calls). However, folder-based extensions are now given priority over packed extensions. (i.e. if folder and .ext file have same name, then folder will get loaded) This was done to make it consistent with the ruleset loading algorithm, and to make it easier for ruleset/extension developers.

The issue of extension load priority was briefly discussed in another thread:
https://www.fantasygrounds.com/forums/showthread.php?t=14419&highlight=load+order

The gist I got from that discussion is that dependencies will work, but the extension developers have to coordinate the dependencies and/or make multiple versions.

The reason I haven't touched this one yet is that it is fairly complex, and something I would need to hash out with the community developers.

Some random thoughts / questions:

* This will not solve collisions between extensions that try to overwrite the same objects. There will still be errors and unpredictable behavior if objects overwrite one another (i.e. changing a windowclass to modify dimensions which is also overwritten by another extension to add rules functionality).

* Extension load order is not a simple priority system. (i.e. you don't always load graphics last, see above) So, it seems to me that load order would be highly specific to the extensions you are trying to use together.

* Given that load priority (i.e. graphics before logic, or logic before graphics) can not necessarily be assumed given the above, we don't have a baseline for what the default priority numbers should be for any given extension.

* Part of this problem results from the tight integration of logic and graphics in many rulesets. (i.e. you can't change the size of a windowclass without overriding the whole windowclass) Perhaps a better solution would be to add a new capability to allow extensions to override or inject XML tags for existing templates or windowclasses. In that way, the theme developers could make graphics type changes (images, size) without affecting the logic or layout of other objects within the windowclasses. (i.e. I avoided modifying the modifier stack in the 3.5E dungeon theme exactly for this reason)

And that's just off the top of my head. I'm sure if we dug in, I would come up with more questions and items to consider, as well as the larger discussion from the community.

I think we should keep this discussion going so that we can come up with a good long term solution. However, as a caveat, it is unlikely that this will be part of the v2.8 release given the complexity of the issue.

Cheers,
JPG

Ikael
June 5th, 2011, 14:18
Thanks for answering, all are valid points.

However, I am not telling that there is global way of load ordering that works with all rulesets and I strongly believe that because its just overloading mechanism there won't be sound option for handling it for all rulesets. In my case I have found an issues that if I won't load icons before loading revised windowclasses then the new icons won't be there. There are also some other examples that can go wrong when load order is not correct but at the moment I don't recall how they manifested in ruleset.

My suggestion about the new tag is for moving the responsible of determining load order to developers. If there is need to make global load order mechanism then it will defenitely be complex and error-prone, but if you force developers to configure the load order then it's less error prone as it can differ from ruleset to ruleset.

Anyways, as it won't be included to next version I am forced to figure out a way to make it work for my purpose.

PS: The load order changed when updating from 2.7.2 to 2.7.6 and it might be because of the new support for non-latin based charsets. If you check the list of extensions (campaign details) in launcher you can see that in 2.7.2 the order is that non-capital starting names are after capital starting names; but in 2.7.6 this order has changed.

Moon Wizard
June 7th, 2011, 07:54
Thanks for your understanding. There is only one of me, and so many features to implement.

The dependency checks actually force the load order as well (i.e. dependencies are loaded before the extensions that depend on them.) So, that is probably the way to go in order to enforce load order currently.

I didn't realize that you were moving from 2.7.2, so I may not have checked that far back in my code history to see if any changes would have affected load order. I only checked from 2.7.4.

In some ways, I think that load order is very similar to dependencies, since developers have to edit the files to adjust and specify ordering. Let me know how it works out with the dependency tags.

Thanks,
JPG

Ikael
June 7th, 2011, 16:36
Like I tried to explain, I can't use extension dependency tag because it only make an extensions to be dependent to one another specific extension (but you can have several depedencies to different extensions). I will give short example why I can't use it:

You have theme/graphic extensions named A1,A2,A3,A4,A5 and rules extensions called B1,B2,B3. The load order should be that theme extension is loaded first in and rule extension is loaded after that. Now you must have ability to make any combination of themes and rules, so you could have A1+B1, A1+B2,A5+B3,A2+B2 etc. You cannot use dependency tag in this situation as it makes one extension dependent to another extension by name (ie. B1 extension is dependent to extension named A1).

If there could be a way to have dependencies by groups, for example using the exclusiongroup tag name (all theme extensions have this exclusiongroup), then it would work very well! This would actually be much better and flexible solution for this issue.

Moon Wizard
June 7th, 2011, 19:48
I'm not trying to be obtuse here. I'm just not following why your example will not work.

From your example:
* B1 has dependency A1
* B2 has dependency A1 and A2
* B3 has dependency A5

When they are loading, let's assume that there would be a default load order, say:
B1, B2, B3, A1, A2, A3, A4, A5.

Here is the load order after processing B1 (due to dependency):
A1, B1

Load order on next step for B2 (next in default load order):
A1, B1, A2, B2

Next step for B3:
A1, B1, A2, B2, A5, B3

Final load order
A1, B1, A2, B2, A5, B3, A3, A4

This is how it should work. If not, I need to address anyway.

Also, are you saying that the dependencies don't chain (C1 requires B1 and B1 requires A1), and that is the issue? If so, I can look at fixing that up as well.

Finally, perhaps a concrete example would work better, then I could see the issue in action. Which ruleset and extensions are the culprits? I can try loading them up here.

Thanks,
JPG

Zeus
June 7th, 2011, 20:28
I'll wade in with my wellies.

I think Ikael's challenge with the current dependency approach is that is requires the dependencies to be hard specified when in fact the modular approach requires a generic dependency capability

e.g. theme/graphic extensions named A1,A2,A3,A4,A5 and rules extensions called B1,B2,B3

B1 is dependant upon A1 or A2 or A3 or A4 or A5
B2 is dependant upon A1 or A2 or A3 or A4 or A5
etc. etc.

This as, Ikael wants to leave it to the GM to select which rule/theme combinations to use.

One way to perhaps workaround this issue using current functionality would be to commonly name the theme extensions (at an FGII level) with perhaps each extension having a different description and version number so that the dependencies can be commonly grouped but still be individually available for selection in the Launcher.

e.g. A1 to A5 are renamed to use a common name of "A"

Now by making B1 to Bn extensions dependant upon extension A, we enable the generic approach.



<dependency>
<name>A</name>
<minversion>1.00</minversion>
<maxversion>1.99</maxversion>
</dependency>


FGII should then allow any of the A theme extensions to be used with any of the B rule extensions and A themes will always be loaded first.

Ikael
June 7th, 2011, 21:20
thanks DrZeuss, you clarified it well and that's the issue. Sorry Moon_Wizard that I wasn't so clear myself, I was writing in rush.

Anyways, I think I will use a fudge way to handle this situtaion until proper built-in way is implemented. This fudge way involves renaming each theme extension names to start with ' (ie. 'Theme - Outbreak) which will make it load before other extensions (and there is no depedency tags used).

Moon Wizard
June 7th, 2011, 22:08
DrZ to the rescue, as usual. ;)

OK, I have a better picture now of what you are asking with DrZ's clarification.

I also assume that you would want to load a rules extension without any
themes potentially, so the dependency would get in the way in that scenario.

Let me think about this a bit, and see what pops out.

Cheers,
JPG

Zeus
June 7th, 2011, 22:39
Your welcome, glad to be of help.

One thought, if it were possible to add a <group> tag to an extensions properties in extension.xml perhaps then the dependency routines could be expanded to support groups.

e.g B1 is dependant upon Group Theme

Moon Wizard
June 7th, 2011, 22:56
It seems like we need an optional flag as well as a group flag, in order to make that work. (i.e. want to be able to load rules extension even if no theme extension used.) If we have theme group extensions, then load them first. Otherwise, just load normally.

Cheers,
JPG

Zeus
June 7th, 2011, 23:27
I guess Yes, if we are assuming the base ruleset carries default theme/graphics, which of course it may not if the strictly modular approach Ikael is proposing is truly segregating the rules logic from the graphics - I guess Ikael would need to confirm this.

Moon Wizard
June 8th, 2011, 06:37
I've been running over the various scenarios in my head. After reviewing all the options for building into current functionality (group and optional tags) as well as thinking through other scenarios (tags to delay loading to last); I've come to the conclusion that the load priority tag you have been fighting for is actually the simplest solution.

Ikael, thanks for sticking through this process with me. Sometimes, I need to understand the issues better before I feel comfortable making changes, especially in a part of the code that I haven't worked on much (i.e. extensions).

That said, here is my proposal:
* Add a loadingorder tag under properties tag in extension.xml.
* By default, the loading order for any file is zero.
* Extensions will be loaded in increasing numerical order based on the loading order tag.
* If extensions have the same loading order number, then the load order is undefined. (i.e. load order is not guaranteed)

* Handling of dependencies with respect to loading order
** Dependencies will bypass loading order
** Example: If extension A with loading order of 2 specifies a dependency B that has a loading order of 5, the load order will be B then A. If no dependency was specified, then the load order would be based on the loading order number (A then B).

How's that sound?

Thanks,
JPG

Ikael
June 8th, 2011, 19:36
Moon_Wizard, that sounds very much how I visioned it and I totally approve it. But to point out one more thing I think the loadorder tag should also affect between extensions that have dependency to the same extension. For example A is the "master" extension and B1 and B2 are both dependent to A. B1 has loadorder tag set to 5 and B2 has loadorder tag set to 3, which would make the final load order be A, B2, B1.

Thank you both for giving such a good effort on this subject!

Moon Wizard
June 11th, 2011, 01:56
After our discussion, I decided to go ahead and add to v2.8 in testing.

When loading extensions, the load order will be determined by the loadorder tag (in increasing order, defaults to zero), and then by case-sensitive alphabetical order.

After the load order is determined, then exclusions and dependency ordering will be applied. So, dependency will trump load order.

The "loadorder" tag is placed inside the "properties" tag in the extension.xml file.

When you get a chance, please check it out to make sure it works the way you expect.

Thanks,
JPG

Zeus
June 11th, 2011, 09:11
Thanks JPG.

I can confirm the alphabetical ordering and dependency chains appear to be working OK. I have 11 extensions attached to my campaign and they appear to load in the expected (correct) order.

I haven't updated the extensions to use <loadorder> yet, I'll leave that to Ikael to test as I have no strict need for it in my extensions.

Ikael
June 21st, 2011, 18:29
Thanks both again! Now I got the time for testing the new loadorder-tag and it works very fine! I declared all my theme extensions to have loadorder with value of 10, all rule extension to have loadorder value of 20 and setting extension to have loadorder value of 30. Works perfectly! All the XMLs are loaded in the correct order no matter of extension's name. No more need to have ugly hacks in extension names :) Now I will eagerly wait that 2.8 becomes stable release.