UWP Import Tool & Issues with the ruleset

[Stargrove]

I have created an extension that will take JSON data from travellermap.com (for example: https://travellermap.com/data/spinward%20marches/2124) which would give you this:

{"Worlds":[{"Name":"Lunion","Hex":"2124","UWP":"A995984-D","PBG":"810","Zone":"","Bases":"NS","Allegiance":"ImDd","Stellar":"G5 V","SS":"K","Ix":"{ 5 }","Ex":"(C8H+3)","Cx":"[7E3B]","Nobility":"BEF","Worlds":11,"ResourceUnits":4896,"Subsector":10,"Quadrant":5,"WorldX":-108,"WorldY":-56,"Remarks":"Hi In Cp Ht","LegacyBaseCode":"A","Sector":"Spinward Marches","SubsectorName":"Lunion","SectorAbbreviation":"Spin","AllegianceName":"Third Imperium, Domain of Deneb"}]}

You can then copy/paste it into a field on my form and after pressing a button it will be processed into a new System entry. However, there is some oddity with the Hex Location and Bases being processed into their respective fields properly. I can see in the db.xml that the <hexlocation> tag is populated properly and the <bases> XML tag is populated properly. The issue seems to show up when you tab through the newly created system record where that triggers something in the MGT2E ruleset to fill in all the extra data to the right of the UWP fields (Starport, Size, Atmosphere, etc) which will then erase the Hex Location and Bases data that my code filled in from the JSON.

If I just import the data to fill in the record and restart FGU, I can see the proper data in the db.xml file:

<id-00003>
<allegiances type="string">Third Imperium</allegiances>
<bases type="string">N S</bases>
<domain type="string"> Domain of Deneb</domain>
<hexlocation type="string">2124</hexlocation>
<name type="string">Lunion</name>
<sector type="string">Spinward Marches</sector>
<subsector type="string">Lunion</subsector>
<system type="string">G5 V</system>
<trade_codes type="string">Hi In Cp Ht</trade_codes>
<travel_code type="string">G</travel_code>
<uwp type="string">A995984-D</uwp>
</id-00003>


But, as soon as I open up the record and tab through all the fields, the Hex and Bases data disappears. I can get it to finally stick by manually entering it but would obviously prefer to not have to do that after the data import. I am also seeing that Bases will no longer hold more than one entry, and Trade Code entries are getting deleted and cannot be re-added and the new entries get deleted.

The extension works well, and it could be a great tool for these annoying issues. I am unsure what is happening under the ruleset's hood, but something is not working properly. I don't think I really need help with my extension, per se, just for someone to look at what is happening with the ruleset.

[Stargrove]

As a test, I took the data above and pasted it into a new campaigns db.xml with no modules or extension loaded, opened up the System entry, and tabbed through all the fields several times and can easily reproduce the issues I described above (Hex and Bases entries disappearing, and Trade Code data getting dropped). Hopefully, someone can figure out why this is happening.



<worlds>
<id-00002>
<allegiances type="string">Third Imperium</allegiances>
<bases type="string">N S</bases>
<domain type="string"> Domain of Deneb</domain>
<hexlocation type="string">2124</hexlocation>
<name type="string">LunionTest</name>
<sector type="string">Spinward Marches</sector>
<subsector type="string">LunionTest</subsector>
<system type="string">G5 V</system>
<trade_codes type="string">Hi In Cp Ht</trade_codes>
<travel_code type="string">G</travel_code>
<uwp type="string">A995984-D</uwp>
</id-00002>
</worlds>

[Moon Wizard]

I've asked Tarkonian to take a first look at this using the steps you described.

Regards,
JPG

[Tarkonian]

As a test, I took the data above and pasted it into a new campaigns db.xml with no modules or extension loaded, opened up the System entry, and tabbed through all the fields several times and can easily reproduce the issues I described above (Hex and Bases entries disappearing, and Trade Code data getting dropped). Hopefully, someone can figure out why this is happening.



<worlds>
<id-00002>
<allegiances type="string">Third Imperium</allegiances>
<bases type="string">N S</bases>
<domain type="string"> Domain of Deneb</domain>
<hexlocation type="string">2124</hexlocation>
<name type="string">LunionTest</name>
<sector type="string">Spinward Marches</sector>
<subsector type="string">LunionTest</subsector>
<system type="string">G5 V</system>
<trade_codes type="string">Hi In Cp Ht</trade_codes>
<travel_code type="string">G</travel_code>
<uwp type="string">A995984-D</uwp>
</id-00002>
</worlds>



It took a few shots but I was able to recreate this, and then I was able to figure out the "most likely" culprit for it. But it's going to take me a hot-minute to unwind what this function is doing and how to fix it. I will try to have an update done for this soon. For now, when a value changes on that record, it calls a "rebuildUWP" function that rewrites the sheet. If you create the world and then lock the record, it should stop it from overwriting anything you set up in the DB with JSON like you were doing. Not ideal, but for now that should stop it from rewriting everything on you.

[Moon Wizard]

Just to be clear, this was something that existed in the ruleset before we took it over; so we're trying to figure out why it was implemented in the first place and how to prevent it from making those changes when undesired.

Regards,
JPG

[Stargrove]

Thank you for looking into this. I will investigate how to lock a record to prevent data loss.

[Stargrove]

You say, "before we took it over". Does this mean that MBM is no longer the primary dev on the ruleset?

[Moon Wizard]

As far as I know, he stepped down from direct development with Mongoose; so we've taking over maintenance until another developer is interested in taking over to add more features.

Regards,
JPG

[Tarkonian]

As a test, I took the data above and pasted it into a new campaigns db.xml with no modules or extension loaded, opened up the System entry, and tabbed through all the fields several times and can easily reproduce the issues I described above (Hex and Bases entries disappearing, and Trade Code data getting dropped). Hopefully, someone can figure out why this is happening.



<worlds>
<id-00002>
<allegiances type="string">Third Imperium</allegiances>
<bases type="string">N S</bases>
<domain type="string"> Domain of Deneb</domain>
<hexlocation type="string">2124</hexlocation>
<name type="string">LunionTest</name>
<sector type="string">Spinward Marches</sector>
<subsector type="string">LunionTest</subsector>
<system type="string">G5 V</system>
<trade_codes type="string">Hi In Cp Ht</trade_codes>
<travel_code type="string">G</travel_code>
<uwp type="string">A995984-D</uwp>
</id-00002>
</worlds>



Okay, sorry for the slow response but I have two different fixes for you.
The issue is your import tool is trying to fill in details that the ruleset is trying to auto-generate for you. If you put a numbercode in the UWP section and then tab away from it, it will use the numbers to auto-generate a new world for you, and does not care if anything is already there.

Solution 1: I have added a houserule to disable the UWP Auto-Generation (UWPAG) completely. This will completely turn it off.
62942

Solution 2: You may want to keep using the UWPAG feature after importing, so I have created a secret DB flag for you to use with your import tool. Your tool will have to add this to the DB data. If you set the flag to 1 it will tell the UWPAG to skip that record and not auto-generate anything for it. This will let you import the records, but keep auto-generating new worlds as you need them without disabling everything. The flag is:
<skipUWPAG type="number">1</skipUWPAG>

62943

[Stargrove]

I appreciate you looking into this.

I am beginning to think that it may be better if I could not enter any data into the UWP field (<uwp /> db.xml field) at the top of the form and instead parse out the UWP JSON to create the entries in the individual UWP fields below the Hex Location field. That would not be that hard to do and could eliminate the issue I was seeing. I will try that out and see what happens. At one point I was filling in those individual entries, but it just seemed easier to fill in the one field since that is how it is in the JSON.

[Stargrove]

My idea seems to work well and shows no issues if I parse out the UWP data into those individual fields for entry into the db.xml. I am no longer entering any data into the <uwp /> db.xml tag.

Ironically, I found that there are some conflicts between abbreviations for Bases in the ruleset vs what is on Travellermap. I will submit a bug post about it.

I will get an updated version of my extension submitted. Thanks.