Page 1 of 2 12 Last
  1. #1

    Proof of concept/experimental, JSON, reduced "node" use

    I've been trying to figure out the best way to improve load times for creatures placed into the Combat Tracker in the AD&D ruleset. It's kinda sluggish and I'd like to make it better. To that end I did some review of the code and the nodes that I might best "trim" out to reduce. As you can guess it's mostly Attacks, Powers and Inventory items.

    My first experiment was to remove the inventory and "abilitynotes" nodes and all their children to npcs placed into the CT and replace those nested nodes with a single string of "node.getPath();node.getPath()" paths back to their source. The idea was I would use createControl for the view of those items showing them from the original source NPC.

    It worked but the reduced feature set (can't edit really) really bothers me... but inventory and ability notes are not that big of a deal. Attacks are.

    Attacks need a little more work to support. I started with the idea I did for inventory/ability notes and realized it wasn't really a good way to handle it. Someone had posted code that showed someone else using JSON strings in FG and it got me thinking, hell... I can store just about anything in a JSON string. I found a library that was opensource and began tinkering with it. After a bit I was able to get the weaponlist nested node and all the components stuffed into a single string entry on a npc in the CT and be able to "edit" the fields there.

    Once I had it in JSON I began working on a UI window for it to display to tinker with it, see how it felt and test the load times.

    The UI ended up being a intense learning experience and it game me a lot of ideas on how to to come work on the CT ui in general (I want to compact the npc views) but in the end it really didn't seem to make all that much difference in load times. I mean, it DOES make it load slightly faster but I think the added overhead of all the createControl() handling ate about 1/4 of the improvement or more.

    I did notice some odd behavior in FG in general unrelated to these tweaks. I could close FG and start it and the first encounter I dropped into the CT was fairly spunky. If I cleared them and re-did it... each time it seemed to get slower (about 3-5th time it seemed to reach it's max slowness). I tested with my ruleset unmodified and modified and both seemed to experience this issue. Even tried the 5E one and seemed to be the same there. My guess is that it has something to do with memory management in the 32 bit app? Tho that's just speculation.

    I'm going to keep tinkering with this a bit more to see what I can do but ... right not im mostly disappointed in the results. I don't think it's really worth the effect of losing the nodes (ease of manipulation) for the "slight" CT load times. Inventory items however I will probably continue to work with regardless so that people can use the Inventory tab and not dramatically affect CT load times. The benefits of inventory on ct npcs is worth it I think... it might end up being a "read only" mode. Meaning you'd need to have the npc's inventory setup the way you want before to drop them in the CT (which for me is 99.9% of the time anyway).

    I say all that to solicit other's insight into this, perhaps someone has a method they use to help with these types of situations.

    Here is the way I took the nodeNPC.weaponlist.* and converted it to a table then converted that to a JSON string.

    Code:
    -- pass a nodeNPC node and return the "weaponlist" children as a json string to be stored into a node as one single entry.
    -- Once we have it in json style text we can use it like:
    -- local aWeaponList = JSON.decode(DB.getValue(nodeCT,'weaponlist_json',"");
    -- DB.setValue(nodeCT,'weaponlist_json',"string",JSON.encode(aWeaponList));
    function getWeaponListAsJSONText(node)
      local aWeaponsList = {};
    
      for sID, nodeWeapon in pairs(DB.getChildren(node,"weaponlist")) do
        local aWeapon = {};
        aWeapon.sSource = nodeWeapon.getPath();
        aWeapon.sID = sID; 
        aWeapon.sName = DB.getValue(nodeWeapon,"name","");
        aWeapon.nAttackCurrent = DB.getValue(nodeWeapon,"attackview_weapon",0);
        aWeapon.sAttackStat = DB.getValue(nodeWeapon,"attackstat","");
        aWeapon.nAttackBonus = DB.getValue(nodeWeapon,"attackbonus",0);
        aWeapon.nSpeedFactor = DB.getValue(nodeWeapon,"speedfactor",0);
        aWeapon.nType = DB.getValue(nodeWeapon,"type",0);
        aWeapon.nCarried = DB.getValue(nodeWeapon,"carried",0);
        aWeapon.nMaxAmmo = DB.getValue(nodeWeapon,"maxammo",0);
        aWeapon.nAmmo = DB.getValue(nodeWeapon,"ammo",0);
        aWeapon.nLocked = DB.getValue(nodeWeapon,"locked",1);
    --    aWeapon.sShortcutClass, aWeapon.sShortcutRecord = DB.getValue(nodeWeapon,"shortcut","","");
        aWeapon.ItemNoteLocked = DB.getValue(nodeWeapon,"itemnote.locked",1);
        aWeapon.ItemNoteName = DB.getValue(nodeWeapon,"itemnote.name","");
        aWeapon.ItemNoteText = DB.getValue(nodeWeapon,"itemnote.text","");
        aWeapon.aDamageList = {};
        for sDMGid, nodeDamage in pairs(DB.getChildren(nodeWeapon,"damagelist")) do
          local aDamage = {};
          aDamage.sID = sDMGid;
          aDamage.sDamageAsString = DB.getValue(nodeDamage,"damageasstring","");
          aDamage.nBonus = DB.getValue(nodeDamage,"bonus",0);
          aDamage.dDice = DB.getValue(nodeDamage,"dice","");
          aDamage.sStat = DB.getValue(nodeDamage,"stat","");
          aDamage.sType = DB.getValue(nodeDamage,"type","");      
          table.insert(aWeapon.aDamageList,aDamage);
        end
    
        -- sort damage by id so they appear as they do in weapons tab right now
        local sort_byID = function( a,b ) return a.sID < b.sID end
        table.sort(aWeapon.aDamageList, sort_byID);
    
        -- add this weapon to weaponslist
        table.insert(aWeaponsList,aWeapon);
    
        -- Sort the weapons by name like they appear in list currently
        local sort_byName = function( a,b ) return a.sName < b.sName end
        table.sort(aWeaponsList, sort_byName);
      end
    
      local sJson = JSON.encode(aWeaponsList);
      
      return sJson;  
    end


    This is an example of how it turned out in the CT. I picked one of the most "nodey" npcs I had.



    The simple JSON library I used was this. I made some minor updates to it but it worked almost out of the gate.

    https://gist.github.com/tylerneylon/...f316be525b30ab
    ---
    DM/Play: AD&D (any flavor)
    Coding the Official AD&D Ruleset

  2. #2
    Here is the bulk of the code I used to generate the subwindow contents of the weapons.

    Code:
    function onInit()
      local nodeCT = getDatabaseNode();
      
      DB.addHandler(DB.getPath(nodeCT, "weaponlist_json"), "onUpdate", buildWeaponsListForCTView);
      
      buildWeaponsListForCTView();
    end
    
    function onClose()
      DB.removeHandler(DB.getPath(nodeCT, "weaponlist_json"), "onUpdate", buildWeaponsListForCTView);
    end
    
    -- return the ID for this weapon entry
    -- name_id-00001
    function getMyID(sThisControl)
      return sThisControl:match("_(id%-%d+)$");
    end
    -- return the ID for this damage entry
    -- dmg_id-00004_id-00001
    -- returns sDamageID, sWeaponID
    function getMyDamageID(sThisControl)
      return sThisControl:match("^dmg_(id%-%d+)_(id%-%d+)$");
    end
    
    -- populate rWeapon using the passed weapon id's record.
    function getWeaponRecord(sWeaponID)
      local rWeapon = {};
      local nodeCT = getDatabaseNode();
      local sWeaponsList = DB.getValue(nodeCT,"weaponlist_json","");
      if sWeaponsList and sWeaponsList ~= '' then
        -- decode the json style string weaponlist entry for this npc
        local rWeaponList = JSON.decode(sWeaponsList);
        for _,rWeaponSource in pairs(rWeaponList) do 
          if rWeaponSource.sID == sWeaponID then
            rWeapon = rWeaponSource;
            break;
          end
        end
      end -- no weaponlist
      return rWeapon;
    end
    -- get damage record from weapon
    -- id-00001, id-00004
    function getDamageRecord(sWeaponID,sDamageID)
      local rDamage = {};
      local rWeapon = getWeaponRecord(sWeaponID);
      for _,rDMGSource in pairs(rWeapon.aDamageList) do 
        if rDMGSource.sID == sDamageID then
          rDamage = rDMGSource;
          break;
        end
      end
      
      return rDamage;
    end
    
    -- set a new value to the JSON array stored on nodeCT, sWeaponID, sTag, sValue, sType
    -- setWeaponRecordValue(nodeCT, id-00001,"nAmmo",3,"number")
    function setWeaponRecordValue(nodeCT,sWeaponID,sTag,sValue,sType)
      local sWeaponsList = DB.getValue(nodeCT,"weaponlist_json","");
      if sWeaponsList and sWeaponsList ~= '' then
        -- decode the json style string weaponlist entry for this npc
        local aWeaponList = JSON.decode(sWeaponsList);
        for nID,rWeaponSource in pairs(aWeaponList) do 
          if rWeaponSource.sID == sWeaponID then
            if sType == "string" then
              aWeaponList[nID][sTag] = tostring(sValue) or "";
            elseif sType == "number" then
              aWeaponList[nID][sTag] = tonumber(sValue) or 0;
            else
              aWeaponList[nID][sTag] = sValue;
            end
            local sWeaponListChanged = JSON.encode(aWeaponList);
            DB.setValue(nodeCT,"weaponlist_json","string",sWeaponListChanged);
            break;
          end
        end
      end -- no weaponlist
    
    end
    
    -- this builds all the controls into the viewspace for weapons
    function buildWeaponsListForCTView()
    -- Debug.console("npc_weapons_ct.lua","buildWeaponsListForCTView","aControls[i].getName()",aControls[i].getName());
      -- we remove existing controls and re-build from scratch
      -- this is incase we eventually allow editing (add/delete items)
      local aControls = getControls();
      if #aControls > 0 then
        for i=1 , #aControls do
          local sControlName = aControls[i].getName();
          if sControlName ~= 'contentanchor' then
            aControls[i].destroy(); 
          end
        end
      end
      
      local nodeCT = getDatabaseNode();
      local sWeaponsList = DB.getValue(nodeCT,"weaponlist_json","");
      if sWeaponsList and sWeaponsList ~= '' then
        -- decode the json style string weaponlist entry for this npc
        local rWeaponList = JSON.decode(sWeaponsList);
    
        local bRowShade = false;
        for _,rWeapon in pairs(rWeaponList) do 
          local sFrame = nil;
          if bRowShade then
            sFrame = "rowshade";
          end
          --- VARS
          local sControlInit           = "init_" .. rWeapon.sID;
          local sControlAttack         = "attack_" .. rWeapon.sID;
          local sControlName           = "name_" .. rWeapon.sID;
          local sControlType           = "type_" .. rWeapon.sID;
    
          --- CONTROLS
          -- initiative roll
          local controlInit = createControl("initiative_weapon_ct", sControlInit);
          controlInit.setFrame(sFrame);
          controlInit.setValue("[INIT:" .. rWeapon.nSpeedFactor .. "]");
          controlInit.setReadOnly(true);
          controlInit.setTooltipText("INITIATIVE");
    
          -- attack name    
          local controlName = createControl("name_weapon_ct", sControlName);
          controlName.setFrame(sFrame);
          controlName.setValue(rWeapon.sName);
          controlName.setReadOnly(true);
          -- this sets height
          controlName.setAnchor("top", sControlInit,"top","absolute",0);
          -- this sets width
          controlName.setAnchor("left", sControlInit,"right","absolute", 0);
          controlName.setAnchor("right", 'contentanchor',"center","absolute",-100);
    
          -- type of weapon
          local controlType = createControl("type_weapon_ct", sControlType);
          controlType.setFrame(sFrame);
          controlType.setValue(rWeapon.nType);
          controlType.setReadOnly(true);
          controlType.setTooltipText("WEAPON TYPE");
          -- this sets height
          controlType.setAnchor("top", sControlName,"top","absolute",0);
          controlType.setAnchor("bottom", sControlName,"bottom","absolute",0);
          -- this sets width
          controlType.setAnchor("left", sControlName,"right","absolute", 0);
          controlType.setAnchor("right", sControlName,"right","absolute",20);
    
          -- attack roll
          local controlAttack = createControl("attack_weapon_ct", sControlAttack);
          controlAttack.setFrame(sFrame);
          -- make the attack label concise.
          local sAttackString = "[ATK";
          if rWeapon.nAttackCurrent ~= 0 then
            sAttackString = sAttackString .. ":" .. rWeapon.nAttackCurrent .. "]";
          else
            sAttackString = sAttackString .. "]";
          end
          --
          controlAttack.setValue(sAttackString);
          controlAttack.setReadOnly(true);
          controlAttack.setTooltipText("ATK");
          -- this sets height
          controlAttack.setAnchor("top", sControlType,"top","absolute",0);
          -- this sets width
          controlAttack.setAnchor("left", sControlType,"right","absolute", 0);
          controlAttack.setAnchor("right", sControlType,"right","absolute",50);
    
          -- add damage rolls
          for nID, rDamage in pairs(rWeapon.aDamageList) do
            local sControlDMG = "dmg_" .. rDamage.sID .. "_" .. rWeapon.sID;
            local sDiceAsString = StringManager.convertDiceToString(rDamage.dDice, rDamage.nBonus);
            
            -- take first letter of each type in the damage type string and uppercase it for compact view.
            local aTypes = StringManager.split(rDamage.sType,",",true);
            local sTypeLetters = "";
            if #aTypes > 0 then
              sTypeLetters = " ";
            end
            for nCount, sAType in pairs(aTypes) do
              local sSep = ",";
              if nCount >= #aTypes then
                sSep = "";
              end
              sTypeLetters = sTypeLetters .. string.upper(sAType:sub(1,1)) .. sSep;
            end
            --
            
            -- Damage control
            local controlDMG = createControl("damage_weapon_ct", sControlDMG);
            controlDMG.setFrame(sFrame);
            local sDamageStringFinal = "[" .. sDiceAsString .. sTypeLetters .. "]";
            local nSizeOfString = string.len(sDamageStringFinal);
            controlDMG.setValue(sDamageStringFinal);
            controlDMG.setReadOnly(true);
            controlDMG.setTooltipText("Damage:" .. rDamage.sDamageAsString);
            -- this sets height
            controlDMG.setAnchor("top", sControlAttack,"top","absolute",0);
            -- this sets width
            controlDMG.setAnchor("left", sControlAttack,"right","relative", 0);
            controlDMG.setAnchor("right", sControlAttack,"right","relative",(nSizeOfString*6));
          end
          
          -- add spacer to end so that it will shade entire line on frame
          local controlSpacer = createControl("spacer_weapon_ct", 'spacer');
          controlSpacer.setFrame(sFrame);
          controlSpacer.setAnchor("top", sControlAttack,"top","absolute",0);
          -- this sets width
          controlSpacer.setAnchor("left", sControlAttack,"right","relative", 0);
          controlSpacer.setAnchor("right", 'contentanchor',"right","absolute",0);
          
          -- done ... NEXT!
          bRowShade = not bRowShade;
        end
      end -- no sWeaponsList
    end
    ---
    DM/Play: AD&D (any flavor)
    Coding the Official AD&D Ruleset

  3. #3

    Join Date
    Apr 2008
    Location
    Virginia Beach
    Posts
    2,934
    celestian, why not just write it out to xml file instead of json using the built-in import/export tools? Just curious why you thought using json was the way to go?

  4. #4
    Quote Originally Posted by Bidmaron View Post
    celestian, why not just write it out to xml file instead of json using the built-in import/export tools? Just curious why you thought using json was the way to go?
    Data stored as an xml node is the cause of the "node" creation problem. You can replace 50+ nodes with a single node storing all the data as a json "string". The downside is you have to manage all your UI programmatically for those similarly stored objects.

    JPG can explain it better than me but my understanding is the creation of nodes in FG takes significant time. (something to do with the backend processing of these objects with LUA). You don't notice it that much in rulesets like 5E because the system was built around avoiding extensive nodes. The 2E ruleset makes use of many and lots of them. The actions/weapons&powers sections specifically. Then adding those same sections to items in addition to inventory slots for npcs.

    That was the impetus for this experimentation.
    ---
    DM/Play: AD&D (any flavor)
    Coding the Official AD&D Ruleset

  5. #5
    How do I get up to speed on all this? I am both absolutely flabbergasted by this, and interested in learning more. (It helps that I essentially grew up with basic/2e, and matured playing 3.0/3.5. I can't tell you how many fights over rules I had. We could only dream of a program like this.)
    Ultimate Licence holder

    I've had FG for so LONG I DON'T KNOW HOW TO USE IT!

    But I'm learning!

  6. #6
    Quote Originally Posted by kalmarjan View Post
    How do I get up to speed on all this? I am both absolutely flabbergasted by this, and interested in learning more. (It helps that I essentially grew up with basic/2e, and matured playing 3.0/3.5. I can't tell you how many fights over rules I had. We could only dream of a program like this.)
    Fortunately if you just want to play AD&D you can with the ruleset without having to know about these sorta things. All this is to try to do is come up with a better wheel I guess, or at least some axle grease?
    ---
    DM/Play: AD&D (any flavor)
    Coding the Official AD&D Ruleset

  7. #7

    Join Date
    Apr 2008
    Location
    Virginia Beach
    Posts
    2,934
    Is this something that will get fixed with FGU, Celestian?

  8. #8
    Quote Originally Posted by Bidmaron View Post
    Is this something that will get fixed with FGU, Celestian?
    Based on the replies from JPG, no. If might be less of an issue with more memory because of the 64 bit application (my experimentation here seemed to link some of this to memory use) but that is pure speculation on my part.
    ---
    DM/Play: AD&D (any flavor)
    Coding the Official AD&D Ruleset

  9. #9

    Join Date
    Apr 2008
    Location
    Virginia Beach
    Posts
    2,934
    Well, that’s a shame....

  10. #10
    I have a hypothesis that removing database nodes as Lua objects would improve performance and load times a fair amount; but it's only an educated guess, and it breaks every single ruleset. Since every ruleset would be broken to make that change, it's a VERY large project to make the change, and fix every ruleset to work again with the revised API. At this point in time, it's not something that will be pursued, since that would be a large delay for FGU.

    In the interim, celestian and I have been chatting back and forth on his findings and his UI re-engineering. I've been interested in revising the CT at some point, so his work is a way to see how people react to various UI changes.

    Regards,
    JPG

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

Log in

Log in