ToME: the Tales of Maj'Eyal

Everything about ToME
It is currently Thu Mar 30, 2017 6:49 am

All times are UTC




Post new topic Reply to topic  [ 4 posts ] 
Author Message
PostPosted: Sat Jul 27, 2013 7:39 pm 
Offline
Thalore

Joined: Tue Feb 28, 2012 6:36 am
Posts: 148
T-Engine has basic serialization support implemented in game/loader/pre-init.lua. This serialization is fine for what T-Engine needs, except that it is really slow in the current git* code. So, I've implemented a buffered version of serialize.

After loading a fresh session of ToME, the profile thread flushes CSETs. On my system, the current git code takes about 40 seconds to finish flushing csets, with the profile thread eating all the cpu cycles it can. Using my table.serialize implementation, flushing csets is done in about one second. Note that these times include zipping the resulting serialized data and sending the result to te4.org.

Code:
function table.serialize(src, sub, no_G, base)
   local chunks = {}
   if sub then
      chunks[1] = "{"
   end
   for k, e in pairs(src) do
      local nk = nil
      local nkC = {}
      local tk, te = type(k), type(e)
      if no_G then
         if tk == "table" then
            nkC[#nkC+1] = "["
            nkC[#nkC+1] = table.serialize(k, true)
            nkC[#nkC+1] = "]"
         elseif tk == "string" then
            nkC[#nkC+1] = k
         else
            nkC[#nkC+1] = "["
            nkC[#nkC+1] = tostring(k)
            nkC[#nkC+1] = "]"
         end
      else
         if not sub then
            nkC[#nkC+1] = (base and tostring(base) or "_G")
         end
         nkC[#nkC+1] = "["
         if tk == "table" then
            nkC[#nkC+1] = table.serialize(k, true)
         elseif tk == "string" then
            -- escaped quotes matter
            nkC[#nkC+1] = string.format("%q", k)
         else
            nkC[#nkC+1] = tostring(k)
         end
         nkC[#nkC+1] = "]"
      end

      nk = table.concat(nkC)

      -- These are the types of data we are willing to serialize
      if te == "table" or te == "string" or te == "number" or te == "boolean" then
         chunks[#chunks+1] = nk
         chunks[#chunks+1] = "="
         if te == "table" then
            chunks[#chunks+1] = table.serialize(e, true)
         elseif te == "number" then
            -- float output matters
            chunks[#chunks+1] = string.format("%f", e)
         elseif te == "string" then
            -- escaped quotes matter
            chunks[#chunks+1] = string.format("%q", e)
         else -- te == "boolean"
            chunks[#chunks+1] = tostring(e)
         end
         chunks[#chunks+1] = " "
      end
      
      if sub then
         chunks[#chunks+1] = ", "
      end
   end
   if sub then
      chunks[#chunks+1] = "}"
   end
         
   return table.concat(chunks)
end


Top
 Profile  
 
PostPosted: Wed Oct 02, 2013 4:21 pm 
Offline
Thalore

Joined: Tue Feb 28, 2012 6:36 am
Posts: 148
Update: the patch has been accepted.
Code:
commit 6f43ddef9b0e454c7afc272c20c6227ce7f8f5c8
Author: DarkGod <darkgod@net-core.org>
Date:   Tue Oct 1 18:11:31 2013 +0200

    use faster serialization


Top
 Profile  
 
PostPosted: Thu Oct 03, 2013 8:12 am 
Offline
Archmage

Joined: Sat Oct 22, 2011 10:31 am
Posts: 393
Good job!


Top
 Profile  
 
PostPosted: Thu Oct 03, 2013 9:22 pm 
Offline
Wayist

Joined: Tue Dec 13, 2011 2:11 pm
Posts: 18
Draw attention to Serializing table data into a string
It can proceed cycled tables.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 4 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group