Hello,
I am a Developer at Chaos Age Server.
The server is quite old now and, as a consequence, we have a huge amount of items in storage (storage.txt has about 500mb, mostly in bank boxes and secure chests). I believe this might be an important reason why SaveWorldState() takes about 1 minute to run. That also seems to be the main reason for the long load time of the server.
What seems to be at our advantage is that, although storage is huge, only a little bit of it changes between each world save. Does SaveWorldState() take advantage of this sparsity?
If not, I had the following idea. Bank boxes and secure chests wouldn't be loaded to storage area when the server is loaded. Instead, we would keep a datafile for the contents of each bankbox or secure chest . When a player opens a bankbox, the content of that datafile would be created in a storage area. We would save the world in the following way:
a) Transfer all items in storage area to their respective datafiles.
b) Destroy items in storagearea.
c) SaveWorldState().
Does this make sense or am I missing something devious?
If this does make sense, is there a good way to save UO Objects to datafiles?
Thanks!
Optimizing SaveWorldState()
Re: Optimizing SaveWorldState()
I don't believe POL does incremental saves to the world data only saving changed data. It writes the entire world data during a world save.What seems to be at our advantage is that, although storage is huge, only a little bit of it changes between each world save. Does SaveWorldState() take advantage of this sparsity?
Wouldn't you just be transferring the delay from writing out the storage.txt files to writing out the datafiles? I don't know how much time would be saved because when you move the object info to the datafiles you would have to copy any CProps attached to any item as well as any special info such as item name etc. All that data manipulation being done in the virtual machine might not yield much savings and in fact might make the process even longer. Plus the storage datafiles would still have to be written to the disc. I suppose there might be some space savings, and consequently some time savings, by eliminating the additional formatting space taken up in a storage.txt file but would it be enough considering the additional overhead of transferring the storage.txt data to the datafiles?If not, I had the following idea. Bank boxes and secure chests wouldn't be loaded to storage area when the server is loaded. Instead, we would keep a datafile for the contents of each bankbox or secure chest . When a player opens a bankbox, the content of that datafile would be created in a storage area. We would save the world in the following way:
a) Transfer all items in storage area to their respective datafiles.
b) Destroy items in storagearea.
c) SaveWorldState().
Does this make sense or am I missing something devious?
I wonder if it might be possible to script a custom storage management system that doesn't use the standard Core storage system but rather uses datafiles. Thus eliminating the need to transfer the object info from the storage.txt files in the first place. If I am not mistaken, datafile entries are written as members are appended to them. This would spread out the writes and thus solve the long write time during a world save.
Re: Optimizing SaveWorldState()
Switching to hosting with SSD disks is best way to get better save times. We've moved to vps with ssd disk and save time dropped from over 34 to 16-18s and that is just shared hosting. I've tested dedicated server with 2xSSD and save time was around 6-9s there. Our /data is over 700 MB including datafiles but w/o .bak files.
Datafiles can be a bit problematic because items in them will not be checked agains itemdesc.cfg files when pol starts. So when you change something in items you'll have to check your DS manually to ensure you can recreate items when needed.
You can try to mark inactive accounts, move banks to ds and recover them when account is reactivated but item issues will remain. I'm not really sure but i think that pol saves datafile on each worldsave even if file itself stays unmodified.
There is a bit of code for incremental saves in pol core but it looks like a work in progress.
Datafiles can be a bit problematic because items in them will not be checked agains itemdesc.cfg files when pol starts. So when you change something in items you'll have to check your DS manually to ensure you can recreate items when needed.
You can try to mark inactive accounts, move banks to ds and recover them when account is reactivated but item issues will remain. I'm not really sure but i think that pol saves datafile on each worldsave even if file itself stays unmodified.
There is a bit of code for incremental saves in pol core but it looks like a work in progress.
Re: Optimizing SaveWorldState()
Hi Yukiko,
Thank you very much for your considerations.
Of course, there is a processing cost to this. Before, all the items are loaded to the server only once (at server load). Now, they are constantly created between saves and destroyed at save. This could probably be avoided by changing SaveWorldState() to exploit this information, instead of hacking ourselves out of it.
Hi AsYlum,
Thank you very much for your considerations.
I think that the gain is that no more than 1% of banks/secures are opened between saves (Hence, they won't even be loaded into the server). We would only have to save back 1% of what today is storage.txt and that is loaded into the server back to the datafiles. There is no gain in using datafiles, per se. The gain is exploiting that, if something wasn't opened (and was't loaded to the server), it wasn't changed.Wouldn't you just be transferring the delay from writing out the storage.txt files to writing out the datafiles?
Of course, there is a processing cost to this. Before, all the items are loaded to the server only once (at server load). Now, they are constantly created between saves and destroyed at save. This could probably be avoided by changing SaveWorldState() to exploit this information, instead of hacking ourselves out of it.
Yep. I said "datafile" as a generic name. Since the "POL object" datafile only contain strings and integers, a lot would have to be done by hand to save the objects in such a file (CProps, etc...). It seems there is no function which automatically gets all information in an UObject and pickles it to some kind of .txt file (accessible to scripters). Of course, SaveWorldState() probably has something like that internally written to it but not accessible to usI wonder if it might be possible to script a custom storage management system that doesn't use the standard Core storage system but rather uses datafiles. Thus eliminating the need to transfer the object info from the storage.txt files in the first place. If I am not mistaken, datafile entries are written as members are appended to them. This would spread out the writes and thus solve the long write time during a world save.
Hi AsYlum,
Thanks for this useful information. I'll see what I can do about the SSD disk. Nevertheless, I'm hyped up about this idea of incremental saves, neverthelessSwitching to hosting with SSD disks is best way to get better save times. We've moved to vps with ssd disk and save time dropped from over 34 to 16-18s and that is just shared hosting. I've tested dedicated server with 2xSSD and save time was around 6-9s there. Our /data is over 700 MB including datafiles but w/o .bak files.
Don't you get errors anyway if you try to load the server and storage.txt has undefined items?Datafiles can be a bit problematic because items in them will not be checked agains itemdesc.cfg files when pol starts. So when you change something in items you'll have to check your DS manually to ensure you can recreate items when needed.