Phantasmal Site > DGD > What Does It Do? > Swapping

DGD's Swapping Behavior

DGD calls itself a "disk-based" server. That means that DGD will swap out objects after they've been unused for awhile. It uses a standard LRU (Least Recently Used) metric to determine which objects should be removed from memory. This swapping is entirely transparent from the point of view of the LPC code. All LPC code executes as though every object was in RAM at all times, and objects will be pulled into memory as they are referenced.

The Configuration File

DGD's configuration file contains a parameter called swap_fragment. That's the fraction of all objects that should be removed from memory. For instance, if swap_fragment is equal to 20, then one-twentieth of all objects will be swapped out at the end of every execution round. A smaller swap_fragment will cause DGD to use less memory at a time, but will also mean more disk access happens after each execution round as objects are written to disk.

Objects are stored on disk as sectors. The sector_size parameter tells DGD how large a single sector is, in bytes. Every object takes up a multiple of the sector size in the swap file. So if the sector size were 1024, no object could take less than one kilobyte to store in the swap file. An object with 3217 bytes of data would wind up actually taking 4096 bytes in the swap file, or four sectors. A small sector size can save some space that way, though a very small sector size (like, say, 64 bytes) will require the DGD server to use up a lot of processor time and memory in managing so many sectors.

The swap_cache config parameter says how many sectors can be in RAM at once. If this is large, DGD runs fast. If it's small, DGD takes less memory.

Transparency

All of this nifty swapping stuff Just Happens(tm). That is, you should never, as a programmer, specifically need to request it. Objects are swapped out at the end of execution rounds according to the swap_fragment configuration parameter, and they're swapped in when you use them. It should never be necessary to explicitly request a swap-in for an object - just call a function of the object, even if that function doesn't exist, and the object will be swapped in.

If you ever wish to explicitly request a swap-out, there's a kfun called swapout() which will swap every object out. The function will take effect at the end of the execution round that requested it.

An object is swapped back into memory when one of its functions is called. Just the first sector will be swapped in when status() is called on the object. Note that calling find_object() does not swap the object in. Only operations that access the object's individual information need to do so.

When an object is swapped out, it will only be written out if it changes. Having large objects that change frequently can make for much slower swapping. Consider separating the highly-changeable parts of the objects into smaller DGD objects that will swap separately.

Memory Use

You'll find that the in-RAM memory use of your library, measured in sectors, is driven by several things. Roughly in order of importance, those things are: 1) the number of rooms and portable items in your world. 2) The time your MUD has been running — DGD is disk-based, so content is loaded as it is visited. 3) The number of active, moving users. Active users will continue to load content by seeing and using it. It would otherwise be swapped out. And 4) the aggressiveness of your cleanup routines, including things like swapping. The more aggressively you clean up, the shorter the time that 'transient' memory (memory that is used briefly and discarded) will take up space after it is used.

Disabling Swapping

If you experience very high usage by your MUD of your machine's CPU, but DGD's usage of ticks doesn't account for it, then DGD's housekeeping tasks, especially swapping, may account for the difference. If your MUD can fit entirely in RAM, you can find out if swapping is your problem by temporarily turning off swapping and only using RAM.

In your DGD configuration file, set swap_fragment to 0. That will turn off ordinary swapping, but not certain operations where all objects are swapped out to make room for larger contiguous blocks of static memory. By setting static_chunk to 0 as well, you can entirely eliminate all swapping in your application.

Remember to turn swapping back on afterward!

Massive Numbers of Objects

For lots of objects, you'll have to first increase the maximum number of objects and maximum number of sectors in the DGD configuration file. Both are quite small initially. DGD was initially designed to run in as little as a single megabyte of memory, so its starting configuration isn't appropriate for large MUDs. You'll have to change the file a bit.

Luckily, there is essentially no penalty for have a very large maximum number of objects, at least if you have the disk space to permit it and the RAM to hold the ones that are currently in memory.

The maximum number of possible objects depends on your operating system and your version of DGD. But remember that your swapfile will have to be no larger than the largest file your operating system supports. That can be as small as 2GB or smaller for some operating systems. Also, with the default data type for object numbers, you can't have more than 4 billion objects total. You can change that limit, but it would require significant effort.

From: dgd@dworkin.nl (Felix A. Croes)
Date: Tue Apr  5 01:29:01 2005
Subject: [DGD] Question about object references

Steve Wooster wrote:

>      I just remembered another question I had... an object isn't loaded 
> from the swap-file unless a function is called in it, right? For example, 
> if I had a list of, say, 10000 objects, and I wanted to process it (maybe 
> calling call_outs or something each with a different object as an argument) 
> without calling functions in the object or looking at their status or 
> anything like that, I could iterate through the whole array without using 
> much more memory than what the array takes, right? Thanks.

As long as you don't access the objects themselves, you can safely
manipulate an array of them (including testing whether objects have
been destructed) without loading any into memory.

Regards,
Dworkin