World syncing

Preliminary knowledge

Binary data blocks

Sessions

[article=p2p]Player-to-player messages[/article]

General knowledge of GML

Note: This tutorial uses the 'Blocks' example included in the download since GameMaker Server 2.0. For easy navigation, this tutorial is split into different sections that each explain a different part of the example. We assume that you're familiar with GameMaker Server, so the basics like logging in, player objects, etc. will not be explained. For automatic login, please see this tutorial.

General world structure

This section will not contain any implementation details. Instead, it will cover the way the example is representing the world. This has implications for both the saving and synchronizing parts, so please don't skip over the next bit!

A 'world' is...

Many objects aligned on a grid of one particular size (the example uses 32x32)

In each grid cell, there is only one object. We represent different objects using different numbers. For example, 1 might be a grass block and 2 might be a stone block.

There may be multiple grids layered on top of eachother, like a 'ground' layer and a 'buildings' layer.

There might be some objects that only appear very sporadically, like a sign. It doesn't make much sense to put them in a grid, since most of the grid would just be empty. These sporadic, or 'sparse' objects might need more than just a number to identify them. For example, the sign may need a 'text' property.

Our definition of a 'world' should fit most of the use-cases I've seen. However, if you have a use-case that doesn't, please let me know in the comment section below.

In the example, all world items inherit from 'parent_worldobject'. Sparse objects inherit from 'parent_sparse_worldobject'.

How to run the example

Start by running the example from GameMaker. It might log you in automatically, if you already logged in before. After logging in, you'll enter the world select screen. You can enter any number between 0 and 7, which will create or join a session for that world. If you're creating a session, you can enter a session name.

When in a world, press Q/W/E to place, update and remove signs, A/R/S to place and remove walls, and Z/X/C/V/B to change the ground type.

Note: Before making any changes, please change the GameID to something else in the Global Game Settings.

Session management

Our session management makes use of the new 'tags' included in version 2.0. The tags have the following format: "[bdb_id]|[world name]". We use string_extract (http://www.gmlscripts.com/script/string_extract) to extract one of the two parts of the tag, splitting on the '|'.

The world name has no further effect on the game. It's included just to show off the new tags feature. The bdb_id, however, is used. It indicates which world is being played in the session. If you enter a BDB ID that's already being played, we join the existing session instead of creating a new one.

To be honest, there's not much code here that's not explained in the session tutorial already. Most of it can be found in obj_world_select's <Enter> press event. The last bit is inside the on_session_change script.

Sparse objects

Sparse object (objects that appear only very sporadically) are implemented differently compared to normal blocks in a layer of the grid. Sparse objects all have a ds_map named object_data that may contain any additional data for the object, like the text on a sign. Normal blocks cannot store any extra data besides the block type.

Synchronizing the world

The example exclusively uses p2p messages to synchronize the world. The main reason for this is that instances are mainly intended to support objects that have a lifecycle shorter or equal to the life of the session -- in other words, instances are expected to be destroyed before the session ends. Manually syncing the blocks using p2p messages gives us more control over how long we want to keep objects alive. On login, we request that one of the other players immediately save the entire world to the BDB so that we can load the latest version. It looks something like this:

1. A logs in, and notices someone else (B) online

2. A sends p2p_request_world_save to B

3. B saves the world to the BDB

4. B sends p2p_world_has_been_saved to A

5. A loads from BDB that B just wrote to

The actual loading is discussed in 'Saving the world' below. After loading the full world when joining, we only need to send much smaller updates whenever someone destroys or builds a block. Note that for the normal (not sparse) blocks, we don't actually have a 'destroy' action. Instead, it's 'update block to blockid_nothing', which is practically equivalent but means that we don't have to write a separate destroy. The following three types of update are defined:

p2p_update_block - updates a single block on a single layer. For example, setting a grid cell to blockid_grass. It is sent every time you change a block on any layer. Implementation can be found in world_set_and_sync

p2p_update_sparse - creates or updates a sparse object. Implementation can be found in world_sync_sparse. This also sends over the entire contents of the object_data ds_map. (which for our sign, contains the text on the sign)

p2p_destroy_sparse - synchronizes a destroy on a sparse object. Implementation can be found in world_sync_sparse_destroy

Essentially, all we're doing is sending these updates to other players whenever we're creating, destroying, or updating objects.

Saving the world

To save the world, we're using BDBs. You should have read that tutorial by now, so I'll only explain the save format, and not how the BDBs work. The main part of the savefile is the saving of the grids. We store the following:

World width

World height

For each x between 0 and world width, and for each y between 0 and world width, and for each layer, we write the blockid. Imagine having a big table of letters on a piece of paper, cutting the paper vertically after each column, and then taping the bottom of each column to the top of the next. You now have a 'flat' string of letters, instead of the 2d table of letters you started with. That's basically what we're doing, only with blockIDs instead of letters.

Note in particular one trick that we're using to reduce file size: Each bdb_u8 can store values up to 255. However, the blockids in the example don't even go over 16. We can use something called bitpacking to squeeze two numbers into one. Without going too far off-topic, we're storing one of the numbers in multiples of 16, and the other in the remainder. Written out, it looks something like this:

(0, 0) stored as 0

(0, 1) stored as 1

(0, 2) stored as 2

(1, 0) stored as 16

(1, 1) stored as 17

(5, 7) stored as 5 * 16 + 7 = 87

(9, 3) stored as 9 * 16 + 3 = 147

When loading, we can revert this again to turn the single number back into two separate numbers. Going from two numbers to one number might seem like an insignificant difference, but remember that the example is already saving over 8000 blocks! Doing this just doubled our maximum world size!

For the sparse objects, we're doing things a little different. Storing them in a grid, makes little sense considering how sporadic these objects are. Instead, we're simply storing their position and the object_data. More concretely, the following is written to the BDB for each sparse object:

Object type. Because object_indexes can change, we're manually converting the object_index to a number. See world_init for the mapping.

X position

Y position

The object_data ds_map, stored as a string

Note that if your 'sparse' objects are too frequently placed in the world, they will use much more space than the grid-based solution!

Extending the example

The example uses a fixed world size. The current version of GameMaker Server will never be suitable for making seamless, infinite, worlds, however you can simulate this effect by using multiple BDBs for one single world. Each room will load and show one BDB, and by walking out of the room another BDB will be loaded. Since BDBs are not infinite, the world will still not be infinite, but it can be many times bigger than what it is right now.

To add more blocks, define macros for the blockids, and add more code to world_set. Note that each layer can re-use the same block IDs! For example, on layer 0, blockid 1 is grass. On layer 1, blockid 1 is a horizontal wall.

To add more sparse objects, create new objects that inherit from parent_sparse_worldobject. Note that you need to call event_inherited() if you add a create or destroy event to the object! You'll also need to call world_sync_sparse every time you update the sparse object.

Layers should be updated by calling world_set_and_sync. If you start changing the grids directly, or start using instance sync it will not properly save.

Make sure that you keep object_data in the sparse objects as small as possible. There's not much room to start saving entire essays inside the BDBs, since most of the space is already used for the world itself.

BDB size

By default any new game will have 8 BDBs available, each 16KiB in size. Please contact gamemakerserver@outlook.com if your game requires bigger BDBs, or if you need more BDBs so we can discuss options. If you mail me, please include the following: