Download

Related posts:

1. Python Image Code
2. A simple texture algorithm – faster code and more results
3. A Simple Procedural Texture Algorithm – More Results and Code

In the Dev.Mag article Guerrilla Tool Development, I give some ideas for getting some useful tools on a tight budget. Check it out!

Related posts:

1. About Me
2. How to Turn XSI Mod Tool into a Level Editor for your XNA Games: Updated for XNA 3.0.
3. How to Turn XSI Mod Tool into a Level Editor for your XNA Games: Example Updated for XNA 3.0.

(Original image by Hljod.Huskona / CC BY-SA 2.0).

I used to hate neural nets. Mostly, I realise now, because I struggled to implement them correctly. Texts explaining the working of neural nets focus heavily on the mathematical mechanics, and this is good for theoretical understanding and correct usage. However, this approach is terrible for the poor implementer, neglecting many of the details that concern him or her.

This tutorial is an implementation guide. It is not an explanation of how or why neural nets work, or when they should or should not be used. This tutorial will tell you step by step how to implement a very basic neural network. It comes with a simple example problem, and I include several results that you can compare with those that you find.

I tried to make the design as straightforward as possible. The training algorithm is simple backpropagation. There are no hidden layers (I will treat that in an upcoming tutorial), no momentum, no adaptive learning rates, and no sophisticated stopping conditions. Those are, in a sense, easy to add once you have a working neural net against which you can benchmark more elaborate designs.

To keep the implementation simple, I did not bother with optimisation. This too can easily be addressed once a working neural net is in place against which you can verify correctness and measure performance improvements.

The brief introduction below is a very superficial explanation of a neural net; it is included mostly to establish terminology and help you map it to the concepts that are explained in more detail in other texts.

Preliminary remarks and overview

What we are doing

The problem we are trying to solve is this: we have some measurements (features of an object), and we have a good idea that these features might tell us in which class the object belongs. For example, if we are dealing with fruit, knowing the size, colour, and “roughness” of the skin, we might deduce which type the fruit is. Of course, we want a program for this.

Now in heaven, there is a function for exactly that task – we give the function the features, and it spits out the class.

Down here on earth, we are not so lucky (well, not for most problems anyway). So instead, we have to do with some approximation. A neural net is one possibility – there are also others. A neural net (one without any hidden layers) is parameterised by a weight matrix. Different problems in general have different weight matrices. To solve our problem, we need to find a suitable matrix.

If both our set of known samples and the problem itself are reasonable, we might expect to find such a matrix. But not directly – we have to use some kind of iterative scheme – we have to “train” our neural net. We use some of the known samples for this.

Behind the scenes, the weight matrix and the feature vector are combined using some matrix operations to give the output vector, which is converted to a class. The mathematical details of this can be found elsewhere (more).

After we have trained the neural net, we can use it in an application to classify objects that we may or may not have encountered before. Generally, the training and application programs are separate. The training program is the difficult part; most of the tutorial deals with training.

It is common to divide the known samples in three sets, called the training set, validation set, and test set, typically divided in the ratio 50%, 25%, and 25%. The training set is used to update the weights iteratively; the validation set is used to stop the training algorithm, and the test set is used to estimate how well our trained neural net will do in the wild.

Each iteration of the training algorithm is called, quite poetically, an epoch. Many different things affect the number of epochs we will need to train the neural net: the problem complexity, how we scale our updates, and how the weight matrix has been initialised.

We can control the speed with a parameter called the learning rate. In general, higher learning rate means faster learning (although, when it is set too high, the network might become unstable and not learn at all).

Representation

We usually represent the features (our measurements) as a vector of numbers, like this:

[0.01, 1.07, 3.60]

In cases of non-numeric values, we convert whatever we have to numbers with some scheme. For instance, when dealing with colours, we might want to use RGB values, or we might want to make a list of colours (red, yellow, green, etc.), assign a number to each of these labels, and use the corresponding number.

Classes, usually not being numbers, are similarly treated: we assign a number (often arbitrarily) to each class, and use that in our computations. For the actual training, however, we use an output vector. The output vector has a 1 in the position of the class number, and 0 everywhere else.

Thus, if we are dealing with three classes of fruit (apple, pear, banana), and number the classes 1, 2, and 3 respectively, we will have the following output vectors:

It is convenient to put all the inputs of a set together in a single matrix, where each row is a sample. Similarly, outputs and classes are also put into matrices, with input sample in a row (say row number n) corresponds to output sample in row n, and also the class in row n.

The implementation below makes use of high-level matrix operations. This avoids many of the errors that can creep in loop-dense code. The following table will give dimensions of all the matrixes involved; this will be helpful during implementation (especially for assert statements). The numbers are given in terms of the number of inputs (features) of the problem, the number of outputs.

Overview

The basic idea of the training algorithm is the following:

First we load in the data: the training samples, the validation samples, and the test samples.

Then we start to train: we run the backpropagation algorithm on random samples. After each iteration, we see how our network is doing so far (on the validation set), and then we decide whether to keep training or not.

After we stopped, we do a final evaluation of our network on the test set – this gives us an indication of whether the neural net will generalise well to samples not originally in the training set.

After we have found a weight matrix that we can live with, we cfan incorporate this in the application were we need the functionality.

The training program has x functions:

• train
• load data
• feed-forward
• evaluate network
• backpropagation
• output to class
• class to output
• activation
• activation derivative

The application program uses these functions:

• feed-forward
• output vector to class

Implementation

1. Gather the necessary libraries (or write them)

You will need the following libraries:

• A library that supports matrix algebra; and
• A library that plots graphs (x versus y).

If you can’t find a matrix library for your implementation language, then you can write a simple library yourself. Since neural nets do not require matrix inverses or long chains of matrix products, you need not worry (much) about numerical stability, thus the implementation is straightforward.

The implementation needs to support the following matrix operations:

• matrix transposition;
• matrix addition;
• matrix multiplication with a scalar;
• ordinary matrix multiplication;
• Hadamard multiplication (component-wise multiplication);
• Kronecker multiplication (only necessary for between row and column vectors); and
• horizontal matrix concatenation.

The first few operations are standard matrix operations, but you might be less familiar with the last three. (Also check out the Wikipedia article on matrix multiplication – it covers all the types of multiplication mentioned here.)

Hadamard multiplication of matrices is defined for two matrices of equal dimensions. Each component of the new matrix is the product of corresponding components in the two multiplicands, that is:

Z[i][j] = X[i][j] * Y[i][j]

The Kronecker product of a row vector and column vector is defined as a matrix whose components are given by:

Z[i][j] = X[0][i] * Y[j][0]

It is possible to define the product for arbitrary matrices, but we don’t need it.

The horizontal concatenation combines two matrices with the same number of rows. For example, the matrices A and B below will be concatenated to form the new matrix C:

A simple implementation simply constructs a new matrix whose components are given by

if j < X_width
  Z[i][j] = X[i][j]
else
   Z[i][j] = Y[i, j – X_width]

If no graph libraries are available, simply write a function that will output a tab-separated list of the input and output sequences to plot. You can then load or paste this into your favourite spreadsheet program to make the necessary plots.

2. Implement Output and Class conversion functions

This is very simple: implement a function that converts an output matrix to a class number vector, and another that converts a class number to an output vector.

For example, the output_to_class function will take the following matrix

1 0 0

0 1 0

0 0 1

1 0 0

0 0 1

and convert it to:

1

2

3

1

3

(The second function will convert the second matrix back to the first matrix).

3. Implement a function to read in data files

For this tutorial you can use the following three files:

• iris_training.dat
• iris_validation.dat
• iris_test.dat

These three files contain samples from the ICU iris dataset, a simple and quite famous dataset. In each file, samples or contained in rows. Each row has seven entries, separated by tabs. The first four entries are features of irises (sepal length, sepal width, petal length, and petal width); the last three is the outputs denoting the species of iris (setosa, versicolor, and virginica). I have preprocessed the values a bit to get them in the appropriate ranges.

You must read in the data so that you can treat the inputs of each set as a single matrix; similarly for the outputs.

I find it useful to store all the data in a structure, like this:

• data_set
  • input_count
  • output_count
  • training_set
    • inputs
    • outputs
    • classes
    • count
    • bias
  • validation_set
    • inputs
    • outputs
    • classes
    • count
    • bias
  • test_set
    • inputs
    • outputs
    • classes
    • count
    • bias

This makes it more useful to send the data as parameters.

4. Implement an activation function and its derivative

The activation function must take in a matrix X, and return a matrix Y. Y is computed by applying a function component-wise to X. For now, use the hyperbolic tangent function:

5. Implement the feed-forward function

The function must take as arguments an input matrix, weight matrix, and a bias node matrix.

The function should return an output matrix, and a net matrix

These are computed as follows:

net = mul(weights, horcat(inputs, bias))
output = activate(net)

The bias matrix is a constant column vector of 1s with as many rows as the input matrix. This vector corresponds to the bias nodes. The implementation here is a bit clumsy, but for now, the approach used here minimises the potential for error.

6. Implement a weight initialisation function

This function must take in a maximum weight, a width and height, and return a matrix of the given width and height, randomly initialised in the range [-max_weight max_weight].

7. Implement a function that evaluates the network error.

The function must take in:

• an input matrix,
• a weight matrix,
• a target output matrix,
• a target class matrix,
• a bias matrix.

The function must return the error e, and the classification error c.

To compute these, first compute the output matrix Z using the feed-forward function (you can ignore the net matrix).

[output net] = feedforward(inputs, weights, bias)

Now subtract the target output matrix from the output matrix, square the components, add together, and normalise:

error = sum_all_components((target_outputs – outputs)^2) ...

   / (sample_count * output_count)

From the output matrix, calculate the classes:

classes = classes_from_output_vectors(outputs)

Count the number of classes that corresponds with the target classes, and divide by the number of samples to normalise:

c = sum_all_components(classes != target_classes)/sample_count

(Here, our inequality returns a matrix of 0s and 1s, with 1s in positions where the corresponding components in classes and target_classes are not equal.)

8. Implement a dummy backpropagation function

The function should take in:

• An input matrix
• A weight matrix
• a learning rate (eta, as in the Greek letter)
• a bias vector

The function must return an updated weight matrix. For now, return W as is.

9. Implement the train function

The training function should take in three sets, the training_set, validation_set, and test_set. Implement a way to limit the maximum number of samples that will actually be used for training (you can also do his in the main program described in the next section). This is very helpful for debugging purposes (especially if you plan to later replace the backpropagation algorithm with something a little faster – and more complicated).

The function should return a weight matrix, and error values as floats.

Initialise a value plot_graphs to true. This is a debug flag, so it is appropriate to implement this as a macro if it is supported by the implementation language.

The function should initialise a weight matrix using initialise weights. For now, use a max_weight of 1/2.

The function should also construct three bias vectors bias_training, bias_validate, and bias_test. Each must contain only 1s, with as many rows as there are inputs in the training, validation and test sets respectively.

Implement a while loop that stops after 500 iterations. (We will change the while condition later to something else, so do not use a for loop).

Inside the loop, call the backpropagation algorithm. Use the training set inputs, the weights, (for now) a fixed learning rate of 0.1, and bias vector bias_train. Assign the result to weights.

Still inside the loop, call the network error function three times: one time for each of the training, validation, and test sets. Use the weight matrix, and the appropriate bias vector. Wrap these calls in an if-statement that tests for a value plot_graphs. (If your language supports it, you can use conditional compilation on the value of plot_graphs).

Store the errors in six arrays (error_train, classification_error_train, etc.), with the current epoch number as index.

After the loop, plot the six error arrays as a function of epoch number. Wrap this in an if-statement (or conditional compilation statement) that tests for the value plot_graphs.

Call the network error function again, on all three sets as before.

Return the weights, and the six errors.

10. Implement the main training program

The program should load in the sets (using the load_sets function), and pass these to the training algorithm.

11. Run the program

The important thing is that everything should run. You should see your error plots; at this stage they should be straight, horizontal lines. Because of the random weight initialisation, we cannot predict where these lines will lie (so do not be alarmed if they do not look exactly the same as below – as long as they are straight and horizontal).

12. Implement the backpropagation function

You have already created the dummy function; now you can put in the actual calculations.

First, select a random sample.

Now, calculate the net matrix and output matrix using the feed-forward function.

[output, net] = feedforward(random_sample, weights, bias)

Calculate the error vector

error_vector = target_outputs - outputs

Calculate the sensitivity.

delta = hammard(error_vector, activation_diff(net))

The corresponding mathematical expression in the textbook might look like this:

Calculate the weights delta:

weights_delta = scalar_mul(eta, kronecker(transpose(outputs), delta))

The corresponding mathematical expression in the textbook might look like this:

Update the weights:

weights = add(weights, weights_delta)

and return the matrix.

13 Run the program (AGAIN)

First, set the debug option to train on only one sample. The curve should like like this:

Notice that the error curves are smooth, the training error rate goes to 0, and the other error rates go to 0.4, plus or minus – this depends on the initial weights. It will also be different if you use a sample other than the first.

If your curve looks right, change the debug option to train on all samples. Your error curves should now look something like this:

The important feature of the error curves is that they should steadily descent on average. If your curve does not resemble the one shown here; there is a mistake in your implementation. It is often helpful to limit the training set to a single sample (thereby eliminating the randomisation of sampling during training) to see what is going on.

Although the errors often follow the relationship training < validation < test, this is not always the case, so do not be alarmed if this is not true on a run (You can see it is not always true in the run above).

14 Implement a proper stopping condition

Change the while loop to stop when the validation error drops below a threshold. Note that this threshold usually depends on the problem. There are better stopping conditions that are less sensitive to the problem at hand, but this one will do for now.

15 Implement a statistical analysis

This part is important for you to get an idea of the robustness of the neural net. In practice, a very simple analysis will suffice.

This part need to train the algorithm 30 times, and then report the mean, standard deviation and maximum of the

• training time,
• regression error, and
• classification error (on the test sets).

In general, you would like all these values to be “low”. Here are some experiments for the iris data set with different learning rates. For each, 30 runs were made; other parameters are as described earlier (max_weight = 1/2, validation_stop_threshold = 0.1).

Using the neural net in a program

To use the neural net in a program after you have trained it, you need to save the weights found by the training program to a file. Your application must then read in this weights, and use it with the feedforward function to calculate the class.

Here is pseudo-code for the program. Note that the training program and classification program needn’t be implemented in the same language. This allows you to take advantage of speed and interface components that might not be available in your target platform. It is a very good idea to implement your training algorithm on a computer algebra system (such as Matlab or Octave) where you can take advantage of both matrix and graphing capabilities (the code provided below works in both).

Using the training algorithm on other problems

When using your neural net for other algorithms, you might need to change the learning rate, stopping threshold, and weight_max for weight the initialisation. The error plots are indispensible for this purpose.

As long as the learning rate is not too high, it should not affect the quality of the solution, only the number of iterations necessary to obtain it. The stopping threshold, however, has does affect the quality of the solution: if it is tool low, the problem might not be solved, or the neural net might train very long; if it is too high, you will get poor performance. There are better stopping conditions available; once you have everything working, you should investigate these.

Remember that you should not base your decisions on a single run, as runs can differ quite drastically from one another. Perform a few runs, and base decisions on these.

The value weight_max can usually be chosen as 1/sqrt(input_count) with good results, but here too it depends on the problem.

For further benchmarking, check out the Proben1 datasets, described here. (Also check out the erratum note on this site – search for proben on the page.)

Download

The following code works in Matlab and Octave. (Included is a randint function; if you are using Matlab you can remove it, because it is already implemented in Matlab).

iris_data_files.zip (3 KB)

basic_neural_net_0_1.zip (10 KB)

The program provided cheats a little – instead of reading in the raw .dat files, it reads in the .mat file that already has the data in the right structure.

Related posts:

1. Generating Random Numbers with Arbitrary Distributions
2. Cellular Automata for Simulation in Games
3. 5 Tips for Prototyping Slow Algorithms

I will cover the case for 2D sets here, but the ideas are easily extended to any dimension.

Visual Inspection From a Scatter Plot

The easy way to estimate the distribution is to simply look at a scatter plot of the samples.

	A scatter plot of a 2D sample set. (A pixel was simply drawn at each point of the sample. The images was slightly blurred to give the pixels more substance).Here we can already estimate the distribution, as we can intuitively “see” the shape.

If the distribution is simple, we can often “guess” suitable parameters for a mathematical function. But this method is not suitable when more accuracy is required. And obviously this method does not work for higher dimensions.

I always use a visual inspection as a first step when using the other methods described here. First, it allows you to decide which approach to use. Second, it serves as a rough benchmark to test the results of a better method against.

Averaging Over a Grid

For this method, we divide the domain in cells, and count the number of points in each cell. To normalise, we divide the counts by the total number of points to get a distribution.

Typically, we choose a cell size large enough so that all cells contain a few points.

It is possible to use different cell sizes, so that regions of higher density has more cells, for instance, by using a quad tree. However, this approach makes it difficult to interpret the values (they should be scaled down by a factor of the area they represent), and to generate random numbers from this distribution. I can’t really imagine a situation where this approach will be preferred above using a regular grid or convolution.

Convolution

There are two ways to implement convolution. The first should be used when the sample set is small relative to the size of the domain, otherwise the second method will be more efficient.

Sparse Point Convolution

Generate a very granular empty grid over the domain.

For each point p in the sample set,

• map p to the grid (calculate the coordinates x,y of p in the grid),
• add a number to all the cells in the neighbourhood of p in the grid.

Now normalise the grid.

The neighbourhood is typically a circle or a square. The number you add can be the same for all neighbours (in which case any positive number will do), or it can be scaled depending on the distance from p.

The size of the neighbourhood depends on the density of your sample. In general, the denser it is, the smaller the neighbourhood can be. Smaller neighbourhoods lead to faster execution. To prevent “holes” in the approximation, use a radius that corresponds to the maximum of the distances between a point and its closest neighbour.

	Estimation with a constant square neighbourhood.
	Estimation with a constant circular neighbourhood.
	Estimation with a circular neighbourhood with a falloff.

Traditional Convolution

Generate a very granular empty grid over the domain.

For each point p in the sample set, map p to the grid (calculate the coordinates x,y of p in the grid), and add one to that location in the grid.

Now choose a square, symmetrical convolution matrix, and perform a discrete convolution on the grid:

new_grid[i, j] = sum_{i,j} grid[i][j] * c[i][j]

Here the i, j go over the indices of the convolution matrix. (Normally, the convolution is defined as new_grid[i, j] = sum_{i,j} grid[n - i][n - j] * c[i][j]. However, since we are suing a symmetrical matrix, these definitions are equivalent, and we need not perform the extra calculation).

Now normalise the grid.

The convolution can be a square or circle of 1s, or be filled with numbers that grow smaller outwards. These correspond to the three neighbourhoods described for the sparse convolution.

Note that the new_grid is larger than the original by one less than the size of the convolution matrix in each dimension. The centre of the new grid corresponds with the original grid.

For example, if the original grid was 100×100, and the convolution matrix was 5×5, the new grid will be 104×104. The point (2, 2) in the new grid corresponds to point (0, 0) in the original grid.

About Normalisation

It is customary to normalise the distribution so that all the probabilities add to 1. But for many purposes we need only relative probabilities, and this step can be skipped. For example, the method of generating random numbers described in a previous post uses only relative probabilities.

A Few Tips

• Always test your distribution by generating a random set from it, and comparing it with the original sample. They should match qualitatively.
• The smaller your sample set, the cruder the approximation should be. That is, cells, neighbourhoods or convolution matrices should be big. There is a limit to the accuracy you can obtain from any sample – if you try to exceed it, your results will be poor.
• The most common implementation errors are made at the borders (of any grid or matrix) – watch out for them!

Download

There is an example of implementation in 2D in with the Python Image Code. See the file random_distributions_demo.py.

Related posts:

1. Generating Random Points from Arbitrary Distributions for 2D and Up
2. Generating Random Integers With Arbitrary Probabilities
3. Generating Random Numbers with Arbitrary Distributions

A cellular automata system is one of the best demonstrations of emergence. If you do not know what cellular automata (CA) is, then you should go download Conway’s Game of Life immediately:

Conway’s Game of Life

Essentially, CA is a collection of state machines, updated in discrete time intervals. The next state of one of these depends on the current state as well as the states of neighbours. Usually, the state machines correspond to cells in a grid, and the neighbours of a cell are the cells connected to that cell. For a more detailed explanation, see the Wikipedia article.

Even simple update rules can lead to interesting behaviour: patterns that cannot be predicted from the rules except by running them. With suitable rules, CA can simulate many systems:

• Natural phenomena: weather, fire, plant growth, migration patterns, spread of disease.
• Socio-economic phenomena: urbanisation, segregation, construction and property development, traffic, spread of news.

In games, these can be used as game elements (such as fire and disease), or they can also add some decorative life to a game (such as city development in a simulation game).

The great thing about CA systems is that they are not hard to implement, and they have intrinsic fun value (I mean here, for the programmer!).

Before we get into the details, let me add a small disclaimer here:

• This tutorial is not comprehensive. I only touch on a few ideas in a wide field. See some of the resources at the end for more information.
• This tutorial is not representative. I merely chose topics I found interesting or thought would be useful for game developers.
• I am not suggesting that using cellular automata is the best way to simulate any of the examples given here. For any problem, there might be solutions that are faster, easier to implement, or yield better results.

Implementation

To implement a CA system, you need to do the following:

• Use a suitable system to store states, for example, a 2D array.
• Decide on a suitable neighbourhood shape, and work out some preliminary rules. Decide how you will handle edges.
• Implement a suitable way to describe your rules.
• Implement an update function that hooks into the main game loop.
• Implement a visualisation scheme (for example, drawing tiles depending on states in the array).

Some details of these are described below.

Grid

A typical implementation uses two grids for storing states. One is the current world state; the other is the new world state. Once all cells have been updated for the current iteration, pointers to the arrays can be swapped. This way updating the cells one by one will not have an effect on other cells until the next iteration, and you do not have to create a new array in each iteration.

Using grids with square cells are the easiest to implement, but the concept is also applicable to other grids: hexagonal and triangular grids are also common, and even polar grids have been used to simulate spiral star systems.

It is also possible to use irregular systems, although the implementation then becomes a challenge. One example of an irregular system is a presentation based on a Poisson distribution of points. The trick here is to find an efficient way to locate neighbours. Fortunately, this problem has already been solved in creating the points (see the Poisson Sampling tutorial in Dev.Mag and download code from Luma Labs). It is in fact done by using an invisible grid! Of course, visualising a state is also a bit more difficult.

Neighbourhoods

For 2D cellular automata, the most common neighbourhoods are the Von Neumann neighbourhood and the Moore neighbourhood, shown below (the blue cell’s neighbourhood is shown in red). Other neighbourhoods are also possible.

Von Neumann neighbourhood	Moore neighbourhood	Extended Von Neumann neighbourhood

The cell can be included in the neighbourhood or not. It is advisable to keep the neighbourhoods as small as possible: not only does the neighbourhood size impact performance, it also affects the number of rules to be specified and thus the tweaking complexity.

Specifying Update Rules

Specifying the rules concisely is the most challenging part of implementing CA. With very simple models, you might want to use a few if/else statements. This, however, makes it hard to modify (tweak or extend), and is therefore not suitable for more complicated systems.

It is best to implement rules as a table that can be indexed using the neighbourhood states.

Table as an array. In this method, each row represents the combined state of the surrounding cells. The final entry in the row is the new state of the current cell. Each of the remaining columns represents a state. The entries in these columns represent the number of cells surrounding the current cell in that state.

A cell surrounded with three cells in the Dry Bush state and one cell in the Burning Bush state will go into the Burning Bush state.

This table can be quite large, so that you must take care to implement lookups efficiently. Keep the table sorted, and perform binary search to lookup entries. Below are some improvements on this scheme.

Table as a tree. In the table above, it is easy to see that if there are three dry bush nodes, there are two possibilities for the other states (either 1 0, or 0 1). We can use this to implement a tree. Each level in the tree represents a state. A value in a node represents how many cells in the neighbourhood are in that state. A path from root to leaf fully describes the states of the entire neighbourhood. We do not add nodes for the last state, as it is uniquely determined by the other states.

Below is a tree corresponding to the piece of table above.

BB = Burning Bush

DB = Dry Bush

This is a more efficient implementation; unfortunately, it is also more complicated. It is inconvenient to specify the tree in a tree-friendly format (XML or s-expressions, for example). You still need to specify it in table format (space separated text file, for example) and build the tree from that.

Implementation as a hash table. Here you use tuples of state counts as keys. For example, the third entry in the table above can be accessed with the tuple (3, 0, 1). If your implementation language does not support tuples, you might need to implement them yourself. Be sure to make them immutable!

Implementation as an N-dimensional array. This is a very simple method, although it wastes a lot of memory. N is the number of states, and we use the state counts to index into the array. In our example, we will use a 3D array, so that the third element can be accessed using transition_array[3, 0, 1].

The method above can be used even when the implementation language does not provide N dimensional arrays. Simply use a 1D array, and interpret the state counts as the digits of a base-4 number. Thus, the third row will be located at 3*16 + 0*4 + 1*1 = 49.

Note that since the numbers are restricted by the requirement that they add to four, there will be unused array positions in both these last two methods. You can save memory by not using the last state to index (since it is uniquely determined by the other states). Thus, we can use a 2D array, and access the 3,0,1 rule with 3,0 alone: transition_array[3, 0]. Alternatively, if we need to use a 1D array, we can store the rule in the position 3*4 + 0*1 = 12. Although the same scheme is possible for the hash table implementation, I would advise you to use the full indexing scheme instead, since it will not waste more memory if you do, and it is somewhat easier to understand if you use full indices.

Some rules do not need the detailed tables above. For instance, in a system of only two states, where transitions depend on the count of the states of the neighbourhood, we can use a simple array, indexed by one of the states’ counts. Many other schemes are possible: in general, aim for quick lookups and easy specification.

Edges

The edges of your simulation might present some problems: edges often serve as unwanted sources, sinks or stabilisers.

A source is the origin of outward growth (states change progressively to a specific state in an outward direction).

A sink is the exact opposite:

Sources and sinks are emergent properties of the CA system, and are not physical attributes of the cells.

A stabiliser is a section that keeps neighbouring states fixed. For example, in the fire simulation (described below), there is an unnatural wall of fire around the edges at the end of the simulation.

There are several ways you can handle edges:

Ignore them. When the edge anomalies are acceptable, it is best not to complicate the model. This is only possible if the rules are specified in a way that does not require state counts to sum to the size of the neighbourhood. For example, if there is a rule “if num_states_fire > 1, then …”.

Wrap around. This solution is very elegant, easy to implement, and its effects are easy to understand. However, it is usually easy to see that the world has been wrapped in this way, and hence it is not always a visually a satisfactory solution. It is possible, however, to simulate a grid bigger than displayed area (say about double). This will hide the fact that it is wrapped, but still give you the benefits of having a world without any discontinuities.

In the CA system shown left, we want to calculate the next state of the grey cell. Two neighbours fall outside the edges. If we wrap around, we use the four red squares as shown.

Reflection. Use a neighbouring state to perform calculations.

The CA system shown left is the same situation as above. If we use neighbouring states for the cells beyond the edges, we use the state of the grey cell (twice!), in addition to the two red cells shown.

Fix their states. The edges can be given special, constant states that make the CA behave as intended. Finding the right state(s) for the edges might be difficult.

An implementation trick that applies to the last three edge-handling techniques is to include a border in your simulation. For the Von Neumann and Moore neighbourhoods, this border is only one cell thick.

This border is used for updates like all other cells, but is itself updated differently. The update rules for wrapping and reflection will simply copy states from the right places; the border for fixed states will never be updated.

In the figure, hatched cells are updated according to the special rules of the edge-handling scheme – in this case, wrapping is used, so they are simply copied from the appropriate cells. All other cells are updated normally. Note that here we do not need special logic for updating the grey cell.

A Few Examples

(Also see the downloads at the bottom of the post).

Heat

Here, continuous states (floats between 0 and 1) are used. Every cell is updated to a weighted average of its Moore neighbourhood cells: corners have a smaller weight than other cells. For this simulation, heat sinks and sources have been added: they keep the cell’s temperature at 0 and 1 respectively.

City Roads

This simulation use separate neighbourhoods for creating and destroying a road. The creation neighbourhood is an extended Von Neumann neighbourhood; the destruction neighbourhood is a Moor neighbourhood. Creation and destruction is according to probabilities based on the count of neighbouring roads.

Influence

This simulation models human emotion as influenced by others. A character has an internal state, and an externally visible state. Characters can only observer the external state of their neighbours. The internal state is then a modified: a character surrounded by angry people will itself become angry, and so on. The external state is probabilistically determined by the internal state.

Disease Spread

For our game Epidemic, Chris and I used a simple diffusion model. We used a floating-point number for the state, and represented the number of infected people in a cell. We used different colours for different ratios to visualise infection. At each time step, a cell would become infected by an amount proportional to that of the neighbourhood, with some random scaling.

Visualising States

Versions of tiles

A very effective way of making CA simulations more impressive is to have more than one tile for each state, and display them randomly. This is especially helpful for hiding regularity in off-line simulations. Tile sets with different versions for corners, edges, and central regions can drastically improve the look of the game, although it complicates choosing the correct tile version. In addition, it means you need tiles for various combinations of states. Assuming we can rotate and reflect tiles, in the fire example with three states, we need the following tiles:

3 centre tiles
3 straight-edge tiles
3 diagonal line tiles
3 cross tiles
6 corner tiles
3 half-straight tiles

This is 21 tiles for only three states. It is advisable to automate tile creation.

Tile

States can be animated. Instead of displaying a single tile, display an animation sequence. You will need to think about how to start and stop these animations so that they are seamless. For instance, you can make all animation sequences the same length, and only update states between animation sequences.

You can also increase the seamlessness of the simulation by animating transitions between states. This can be as easy as blending between two tiles over a period of time (assuming that updates are made far enough apart).

No Tiles

Instead of using tiles to represent a state of a cell, you can put objects in a cell. For instance, in a plant growth simulation, you can put a number of trees in the cell depending on the state. The advantages of this are:

• You do not have to create dozens of tiles for variety.
• The player can interact with the objects. For instance, the plant placement of trees can hinder a player’s movement.

This shows a simple forest simulation. Trees grow and die depending on the surrounding trees. This system uses a grid, but not tiles: trees can grow in random positions in tiles. For this simulation, three random coordinates have been chosen for each cell at the beginning of the simulation. Depending on the states, one, two, or three trees will be drawn at these points. Choosing a fixed set of points makes it easier to draw trees at the same coordinates every tick. A somewhat more complicated simulation can choose new points whenever a new tree comes into being.

CA Enhancements

The images below show three frames from a simple fire simulation.

The green is grass (Dry Bush), the orange is fire (Burning Bush) and the brown is burnt grass (Charcoal).

As you can see, the simulation captures some features of a fire:

• fire grows outward,
• grass only burns once,
• grass needs fire to catch fire (there is no spontaneous combustion).

Even so, the symmetry and regularity makes this fire very unconvincing. Below we look at some ideas to enhance a simple CA simulation.

Probabilistic State Transitions

Instead changing to some specific state based on the surrounding states, you can change to any state with some probability. These probabilities depend on the surrounding states, and can possibly be zero.

Adding probability to the mix can have several advantages:

It introduces variability in patterns. This means a specific (perhaps the start-up pattern) will not always produce the same results.

It can enhance the organic look of the system. Cross and squares patterns are common in deterministic CA systems. For many purposes, this is unsatisfactory (have you ever seen a fire spreading as a growing square?) Adding probabilistic state changes will make the patterns more organic and varied.

It can make discreet behaviour more continuous. For example, two common “problems” with CA systems are exponential growth (where all cells go to a certain state outwards very quickly), or sudden death (where all cells go to a certain state very quickly inwards). The problem is often that a single change in the update rules results in a exponential growth system to become a sudden death system, with seemingly no way to get a “sweet spot”. If states can only change with a certain probability, tweaking the probabilities allows you to get intermediate patterns (stable growth, equilibrium, or stable death).

It allows you to control the speed of the simulation. It is especially helpful to slow down a simulation without making it more granular (in the time dimension). It is useful to multiply all your probabilities with a global parameter that you can tweak to control the speed.

Note that in the example above, certain cells did not burn at all – per chance, they did not caught fire when their neighbours were burning. Timers (see below) can reduce or eliminate this effect.

Adding randomness can increase the emergence, and hence also unwanted emergence, so care must be taken that the simulation is always valid.

Tile Interpolation

Normally, you want your simulation to be as simple as possible. However, when such simple simulations are displayed (visualised), the results can be unsatisfactory. One way to get better visual quality is to use tile (cell) interpolation to make the simulation seem more complicated than it really is. When combined with a bit of randomisation, the effects can be very convincing.

The basic idea is to simulate only every nth cell with the CA approach. Other cells are interpolated. The number of interpolated cells can be adjusted, depending on the detail level required (balanced with the combinatorial number of tiles required!)

In the fire simulation, using only three states makes the simulation seem very primitive. Suppose that we simulate only every second row, and in these, only every second column, and fill the remaining cells depending on their neighbours. We have to create a function that will return a tile for each combination of surrounding tiles:

get_center_tile(north, east, south, west)

Interpolation rules are similar to update rules of the simulation, and hence can easily be as complicated. If this is the case, you would do better to implement more states and not do any interpolation. Interpolation should only be used if it significantly reduces the implementation complexity.

Timers

Some interesting effects can be obtained if each cell has a timer, that is, each cell only updates once the timer expires. Some rule determines when the timer is reset. For instance, the timer can be set:

• randomly,
• as a function of the state,
• as a function of the surrounding cells,
• as a function of another property of the tile (*1),
• as a function of some external process (*2).

(*1) For instance, if each cell has an associated “burning capacity” parameter, (simulating material types), a fire state timer can be set in proportion to this parameter.

(*2) For example, we can set timers according to a wind factor.

Timers can have the following effects:

• add some randomness to a simulation,
• slow the simulation down,
• smear the simulation (for example, in the fire simulation, the burning border is much thicker, and no cells are skipped).

Although you can often implement your grid cells as classes (or structs, or whatever aggregating construct your language provides) with states and timers embedded, it is simpler (on more efficient) to have separate integer grids – one for states and one for timers.

Note that if timers do not have the same expiration time, you the CA system is asynchronous and the behaviour can differ dramatically from a synchronous CA system. Thus, you should include timers early in the development process, so that their effect does not force you to re-tweak all the parameters.

If your world is big, you cannot run the CA for the whole world at every time step – it will kill performance. Below are some ideas for improving performance.

Instead of simulating the entire world, you can only simulate the part that is relevant to the game, usually the visible region of the game. A simple technique is to use a wrapped grid, similar to the grids used for the visual port in scrollable 2D games. This grid can be slightly larger than the visible grid, so that it allows cells to update naturally for a few iterations before they become visible, and so increase the illusion of a persistent world.

To hide that fact that you are only simulating a very small piece of the world, you must update “new” cells appropriately. There are several options:

Update cells to a natural starting state. In many cases, this solution is adequate, especially if you simulate a slightly larger grid than is visible.

Copy states from the existing simulation. Here you can copy neighbouring cells, or randomly from a nearby region. Copying from nearby cells promotes spatial coherence.

Initialise states statistically. Based on a neighbourhood of cells, each cell is initialised probabilistically. For example, each cell can be initialised randomly to one of three neighbouring cell’s states.

Use hierarchical modelling. Use another CA system that represents the entire world in very low detail, and use this to initialise the cells of the high detail version. Note that both the states and the update rules will differ for the high and low detail simulations. Use probabilities to prevent homogeneous systems. You can even use several levels of detail, and incorporate them as needed. Beware the extra implementation complexity, though!

This technique does not work when the world is should be changed permanently.

Instead of only simulating a part of the world, you can only update a part of the world. This requires enough memory for the whole world, but only processing power for the cells you wish to update. This method has the advantage of real persistence, thus permanent changes can be reliably simulated. Which cells you update will determine how your simulation will play out:

• Update all visible cells (or a slightly larger region). This is the most natural update scheme, and its effects are similar to partial simulation described above.
• Divide the world into discontiguous regions, and update a different region each time step. For example, divide the world into two sets: one contains every other cell, the other the rest, and update these two sets alternately. To save a significant amount of processing, you need many sets. To hide patterns in the updates that might break the illusion of a continuous world, the sets should be irregular. This method has the advantage that real things happen off screen. On the down side, it slows down the simulation significantly.
• Combine the two methods above.

Partial updating, like the inclusion of timers, lead to an asynchronous system, and hence the behaviour might be very different from that of a fully updated system.

In certain cases, simulations can be baked offline and be used in games. This means you perform the simulation before the time, and record each frame. The game then reads these frames, and plays them back.

Various schemes are possible to get more with less. For instance, you can also interpolate the simulation in time. With some thought, you can simulate events in a way that they can be looped, stacked together, and started and stopped unobtrusively. This means you can blend them with the interaction in a way that will not make it obvious that the simulation has been performed off-line.

Unwanted Emergence

Not all emergence is good, and you might find it hard to prevent certain behaviours. If the unwanted behaviour is only visual, and it occurs only rarely, it might be acceptable. However, if behaviour impacts play negatively, it must be prevented.

As with all tweaking, you should not spend too much time fumbling around. To proceed more “scientifically”, ask yourself the following questions:

• Is there a way to detect (measure objectively) the unwanted behaviour?
• Instead of tweaking your CA, can you detect-and-recover system be used?
• Is the behaviour continuous? In other words, is there a degree of badness, or is it either good or bad?
• If it is continuous, is it feasible to kick in a preventative system? (When the badness goes above a threshold, updates are processed differently).
• Is it feasible to find a set of parameters through an optimisation algorithm (for instance, genetic programming)? It might even be possible to search exhaustively for a solution.
• Is my model too complex? Try to eliminate states and simplify transition rules. You might even consider breaking your simulation into two or more independent CA systems.
• If you find yourself struggling to tweak the parameters for too long, you should perhaps look for a better model that is less sensitive to the exact parameters.

Links

Online Demo

http://blog.soulwire.co.uk/laboratory/flash/2d-cellular-automata Browswer-based demo with ActionScript source code.

General

http://cell-auto.com/ A nice breakdown of various topics related to Cellular Automata. The article on optimisation is especially interesting.

http://www.collidoscope.com/modernca/ Contains many non-traditional variations of cellular automata.

http://www.fourmilab.ch/cellab/ Application for exploring cellular automata.

Applications and Implementation

http://realtimecollisiondetection.net/blog/?p=57 Path finding using cellular automata.

http://www.ibm.com/developerworks/java/library/j-camusic/ Cellular Automata and music (interesting way of creating procedural music).

http://blog.sigfpe.com/2006/12/evaluating-cellular-automata-is.html About using lazy, infinite containers for cellular automata.

http://madeira.cc.hokudai.ac.jp/RD/takai/automa.html Graphics applications for cellular automata – lots of videos!

Game Design

http://www.gamestudies.org/0102/pearce/ An article about game design.

http://www.cp.eng.chula.ac.th/~vishnu/gameResearch/Sweetser_Thesis.pdf A thesis that looks at game design from a emergence perspective.

http://fora.tv/2006/06/26/Will_Wright_and_Brian_Eno#chapter_01 Will Wright and Brian Eno on generative systems.

Downloads

Game Maker Demos

• Requires Game Maker 7.
• Contains 9 demos.

ca_demos.zip (135 KB)

Related posts:

1. Random Steering – 7 Components for a Toolkit
2. A Simple Trick for Moving Objects in a Physics Simulation
3. Estimating a Continuous Distribution from a Sample Set

Last year I wrote a tutorial explaining how to use XSI Mod Tool as a level editor, specifically for XNA. Below is the same tutorial, updated for XNA 3.0. There are only a few minor changes:

• You need not copy compiled assets from the batch file as before.
• A section is included that describes how to use the content pipeline classes for easy reading of XML files (useful for level files, etc.).

I also corrected quite a few typos.

Link

The article Integrating Your XNA Engine With XSI ModTool is very relevant to this tutorial. Among other things, it explains how to make XNA shaders work in XSI.

Download

PDF Tutorial

How to Turn XSI Mod Tool into a Level Editor for your XNA Games v3.pdf (463 KB)

Example

(XSI Plug-in and C# code. Includes the tutorial PDF)

XSIModToolLevelEditor3.zip (5.2 MB).

Related posts:

1. How to Turn XSI Mod Tool into a Level Editor for your XNA Games: Example Updated for XNA 3.0.
2. Google App Engine for Games
3. Guerrilla Tool Development

For many applications, detailed statistical models are overkill. Instead, we can get away with a rough description of the distribution – not in mathematical formula form, but just as a graph with a few sample points.

For example, when trying to model the traffic around a school, you might know that the graph looks something like this:

The input is the number of minutes before the first bell rings, and the output the number of children dropped off at that time. You know that most kids are brought before the bell rings, and that the closer to the bell, the more kids are being brought every minute. Only a few kids are late.

This tutorial describes how to generate random numbers that can generate a distribution described by an arbitrary (piece-wise linear) curve, as the one above.

Implementation

1. Calculate an accumulative probability

The first step is to run an accumulative sum of our original sample points. The following code snippet shows how it can be done in C++; the array “samples” contains the original samples.

for(int i = 1; i < n; i++)
{
    samples[i] += samples[i - 1];
}

  The snippet above does the calculations in-place, but it is not necessary, as is shown in the snippet below:

newSamples[0] = samples[0];

for(int i = 1; i < n; i++)
{
    newSamples[i] = samples[i] + newSamples[i - 1];
}

  The graph above shows the accumulative probability density.

2. Calculate inverse sample points

This step is easy – we simply swap the input and output samples. It is not necessary to do this explicitly, just swap them in the argument list of the function call in the next step.

3. Interpolate between samples

I use a special data structure that makes it very easy to compute the interpolations from discreet sample points: the response curve. There are two varieties. The ordinary response curve, with uniformly spaced samples, and the xy-response curve, where samples can be arbitrarily spaced. Ordinary Response Curve. XY-Response Curve. The implementation is very simple, so I won’t describe it here. A C++ implementation is available in the Special Numbers Library; a Python implementation is available with the example below. Here I simply explain the usage as it relates to this tutorial. We proceed as follows:

• First, we construct an xy-response curve from our samples of the inverse accumulative probability.
• Then we sample this curve at regular intervals, and load it into a ordinary response curve (we do this, simply because the ordinary response curve is much faster than the xy-version).

//...

int oldSampleCount = 7;

int newSampleCount = 20;

float inputMin = 0.0f;
float inputMax = 3.2f;

//note: input and output is swopped arround, because we want the inverse!

XYResponseCurve<float, oldSampleCount> xyCurve(outputSamples, inputSamples);

for(int i = 0; i < newSampleCount; i++)

{

    input = ((float) i / (newSampleCount - 1)) * (inputMax - inputMin) + inputMin;

    uniformOutput[i] = xyCurve(input);

}

ResponseCurve<float, newSampleCount> curve(inputMin, inputMax, uniformOutput);

4. Map uniform random numbers to the input range of the IAPDF, and calculate the output

Now that we have the response curve, we can map uniform random numbers to the appropriate input range. For the example above, we need to map to the range [0, 3.2]. The snippet below shows how to generate random numbers with the distribution shown above:

//...create the curve c

float input = random(); //Uniformly distributed function between 0 and 1.

float scaledInput = r * (inputMax - inputMin) + inputMin;

float output = curve(scaledInput);

  The graph below shows how 10 000 random numbers are distributed. It follows the original graph closely; the discrepancy at -10 is caused by the way samples are counted (all samples between -10 (inclusive) and 0 (exclusive) are counted and plotted at -10.  

Tips and Pitfalls to Avoid

1. Generate sequences for all intermediary steps.
2. Use Excel, Calc, or some other spread sheet program to debug these sequences visually when things go wrong.
3. It is very easy to get confused with input, and output, especially after the swap. Watch out for this!
4. It is easy to get confused with the number of samples for the various sequences.
5. If you implement your own Response Curve data structure, unit tests will save you huge amounts of time.
6. Always make sure that you sample enough points, especially if your original distribution graph has rapid changes in it.
7. Always confirm that your random output follows the distribution you wanted.
8. It might be faster to use this method even when mathematical formulas are available.

Downloads

Example C++ Source Code

http://code-spot.co.za/downloads/cpp_examples/arbitrary_distribution.cpp Requires Special Numbers Library

Example Python Source Code

http://code-spot.co.za/downloads/python_examples/random_distributions.py

Related posts:

1. Generating Random Points from Arbitrary Distributions for 2D and Up
2. Generating Random Integers With Arbitrary Probabilities
3. Random Steering – 7 Components for a Toolkit

• Python runtime (http://www.python.org/download/).
• A Google App Engine account (http://appengine.google.com/).
• The Google App Engine SDK (http://code.google.com/appengine/downloads.html).
• The files for this tutorial (http://code-spot.googlecode.com/files/code-spot_0.1.zip).
• Your favourite web browser and general purpose code editor.

Below are some links to related web sites.

Links

Reference

http://appengine-cookbook.appspot.com/ Google App Engine Cookbook.

http://www.poromenos.org/tutorials/python Learn Python in 10 minutes.

http://www.djangoproject.com/documentation/templates/ Django template reference.

http://www.djangobook.com/en/1.0/chapter04/ Django templates from the Django book.

http://code.google.com/appengine/articles/rpc.html Using AJAX with Google App Engine.

http://aralbalkan.com/1318 Using Flash with Google App Engine.

Games on Google App Engine

http://www.ajaxbattle.net/ AjaxBattle is a multi-player real-time strategy game for 2 to 4 players. It is a web-based version of the old X-Windows game xbattle. Works best with Firefox 2.

http://uboggle.appspot.com/ uBoggle is a app to play Boggle – the popular word making game.

http://mnk.appspot.com/ This game lets users set up most types of x-in-a-row type games – can be played against another online player, or the computer.

http://gaesudoku.appspot.com/ Sudoku.

http://games.wtanaka.com/ Card and board games.

http://www.guessasketch.com/ A multiplayer game where players guess what other players draw.

http://achi.appspot.com/ A tic-tac-toe-like board game.

http://photomunchers.appspot.com/ and http://jamendogame.appspot.com/. These two games are really ways to get users to tag images and music, respectively. They illustrate how to tap into other networks.

Related posts:

1. How to Turn XSI Mod Tool into a Level Editor for your XNA Games: Updated for XNA 3.0.
2. Cellular Automata for Simulation in Games
3. Quadtrees

General

Use postfixes to prevent name clashes

It is possible (but incorrect) to give, for example, an object and sprite the same name. To prevent this from happening accidentally, use postfixes to distinguish different types of game-elements (sprites, sounds, objects, rooms, and so on). Use postfixes instead of prefixes – this makes it easier to find specific objects in the object tree, and to read your code.

[EDIT]

The following Game Maker file illustrates how name clashes can lead to bugs:

test_name_clash.gmk (19.2KB)

Use folders to structure resources

Generally, the object folder structure should follow the inheritance hierarchy, with further separation as necessary. Other resource folders should reflect the groups of the objects to which they belong, or be subdivided by their function.

Use a Development Standards Document

Compile and work from a development standards document to promote consistency across projects and between developers. At the very least, this document should define:

• your naming convention;
• your folder structure;
• your layering convention (see below);
• positions of controllers; and
• icon code (if you use one – see below).

Resources

Do not make the embedded drawing tool part of the pipeline

…if your main drawing tool is an external tool. This prevents every change from becoming an error-prone two-step process.

Use dynamic content

Loading images from files, creating sprites and objects dynamically, and constructing levels programmatically makes it a lot easier to change assets, since the Game Maker file doesn’t need to change whenever the asset changes.

Design art around a grid

Use a grid, and design objects for that grid to simplify world construction and object placement.

GUI

Draw your GUI in a separate room

…and overlay this room over every level.

Use a widget-and-style approach for complicated GUIs

For complicated GUIs, it is worth defining generic widgets (such as buttons or menus) that can be configured using parameters. Keep style elements (colour, font, borders) in separate objects. This makes it easier to change styles in one place, instead of in every widget. It also makes it easier to make the GUI skinnable.

Do as much of the GUI layout programmatically as is possible

Laying out a complicated GUI can be extremely labour intensive, so that changes to the GUI layout can become unmanageable. Using layout functions can greatly improve the ease with which the GUI can be extended or otherwise modified.

Extensions

Make extensive use of Game Maker extensions

Using extensions provide several advantages:

• extensions make it possible to reuse functions more effectively;
• extensions make it easier for more people to work together (people making extensions needn’t modify the main Game Maker file); and
• extensions can be put under separate version control, and so reduce the impact of a roll-back.

Every script function you define should be considered for implementation as a DnD action in an extension. It is useful to have a general usage extension, that you can use across projects, as well as a project-specific extension. Remember to check whether somebody has not already implemented a particular function in an extension!

Document extension actions properly

Don’t leave any of the fields (such as Description, List Text, and Hint Text) in the extension maker empty, and take care to make the List Text and Hint Text display as much field information as possible (by using the @keywords – see the Extension Maker Help Documentation).

Spend time making (or searching for) proper icons for your extension actions

Forgetting what an action does kills maintainability; a descriptive icon will significantly reduce this.

Always define a script function for every DnD action

This way it is easier to unit-test features (see below), and possible to use the functionality from a script.

Actions and Scripts

Learn how to use the merge facility effectively to facilitate for teamwork

This is not Game Maker’s safest feature: there are many pitfalls, for example:

• be careful for name clashes; and
• all elements get new IDs (so be careful if you use hard=coded IDs).

However, with careful planning, this feature can help several developers can work together on a project. for example, every developer works on his / her own file, which is merged into a final game.

Avoid complicated branch logic in the GUI action editor

Complicated branch statements are hard to read in the DnD interface, and editing these constructs is error-prone. Put complicated branch logic in a script.

Avoid assigning too many variables in the GUI action editor

Rather use a script. This way, tweaking the values of several variables becomes a lot easier.

Never use the Execute Code action

Always use the Run Script action, and put the code in a separate script. This way the code is easier to find, reuse, and use in an extension.

Always use blocks to demarcate if-then-else constructs in the DnD interface

…even when a statement block is only one line. This makes it much more convenient to add stements to the block in future, and reduces the risk of introducing a bug because of incorrect blocks.

Make use of named global constants

Using named constants makes code easier to read, less error-prone, and easier to change the value. If you define them in the Global Game Settings dialog, they appear highlighted in your code.

Use expressions as constant values

You can use expressions when you define constants (in the Game Settings dialog). For instance, to define a colour, you can set the value as make_color(255, 128, 64).

Initialise the variables of an object in one place

…either in a single file, or as DnD actions in the Create event of that object. It might be better to do it in a file if you need to document usage.

Declare global variables in one file only

This serve as an easy way to see all the global variables, and is a good place to document their usage.

Always assign function arguments to named variables

Like this:

   1: {

   2:     var width, height;

   3:     width = argument0;

   4:     height = argument1;

   5:

   6:     return width * height;

   7: }

Notice that you can immediately deduce what the arguments should be, and that this function returns an area.

Use the with-statement as a for-each loop

The with-statement can be used as a for-each loop over the instances of an object (where the variable self serves as the loop counter).

The following two examples illustrate this:

   1: //kills all monsters

   2: with(monster)

   3: {

   4:     instance_destroy();

   5: }

   1: //Resets all object positions to 0 0

   2: with(all)

   3: {

   4:     x = 0;

   5:     y = 0;

   6: }

Objects

Use sprite frames to represent object states,

…rather than using separate sprites or objects per state.

Use inheritance

… and refactor aggressively throughout development. See The Power of Inheritance.

Use tiles instead of sprites for the visual appearance of objects

See for example http://www.youtube.com/watch?v=zM3iQbzM4C4.

Use unique sprites even for invisible and control objects

This makes it easier to place these objects in rooms, and identify them properly. If you cannot spend time making proper visually descriptive icons, use a coding system (based on colour, shape and text).

Avoid creation order dependence…

Creation order dependence is hard to remember, and thus something that is easy to break. Creation order dependence can often be eliminated by removing initialisation code, and making sure that the update function will do the work the first time it is called.

…Or use spawning to control order dependence

When creation order dependence is unavoidable, let objects spawn those objects which should be created after them. This way you control the creation order, but also make it explicit (and thus, not necessary to remember).

Use parameterised objects instead of separate objects where possible…

For example, when the only difference between two objects is their sprites, you could do better to make them the same object, and control the sprite that is used by a parameter. The fewer objects you have, the easier it is to restructure or otherwise change the objects.

…But use separate objects to avoid type checking

If you have to do type checking to distinguish between objects to differentiate behaviour, you should use separate objects. This is especially true when you have to perform group operations.

Use Game Maker objects to mimic record data types

For example, you might create an Abilities object that contains variables for strength and magic power. Every character in the game can make its own Abilities object, which allows you to write code like this:

   1: if (monster.abilities.strength > human.abilities.strength)

   2: {

   3:     kill_human(human);

   4: }

Of course, nowhere in the game the user will actually see an Abilities object, nor will it ever be put in a room (as would monsters, for example).

Use object composition

Object composition is a great way to reuse objects, and is a great substitute for multiple inheritance (which is not available in Game Maker). For example, a monster can be composed of its static attributes (strength, aggressiveness), its current state (health, anger) and its items (weapons and potions).

Avoid Using the Test Chance action

Because it is only possible to express a limited set of probabilities with a single Test Chance action, it is not easy to tweak it. Rather use a Test Expression action, with the expression something like:

random(100) <= 60

Idioms

Choose a layering convention

Choose it up-front, document it, and keep to it! For example, choose the layers for the background, static elements, dynamic elements, main character, and effects (such as particles and glows).

Use the controller idiom

Controllers are used to group together logic that does not belong to an in-game-object (such as the player, a monster, or item). Typically, controllers are used to maintain room state and global game state, and to draw interface elements.

Choose a standard position on screen for controllers

This makes them easy to find, and avoids duplication.

Standardise idioms that are relevant to your game

For example, I always use the init-update-draw idiom for each interesting object (i.e. init_monster, update_monster, draw_monster). Recognise the patterns in your own games, document them, and follow these as standards.

Avoid putting code in the room initialisation section

It is usually better to put initialisation code in a controller, and it is necessary if you want the code to execute before other objects are created. It is easy to forget about code in the room initialisation section, which can result in code duplication and hard-to-track bugs.

Time

Use timelines with care

Step numbers can only be hard-coded values. Thus, tweaking a set of timelines can become a maintenance nightmare if their step numbers are somehow related to each other or other game parameters. It might be a better idea to use a script and counter approach, which will allow you to define these relationships (for example, the growling time of a monster is always proportional to it’s size).

Use a script file to document important global issues, such as script entry points, general architecture of the game, and so on.

Design your game around fixed unit time intervals

For example, everything in the game happens in intervals of multiples of ten steps. Using this scheme makes constructing time-dependent events easier for the same reason that using a grid make level-construction easier.

Never use frame rate to change simulation speed

This can lead to ugly glitches in animation. Simulation speed is better controlled with a time scale variable independent of the frame rate.

Always animate sprites at the same frame rate

It is just easier not to have to think about this for every object. And if you need to tweak the frame rates of sprites individually, you can easily find yourself in a maintenance nightmare.

Do not use any text in your graphics

When text is part of a graphic, it is very hard to change it. Also, it means that you need a new graphic for each piece of text, which puts a bottleneck on the amount you can produce. Drawing text with Game Maker is a pain, but it will save a lot of work in the long run, and will make it easier to make the game data-driven and scale gracefully.

Events

Choose a fixed place to call the parent event

You should always call the parent event either first, or last, and do it consistently throughout (the game, or even better – all your games). This reduces the chance of having hard-to-debug logic as a result of calling the parent event, especially along complicated hierarchies.

Never rely on the order of events for something to work

Even though the order is well defined, and can be expected to stay the same for future releases of Game Maker, it is hard to remember this order, and hard to remember that the order for a piece of logic is important. Therefor mistakes are easily made. This is especially true in teamwork, where  reliance on event order might not be communicated properly.

Avoid using custom events when possible…

Custom events are not named (you have to use indices to refer to them), and hence it is hard to remember which event does what.

…But use custom events instead of global states

Custom events can be used to broadcast changes in game conditions. This not only eliminates global variables, but also the if-statements that are necessary to check the global state at every step.

Testing and Debugging

Use asserts

Define an assert function, and use it to confirm game conditions. Using asserts make it much easier to make changes safely.

Always make sure that the number of enums is correct

When you iterate over enums, you usually have a constant that holds the number of enums in a set. It is important to assert that this number is correct (it is easy to forget to update this number when inserting or deleting enums). Do it in a place where it is easy to see how many there should be. Below is an example.

   1: globalvar activity_dict;

   2:

   3: activity_dict[0] = bed;

   4: activity_dict[1] = tv;

   5: activity_dict[2] = stove;

   6: activity_dict[3] = washingmachine;

   7: activity_dict[4] = radio;

   8: activity_dict[5] = noone;

   9: activity_dict[6] = computer;

  10: 

  11: assert(ACTIVITY_COUNT == 7, "The wrong number of activities have been initilised in the script 'init'");

Always check that functions do not return noone

When searching for the nearest instance of an object, for example, make sure that the function does not return noone (even when you think it is impossible). At the very least, display a helpful message. It is common that your assumption of impossibility might be violated by future changes.

Use unit tests

Write unit tests for all your script functions. You can also set up unit tests in special test rooms, that will test object behaviour in specific preset conditions. When you are protected with unit tests, you make make changes boldly. Log success and failures to a file.

Make unit tests runnable with command line parameter

This way, unit testing can be automated. Use the log file to detect failures. You can use the parameter_count and parameter_string functions to get the command line parameters fed to the game.

Use a Game Maker file to put in unit tests for the general purpose library

This special file will not only help detect bugs in your extensions, but also serve as a usage document.

Implement a debug menu screen

Implement a screen with special debug functionality, and only display this when the game is run under a special mode. The debug screen should have ways to:

• run any level;
• run scratchpad levels (see below); and
• run unit tests.

Implement a test room for a scratch pad

It is often necessary to test a new feature in a room that is already part of the game. Resist the temptation to test new features in existing rooms. It is easy to mess up a room when implementing a new, untested feature, and it can be hard to remove this feature if it doesn’t work out. Rather, have a minimal room ready that is fully integrated into the game, and use it as a scratchpad to test new features.

Implement debug drawing mode

In many cases your game can be too complex to rely on logging or inspecting variables one by one to perform adequate debugging. Drawing extra information with instances, and using colours and other graphical elements effectively, can make it a lot easier to spot errors. This technique is especially useful for AI and simulation.

Implement cheats

It is not only a useful debugging aid, but also guides the design (the technical design – not the game design!) to be more modular and flexible. A cheat (for example, a God-mode) that is hard to implement, can indicate a problem in the design (for instance, deductions to health is made in several objects, instead of in a single one or just a few).

Use a global game options to switch on and off some features

For example, this can be used to display debug info, or skip some eye-candy to increase load speeds. Do not chaotically comment in and out pieces of code or action logic to achieve this effect: it is error prone in larger projects and in teams.

Related posts:

1. Cellular Automata for Simulation in Games
2. Random Steering – 7 Components for a Toolkit
3. Google App Engine for Games