(Original Image by Valerie Everett)

It is sometime necessary to move an object in a physics simulation to a specific point. On the one hand, it can be difficult to analyse the exact force you have to apply; on the other hand it might not look so good if you animate the object’s position directly.

A compromise that works well in many situations is to use a spring-damper system to move the object.

The trick is simple: we apply two forces—the one is proportional to the displacement; the other is proportional to the velocity. Tweaked correctly, they combine to give realistic movement to the desired point.

The spring force is proportional to the difference between the current position and the position where we want the object:

Here, is a positive value called the spring constant.

As you can see, the force gets smaller as our object approaches the desired position, and becomes zero when it reaches that position. Unfortunately, in the absence of friction or drag, the velocity is not zero at this point, so the object overshoots the desired position, and moves past it. The force becomes bigger, but in the opposite direction. The object keeps on moving, slowing down, and finally starts moving in the opposite direction towards the desired position. This goes on indefinitely.

When there is friction or drag, we might be lucky enough for the system to slow down the object sufficiently so that its velocity becomes zero when the object reaches the desired position. This can be tricky to accomplish, though, and might impact the simulation environment in undesirable ways.

It is better to add a counteracting force explicitly. We add a damper force that is propostional to the velocity of the object, again opposite in direction. Here is the viscous damping coefficient, also a positive number.

We then apply the sum of the forces to our object:

The trick is to choose to get the behaviour we want.

Fortunately, this is easy. The following table summarizes how the damping constant affects behaviour. Here, is the mass of the object.

	The object oscillates.
	The object oscillates, but the oscillations die down.
	The object moves to the desired position without oscillating in minimum time.
	The object moves to the desired position without oscillating, and takes longer as c increases.

If you want to see an explanation of how this works, see the Wikipedia article on damping.

By choosing , we are left with only one parameter to tweak (the spring constant), with which we can adjust the time it will take for the object to reach the desired spot.

A simple implementation of this idea is given by the following function. The function should be called for every simulation frame, until we are satisfied that the object reached its spot:

private void MoveTo(Rigidbody rigidbody, Vector3 newPosition,
      float springConstant)
{
  Vector3 desiredDisplacement = rigidbody.position - newPosition;
  Vector3 springForce = -springConstant * desiredDisplacement;

  float viscousDampingCoefficient
    = 2 * sqrt(rigidbody.mass * springConstant);

  Vector3 dampingForce
    = -viscousDampingCoefficient * rigidbody.velocity;

  Vector3 totalForce = springForce + dampingForce;
  rigidbody.AddForce(totalForce);
}

This will work even when there is drag or friction, except that the object will move slower (in this case we can decrease the artificial damping, although it is a bit risky). When there is an external force applied to the object, the object will come to rest at some point away from the desired position. By increasing the spring constant, the distance between this point and the desired point can be made smaller. Thus, we can also use this scheme to maintain objects at a certain height, for example, which can give a rather realistic simulation of a hovercraft or even a drifting object.

Update: This is an example of a PID controller (proportional–integral–derivative controller), for which there is a C++ implementation in my Special Numbers Library.

Related posts:

1. Cellular Automata for Simulation in Games
2. Random Steering – 7 Components for a Toolkit
3. Force Field Editor v1.0

A cellular automata system is one of the best demonstrations of emergence. If you do not know what cellular automata (CA) is, then you should go download Conway’s Game of Life immediately:

Conway’s Game of Life

Essentially, CA is a collection of state machines, updated in discrete time intervals. The next state of one of these depends on the current state as well as the states of neighbours. Usually, the state machines correspond to cells in a grid, and the neighbours of a cell are the cells connected to that cell. For a more detailed explanation, see the Wikipedia article.

Even simple update rules can lead to interesting behaviour: patterns that cannot be predicted from the rules except by running them. With suitable rules, CA can simulate many systems:

• Natural phenomena: weather, fire, plant growth, migration patterns, spread of disease.
• Socio-economic phenomena: urbanisation, segregation, construction and property development, traffic, spread of news.

In games, these can be used as game elements (such as fire and disease), or they can also add some decorative life to a game (such as city development in a simulation game).

The great thing about CA systems is that they are not hard to implement, and they have intrinsic fun value (I mean here, for the programmer!).

Before we get into the details, let me add a small disclaimer here:

• This tutorial is not comprehensive. I only touch on a few ideas in a wide field. See some of the resources at the end for more information.
• This tutorial is not representative. I merely chose topics I found interesting or thought would be useful for game developers.
• I am not suggesting that using cellular automata is the best way to simulate any of the examples given here. For any problem, there might be solutions that are faster, easier to implement, or yield better results.

Implementation

To implement a CA system, you need to do the following:

• Use a suitable system to store states, for example, a 2D array.
• Decide on a suitable neighbourhood shape, and work out some preliminary rules. Decide how you will handle edges.
• Implement a suitable way to describe your rules.
• Implement an update function that hooks into the main game loop.
• Implement a visualisation scheme (for example, drawing tiles depending on states in the array).

Some details of these are described below.

Grid

A typical implementation uses two grids for storing states. One is the current world state; the other is the new world state. Once all cells have been updated for the current iteration, pointers to the arrays can be swapped. This way updating the cells one by one will not have an effect on other cells until the next iteration, and you do not have to create a new array in each iteration.

Using grids with square cells are the easiest to implement, but the concept is also applicable to other grids: hexagonal and triangular grids are also common, and even polar grids have been used to simulate spiral star systems.

It is also possible to use irregular systems, although the implementation then becomes a challenge. One example of an irregular system is a presentation based on a Poisson distribution of points. The trick here is to find an efficient way to locate neighbours. Fortunately, this problem has already been solved in creating the points (see the Poisson Sampling tutorial in Dev.Mag and download code from Luma Labs). It is in fact done by using an invisible grid! Of course, visualising a state is also a bit more difficult.

Neighbourhoods

For 2D cellular automata, the most common neighbourhoods are the Von Neumann neighbourhood and the Moore neighbourhood, shown below (the blue cell’s neighbourhood is shown in red). Other neighbourhoods are also possible.

Von Neumann neighbourhood	Moore neighbourhood	Extended Von Neumann neighbourhood

The cell can be included in the neighbourhood or not. It is advisable to keep the neighbourhoods as small as possible: not only does the neighbourhood size impact performance, it also affects the number of rules to be specified and thus the tweaking complexity.

Specifying Update Rules

Specifying the rules concisely is the most challenging part of implementing CA. With very simple models, you might want to use a few if/else statements. This, however, makes it hard to modify (tweak or extend), and is therefore not suitable for more complicated systems.

It is best to implement rules as a table that can be indexed using the neighbourhood states.

Table as an array. In this method, each row represents the combined state of the surrounding cells. The final entry in the row is the new state of the current cell. Each of the remaining columns represents a state. The entries in these columns represent the number of cells surrounding the current cell in that state.

A cell surrounded with three cells in the Dry Bush state and one cell in the Burning Bush state will go into the Burning Bush state.

This table can be quite large, so that you must take care to implement lookups efficiently. Keep the table sorted, and perform binary search to lookup entries. Below are some improvements on this scheme.

Table as a tree. In the table above, it is easy to see that if there are three dry bush nodes, there are two possibilities for the other states (either 1 0, or 0 1). We can use this to implement a tree. Each level in the tree represents a state. A value in a node represents how many cells in the neighbourhood are in that state. A path from root to leaf fully describes the states of the entire neighbourhood. We do not add nodes for the last state, as it is uniquely determined by the other states.

Below is a tree corresponding to the piece of table above.

BB = Burning Bush

DB = Dry Bush

This is a more efficient implementation; unfortunately, it is also more complicated. It is inconvenient to specify the tree in a tree-friendly format (XML or s-expressions, for example). You still need to specify it in table format (space separated text file, for example) and build the tree from that.

Implementation as a hash table. Here you use tuples of state counts as keys. For example, the third entry in the table above can be accessed with the tuple (3, 0, 1). If your implementation language does not support tuples, you might need to implement them yourself. Be sure to make them immutable!

Implementation as an N-dimensional array. This is a very simple method, although it wastes a lot of memory. N is the number of states, and we use the state counts to index into the array. In our example, we will use a 3D array, so that the third element can be accessed using transition_array[3, 0, 1].

The method above can be used even when the implementation language does not provide N dimensional arrays. Simply use a 1D array, and interpret the state counts as the digits of a base-4 number. Thus, the third row will be located at 3*16 + 0*4 + 1*1 = 49.

Note that since the numbers are restricted by the requirement that they add to four, there will be unused array positions in both these last two methods. You can save memory by not using the last state to index (since it is uniquely determined by the other states). Thus, we can use a 2D array, and access the 3,0,1 rule with 3,0 alone: transition_array[3, 0]. Alternatively, if we need to use a 1D array, we can store the rule in the position 3*4 + 0*1 = 12. Although the same scheme is possible for the hash table implementation, I would advise you to use the full indexing scheme instead, since it will not waste more memory if you do, and it is somewhat easier to understand if you use full indices.

Some rules do not need the detailed tables above. For instance, in a system of only two states, where transitions depend on the count of the states of the neighbourhood, we can use a simple array, indexed by one of the states’ counts. Many other schemes are possible: in general, aim for quick lookups and easy specification.

Edges

The edges of your simulation might present some problems: edges often serve as unwanted sources, sinks or stabilisers.

A source is the origin of outward growth (states change progressively to a specific state in an outward direction).

A sink is the exact opposite:

Sources and sinks are emergent properties of the CA system, and are not physical attributes of the cells.

A stabiliser is a section that keeps neighbouring states fixed. For example, in the fire simulation (described below), there is an unnatural wall of fire around the edges at the end of the simulation.

There are several ways you can handle edges:

Ignore them. When the edge anomalies are acceptable, it is best not to complicate the model. This is only possible if the rules are specified in a way that does not require state counts to sum to the size of the neighbourhood. For example, if there is a rule “if num_states_fire > 1, then …”.

Wrap around. This solution is very elegant, easy to implement, and its effects are easy to understand. However, it is usually easy to see that the world has been wrapped in this way, and hence it is not always a visually a satisfactory solution. It is possible, however, to simulate a grid bigger than displayed area (say about double). This will hide the fact that it is wrapped, but still give you the benefits of having a world without any discontinuities.

In the CA system shown left, we want to calculate the next state of the grey cell. Two neighbours fall outside the edges. If we wrap around, we use the four red squares as shown.

Reflection. Use a neighbouring state to perform calculations.

The CA system shown left is the same situation as above. If we use neighbouring states for the cells beyond the edges, we use the state of the grey cell (twice!), in addition to the two red cells shown.

Fix their states. The edges can be given special, constant states that make the CA behave as intended. Finding the right state(s) for the edges might be difficult.

An implementation trick that applies to the last three edge-handling techniques is to include a border in your simulation. For the Von Neumann and Moore neighbourhoods, this border is only one cell thick.

This border is used for updates like all other cells, but is itself updated differently. The update rules for wrapping and reflection will simply copy states from the right places; the border for fixed states will never be updated.

In the figure, hatched cells are updated according to the special rules of the edge-handling scheme – in this case, wrapping is used, so they are simply copied from the appropriate cells. All other cells are updated normally. Note that here we do not need special logic for updating the grey cell.

A Few Examples

(Also see the downloads at the bottom of the post).

Heat

Here, continuous states (floats between 0 and 1) are used. Every cell is updated to a weighted average of its Moore neighbourhood cells: corners have a smaller weight than other cells. For this simulation, heat sinks and sources have been added: they keep the cell’s temperature at 0 and 1 respectively.

City Roads

This simulation use separate neighbourhoods for creating and destroying a road. The creation neighbourhood is an extended Von Neumann neighbourhood; the destruction neighbourhood is a Moor neighbourhood. Creation and destruction is according to probabilities based on the count of neighbouring roads.

Influence

This simulation models human emotion as influenced by others. A character has an internal state, and an externally visible state. Characters can only observer the external state of their neighbours. The internal state is then a modified: a character surrounded by angry people will itself become angry, and so on. The external state is probabilistically determined by the internal state.

Disease Spread

For our game Epidemic, Chris and I used a simple diffusion model. We used a floating-point number for the state, and represented the number of infected people in a cell. We used different colours for different ratios to visualise infection. At each time step, a cell would become infected by an amount proportional to that of the neighbourhood, with some random scaling.

Visualising States

Versions of tiles

A very effective way of making CA simulations more impressive is to have more than one tile for each state, and display them randomly. This is especially helpful for hiding regularity in off-line simulations. Tile sets with different versions for corners, edges, and central regions can drastically improve the look of the game, although it complicates choosing the correct tile version. In addition, it means you need tiles for various combinations of states. Assuming we can rotate and reflect tiles, in the fire example with three states, we need the following tiles:

3 centre tiles
3 straight-edge tiles
3 diagonal line tiles
3 cross tiles
6 corner tiles
3 half-straight tiles

This is 21 tiles for only three states. It is advisable to automate tile creation.

Tile

States can be animated. Instead of displaying a single tile, display an animation sequence. You will need to think about how to start and stop these animations so that they are seamless. For instance, you can make all animation sequences the same length, and only update states between animation sequences.

You can also increase the seamlessness of the simulation by animating transitions between states. This can be as easy as blending between two tiles over a period of time (assuming that updates are made far enough apart).

No Tiles

Instead of using tiles to represent a state of a cell, you can put objects in a cell. For instance, in a plant growth simulation, you can put a number of trees in the cell depending on the state. The advantages of this are:

• You do not have to create dozens of tiles for variety.
• The player can interact with the objects. For instance, the plant placement of trees can hinder a player’s movement.

This shows a simple forest simulation. Trees grow and die depending on the surrounding trees. This system uses a grid, but not tiles: trees can grow in random positions in tiles. For this simulation, three random coordinates have been chosen for each cell at the beginning of the simulation. Depending on the states, one, two, or three trees will be drawn at these points. Choosing a fixed set of points makes it easier to draw trees at the same coordinates every tick. A somewhat more complicated simulation can choose new points whenever a new tree comes into being.

CA Enhancements

The images below show three frames from a simple fire simulation.

The green is grass (Dry Bush), the orange is fire (Burning Bush) and the brown is burnt grass (Charcoal).

As you can see, the simulation captures some features of a fire:

• fire grows outward,
• grass only burns once,
• grass needs fire to catch fire (there is no spontaneous combustion).

Even so, the symmetry and regularity makes this fire very unconvincing. Below we look at some ideas to enhance a simple CA simulation.

Probabilistic State Transitions

Instead changing to some specific state based on the surrounding states, you can change to any state with some probability. These probabilities depend on the surrounding states, and can possibly be zero.

Adding probability to the mix can have several advantages:

It introduces variability in patterns. This means a specific (perhaps the start-up pattern) will not always produce the same results.

It can enhance the organic look of the system. Cross and squares patterns are common in deterministic CA systems. For many purposes, this is unsatisfactory (have you ever seen a fire spreading as a growing square?) Adding probabilistic state changes will make the patterns more organic and varied.

It can make discreet behaviour more continuous. For example, two common “problems” with CA systems are exponential growth (where all cells go to a certain state outwards very quickly), or sudden death (where all cells go to a certain state very quickly inwards). The problem is often that a single change in the update rules results in a exponential growth system to become a sudden death system, with seemingly no way to get a “sweet spot”. If states can only change with a certain probability, tweaking the probabilities allows you to get intermediate patterns (stable growth, equilibrium, or stable death).

It allows you to control the speed of the simulation. It is especially helpful to slow down a simulation without making it more granular (in the time dimension). It is useful to multiply all your probabilities with a global parameter that you can tweak to control the speed.

Note that in the example above, certain cells did not burn at all – per chance, they did not caught fire when their neighbours were burning. Timers (see below) can reduce or eliminate this effect.

Adding randomness can increase the emergence, and hence also unwanted emergence, so care must be taken that the simulation is always valid.

Tile Interpolation

Normally, you want your simulation to be as simple as possible. However, when such simple simulations are displayed (visualised), the results can be unsatisfactory. One way to get better visual quality is to use tile (cell) interpolation to make the simulation seem more complicated than it really is. When combined with a bit of randomisation, the effects can be very convincing.

The basic idea is to simulate only every nth cell with the CA approach. Other cells are interpolated. The number of interpolated cells can be adjusted, depending on the detail level required (balanced with the combinatorial number of tiles required!)

In the fire simulation, using only three states makes the simulation seem very primitive. Suppose that we simulate only every second row, and in these, only every second column, and fill the remaining cells depending on their neighbours. We have to create a function that will return a tile for each combination of surrounding tiles:

get_center_tile(north, east, south, west)

Interpolation rules are similar to update rules of the simulation, and hence can easily be as complicated. If this is the case, you would do better to implement more states and not do any interpolation. Interpolation should only be used if it significantly reduces the implementation complexity.

Timers

Some interesting effects can be obtained if each cell has a timer, that is, each cell only updates once the timer expires. Some rule determines when the timer is reset. For instance, the timer can be set:

• randomly,
• as a function of the state,
• as a function of the surrounding cells,
• as a function of another property of the tile (*1),
• as a function of some external process (*2).

(*1) For instance, if each cell has an associated “burning capacity” parameter, (simulating material types), a fire state timer can be set in proportion to this parameter.

(*2) For example, we can set timers according to a wind factor.

Timers can have the following effects:

• add some randomness to a simulation,
• slow the simulation down,
• smear the simulation (for example, in the fire simulation, the burning border is much thicker, and no cells are skipped).

Although you can often implement your grid cells as classes (or structs, or whatever aggregating construct your language provides) with states and timers embedded, it is simpler (on more efficient) to have separate integer grids – one for states and one for timers.

Note that if timers do not have the same expiration time, you the CA system is asynchronous and the behaviour can differ dramatically from a synchronous CA system. Thus, you should include timers early in the development process, so that their effect does not force you to re-tweak all the parameters.

If your world is big, you cannot run the CA for the whole world at every time step – it will kill performance. Below are some ideas for improving performance.

Instead of simulating the entire world, you can only simulate the part that is relevant to the game, usually the visible region of the game. A simple technique is to use a wrapped grid, similar to the grids used for the visual port in scrollable 2D games. This grid can be slightly larger than the visible grid, so that it allows cells to update naturally for a few iterations before they become visible, and so increase the illusion of a persistent world.

To hide that fact that you are only simulating a very small piece of the world, you must update “new” cells appropriately. There are several options:

Update cells to a natural starting state. In many cases, this solution is adequate, especially if you simulate a slightly larger grid than is visible.

Copy states from the existing simulation. Here you can copy neighbouring cells, or randomly from a nearby region. Copying from nearby cells promotes spatial coherence.

Initialise states statistically. Based on a neighbourhood of cells, each cell is initialised probabilistically. For example, each cell can be initialised randomly to one of three neighbouring cell’s states.

Use hierarchical modelling. Use another CA system that represents the entire world in very low detail, and use this to initialise the cells of the high detail version. Note that both the states and the update rules will differ for the high and low detail simulations. Use probabilities to prevent homogeneous systems. You can even use several levels of detail, and incorporate them as needed. Beware the extra implementation complexity, though!

This technique does not work when the world is should be changed permanently.

Instead of only simulating a part of the world, you can only update a part of the world. This requires enough memory for the whole world, but only processing power for the cells you wish to update. This method has the advantage of real persistence, thus permanent changes can be reliably simulated. Which cells you update will determine how your simulation will play out:

• Update all visible cells (or a slightly larger region). This is the most natural update scheme, and its effects are similar to partial simulation described above.
• Divide the world into discontiguous regions, and update a different region each time step. For example, divide the world into two sets: one contains every other cell, the other the rest, and update these two sets alternately. To save a significant amount of processing, you need many sets. To hide patterns in the updates that might break the illusion of a continuous world, the sets should be irregular. This method has the advantage that real things happen off screen. On the down side, it slows down the simulation significantly.
• Combine the two methods above.

Partial updating, like the inclusion of timers, lead to an asynchronous system, and hence the behaviour might be very different from that of a fully updated system.

In certain cases, simulations can be baked offline and be used in games. This means you perform the simulation before the time, and record each frame. The game then reads these frames, and plays them back.

Various schemes are possible to get more with less. For instance, you can also interpolate the simulation in time. With some thought, you can simulate events in a way that they can be looped, stacked together, and started and stopped unobtrusively. This means you can blend them with the interaction in a way that will not make it obvious that the simulation has been performed off-line.

Unwanted Emergence

Not all emergence is good, and you might find it hard to prevent certain behaviours. If the unwanted behaviour is only visual, and it occurs only rarely, it might be acceptable. However, if behaviour impacts play negatively, it must be prevented.

As with all tweaking, you should not spend too much time fumbling around. To proceed more “scientifically”, ask yourself the following questions:

• Is there a way to detect (measure objectively) the unwanted behaviour?
• Instead of tweaking your CA, can you detect-and-recover system be used?
• Is the behaviour continuous? In other words, is there a degree of badness, or is it either good or bad?
• If it is continuous, is it feasible to kick in a preventative system? (When the badness goes above a threshold, updates are processed differently).
• Is it feasible to find a set of parameters through an optimisation algorithm (for instance, genetic programming)? It might even be possible to search exhaustively for a solution.
• Is my model too complex? Try to eliminate states and simplify transition rules. You might even consider breaking your simulation into two or more independent CA systems.
• If you find yourself struggling to tweak the parameters for too long, you should perhaps look for a better model that is less sensitive to the exact parameters.

Links

Online Demo

http://blog.soulwire.co.uk/laboratory/flash/2d-cellular-automata Browswer-based demo with ActionScript source code.

General

http://cell-auto.com/ A nice breakdown of various topics related to Cellular Automata. The article on optimisation is especially interesting.

http://www.collidoscope.com/modernca/ Contains many non-traditional variations of cellular automata.

http://www.fourmilab.ch/cellab/ Application for exploring cellular automata.

Applications and Implementation

http://realtimecollisiondetection.net/blog/?p=57 Path finding using cellular automata.

http://www.ibm.com/developerworks/java/library/j-camusic/ Cellular Automata and music (interesting way of creating procedural music).

http://blog.sigfpe.com/2006/12/evaluating-cellular-automata-is.html About using lazy, infinite containers for cellular automata.

http://madeira.cc.hokudai.ac.jp/RD/takai/automa.html Graphics applications for cellular automata – lots of videos!

Game Design

http://www.gamestudies.org/0102/pearce/ An article about game design.

http://www.cp.eng.chula.ac.th/~vishnu/gameResearch/Sweetser_Thesis.pdf A thesis that looks at game design from a emergence perspective.

http://fora.tv/2006/06/26/Will_Wright_and_Brian_Eno#chapter_01 Will Wright and Brian Eno on generative systems.

Downloads

Game Maker Demos

• Requires Game Maker 7.
• Contains 9 demos.

ca_demos.zip (135 KB)

Related posts:

1. Random Steering – 7 Components for a Toolkit
2. Estimating a Continuous Distribution from a Sample Set
3. A Simple Trick for Moving Objects in a Physics Simulation

(Photo by  Darren Hester)

Some algorithms take a long time to return their results. Whether it is because the algorithm has to operate on a huge data set, or because it has combinatorial complexity; every time you run it you have to wait minutes or even hours for the thing to finish, making errors very expensive.

This post gives some advice on how to prototype slow algorithms with as little frustration as possible. We assume that this algorithm is being implemented experimentally – that is, you will tweak it and change it often before it is finished (it is not the kind of algorithm you type in straight from a text book). For example, I used the ideas outlined here while playing with the texture generating algorithm of the previous post.

This post is not about optimisation, looking for another algorithm, implementing it in another (faster) language (C++) or writing correctness tests for it. Some of the suggestions go against good programming practices for production code, and you should not use them for that.

The idea behind these suggestions is:

• to minimise error (that would force a rerun after correcting it);
• to minimise duplication of calculation between successive runs; and
• to minimise losing a better procedure or parameter set (that would force several reruns in trying to reconstruct it from memory).

I will assume that your code is divided in two parts:

• a function (and its helper functions) that forms the algorithm; and
• a test harness.

To illustrate the ideas proposed below, we will use the following experimental blur algorithm as an example. I give it here in Python, but the tips are not language specific.

def blur(grid, repeat_count, window_radius):
    window_area = (window_radius * 2 + 1)**2
    tmp_grids = [None] * repeat_count

    for i in range(repeat_count):
        tmp_grid[i] = Grid2D(grid.dims)

        for index in grid.index_iter():
            for window_cell in grid.wrapped_square_iter(index, window_radius):
                sum += window_cell
            tmp_grid[i][index] = sum / window_area

        grid = tmp_grid[i]

        new_grid = Grid2D(grid.dims)

        for index in new_grid.index_iter():
            new_grid = tmp_grid[floor(random() * repeat_count)]

        return new_grid

def demo_blur():
    grid = load_image(‘bar.png’)
    grid = to_grey(grid)
    grid = blur(grid, 20, 5)
    write_image(grid, ‘bar_blur.png’)

1. Implement a quick-run feature

A quick-run is where all loops and data structure sizes are set to the minimum necessary to make all the code run. This is to prevent large amounts of processing time before hitting a run-time error because of a small mistake near the end of the algorithm. This step is especially helpful if you use a dynamic language, where there is less help from the compiler to check validity of code.

In our example, there are three factors that determine the length of the algorithm:

• the image size;
• the repeat count; and
• the window radius.

We can implement a quick-run feature by changing our harness as follows:

quick_run = True

if quick_run:
    test_base_name = '8x8'
    test_image = test_base_name +'.png' #This is an 8 x 8 image.
    repeat_count = 2
    window_radius = 1
else:
    test_base_name = 'bar'
    test_image = test_base_name +'.png'
    repeat_count = 20
    window_radius = 5

def demo_blur():
    grid = load_image(test_image)
    grid = to_grey(grid)
    grid = blur(grid, repeat_count, window_radius)
    write_image(grid, ‘blur.png’)

Every time we make a change, we first run the program with quick_run set to True just to make sure we have not introduced any errors in the code. If you use a language that supports #define macros (C and C++) or some similar functionality, this can be implemented much more seamlessly.

Be careful for degenerate cases that skip loops – make sure all loops run at least once.

2. Consider using disk-caching for intermediate results

If it is faster to load a data structure from disk than to recreate it, you might gain a considerable saving in time by saving it to disk the first time it is created, and loading it thereafter.

In our example, we can store the temporary grids cache the temporary grids on disk by changing the function, allowing us to tweak part of the algorithm following the comment more effectively:

def blur_once(grid, window_radius, step):
    fname = test_image_base + '_blur_once_' + str(window_radius) + '_' + str(step)
    f = file(fname)

    if f.exists():
        return load_image(f)
    else:
        tmp_grid = Grid2D(grid.dims)
    for index in grid.index_iter():
            for window_cell in grid.wrapped_square_iter(index, window_radius):
                sum += window_cell
            tmp_grid = sum / window_area
        save_image(tmp_grid, fname)
    return tmp_grid

def blur(grid, repeat_count, window_radius):
    window_area = (window_radius * 2 + 1)**2
    tmp_grids = [None] * repeat_count

    for i in range(repeat_count):
        tmp_grid[i] = blur_once(grid, window_radius, i)
        grid = tmp_grid[i]

    ### Part that can be tweaked follows ###
    new_grid = Grid2D(grid.dims)

    for index in new_grid.index_iter():
        new_grid = tmp_grid[floor(random() * repeat_count)]

    return new_grid

Guidelines

When caching is used carefully, it can save time. But it can also waste a lot of time.

• Use it if you use one test case often. In our example, we would not gain much if we tested with a different image every time. But if we used one or perhaps a few images, you can reap the benefits.
• When tweaking different parts of the algorithm, caching might become a hindrance. Never cache results that depend on your current tweaking. Not only will the extra disk saves slow things down, but you might be confusing old results from the cache instead of the results you actually need.
• Make sure that the filename is unique for the function and all the input parameters. You might use results from the cache for the incorrect inputs without realizing it, leading to very frustrating debugging. This works hand-in-hand with tip 4.
• Remember to stabilise randomness; that is, seed the randomiser at appropriate spots. (It is, of course, something you should generally do for debugging and unit tests / regression tests).
• For a little extra work, you can avoid bugs due to a non up-to-date cache, at least in some cases. Perform the algorithm on a subset or sample of the data, and compare this with the cache. Signalling inconsistencies can help flush out bugs. You can even modify the caching to update automatically when such an inconsistency appears – but you should still always signal cache misses, so that you can immediately see when your cache is failing, perhaps because of a bug or unstable randomisation.
• Make your cache easy to clear (define a simple function for this).
• Make it easy to disable caching for a run.
• If you use an interpreted language, you can often use a “soft-cache” instead. This just means that you store intermediate results in variables, and only recalculate them when necessary. This can be a powerful method, but in my experience it requires a lot of mental maintenance, and is therefore error-prone. Because we are dealing with slow algorithms, a crash during an interactive session might be disastrous.

3. Use “yield” to return intermediate results for easy inspection…

…if your programming language allows this (C#, Ruby, Python). When prototyping an algorithm, it is often helpful to inspect (or analyse) intermediate results. Doing this can clutter up the code, especially since you want to put it in and take it out as the need arises.

Here is how we can modify our code:

RESULT = 0
BLUR_ONCE = 1

def blur(grid, repeat_count, window_radius):
    window_area = (window_radius * 2 + 1)**2
        tmp_grids = [None] * repeat_count

    for i in range(repeat_count):
        tmp_grid[i] = blur_once(grid, window_radius, i)
        yield tmp_grid[i], BLUR_ONCE, i

        grid = tmp_grid[i]
    new_grid = Grid2D(grid.dims)

    for index in new_grid.index_iter():
        new_grid = tmp_grid[floor(random() * repeat_count)]

    yield new_grid, RESULT, 0

def demo_blur():
    grid = load_image(test_image)
    grid = to_grey(grid)

for result, result_id, step in blur(grid, repeat_count, window_radius)
    if result_id == BLUR_ONCE:
        if step % 2 == 0:
            write_image(res, ‘blur_once_’ + test_base_name + ‘_’ + str(step) + ‘.png’)
        elif RESULT_ID == RESULT:
            write_image(result, ‘blur.png’)

Note that you can change what you want to inspect in one place. This is even more useful in complicated or deeply nested algorithms. Using an extravagant language feature such as yield also makes it easier to port the code to the production version – it is easier to spot and remove.

If your language does not support yielding, you can implement a inspect function and still have s astructure very similar to the above:

RESULT = 0
BLUR_ONCE = 1

def blur(grid, repeat_count, window_radius):
    window_area = (window_radius * 2 + 1)**2
    tmp_grids = [None] * repeat_count

    for i in range(repeat_count):
        tmp_grid[i] = blur_once(grid, window_radius, i)
        inspect(tmp_grid[i], BLUR_ONCE, i)
        grid = tmp_grid[i]

    new_grid = Grid2D(grid.dims)

    for index in new_grid.index_iter():
        new_grid = tmp_grid[floor(random() * repeat_count)]

    inspect(new_grid, RESULT, 0)

    return grid

def inspect(result, result_id, step):
    if result_id == BLUR_ONCE:
        if step % 2 == 0:
            write_image(res, ‘blur_once_’ + test_base_name + ‘_’ + str(step) + ‘.png’)
    elif RESULT_ID == RESULT:
        write_image(result, ‘blur.png’)

def demo_blur():
    grid = load_image(test_image)
    grid = to_grey(grid)

    #The result is inspected in the inspect function; here we can ignore it.
    #We still put it in a variable to remind us that the function returns a result.

    ignore_result = blur(grid, repeat_count, window_radius)

Do not split the function (using polymorphism or overloading to prevent the if-elif logic). The point is that we want a single place where we can choose whether to output the result to disk or not.

4. When tweaking functions, make copies rather than tweaking in-place

Do this even for relatively small changes. Number functions, and associate saved results with the version of that function. One of the most frustrating things is when a change has unwanted effects, and you can’t undo it (and can’t remember how it was when working). Just as frustrating is when you have several results, and do not know which result belongs with which version of the algorithm. Being careful in this regard will prevent long searches for the right settings. It will also make caching (Tip 2) safer.

Under normal coding circumstances, you would refractor these versions to save code and promote maintainability. For prototyping code, I would not suggest this – just copy and paste the function, change the version number, and make the necessary changes in place. Refactoring introduces more room for error – enough to invalidate this tip. Of course, in situations where you want to check out different combinations of algorithms, you might need to refactor after all. As always, do not refactor without the protection of unit tests.

The above applies especially when changing library code – rather make a copy of the library code outside the library (perhaps only a single function or class), make the change there, and use the copy. This allows you to be quick and dirty; you can always refactor the change back in if the change is also useful in other application.

And if you think this approach will lead to a refactoring nightmare later on, you are right. But later on is the right time to deal with it – now you are designing an algorithm, and you need all your brain power. The point of this is to not mess up any existing (tested and working) code.

5. Implement an early-warning system

Even when you do not always know exactly what the state of a data structure should be, you often know what it should not be (for instance, empty). One mistake I make often is to process the wrong variable, for instance:

grid = do_something(grid)
new_grid = so_something_else(new_grid) #Should really work on grid

To help eliminate this problem, I implement a few very quick tests to flag suspicious conditions. What tests you implement depends on your application; here are a few generic ones:

• Test for emptiness
• Test for uniformity (for example, to detect all-black images)
• Test for no-change after processing
• Test for excessive deviations after processing

Guidelines

• Use these tests in assert statements if your language supports them.
• When working with large data structures (such as images), it is important not to test the entire structure, but rather just a sample. Your tests should not take any time to run at all! For images, even a sample of 10 pixels can be enough in many situations.
• If you take a random sample, it is a good idea to use a separate randomiser for these tests. This way seeding the randomiser elsewhere (to stabilise for caching, for example) won’t affect the test samples negatively – it is better to test with different sample sets each time.
• Do not strive for mathematical correctness and completeness (unless this is easy to achieve) – rather use simple rules that will flag extreme deviations. For example, a blur should not significantly change the average brightness of a sample on an image. Although bounds for his can be calculated exactly, broader bounds (that are easy to estimate) will already flag many mistakes in the code.
• Avoid complicated test code; in particular, avoid complicated sampling. Again, we are not striving for the mathematical best sampling method – that is for the production version.
• Test your tests aggressively! There is nothing like a faulty test to waste time.
• Do not halt the algorithm on a failed test where only a sample is used – output the suspect data structure to disk and inspect it first to make sure it is not a false alarm. If it is a false alarm, the algorithm is not stopped needlessly, and you do not have to put in (unnecessary) code to check what is going on. Remember to increase the sample size of the sample algorithm before the next run.
• It is good to start with very crude tests, and refine them as you get closer to the final version of the prototype. Obviously, tests for the production implementation should be as exact as possible.

Other methods

What protection devices do you use for prototyping slow algorithms? Let me know in the comments!

Related posts:

1. A simple texture algorithm – faster code and more results
2. Estimating a Continuous Distribution from a Sample Set
3. Cellular Automata for Simulation in Games

Vector fields are used in certain AI and simulation techniques. The tool below allows you to paint a vector field. These files can be saved in XML format, that can easily be loaded into another application.

There are five brushes:

• The Set Direction brush keeps the size of the underlying vector the same, but changes the size to a constant value.
• The Set Size brush keeps the size of the underlying vector the same, but changes the direction to a constant angle.
• The Attract brush draws all the vectors closer to the mouse cursor.
• The Repel brush pushes all vectors away from the mouse cursor.
• The Comb brush moves vectors in the same direction as the mouse.

The XML file arranges the field into a quadtree. The quadtree compression can be viewed from the program.

To change the brush size, use the ‘[' and ']‘ keys.

To change the brush options, use the ‘-’ or ‘=’ keys.

This is the first version of this program, so there are many missing features. For example, the size of the field can not be changed yet. If you find it useful, let me know in the comments!

Download

Windows Executable

force_field_editor_v1_0.exe 3.3 MB

Requires Java Runtime Environment.

Related posts:

1. Random Steering – 7 Components for a Toolkit
2. Quadtrees
3. Cellular Automata for Simulation in Games

For many applications, detailed statistical models are overkill. Instead, we can get away with a rough description of the distribution – not in mathematical formula form, but just as a graph with a few sample points.

For example, when trying to model the traffic around a school, you might know that the graph looks something like this:

The input is the number of minutes before the first bell rings, and the output the number of children dropped off at that time. You know that most kids are brought before the bell rings, and that the closer to the bell, the more kids are being brought every minute. Only a few kids are late.

This tutorial describes how to generate random numbers that can generate a distribution described by an arbitrary (piece-wise linear) curve, as the one above.

Implementation

1. Calculate an accumulative probability

The first step is to run an accumulative sum of our original sample points. The following code snippet shows how it can be done in C++; the array “samples” contains the original samples.

for(int i = 1; i < n; i++)
{
    samples[i] += samples[i - 1];
}

  The snippet above does the calculations in-place, but it is not necessary, as is shown in the snippet below:

newSamples[0] = samples[0];

for(int i = 1; i < n; i++)
{
    newSamples[i] = samples[i] + newSamples[i - 1];
}

  The graph above shows the accumulative probability density.

2. Calculate inverse sample points

This step is easy – we simply swap the input and output samples. It is not necessary to do this explicitly, just swap them in the argument list of the function call in the next step.

3. Interpolate between samples

I use a special data structure that makes it very easy to compute the interpolations from discreet sample points: the response curve. There are two varieties. The ordinary response curve, with uniformly spaced samples, and the xy-response curve, where samples can be arbitrarily spaced. Ordinary Response Curve. XY-Response Curve. The implementation is very simple, so I won’t describe it here. A C++ implementation is available in the Special Numbers Library; a Python implementation is available with the example below. Here I simply explain the usage as it relates to this tutorial. We proceed as follows:

• First, we construct an xy-response curve from our samples of the inverse accumulative probability.
• Then we sample this curve at regular intervals, and load it into a ordinary response curve (we do this, simply because the ordinary response curve is much faster than the xy-version).

//...

int oldSampleCount = 7;

int newSampleCount = 20;

float inputMin = 0.0f;
float inputMax = 3.2f;

//note: input and output is swopped arround, because we want the inverse!

XYResponseCurve<float, oldSampleCount> xyCurve(outputSamples, inputSamples);

for(int i = 0; i < newSampleCount; i++)

{

    input = ((float) i / (newSampleCount - 1)) * (inputMax - inputMin) + inputMin;

    uniformOutput[i] = xyCurve(input);

}

ResponseCurve<float, newSampleCount> curve(inputMin, inputMax, uniformOutput);

4. Map uniform random numbers to the input range of the IAPDF, and calculate the output

Now that we have the response curve, we can map uniform random numbers to the appropriate input range. For the example above, we need to map to the range [0, 3.2]. The snippet below shows how to generate random numbers with the distribution shown above:

//...create the curve c

float input = random(); //Uniformly distributed function between 0 and 1.

float scaledInput = r * (inputMax - inputMin) + inputMin;

float output = curve(scaledInput);

  The graph below shows how 10 000 random numbers are distributed. It follows the original graph closely; the discrepancy at -10 is caused by the way samples are counted (all samples between -10 (inclusive) and 0 (exclusive) are counted and plotted at -10.  

Tips and Pitfalls to Avoid

1. Generate sequences for all intermediary steps.
2. Use Excel, Calc, or some other spread sheet program to debug these sequences visually when things go wrong.
3. It is very easy to get confused with input, and output, especially after the swap. Watch out for this!
4. It is easy to get confused with the number of samples for the various sequences.
5. If you implement your own Response Curve data structure, unit tests will save you huge amounts of time.
6. Always make sure that you sample enough points, especially if your original distribution graph has rapid changes in it.
7. Always confirm that your random output follows the distribution you wanted.
8. It might be faster to use this method even when mathematical formulas are available.

Downloads

Example C++ Source Code

http://code-spot.co.za/downloads/cpp_examples/arbitrary_distribution.cpp Requires Special Numbers Library

Example Python Source Code

http://code-spot.co.za/downloads/python_examples/random_distributions.py

Related posts:

1. Generating Random Points from Arbitrary Distributions for 2D and Up
2. Generating Random Integers With Arbitrary Probabilities
3. Random Steering – 7 Components for a Toolkit