Balancing a life simulation with bots

The worst bug in a life game is the one a player will not notice: every path looks sensible on its own, but together everyone ends up in the same place. You will not see that by playing one life at a time. So we wrote bots to play the game.

One thing first: a bot is not a player. What a bot does does not tell you what a person will do. We use the bot as a measuring instrument: it shows where the rules can lead, not where people will go.

Two bots

• The random bot. It flips a coin at every decision. If the rules are unbalanced, it shows it in the plainest way.
• The purposeful bot. It picks a leaning for each life (there are four) and takes the option that fits that leaning at every decision. A rough imitation of a player with a consistent character, always drawn to the same "type" of option.

They are reported separately. The seed is fixed: repeat a run and you get the same result. A run plays 600 random and 600 purposeful lives, 1,200 in all; about 1,090 to 1,100 of them reach adulthood (the sixth season).

The goal

For this round we set three goals:

1. With the random bot, no adult path above thirty percent and none below eight.
2. The purposeful bot can actually reach the path it aims for.
3. All thirty-three jobs appear, and none accounts for more than ten percent of lives.

The skew we found

Adulthood splits into six paths: precarious work, craft, public service, a degree, returning home, and going abroad. In the first measurement the precarious path took 65.8 percent. Even with the random bot, 51 percent. The degree path got 3.1 percent.

Why? The precarious path was a "everyone else" bucket at the end of the rules, and three things pulled whole lives into it:

1. Having gone back home or resigned acted as a veto on the degree, craft and public paths. With the random bot, 71 percent of the precarious lives were there for this reason.
2. People who finished university did not go to the degree path unless a flag said so; they fell into precarious work. That was 48 percent of the graduates.
3. A graduate whose appointment fell through was thrown out of the degree path: 145 of the 249 lives of the studious purposeful bot.

So the game was telling someone who had studied and graduated "it didn't work out, you are precarious". In real life that can happen; but it should not be the thing the game says every time.

We changed the order of the rules and narrowed the vetoes. The result: with the random bot the largest path is 26.3 percent (craft) and the smallest 11.3 percent (abroad). With the purposeful bot, those who always study now reach the degree path (12.4 percent) and public service (33.6 percent); before, they all ended up in public service or precarious work, and the degree path was zero.

Jobs

The same measurement looked at jobs. In the first round only thirty-one of the thirty-three jobs came up; photographers and unemployed graduates never, police once. Meanwhile musician appeared in 173 lives and content creator in 157.

After the fix all thirty-three come up. The largest: doctor 85 lives (7.1 percent), precarious day labourer 77 (6.4 percent), unappointed graduate 76 (6.3 percent). None is above ten percent. Police 38, photographer 36, unemployed graduate 27.

Not fooling ourselves

To keep real bot runs to a minimum we did this: first we recorded the state of the 1,200 lives at the end of season five once, and tried six candidate rule orders on that record offline. The offline "before" numbers matched the real bot measurement exactly. That told us the method works. Then we confirmed the version we chose with the real bot, once.

There are limits, and we are not hiding them:

• The sample is small. A path's share can move by two points.
• The purposeful bot has three routes. It has no route for returning home or going abroad, so both are zero in its table. That is a limit of the bot, not a problem with the rules. We measure rule balance on the random bot.
• Public service at 33.6 percent with the purposeful bot comes from a single route ("always studies, takes the public exam"). That is a correct result.
• A bot is not a player. People choose differently.

After this measurement we re-ran the job and scene tests and got zero errors. The work is not finished. While we shortened the seasons, some chapters changed their counter contributions, so re-measuring the ending thresholds with the same bots is one of the open items in our notes.