Putting the 'role' back in role-playing games since 2002.
Donate to Codex
Good Old Games
  • Welcome to rpgcodex.net, a site dedicated to discussing computer based role-playing games in a free and open fashion. We're less strict than other forums, but please refer to the rules.

    "This message is awaiting moderator approval": All new users must pass through our moderation queue before they will be able to post normally. Until your account has "passed" your posts will only be visible to yourself (and moderators) until they are approved. Give us a week to get around to approving / deleting / ignoring your mundane opinion on crap before hassling us about it. Once you have passed the moderation period (think of it as a test), you will be able to post normally, just like all the other retards.

Microtown - One Guy Makes a Settlers Clone

Maxie

Guest
Microtown is a cute little Settlers-type game made from scratch by one guy in his spare time, both as a game and as a programming project. It's been released in Early Access in 2019 and continuously updated since, adding more mechanics, more production chains, streamlining the performance and fiddling with the art - simplistic pixel doodles on a hexagon grid.

Fair warning - it's still a very tentative title. Rudy (the dev) does his best and puts a lot of thought and effort into design, which he always explains in detail in patch notes. I do enjoy it as a little time sink to re-visit every now and then. There's no campaign mode and no in-game deadlines to meet, so feel free to leisurely build your town (or a viciously optimized labour camp).

It's never on sale (except for a brief -10% back in 2019), so there's little point in waiting for one, if you're interested.


I'll keep dumping patch notes here to keep the thread alive.
 

vortex

Fabulous Optimist
Joined
Mar 25, 2016
Messages
4,221
Location
Temple of Alvilmelkedic
I'll keep dumping patch notes here to keep the thread alive.

boris-thumbs-up.jpg
 

Maxie

Guest
EA Update #12 - Micropedia

This UI-focused update adds Micropedia, a browseable in-game information "encyclopedia". Together with improvements to tooltips and presentation as well as nicer-looking icons for all the important buildings and items.

Icons

Over time, I have been drawing icons for buildings in the main construction menu. I wanted to replace them because of how unrecognizable they are. It's one of the first things a player sees in the game and it remains the primary mechanic, and all new building unlocks appear here. So it sucks when the sprites are so interchangeable and it takes clicking and reading tooltips to find what one is looking for. (And it is unlikely that the in-world sprites could ever not only be distinct enough but also fit in the small buttons.)

Long story short, here's a comparison between an old and new construction menu page:

69e682418a1904f9d0460d45e1017b13932a4b3a.png


Even if not immediately obvious, these should be infinitely more memorable and recognizable. You don't even have to play the game to guess most of these. There are still tons of auxiliaries, but these mainly appear under their own parent buildings, so it's not such a big problem.

Originally, I was trying to come up with unique icons only for buildings in the construction menu. I was mostly using their tools (pickaxe for quarry, saw for sawmill, etc.) as icons:

38018348e2740a1fe18fb9da3139a1a37f270cf5.png


But I very soon ran out of tools, so to speak. Some buildings I could still represent with unique objects (like an anvil for smith), but others I had no idea how to represent (e.g. an icon for paper mill that isn't paper). So I ended up just drawing the primary items produced. And this actually worked out better than tools, so I drew items for most buildings instead.

Before long, I had about 70% of in-game items drawn, so at that point I might as well draw the rest. So now I had icons that I can use in all the HUD/UI elements with items, "automatically" improving these:

d8ffce82f299504502fa38cd17ee15bf8f108ad3.png


Tooltips

The majority of the work was actually for the tooltips. Micropedia is just a consequence of creating a structured easily-browseable repository for "all the tooltips". It has a bunch of extra information, but the data backend is the same.

Now, old tooltips did and still do their job:

752a07528fe8cd6e5141d199d7b8ff8e15623df0.png


Not great, not terrible.

However, my primary goal was to create extremely simple tooltip presentation that is also visually appealing. Like construction menu, this is the first place where a player looks to understand a building or item or anything. For example, the first thing a new player might learn is these:

c52eb482922472f964f8bee864d86042531846be.png


Lumberjack and sawmill aren't exactly complicated, but a player who has learned this "language" can look at a more complex tooltip and understand what is required:

fcb3ae61a423feefe80eb7b8d7e9bb8f29e31e48.png


Then, from there I can use the same formula and expand until I have, for example, cheese inspection:

7c65356bae39752949dc7a8300b1fe6e8dbe54aa.png


There's a whole lot of different things I can show. My main design problem was actually limiting which info I show. For example, tooltips are limited to the most important stuff.

Micropedia

And so the Micropedia is a separate in-game window that has an organized list of every entity and concept in the game. (Well, not every every one, but all the main ones anyway and then some.)

1627c213577a666a89592bb3f2f96e30f5809d41.png


I expect that Micropedia would serve two primary use cases - (1) looking up something specific and (2) just browsing.

For the (1) case -- the player likely has a question about something, probably what some entity is or does or needs. If Micropedia can actually answer such question, then the quickest way to the answer is most likely through a search:

84bc1a61fbbb92cc74dcb2fedc70a7631bcd6671.png


Opening Micropedia instantly focuses on the search bar and you can start typing. Hit Enter and it will go to the best result.

Or alternatively for (2) browsing, everything in Micropedia is broken down by categories (items, buildings, animals, techs, etc.) and then each item is listed (mostly, alphabetically). So you can see and browse all entries.

Currently, Micropedia is rather technical and doesn't list many specific values or go into explainations. I will be adding more of these in the future.

Intermediary items

I combined several buildings, and removed all intermediary items (see the full notes for the list). This may or may not break some games, depending on how those building were arranged. The merged building are Brickyard, Paper Mill and Textile Mill:

40ff6ed08b6a400a93b32063b670c598f2da8e19.png


There are a few items in the game that are "intermediate". For example, Cheesemonger first brews Milk into Curd and then pressed Curd into Cheese. This Curd item is a real in-game item with all the related features. Then there's buildings like Brickworks and Brickyard which not only have intermediate Grout, but also Unfired Bricks delivered between them.

I think it was a cool idea to show these items and to see them carried around. The problem is that these clutter up the UI -- every place that lists items, like item report, also lists these items. (And if I were to just hide them, then what's the point of having them?)

I also adjusted the inspection HUD to better show to the player that there's a "virtual" item that gets transferred between auxiliaries. For example, Brick Moulder:

05ae6df7897ea402a21220630f66e2dc838be64d.png


Future plans

This update was a little more chill in terms of workload. It still took twice as long than I had planned. And the UI work is always tricky. But it was still "holiday reduced workload update". However, I do need to get back to some of the tough changes.

I want to get back to expanding the end goal and making the monument a larger objective. And there are numerous steps along the way. I'm not even sure how large and complex each step would be. For example, I have to make multi-tile buildings. I need to change how construction tasks work and allow multiple builders. Builders need to be able to carry items. I need to rework how construction input items are stored and used/reclaimed/counted. I want to add multi-step construction for the monument. This probably means building upgrading. Likely I need foundation/platform construction. I need to add more and new materials for all of it. Etc. In short, my main goal for "endgame" has a lot of mini-goals that I will likely add one at a time.

Full changelog

Changes

• Remove Brickworks, Grout and Unfired Bricks
• Grout Mixer and Brick Moulder are now part of Brickyard
• Brickyard now directly receives Sand and Clay instead
• Remove Pulp Mill and Paper Pulp
• Pulp Vat is now part of Paper Mill
• Paper Mill now directly receives Hemp and Water instead
• Remove Carding Mill
• Carding Machine and Spinning Mule are now part of Textile Mill
• Textile Mill now directly receives Hemp and Wool instead
• Remove Linen Sliver, Linen Yarn, Wool Sliver and Wool Yarn
• Remove Unfired Pottery
• Remove Clean Fleece
• Remove Curd
• Remove Honeycomb
• Add Micropedia window - an organized information repository - split into categories and their entries
• Add categories for Micropedia - concepts, items, buildings, roads, villagers, animals, props, tiles, technologies and seasons
• Each entry in Micropedia shows its information - description, sprite(s), usage and some hard-wired related info
• Entities and links are clickable in Micropedia, switching to the corresponding entry
• Add unique recognizable icons for all items, roads and most buildings
• Most places in UI/HUD use the new icons for entities
• Expand entity tooltips to include usage/production information and other most-relevant information
• Add various new concepts
• Add and tweak various icons
• Add and tweak various sprites
• Change descriptions for all the changed buildings, items and others
• Remove Water Tower
• Add "Hide UI" button to pause menu that hides UI/HUD in-game (and prevents input for the duration of it)
• Building workers are now more reluctant to carry their own items
• Beehive now shows a warning when no Flowers are in the operating range
• Add a privacy notice and some extra info to save upload window; make toggle for optional Steam ID submission
• Remove the Population count requirement to unlock the Technology Branches
• Add a goal for building Town Hall
• Add wild Animals - Deer, Boars, Foxes, and Wolves
• Wild animals will spawn in Animal Habitats up to a limit
• Animal Habitats spawn and despawn in the world based on surrounding forest "score"
• New worlds and worlds loaded from older saver will have up to the default number of Animal Habitats spawned
• Animal Habitats show an in-world indicator and the mouseover tooltip provides additional information
• Rename Pork to Meat and Ham to Roast
• Add Hunting technology available for research from the start
• Add Hunter's Lodge building unlocked with Hunting
• Wild animals can be hunted - the worker will approach, shoot and harvest the animal for Meat
• Recently deceased wild animals can be skinned - the worker will harvest the animal for Meat
• Hunter's Lodge can individually toggle which animals are hunted
• Wild animals that die leave corpses instead of despawning immediately; the corpses eventually decay
• Add Game Warden's Lodge who will tend nearby habitats
• Herding now requires Hunting instead of Farming
• Smokehouse and Drying Rack now unlock with Hunting instead of Herding
• Add more particle effects to Drying Rack
• Add Animal and Animal Habitat concepts
• Trees no longer grow during Winter
• Forester and Florist no longer plant during Winter
• Various text and sprite adjustments
• Add wild Animal "baby" versions
• Only grown Animals can be hunted or skinned by the Hunter's Lodge
• Animal Habitats in new worlds generate with the appropriate number of Animals

Fixes

• Animal Habitat improved quality score from Game Warden's Lodge not being deducted after the improvement time runs out
• Workers at some buildings with multiple auxiliaries would not complete work at auxiliary but run off to another auxiliary
• Tooltips showing "through" opaque/blocking UI elements
• Mouse scroll wheel no longer zooms in/out when over HUD/UI
• Keyboard numpad +/- zoom keys now have a reasonable delay to zoom in/out one "step" at a time
• Water not being "unlocked" from the start and inspection HUD not showing Well items
• Barn is no longer unlocked if none of the items it stores are unlocked yet
• HUD pause menu button not playing a "click" sound
• Incorrect distance calculation for pending deliveries
• Farm, Gardener and Arborist planting and collecting their production entities on unfinished auxiliary buildings and on untilled tiles
• Transit toggle buttons blocking storage selection in storage building inspection HUD panel
• Fix Water Tower being treated like a Tavern and fulfilling Houses tavern resident sentiment
• Fix rare exception/stall bug when a worker switches a harvest task at the same as the target plant decays
• Fix rare exception bug when an animal regains its product at the same time as dying
• Fix Gardener and Arborist occasionally not tilling a tile just after planting in all other tiles
• Fix wrong Drying Rack meat sprite layer
• Correct Unfired Pottery plural name typo
• Play the correct UI sound for road upgrading via the U shortcut
• Fix a very rare error where a task for livestock animal is assigned right as it is dying
• Fix animal inspection HUD box lingering after animal leaves the selected tile
• Fix Windmill starting milling and creating Flour without any Wheat
• Fix Hunter's Lodge workers sometimes running on the spot when approaching animals
• Fix Farms, Gardeners, Arborists planting things in the wrong Seasons
• Fix building workers occasionally not doing anything (because they had another task planned, which is no longer valid), for example Farm workers not tilling a field
• Animals in new Habitats start aged proportionally (instead all dying at the same time)
• Closing early access intro window with Esc doesn't open patch note or link window with main menu

Balancing

• Wild Animals spawn a bit slower in Animal Habitats

Optimizations

• Faster tile border and indicator re-drawing and minimize the lagspike on swapping/relocating them
• Buildings to not look for tasks for disabled worker slots
• Remove redundant task condition checks when first deciding on a task
• Add delay between buildings failing to assign potential tasks
• Remove expensive periodic task checks that can instead be aborted later
• Check building tasks according to priority to avoid unnecessary checks

Rudy
Snowy Ash Games
 

Maxie

Guest
EA Update #13 - Construction
This update focuses on the backend features for building construction, broadly speaking. My main eventual design goal is to make construction feel like a larger part of village development. This means things like longer build times, additional construction steps, more required materials, more builders, larger buildings, more builder animations and actions, more visuals, more feedback, etc. As part of it, I also added a lot of smaller features that I wanted to add but couldn't before.

Multi-tile buildings

The biggest change (at least, internally) is the support for multi-tile buildings. Here is the "new" town hall on a 2x2 footprint (or 7 tiles):

d54fc3f2e4135dddc0c5da3c0d59fda987e641ca.png


I have not yet "converted" any other production buildings, because this would basically remove them without replacing and break any games. Besides, I need new art for them all and this is a out of scope for this update (see also future plans).

And here is the new (unfinished) monument on a 3x3 footprint (or 19 tiles):

eea6de71da8caf33a4a1ac3ff1780c6b02d910e5.png


With this, I also added support for multiple builders at the same construction site. Larger (future) buildings will require more time to complete, but this will also be offset by more builders working on them. This is probably the primary way in which I can visually "explain" that constructing buildings is a significant endeavor.

Notably on visual side, I added the graphics for construction progress for buildings:

39aa719e1a131cd668687316a42035bdce0b1e29.png


This was one of the visual things that has been bugging me forever, but I was not able to add it due to how the construction and deconstruction progress was stored. It is now self-consistent (and you can no longer cheese the materials by toggling between de/construction).

Buildings can now also support more than 3 materials for construction, notably the monument will now also require planks and iron ingots.

Dirt

I added a new tile type "dirt", which is basically dug-up grass. This will be the "default" tile under all buildings:

1397dd63d2f91e2e22ead9ac7ed7b9ff600d3482.png


Builders will dig up the tile at the start of construction and will now actually convert it to dirt (if it's not already dirt). Previously, only workers at places like farms and ranches would dig up the working tiles, but these were just a variant of grass. They will still do it, but now they will work directly from dirt and further prepare them for use.

This makes the village look more lived in and creates a starker visual contrast between the village and the surrounding nature.

The dirt will also slowly regrow back into grass if there's no building on it:

579fd7dd2f8e71491be7b471a1f6c91cf85c7876.png


With dirt tiles being their own "type", I can also add transitions between different tile types. (Before, "fake dirt" was only at the borders of crop fields, animal pens, etc., so it didn't really matter if it had a "sharp edge" with the grass outside the area.)

e2dd736330d502428084e0676042f569b40b6879.png


I numbered the different transitions above and you can see how just such a small location can have transitioning surfaces from everywhich side.

I even had to work on improving my pipeline for transition creation, such as a quick preview so I can tweak any issues:

7d247b53d25b3912bde0a568571bd2f515470b67.png


This change is similarly mostly about the backend. Previously, I would never change the tile after creating it (dirt was just a variant of grass). But this also meant that everything assumed that it can "connect" to tiles safely and the tile would never disappear or change its type. So now I have the ability to transform the tiles as desired, which paves the road for various tile-related features.

Self-delivery toggles

Buildings can now "decide" if the workers will self-deliver items, that is, bring nearby input items and deliver nearby output items to other buildings or storage instead of waiting for a carrier. This behaviour is both a positive and a negative feature and the line is very blurry and based on so many factors. When it works, it's great. When it doesn't, it can ruin production. I thought about this a lot and in the end I decided to allow the players to enable and disable it. This would still be important even after upcoming updates and changes to logistics, so I might as well do it right.

Firstly, each building with workers now has a toggle to either allow or disallow self-deliveries or keep it at the default setting:

a8b142f5c6fcb93d3971727efbab0cc4d5086515.png


Secondly, the item distribution window now offers a building self-delivery toggle tab that lists all the building types and each has its own toggle for all buildings of that type:

d1efcb03abbc654e3bb3b343fe1e76647df63381.png


There is also a global toggle to simply disallow all self-deliveries by default. This can result in very different layout and carrier requirements. With a small village, it almost always works better with self-deliveries, and this is the default option and this is also part of my "just works" design goal. In the end, I am allowing the players to choose how their workers spend their time and if they want to optimize long-term or react short-term, I'm happy either way.

Markers

Selecting a building, will now show the a small in-world marker above the items that are being delivered/imported to (blue, arrow in) or taken away/exported (orange, arrow out) from the building:

5b0346bbf965fa204feb3cfd98dcca5ac82bbb22.png


This will also show the villagers performing the delivery before the items is yet picked up:

32932fdfcf452e8203bd8faa398b1e74884b2f3d.png


This should make it much easier to see what items are being delivered without needing to check the tooltip that can only show a list of items and their approximate distances.

I will likely expand this to "mark" other related things as well, like workers themselves. However, this is a finicky feature that takes a lot of time, so I will work on this at a later date. My primary goal was to show the items and carriers since these cause the most confusion, especially for construction. This will also be particularly helpful for the future logistics updates.

Various

Most of the work was in the backend, but here as some notable changes.

All buildings with workers now track their efficiency and show it in the HUD:

63e341cc5839ad47f95e5dc88d9e61be3e4e0b7a.png


This is simply the percentage of time that the workers spent working. In addition, I also show the time spent self-delivering items so it's clear when workers are spending time hauling items versus doing the building's designated tasks, which is not necessarily bad but also not necessarily good. In any case, the player can decide what they want to do with this information and possibly use the self-delivery toggles.

I (finally) changed the minimum carrier value from an absolute number to a proportion and changed the UI to be a percentage slider:

1f1424932f5d028fe15a71355c00277a8d6c93ca.png


Importantly, I made it so that going under this limit will start dismissing workers from the buildings. For this, I use the most occupied and then least efficient workers.

This now functions more intuitively like "I want X% of villagers to be carriers" rather than the more convoluted "minimum".

I also changed how the UI shows the de/construction material items:

8fec4cea368512d368c726ae21584c485c216d99.png


This is an extreme case for the monument, so there a lots of items. But for regular buildings, this should work up nicely. The old "button" was just too unclear and cramped.

I split the top HUD overview and their tooltips into the tradesmen/carrier/builder sections:

d9524ca4a57742c3684e5140d6169cad52c1f11d.png


I removed trying to show green/red status icons for these, since they are almost always just a little wrong. The end result is that a village working 98% of the time has red icons while a village not doing anything can have green icons (since technically all tasks are fulfilled). I think I was trying to be too smart trying to "tell the player" what the goal should be when it's not actually a goal at all.

Future plans

This update took way longer than anticipated because of a fundamental backend assumption that tiles map 1:1 to buildings and that tiles remain "fixed" throughout. Adding such fundamental features retroactively is never fun. For example, path-finding had no idea how to handle not being able to walk between multi-tile buildings. I think it was worth it just to allow the larger monument. But, of course, there are tons of buildings that could be multi-tile.

The next thing I am working on before anything else is a delivery "caravan" system. There is a
lot
to discuss about it and I will probably leave it for the next update notes. I didn't do many logistics changes in this update mostly to not delay it any longer. For example, there are still various issues with construction, particularly with priorities and deliveries. But much of it needs logistics "fixed" first.

For the more distant future, the next part of building changes would be making all primary buildings be 3x3 tiles. That is, for example, a farm would be 3x3 while the fields would remain 1x1. However, I will not do this change right now as there's a specific problem I want to focus on.

Of course, there are all the other things I have mentioned here and there before - markets, better civilian needs, happiness overhaul, more production chains, trading, etc. etc.

Full changelog

Changes

• Monument is now 3x3 (19 tiles)
• Town Hall is now 2x2 (7 tiles)
• Multi-tile buildings are now supported by the backend: currently used footprints are 2x2 or 7 tile "circle" and 3x3 or 19 tile "circle"
• Add multi-tile outlines for building selection and construction
• Selecting any tile of a multi-tile building selects both the exact tile and the building itself and with separate outlines
• Add (de)construction markers for multi-tile buildings
• Buildings are now (de)constructed through several visual construction steps; add in-progress sprites for primary and auxiliary buildings
• Update (de)construction item progress UI to be shown as a progress bar with sections for used/delivered/pending/needed items with appropriate icons
• Make (de)construction progress bar advance smoothly and more accurately than before
• Town Hall's footprint is now 2x2 or 7 tiles
• New larger Town Hall sprite
• Monument's footprint is now 3x3 or 19 tiles
• Monument now also requires Iron Ingots and Planks for construction
• Add Iron Ingot to building proportion item distribution window
• New large Monument sprite and its construction steps
• Make Dirt tile a separate tile for "used" tiles, including its previous "variants": Tilled Dirt for farming and Stomped Dirt for animal husbandry
• Builders clearing a tile for construction will now convert it to Dirt
• Dirt tiles will slowly revert back to grass as Regrowing Dirt and Regrowing Dirt if there is no building on top; other dirt types will first turn into "regular" dirt
• Tilled Dirt under Crop Fields and Garden Plots will turn into Wasted Dirt at the arrival of Winter and workers will need to retill it
• Tree Nurseries don't need the tile tilled anymore
• Buildings with workers will show their work efficiency percentage in inspection HUD based on their time spent working
• Building workers will now follow a self-delivery setting and not bring or deliver items to and from their building if self-delivery is disabled
• Add self-delivery toggle (enabled, default, disabled) to building inspection HUD
• Add a "Self-deliveries" tab to item distribution window with a global self-delivery toggle (enabled/disabled) and per-building type (enabled, default, disabled) delivery toggles
• Change profession distribution carrier minimum value from an absolute value to a percentage value
• Workers will now leave their buildings to become carriers to reach the global carrier threshold
• Split top overview work section into individual "Tradework", "Deliveries" and "Construction" sections and simplify the tooltips
• Remove unclear and often incorrect green/red-colored status colors from top overview worker numbers/labels
• When a building is selected, items being delivered as inputs or carried away from outputs and carriers coming to take the items are indicated with in-world markers with appropriate icons (blue - incoming, orange - outgoing)
• Add UI/HUD button pressed variant and change various buttons that are "toggled" to have the pressed version (instead of just color change)
• Add water tile animation
• Update and improve various tile transition sprites
• Remove old save potential upgrade notice from save/load menu

Fixes

• Toggling between construction and deconstruction no longer returns material items randomly, but proportional to what was actually used
• Item distribution UI stuck with previous world's values after load or new game
• Rare exception when moving obstructing items from a construction site when the items are chosen for delivery
• Incorrect partial tile border outline for selected buildings with no auxiliaries
• Rare exception when selecting between buildings that have differing border logic
• Switching between different building construction would not update the invalid tile outlines in some cases
• Butcher producing Meat without Pigs
• Young Deer walking sprite animation mismatch
• Wrong tooltip info for Codex production "recipe"
• Wool Fabric production chain missing from Carding Machine tooltip
• Shift-selecting unbuildable building would change selection but not visuals
• Certain world changes would not be reflected as current selection changes
• Wrong Sapling removal animation
• Worker digging animation not showing particle effects in many cases
• Minor selection change trigger issues
• Occasional tile transition sprite overlap flicker
• Minor tile transition sprite issues
• Brewing Vat to not individually select production, rather parent Brewery to decide what to brew
• Confirmation window to use the Enter/Submit input instead of passing it to window under it
• Saplings to not grow in Winter

Balancing

• Increase various construction costs
• Increase various construction action durations
• Saplings grow faster
• Trees grow slower
• Tree Nursery workers plant and dig up Saplings faster

Optimizations

• Optimize tile updating speed (however, all water tiles need constant updates, so overall the game is slower)
• Improve certain redundant frequent update calls

Rudy
Snowy Ash Games
 

Maxie

Wholesome Chungus
Patron
Glory to Ukraine
Joined
Nov 13, 2021
Messages
6,753
Location
Grantham, UK
EA Update #15 - Fishing
This smaller update adds fish animals and fishing to the game.

Fishing

Firstly, fish will now spawn in their own water-based habitats:

227fd64348aebb443a73863bf83184cd5c8947e7.png


These work very similarly to land-based habitats, except for the requirement to be in shallow water. There are three types of fish/habitats: Striper, Salmon and Marlin.

After unlocking the Fishing technology, a Fisherman's Hut can be built. The workers will go to the nearest coast with fish and cast a rod and wait for a fish to hook:

c24c358fffa95b15e13f1844ede55ff52ad84984.png


This catches the fish corresponding to the animal's species:

3ebee4e9408c10f6eb66231227ee1f4fd55c4ee6.png


(I considered whether a "caught fish" should be the same item or if each species should have its own. There is no functional difference and I doubt there ever will be. But the extra items do take up the extra space in UIs. I don't really mind this too much and I find it cool to have variety. Although I might need to eventually figure out some sort of grouping of similar items, like in the Item Report window.)

The raw fish then has to be taken to the Fishmonger building, where it's prepared into a Fish Steak at a Gutting Table:

bf347c2e4fa9d03911580c1dcc7bb7252f2d0e86.png


From there, Fish Steaks are taken to the Smokehouse, exactly the same as raw Meat:

d777a176ec5d90faca3306b3ecbea44a41da745a.png


In fact, this is so exactly like Meat, that the item produced is also a Roast. I guess no one can tell the difference. But, on a serious note, I do not want to add more food or supply items at this time. There are already 12 of them and the way Markets and Houses handle them is rather subpar. Taking each individual item to houses and having houses track each of the 12 items is simply too cumbersome. So this is a compromise until I rework the villager "shopping experience".

Path-finding

No day goes by when I don't discover there is some complex backend code that I need to work on to enable a feature. Of all the things to break, I didn't imagine imagine fishing would break path-finding so badly.

First of all, fish "walk" in water, so I had to create rules and separate tile "swimmability" flags that path-finding can use depending on how a unit moves (i.e. on land or on water). But that's relatively easy.

Instead, I discovered a bug where workers could not find a path to some buildings that were built on the coast (i.e. if grass would touch the water). To explain the problem, I need to quickly explain how path-finding coordinates work in the game.

The game uses hex tiles, and so the coordinate system is not a straight-forward X-Y horizontal-vertical axis. But for that the path-finding is still not too difficult to "solve", I just have to be careful how coordinates are assigned and what directions are allowed for movement:

907cd5d564253141d77d665b5d579c64dbd02e24.png


(If you want to know more technical details on the basics of hex grids, then look no further than the excellent https://www.redblobgames.com/grids/hexagons/ guide.)

The tricky part is that I don't actually want movement just between tile centers. You likely have noticed that workers run between buildings, although it would not be obvious how path-finding does this when you consider the coordinate system it works with. For MicroTown, I only allow movement between the adjacent centers and corners:

709677196ad0c7976d7f8a9199cc641f61be3279.png


In fact, internally, I have 3 times as many path-finding "nodes" as tiles in each axis direction. Here are the node relative "distances" and "missing" nodes from the full path-finding data:

3bf8fc68829b34b503c4496b46eae5a97dc19ac4.png


So in fact units cover many more "locations" and "routes" to reach them that just moving between tiles when looking for paths:

9148eb08e4bf6ab19427d894bb63ec02c52cf96b.png


Besides walking between structures, another important reason why I am doing this is because units need to access certain buildings from a certain direction. For example, the fisherman has the door pointing South-West, so workers need to use the South-West tile or corner:

228c27e1f628ddb15439135f977b9d1aaba6535f.png


And this was where the bug happened. The design-to-implementation question here is: which tile does a corner belong to? And in what directions does each corner enable access to buildings? Because each corner is part of 3 tiles:

0dc9f9d47d0c53e70a3155b581e3e76f9ed1083f.png


This does not matter on land. But I had not carefully debugged what happens near the coast (in fact, I had to implement the above visual debugging to see what was going on). Depending on direction, corners would use the water tile to decide if they can be traversed, thus making buildings inaccessible.

So to do this properly, when I update path-finding data for a tile, I am also updating corner path-finding data. I need to check all the adjacent tiles, because every corner has a combination of 3 different tiles. For land-based path-finding, this means allowing corner movement if any "connected" tile is a land tile. So, in essence, every coastal tile is fully walkable:

a0459d9e07f5f90550cdde298602580ec3b37e5c.png


Of course, this is just one aspect of what is needed and there is much more happening, like road priorities, multi-tile building avoidance, full-tile movement preference, direction-dependent movement validation, preferred route distance estimation, etc. Much of the difficulty of changing such a fundamental system is all these additional factors that have to keep working.

Other changes

One of bigger UI changed is that all technologies are now in the same window and tab:



I basically removed technology branches, because they served no real purpose other than to have empty unfinished-looking tabs in the tech window. It also seems like most technologies are some sort of production ones anyway.

Future plans

In the future, for the fishing specifically, I can imagine adding something like fish oil item for... some purpose. I can also imagine more fancy foods, like crabs, or specific items, like pearls that could be acquired. I originally thought that fishing would be done in boats, but decided not to over-complicate the matters at this point.

I also think Hunters would eventually have a dedicated skinning/preparation building where additional items are produced, such as furs. Then I would have individually-itemed animal carcasses similar to fish species.

However, the big plan is to improve the Market logic. I am more than happy that many long production chains culminate in arrival at the market. But I am rather unhappy with how individual per-item market stalls have to then distribute the items to individual House slots. I think this is boring linear micro-management. Basically, the market may as well have come with the stalls "pre-installed" for all the use their exact layout does.

So I will likely "convert" stalls into shops of sorts, like a Meat shop or a Produce shop or a Clothes shop. That way, I can have a similar number of shops as stalls, but they can "sell" several items and thus overall, it would be a much larger variety of items. This would gradually enable me to add currency as well. Then the central Market "authority" would be responsible for taxing the shops or some such. This is very likely what I will work on next.

Full changelog

Changes

• Add Fisherman's Hut building
• Add Striper, Salmon and Marlin as "wild Animals"
• Add Striper, Salmon and Marlin items fished from respective fish animals
• Add Fishmonger building with Gutting Table
• Add Fish Steak item produced from fish
• Add water-based Animal Habitats for the new fish species placed along the shore
• Land and water Animal Habitats have separate spawn rules, locations, habitat score calculation rules, visuals and various other properties
• Game Warden's Lodge is now specifically for land-based Animal Habitats only (and mentions this in description)
• Add Fishing tech, unlockable from start
• Add fishing Goal to catch fish
• Add "Animals hunted" and "Fish caught" (internal) statistics
• Smokehouse and Drying Rack can now also cook Fish Steaks into Roast (for now)
• Adjust building worker idling locations
• Idle building workers will now occasionally rotate
• Add concepts: Livestock, Game, Fish
• Not showing icons anymore for in-text concepts
• Canceling building deconstruction before it begins will "suck back up" building's compatible items
• Import Station to show item stacks separated
• Remove technology branches -- all technologies are now on the same page
• Technology window entries no longer show their name labels
• Technology window will resize to fit more of the screen vertically and can now scroll its content vertically
• Mr. Piggles' portrait is now the game's icon
• Newly-available building output and stored items now have a short delay during which they won't get assigned to regular deliveries, but can still be picked up by large storage building and Export Stations workers thus prioritizing them over carriers
• Building workers can now discard the resulting item of an operation if this item is no longer compatible, this applies to Brewing Vat and Loom
• Import Station can now accept and store up to 6 types of items independently; it will no longer get "clogged" by one type of item when receiving multiple types of items
• Up to 3 types of incoming items to Import Station are now shown in building inspection HUD with "progress" bars
• Loading saves where Export Station or Import Station has to be reset will preserve previous shipping routes (although active deliveries will be cancelled)
• Shipping routes can now only be set to Import Stations that are complete or under construction (but not deconstructing)
• Export Station inspection shipping routes selection will only show if it is complete or under construction (but not deconstructing)

Fixes

• Add missing Hunting Goal to hunt animals
• Some valid coordinates were not considered for Animal Habitat placement
• Fix some minor animal animation issues
• Pathfinding would not check coastal corners of tiles and workers could not go to buildings placed adjacent to coasts
• Fix profession distribution panel builder increase button decreasing the value instead; fix tooltips on all buttons being on the wrong buttons
• MacOS builds are now Intel only until Steam-injected libraries support Universal (i.e. ARM) builds and work on M1 chip systems
• Tooltips getting stuck when showing entries that check game progression while not in a game (i.e. main menu)
• Internal exception when removing a shipping route at a Export Station and cancelling tasks involved with the route due to not accounting for certain tasks failing to cancel remaining tasks and corrupting the internal route state
• Import Station clearing its incoming routes after construction
• Export Station not clearing its outgoing routes when deconstructing
• Some rare issues adding/removing shipping routes during building construction
• Import Station workers would only distribute the (internally) first stack of items

Balancing

• Smokehouse can now hire 3 instead of just 1 workers
• Drying Rack takes longer to "cook" meat
 

Maxie

Wholesome Chungus
Patron
Glory to Ukraine
Joined
Nov 13, 2021
Messages
6,753
Location
Grantham, UK
EA Update #16 - Shops
This update replaces Markets with Shops and adds significant rendering optimizations. And I might as well call this "Rendering Update", because I spent most of the time working on optimizing sprite drawing.

Shops

Shops are new buildings that are set to replace Markets and Market Stalls.

3e244c04b993a59f68e0673ee51db598779a5341.png


They essentially do the same thing that Stalls do, except each shop can store and distribute several item types. For example, a Produce shop would sell Tomatoes, Carrots and Potatoes at the same time:

e7f2b8100fe66c111bd8f1c3e9a43a35eca227e6.png


Shops allow me to add more items without adding a lot of micromanagement to Markets and Stalls. I can instead focus on logical item "groups", such as foods, supplies, clothing, well-being, luxuries, etc. The gameplay would then not be so much about having exact items, but a decent variety of items and an overall satisfaction for that item group. It's also much easier for me to design for and players to work with fewer categories.

For now, Markets and Stalls will still work, but they are tagged as obsolete and will be removed (probably) in the next update. Basically, I'm leaving them working so anyone who wants to update their world can do so. Unfortunately, I cannot maintain two versions of item distribution, so I will have to completely remove Stalls later. I also cannot automatically replace them in-game with shops, because shops are 7-tile buildings.

Miscellaneous changes

There are now 2 more larger world size options. Thanks to the optimizations, I am finally able to create bigger islands without lagging everything. These won't have better terrain generation yet and the camp still spawns in the middle, but at least they are an option now.

Animal habitats now have to be selected to see their information:

2c93785802345bab592458732d1ffd4bacfc3608.png


This is mostly because I could not easily re-implement in-world tooltips following the rendering changes. But I also had this planned regardless, so I just went with it. There isn't any new information yet (notably, about quality factors).

Rendering optimization

So the problem is: MicroTown's rendering is fairly slow because there are a lot of things to draw. I should note that MicroTown being "just" pixel art is rather deceptive. The game needs to render more geometry than many 3D games and, for various reasons related to how GPUs work, it's not much cheaper to send "just" sprites to render than it is to send 3D models.

Game world renderers

Optimization should generally be done based on profiling, i.e. evidence. So, most importantly, I need to count the number of sprites that are required to draw everything in the game world. For my test case, I am "looking" at a 1440p resolution zoomed-out 160-size world with 2000 villagers. These are the numbers of in-world entities that exist in the world:

ea785bba745149cfb676b3c34963386542d62409.png


This is a rough indicator of how much "stuff" needs to be drawn, but it's inaccurate because each entity does not correspond to a single sprite. There could be zero to a dozen of sprites needed for an entity, depending on many factors. This also does not account for many extra things, like effects, tile overlays and outlines, markers, building icons, etc.

So here are the numbers of individual sprites:

41a4e2f0168ec1cf551634903c6c9c9274db466b.png


These can be further broken down into broader categories:

0c7860546ed158a88bd316fc391b16bd74826e0e.png


The numbers I am most interested in are the 25k/54k renderers. This is how many sprites need to actually be drawn out of the total sprites that exist. Importantly, there are 5k updates per frame, which means that that many sprites were changed, added or removed.

Which means there are two main focuses for entity processing optimization - knowing what to render and processing changes quickly. Subsequently, the main rendering optimization focus is to render the visible sprites as fast as possible. And these two sentences are so much easier said than done.

CPU rendering bottleneck

The main bottleneck is the CPU time it takes to render the game world. Unity works with "game objects" to do things, broadly speaking. For the game world, I only use game objects exclusively for sprite rendering (no data or logic there). Each sprite renderer requires a separate game object. So every sprite I mentioned above requires a game object, which is easily 50k objects. Meanwhile, for Unity, general consensus is that 10k objects is too many.

I basically have these long long lists of objects:

113adc337e6af1f7542b8336c7e855aabd581d00.png


Unfortunately, there is only so much I can do before Unity itself bottlenecks like this. I've squeezed out as much performance as I could with this setup. Without going into too much detail, there are too many engine-dependent things happening that I cannot change in any way like sorting, culling and communication with the GPU. Unity simply isn't built to support so many objects via its "conventional" game object means. It doesn't matter that the GPU renders everything in 0.5 ms when it takes 25 ms for Unity to prepare and send the data. It still has the overhead of all those universal game objects regardless how many are visible and how cleverly they are optimized for rendering.

So here comes the technical and difficult part...

Custom rendering pipeline

After lots of research and experiments, I decided that I would need to bypass almost all of Unity's "standard" rendering overhead and send sprites to the GPU more-or-less directly. This is essentially how most GPU-based 2D games have always done this. Simply put, "draw sprite S at X, Y" for every sprite. Unfortunately, this is much easier said than done nor does it capture the many complexities. Unity isn't a 2D game engine and all its features have many layers of expensive abstraction. But thankfully Unity does provide rough access to lowish-level graphics logic.

In short, I can send a mesh to render directly to the GPU. A mesh is just a bunch of triangles connected at their vertices. In my case, to show a sprite, I only need a quad -- a rectangular mesh that is 4 vertices and 2 triangles.(This is also what Unity uses by default for sprite renderers, except with a lot more stuff.)

ad977b48e0039a9c85d1b2b4673be77e8b7f1c01.png


I can not only send a mesh once, but "send" the same mesh again and again, which is called GPU instancing and which saves the day. Technically, I am not even sending multiple meshes - just the one quad, but I am telling the GPU where to draw copies of it. So each quad mesh would correspond to one sprite and I would send a batch of such sprite data to the GPU to get them all rendered at once really, really fast. This lets me render any arbitrary layout:

848e20a7faab80127a3b9e9bdcea73f4676f0e00.png


Fortunately, I have built the game in a way that allows me to fairly easily "collect" all the sprites I need to render (which still massively downplays the difficulty of this conversion). As I described, I have game objects that correspond somewhat to game world entities. I know where they are located and roughly in what order. So I "just" need to pass these to the GPU myself. It would look something like this (with a colorful test texture):

c44ee5fe8fab2765915faeddb3cb65de6411cd9e.png


Except, there are no "sprites" on a GPU, there are definitely no sprite atlases. All the "sprites" here look the same. Because a "sprite" is a high-level engine abstraction done through custom assets, custom shaders and lots of engine code. All I get at low-level is a texture and raw vertices and pixels. What I really need to do it specify locations on the final atlas texture, so I can draw specific sprites that would be arranged there (these are 16x16 "squares" from the same 2048x2048 test texture):

9e5dc55fe86306e43028248e936159980e55951b.png


The next step is to somehow combine individual meshes and send things to the GPU in batches. The problem is communicating what data I need to send per mesh, that is, per sprite. Naively, I can only set one property per communication, which basically results in every sprite in a batch being the same:

f25ac212d27a35e185bc017120602d810cd8d865.png


The solution is that modernish GPU shaders can be compute shaders, which can receive and process large chunks of arbitrary data and run (parallel) processing on the GPU. This means I can actually pass all the sprite information to the shader in one big batch very efficiently. This data can then be sampled to select the correct region from the atlas texture from the sprite's location data for each mesh/quad.

And this provides the starting basic functionality of rendering sprites directly to the GPU almost the same as game object sprite renderers, but for a tiny fraction of the cost. Here is the pipeline itself working correctly if not the logic of drawing everything:

2f830eea79a7fdfef573557269a5ba3817c3f464.png


This does unfortunately come with a bunch of significant drawbacks that can be summarized as "additional complexity" and whatever is the opposite of "ease of use". But I can live with these considering the speed benefits.

The new problem is that now everything that Unity did -- sort, cull, toggle, offset, scale, etc. -- is gone. I now need to make it all myself.

Sorting

Most importantly, I can't just "render all sprites" in whatever order they are (unintentionally) stored. I can technically do that for tiles, roads, and tile overlays, because they never overlap and are exactly spaced. But every other entity must obey visual depth sorting. Simply put, a unit can walk in front and behind a building while still being in the same tile. But sending sprites to the GPU is fast precisely because it ignores such pre-processing details and just draws things sequentially. Just rendering naively would result in this:

a796ef34ab6204fc190447c92b3a6a019e8c8baf.png


Thankfully, my job is simpler than it could have been trying to sort 50k entities -- I already have a game tile hex grid. Every entity has a whole tile coordinate and I can hugely optimize sorting by looping through the tile coordinates in "visual order".

Entities also have in-tile fractional coordinates. So I have to loop through the entities in a tile back to front. For optimization purposes, I have to keep the entity list pre-sorted and add and update entities as their coordinates change. And this basically correctly sorts 90% of sprites.

The final consideration is that for entities at the same coordinate (like parts of the same building), I need an extra hard-coded depth-sorting value so they appear in the right order even though they are technically at the same location.

Sorting is probably the most time-consuming part to implement because I have to change and adjust so much code. Every one of these "considerations" is another layer of complexity and difficulty. And with all that, I am only now approaching the same visual results as I had at the beginning:

e8b7f11b0451e9c4a7c3c20c615d37e178205012.png


Culling

Another important consideration is to cull anything that is not visible, that is, literally not render sprites that are not on screen. It's a simple problem to explain, but deceptively difficult to implement efficiently. I cannot just check 50k items every frame. So I keep a list of entities per-coordinate, update the lists when entity coordinates change, and loop only through visible coordinates when rendering:

b2b420a3898b44be2596620f3e2717e0a9476947.png


The biggest consideration is that whatever code logic I do, I will be CPU bottle-necked and I cannot offload this work to GPU in any reasonable way.

Effects

All the small effects in the game were provided by Unity's particle system -- smoke, dirt, chips, sawdust, etc. It was all relatively easy to set up. And none of this works anymore with the custom rendering pipeline. Unity's particle system wasn't compatible with how I rendered regular sprites. So I had to recreate the effects myself -- all the logic, animation, sprites, visuals, previews, etc.

There are now some new optimization considerations. For example, various effects like digging dirt used the same basic logic where a 1-pixel particle would fly out. There would be some 12 particles, which equates to 12 sprites. This is actually quite a lot when you consider they have to follow all the new rendering/sorting logic I implemented. I now have to design effects paying attention to the number of sprites they produce and optimize when I can. For example, I can use only 4 sprites if I make an "animation" of the particles splitting up as they spread over time:

146dadd04262b5b2026764f9dee10971937e5efc.png


My favorite effect -- smoke, which I painstakingly recreated -- takes up 8 sprites:

a919c0cb042b571b840d11418b818a12571bdbb3.gif


Here, there's nothing I can do to reduce the number of sprites, and it will be slower than Unity's particles were originally. Of course, I am considering the big picture performance and presentation, and nice effects are definitely a worthy "expense".

Shadows

There is so much to talk about, because I am now revisiting 3 years worth of various visual features. But one of the cool easy-to-add changes was adding shadows for units:

f8f8e55f1b2b5583f82f51ddb41666b1ec4975af.png


These are very subtle and usually get lost in the "noise". But they subconsciously feel good. I couldn't add these before because they would get incorrectly rendered on top of other things - buildings, other units, etc. - because units would run all over the place and cause depth-sorting issues. However, now that I am "sending" sprites into layers for rendering, I converted all shadows into a "shadow layer", so they get drawn on the ground before any sortable objects, thus they are always on the bottom. This fixes a lot of shadow glitches I had, as well as lets me add shadows without worrying about problems like this.

In fact, I have a lot of new debug tools I had to make to visualize and find all the different parts. For example, a shadows-only preview:

c13b0d3aba66abb11547a573c29e0f224435243b.png


Fallback rendering

All this fancy shader stuff is great and all... except many older systems don't actually support it. The whole GPU-based compute shader processing is a relatively new concept. In fact, systems older than 3 years are increasingly unlikely to support it. Whereas even 15-year old machines could run the game before. Virtual machines also don't support advanced 3D accelerated graphics, which means I cannot test the game on macOS and Linux with the new rendering logic. This is also not something I can afford.

I did not enable analytics in the game, so I have no idea what systems MicroTown's players have. But the game has a strong retro vibe and I would not be surprised nor could I fault any players for trying it on an older system. So I have to support them. Which means implementing a fallback method for rendering everything.

(Un)fortunately, I cannot just fall back to my original Unity-based rendering, as I have essentially changed and rewritten everything. Thankfully, I have about 90% of actual logic reusable, because it's only the communication with the GPU that cannot utilize fancy data transfer logic. I have my list of sprites, their location, atlas information, etc. So "all" I need to do is replace the fancy shader stuff with a dumber, slower version.

Naively, I would basically have to send a mesh per sprite to the GPU one at a time. This is too slow (even with native batching), but I can also combine these meshes together. Which is also too slow with Unity's tools (but comparable to how fast the game ran before). So in the end, I am manually re-constructing meshes vertex-by-vertex to match the expected "sprite layout". It still makes things very CPU-bound, but it's still better than before. The biggest problem is just the human time it takes to get everything implemented and running smoothly.

Final words

Most of the time I was looking either at a complete mess like this:

d37a98c6483264a9bc09cf674cac2613676aafce.png


or this:

036dedfd2a89a0525f4aa17ba65a961e83ea1e2c.png


Or trying to fix one of the 100 different things that were glitchy:

40393d4df8524865412b835fdbb55865d8ca95a9.png


I even discovered 2D MicroTown:

0589325f01b8fed0b03643a7361def2f01b3543f.png


As a final thought, I only summarized the "correct" path to implement this. I spent a lot of time experimenting, agonizing why something doesn't work and getting exhausted by debugging. And this is besides Unity crashing every time it doesn't like something. The power of working directly with shaders is great and the results below speak for themselves, but it's also equally mind-numbingly tedious and disproportionately time-consuming. As a one-man team, I can't detour through such rabbit holes unless it's absolutely necessary.

So I'll end this with some benchmark comparisons. (I am not comparing FPS, because this isn't an accurate measure for games like MicroTown. The game doesn't have a fixed time step and fewer updates just means it has to process more changes.) These benchmarks are done on a 1440p resolution zoomed-out 160-size world with 2000 villagers drawing 25k entities (same as in the section on entities above) on 4-year old high-end hardware:

Method
Frame time
Render time
New
34 ms
6 ms
Fallback
54 ms
25 ms
Old
63 ms
35 ms


A full frame now takes 34 ms instead of 63 ms, which is almost twice as fast. Notably, specifically rendering now takes 6 ms instead of 35 ms before, which is more than 5x times faster! Even the fallback method takes "only" 25 ms. It's still only 30 FPS, but this is also an extreme case. I think I have squeezed out as much performance as I will be able to from rendering. But I could also easily add more entities and sprites without a significant performance hit. (In fact, I imagine most of the future optimization will be to the world update logic. In these examples, world update takes around 20 ms.) Of course, this will vary system by system, but there should be a significant improvement for all systems because the game is CPU-bound.

Future plans

I originally planned for "Shops and Houses" to be the next update (i.e. this update). This would have allowed Houses to use more items so that Shops can sell more items and then I could add more production chains and variety. But I only managed to get the Shops working, so the next main goal is getting Houses working. This will likely be a smaller update, but with bigger breaking changes. The Houses will likely also become 7-tile buildings.

Full changelog

Changes

• Market Square, Food Stall and Supply Stall are scheduled to be deprecated; they will remain functional for the current version, but will need to be replaced by Shops
• Optimized rendering requires compute shader support: at least DirectX 11, OpenGL 4.3, OpenGL ES 3.1, Vulkan or Metal
• Older system will fall back to a slower rendering method
• Add shop buildings: Grocery Shop, Produce Shop, Meat Shop, Clothes Shop, Supplies Shop, Medicine Shop with corresponding sold items
• Shop building HUD inspection to show full item "progress" (unlike market stalls)
• Add Shop concept
• Tooltips to show sold items at shops (similar to storage building tooltip)
• Export Station and large storage buildings will distribute items directly to nearby shops
• Market Square, Food Stall and Supply Stall are no longer constructible but will remain functioning as before in the world
• Rename "Fish Steaks" to Raw Fish
• Rename new game world size options from "small", "medium" and "large" to "tiny", "small" and "medium"
• Add "large" and "huge" new game world size options
• Fish Animal Habitats will no longer spawn far away from coast
• Remove tutorial steps for Market Square and Food Stall construction and replace with Produce Shop construction
• Adjust tooltips and explanations mentioning Market Square logic to instead describe Shops
• Animal Habitats no longer show tooltips on mouseover; instead Habitats can be selected in-world and show an inspection panel with the same information and changed display
• Add in-world markers for Animal Habitats animals
• Shadows now appear on the "tile layer" and don't overlap other objects
• Tile outlines, selection indicators and other tile overlays appear above roads
• Markers and various indicators now appear above in-world objects
• Mine, Sand Pit and Clay Pit prepared ground sprite now appears on the tile "layer"; adjust worker position and mining effect offset
• Villagers and animals now have shadows
• Add water splash effect to fisher casting and reeling animation
• Micropedia now has "Tutorials" section/list
• Item tooltips and Micropedia descriptions now combine storage entries to multiple buildings
• Add tooltip and description for Beer and Mead usage at Brewery
• Add amount label to (de)construction item progress for building and road inspection
• Add warning in main menu if compute shaders are not supported and the game will run a slower fallback rendering method
• Path-finding will discourage walking between buildings as much
• Units will no longer stick to exact hex tile centers when path-finding and choose straighter paths
• Workers with carts entering and exiting Export Station and Import Station will now move at a non-slowed down speed
• Hunting goal now counts skinned (but not shot/hunted) animals as part of the completion number
• Add internal "animals skinned" stat
• Roads to have separate sprites for vertically-mirrored bits and new dedicated sprites for the straight segments
• Add a more-expensive Boardwalk type of Roads that can built on otherwise unbuildable land tiles (Sand, Rocks, Clay)
• Add Boardwalks tech
• Add Boardwalk construction Goal
• Roads to have a new dedicated sprites for the three-way segments
• Animals now walk instead of running (internally, since visually these are the same speed and animation currently)
• Fish move faster to bait/hook
• Idle villagers will walk more instead of running, especially for short distance
• Adjust Import Stations item positioning so all stacks fit on the same tile
• Import Stations will also distribute their items to Food Stalls and Supply Stalls
• Large Warehouses and Large Granaries will also distribute their items to Food Stalls and Supply Stalls
• Add issue warning for large storage buildings that have dedicated workers but no compatible item sources or targets
• Large storage buildings to mention how workers operate in the description and change range explanation label for construction

Fixes

• Micropedia search bar results would disappear when clicked before the click is processed and would not navigate to the clicked entry
• Raw Fish item distribution proportion unintentionally defaulting to weight of 1 instead of 3
• Oven operation progress not showing correctly when the worker is activating the cooking process
• Export Station and Import Station not showing all item delivery marker icons
• Florist missing tooltip/Micropedia information line about Flower "production" from Seedlings
• Brewing Vat not showing cooking sprite and smoke
• Building workers still accepting tasks from far away blocking other workers from and choosing them until they arrive
• Building worker with no task going to the building from far away even when the slot is disabled
• Building construction and deconstruction markers now appear on the tile "layer" and don't overlap other objects
• Mouseover tile would occasionally be momentarily incorrectly calculated
• Various sprite fixes
• Path-finding will avoid leading units through single-tile buildings
• Fix path-finding not considering "exiting" a building tile in a proper direction
• Path-finding will prioritize "front" tiles for multi-tile buildings and avoid going through "occupied" building tiles
• Path-finding will again avoid leading units through fields/plots/pens, but take corners instead
• Animals will properly ignore pathing restrictions and walk in pens as desired
• Fix path-finding choosing sub-optimal shortcuts
• Fix villager slowdowns when running between certain road tiles, most noticeable with carts
• Animal Habitats not spawning
• Graph window not working when loading previous version saves
• Acquiring items not present in loaded saves causing an exception in internal graph logic
• Export Stations sometimes not delivering items when one of its target Import Stations has a full stack of that item
• Technology panel buttons having mouseover detection region larger than the button itself
• Skinned animal stat not being recorded in saves
• Animal Habitats not despawning when reaching a low score
• Fix large storage building worker incorrect waiting locations
• Import Stations and large storage buildings would not fill up compatible building items fully
• Fix exception when loaded saves would fail to assign unit idle locations
• Remaining items not appearing in the world when extracted from stacks
• Internal ID conflicts between units (namely Pigs that are led to Butcher) and items would cause one or the other to be excluded from future deliveries for the ongoing session
• Import Station issue verification causing internal exceptions while it's constructing
• Large storage building issue verification incorrectly triggering when there are compatible item source buildings in range
• Building issue verification for required auxiliaries would incorrectly show double-issue for buildings with multiple auxiliaries
• Building issue verification for issues that have a building of item list to report would not update the tooltip if these lists changed

Balancing

• Garden Plots, Crop Fields, Tree Nurseries and Animal Pens now cost 2 Planks instead of 3 (and 1 Stone Slab)
• Import Stations can now distribute up to 6 items to nearby targets
• Large storage buildings can now distribute up to 6 items to nearby targets
• Double Large Barn and Large Granary capacity
• Increase Forester operation range from 3 to 4 to match Lumberjack operation range

Optimizations

• Redo rendering pipeline and significantly reduce CPU usage and overhead
• Most visual-related logic has different performance cost, usually reduced
• Small, emptier worlds run slightly slower, but large worlds run significantly faster
• Increased memory usage
• All particle effects are redone and slightly different due to using a new system
• World entering is slightly faster
• Reduce redundant unit internal animation processing
• Speed up villager idling location lookup
• Entering (generating, loading) and exiting a world is slightly slower due to the pooled objects now having inefficient hierarchy for mass changes
• Processing (pathfinding and delivery processing) threads sleep longer and more frequently when idle freeing CPU usage
• Internal game logic is now capped at 90 FPS, so faster machines do not do needless processing
• Slightly fewer HUD rendering batches
• Building issue periodic checking is faster
• Many minor optimizations

Rudy
Snowy Ash Games
 

Maxie

Wholesome Chungus
Patron
Glory to Ukraine
Joined
Nov 13, 2021
Messages
6,753
Location
Grantham, UK
In case you're into that, our man's sperging out about map generators
EA Update #24 - Map Generator 1/3
This update replaces the map generator with a new version.
This update replaces the game's island map generator with a new more robust and expandable version.

ddfa094cd6affd54e6c6dcb1d70588f74b232d34.png


I had two primary goals for the new level generator. Firstly, to bring it up to at least the same level as the old generator - but do this with a modular, extendable and configurable approach. And secondly, to fix the biggest problems with the old one - excessive unusable land, poor scaling with world sizes, bad coastal shapes and many minor issues.

There is a lot to talk about designing a complex system like this. I won't go into a huge essay about my software architecture here. There are many reasons why I'm doing the things I'm doing even if they oftentimes feel like over-engineering convoluted solutions to simple problems. But I will go over the most important parts.

Due to Steam post limit, I had to split this up into three parts. This post will focus on the overview and general approach, and the goal of the generator. The other two parts will go through specific generation steps: Post #2 and Post #3.

Iteration
The hardest part of making a complex parametrized system is testing and iterating its results. The faster I can preview my changes and the easier I can understand the impact of these changes, the faster I can work on it and by extension produce better results.

This is my best friend:

a03ae29b3e9467e444967ea348e1b28fb47fe2fd.png


To accomplish fast iteration, the system has to be designed with this in mind from the start.

A map generator makes hundreds of decisions when building a map from high-level choices (for example, the number of biomes) to specific low-level choices (for example, number of trees per tile). Usually, these are encoded as direct and indirect parameters that I can quickly tweak to achieve the desired results.

The difficulty comes from having lots of parameters and wanting to understand their impact at the same time, which means previewing the difference they make to the map generation. And this is a difficult problem because any changes that happen after the relevant ones can obscure the results. Most parameters have a huge range of possible values and only a narrow band of value produce desirable (or even usable) results.

So I want to preview not just the final result, but every step of the way. To do that, I am splitting the generator's logic into discrete steps or "passes" so that I can look at the independent output(s) of any pass in the sequence.

It is also critically important that the result preview is visual. This may sound like an obvious criterion, but implementing clear and useful previews takes significant additional time and effort. It is thus a longterm strategy, which means it has all the pitfalls of additional code to implement and maintain.

a97ad645e22934e612d5e975d8f327bc5397c890.png


Another important consideration is coding the generator, not just running it. The last thing I want is to break everything by making changes. Implementing separate passes will take more time but not nearly as much time as having to rewrite large parts of the code if I interconnect everything. If I mess up some pass, then I will likely only break its inputs and outputs to the previous and next passes. And realistically I am almost guaranteed to rewrite, remove, add or majorly change many of the passes because I simply don't know how exactly they will work.

And one final and often overlooked benefit is personal motivation. Actually seeing the system work after every change is a huge boost to my motivation. It's the difference between just making something work and investing that extra polishing time. In a large system like this, small details quickly compound.

Passes
I would like to describe every step the generator takes, because I personally find it all very interesting and I could talk about it for hours. Unfortunately, this will end up with a hundred pages long post (and about 20 times over the Steam post limit). So I will try to summarize the most important steps and parts. And even with me being "brief", it is still about 3 times over the post limit. So as I'm splitting the update into three posts - this one (broad overview) and two step-by-step overviews of the generation process: Post #2 and Post #3.

Scaling
One thing I barely considered for the old generator is scaling parameters with the generated world size. This is where the huge sand areas of the old generator in the large maps come from. And this is actually a much more difficult problem than it seems, which needs to be "baked into" logic from the start. Every variable that controls something size-dependent for the generator must decide how it behaves with different map sizes and I need to be able to quickly control it. In fact, there must actually be all the variables to control size-dependent features (for example, I cannot scale parts of a noise map without scaling everything). For example, I would want to specify that a beach is 2 tiles wide or 5% of map size, or 3% of map area, etc. Choosing an incorrect value or approach might work fine for the reference size, but may produce terrible result for different sizes.

So when specifying parameters, I make sure to specify (and preview) how they scale:

2a3794cd6adda0a6d0183685bc4b2e987be1bb87.png


Generally, the options are "static", "linear" and "exponential" (with inverse variants). Static mean the same value for all sizes - e.g. number of spawn locations. Linear means it scales with the map size (width) - e.g. allowed water edge distance. And exponential means it scales with the map area - e.g. number of stone deposits.

And when testing the generator, I can run any combination to verify that they all work correctly:

af8e13f05428a607a8566802d0f62a0f06d447c1.png


Major changes
As mentioned, this update is not meant to majorly change the gameplay beyond (A) fixing and adjusting most problems with the old generator and (B) those changes that are best done now to avoid too much disruption later. So there are still a few bigger changes that do impact gameplay:

Rocks no longer spawn all over the place, but rather in discrete patches focused towards one area of the map (this preview is from the tiny map, because I cannot get a better screenshot from a large map to demonstrate the idea clearly):

3a706214fadbe49004c411102738008c938eacbf.png


Furthermore, rocks now come in 4 variants - Rocks, Coal Vein, Iron Ore Vein and Limestone Vein, with their respective deposits only spawning there:

2415ab347f1ac8f2d04381ca2d99260d951526a3.png


Consequently, the Mines now need to be placed on the appropriate type of rock:

5dee779649c58a82bb7813b6cb8fb144b130fde9.png


(That is one chonker of a tooltip... I haven't really come up with a way to shorten it either other than having 8 separate lines.)

Sand and Beach are now different tiles. Minable and usable Sand now spawn in discrete clumpy patches. Beaches spawn along the coast (sometimes in clumps) and they cannot be mined.

c19a833a038258d27947b7b8e3c4973051e713e8.png


They look a bit empty (again, because I did not want to expand the scope of the update), but there is potential to add something here.

Overall, the island shape has somewhat changed. The generator attempts to "fit a hex shape" into the game area, with a roughly-expected distance to edges, but still some larger coastal variation (I explain the various constraints in the other posts). A more "extreme" example of an island that is basically a hex:

3252e31ae74b03278b28ddb9326e8e0fdb7043ab.png


The coastline of the island is also much cleaner. There are no extended peninsulas or closed bays - the vast majority of the map should be accessible. But there can still be significant variation of the coastline. Again, a more "extreme" example with lots of "shapeness":

90f2a7ebffc545ad66bd0db2a3f46b4b5601c39f.png


Most islands end up somewhere in between the two, leaning towards more varied coast.

I also added a new grass type for some visual variety and as a proof of concept that I can vary vegetation:

f75d7eae1bc75e5843a6e8d1a2a649436e585f53.png


Which kind of lead to discussing the possibilities...

Potential
One of the main goals of the new level generator was to be able to extend it in linear time. In other words, add new stuff quickly. This means there are a lot of things I can now add to the map that I previously couldn't. Some are very easy, some are more difficult, some are impractical. For example, I won't be making multiple islands (impractical), but I can easily add more biomes (easy) or cool features like a river or two (difficult). The great thing is that I am not limited to any particular single algorithm that has to somehow produce all the results in bulk - the generator consists of many individual steps and I have a lot of freedom to add and modify those.

A particular direction I want is to specialize and utilize different parts of the map, so that the terrain shapes the village more than it currently does (i.e. not at all). For example, you might have a swamp in one end that is needed for medicinal plants or flood plains in another part, where crops grow best, and so on. Following this, I might be able to force some production chains to actually need transportation from their production location to their processing location rather than having everything clumped together.

Another big potential feature is allowing player customization for the generated map. For example, forest density, deposit amount, climate bias, etc. I haven't implemented any of this, but I have kept this in the back of my mind when designing systems. I basically comes down to having user values selecting or adjusting one or more generation parameters.

Finally, I could potentially add--if not full replayability with different climate zones and resources--then at least enough differences between maps and their layout to not all look the same.

So this is all something that I'm looking forward to (admittedly, I am also exhausted having spent so much time on the generator.) I won't even speculate on what I want to add next, because there are just so many possibilities. In fact, deciding what to add and--more importantly--what not to add is in itself a big design decision/direction.

Unity engine
I am currently using Unity engine for the game. You my have heard about the recent Unity trust debacle. I won't go into details here, but the main point is that I do not wish to continue using the engine. Unfortunately, MicroTown is many years into development and strongly tied with the engine's features. This makes porting to a different engine very difficult and time consuming. I don't know if I can realistically invest that much time into this. I spent a couple weeks investigating engines and experimenting to see what such a transition would require. For now, I am still undecided. But it's likely I'll have to stay with the current engine.

Future plans
I had to really stop myself from starting to add more features to the level generator. It has already turned out to be the largest "feature" that I have ever worked on. And the final touches and fixes again took way longer than I anticipated.

I wish I could have split this update into multiple parts over past year or so. But unfortunately this is one of those "all or nothing" features. I cannot implement half the generator or even 90% of the generator - it has to be all of it. So updates have been, shall we say, a bit slow (in addition to some other real-life stuff). So I tried to work on it 50/50 mixed with other features/updates, but that hasn't really been a good approach. I will definitely avoid splitting major workloads like that in the future.

So, hopefully, I will be making several "terrain updates" adding something to the generator and either adding or modifying something in related gameplay.

Full changelog
Changes

• Implemented new level generator algorithm
• Overall terrain shape is now properly based on a hexagonal shape, but there is more coastal variation and most landmass is slightly further from the edge of the map, but with more variation
• Terrain is now constructed in a more cell-like pattern, where "cells" correspond to biomes/regions, such as meadows or forests. Trees and other props will have way fewer unnatural "stretches" and "clumps". These "regions" will tend to be more uniform and connect more naturally.
• Islands overall shape will avoid generating with no coastline variation or excessive water or land
• Island now has several larger "climate" or "biome" sections that somewhat alter its layout and features - a part of the island has more forests, a part has more plains and a part has more rocks along with other slight vegetation biases.
• Deep Water to Shallow Water transitions are now smoother
• Water will not have excessive "patches" anymore
• Coastline is less jagged now and fewer extended unusable peninsulas or bays will spawn
• Add Switchgrass, a type of grass that spawns in Grassland
• Rename "Flower" to Marigold
• Add generic Flower concept
• Vegetation (Tree layout, types, tile variation, and Shrubbery/Marigold) layout are all now based on a separate noise patterns to avoid unnatural patterning
• Add new tile Beach that replaces Sand along the coastline, not suitable for quarrying
• Coastal tiles have much less Sand (that is, fewer Beach tiles) now
• Sand, Clay and Rocks now spawn in dedicated patches and cover much less of the terrain
• Patches of Rocks, Clay and Sand won't spawn next to similar patches anymore
• Rocks patches will spawn more in "rocky biome" and will generally include all the ores in that region (although they will still spawn randomly elsewhere)
• Clay patches now only spawn further inland
• Sand patches only spawn closer to coastline
• Several larger Beach locations now spawn randomly along the coast
• Rocks now have resource vein variants - Coal Vein, Iron Ore Vein and Limestone Vein (and their excavated variants - Excavated Coal Vein, Excavated Iron Ore Vein and Excavated Limestone Vein)
• Coal Mine, Iron Ore Mine, Lime Mine can now only be built on their resource tile variant, while Stone Mine can be built on plain Rocks
• Spawn point is now randomly offset from the middle of the map
• Spawn region is now clear of obstructing terrain
• Spawn region will always have some free-standing rocks around it
• Spawn region will only be set in "regular" climate avoiding inconvenient terrain for starting
• Patches with resources will avoid spawning in poorly-accessible unbuildable locations, like surrounded by water or unbuildable terrain
• Animal Habitats now spawn based on distance to coast rather than distance to map edge/center making for more uniform spawning and avoiding unreachable locations
• Animal Habitats now pre-spawn properly instead of appearing one at a time after the world starts
• All the internal properties of level generation now scale with the world size appropriate to their purpose. For example, larger worlds do not get larger beaches or excessive rock patches while smaller worlds still contain sufficient rock patches. Terrain features, like biomes stay proportionally-sized.
• World generation will no longer stall UI (some other operations will also have fewer UI stalling portions)
• Older saves with Mines will have their tiles converted to respective resource tile
• Older saves will have empty stone patches randomly converted to resource patches
• Add worker self-delivery max distance indicator "circle" when building or selecting buildings
• Add the worker self-delivery max distance value to several relevant tooltips
• Item deliveries during carrier shortage will allow self-deliveries, ignoring regular priorities and proportions, but not not stalling/failing to deliver
• HUD delivery info tooltip will also show self-delivery count

Fixes

• Selecting Observer would not show the empty icons above observable building until they had done at least some work
• Fix worker self-deliveries not working above dedicated carrier max delivery distance
• Fix workers delivering 2 or 3 items of their building's output to storage removing/replacing them from the building as worker
• HUD delivery info tooltip showing the wrong number of available (idle) carriers and some other slight inconsistencies due to self-deliveries
• Fix props attempting to transform while a worker is using them, for example a sapling rotting while being harvested by an arborist (and this invalid combo causing games saved at this moment to fail to load)

Balancing

• New terrain generation alters various gameplay elements

Rudy
Snowy Ash Games

EA Update #24 - Map Generator 2/3
This the second of three update posts about the new level generator. The introduction is at Post #1 and the third/last part is at Post #3.

I will now go into step-by-step process of the generation passes. For clarity, I will try to explain things in the way that makes most sense rather than sticking to exact technical pass layout. I'll try to group sections by generation "topic", although there is a lot of overlap.

Clusters
Previously, the generated terrain was based purely on a noise map (Perlin noise). This created decent-looking gradual layouts, but this did not let me actually change anything in absolute values. That is, I could adjust things relatively - for example, have sand at 5% threshold, but then it would be random amount of sand up to 5% on any size map, which might be terrible on huge maps. There was no way for me to say - I want
only
3 "patches" of sand, because there was no concept of a "patch" or "scale".

So the first and main terrain step is to produce discrete "clusters" (based on the map size). First, I generate a bunch of random (shapeless) points:

6350d0a811a7c6f30bb7ed3dd6f0672104340bed.png


Then I tessellate these (basic Voronoi diagram) into individual (shaped) clusters:

de13a9d31f658e8573034e8cd428d6496b48e40c.png


Due to randomness, some of these often end up being too large or too small, so I run an extra smoothing/averaging pass (Lloyd's relaxation):

c982fe5f31dda0a268fbc2190cac84d961a54e60.png


This produces a nicely distributed and organically-looking pattern (kind of like cells). Jumping way ahead, this is what will give the terrain the "clustered" look I'm going for (warning: now you will never unsee it):

0f01aedd780b3e373df87b9cbbcbca899f82fcce.png


Water
The next big thing that I generate is the separation between water and not-water by heightmap, which is a convoluted way of saying that I'm building a coastline. This is where a noise map like Perlin noise works really well, so that's what I'm using:

6af07eca73f34b3b7b4ea03b21ddcf4c9129d07f.png


Now, I cannot directly apply this to clusters, because the edges need to be water and the middle needs to be the island mass - by design. Previously, I was using a squashed circle on top of the height map to "cut out" the rough shape, but this created a lot of water in some directions and too much land in others. So now I implemented a much more accurate hex shape with proper "distance to edge" calculations:

d5126130126cd8239e5e9e14b258acd2ffc8a018.png


I can then overlay this shape over the actual heightmap, reducing or increasing the random value to force it below or above certain thresholds:

d344353d81b5369f6caaaeed630da2679da2df4b.png


This makes sure that anything outside the "big" hex becomes water and anything inside the "small" hex becomes land. The in-between area is then where the noise actually affects the shape. With some visual experimentation in later passes, I can tweak the noise parameters to produce satisfactory results.

I can then apply the "clamped" height map values to the clusters themselves and decide which clusters become coastal shallow water and deep water:

439db1e87b44c50e3ad0976e9bb2412a3d015aa0.png


Of course, due to the very nature of applying thresholds to randomness over an area, I sometimes end up with little inland pond and offshore islets. For gameplay purposes (and all the issues these cause), I am removing these when I find them:

9326388da198115a5d853e47f9d46f41d27f951b.png

5d23569a0dbcc362684448a8204ed99f3ede455c.png


And this is just the start of the "battle" against randomness "ruining" my islands. I'll talk about this more a little later.

Island shape
There is no perfect way to generate an island shape. Perhaps there are more suitable algorithms just for the island shape generation, but I also have all the other generator's parts to consider and they have to all be compatible. So instead of seeking some perfect algorithm, I'm just brute-forcing the generation a bunch of times until it doesn't get rejected. Some passes are there purely to reject the result and tell the generator to restart.

One such pass is for, broadly, "island shape". I generate a few "control points" in a rough approximate ellipse around the island and check if they are land or water:

8aeae60f8887b0635ef01721c3031f0baf4b7bdd.png


If there's too much water, too much land or too much repetition, I reject the shape. Otherwise, it's probably interesting enough to keep. There is no science behind this - I was just looking at hundreds of islands until I came up with heuristics and parameters that struck a good balance between random and still somewhat hex-like shaped.

This is a good time to reflect on just how many maps I generate throughout the process:

07f8a206fe364e24b97f694ba85b490ad23e0390.png


I have set up my generator to support output export at every step, so that I can leave it running for many iterations and then check them for any systematic problems. This is not something I could do before, and this is an invaluable methodology for designing and implementing a system like this.

Climate
Among many experiments to make different regions of an island feel more varied, I settled on a "climate" concept, which flags clusters as belonging to a certain climate (to a certain amount) and other passes can use this data to alter their generation. So this doesn't do anything by itself, but it provides additional customization for later passes. It's kind of like biomes, but currently there isn't any biome-specific terrain, so I'm calling it "climate".

For start, I select a few "control points" in a circle around the map (I chose this shape for simplicity and because the maps aren't large enough to come up with complicated layouts). Each climate has a area it can influence:

900537ec35d40ff4493c0e82b775a24439011baf.png


Then several climates are assigned randomly, specifically 2 forest, 2 plains and 1 rocks (and the rest stay default):

d6f000b5457be2ca408b1274e241de66e856273c.png


These are just thematic names that describe my intentions. The climate regions then flag clusters within their influence with an appropriate climate value/weight for later use:

d6f000b5457be2ca408b1274e241de66e856273c.png


Again, jumping way ahead, slight variations is later passes would create noticeable differences:

2149de10cf78addb5a5720a1df293c6c1b63a7bc.png


Spawn
Previously, I placed the starting area or spawn in the middle of the island and manually cleared it from obstructions and non-buildable tiles. And while this works, it's implemented backwards. Instead, what I really want is to decide on the spawn location and then make sure I don't obstruct it with anything and place the relevant things around it.

So now I pick a random location on the island within a spawn selection area torus:

01ea801022f7929c38e715614b5bdead1db9fb69.png


From there, I pick a valid (for example, not too close to water) closest cluster and construct a spawn area (this uses the same logic as regions, which I will describe later):

0a48badc63b552f0fe79ed2e72f5de7c874b0dea.png


This creates one "main" point for the spawn and "grabs" nearby clusters to expand the area to a specified desired size. In addition, I place several "meadow rocks" areas, so I can spawn starting stone deposits around but not on top of the spawn area.

Finally, I create a small spawn exclusion zone, where many things can generate normally, but it will mostly remain free of obstruction:

24ece4a24ad711663ba4b099773770f4a161dbb7.png


While this may sound simple, it's extremely easy to get wrong and took me enough experimentation to make a working version. There was a good reason why I previously kept the spawn centered and it was to avoid the many issues of random placement of potentially-overlapping terrain features.

Regions
Most of the map is generated in two big parts - discrete features like rocks and clay patches and continuously semi-random ones like forests or flowers. Regions are for preset discrete locations, so they have to be generated first before everything else is "filled in".

It is very difficult to break down region generation into multiple passes, because they are basically all identical except their rulesets. And separating these rulesets is only useful for visualization (which would take huge amounts of work, so I didn't really do it). So what I end up is producing the region map in a single pass:

667f25afaff3375c9e00402e9c28cfdd331fec90.png


Generally speaking, regions are groups of neighboring clusters that form an area for some purpose. For example, there could be a 4 cluster region for coal rocks. Each type of region has a long list of hand-crafted rules and requirements, mostly derived from experimenting. Plus, most of these have some sort of rules about distances in relation to each other. So it makes more sense to just see the results for all at once.

For example, here is a ruleset for placing sand regions:

ba692d481cfc443ea89a3880428613bf13e12fdd.png


What this says is that it will place 2-3 region with 2 clusters semi-randomly; it won't place two regions close to each other and will place regions close to but not next to the coastline.

There are many more rules and nuances I could add, but it's best to keep it simple - more parameters just means more work adjusting them and debugging issues. And there are already 12 different region groups just to match the basic level generation goals.

There are some notable rules though worth mentioning. For instance, some regions are much more likely to generate in certain climates. Notably, all rocks-based minable regions are more likely to appear in the rocks climate:

31cf9c7e3a72c675dcacd3bef958121f02d5e974.png


Jumping ahead to final generation, this would allow the terrain to shape village direction and create a part of the map that would naturally focus on mining and industry:

650a15ceeebfedef56b256f67e66b2f80393eab0.png


This is not something I could have influenced like this before and is only possible because I can "direct" such discretely-selected regions towards certain results I'm looking for.

Another example is a beach region (in addition to regular thin noise-based beaches discussed later), which randomly generates along the coast in a few locations:

9b81f909dcafad27c7b8268f392cc650ed44f6a7.png


This adds variety to the map without covering half the map in sand, like it used to do before.

I should note that while this is how I primarily use the regions for now, they do not specifically just control tiles (grass, sand, rocks, whatever), rather they flag the clusters as belonging to that region. In fact, the number of tiles that later get converted to their "region designation" is more strict than "cover the whole region". So regions in that sense more of an abstract layer informing later generation what to select but not necessarily exactly how. More like "this is a good place for X".

Filler regions
The second type of regions are the "filler" regions that, as the name implies, fill out the rest of the clusters that aren't already designated for something. Because this doesn't require any discrete counts, this can be done with a noise map. I'm using several slightly different noise maps for different purposes, specifically tree cover, tree types, plant type and tile variation. It's also fairly simple to add new variants and then use them in later passes.

For example, here is (part of) the tree cover noise:

ebc5c30ab61d45065ff0b15a3c68b1c252bf5bec.png


This assigns a "tree cover value" to each cluster (point):

782ade4d0dabccfd918f583af4dd39843f7f9a48.png


Then a bias to increase or decrease the noise value is applied based on the climate it's in:

ed5899d6bc0db9782b08be78b4f32b7b19cc0361.png


These are small adjustments that each climate provides, but they can create a significant variation for each "filler":

a6e1efd25f89a28eb00eb00e7beb592e027fb9e4.png


Now with the noise values decided, empty clusters can decide what sort of content they want. For example, tree cover decides where the forests will be located:

7145104fecf84219cc4ba2dec52cc7a5af9adb14.png


These noise maps will be further used in later passes for relevant selections. The above selection was more or less binary, but there are many more things that can be decided based on the full range of the random value.

And more!
I'll continue describing the remaining steps in the next, but this seems like a good time to reflect upon all the things that I am not doing. As I mentioned, it's important to set myself some limits on what I'm implementing, otherwise I will never finish this. I am certainly thinking about a lot of cool things that I could do.

As an example, I used the cluster layout and the Voronoi diagram edge connectivity to build a cluster "height map", spawn a "lake" region and then build a river running from the lake to the nearest coast:

416b66cc11edc80f73a66a7f2f7822215843469e.png


Of course, this is far from something that I can translate into tiles and gameplay, so it's more of a proof of concept. But I thought to mention this just to give an idea of the possibilities.

Next post: Post #3

EA Update #24 - Map Generator 3/3
This the third/last of three update posts about the new level generator. The introduction is at Post #1 and the second partis at Post #2.

At this point we have something resembling a map:

ac2214ac2a70a65015be04ab98e88fc6d9e91ec1.png


It's probably fair to say that the generation logic so far is already complex. But this is only perhaps third of the way to the final map implementation-wise. As more passes work with the data, they make smaller and more specialized changes. There aren't too many steps left, but they are generally an order of magnitude more complex in implementation than most preceding steps. This is also why I attempt to get as much top-level decision making done as soon as possible, so that the later passes work with smaller and more specific parts of the map without making any significant changes to the overall direction.

Tiles
The next big transition between the two generation parts is to actually create a tile layout on top of the clusters. This is quite literally how it works:

fbf12376a18b8fadd8608a35e968afa9358f8bc5.png


The main change is that the tiles actually correspond to the in-game tile hex layout (and all the math from here is in hex grid). It's also important to note that they are much denser than the underlying clusters, so all operation are way more fine-grained:

067e192c149929f3e627e9b73e1a09c0357ed9b3.png


In other words, a single cluster is represented by a collection of tiles. This is how the cell-like structure of the clusters gets preserved to the final version.

The tiles that get selected (at least, the initial version) pretty much just match the cluster they are on:

f1a2fac81215ce2909ef372da76d25d2f4627b92.png


While it was not really productive to design the main shape of the island using tiles are the basic "unit", it's important to start seeing individual tiles once the main shapes are determined, so going forward all changes will be done to tiles (directly and indirectly):

f65e766a3af1c90694988487af1c55df83a04da7.png


The downside of using purely random points to determine the clusters is that sometimes they are just too small or too large when they get converted to tiles. And there's no good way to handle this generally. These are basically statistical outliers, so after "placing" regions into tiles, I also either trim or expand them to a maximum or minimum number of allowed tiles:

461549105e2d5f045bfffc5c8bfc40ac0649d380.png


The perfectionist in me detests this kind of bodging. So this may be a good time to reflect on this.

Randomness
Randomness is always battling with order. As a concrete example, to create interesting rocky patch shapes I have to allow the random values to be random enough that these shapes get generated. I cannot restrict it too much or I will end up generating the same blob over and over. The problem of course is that random generation has no concept of subjective values like "interesting" or "balanced" or "doesn't break pathfinding horribly". To put it another way, the more cool stuff I let in, the more bad stuff sneaks in. Consequently, the more freedom I allow, the more rules I have to write.

There are several ways to ensure that the generated result is compatible with my goals. Barring just hard-coding all the values, I broadly label these "threshold", "fix", "retry" and "accept". This might sound like nitpicking (and I certainly didn't foresee myself writing an essay on randomness philosophy). But given that I have to make dozens if not hundreds of decisions as to how I will handle each such value, there emerge several patterns.

"Threshold" means setting a strict limit to the possible output of random generation. It may be strong bias, it may be a preset range, it may be a hard limit that cannot statistically be outside the designed range. For example, the middle of the island is always solid land - the threshold for land simply exceeds the noise values that are clamped to the hex shape there. The downside is that such thresholds are often fairly restrictive and I may miss out on interesting outputs simply because I deny too many.

"Fix" is the consequence of letting a lot of randomness through. Subpar outputs and awesome outputs are different ends of the same spectrum. If I aim to preserve the good bits, I have to be prepared to handle the bad bits. Besides reading like a fortune cookie, this just means applying certain fixes to expected problems, like the above example of under/over-sized regions. There is a downside though - I have to actually implement it, test it and forever maintain and adjust it with the rest of the generator, which is all the more complex because of it. This is also an excellent reason to keep such fixes self-contained in separate passes and easily reviewable.

"Retry" means literally that - just run it again. There are two general cases when I would do this. Firstly, if something breaks or I cannot generate something. For example, I might want to place a sand patch 5 tiles away from all the clay patches, but I just happened to generate clay so uniformly that there's no place on the map to place sand now. Secondly, when I don't like the result. Like the earlier example with the island shape - I might hard-code some subjective rules, filters or conditions. As easy as this solution is, it's also dangerous to start rejecting too often rather than have better rules. It's also too easy to filter too much and not give randomness a chance to produce neat results. And with a complex generator like this, one must consider that it becomes computationally costly - I just cannot retry things hundreds of times, especially during later passes.

And finally "accept". Sometimes I just have to give up. The result will never be without fault. That's statistically impossible. The good news is that this is actually mitigated by the layered complexity of the generator as well as high error tolerance of the game itself. It's actually surprisingly difficult to generate a map that's unplayable. Even a tiny map that can barely fit anything and fails half the checks is perfectly playable (and I didn't even cherry-pick the example, I just ran the game once):

c2fa341abdee47c944d333e945f1e68fde78d744.png


Shape cleanup
Similarly to clusters, tiles may end up with small ponds and islets because cluster connections do not always translate to tiles perfectly and it's occasionally possible to create "gaps" between clusters when represented as tiles:

ce223d80f463e3e6adb7e6dd58e913aa8c152553.png


These are really rare and they also don't really matter for other non-water clusters. For example, a gap between a forest in the final layout just makes it more varied, but doesn't cause any gameplay issues:

18af4ca346a38cf9acf68af18b95991d0db90dd2.png


A bunch of logic depends on coast distance, so I calculate each tile's distance to water and land:

68b9f466ade067994e8ea9410ab27864f5d257c0.png


Previously, I often encountered islands with annoying unusable coastal extrusions (long peninsulas) that mostly ended up as sand anyway. Similarly, there often appeared long water "tunnels" into land. I still occasionally get these:

b70575cb3599ac16d386c9c1fe7285b6e53aa304.png


However, now I attempt to find "landmass" and "water mass" areas, which are basically tiles that are deep inland or far from coast. This identifies any regions that are only connected by narrow strips of land/water:

4da573f4427879b995573c38aabe729b752bc61f.png


As these are undesirable, I reject these results and retry the whole generation. It's fine for there to be extrusions like this, but they have to be "reasonable" and ensuring "wide connectivity" is the approach I found best to preserve enough interesting but not broken shapes:

d9518e80625b088590cc60e2dceb217e1ca94d78.png


Another common issue is an undesirable proportion of land-to-water ratio. Some shapes end up being too large (basically a hex), others too small (boring blob in center). So I calculate how much water there is that's not close to the coast and reject shapes with bad proportions:

54485ed3a6a74e673ff7efcc1bab32e0c5689824.png


A similar common issue is the whole shape being "offset" from the center so that some coastlines are really far from the "world edge", making for huge undesirable water stretches. I reject these shapes as well:

595af9053fea71f53b0ec192adc522219b769b80.png


Coastal cleanup
The height map random noise does a decent job of creating shallow and deep water around the island, but it is also prone to creating excessive shallow water. Since I cannot "fix" it during the initial noise pass (I would also affect how much land generates), I manually trim away shallow water too far from land (convert to deep water):

3fb965af27d74ef085041a7fd50b1ab41ac267a8.png


This basically makes deep water transitions more "aggressive". This is also mostly a result of visually confirming what sort of shallow water amounts and variations look goods.

Similarly, sometime the noise transitions are so "steep" that deep water ends up touching the coast, so I offset any deep water too close to the coastline (although I fine with there being only a little gap):

ca9aa105becf919cd6fa3aac13de686dfe8d8ffa.png


Finally, all this fiddling with water level may end up shallow offshore patches. I did consider leaving them in as small cool variations in the ocean, but they almost never actually look good because they are just extreme outliers of the height map noise. So I basically remove them:

fb6781788cbf0655a9e63a50c9b656dafaf81a2f.png


Previous height map also controlled the beach elevation. That is, I spawned beaches based on the height map, which was based on the map size. This created really huge sand areas around the coastline, especially where the coastline wasn't regular. So far, I have completely removed any sand at the coastline except the specially designated beach regions. But I do actually want some narrow beach tiles to randomly appear. So I'm using some noise values to randomly change coastal tiles to sand but only with specific thresholds to avoid any excess:

a126347e94f04f50e2787d862049a27b022f06a8.png


Despeckle
A common issue with the resulting tile layout is that they sometimes have "sharp" edges - tiles that are almost completely surrounded by another tile type. This just doesn't look good in the final version, so I added a pass to trim these to match their surroundings:

40c7b2131802adbdb015ae412bdbba40de25cc53.png


With several iterations, this cleans up any unnatural looking extrusions.

A special case of this are coasts that may end up with micro-extrusions of single-tile peninsulas. I really don't like these in the final resulting tile layout, so I trim these:

86214c50c482d001368085fe257b9be3deb8b854.png


Technically, this trimming happens before all the water coastline logic because I cannot modify water/land tiles once I have established the distances. (It was a "fun" bug to chase that caused the generator to crash every 1000th run or so.)

Tile variation
At this point I assign tile variation to the map based on a noise map:

8b467c3dddd5205b6204d6fe9d985e57d7959b00.png


Currently, this only affects grassland tiles (and technically forest tiles that may revert back to grassland). Compared to previous generation, these patches and transitions are a less messy shape and don't overlap contrasting variants as much. Similar to other noise maps, climate affects these for some extra variation between "biomes":

448ea1e8cb59eba4b5065f2ea3e9dd267cf87fa0.png


Previously, I used the same height noise map for these as all the other features and while it looked fine, it was also very obvious purely random and not following any sort of natural "land" layout.

This might seem like a fairly trivial decorative feature to discuss in such detail, but the map looks really ugly with all the grass tiles being the same variant, especially after seeing the varied background for so long. Seeing such a huge difference made by such a small variation makes me consider all the other places I can vary in minor ways but to great effect.

Fillers
Finally, I get to filling the contents of regions. This is also what I call the process and the objects - fillers. Specifically, I will spawn trees, shrubbery and deposits all with the same general methodology and much shared logic but different details. Similarly to region creation, this will involve fewer passes that instead process numerous entries but in different ways.

First of all, I need to determine the "depth" of the region tiles, i.e. how far from the region's edge they are:

ed3507c55b7cf5516e4a8d28b72a342c8646126a.png


This may sound like a small thing, but it's a very important step that has huge impact on the resulting visual layout. Without this information, I wouldn't be able to make any transitions between region sufficiently gradual (or sharp) to look decent. It's also important for spawning the right amount of fillers, like deposits. This is not something I could not have done previously, because the map had no concept of "region" - all locations where only as gradual as the noise map at that location.

Currently, I do 3 runs for the different groups. Starting with trees, which get spawned in the forests (regions):

9514ed16877b99c7b3373cab775a6a0bf1392cbb.png


The placement rules, which are basically "how many trees per tile in which tiles" comes from the ruleset I specify:

6ac87d7040ea1ac5b17a90ffca690df0a3560476.png


An earlier generated noise map for tree types (which was affected by the climate) is then used to decide what type of tree to place - spruce, maple to birch. This all ends up creating a nice mix of trees in semi-contained regions:

f42217e6239f26e2c7f769b465972bd79d664f83.png


Very similarly to trees, I place minor vegetation - shrubs, grass and flowers. They also have their own ruleset and noise map, but things are just tweaked to accomplish what I want visually (which really is 90% of how I come up with the parameters):

4f1f91945e3fd7267013da22ea9bf98e16624468.png


And finally, I place deposits in the earlier established deposit/minable regions:

be5ab2dedca2af829a89b6830cdedffac38889b4.png


Most importantly, these are not just randomly spewed across the map with too many or too few tiles, but rather deliberately chosen to occupy certain areas and be of certain size (as per their region). This is one of the main problems I had with the old generator. Now, deposits follow specific rulesets, including actual deposit counts:

534f38dc088292465d565415c9e30ecd86dabdc9.png


It is also important to note that deposits come with the "amount" value. While count means "how many per tiles" - 1 to 3, amount means how many "logical states" they get in gameplay, which translates to how many times they can be mined.

d624c869af1e0ce2f017e381aed1f6c4d35f7aa3.png


82b48b97203e5f22786f73590d39a61667869c59.png


Even a simple algorithm for "filling" the deposits into a region to look nice and natural ends up being a fairly complex process. For example, I prefer to fill in the deeper region tiles before filling the outer tiles. I also want deeper tiles to have more minable amount than outer ones. But all of this is still randomized.

Output
At this point, we reach the end of the generation. The way I have structured the generator, I technically started implementing it with an input and an output pass, so I have always had "the output". This really is an arbitrary point, which is decided by what the game needs and when I stop. As I mentioned at the start, the primary goal was to get it back to working to the same level as the old generator plus fix the primary problem. At this time, that is roughly where it's at. Of course, it's easily extendable in all sort of ways now, so when I'm adding new features, I can now also consider making changes to the map generator.

I skipped describing much of minor logic and a lot of details. I also skipped all the things that didn't work - a lot of the above is the result of experimenting and making multiple versions for passes. I might mention some if and when I add some feature that is tied specifically to some part of the generation.

Combining all the outputs and data from the passes, I get my final "image":


e65b583596a60ab3cbcf5e48f3b074058de1ab00.png


Importantly, this is all not presented in the actual game, but rather only in the separate generator "project" visualized with a quick debug texture creation. And this also contains all the important data from the preceding passes, that does not appear in the final game in any form. For example, none of the cluster and region information, none of the noise maps or stuff like climates is part of the gameplay. Probably 90% of the data gets discarded. In fact, the only new data that I store compared to the previous version is the spawn location. But this data can be fed into the game now, which really "only" has to spawn the things that the output asks for - tiles and props plus whatever hard-coded spawn features. Notably, the game doesn't have to "fix" anything that the generator messed up - it's basically expected to be correct.

Final thoughts
I've been writing the main points to these posts along with implementing the generator. By the time I'm publishing the update, I do a full write-up from these key points. And, oh boy, were there a lot of those! By the end of it, I could hardly remember what I meant by half of the stuff I wrote down at the beginning. These posts also serve not just to discuss my progress publicly, but also to help me think through many design decisions by "talking them out". And there is still a lot to talk about, but I have to keep it relatively short (I say while having to split this up into 3 parts due to length).

The main conclusions and future plans can be found in the first part: Post #1.
 

GrafvonMoltke

Shoutbox Purity League
Shitposter
Joined
Dec 2, 2016
Messages
2,527
Location
Land of the Great Steppe
In case you're into that, our man's sperging out about map generators
EA Update #24 - Map Generator 1/3
This update replaces the map generator with a new version.
This update replaces the game's island map generator with a new more robust and expandable version.

ddfa094cd6affd54e6c6dcb1d70588f74b232d34.png


I had two primary goals for the new level generator. Firstly, to bring it up to at least the same level as the old generator - but do this with a modular, extendable and configurable approach. And secondly, to fix the biggest problems with the old one - excessive unusable land, poor scaling with world sizes, bad coastal shapes and many minor issues.

There is a lot to talk about designing a complex system like this. I won't go into a huge essay about my software architecture here. There are many reasons why I'm doing the things I'm doing even if they oftentimes feel like over-engineering convoluted solutions to simple problems. But I will go over the most important parts.

Due to Steam post limit, I had to split this up into three parts. This post will focus on the overview and general approach, and the goal of the generator. The other two parts will go through specific generation steps: Post #2 and Post #3.

Iteration
The hardest part of making a complex parametrized system is testing and iterating its results. The faster I can preview my changes and the easier I can understand the impact of these changes, the faster I can work on it and by extension produce better results.

This is my best friend:

a03ae29b3e9467e444967ea348e1b28fb47fe2fd.png


To accomplish fast iteration, the system has to be designed with this in mind from the start.

A map generator makes hundreds of decisions when building a map from high-level choices (for example, the number of biomes) to specific low-level choices (for example, number of trees per tile). Usually, these are encoded as direct and indirect parameters that I can quickly tweak to achieve the desired results.

The difficulty comes from having lots of parameters and wanting to understand their impact at the same time, which means previewing the difference they make to the map generation. And this is a difficult problem because any changes that happen after the relevant ones can obscure the results. Most parameters have a huge range of possible values and only a narrow band of value produce desirable (or even usable) results.

So I want to preview not just the final result, but every step of the way. To do that, I am splitting the generator's logic into discrete steps or "passes" so that I can look at the independent output(s) of any pass in the sequence.

It is also critically important that the result preview is visual. This may sound like an obvious criterion, but implementing clear and useful previews takes significant additional time and effort. It is thus a longterm strategy, which means it has all the pitfalls of additional code to implement and maintain.

a97ad645e22934e612d5e975d8f327bc5397c890.png


Another important consideration is coding the generator, not just running it. The last thing I want is to break everything by making changes. Implementing separate passes will take more time but not nearly as much time as having to rewrite large parts of the code if I interconnect everything. If I mess up some pass, then I will likely only break its inputs and outputs to the previous and next passes. And realistically I am almost guaranteed to rewrite, remove, add or majorly change many of the passes because I simply don't know how exactly they will work.

And one final and often overlooked benefit is personal motivation. Actually seeing the system work after every change is a huge boost to my motivation. It's the difference between just making something work and investing that extra polishing time. In a large system like this, small details quickly compound.

Passes
I would like to describe every step the generator takes, because I personally find it all very interesting and I could talk about it for hours. Unfortunately, this will end up with a hundred pages long post (and about 20 times over the Steam post limit). So I will try to summarize the most important steps and parts. And even with me being "brief", it is still about 3 times over the post limit. So as I'm splitting the update into three posts - this one (broad overview) and two step-by-step overviews of the generation process: Post #2 and Post #3.

Scaling
One thing I barely considered for the old generator is scaling parameters with the generated world size. This is where the huge sand areas of the old generator in the large maps come from. And this is actually a much more difficult problem than it seems, which needs to be "baked into" logic from the start. Every variable that controls something size-dependent for the generator must decide how it behaves with different map sizes and I need to be able to quickly control it. In fact, there must actually be all the variables to control size-dependent features (for example, I cannot scale parts of a noise map without scaling everything). For example, I would want to specify that a beach is 2 tiles wide or 5% of map size, or 3% of map area, etc. Choosing an incorrect value or approach might work fine for the reference size, but may produce terrible result for different sizes.

So when specifying parameters, I make sure to specify (and preview) how they scale:

2a3794cd6adda0a6d0183685bc4b2e987be1bb87.png


Generally, the options are "static", "linear" and "exponential" (with inverse variants). Static mean the same value for all sizes - e.g. number of spawn locations. Linear means it scales with the map size (width) - e.g. allowed water edge distance. And exponential means it scales with the map area - e.g. number of stone deposits.

And when testing the generator, I can run any combination to verify that they all work correctly:

af8e13f05428a607a8566802d0f62a0f06d447c1.png


Major changes
As mentioned, this update is not meant to majorly change the gameplay beyond (A) fixing and adjusting most problems with the old generator and (B) those changes that are best done now to avoid too much disruption later. So there are still a few bigger changes that do impact gameplay:

Rocks no longer spawn all over the place, but rather in discrete patches focused towards one area of the map (this preview is from the tiny map, because I cannot get a better screenshot from a large map to demonstrate the idea clearly):

3a706214fadbe49004c411102738008c938eacbf.png


Furthermore, rocks now come in 4 variants - Rocks, Coal Vein, Iron Ore Vein and Limestone Vein, with their respective deposits only spawning there:

2415ab347f1ac8f2d04381ca2d99260d951526a3.png


Consequently, the Mines now need to be placed on the appropriate type of rock:

5dee779649c58a82bb7813b6cb8fb144b130fde9.png


(That is one chonker of a tooltip... I haven't really come up with a way to shorten it either other than having 8 separate lines.)

Sand and Beach are now different tiles. Minable and usable Sand now spawn in discrete clumpy patches. Beaches spawn along the coast (sometimes in clumps) and they cannot be mined.

c19a833a038258d27947b7b8e3c4973051e713e8.png


They look a bit empty (again, because I did not want to expand the scope of the update), but there is potential to add something here.

Overall, the island shape has somewhat changed. The generator attempts to "fit a hex shape" into the game area, with a roughly-expected distance to edges, but still some larger coastal variation (I explain the various constraints in the other posts). A more "extreme" example of an island that is basically a hex:

3252e31ae74b03278b28ddb9326e8e0fdb7043ab.png


The coastline of the island is also much cleaner. There are no extended peninsulas or closed bays - the vast majority of the map should be accessible. But there can still be significant variation of the coastline. Again, a more "extreme" example with lots of "shapeness":

90f2a7ebffc545ad66bd0db2a3f46b4b5601c39f.png


Most islands end up somewhere in between the two, leaning towards more varied coast.

I also added a new grass type for some visual variety and as a proof of concept that I can vary vegetation:

f75d7eae1bc75e5843a6e8d1a2a649436e585f53.png


Which kind of lead to discussing the possibilities...

Potential
One of the main goals of the new level generator was to be able to extend it in linear time. In other words, add new stuff quickly. This means there are a lot of things I can now add to the map that I previously couldn't. Some are very easy, some are more difficult, some are impractical. For example, I won't be making multiple islands (impractical), but I can easily add more biomes (easy) or cool features like a river or two (difficult). The great thing is that I am not limited to any particular single algorithm that has to somehow produce all the results in bulk - the generator consists of many individual steps and I have a lot of freedom to add and modify those.

A particular direction I want is to specialize and utilize different parts of the map, so that the terrain shapes the village more than it currently does (i.e. not at all). For example, you might have a swamp in one end that is needed for medicinal plants or flood plains in another part, where crops grow best, and so on. Following this, I might be able to force some production chains to actually need transportation from their production location to their processing location rather than having everything clumped together.

Another big potential feature is allowing player customization for the generated map. For example, forest density, deposit amount, climate bias, etc. I haven't implemented any of this, but I have kept this in the back of my mind when designing systems. I basically comes down to having user values selecting or adjusting one or more generation parameters.

Finally, I could potentially add--if not full replayability with different climate zones and resources--then at least enough differences between maps and their layout to not all look the same.

So this is all something that I'm looking forward to (admittedly, I am also exhausted having spent so much time on the generator.) I won't even speculate on what I want to add next, because there are just so many possibilities. In fact, deciding what to add and--more importantly--what not to add is in itself a big design decision/direction.

Unity engine
I am currently using Unity engine for the game. You my have heard about the recent Unity trust debacle. I won't go into details here, but the main point is that I do not wish to continue using the engine. Unfortunately, MicroTown is many years into development and strongly tied with the engine's features. This makes porting to a different engine very difficult and time consuming. I don't know if I can realistically invest that much time into this. I spent a couple weeks investigating engines and experimenting to see what such a transition would require. For now, I am still undecided. But it's likely I'll have to stay with the current engine.

Future plans
I had to really stop myself from starting to add more features to the level generator. It has already turned out to be the largest "feature" that I have ever worked on. And the final touches and fixes again took way longer than I anticipated.

I wish I could have split this update into multiple parts over past year or so. But unfortunately this is one of those "all or nothing" features. I cannot implement half the generator or even 90% of the generator - it has to be all of it. So updates have been, shall we say, a bit slow (in addition to some other real-life stuff). So I tried to work on it 50/50 mixed with other features/updates, but that hasn't really been a good approach. I will definitely avoid splitting major workloads like that in the future.

So, hopefully, I will be making several "terrain updates" adding something to the generator and either adding or modifying something in related gameplay.

Full changelog
Changes

• Implemented new level generator algorithm
• Overall terrain shape is now properly based on a hexagonal shape, but there is more coastal variation and most landmass is slightly further from the edge of the map, but with more variation
• Terrain is now constructed in a more cell-like pattern, where "cells" correspond to biomes/regions, such as meadows or forests. Trees and other props will have way fewer unnatural "stretches" and "clumps". These "regions" will tend to be more uniform and connect more naturally.
• Islands overall shape will avoid generating with no coastline variation or excessive water or land
• Island now has several larger "climate" or "biome" sections that somewhat alter its layout and features - a part of the island has more forests, a part has more plains and a part has more rocks along with other slight vegetation biases.
• Deep Water to Shallow Water transitions are now smoother
• Water will not have excessive "patches" anymore
• Coastline is less jagged now and fewer extended unusable peninsulas or bays will spawn
• Add Switchgrass, a type of grass that spawns in Grassland
• Rename "Flower" to Marigold
• Add generic Flower concept
• Vegetation (Tree layout, types, tile variation, and Shrubbery/Marigold) layout are all now based on a separate noise patterns to avoid unnatural patterning
• Add new tile Beach that replaces Sand along the coastline, not suitable for quarrying
• Coastal tiles have much less Sand (that is, fewer Beach tiles) now
• Sand, Clay and Rocks now spawn in dedicated patches and cover much less of the terrain
• Patches of Rocks, Clay and Sand won't spawn next to similar patches anymore
• Rocks patches will spawn more in "rocky biome" and will generally include all the ores in that region (although they will still spawn randomly elsewhere)
• Clay patches now only spawn further inland
• Sand patches only spawn closer to coastline
• Several larger Beach locations now spawn randomly along the coast
• Rocks now have resource vein variants - Coal Vein, Iron Ore Vein and Limestone Vein (and their excavated variants - Excavated Coal Vein, Excavated Iron Ore Vein and Excavated Limestone Vein)
• Coal Mine, Iron Ore Mine, Lime Mine can now only be built on their resource tile variant, while Stone Mine can be built on plain Rocks
• Spawn point is now randomly offset from the middle of the map
• Spawn region is now clear of obstructing terrain
• Spawn region will always have some free-standing rocks around it
• Spawn region will only be set in "regular" climate avoiding inconvenient terrain for starting
• Patches with resources will avoid spawning in poorly-accessible unbuildable locations, like surrounded by water or unbuildable terrain
• Animal Habitats now spawn based on distance to coast rather than distance to map edge/center making for more uniform spawning and avoiding unreachable locations
• Animal Habitats now pre-spawn properly instead of appearing one at a time after the world starts
• All the internal properties of level generation now scale with the world size appropriate to their purpose. For example, larger worlds do not get larger beaches or excessive rock patches while smaller worlds still contain sufficient rock patches. Terrain features, like biomes stay proportionally-sized.
• World generation will no longer stall UI (some other operations will also have fewer UI stalling portions)
• Older saves with Mines will have their tiles converted to respective resource tile
• Older saves will have empty stone patches randomly converted to resource patches
• Add worker self-delivery max distance indicator "circle" when building or selecting buildings
• Add the worker self-delivery max distance value to several relevant tooltips
• Item deliveries during carrier shortage will allow self-deliveries, ignoring regular priorities and proportions, but not not stalling/failing to deliver
• HUD delivery info tooltip will also show self-delivery count

Fixes

• Selecting Observer would not show the empty icons above observable building until they had done at least some work
• Fix worker self-deliveries not working above dedicated carrier max delivery distance
• Fix workers delivering 2 or 3 items of their building's output to storage removing/replacing them from the building as worker
• HUD delivery info tooltip showing the wrong number of available (idle) carriers and some other slight inconsistencies due to self-deliveries
• Fix props attempting to transform while a worker is using them, for example a sapling rotting while being harvested by an arborist (and this invalid combo causing games saved at this moment to fail to load)

Balancing

• New terrain generation alters various gameplay elements

Rudy
Snowy Ash Games

EA Update #24 - Map Generator 2/3
This the second of three update posts about the new level generator. The introduction is at Post #1 and the third/last part is at Post #3.

I will now go into step-by-step process of the generation passes. For clarity, I will try to explain things in the way that makes most sense rather than sticking to exact technical pass layout. I'll try to group sections by generation "topic", although there is a lot of overlap.

Clusters
Previously, the generated terrain was based purely on a noise map (Perlin noise). This created decent-looking gradual layouts, but this did not let me actually change anything in absolute values. That is, I could adjust things relatively - for example, have sand at 5% threshold, but then it would be random amount of sand up to 5% on any size map, which might be terrible on huge maps. There was no way for me to say - I want
only
3 "patches" of sand, because there was no concept of a "patch" or "scale".

So the first and main terrain step is to produce discrete "clusters" (based on the map size). First, I generate a bunch of random (shapeless) points:

6350d0a811a7c6f30bb7ed3dd6f0672104340bed.png


Then I tessellate these (basic Voronoi diagram) into individual (shaped) clusters:

de13a9d31f658e8573034e8cd428d6496b48e40c.png


Due to randomness, some of these often end up being too large or too small, so I run an extra smoothing/averaging pass (Lloyd's relaxation):

c982fe5f31dda0a268fbc2190cac84d961a54e60.png


This produces a nicely distributed and organically-looking pattern (kind of like cells). Jumping way ahead, this is what will give the terrain the "clustered" look I'm going for (warning: now you will never unsee it):

0f01aedd780b3e373df87b9cbbcbca899f82fcce.png


Water
The next big thing that I generate is the separation between water and not-water by heightmap, which is a convoluted way of saying that I'm building a coastline. This is where a noise map like Perlin noise works really well, so that's what I'm using:

6af07eca73f34b3b7b4ea03b21ddcf4c9129d07f.png


Now, I cannot directly apply this to clusters, because the edges need to be water and the middle needs to be the island mass - by design. Previously, I was using a squashed circle on top of the height map to "cut out" the rough shape, but this created a lot of water in some directions and too much land in others. So now I implemented a much more accurate hex shape with proper "distance to edge" calculations:

d5126130126cd8239e5e9e14b258acd2ffc8a018.png


I can then overlay this shape over the actual heightmap, reducing or increasing the random value to force it below or above certain thresholds:

d344353d81b5369f6caaaeed630da2679da2df4b.png


This makes sure that anything outside the "big" hex becomes water and anything inside the "small" hex becomes land. The in-between area is then where the noise actually affects the shape. With some visual experimentation in later passes, I can tweak the noise parameters to produce satisfactory results.

I can then apply the "clamped" height map values to the clusters themselves and decide which clusters become coastal shallow water and deep water:

439db1e87b44c50e3ad0976e9bb2412a3d015aa0.png


Of course, due to the very nature of applying thresholds to randomness over an area, I sometimes end up with little inland pond and offshore islets. For gameplay purposes (and all the issues these cause), I am removing these when I find them:

9326388da198115a5d853e47f9d46f41d27f951b.png

5d23569a0dbcc362684448a8204ed99f3ede455c.png


And this is just the start of the "battle" against randomness "ruining" my islands. I'll talk about this more a little later.

Island shape
There is no perfect way to generate an island shape. Perhaps there are more suitable algorithms just for the island shape generation, but I also have all the other generator's parts to consider and they have to all be compatible. So instead of seeking some perfect algorithm, I'm just brute-forcing the generation a bunch of times until it doesn't get rejected. Some passes are there purely to reject the result and tell the generator to restart.

One such pass is for, broadly, "island shape". I generate a few "control points" in a rough approximate ellipse around the island and check if they are land or water:

8aeae60f8887b0635ef01721c3031f0baf4b7bdd.png


If there's too much water, too much land or too much repetition, I reject the shape. Otherwise, it's probably interesting enough to keep. There is no science behind this - I was just looking at hundreds of islands until I came up with heuristics and parameters that struck a good balance between random and still somewhat hex-like shaped.

This is a good time to reflect on just how many maps I generate throughout the process:

07f8a206fe364e24b97f694ba85b490ad23e0390.png


I have set up my generator to support output export at every step, so that I can leave it running for many iterations and then check them for any systematic problems. This is not something I could do before, and this is an invaluable methodology for designing and implementing a system like this.

Climate
Among many experiments to make different regions of an island feel more varied, I settled on a "climate" concept, which flags clusters as belonging to a certain climate (to a certain amount) and other passes can use this data to alter their generation. So this doesn't do anything by itself, but it provides additional customization for later passes. It's kind of like biomes, but currently there isn't any biome-specific terrain, so I'm calling it "climate".

For start, I select a few "control points" in a circle around the map (I chose this shape for simplicity and because the maps aren't large enough to come up with complicated layouts). Each climate has a area it can influence:

900537ec35d40ff4493c0e82b775a24439011baf.png


Then several climates are assigned randomly, specifically 2 forest, 2 plains and 1 rocks (and the rest stay default):

d6f000b5457be2ca408b1274e241de66e856273c.png


These are just thematic names that describe my intentions. The climate regions then flag clusters within their influence with an appropriate climate value/weight for later use:

d6f000b5457be2ca408b1274e241de66e856273c.png


Again, jumping way ahead, slight variations is later passes would create noticeable differences:

2149de10cf78addb5a5720a1df293c6c1b63a7bc.png


Spawn
Previously, I placed the starting area or spawn in the middle of the island and manually cleared it from obstructions and non-buildable tiles. And while this works, it's implemented backwards. Instead, what I really want is to decide on the spawn location and then make sure I don't obstruct it with anything and place the relevant things around it.

So now I pick a random location on the island within a spawn selection area torus:

01ea801022f7929c38e715614b5bdead1db9fb69.png


From there, I pick a valid (for example, not too close to water) closest cluster and construct a spawn area (this uses the same logic as regions, which I will describe later):

0a48badc63b552f0fe79ed2e72f5de7c874b0dea.png


This creates one "main" point for the spawn and "grabs" nearby clusters to expand the area to a specified desired size. In addition, I place several "meadow rocks" areas, so I can spawn starting stone deposits around but not on top of the spawn area.

Finally, I create a small spawn exclusion zone, where many things can generate normally, but it will mostly remain free of obstruction:

24ece4a24ad711663ba4b099773770f4a161dbb7.png


While this may sound simple, it's extremely easy to get wrong and took me enough experimentation to make a working version. There was a good reason why I previously kept the spawn centered and it was to avoid the many issues of random placement of potentially-overlapping terrain features.

Regions
Most of the map is generated in two big parts - discrete features like rocks and clay patches and continuously semi-random ones like forests or flowers. Regions are for preset discrete locations, so they have to be generated first before everything else is "filled in".

It is very difficult to break down region generation into multiple passes, because they are basically all identical except their rulesets. And separating these rulesets is only useful for visualization (which would take huge amounts of work, so I didn't really do it). So what I end up is producing the region map in a single pass:

667f25afaff3375c9e00402e9c28cfdd331fec90.png


Generally speaking, regions are groups of neighboring clusters that form an area for some purpose. For example, there could be a 4 cluster region for coal rocks. Each type of region has a long list of hand-crafted rules and requirements, mostly derived from experimenting. Plus, most of these have some sort of rules about distances in relation to each other. So it makes more sense to just see the results for all at once.

For example, here is a ruleset for placing sand regions:

ba692d481cfc443ea89a3880428613bf13e12fdd.png


What this says is that it will place 2-3 region with 2 clusters semi-randomly; it won't place two regions close to each other and will place regions close to but not next to the coastline.

There are many more rules and nuances I could add, but it's best to keep it simple - more parameters just means more work adjusting them and debugging issues. And there are already 12 different region groups just to match the basic level generation goals.

There are some notable rules though worth mentioning. For instance, some regions are much more likely to generate in certain climates. Notably, all rocks-based minable regions are more likely to appear in the rocks climate:

31cf9c7e3a72c675dcacd3bef958121f02d5e974.png


Jumping ahead to final generation, this would allow the terrain to shape village direction and create a part of the map that would naturally focus on mining and industry:

650a15ceeebfedef56b256f67e66b2f80393eab0.png


This is not something I could have influenced like this before and is only possible because I can "direct" such discretely-selected regions towards certain results I'm looking for.

Another example is a beach region (in addition to regular thin noise-based beaches discussed later), which randomly generates along the coast in a few locations:

9b81f909dcafad27c7b8268f392cc650ed44f6a7.png


This adds variety to the map without covering half the map in sand, like it used to do before.

I should note that while this is how I primarily use the regions for now, they do not specifically just control tiles (grass, sand, rocks, whatever), rather they flag the clusters as belonging to that region. In fact, the number of tiles that later get converted to their "region designation" is more strict than "cover the whole region". So regions in that sense more of an abstract layer informing later generation what to select but not necessarily exactly how. More like "this is a good place for X".

Filler regions
The second type of regions are the "filler" regions that, as the name implies, fill out the rest of the clusters that aren't already designated for something. Because this doesn't require any discrete counts, this can be done with a noise map. I'm using several slightly different noise maps for different purposes, specifically tree cover, tree types, plant type and tile variation. It's also fairly simple to add new variants and then use them in later passes.

For example, here is (part of) the tree cover noise:

ebc5c30ab61d45065ff0b15a3c68b1c252bf5bec.png


This assigns a "tree cover value" to each cluster (point):

782ade4d0dabccfd918f583af4dd39843f7f9a48.png


Then a bias to increase or decrease the noise value is applied based on the climate it's in:

ed5899d6bc0db9782b08be78b4f32b7b19cc0361.png


These are small adjustments that each climate provides, but they can create a significant variation for each "filler":

a6e1efd25f89a28eb00eb00e7beb592e027fb9e4.png


Now with the noise values decided, empty clusters can decide what sort of content they want. For example, tree cover decides where the forests will be located:

7145104fecf84219cc4ba2dec52cc7a5af9adb14.png


These noise maps will be further used in later passes for relevant selections. The above selection was more or less binary, but there are many more things that can be decided based on the full range of the random value.

And more!
I'll continue describing the remaining steps in the next, but this seems like a good time to reflect upon all the things that I am not doing. As I mentioned, it's important to set myself some limits on what I'm implementing, otherwise I will never finish this. I am certainly thinking about a lot of cool things that I could do.

As an example, I used the cluster layout and the Voronoi diagram edge connectivity to build a cluster "height map", spawn a "lake" region and then build a river running from the lake to the nearest coast:

416b66cc11edc80f73a66a7f2f7822215843469e.png


Of course, this is far from something that I can translate into tiles and gameplay, so it's more of a proof of concept. But I thought to mention this just to give an idea of the possibilities.

Next post: Post #3

EA Update #24 - Map Generator 3/3
This the third/last of three update posts about the new level generator. The introduction is at Post #1 and the second partis at Post #2.

At this point we have something resembling a map:

ac2214ac2a70a65015be04ab98e88fc6d9e91ec1.png


It's probably fair to say that the generation logic so far is already complex. But this is only perhaps third of the way to the final map implementation-wise. As more passes work with the data, they make smaller and more specialized changes. There aren't too many steps left, but they are generally an order of magnitude more complex in implementation than most preceding steps. This is also why I attempt to get as much top-level decision making done as soon as possible, so that the later passes work with smaller and more specific parts of the map without making any significant changes to the overall direction.

Tiles
The next big transition between the two generation parts is to actually create a tile layout on top of the clusters. This is quite literally how it works:

fbf12376a18b8fadd8608a35e968afa9358f8bc5.png


The main change is that the tiles actually correspond to the in-game tile hex layout (and all the math from here is in hex grid). It's also important to note that they are much denser than the underlying clusters, so all operation are way more fine-grained:

067e192c149929f3e627e9b73e1a09c0357ed9b3.png


In other words, a single cluster is represented by a collection of tiles. This is how the cell-like structure of the clusters gets preserved to the final version.

The tiles that get selected (at least, the initial version) pretty much just match the cluster they are on:

f1a2fac81215ce2909ef372da76d25d2f4627b92.png


While it was not really productive to design the main shape of the island using tiles are the basic "unit", it's important to start seeing individual tiles once the main shapes are determined, so going forward all changes will be done to tiles (directly and indirectly):

f65e766a3af1c90694988487af1c55df83a04da7.png


The downside of using purely random points to determine the clusters is that sometimes they are just too small or too large when they get converted to tiles. And there's no good way to handle this generally. These are basically statistical outliers, so after "placing" regions into tiles, I also either trim or expand them to a maximum or minimum number of allowed tiles:

461549105e2d5f045bfffc5c8bfc40ac0649d380.png


The perfectionist in me detests this kind of bodging. So this may be a good time to reflect on this.

Randomness
Randomness is always battling with order. As a concrete example, to create interesting rocky patch shapes I have to allow the random values to be random enough that these shapes get generated. I cannot restrict it too much or I will end up generating the same blob over and over. The problem of course is that random generation has no concept of subjective values like "interesting" or "balanced" or "doesn't break pathfinding horribly". To put it another way, the more cool stuff I let in, the more bad stuff sneaks in. Consequently, the more freedom I allow, the more rules I have to write.

There are several ways to ensure that the generated result is compatible with my goals. Barring just hard-coding all the values, I broadly label these "threshold", "fix", "retry" and "accept". This might sound like nitpicking (and I certainly didn't foresee myself writing an essay on randomness philosophy). But given that I have to make dozens if not hundreds of decisions as to how I will handle each such value, there emerge several patterns.

"Threshold" means setting a strict limit to the possible output of random generation. It may be strong bias, it may be a preset range, it may be a hard limit that cannot statistically be outside the designed range. For example, the middle of the island is always solid land - the threshold for land simply exceeds the noise values that are clamped to the hex shape there. The downside is that such thresholds are often fairly restrictive and I may miss out on interesting outputs simply because I deny too many.

"Fix" is the consequence of letting a lot of randomness through. Subpar outputs and awesome outputs are different ends of the same spectrum. If I aim to preserve the good bits, I have to be prepared to handle the bad bits. Besides reading like a fortune cookie, this just means applying certain fixes to expected problems, like the above example of under/over-sized regions. There is a downside though - I have to actually implement it, test it and forever maintain and adjust it with the rest of the generator, which is all the more complex because of it. This is also an excellent reason to keep such fixes self-contained in separate passes and easily reviewable.

"Retry" means literally that - just run it again. There are two general cases when I would do this. Firstly, if something breaks or I cannot generate something. For example, I might want to place a sand patch 5 tiles away from all the clay patches, but I just happened to generate clay so uniformly that there's no place on the map to place sand now. Secondly, when I don't like the result. Like the earlier example with the island shape - I might hard-code some subjective rules, filters or conditions. As easy as this solution is, it's also dangerous to start rejecting too often rather than have better rules. It's also too easy to filter too much and not give randomness a chance to produce neat results. And with a complex generator like this, one must consider that it becomes computationally costly - I just cannot retry things hundreds of times, especially during later passes.

And finally "accept". Sometimes I just have to give up. The result will never be without fault. That's statistically impossible. The good news is that this is actually mitigated by the layered complexity of the generator as well as high error tolerance of the game itself. It's actually surprisingly difficult to generate a map that's unplayable. Even a tiny map that can barely fit anything and fails half the checks is perfectly playable (and I didn't even cherry-pick the example, I just ran the game once):

c2fa341abdee47c944d333e945f1e68fde78d744.png


Shape cleanup
Similarly to clusters, tiles may end up with small ponds and islets because cluster connections do not always translate to tiles perfectly and it's occasionally possible to create "gaps" between clusters when represented as tiles:

ce223d80f463e3e6adb7e6dd58e913aa8c152553.png


These are really rare and they also don't really matter for other non-water clusters. For example, a gap between a forest in the final layout just makes it more varied, but doesn't cause any gameplay issues:

18af4ca346a38cf9acf68af18b95991d0db90dd2.png


A bunch of logic depends on coast distance, so I calculate each tile's distance to water and land:

68b9f466ade067994e8ea9410ab27864f5d257c0.png


Previously, I often encountered islands with annoying unusable coastal extrusions (long peninsulas) that mostly ended up as sand anyway. Similarly, there often appeared long water "tunnels" into land. I still occasionally get these:

b70575cb3599ac16d386c9c1fe7285b6e53aa304.png


However, now I attempt to find "landmass" and "water mass" areas, which are basically tiles that are deep inland or far from coast. This identifies any regions that are only connected by narrow strips of land/water:

4da573f4427879b995573c38aabe729b752bc61f.png


As these are undesirable, I reject these results and retry the whole generation. It's fine for there to be extrusions like this, but they have to be "reasonable" and ensuring "wide connectivity" is the approach I found best to preserve enough interesting but not broken shapes:

d9518e80625b088590cc60e2dceb217e1ca94d78.png


Another common issue is an undesirable proportion of land-to-water ratio. Some shapes end up being too large (basically a hex), others too small (boring blob in center). So I calculate how much water there is that's not close to the coast and reject shapes with bad proportions:

54485ed3a6a74e673ff7efcc1bab32e0c5689824.png


A similar common issue is the whole shape being "offset" from the center so that some coastlines are really far from the "world edge", making for huge undesirable water stretches. I reject these shapes as well:

595af9053fea71f53b0ec192adc522219b769b80.png


Coastal cleanup
The height map random noise does a decent job of creating shallow and deep water around the island, but it is also prone to creating excessive shallow water. Since I cannot "fix" it during the initial noise pass (I would also affect how much land generates), I manually trim away shallow water too far from land (convert to deep water):

3fb965af27d74ef085041a7fd50b1ab41ac267a8.png


This basically makes deep water transitions more "aggressive". This is also mostly a result of visually confirming what sort of shallow water amounts and variations look goods.

Similarly, sometime the noise transitions are so "steep" that deep water ends up touching the coast, so I offset any deep water too close to the coastline (although I fine with there being only a little gap):

ca9aa105becf919cd6fa3aac13de686dfe8d8ffa.png


Finally, all this fiddling with water level may end up shallow offshore patches. I did consider leaving them in as small cool variations in the ocean, but they almost never actually look good because they are just extreme outliers of the height map noise. So I basically remove them:

fb6781788cbf0655a9e63a50c9b656dafaf81a2f.png


Previous height map also controlled the beach elevation. That is, I spawned beaches based on the height map, which was based on the map size. This created really huge sand areas around the coastline, especially where the coastline wasn't regular. So far, I have completely removed any sand at the coastline except the specially designated beach regions. But I do actually want some narrow beach tiles to randomly appear. So I'm using some noise values to randomly change coastal tiles to sand but only with specific thresholds to avoid any excess:

a126347e94f04f50e2787d862049a27b022f06a8.png


Despeckle
A common issue with the resulting tile layout is that they sometimes have "sharp" edges - tiles that are almost completely surrounded by another tile type. This just doesn't look good in the final version, so I added a pass to trim these to match their surroundings:

40c7b2131802adbdb015ae412bdbba40de25cc53.png


With several iterations, this cleans up any unnatural looking extrusions.

A special case of this are coasts that may end up with micro-extrusions of single-tile peninsulas. I really don't like these in the final resulting tile layout, so I trim these:

86214c50c482d001368085fe257b9be3deb8b854.png


Technically, this trimming happens before all the water coastline logic because I cannot modify water/land tiles once I have established the distances. (It was a "fun" bug to chase that caused the generator to crash every 1000th run or so.)

Tile variation
At this point I assign tile variation to the map based on a noise map:

8b467c3dddd5205b6204d6fe9d985e57d7959b00.png


Currently, this only affects grassland tiles (and technically forest tiles that may revert back to grassland). Compared to previous generation, these patches and transitions are a less messy shape and don't overlap contrasting variants as much. Similar to other noise maps, climate affects these for some extra variation between "biomes":

448ea1e8cb59eba4b5065f2ea3e9dd267cf87fa0.png


Previously, I used the same height noise map for these as all the other features and while it looked fine, it was also very obvious purely random and not following any sort of natural "land" layout.

This might seem like a fairly trivial decorative feature to discuss in such detail, but the map looks really ugly with all the grass tiles being the same variant, especially after seeing the varied background for so long. Seeing such a huge difference made by such a small variation makes me consider all the other places I can vary in minor ways but to great effect.

Fillers
Finally, I get to filling the contents of regions. This is also what I call the process and the objects - fillers. Specifically, I will spawn trees, shrubbery and deposits all with the same general methodology and much shared logic but different details. Similarly to region creation, this will involve fewer passes that instead process numerous entries but in different ways.

First of all, I need to determine the "depth" of the region tiles, i.e. how far from the region's edge they are:

ed3507c55b7cf5516e4a8d28b72a342c8646126a.png


This may sound like a small thing, but it's a very important step that has huge impact on the resulting visual layout. Without this information, I wouldn't be able to make any transitions between region sufficiently gradual (or sharp) to look decent. It's also important for spawning the right amount of fillers, like deposits. This is not something I could not have done previously, because the map had no concept of "region" - all locations where only as gradual as the noise map at that location.

Currently, I do 3 runs for the different groups. Starting with trees, which get spawned in the forests (regions):

9514ed16877b99c7b3373cab775a6a0bf1392cbb.png


The placement rules, which are basically "how many trees per tile in which tiles" comes from the ruleset I specify:

6ac87d7040ea1ac5b17a90ffca690df0a3560476.png


An earlier generated noise map for tree types (which was affected by the climate) is then used to decide what type of tree to place - spruce, maple to birch. This all ends up creating a nice mix of trees in semi-contained regions:

f42217e6239f26e2c7f769b465972bd79d664f83.png


Very similarly to trees, I place minor vegetation - shrubs, grass and flowers. They also have their own ruleset and noise map, but things are just tweaked to accomplish what I want visually (which really is 90% of how I come up with the parameters):

4f1f91945e3fd7267013da22ea9bf98e16624468.png


And finally, I place deposits in the earlier established deposit/minable regions:

be5ab2dedca2af829a89b6830cdedffac38889b4.png


Most importantly, these are not just randomly spewed across the map with too many or too few tiles, but rather deliberately chosen to occupy certain areas and be of certain size (as per their region). This is one of the main problems I had with the old generator. Now, deposits follow specific rulesets, including actual deposit counts:

534f38dc088292465d565415c9e30ecd86dabdc9.png


It is also important to note that deposits come with the "amount" value. While count means "how many per tiles" - 1 to 3, amount means how many "logical states" they get in gameplay, which translates to how many times they can be mined.

d624c869af1e0ce2f017e381aed1f6c4d35f7aa3.png


82b48b97203e5f22786f73590d39a61667869c59.png


Even a simple algorithm for "filling" the deposits into a region to look nice and natural ends up being a fairly complex process. For example, I prefer to fill in the deeper region tiles before filling the outer tiles. I also want deeper tiles to have more minable amount than outer ones. But all of this is still randomized.

Output
At this point, we reach the end of the generation. The way I have structured the generator, I technically started implementing it with an input and an output pass, so I have always had "the output". This really is an arbitrary point, which is decided by what the game needs and when I stop. As I mentioned at the start, the primary goal was to get it back to working to the same level as the old generator plus fix the primary problem. At this time, that is roughly where it's at. Of course, it's easily extendable in all sort of ways now, so when I'm adding new features, I can now also consider making changes to the map generator.

I skipped describing much of minor logic and a lot of details. I also skipped all the things that didn't work - a lot of the above is the result of experimenting and making multiple versions for passes. I might mention some if and when I add some feature that is tied specifically to some part of the generation.

Combining all the outputs and data from the passes, I get my final "image":


e65b583596a60ab3cbcf5e48f3b074058de1ab00.png


Importantly, this is all not presented in the actual game, but rather only in the separate generator "project" visualized with a quick debug texture creation. And this also contains all the important data from the preceding passes, that does not appear in the final game in any form. For example, none of the cluster and region information, none of the noise maps or stuff like climates is part of the gameplay. Probably 90% of the data gets discarded. In fact, the only new data that I store compared to the previous version is the spawn location. But this data can be fed into the game now, which really "only" has to spawn the things that the output asks for - tiles and props plus whatever hard-coded spawn features. Notably, the game doesn't have to "fix" anything that the generator messed up - it's basically expected to be correct.

Final thoughts
I've been writing the main points to these posts along with implementing the generator. By the time I'm publishing the update, I do a full write-up from these key points. And, oh boy, were there a lot of those! By the end of it, I could hardly remember what I meant by half of the stuff I wrote down at the beginning. These posts also serve not just to discuss my progress publicly, but also to help me think through many design decisions by "talking them out". And there is still a lot to talk about, but I have to keep it relatively short (I say while having to split this up into 3 parts due to length).

The main conclusions and future plans can be found in the first part: Post #1.
Maybe you should just fucking kill yourself; see how that works out.
 

As an Amazon Associate, rpgcodex.net earns from qualifying purchases.
Back
Top Bottom