[MODDING TOOL] GIS Construction Set
Post by
carademono » Mon Jun 07, 2021 11:00 pm
GIS Construction Set
by carademono
Description
Daggerfall is really big. Far, far too big to work with conventional modding construction sets like those for Morrowind, Oblivion and Skyrim. Geographic Information Systems (GIS) tools, however, are perfect for working at such large scale.
This post presents a QGIS (
www.qgis.org) project designed to help modders place location prefabs that load into DF Unity with Kamer's World of Daggerfall. Using the Construction Set, modders can individually hand-place prefabs with considerable precision, or they can use QGIS's random and algorithmic point placement functions to place prefabs by the dozens, hundreds, thousands, or tens of thousands.
The project currently contains four base maps:
1. Iliac Bay Base Map: The Iliac Bay MapPixel map by Meritamas, included in BadLuckBurt's world data editor (
https://github.com/BadLuckBurt/dfu-worlddata-editor).
2. Paper Map: A beautiful Iliac Bay paper map created by Rubikia (
https://www.reddit.com/r/Daggerfall/com ... iliac_bay/)
3. Travel Map: The in-game travel map.
4. Vanilla Height Map: The height map for vanilla Daggerfall.
It also currently includes seven reference layers:
5. Water: A simple vector land and water map.
6. Regions: A vector map of the Iliac Bay's regions. This can be used to place points within specific regions.
7. Locations: A layer map containing all 15,000+ points of Daggerfall's locations.
8. Basic Roads: A raster map of Hazelnut's Basic Roads network.
9. Roadside - Roads & Tracks: A vector map of the Basic Roads network divided by regions, with polygons extending about a map pixel from the roads. You can use this to place points near roads -- for example, bandit camps and Imperial watch towers placed in this layer will be visible from the road.
10. Roadside - Roads: Like the above, but
only includes roads, not tracks. You might place Imperial watch towers here.
11. Roadside - Tracks: Like the above, but
only includes tracks, not roads. You might place bandit camps here.
Finally and most importantly, it includes:
12. Prefab Placer: All points created in or copied to this layer will have their locational fields autopopulated with values that can be read directly by the LocationLoader or World of Daggerfall mods. This includes the
worldX,
worldY,
terrainX, and
terrainY coordinates. The layer also calculates a unique
locationID for every possible point by concatenating its coordinate values. You will need to enter in only
name (which does not need to be unique),
prefab, and
type (more on this below). These can be entered either when adding points individually, in QGIS's attribute table, or in a spreadsheet program after export.
You can download all the project files from my GitHub:
https://github.com/drcarademono/GIS-construction-set. It requires QGIS (
https://qgis.org/en/site/forusers/download.html), and Kamer's World of Daggerfall (which is based on Uncanny_Valley's LocationLoader).
Getting Started: Adding Prefabs
Adding points to the Prefab Placer layer works like adding points to any QGIS layer. Select Prefab Placer in the layer list and click the pencil icon (Toggle Editing) in the top tool bar. Now click the Add Point Feature icon in the toolbar and click anywhere on the map where you'd like to add a prefab. Excellent! You've added a point.
Of course, adding points one by one would take a long time if, say, you'd like to add 20,000 rocks to the Alik'r Desert. I'll introduce QGIS's abiltity to place large numbers of points randomly or algorithmically within polygons later in the tutorial. Anything that you can do in QGIS, you can do to the Prefab Placer layer... and QGIS can do quite a hell of a lot.
Export & Data Management
Once you've filled it with one or a million points, you should export your Prefab Placer layer to a CSV file. Right-click on the layer, Export -> Save Features As... and select Comma Separated Values [CSV] from the drop-down list. Choose your file location and save. If you're dealing with a large number of points, this might be a good time to open up the CSV file in a spreadsheet program and mass-enter
name,
prefab, and
type.
LocationLoader locations files are simply XML-formatted text files. In order to actually get the prefabs into the game, you'll need to reshape the CSV into one (or many) of those. I've put together a Python script (
https://github.com/drcarademono/GIS-con ... convert.py) that automates the process of splitting and converting your CSV into smaller txt files that World of Daggerfall can load. To use it, simply place the script in the same folder as your CSV file and type at the command line:
CODE:
SELECT ALL
python gisconvert.py 'input.csv' 100
If you are on Windows and don't have Python, I've created a command-line application (
https://github.com/drcarademono/GIS-con ... onvert.exe) with similar usage:
CODE:
SELECT ALL
gisconvert 'input.csv' 100
This will split the CSV into txts with 100 prefabs each, which is an ideal length for World of Daggerfall. You can find these files in the out folder after running the script. To see your new locations in the game, put the txt file(s) in the StreamingAssets\Locations folder and activate
world of daggerfall.dfmod.
Note: Using numbers higher than 100 with gisconvert.py is not recommended as it increases loading times and may lead to some prefabs not being placed in-game.
Now for some more specific tutorials on how to use this tool.
Tutorial #1: Adding a dock to Kalunnunu
We'll start with an easy project: adding a single dock prefab to the north side of Kalunnunu.
Let's start up QGIS and open
GIS Construction Set.qgz to get started on our first project. If all went well, you should be greeted with something like this:
See the Layers list in the lower left corner? It should have the Prefab Placer layer right at the top. However, we don't want to work directly with that layer -- let's make a copy of it to work with instead. Right-click on Prefab Placer, select Duplicate Layer, activate the new layer by clicking on the checkbox to its left, drag it to the top, and rename it something like "Kalunnunu Docks."
Now let's go find Kalunnunu. It's in the Lainlyn region. If you're not sure where that is, activate the Regions layer for a moment and it will superimpose the region borders on top of the base map. Kalunnunu is right on the coast:
Next we're going to place a prefab! Click on the Kalunnunu Docks layer you created, and select Toggle Editing on the top toolbar (it's the pencil icon). Now select Add Point Feature (the three dots icon) and your mouse should change to a crosshairs. Let's try placing the dock
just north of Kalunnunu. Click on the map and a little window will pop up, asking us to us fill in various attributes.
Name can be set to anything (I went with "Docks"), while
type has three potential settings in World of Daggerfall:
type 0 smooths the terrain underneath the prefab
type 1 does not smooth the terrain underneath the prefab
type 2 allows prefabs to be placed in map pixels with locations and roads.
Note: World of Daggerfall ordinarily prevents prefabs placed near locations and roads from appearing in the game, as Basic Roads does not currently distinguish between world locations and prefab locations. Because of this, if prefabs are placed too close to a road, players might get pulled off the road and sucked into the prefab.
type 2 overrides this limitation, allowing prefab placement anywhere, but should be used with caution. However, if a prefab you've placed isn't appearing in-game, you should try setting it to
type 2.
Because the Kalunnunu dock is very close to the city of Kalunnunu itself, we will need to set
type to 2 in order to see it in the game. Finally, the
prefab field is where we specify which specific prefab we want to place. Let's go with WA_Poor_Docks_North_00 from Kamer's World of Daggerfall mod. Click OK. There's now a pink prefab symbol where we placed the dock.
Now that we've placed a prefab, it's a good moment to check out the Kalunnunu Docks layer's attribute table. Right-click on the layer and select Open Attribute Table. If everything went right, you'll observe that
worldX,
worldY,
terrainX,
terrainY, and
locationID have all been autopopulated with the correct values:
Click Toggle Editing again to turn off editing and save your changes to the layer. We're ready to export our new docks to a CSV file. Right-click Kalunnunu Docks and select Export -> Save Features As... Change format to Comma Separated Values [CSV], give the file a name, and hit OK. You can close QGIS now, saving your work first, of course.
No need to edit the CSV file -- we already filled out all the necessary fields in QGIS. Let's convert it to an XML-formatted txt file. Assuming you have Python installed, place the gisconverter.py script in the same folder as your CSV file and type at the command line:
CODE:
SELECT ALL
python gisconvert.py 'Kalannunu Docks.csv' 100
Kalannunu Docks.txt will appear in the out folder. Place the file in your StreamingAssets/Locations folder. Boot up DF Unity and fast travel to Kalunnunu...
Pretty darn good placement! We can edit the txt file to get it just right. The docks are a few meters too far to the north so try changing
terrainY from 9 to 5...
Perfect!
Kalunnunu Docks.zip
(334 Bytes) Downloaded 6 times
Tutorial #2: Launching a Crime Wave
Let's suppose that we decide there's been a prison break in Arkmoth End, and now the hills are full of bandits. Let's randomly place 30 bandit camps in a polygon around Arkmoth End.
The first thing to do here (after duplicating the Prefab Placer layer again and renaming it Prison Break) is to draw a polygon around Arkmoth End. To do this, we'll need to create a new polygon layer. Select Layers -> New Shapefile Layer... from the menu. Set the layer to have a Polygon geometry type, name it whatever you'd like, and hit OK.
Select the new layer, select Toggle Editing, and select Add Polygon Feature. You can now draw the polygon by clicking in a circle around Arkmoth End (give it any id number).
Once you've made your polygon, it's time to fill it with bandit camps. Go to Vector -> Research Tools -> Random Points inside Polygon in the top menu. In the popup box, specify that you want to add 30 points. Set a minimum distance of .3 between them so that they don't spawn right on top of each other, and hit OK.
Your polygon will now be full of points, but all on a new layer called Random Points. Select this layer, hit Toggle Editing, right-click on it and open its Attribute Table, and select all the points by shift-clicking the numbers 1-30 on the left side.
Then select Edit -> Copy Features in the top menu. Select your duplicate Prefab Placer layer, hit Toggle Editing for this layer too, then Edit -> Paste Features.
Note: Random placement might place your prefabs right on top of cities, dungeons, and other locations. You should do a once over and remove points that are right on top of locations on the map.
Your bandit camps are now placed! Go ahead and export the layer as a CSV file. Open the CSV file in a spreadsheet app and finish filling out the empty columns. As before, you can put any value in the name column, and they can all have the same name. We want terrain smoothing, so type should be 0. Finally, we're going to use the
WA_Daggerfall_BanditCamp_01,
02, and
03 prefabs from Kamer's World of Daggerfall.
Save the CSV, convert it to an XML-formatted txt file using gisconvert.py, put it in your StreamingAssets/Locations folder, and fire up DF Unity....
Tip: Modders who are accustomed to games like Morrowind or Skyrim might have some trouble adjusting to the sheer scale of Daggerfall. For comparison, the small, approximately 7x6 MapPixel polygon we drew around Arkmoth End is larger than the island of Vvardenfell! From that perspective, 30 bandit camps doesn't seem all that excessive.
Prison Break.zip
(830 Bytes) Downloaded 5 times
Tutorial #3: Dropping Rocks in the Alik'r Desert
How can we use the GIS Construction Set to cover the Alik'r Desert with rocks? We'll be doing much the same thing as before -- filling a polygon with random points -- except that the Alik'r Desert is a much, much bigger polygon. We're going to need a lot of rocks. Lets try 20,000.
First, we need to select our polygon. No need to draw a new one, as the Alik'r Desert polygon can be found on the Regions layer. Activate and select the Regions layer, click the "Select Features by area or single click" icon from the top toolbar, and click on the Alik'r Desert. It should light up in yellow like this:
Once again, select Vector -> Research Tools -> Randomly Add Points to Polygon. Set the function to generate 20,000 points (or alternately, play with the point density field).
Copy your random points to the duplicate Prefab Placer layer, and export to a CSV file. Open the CSV in a spreadsheet program. We can name all 20,000 rows "Rocks," and as we don't need smoothed terrain for rocks, we can set all the types to 1. The prefab column will take a bit more thought: after all, it would be a bit strange if we were to just see 20,000 instances of the same rock in the Alik'r Desert. You can make a rock prefab mix using simple spreadsheet tricks. Remember that these randomly generated rocks are in totally random order, so this strategy will have the rock types nice and mixed up. Save the CSV file.
Split and convert the file using gisconvert.py. That accomplished, we drop the txts into StreamingAssets/Locations and fire up DFUnity...
Congratulations! You are now ready to transform the Iliac Bay.
Alikr Rocks.zip
(397.33 KiB) Downloaded 8 times
Limitations
The primary limitation is that, at least at present, the GIS Construction Set can only place prefabs. It can't modify World Data or Daggerfall's blocks. Prefabs are terrific for bringing life to the wilderness between cities, but they can't integrate with the quest system (at least, not yet).
Roadmap
Future versions of the GIS Construction Set will hopefully include the following:
- A Python-based QGIS plugin to import Locations txt files, allowing for an easier circular workflow between the editor and the game.
- A community-maintained repository of tested, high-quality prefabs, along with SVG symbols for QGIS that show modders their outlines and dimensions, allowing for perfect placement every time.
Can you help me with any of these things, or have a different idea? Please submit a PR!
Have fun, everybody!
