Putting the 'role' back in role-playing games since 2002.
Donate to Codex
Good Old Games
  • Welcome to rpgcodex.net, a site dedicated to discussing computer based role-playing games in a free and open fashion. We're less strict than other forums, but please refer to the rules.

    "This message is awaiting moderator approval": All new users must pass through our moderation queue before they will be able to post normally. Until your account has "passed" your posts will only be visible to yourself (and moderators) until they are approved. Give us a week to get around to approving / deleting / ignoring your mundane opinion on crap before hassling us about it. Once you have passed the moderation period (think of it as a test), you will be able to post normally, just like all the other retards.

Underrail saves/data modding

jagged-jimmy

Prophet
Joined
Jan 25, 2008
Messages
1,551
Location
Freeside
Codex 2012
I was tinkering with the underrail files for a while. So i wanted to share and summarize the findings...


What we (seem to) know:
Underrail save files and data are "packed" and obfuscated. You need to "unpack" it, then you can analyze the obfuscated data.
There is a file unpacker/packer. People used it to modify fucked up quest flags.
See here https://underrail.com/forums/index.php?topic=1700.0
Tool: https://github.com/MichaelBurge/underrail-unpacker/releases

Any underrail file is just 24 bytes + Gzipped obfuscated data. So you can write your own program in your favourite language to unpack it. Just google gzip/unzip files...
Coding wise: Ignore/remove first 24 bytes, then unzip the rest. I did it with java...
Packing however is tricky. I assume Gzip implementation might be different for C# and java or i fucked up somehow.
But packing works with the tool above. It is written in C# as Underrail.


What triggered me to start my analysis:
I googled a bit and found the data format of underrail data. Not sure it was known before, could not find any info on this.
See here https://stackoverflow.com/questions/3052202/how-to-analyse-contents-of-binary-serialization-stream

Basically for programmers among us: Styg just probably serializes the data structures (classes) using Micrisoft .NET formatter
with some kind of obfuscation on. But only class and variable names are obfuscated. The overall structure and values are readable.


Approach:
First i tried to parse the whole file acording to the identified format specification. But, unfortunately, either the spec i found
does not fully match the actual format (slightly different version) or the obfuscation fucks the data here and there randomly, preventing my parser to work reliably.

Then i searched for records of the format anywhere in the file. Values and stuff must be stored in classes and so finding all classes and their values would be enough to find the right things to modify. This works, classes and values can be found, but their relationship (the structure) remains unknown.


Findings:
  • I identified the place where stats and skills are in the file. They are saved as "value" / "effective value" and are easily modifiable.
  • I could find the quality of a component in the inventory and change it to my liking. So any unique values such as money, stack sizes are probably easy to find. Finding such values is not just luck, if you know the format spec and can exclude values, which cannot be "data"/"value".
  • I could modify a unique item (damage). This is also easy, as unique items have set values and their files can be unpacked and read the same way. Then just modify and replace (back up, of course)

Tools and analysis loop:
  1. Unpack using the tool. For me it worked without any additional SW installation.
  2. Use a hex editor to check out the data. I use https://mh-nexus.de/en/hxd/
  3. Find and modify value with the editor.
  4. Pack using the tool. This is important. If you modify values, i recommend using the tool to unpack/pack. For analysis only you can unpack using your own tool.
  5. Load up the char and check in game
  6. Search & Guess: create multiple saves (1 item in inventory, 4 items, etc.) and DIFF the binary output with Beyond Compare (https://www.scootersoftware.com/download.php?zz=dl3_en) or similar tools.

So yeah, in theory, one could analyse the complete data structure (manually and by custom written tools) and it might lead to something.

This is a modified "The Claw" with 75-150 Bio damage, instead of 1-3. This works in game.
3IWJzAS
Notice the "Little Endian" notation. I just searched for original Bio damage 1-3 and modified, repacked.
Original was 01 00 00 00 03 00 00 00
 

epeli

Arcane
Joined
Aug 17, 2014
Messages
719
Basically for programmers among us: Styg just probably serializes the data structures (classes) using Micrisoft .NET formatter with some kind of obfuscation on. But only class and variable names are obfuscated. The overall structure and values are readable.

Since you didn't link it directly, the file format spec: https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-nrbf/

As far as I can tell, the "obfuscation" of serialized files is done manually to complement dotfuscator on the executable. Strings aren't obfuscated per se, the devs simply name the serializable data model elements with vague and hard to read abbreviations. You can usually figure them out by partially unobfuscating the dotfuscated .exe with a tool like dnSpy and searching for the data model identifier string. Remember to match whole word, case sensitive.

Packing however is tricky. I assume Gzip implementation might be different for C# and java or i fucked up somehow.

Any proper gzip implementation should work. I used https://github.com/nodeca/pako (a javascript zlib port) for it. It works even though the files it produces are not byte-identical to the originals. The only difference is the OS flag in gzip headers. It's hardcoded to "unix" in pako, whereas C# detects it and sets it to Windows in most cases of Underrail gaming. C# will happily read the data regardless of the gzip header os flag.

Also some small files (.udlg, .obj) aren't even gzipped. Just mind the header and you're good to go as is. And some underrail files might have multiple gzipstreams (or some other data after the first stream) in them, I haven't looked into it.

But packing works with the tool above. It is written in C# as Underrail.
Pack using the tool. This is important. If you modify values, i recommend using the tool to unpack/pack. For analysis only you can unpack using your own tool.

Word of warning about the old underrail-unpacker: Packing files with it can introduce bugs. The underrail header hardcoded into it has a data model version number from 2015. So if you edit any file that has data model elements which have had their functionality changed since then and save it with the 2015 version header, the game will interpret them wrongly.

It might work with global.dat from underrail saves, though, since saving again in-game will update the fucked up version number, hopefully before any of the data model elements are misinterpreted using the old version. But for general-purpose underrail file modding you must preserve the correct data model version.
 

jagged-jimmy

Prophet
Joined
Jan 25, 2008
Messages
1,551
Location
Freeside
Codex 2012
Any proper gzip implementation should work. I used https://github.com/nodeca/pako (a javascript zlib port) for it. It works even though the files it produces are not byte-identical to the originals. The only difference is the OS flag in gzip headers. It's hardcoded to "unix" in pako, whereas C# detects it and sets it to Windows in most cases of Underrail gaming. C# will happily read the data regardless of the gzip header os flag.

Also some small files (.udlg, .obj) aren't even gzipped. Just mind the header and you're good to go as is. And some underrail files might have multiple gzipstreams (or some other data after the first stream) in them, I haven't looked into it.

Then i fucked up somehow probably.

Anyway, all i wanted is to place an item to the inventory. Aaaand this only to have fun on subsequent playthroughs (ex. craft best gear as soon as skills allow it, no dumpster searching for components or merchant camping).
But Styg implemented the inventory in some kind of hash map, with "keys" probably being enumerations of some sort with IDs of items. These are readable strings and can be easily found. But the values of items, like quality, dmg etc are the "values" of the map. This structure is distributed in the file and can be found by exporting a char with different items.
To link both (key+value) together in some serialized C# hash map implementation requires some top notch autism i would say...
 

taxalot

I'm a spicy fellow.
Patron
Joined
Oct 28, 2010
Messages
9,681
Location
Your wallet.
Codex 2013 PC RPG Website of the Year, 2015
Is there such a thing as a modern save editor for this game ?

I think I fucked myself. I was about to do the Kill the Beast quest and I realized I lost my Psi Amplifier. It's not in my locker or in my room at commons. I know, I brought this on me, but rather than searching every container in the game world, is there a way for me to get back the item ? At the very least tell me Kill the Beast is not a mandatory quest.
 

ghostlife

Literate
Joined
Apr 24, 2023
Messages
38
Is there such a thing as a modern save editor for this game ?

I think I fucked myself. I was about to do the Kill the Beast quest and I realized I lost my Psi Amplifier. It's not in my locker or in my room at commons. I know, I brought this on me, but rather than searching every container in the game world, is there a way for me to get back the item ? At the very least tell me Kill the Beast is not a mandatory quest.
you can probably add it with the dev console https://www.reddit.com/r/underrail/comments/11gt2yh/enabling_the_developer_console/
 

CHEMS

Scholar
Joined
Nov 17, 2020
Messages
1,504
Styg is probably sending in a serbian death squad to your house rn
 

Forest Dweller

Smoking Dicks
Joined
Oct 29, 2008
Messages
12,204
Is there such a thing as a modern save editor for this game ?
Wondering this too. This unpacker:

https://underrail.info/unpacker/

no longer works.

I think I fucked myself. I was about to do the Kill the Beast quest and I realized I lost my Psi Amplifier. It's not in my locker or in my room at commons. I know, I brought this on me, but rather than searching every container in the game world, is there a way for me to get back the item ? At the very least tell me Kill the Beast is not a mandatory quest.
It's not mandatory.
 

epeli

Arcane
Joined
Aug 17, 2014
Messages
719
Is there such a thing as a modern save editor for this game ?
Wondering this too. This unpacker:

https://underrail.info/unpacker/

no longer works.

I'm pretty sure it still works, since the only thing it does is (un)gzipping Underrail files while preserving their version header. It's just a proof-of-concept thing, not a save editor. You still have to deal with binary files.

Anyways, just use the console since it's finally public knowledge. The wiki was also written with it in mind: Most item infoboxes include the item datafile (you'll have to click View Source since its not displayed on the actual page due to someone never willing to publicize the console) and there's a list of zone filenames at Zones & Zones/Expedition, among other things.
 

As an Amazon Associate, rpgcodex.net earns from qualifying purchases.
Back
Top Bottom