Let's talk about the GUI

[ryefish]

Howdy All! I'm new to ToEE and I've set out to make a couple tweaks to the UI before starting my first adventure. Let me know if I should start a fresh thread rather than necro this one (or should I be posting to co8?) - I had intended to do so but while scanning previous posts I discovered this one and a number of the changes mentioned are what I've been trying to do - with some success (I think). Specifically: expand window sizes, show all info on charsheet without tabbing (other than different class spells), and center dialog, among a bunch of other things.

I don't know if any or all of these have already been resolved or implemented elsewhere since this was first posted. If so, please let me know. Hopefully something I've discovered is useful. I have a LOT of notes and a number of questions, but for now I'll just try to give a brief description of some of the things I've been experimenting with.

First off: Resizing window backgrounds.

I've had success changing window sizes of some windows (charsheet/rollhistory/townmap) by making changes in overrides/art/interface/[component]_ui/. The fix I've been experimenting with involves modifying 3 file types in concert, the main .img, the main .mes, and a grid of .tga images. I'll use townmap_ui as an example here:

EDIT: map_interface.img
This is a 4 byte file that stores the width (first 2 bytes) and height (second 2 bytes) of the complete background image.

ex: CF 02 BC 01 (hex bigendian?) -> 2CF 1BC (hex) -> 719 x 444 (same size listed in .mes)​

Change this to the larger desired size

ex: 00 04 00 03 for 1024x768​

EDIT/ADD: map_interface.tga files
The .tga files work as tiles in a numbered grid (_COL_ROW) that starts at the lower left corner (_0_0), going up columns and then over rows. The maximum size of an individual .tga appears to be 256x256. I you need more height or width than that, add more rows or columns respectively.

ex: to make a 1024x768 background need 4x256 wide and 3x256 high, that's 12 images named map_interface_[COL]_[ROW].tga
..._0_0, ..._0_1, ..._0_2,... _1_0, ..._1_1, ..._1_2, ..._2_0, ..._2_1, ..._2_2, ..._3_0, ..._3_1, ..._3_2​

Note: I find it easiest to copy and existing 256x256 .tga and rename it to generate new image files.

EDIT: 0_townmap_ui_locations.mes

change main_window width {012} -> 1024
change main_window height (013} -> 768
change mapview_window width {} -> 706
change mapview_window height {} -> 768
add about 300 to x position of other displayed buttons
Note: Unfortunately I've changed my layout since I did this so I don't have exact x pos for the buttons, just push them to right off the map.​

And that should be everything: The .img now sets a larger size for the total image, the .tga tiled grid fills it, and the .mes file positions/sizes the window that gets displayed. And now I can see all of Hommlet to the North and South, I just have to pan a little to see all of the far side East of town, I assume on a widescreen you could see the entire map at once.

Know Issues: This has worked well so far, although while experimenting with setting the localmap fullscreen (I run at 1280x1024) the game has now started to crash whenever I quit. I haven't looked into crash at all yet so could be an issue related to how big I made it or could be something completely unrelated (I've got a lot of 'test' changes floating around in my codebase right now).

Important Note: I have not tested this change or any of my others beyond the tutorial or the starting location in Hommlet. I still have not started the campaign and I am doing my best to remain spoiler free. Which has let me there are things I won't be able to test (like the list of maps in the townmap screen - I only have 2 maps).

It has turned out that even trying to give a brief description of this experimental solution has turned into a rather long post. I'll continue with my changes in another post.

Oh, and thank you so much for the truly incredible mod! I've been looking for something like this for a long time.
And do let me know if you have any problems testing this fix: its possible a typo could have crept in.

 

[ryefish]

Now on to the next topic: An experimental method that modifies the character sheet to always show the Portrait, the 3d model, and the Paperdoll (equipped).
This modification allows for equipping a character or changing loadouts and immediately seeing the effect on the 3d model, as well as always being able too see the portrait.

This change involves editing a c++ file (ui_wrapper.cpp) and then tweaking .mes files so the additional elements don't overlap. It functions by unhiding the Portrait and 3d model windows when the Paperdoll unhides.

Note: From here on out I'm not going to give precise details on repositioning via .mes, I'll assume you know how to do that. I've made so many changes I no longer have the coords for just these fixes. Also I'm writing parts of this from memory so might need tweaks.

EDIT: ui_wrapper.cpp->ui_widget_set_hidden(LgcyWidgetId id, BOOL hidden)
uiManager->SetHidden(id, hidden == TRUE);
//Add this to display port&3d when paperdoll displays
if (id == 596 && hidden == 0) {
uiManager->SetHidden(594, 0);
uiManager->SetHidden(595, 0);
}
EDIT: a number of .mes files
.../interface/char_ui/0_char_ui.mes
move entire window to the right enough to have from for portrait/3d {120}
.../interface/char_ui/char_portrait_ui/2_char_portrait_ui.mes
move portrait to the left {010-013}
move 3d down and to left {000-003}
move button_window and buttons out of the way, x=0 (no longer needed) {031} {061} {081} {101}
NOTE: I've made a lot of changes to these mes files, you may need to tweak additional positions.

This experimental fix was difficult for me to get working (I'm not a C++ or Assembly programmer) and has gone through a number of different incarnations already. I've boiled it down as simply as a could, but I have no idea if I'm putting this where it really should go in the code (unlikely), or if there is some other way entirely different way to accomplish this (like via python?).

Known Issues: There is an issue with NPC-PCs that I recently discovered after I figured out how to add an NPC to my party using the DM interface (which is immensely useful, and the Debug Info window which is even more useful - Excellent tools!) Pre-modification the NPC-PCs did not have any way to show a portrait (the button is greyed out), post-modification it auto-shows. BUT: there are NO portrait sized assets for NPCs and .../interface/portraits/portraits.mes has a blank {} for the big image. That's just an asset problem, the more core issue as that the char_sheet pulls the '_s' small portrait rather than the '_b' big portrait (its using {XXX2} rather than {XXX0}. I suspect I can jump in somewhere and 'shift' it over to read from {XXX0}. I have a fair bit of additional information about this issue, but I'm trying to keep these overviews brief.

Again as a reminder: This has only been tested in Tutorial and Hommlet landing location, although I've had this modification in place for a while now and other than the NPC issue it has worked great.

 

[ryefish]

Next is a related topic: An experimental method the modifies the character sheet to always show Inventory, Skills, Feats, and Spells

This change is similar to the last, but more complex - each of the different windows behave in different way regarding the 'render cycle?' and my limited C++ knowledge impacted how I accomplished this. It functions by unhiding Skills in the same way Portrait/3d were unhid and by initializing Feats and Spells when another button in the Char sheet renders.

EDIT: ui_wrapper.cpp->ui_widget_set_hidden(LgcyWidgetId id, BOOL hidden)
uiManager->SetHidden(id, hidden == TRUE);
//Add this to display port&3d when paperdoll displays
if (id == 596 && hidden == 0) {
uiManager->SetHidden(594, 0);
uiManager->SetHidden(595, 0);
}
//Add this to display inv&skills&feats&spells
if (id == 650 && hidden == 0) { //650 is the last inv box loaded
uiManager->SetHidden(1609, 0);
}
EDIT: ui_char.cpp->UiCharHooks::AlignGenderRaceBtnRender(int widId)
UiRenderer::RenderText(text, rect, style);
UiRenderer:opFont();

//RYEMOD: Trying to get spells to show without cycling error.
//if (uiManager->GetActiveWindows().size() == 60) {
if (std::find(uiManager->GetActiveWindows().begin(), uiManager->GetActiveWindows().end(), 691) != uiManager->GetActiveWindows().end()) {
UiCharHooks::FeatsShow();
} else {
UiCharHooks::FeatsShow();
UiCharHooks::SpellsShow(temple::GetRef<objHndl>(0x10BE9940));
}


Note: There is no good reason this modification changes UiCharHooks::AlignGenderRaceBtnRender other than it was the first place I found that worked, definitely not the right place to put this. Also: UiCharHooks::SpellsShow call requires a temple::GetRef<objHndl> - I used... the characters objHndl I think, wasn't really sure what to pass it so I used whatever I could to get it work. I hope the address I used (0x10BE9940) is the same across systems but I don't really know.

Known Issues: Functionality wise I haven't encountered any problems. I tested metamagic and enchanting items, as well as creating a test characters with a level in 7-8 classes and having multiple spell classes also worked (other than some preexisting layout issues). My concerns are with how I implemented this. This was difficult to get working so as soon as I figured out a way (regardless of how or where I put things) I moved on to other issues. The problems are in how differently the windows behave render wise. Again I have a lot more detail regarding things I tried worked/failed and such.

 

[ryefish]

Next up is where I started this whole exploration: How to show all 21 skills w/o scrolling

Just changing the SKILL_BTN_COUNT to 21 causes a crash (Access violation), and I think I have tracked down the cause: the memory location where button 21 gets stored is being overwritten by temple.dll. It looks like ln:184 'skillsActive = temple::GetRef<BOOL>(0x10D1A388)' is storing to the same place and at some point skillsActive ends up having the button as its value. I tried a number of things (like 'shifting' skillsActive over by 4 (0x10D1A38C) in debugger/memory) but eventually put in a temporary hack: store the 21st button somewhere else and pull it back in when I need it.

EDIT: ui_char.cpp
#define SKILL_BTN_COUNT 20 -> #define SKILL_BTN_COUNT 21
EDIT: ui_char.cpp -> UiCharImpl::SkillsWidgetsInit() - append to the end (after ln 2453):
//Store button[20] in button[23] - b/c it gets overwritten.
skillsBtns[23] = skillsBtns[20];
//For some reason the position of [20] doesn't move with rest of buttons, manually set for now.
skillsBtns[23]->x = skillsBtns[23]->x + 124;
EDIT: ui_char.cpp -> UiCharImpl::SkillsBtnRender() - add after setting auto idx (after ln 2463):
//Restore button[20] from button[23]
skillsBtns[20] = skillsBtns[23];

Known Issues: I don't really like the test solution I came up with for this - I suspect my 'backup of the button' will get overwritten by something I haven't activated yet. I just viewed a list of buttons and saw the [23] was initialized but empty so I reappropriated it thinking if/when [23] does active it will either crash or my 21st button would disappear - not really a good solution. Also, the additional 21st button doesn't line up properly with the other buttons, so I have to go in and change its position in C++ when I reposition the Skills window - this may push it offscreen depending on where you have skills window positioned (this can make it looks like it didn't get added, check Debug Info for its position, or output log to see if anything was offscreen and didn't render).

Note: Ideally it would be possible to show all the skills the engine supports. I had Shattered Gates installed for part of my testing and really want to figure out how to show all of the additional skills it includes (and my top desire is to use the bag of holding or any bag in the main campaign).

 

[ryefish]

And Finally: How to center dialog history
and a few other things I've changed or am trying to change

I haven't found a .mes file for ...\interface\dialog\ and I didn't see any effect when I tried to fabricate one from scratch. I still don't really understand how the mes files work, when where they get read, where they came from, and whether I can make new ones (same question with .img files) - more research I hope to do. Also, even if there were a mes file I haven't see any that 'center', they all appear to offset from left/right/top/bottom. Either way, I suspect this is a know fix since its in the x position of dialog history window is set in a C++ file.

EDIT: ui_dialog.cpp->UiDialogImpl::WidgitsInit() - change dlgWnd x position (ln 449)
//This repositions the dialog history window
LgcyWindow dlgWnd(9, h - 374, 611, 292); -> LgcyWindow dlgWnd(359, h - 374, 611, 292);
EDIT: ui_dialog.cpp->UiDialogImpl::ResponseWidgetsInit() - change responseWnd x position (ln 600)
//This repositions the dialog Q/A window
static LgcyWindow responseWnd(16, h - 221, 611, 139); -> static LgcyWindow responseWnd(366, h - 221, 611, 139);

Known Issues: I am currently using a hardwired x off values, this should be replaced with something like (sceen_width/2 - window_width/2).

Other things I've been able to change:
Some display fields are 'centered' or 'left-aligned', like in ui_char.cpp -> UiCharHooks::ClassLevelBtnRender and UiCharHooks::AlignGenderRaceBtnRender. I've also been overriding these in things directly in ui_char.cpp, not sure if these is a better way to accomplish these kind of layout things without editing c++ files. Similarly some things are truncated when they reach a certain length, these appear changeable to adapt to new width.

Things I was unable to change:
I tried moving the combat icons that display at the top over by ~50 pixels, I had hoped to move a shrunk version of the turn indicator so it would sit to the left of the combatant icons (which I think max out at 2 rows). This failed: the value in the .mes appears to set the spacing b/w character icons, not the distance from left side, and the distance from the left side keeps changing depending on how many characters are displayed. I suspect it is doing some calculation to maximize buttons show or something. Not a big issue, was just an experiment.

Things I'm looking into changing:
It appears the 'help' popup and the 'hit/damage' popup (from clicking blue links in the rollhistory window) are using the same window. I want to identify which way it is being used and change its x position accordingly: I'd like 'help' centered and 'hit/damage' to 'dock' right next to the roll history, which I've expanded in length and centered on the right side. Similarly I'm setting up the centered 'dialog' window to 'dock' directly above the party member windows (which I've also ~ centered and redesigned so damage is on top, nonlethal is its own bar, conditions sit below debuffs not over the character image, but that's all just repositioning)....

Note:
When I found this thread I also found a related thread on co8:
Improving ToEE's User Interface: https://co8.org/community/threads/improving-toees-user-interface.9250/
This includes a lot of info about changes Co8 made to the UI as well as design decisions.
Is the user interface they made along with this post for the most part the current UI?

Ok, this ended up being a lot longer than I had anticipated. I hope that my experimental fixes are in any way useful to others and that my presentation has been clear and I haven't make any test breaking typos in the experimental fixes: if I have please let me know. I am open to any and all feedback: are there already other ways to accomplish what I've been working on? If any one else will to test out these modifications please let me know how it goes?

What I would really like to figure out is some way to have these UI changes bundled together in a way that it can be activated via the Temple+ Configuration, something like a checkbox that says 'Extended UI' or something.

Thanks again for the excellent mod and tools!

 

[Sitra Achara]

Hi ryefish! Glad you found the tools useful.

Your modifications are very interesting (can you post a screenshot?). To state the obvious it needs to be properly implemented rather than thru hacks if it is to be included in the official release. For one, use proper window handles rather than explicit numeric IDs (which are not guaranteed). They're usually stored somewhere in memory, just need to find the address for each one.

I'm not sure what the effect of showing all the char UI sub windows would be. I'll have to check if anything refers to the current state. It's good that you tested it but you could be missing edge cases, subtle bugs, unexpected flows etc. I'll have to get back to you - which may take a while honestly since there's quite the queue already...

As for how to toggle it - On the one hand I'd like to make it as generic and data driven as possible, but OTOH I want to start shifting the focus to OpenTemple so I don't want to overdo it. "single char UI" checkbox or sthg to that effect is the preferred option then.

Regarding how the assets are loaded, there's a ton of ui init functions that are not hooked by Temple+. I can send you the decompiled code so you can look stuff up if you're interested. You can also add your own init func if you want to add stuff rather than modify existing assets.

To go forward, how about you clone the repository and start a branch - this will also help keep track of what you've added. It would also make it easier to go step by step.

 

[ryefish]

Thanks for the excellent feedback, I'll get a fork setup and commit my different changes as separate commits, then let you know (could take me a few days).

First I think I'll focus on the test fix for 'expanding window sizes' - just expanding the windows, not adding simultaneous views. I'd like to get that fully confirmed, since my other changes are built assuming that works. I think I should do a clearer write up or a short video showing how to experiment with it. I'm somewhat limited on what I can test given my constrained video resolution and wanting to not encounter spoilers, so I'm really hoping this grabs someones attention who is willing to do additional tests with it. In general, do most people who monitor Co8 forum also monitor this? Or would you recommend I cross post information about expanding window sizes there as well?


For now here are a couple screenshots taken at 1280x1024 - that's the max resolution of my monitors so I have no way to test widescreen or high-res. Also: maybe because I'm running at 5:4 aspect ratio I get a 1:1 relationship b/w the sizes/positions I set for .mes and .img and .tga and what I see in game, I don't know if that is always the case.

Note: I'm using a single-pixel colored wire frame to outline my images, this to ease positioning.

This is a game play screenshot of the experimental full screen Town Map:

The buttons all work I think: I can jump between maps, center on party (can't really center but does jump), add/del notes/zoom.

This is a game play screenshot of the experimental expanded character sheet:

Again: the buttons all appear to work. Mouseovers, changing spell tabs, enchanting weapons, swapping items/loadouts, rotating 3d model, 3d model auto-updates when loadout changes. I don't know of any functional problems with this (other than NPC-PCs not having/using portrait image so the image gets expanded/distorted).


And lastly for now, this is a MOCKUP of the direction I've been trying to head:

I could write a long blog just about the changes I'm trying to make here (like standardizing item slot sizes) but I'll only point out 2 things: 1) I'm trying to ensure I'll have space for an inventory 'bag' (I really liked what Shattered Gates includes and hope to be able to use that in the main campaign), I hope. 2) I'd like to convert the Skills bonus calculator to display as too ltip.

Ok, I hope these screenshots help to clarify some of the changes that I'm experimenting with and something like what I envision. Let me know if you have any additional questions or recommendations. Thanks again.

 

[ryefish]

Two more things:

First: The primary reason my current char sheet looks so different from the my target is how time consuming it is to move all the buttons (my understanding is they have to be moved individually). So I'm going to reposition everything in one pass, after I have resolved all technical issues and after I have all of my ancillary windows polished, resized, and positioned (like dialog/rollhistory/party).

Second: I forget to mention that the char sheet also works with quick switching party members (left click a different character icon while in inventory screen).

 

[Sitra Achara]

Cool! It's a bit too crowded for my tastes but to each their own. Have you thought about how to adapt the art?

Here's a link to the decompiled code for your reference:
https://fil.email/LwNXiYVK
You can look up how assets are loaded and the various UI inits not covered by Temple+. Feel free to recreate any.
About buttons: they're usually at least defined relative to their parent window (though it can be baked in and not dynamically adjusted to the current window coords). A somewhat less hacky way would be to iterate the window's children and move them together with the window in such cases, rather than sisyphically going one by one.

BTW about bags: I don't know if you're noticed but toee has a backpack icon, and the engine has traces of various additional 'bags', probably for stuff like scrolls and such. One if those things I'd like to get around to eventually...

 

[ryefish]

Yeah - too busy for my taste as well. I have experimented with using the existing buttons to bring up the spells independently window in a toggle fashion (for instance - related to the 'flags' field). But for now my goal is show 'all', find issues/limits, then back off. Reading back past posts I had thought expanding the UI is a core objective, and I have found no other working approach - yet I have found many posts related to scaling the UI - all answers say its not possible. Am I overlooking something? Is there another replaceable way to change the UI window sizes? Please let me know, modding ToEE is somewhat confusing.

These are a couple posts from the c08 UI redesign:

https://co8.org/community/threads/improving-toees-user-interface.9250/page-2
"What we can't do: ... Change the sizes of various interface boxes to accomodate larger text. (At least I haven't figured it out yet. If we can't, some of the other ui changes will have to go because we'll have text spilling out all over the place.)"

"Can't remember if I mentioned this before ... the basic problem here that makes redesigning the ui necessary (assuming the use of the Priory font) is that the basic box can't be enlarged. (At least I can't figure it out.) This means that instead we have to make better use of the existing box by reclaiming whatever idle real estate we can, hence the move to make the text box bigger and eat up the space between it and the inventory box, etc. Hope that makes sense."​

I know it has been many years, but I want to determine if I have figured out how to 'enlarge the basic box'?

 

[ryefish]

And thank you for providing me with the 'decompiled code' - I am eager to learn more about how to leverage it.

--although I am not sure what I am looking at, it looks very interesting. And I am very glad - I thought this might be assembly, but this is c I think. Easier to grok.

 

[Sitra Achara]

Yeah - too busy for my taste as well. I have experimented with using the existing buttons to bring up the spells independently window in a toggle fashion (for instance - related to the 'flags' field). But for now my goal is show 'all', find issues/limits, then back off. Reading back past posts I had thought expanding the UI is a core objective, and I have found no other working approach - yet I have found many posts related to scaling the UI - all answers say its not possible. Am I overlooking something? Is there another replaceable way to change the UI window sizes? Please let me know, modding ToEE is somewhat confusing.

These are a couple posts from the c08 UI redesign:

https://co8.org/community/threads/improving-toees-user-interface.9250/page-2
"What we can't do: ... Change the sizes of various interface boxes to accomodate larger text. (At least I haven't figured it out yet. If we can't, some of the other ui changes will have to go because we'll have text spilling out all over the place.)"

"Can't remember if I mentioned this before ... the basic problem here that makes redesigning the ui necessary (assuming the use of the Priory font) is that the basic box can't be enlarged. (At least I can't figure it out.) This means that instead we have to make better use of the existing box by reclaiming whatever idle real estate we can, hence the move to make the text box bigger and eat up the space between it and the inventory box, etc. Hope that makes sense."​

I know it has been many years, but I want to determine if I have figured out how to 'enlarge the basic box'?

Yeah, as you pointed out this post is very old (almost 10 years), whereas Temple+ and our reversing of the temple didn't start until 2015. So yeah, whatever info you find in co8 about mod limitations is very much outdated.

Something else to keep in mind - I think what most people would really like is bigger fonts to go along with higher res. Co8 has enlarged the fonts from size 10 to 12, which is enough for me but perhaps people playing on 1080p or higher would appreciate a 14pt font. Up to you whether to tackle that.

 

[Sitra Achara]

And thank you for providing me with the 'decompiled code' - I am eager to learn more about how to leverage it.

--although I am not sure what I am looking at, it looks very interesting. And I am very glad - I thought this might be assembly, but this is c I think. Easier to grok.

This is what the IDA decompilation spits out after analyzing the assembly code, so definitely easier than assembly but not quite 'human' c code.
You can however find the memory addtrsses where the widget IDs are stored. You can follow the trail by searching the fioe and folder names where the assets are stored, e.g. ui_char/something.mes.