Age of Empires 4 [Engine: Essence 5.0]

[SunBeam]

[ 04 Nov 21 - Hit #2 ]

Based on the [ Hit #1 ] section below:

RelicCardinal.CT

1.0

(18.61 KiB) Downloaded 1049 times

Which can be used as follows:

• Open Cheat Engine 7.1+, target the game (File > Open Process > Processes tab > RelicCardinal.exe) and open/load the table you've just downloaded.

• Activate [ Enable ] script. Then activate Toggle Console UDF script and this window will show up:
  
  
  
  • white input memo box at the bottom is used to type in text:
    • you can use Enter to move to next line and type some more
    • Shift+Enter key combination will execute what you've typed
    • no, you can't enlarge the memo box to your desired sizes
    • you can scroll it though horizontally and vertically
    • I recommend writing the script in Notepad++, copy-pasting it here, then Shift+Enter
  • purple memo box is used to display what you've executed
    • it doesn't work like a "return result" log-type of window (just prints a copy of your input)
  • Clear button
    • clears the purple memo box
  • Close [x] button
    • will close the UDF window and deactivate "Toggle Console UDF" script

What you can run with this UDF is shown in the next post.

Credits:

• Norway-_-1999

• Zanzer (for all your UDF help)

• < online communities with old research: UC, guru3D, etc. >

[ 02 Nov 21 - Hit #1 ]

Game Name: Age of Empires 4
Game Vendor: Steam
Game Version: 5.0.7274.0
Game Process: RelicCardinal.exe
File Version: 5.0.7274.0



Hello folks. Without any further ado, listing what this topic covers and the knowledge behind it.


[ Game Engine, Data Files ]


Was curious to know which Engine this game was built with. I didn't even look at the developer name. Google says: "Essence Engine". A more in-depth search shows version 5.0 (wiki). Then I noticed the developer: Relic Entertainment. As many of you know, they're also the ones who've created Company of Heroes (1, 2, 3).

Then I opened up the executable in x64dbg and searched for all string references. I then wanted to see if there's any common words like "cheat", "god", etc. and found this:

Code: Select all

00007FF6E9DF52DE | 4C:8D3D 1B83D304 | LEA R15,QWORD PTR DS:[7FF6EEB2D600] | 00007FF6EEB2D600:"data:dev/cheatmenu.lua"

So.. the game is using Lua Which means that, pretty much like in Watch Dogs or Far Cry, we can hook it and execute our own Lua commands. We can also dump a list of all internal commands. Pretty much like in this topic: viewtopic.php?f=4&t=17693.

Back to that "data:dev/cheatmenu.lua" -> looking at that word "data" I deducted that it should be inside some data file. And checking through the folder hierarchy, I found this:



Then I opened up that "sga" file in a hex editor and saw this:



Then I googled "zenhax age of empires 4" and found this: [Link].

So it turns out the [Link] for Company of Heroes 3 also works for this title. And with this viewer in hand I've opened up Data.sga to see this:



Peachy, eh? So while fuckers are battling over who's got the bigger e-penor or insinuating others stole from them, I do this practical crap. Fun times


[ Lua, Baby! ]


In order to run commands inside game's Lua Engine we want to find several requisites:

• Lua version

• Lua state

• several Lua APIs that help with string/script execution (hence why we need to know the first bullet)

After those are determined, you can google up "lua binaries download" and find the ones matching your version. The version doesn't need to be matched 1:1, but you need to have good skills at spotting where exactly are these Lua APIs inlined in the game code. If you take a look inside the game folder you will see there's no "luaX.X.dll" (e.g.: lua5.1.dll) there, so this means whatever's used from Lua is part of the game's code (was compiled as part of the code == inlined).


[ Version ]


Looked through the string references aaaaand:



Then browsed it aaaand:



So all we need to find now is a 5.3 build of Lua somewhere on the net, preferably with debug symbols. The game was most likely compiled with MSVC2017/2019, so maybe we're lucky. There we go: [Link]. What will happen next is we'll use arrays of bytes (signatures) from these Lua APIs to find them in our binary (RelicCardinal.exe)


[ Compile Time! ]


Figured why not. If you have a Visual Studio installation (2017/2019), you can follow this small tutorial to quickly compile Lua 5.3.6. That's the version I'll be looking at and cross-referencing it with our game.

1) Go to [Link] and get [Link].
2) Extract the content to your Desktop:



3) Start > x64 Native Tools Command Prompt for VS 2017
4) Navigate to your Lua src folder (e.g.: cd C:\Users\SunBeam\Desktop\lua-5.3.6\src).
5) Run each of these commands in the console (copy them, right-click to execute):

Code: Select all

cl /MD /O2 /c /DLUA_BUILD_AS_DLL *.c
ren lua.obj lua.o
ren luac.obj luac.o
link /DLL /IMPLIB:lua5.3.0.lib /OUT:lua5.3.0.dll *.obj

If you want to get the .pdb, just add /DEBUG:FULL at the end of last line.

And you'll get these:



Credits: [Link] (adjusted it myself to build the x64 DLL)

There you have it: how to compile Lua uber-fast


[ Lua State ]


Took me a bit of research, as Lua state wasn't that easy to spot. More precisely, plural: states. The game Engine initializes 2-3 states based on scope, all of them being acquired via this function:

Code: Select all

RelicCardinal.exe+402428 - 48 89 5C 24 08        - mov [rsp+08],rbx
RelicCardinal.exe+40242D - 89 54 24 10           - mov [rsp+10],edx
RelicCardinal.exe+402431 - 57                    - push rdi
RelicCardinal.exe+402432 - 48 83 EC 30           - sub rsp,30 { 48 }
RelicCardinal.exe+402436 - 41 B8 04000000        - mov r8d,00000004 { 4 }
RelicCardinal.exe+40243C - 48 8D 54 24 48        - lea rdx,[rsp+48]
RelicCardinal.exe+402441 - 48 8B D9              - mov rbx,rcx
RelicCardinal.exe+402444 - E8 67875F00           - call RelicCardinal.exe+9FABB0
RelicCardinal.exe+402449 - 4C 8B C8              - mov r9,rax
RelicCardinal.exe+40244C - 48 8D 4B 18           - lea rcx,[rbx+18]
RelicCardinal.exe+402450 - 4C 8D 44 24 48        - lea r8,[rsp+48]
RelicCardinal.exe+402455 - 48 8D 54 24 20        - lea rdx,[rsp+20]
RelicCardinal.exe+40245A - E8 29000000           - call RelicCardinal.exe+402488
RelicCardinal.exe+40245F - 48 8B 40 08           - mov rax,[rax+08]
RelicCardinal.exe+402463 - 48 85 C0              - test rax,rax
RelicCardinal.exe+402466 - 74 15                 - je RelicCardinal.exe+40247D
RelicCardinal.exe+402468 - 48 3B 43 20           - cmp rax,[rbx+20]
RelicCardinal.exe+40246C - 74 15                 - je RelicCardinal.exe+402483
RelicCardinal.exe+40246E - 48 8B 40 18           - mov rax,[rax+18]
RelicCardinal.exe+402472 - 48 8B 5C 24 40        - mov rbx,[rsp+40]
RelicCardinal.exe+402477 - 48 83 C4 30           - add rsp,30 { 48 }
RelicCardinal.exe+40247B - 5F                    - pop rdi
RelicCardinal.exe+40247C - C3                    - ret 
RelicCardinal.exe+40247D - 48 8B 43 20           - mov rax,[rbx+20]
RelicCardinal.exe+402481 - EB E5                 - jmp RelicCardinal.exe+402468
RelicCardinal.exe+402483 - 33 C0                 - xor eax,eax
RelicCardinal.exe+402485 - EB EB                 - jmp RelicCardinal.exe+402472

An example is here:

Code: Select all

RelicCardinal.exe+377A6F - BA 52414353           - mov edx,53434152 { "RACS" }
RelicCardinal.exe+377A74 - 48 8B 0D 35E3B107     - mov rcx,[RelicCardinal.exe+7E95DB0] { (24AB9092E30) }
RelicCardinal.exe+377A7B - E8 A8A90800           - call RelicCardinal.exe+402428

If you take a look at the parameter in edx, it stands for "SCAR". This is the "engine", so to speak. Then.. finding the references to this function will lead you to spots where edx is "CHTM" and "GLUA". Meaning CheatMenu and GlobalLua. These are the several consoles you will see in-game when you press Ctrl+Shift+Tilde(`~), Ctrl+Shift+1 and Ctrl+Shift+3 (Global] and CheatMenu]). Additionally to that there's the SCAR console which you can access only via applying in your Steam client the -dev command line argument. Then open it pressing Alt+Shift+Tilde(~`) and you'll see the SCAR] prompt. This will let you run SOME Lua commands/functions.

You can find more information @ these locations:

• [Link]

• [Link]

[ Lua APIs ]


In order to execute some Lua string/command or buffer (script), we'll need a certain execution flow built out of Lua APIs: the Lua state pointer, luaL_loadbuffer (or equivalent) and lua_pcall (or equivalent). In this game the Lua version is 5.3 so we need the "x" version of those APIs, as per the documentation here: [Link]. In short:

• LUALIB_API int luaL_loadbufferx (lua_State *L, const char *buff, size_t size, const char *name, const char *mode)

• LUA_API int lua_pcallk (lua_State *L, int nargs, int nresults, int errfunc, lua_KContext ctx, lua_KFunction k)

What do I mean with this is finding them in the game's executable code. Where they're inlined. Using our 5.3 compiled DLL and aob signatures - what I've done was to copy a certain number of bytes in my DLL and search them in RelicCardinal.exe to get to the inlined Lua code - I found them here:

Code: Select all

<< luaL_loadbufferx >>
RelicCardinal.exe+342968C - 4C 8B DC              - mov r11,rsp
RelicCardinal.exe+342968F - 48 83 EC 48           - sub rsp,48 { 72 }
RelicCardinal.exe+3429693 - 48 8B 44 24 70        - mov rax,[rsp+70]
RelicCardinal.exe+3429698 - 49 89 53 E8           - mov [r11-18],rdx
RelicCardinal.exe+342969C - 48 8D 15 D1151BFD     - lea rdx,[RelicCardinal.exe+5DAC74] { (138578760) }
RelicCardinal.exe+34296A3 - 4D 89 43 F0           - mov [r11-10],r8
RelicCardinal.exe+34296A7 - 4D 8D 43 E8           - lea r8,[r11-18]
RelicCardinal.exe+34296AB - 49 89 43 D8           - mov [r11-28],rax
RelicCardinal.exe+34296AF - E8 08B11BFD           - call RelicCardinal.exe+5E47BC
RelicCardinal.exe+34296B4 - 48 83 C4 48           - add rsp,48 { 72 }
RelicCardinal.exe+34296B8 - C3                    - ret 

Code: Select all

<< lua_pcallk >>
RelicCardinal.exe+5E49E0 - 48 89 5C 24 08        - mov [rsp+08],rbx
RelicCardinal.exe+5E49E5 - 48 89 6C 24 10        - mov [rsp+10],rbp
RelicCardinal.exe+5E49EA - 48 89 74 24 18        - mov [rsp+18],rsi
RelicCardinal.exe+5E49EF - 57                    - push rdi
RelicCardinal.exe+5E49F0 - 48 83 EC 40           - sub rsp,40 { 64 }
RelicCardinal.exe+5E49F4 - 33 F6                 - xor esi,esi
RelicCardinal.exe+5E49F6 - 41 8B E8              - mov ebp,r8d
RelicCardinal.exe+5E49F9 - 44 8B DA              - mov r11d,edx
RelicCardinal.exe+5E49FC - 48 8B F9              - mov rdi,rcx
RelicCardinal.exe+5E49FF - 45 85 C9              - test r9d,r9d
RelicCardinal.exe+5E4A02 - 0F84 81000000         - je RelicCardinal.exe+5E4A89
RelicCardinal.exe+5E4A08 - 41 8B D1              - mov edx,r9d
RelicCardinal.exe+5E4A0B - E8 44470B00           - call RelicCardinal.exe+699154
RelicCardinal.exe+5E4A10 - 4C 8B D0              - mov r10,rax
RelicCardinal.exe+5E4A13 - 4C 2B 57 38           - sub r10,[rdi+38]
RelicCardinal.exe+5E4A17 - 4C 8B 4F 10           - mov r9,[rdi+10]
RelicCardinal.exe+5E4A1B - 41 8D 43 01           - lea eax,[r11+01]
RelicCardinal.exe+5E4A1F - 48 63 C8              - movsxd  rcx,eax
RelicCardinal.exe+5E4A22 - 48 8B 44 24 78        - mov rax,[rsp+78]
RelicCardinal.exe+5E4A27 - 48 C1 E1 04           - shl rcx,04 { 4 }
RelicCardinal.exe+5E4A2B - 4C 2B C9              - sub r9,rcx
RelicCardinal.exe+5E4A2E - 4C 89 4C 24 30        - mov [rsp+30],r9
RelicCardinal.exe+5E4A33 - 48 85 C0              - test rax,rax
RelicCardinal.exe+5E4A36 - 0F85 B0CE6902         - jne RelicCardinal.exe+2C818EC
RelicCardinal.exe+5E4A3C - 4C 2B 4F 38           - sub r9,[rdi+38]
RelicCardinal.exe+5E4A40 - 4C 8D 44 24 30        - lea r8,[rsp+30]
RelicCardinal.exe+5E4A45 - 48 8D 15 CCDF0A00     - lea rdx,[RelicCardinal.exe+692A18] { (138578756) }
RelicCardinal.exe+5E4A4C - 89 6C 24 38           - mov [rsp+38],ebp
RelicCardinal.exe+5E4A50 - 48 8B CF              - mov rcx,rdi
RelicCardinal.exe+5E4A53 - 4C 89 54 24 20        - mov [rsp+20],r10
RelicCardinal.exe+5E4A58 - E8 37000000           - call RelicCardinal.exe+5E4A94
RelicCardinal.exe+5E4A5D - 8B F0                 - mov esi,eax
RelicCardinal.exe+5E4A5F - 83 FD FF              - cmp ebp,-01 { 255 }
RelicCardinal.exe+5E4A62 - 75 0E                 - jne RelicCardinal.exe+5E4A72
RelicCardinal.exe+5E4A64 - 48 8B 47 20           - mov rax,[rdi+20]
RelicCardinal.exe+5E4A68 - 48 8B 4F 10           - mov rcx,[rdi+10]
RelicCardinal.exe+5E4A6C - 48 39 48 08           - cmp [rax+08],rcx
RelicCardinal.exe+5E4A70 - 72 1C                 - jb RelicCardinal.exe+5E4A8E
RelicCardinal.exe+5E4A72 - 48 8B 5C 24 50        - mov rbx,[rsp+50]
RelicCardinal.exe+5E4A77 - 8B C6                 - mov eax,esi
RelicCardinal.exe+5E4A79 - 48 8B 74 24 60        - mov rsi,[rsp+60]
RelicCardinal.exe+5E4A7E - 48 8B 6C 24 58        - mov rbp,[rsp+58]
RelicCardinal.exe+5E4A83 - 48 83 C4 40           - add rsp,40 { 64 }
RelicCardinal.exe+5E4A87 - 5F                    - pop rdi
RelicCardinal.exe+5E4A88 - C3                    - ret 
RelicCardinal.exe+5E4A89 - 4C 8B D6              - mov r10,rsi
RelicCardinal.exe+5E4A8C - EB 89                 - jmp RelicCardinal.exe+5E4A17
RelicCardinal.exe+5E4A8E - 48 89 48 08           - mov [rax+08],rcx
RelicCardinal.exe+5E4A92 - EB DE                 - jmp RelicCardinal.exe+5E4A72
RelicCardinal.exe+5E4A94 - 48 8B C4              - mov rax,rsp
RelicCardinal.exe+5E4A97 - 48 89 58 08           - mov [rax+08],rbx
..
..

Based on Norway-_-1999's script loader for Far Cry series I was able to adapt for Lua 5.3 the functions that load/run a Lua string and load/run a Lua script (buffer) so that Visual Studio 2017 would compile them to usable ASM:

Code: Select all

[ENABLE]

alloc( LuaFuncs, 0x1000 )
registersymbol( LuaFuncs )

label( StringExec )
registersymbol( StringExec )
label( szNull )
label( @L00000001 )
label( @L00000002 )

label( ScriptExec )
registersymbol( ScriptExec )
label( @L00000003 )

label( _get_state )

LuaFuncs:
db 90
align 10 CC

StringExec:
mov [rsp+8],rbx
push rdi
sub rsp,30
mov rbx,rcx
call _get_state
mov rdi,rax
test rax,rax
je short @L00000002
mov r8,-1

@L00000001:
inc r8
cmp byte ptr [rbx+r8],0
jne short @L00000001
mov qword ptr [rsp+20],0
mov r9,szNull
mov rdx,rbx
mov rcx,rdi
call _luaL_loadbufferx
test eax,eax
jne short @L00000002
mov qword ptr [rsp+28],0
mov qword ptr [rsp+20],0
xor r9d,r9d
lea r8d,[rax-1]
xor edx,edx
mov rcx,rdi
call _lua_pcallk
test eax,eax
jne short @L00000002
mov al,1
mov rbx,[rsp+40]
add rsp,30
pop rdi
ret

@L00000002:
mov rbx,[rsp+40]
xor al,al
add rsp,30
pop rdi
ret

align 10 CC

szNull:
dq 0

align 10 CC

ScriptExec:
mov [rsp+8],rbx
push rdi
sub rsp,30
mov rdi,rcx
call _get_state
mov rbx,rax
test rax,rax
je short @L00000003
xor r8d,r8d
mov rdx,rdi
mov rcx,rax
call _luaL_loadfilex
test eax,eax
jne short @L00000003
xor eax,eax
mov [rsp+28],rax
mov [rsp+20],rax
xor r9d,r9d
lea r8d,[rax-1]
xor edx,edx
mov rcx,rbx
call _lua_pcallk
test eax,eax
jne short @L00000003
mov al,1
mov rbx,[rsp+40]
add rsp,30
pop rdi
ret

@L00000003:
mov rbx,[rsp+40]
xor al,al
add rsp,30
pop rdi
ret

align 10 CC

_get_state:
sub rsp,28
mov edx,53434152
mov rcx,RelicCardinal.exe+7E95DB0
mov rcx,[rcx]
call RelicCardinal.exe+402428
lea rax,[rax+30]
mov rax,[rax]
mov rax,[rax+8]
add rsp,28
ret

align 10 CC

[DISABLE]

unregistersymbol( ScriptExec )
unregistersymbol( StringExec )
dealloc( LuaFuncs )
unregistersymbol( LuaFuncs )

Which leads to the table up top (Hit #2) and the Lua commands in the next post Enjoy!

BR,
Sun

How to use this cheat table?

1. Install Cheat Engine
2. Double-click the .CT file in order to open it.
3. Click the PC icon in Cheat Engine in order to select the game process.
4. Keep the list.
5. Activate the trainer options by checking boxes or setting values from 0 to 1

[SunBeam]

Reveal/Unfog Map

Code: Select all

FOW_UIRevealAll()

If you want the shroud back again, then:

Code: Select all

FOW_UIUnRevealAll()

But note that this does not restore the unveiled map fog, just makes it so whatever's under the 'transparent' layer is not updated/shown to you. The mini-map though is correctly shrouded back.

* * *

[dec1337]

I'm using the gamepass version and don't have access to the game data since it's in the WindowsApps folder, so I can't check myself but did you notice any text string references within the cheatmenu.lua file?
They probably won't work yet but AoE usually just has cheats you can activate by hitting enter and typing in a command e.g. "steroids" was instant build in the original game

[SunBeam]

^ It is a different Engine, so no, old shit won't work. What does work, but they're not called 100% the same (the Lua functions) are commands from Company of Heroes 2. See this Steam page: [Link]. Enable the -dev command line argument in Steam client, then Alt+Shift+~ and copy-paste the population cap function listed there:

Code: Select all

Player_SetPopCapOverride(Game_GetLocalPlayer(), 1000)

I was hoping to do more with just Lua itself. But it seems like this Engine is over-convoluted.

Cheers,
Sun

[isosplx]

...

Thank you Sunbeam!
You have always been thorough and helpful when explaining methods and processes.

[Csimbi]

Impressive work SunBeam!
I don't have the game, like the game or want the game, but it was still a great read.
I wish we have more people contributing 'practical crap'.

[SunBeam]

I'm currently mapping some internal functions using CoH2 information. Will only do it for this version of the game (so just this existing Steam binary), as porting the labels over to future updates makes no sense to me. Relic decided to use some code mutation in some core functions (such as the one processing Console variables - add, remove, etc. - so that's more reason not to continue in game updates). The cheats that were implemented in CoH2 (cheat_buildtime, cheat_revealall, etc.) are not compiled in this game, thus not registered to the whole Lua/console flow.

[Csimbi]

Sounds like you are arming yourself for when CoH3 hits the stores

[RaMzZzEsll]

Don't work

[SunBeam]

Tough luck.

[ivindel]

Reveal/Unfog Map

Code: Select all

FOW_UIRevealAll()

If you want the shroud back again, then:

Code: Select all

FOW_UIUnRevealAll()

But note that this does not restore the unveiled map fog, just makes it so whatever's under the 'transparent' layer is not updated/shown to you. The mini-map though is correctly shrouded back.

* * *

Hi SunBeam, I know it's kind of a long stretch, but I was wondering if you were able to isolate which part of the game code/function is responsible for the LUA command for FOW_UIRevealAll()?
I am pretty new to this and trying to make sense and learn these on my own.

Thank you and have a nice day!

[SunBeam]

Please re-read my first post. I am explaining pretty clearly and direct there how to get to the Lua files inside the game's data archives. You can then extract them and do a Ctrl+F yourself.

[Rely5920]

I hope to get 9369 version of single machine defogging

[HeartUnderBlade]

I follow the instruction, update the lua_state function to correct offset, make sure the lua_api is scanned by aob scan and try to type some lua command into. The result is if I type something wrong, nothing happened. If I type something correct, for example, add 200 food to player, the game will crash. Is this a script loader problem or I misunderstanding the instruction?