How To Register As A Ceo In Gta Five

GTA Online. Infamous for its slow loading times. Having picked upward the game once more to finish some of the newer heists I was shocked (/s) to discover that it still loads but equally dull equally the mean solar day it was released 7 years ago.

It was time. Time to get to the bottom of this.

Recon

First I wanted to bank check if someone had already solved this problem. About of the results I constitute pointed towards anecdata almost how the game is so sophisticated that it needs to load so long, stories on how the p2p network compages is rubbish (not saying that information technology isn't), some elaborate ways of loading into story manner and a solo session after that and a couple of mods that allowed skipping the startup R* logo video. Some more reading told me we could relieve a whopping 10-30 seconds with these combined!

Meanwhile on my PC…

Benchmark

1 2 3 four 5 6 7 8

Story manner load time: ~1m 10s Online manner load time: ~6m flat Startup carte du jour disabled, fourth dimension from R* logo until in-game (social gild login fourth dimension isn't counted). Onetime but decent CPU: AMD FX-8350 Cheap-o SSD: KINGSTON SA400S37120G We have to have RAM: 2x Kingston 8192 MB (DDR3-1337) 99U5471 Good-ish GPU: NVIDIA GeForce GTX 1070

I know my setup is dated but what on earth could take 6x longer to load into online mode? I couldn't measure out whatever difference using the story-to-online loading technique as others have found before me. Even if it did piece of work the results would be down in the noise.

I Am (Not) Lonely

If this poll is to be trusted then the issue is widespread enough to mildly annoy more than than 80% of the player base. Information technology's been 7 years R*!

Looking around a fleck to find who are the lucky ~20% that get sub 3 minute load times I came across a few benchmarks with high-end gaming PCs and an online mode load fourth dimension of about 2 minutes. I would kill hack for a ii minute load time! It does seem to be hardware-dependent only something doesn't add up here…

How come up their story mode yet takes almost a minute to load? (The M.two ane didn't count the startup logos btw.) Besides, loading story to online takes them only a minute more while I'yard getting about five more than. I know that their hardware specs are a lot better merely surely non 5x better.

Highly accurate measurements

Armed with such powerful tools as the Chore Manager I began to investigate what resource could be the bottleneck.

After taking a infinitesimal to load the common resources used for both story and online modes (which is nigh on par with loftier-end PCs) GTA decides to max out a single core on my auto for four minutes and do nothing else.

Disk usage? None! Network usage? There's a bit, merely it drops basically to zero after a few seconds (apart from loading the rotating info banners). GPU usage? Zero. Memory usage? Completely flat…

What, is it mining crypto or something? I smell lawmaking. Really bad code.

Single thread-bound

While my old AMD CPU has 8 cores and information technology does pack a dial, it was made in the olden days. Back when AMD'southward unmarried-thread performance was manner backside Intel'due south. This might not explain all of the load time differences but it should explain most of it.

What'south odd is that it's using up just the CPU. I was expecting vast amounts of deejay reads loading upwardly resource or loads of network requests trying to negotiate a session in the p2p network. But this? This is probably a bug.

Profiling

Profilers are a smashing way of finding CPU bottlenecks. There's just one problem - most of them rely on instrumenting the source code to get a perfect picture of what's happening in the process. And I don't have the source code. Nor practise I need microsecond-perfect readings - I have 4 minutes' worth of a bottleneck.

Enter stack sampling: for closed source applications there'south only one option. Dump the running process' stack and current education pointer'due south location to build a calling tree in set intervals. Then add them up to get statistics on what's going on. In that location's only 1 profiler that I know of (might be ignorant here) that tin practise this on Windows. And it hasn't been updated in over ten years. It's Luke Stackwalker! Someone, please give this projection some love :)

Usually Luke would group the same functions together but since I don't have debugging symbols I had to eyeball nearby addresses to guess if it's the same place. And what practice we see? Non ane bottleneck but 2 of them!

Downward the rabbit hole

Having borrowed my friend'due south completely legitimate copy of the industry-standard disassembler (no, I really tin can't afford the affair… gonna learn to ghidra one of these days) I went to accept GTA autonomously.

That doesn't look right at all. Most loftier-profile games come with built-in protection against contrary engineering science to keep away pirates, cheaters, and modders. Not that it has ever stopped them.

At that place seems to be some sort of an obfuscation/encryption at play here that has replaced most instructions with gibberish. Non to worry, we simply need to dump the game's memory while information technology's executing the part we want to await at. The instructions accept to be de-obfuscated before running one style or another. I had Procedure Dump lying around, so I used that, but there are plenty of other tools bachelor to do this sort of thing.

Problem i: Information technology's… strlen?!

Disassembling the now-less-obfuscated dump reveals that one of the addresses has a label pulled out of somewhere! It's strlen? Going downwardly the call stack the next one is labeled vscan_fn and afterwards that the labels end, tho I'g fairly confident information technology's sscanf.

It's parsing something. Parsing what? Untangling the disassembly would have forever so I decided to dump some samples from the running procedure using x64dbg. Some debug-stepping after it turns out information technology'south… JSON! They're parsing JSON. A whopping x megabytes worth of JSON with some 63k item entries.

i 2 3 four 5 vi seven viii 9 10 11

..., { "key": "WP_WCT_TINT_21_t2_v9_n2", "toll": 45000, "statName": "CHAR_KIT_FM_PURCHASE20", "storageType": "BITFIELD", "bitShift": 7, "bitSize": 1, "category": ["CATEGORY_WEAPON_MOD"] }, ...

What is information technology? It appears to be data for a "cyberspace store catalog" according to some references. I assume information technology contains a list of all the possible items and upgrades yous tin can buy in GTA Online.

Clearing up some confusion: I beleive these are in-game money purchasable items, not direct linked with microtransactions.

But x megs? That'south naught! And using sscanf may not exist optimal but surely information technology's not that bad? Well…

Yeah, that'due south gonna have a while… To be fair I had no idea most sscanf implementations called strlen so I can't blame the developer who wrote this. I would assume it only scanned byte past byte and could end on a NULL.

Problem 2: Let's use a Hash- … Array?

Turns out the second offender is called right adjacent to the first 1. They're both even called in the same if statement as seen in this ugly decompilation:

All labels are mine, no idea what the functions/parameters are actually called.

The 2d problem? Correct later parsing an item, it's stored in an array (or an inlined C++ listing? not certain). Each entry looks something like this:

one 2 three iv

struct { uint64_t *hash; item_t *item; } entry;

Just before it'south stored? It checks the entire array, i by one, comparing the hash of the item to see if it's in the list or not. With ~63k entries that's (north^2+n)/2 = (63000^2+63000)/2 = 1984531500 checks if my math is right. Well-nigh of them useless. You accept unique hashes why non use a hash map.

I named it hashmap while reversing but it's clearly not_a_hashmap. And it gets even better. The hash-assortment-list-thing is empty earlier loading the JSON. And all of the items in the JSON are unique! They don't even need to check if information technology's in the list or not! They even have a function to directly insert the items! Only use that! Srsly, WAT!?

PoC

At present that'due south nice and all, but no one is going to have me seriously unless I test this so I tin can write a clickbait title for the post.

The plan? Write a .dll, inject information technology in GTA, claw some functions, ???, profit.

The JSON problem is hairy, I can't realistically supervene upon their parser. Replacing sscanf with one that doesn't depend on strlen would be more realistic. Merely there'southward an even easier way.

• claw strlen
• wait for a long cord
• "cache" the start and length of it
• if it's called once again within the string's range, return buried value

Something like:

1 2 3 iv 5 6 7 8 9 10 11 12 13 14 xv 16 17 xviii 19 20 21 22 23 24 25 26 27 28 29 xxx 31 32 33 34 35 36

size_t strlen_cacher (char* str) { static char* kickoff; static char* end; size_t len; const size_t cap = 20000; if (beginning && str >= start && str <= end) { len = finish - str; if (len < cap / two) MH_DisableHook((LPVOID)strlen_addr); return len; } len = builtin_strlen(str); if (len > cap) { commencement = str; stop = str + len; } return len; }

And every bit for the hash-array problem, information technology's more straightforward - just skip the indistinguishable checks entirely and insert the items directly since we know the values are unique.

ane ii 3 4 5 6 vii 8 9 x xi 12 13 14 15 sixteen 17 xviii 19 twenty 21

char __fastcall netcat_insert_dedupe_hooked (uint64_t catalog, uint64_t* fundamental, uint64_t* detail) { uint64_t not_a_hashmap = catalog + 88; if (!(*(uint8_t(__fastcall**)(uint64_t*))(*item + 48))(particular)) render 0; netcat_insert_direct(not_a_hashmap, key, &detail); if (*primal == 0x7FFFD6BE) { MH_DisableHook((LPVOID)netcat_insert_dedupe_addr); unload(); } return 1; }

Total source of PoC here.

Results

Well, did it work and then?

1 2 3 iv v six

Original online mode load time: ~6m flat Time with only duplication check patch: 4m 30s Fourth dimension with but JSON parser patch: 2m 50s Time with both issues patched: 1m 50s (6*threescore - (1*lx+50)) / (6*sixty) = 69.four% load fourth dimension improvement (nice!)

Hell yes, it did! :))

Well-nigh likely, this won't solve everyone's load times - there might be other bottlenecks on dissimilar systems, simply information technology'due south such a gaping hole that I have no idea how R* has missed it all these years.

tl;dr

• There's a single thread CPU bottleneck while starting up GTA Online
• It turns out GTA struggles to parse a 10MB JSON file
• The JSON parser itself is poorly congenital / naive and
• Afterwards parsing there's a slow item de-duplication routine

R* please fix

If this somehow reaches Rockstar: the bug shouldn't take more than than a 24-hour interval for a single dev to solve. Please do something almost it :<

You lot could either switch to a hashmap for the de-duplication or completely skip it on startup as a faster fix. For the JSON parser - just swap out the library for a more performant i. I don't remember in that location'southward any easier way out.

ty <iii

Small update

I was expecting to become some attention but nowhere virtually this much! After reaching the top of HN this post has spread like wildfire! Thanks for the overwhelming response :)

I'll practice more writing if something interesting comes along, merely don't await anything of this scale soon - at that place was a lot of luck involved.

A few people suggested spamming this post to Rockstar's support - delight don't! I'm sure they've seen this by now. Continuing would only bog down support tickets for everyone else. Social media is fair game in my book tho.

Several HN comments suggested I add a donate button, as they would like to buy me a beer (cheers!) and then I'yard placing a link in the footer.

Thank you for reading and all the back up :)

Update 2021-03-15

• Got confirmation from R* that this is getting a fix soon
• Just got awarded $10k through their H1 in-game compensation equally an exception :)) (usually only for security issues)
• Trying to figure out what's a W8 and how to fill information technology (lol)
• I did try asking for more technical details but they couldn't say anything
• Will do another benchmark on my aforementioned old setup as soon as the update is out, I'm certain their engineers won't disappoint :)

Update 2021-03-sixteen

R* released the update! Downloaded it and got my first run results - same hardware, aforementioned measurement - from R* logo to fully online.

Fully fixed! t0st approves!

Cheers once again for all the coffees, and cheers to R* for taking the time to look into this and the generous bounty!

Source: https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times-by-70/

Posted by: fordambateractly.blogspot.com

Share this post