Question

[Sparky]

the maximum number of tags you can have in a map is 7150, about. I forgot. I do remember posting an extensive topic on that in this forum, but it's generally the number of tags and shaders and models and other resources that are ACTIVE within the scenario (scnr) tag that will make it lag to a standstill or crash. Filesize itself means very little, unless the resources are being set to memory. You mentioned AI, and the actor varient (actv) tag has one of the highest number of references, such as character voice sounds and related weapons and resources.

Fixed, and the maximum number is actually 7,820 without having Zeus modify the max limit counter, and with Zeus extending the limit, probably 32,767-ish.

[Modzy]

Fixed, and the maximum number is actually 7,820 without having Zeus modify the max limit counter, and with Zeus extending the limit, probably 32,767-ish.

The tag limit is related to two separate issues:

The first issue has to deal with tag IDs. All scenario tags, the first tags in the map, start with the base 0xE1740000. Every tag thereafter has exactly 0x10001 added to it from the previous ID.
If we do the math:

0xFFFFFFFF - 0x E1740000 = 0x1E8BFFFF
0x1E8BFFFF / 0x10001 = 0x1E8B = 7819

That means Halo IDs have a limit of 7,819 (you were close!).
Solutions to this issue:

We can modify the added value when rebuilding maps. Instead of 0x10001, we can reduce this to 0x1001. Giving us up to 125,089 unique ID values. It is unclear how small we can actually make the add value. Halo seems to do some rounding when interpreting IDs, as with some testing proves, Halo sees 0xE1740000 to be the same as 0xE1740001. Which means there is a limit to how much we can reduce the add value. More research required, however 125,089 is plenty.

The second issue: Meta length.

Halo loads all meta and index data into memory at exactly 0x40440000. The BSP is loaded at the offset 0x41B40000 - BSP length. Because of the BSP offset, there is a very limited space for tag meta to live.
In the best case scenario, where the BSP is zero bytes, the space for meta data is extremely small. Some maths:

0x41B40000 - 0x40440000 = 0x1700000 = 24,117,248 bytes.

That's barely 24MB! That limit was plenty for the original game, as even the largest campaign maps have meta data sizes of hardly 10MB, but it doesn't leave much room for modders.
Because 0x40440000 and 0x41B40000 and offsets built into the game engine, we cannot overcome this issue without modifying the game itself. Even if we did modify the game to have larger meta data spaces, this modification would make the game unable to play any maps. You would have to convert any map you wanted to play to the new magic (like Pearl does to go from demo to full or reverse). That's a whole mess of trouble.

Conclusion:

We can have hundreds of thousands of unique IDs, but we will always be limited to meta data size of 24MB.

Any issues that arrise not related to these two problems is due to shotty rebuilding process and not Halo limitations.

[Sparky]

I was planning to have Zeus start the scnr tag ID at 0xFFFFFFFF instead of at 0xE1740000. I don't know if there's some calculation with the offset and the memory location, but probably not, right? Memory location will probably not be affected by a change in tag ID. ...um, the tags count from -7820 down to -1 at most, giving you 7819 possible tags in the map (although I've been successful with the one extra tag in my old Eschaton stress testing while compiling the INCY a while back). So why not start from -32,767 and count down to -1? Or an even lower/higher value? That's what I mean.

Modifying the locations of the memory addresses would be something that MacSoft probably did for the development of the Universal Binary version. I wonder if they extended it?

Also, if map files can be literally 500+ MB in size, why can't all the metadata fit into memory? Have you stress-tested this? Certainly, Halo would seem to lag out at that point... and I do recall Sapien throwing errors along the lines of "MAXIMUM_RENDERED_OBJECTS exceeded" and those gloriously non-specific "OUT OF OBJECTS:" errors, the former of which is probably related to OpenGL or DirectX buffer sizes... and there are sound buffer overflows with certain sound_scenery tags like the "stream" tag, where too many sound_scenery tags' sound_looping sounds playing at once crash the game engine since it probably doesn't account for those during the increase of the sound decompression buffer size (but this is seemingly unique to Sapien)... then we have the shader problems that crash the UB version... so many things in this game could be updated.

[Modzy]

I was planning to have Zeus start the scnr tag ID at 0xFFFFFFFF instead of at 0xE1740000. I don't know if there's some calculation with the offset and the memory location, but probably not, right? Memory location will probably not be affected by a change in tag ID. ...um, the tags count from -7820 down to -1 at most, giving you 7819 possible tags in the map (although I've been successful with the one extra tag in my old Eschaton stress testing while compiling the INCY a while back). So why not start from -32,767 and count down to -1? Or an even lower/higher value? That's what I mean.

Modifying the locations of the memory addresses would be something that MacSoft probably did for the development of the Universal Binary version. I wonder if they extended it?

Also, if map files can be literally 500+ MB in size, why can't all the metadata fit into memory? Have you stress-tested this? Certainly, Halo would seem to lag out at that point... and I do recall Sapien throwing errors along the lines of "MAXIMUM_RENDERED_OBJECTS exceeded" and those gloriously non-specific "OUT OF OBJECTS:" errors, the former of which is probably related to OpenGL or DirectX buffer sizes... and there are sound buffer overflows with certain sound_scenery tags like the "stream" tag, where too many sound_scenery tags' sound_looping sounds playing at once crash the game engine since it probably doesn't account for those during the increase of the sound decompression buffer size (but this is seemingly unique to Sapien)... then we have the shader problems that crash the UB version... so many things in this game could be updated.

Tag IDs do not descend in value, they ascend, however it may be possible to descend them while rebuilding. 0xFFFFFFFF has major issues because that is also the common value for nulled references. Starting from 0xFFFFFFFF and working backwards would cause all sorts of problems (especially with lone IDs). Its better to start with the traditional 0xE1740000 and reduce the add value. Although it is also possible to start with a value like 0xA1000000 and use the original 0x10001 add value, giving you 24,319 IDs.

MacSoft did not alter memory addresses for map loading in Universal Binary. As I stated before, any change to the engine to add more space for tag meta would mean converting all maps into that new memory offset/magic.

Maps can be very large' but meta data can't. Meta data is everything from the beginning of the index to the end of the map file. For the largest campaign maps, around 10MB. The 500MBs of data in some maps are due to multiple BSPs, model data, internal sounds/bitmaps, etc. none of which contribute to the meta data size as all raw data like that is stored before meta data in map file, and in somewhat unlimited sections of memory.

Example: the map D20 is 60MB in file size. The meta data is only 10,717,984 bytes, or 10MB.