Map cache file
A map, also known as a cache file, is a bundle of compiled tags which can be loaded and used by Halo. With the exception of resource maps, each map represents a playable campaign, multiplayer level, or main menu.
Tags within a map file are not exactly the same as they exist in your source
tags directory. When tags are compiled into a map, data is prepared for how it will be used at runtime. Tag path references are replaced with pre-calculated pointers, child scenarios are merged, extra fields are calculated, and the metadata for bitmaps and sounds is separated from their raw data.
Maps are found in Halo's
maps directory. Maps in subdirectories are not loaded by the game. Mods like Chimera and HAC2 store downloaded maps in a separate location and force the game to load them regardless.
With the exception of resource maps, the type of a map is determined by the type field in the compiled scenario tag.
The most common type of map, these were compiled with a scenario type of "multiplayer". Multiplayer maps can be loaded through the ingame menu or with the command
sv_map. Loading a multiplayer map using
map_name will trap the player in the level without the ability to use the menu.
Singleplayer/campaign maps can be compiled by setting the scenario's type to "singleplayer". Besides via a modded
ui.map, campaign maps can then be loaded with
map_name. This map type is also used by modders creating scripted firefight maps. When Tool compiles this map type, it strips multiplayer information from globals and applies some balancing tag patches.
ui.map contains resources for the game's main menu, including bitmaps for its UI elements like the server browser and the Halo ring background. The HEK supports the creation of custom UI maps. When Tool compiles a UI map, it strips multiplayer info and fall damage blocks from globals.
Custom UI maps which intend to add a campaign menu to Custom Edition must include a dummy first menu item since the game is hardcoded to remove it.
loc.map are special resource maps, and contain commonly referenced tags which do not need to be duplicated within each playable map referencing them. Instead, these shared tags are excluded by tool at map compilation time (seen as "cache hits" in its output). Compiling custom resource maps from tagshow? can be an effective way to reduce the net filesize of a campaign overhaul mod.
In addition to CEA's compressed map format, some custom map formats have been developed for use by a modified game client:
Maps with the extension
.yelo are compiled with OS_Tool and can only be played using a game client running the OpenSauce mod, which extends Halo's engine with new tag types, higher limits, and extra renderer features. These maps are typically custom campaign missions specifically designed to take advantage of these extensions. Refinery supports extracting OpenSauce tags from these maps.
Maps downloaded by the Chimera mod use a custom compressed format taking up less space. These maps are produced by compiling them using invader-build with the
-c flag, or using
invader-compress. The map's header is uncompressed, but all other data is compressed. These files also have a
.map extension, but when downloaded directly from HaloNet they have a
.inv extension. Although Chimera does not download maps to Halo's main
maps directory, take care not to mix these maps with stock ones since they are not compatible with the base game and are unsupported by other mods at this time. You can, however, decompress them easily with
invader-compresss -d <map>.
Because the game has a 23 MiB tag space limit (22 MiB on Xbox and raised to 31 MiB in CEA), Tool will enforce this limit when compiling a map. Keep an eye on its console output:
total tag size is 8.43M (14.57M free)
Total tag size is comprised of all non-raw tag data (ie. no bitmap or sound raw data) plus the largest BSP size, since the BSP is loaded within the 23 MiB space and there will only be a single BSP loaded at a time.
Care should be taken not to get too close to the tag limit, because even though you may compile a map with a certain set of resource maps (e.g. the English version of the game), Halo players with different languages may actually have larger resource map tag data which now exceeds the limit and prevents your map from loading.
You can toubleshoot which tags are using the most memory by generating
baggage.txt using the Sapien hotkey: Control+Shift+B.
Within a map, tag metadata (most of the fields seen in tag editors) is stored separately from raw data (sounds and bitmaps). BSP data for all BSPs is also stored in its own location. The game is able to find these locations using special headers and indexes in the map file.
Each section is loaded in a different way:
Tag metadata is copied directly into memory at a fixed address. The game has a limited amount of tag space available for the currently loaded map. The size depends on the edition:
Tag metadata is loaded into the start of this region. Because this data has been preprocessed by Tool, it requires no further processing and thus is very fast to load. The address where tag data is loaded is also dependent on the edition:
The active BSP is loaded into the end of the tag space. When a BSP switch occurs, the new BSP data is read from the map file and replaces the previous data in-memory. In CEA, it is loaded at memory address
Raw data is streamed from the map file as needed and dynamically allocated in a loaded resource pool/cache. The texture cache is cleared during maps loads and BSP switches. Some tags like BSPs and scenarios contain a "predicted resources" block which hints to the game which data should be loaded into these caches.
Tags from resource maps are also loaded into the tag space as needed. Halo CEA uses compressed maps and additional files to store sounds and bitmaps, so loads maps differently.
Thanks to the following individuals for their research or contributions to this topic:
- gbMichelle (How maps are loaded into memory)
- Jakey (Tag data limit info)
- Kavawuvi (Map base memory addresses)
- Masterz1337 (Context on OpenSauce capabilities)