A map, also known as a cache file, is a bundle of processed tags which can be loaded and used by Halo. With the exception of resource maps, each map represents a playable campaign, multiplayer level, or main menu.
When tags are compiled into a map, their data is prepared for how it will be used at runtime. Tag path references are replaced with pre-calculated indices or pointers, child scenarios are merged, extra fields are calculated, and the metadata for bitmaps and sounds is separated from their raw data.
Maps are found in Halo's maps
directory and have the ".map" extension. Maps in subdirectories are not loaded by the game. H1CE mods like Chimera and HAC2 store downloaded maps in a separate location and force the game to load them regardless.
Editing and porting maps
The recommended approach to porting or modifying maps is to obtain their source tags and recompile the map using Tool or invader-build. This ensures the greatest flexibility and tags will be processed correctly when the new map is built.
Although maps work mainly the same way in each release of H1, there are a number of differences listed on this page which prevent maps from being reused across them as-is. For example, an H1CE map file cannot be used in H1PC Demo without recompiling it from tags.
It is also possible to directly edit ("poke") the tags within a map using tools like Assembly, but this can be error prone or more limiting than working with source tags. Adding new assets ("injecting") is harder than using the intended asset pipeline.
Map types
The type of a map is determined by the scenario type field when the scenario is compiled, with the exception of resource maps which are not compiled from one.
Multiplayer
In H1CE, multiplayer maps can be loaded through the in-game menu or with the command sv_map <map> <gametype>
. Loading a multiplayer map using map_name <map>
will trap the player in the level without the ability to use the menu.
Singleplayer
To load a singleplayer map in H1CE, you can either use a modded ui.map
which includes menu options to launch it, or load it directly using the map_name <map>
console command. When Tool compiles this map type, it strips multiplayer information from globals and applies some balancing tag patches. These patches are applied at runtime in H1X.
UI
The special ui.map
contains resources for the game's main menu, including bitmaps for its UI elements like the server browser and the Halo ring background. The HEK supports the creation of custom UI maps. When Tool compiles a UI map, it strips multiplayer info and fall damage blocks from globals.
Custom UI maps which intend to add a campaign menu to H1CE must include a dummy first menu item since the game is hardcoded to remove it.
Resource maps
Resource maps provide a way for certain tags to be stored external to a playable map rather than its tags being totally self-contained. These maps themselves are not playable and have a different header structure, but instead contain shared tags referenced by normal map files. This feature was introduced with H1PC with bitmaps.map
and sounds.map
to store bitmap and sound tags respectively, and loc.map
was added in H1CE to store font and unicode_loc.map
except for backwards compatibility with maps compiled for Custom Edition. H1X does not use resource maps.
When playable maps are compiled using Tool, any needed tags for the map which are already present in a resource map (determined by tag path) will be excluded and referenced by pointer to the loaded resource map instead. The resource maps were created once using an internal version of Tool and were not originally intended to be modified, though invader-resource and OpenSauce are capable of compiling new ones. Using incompatible resource maps will result in glitched textures, sounds, and text.
Storing bitmaps and sounds in common files had the benefit of reducing the disk space needed for H1PC's maps because multiple maps are able to reference the same data rather than duplicating it. The other benefit is that the resource map can be swapped out with another to alter tag content without affecting the dependent maps. In the case of loc.map
, the file itself contains common UI messages and prompts but varies by language of the H1CE installation. This means a custom map can be compiled once but still have localized messages when used in another language of the game, as opposed to compiling a version of the map for each language.
To create self-contained maps when using Tool, temporarily removing resource maps from the maps
directory when compiling the map.
Compressed maps
H1X maps use zlib compression for all data following their header. Maps are decompressed into one of multiple disk caches depending on the header's scenario type.
This compression scheme is not supported natively in other releases of the game, but it is supported by the H1CE mod Chimera. Maps downloaded from HaloNet by this mod may be compressed this way. Although Chimera does not download maps to Halo's main maps
directory, take care not to mix these maps with stock ones since they are not compatible with the base game and are unsupported by other mods at this time. Compressed maps can be identified using invader-info.
Engine differences
Each game supports a slightly different set of tag classes. For example, gbxmodel did not yet exist in H1X and H1CE cannot render shader_
Because of these differences, map authors may need to create multiple tag sets to compile their maps if releasing for both H1CE and H1A. For example, the H1CE map would use shader_
OpenSauce .yelo maps
Maps with the extension .yelo
are compiled with OS_Tool and can only be played using H1CE with the OpenSauce mod, which extends Halo's engine with new tag types, higher limits, and extra renderer features. These maps are typically custom campaign missions specifically designed to take advantage of these extensions. Refinery supports extracting OpenSauce tags from these maps.
Protected maps
A protected map is a map which has been intentionally corrupted in a way which still allows it to be loaded and played in-game, but hinders attempts to extract tags from it by removing or scrambling data like tag paths. It is now a discouraged practice.
Map protection was unfortunately common in the H1CE modding scene in the 2000s as a way to prevent others from using ones custom tags, and it has overall negatively impacted the community because the resulting maps are crash-prone, cannot be easily be ported to newer engines like H1A, and new modders cannot extract their tags cleanly for study. H1A even explicitly checks for and refuses to load protected maps.
Refinery can "deprotect" maps for tag extraction but the results may require cleanup.
Map file size limit
The maximum allowable file sizes for playable maps varies by version. Halo will reject maps if their header has a file size that exceeds this limit.
- H1X:
- SP: 278 MiB
- MP: 47 MiB
- UI: 35 MiB
- H1CE: 384 MiB (Tool enforces 128 MiB for MP maps)
- H1A: 2 GiB
invader-build can be used to build cache files which exceeds the stock limits, but this may require the user to use a mod to play the map.
Tag space
The game's buffer for tag data is limited:
- H1X: 22 MiB
- H1CE and H1PC: 23 MiB
- H1A: 64 MiB
Total tag size is comprised of all non-raw tag data (ie. no bitmap or sound raw data) plus the largest BSP size, since the BSP is loaded within the tag space and there will only be a single BSP loaded at a time. Additionally, a maximum of 65535 tags can be in a map.
Tool will enforce this limit when compiling a map. Keep an eye on its console output:
total tag size is 8.43M (14.57M free)
Care should be taken not to get too close to the tag limit, because even though you may compile a map with a certain set of resource maps (e.g. the English version of the game), Halo players with different languages may actually have larger resource map tag data which now exceeds the limit and prevents your map from loading.
You can toubleshoot which tags are using the most memory by generating the baggage.txt
report using the Sapien hotkey: Control + Shift + B.
Map loading
Within a map, tag definitions (sometimes called metadata) are stored separately any raw data used by the tag, such as sounds and bitmaps. BSP data for all BSPs is also stored in its own location. The game is able to find these locations using special headers and indexes in the map file.
Each section is loaded in a different way:
Tag metadata is copied directly into memory at a fixed address. The game has a limited amount of tag space available for the currently loaded map. The size depends on the edition:
- H1X: 22 MiB
- H1PC and H1CE: 23 MiB
- H1A: 64 MiB
Tag metadata is loaded into the start of this region. Because this data has been preprocessed by Tool, it requires no further processing and thus is very fast to load. The address where tag data is loaded is also dependent on the edition:
- H1X:
0x803A6000
- H1PC Demo:
0x4BF10000
- H1PC and H1CE:
0x40440000
- H1A: Dynamic
The active BSP is loaded into the end of the tag space. When a BSP switch occurs, the new BSP data is read from the map file using information stored in the scenario tag and replaces the previous data in-memory. In H1A, it is loaded at a separate dedicated location instead.
Raw data is streamed from the map file as needed and dynamically allocated in the sound cache and texture cache. The texture cache is cleared during maps loads and BSP switches. Some tags like BSPs and scenarios contain a "predicted resources" block which hints to the game which data should be loaded into these caches.
Tags from resource maps are also loaded into the tag space as needed. Halo CEA uses compressed maps and additional files to store sounds and bitmaps, so loads maps differently.
File structure
Generally map files consist of a map header section followed by BSP, model, raw data, and indexed tag definitions. The header structure and/or values vary by game. Not all maps may look like this exactly, due to map protection or differences in map compiler. All data is little-endian.
This page does not give a full accounting of how BSP and model data are stored and loaded. For further information, see this page's acknowledgments section for source material.
Map header
Normal playable (non-resource) cache files begin with a header which is always 2048 bytes long.
Field | Offset (relative) | Type | Comments | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
head magic | 0x0 | char[4] | Always takes the value | ||||||||||||||||||
cache version | 0x4 | CacheVersion: enum32 | Identifies the cache file version, which differs by release of the game. This must match the game's cache file version for the map be loadable. Later games use additional versions, such as H2V's version | ||||||||||||||||||
| |||||||||||||||||||||
file size | 0x8 | uint32 | Length of the map in bytes when uncompressed. Halo checks this to ensure the map is within the size limit. | ||||||||||||||||||
padding length | 0xC | uint32 | Length of the padding after the map in bytes. Only used on Xbox. | ||||||||||||||||||
tag data offset | 0x10 | uint32 | Offset into the file to the tag data header. | ||||||||||||||||||
tag data size | 0x14 | uint32 | Length of the tag data in bytes. | ||||||||||||||||||
0x18 | pad(8) | ||||||||||||||||||||
scenario name | 0x20 | char[32] | The name of the scenario tag which this map was compiled from, e.g. | ||||||||||||||||||
build version | 0x40 | char[32] | Must match the engine version on Xbox, | ||||||||||||||||||
scenario type | 0x60 | ScenarioType: enum16 | On Xbox, this tells the game which disk cache to decompress the map into. The singleplayer is 278 MiB, multiplayer is 47 MiB, and UI is 35 MiB. For example, setting this to | ||||||||||||||||||
| |||||||||||||||||||||
0x62 | pad(2) | ||||||||||||||||||||
checksum | 0x64 | uint32 | CRC32 checksum which verifies the integrity of BSP, model, and tag data. | ||||||||||||||||||
h1a flags | 0x68 | H1AFlags: bitfield32 | A bitfield which customizes how the cache file is treated by H1A in MCC. These are set by set by the | ||||||||||||||||||
| |||||||||||||||||||||
0x6C | pad(1936) | ||||||||||||||||||||
foot magic | 0x7FC | char[4] | Always takes the value |
Demo map header
Demo versions of H1PC use a different cache file header structure with reordered fields and extra padding between them as a means to make it harder to port retail cache files to the demo. The header is still 2048 bytes long.
Field | Offset (relative) | Type | Comments | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0x0 | pad(2) | Filled with garbage values | |||||||||||||
scenario type | 0x2 | ScenarioType: enum16 | On Xbox, this tells the game which disk cache to decompress the map into. The singleplayer is 278 MiB, multiplayer is 47 MiB, and UI is 35 MiB. For example, setting this to | ||||||||||||
| |||||||||||||||
0x4 | pad(700) | Filled with garbage values | |||||||||||||
head magic | 0x2C0 | char[4] | Always takes the value | ||||||||||||
tag data size | 0x2C4 | uint32 | Length of the tag data in bytes. | ||||||||||||
build version | 0x2C8 | char[32] | Must match the engine version on Xbox, | ||||||||||||
0x2E8 | pad(672) | Filled with garbage values | |||||||||||||
cache version | 0x588 | CacheVersion: enum32 | Identifies the cache file version, which differs by release of the game. This must match the demo cache file version ( | ||||||||||||
scenario name | 0x58C | char[32] | The name of the scenario tag which this map was compiled from, e.g. | ||||||||||||
0x5AC | pad(4) | Filled with garbage values | |||||||||||||
checksum | 0x5B0 | uint32 | CRC32 checksum which verifies the integrity of BSP, model, and tag data. | ||||||||||||
0x5B4 | pad(52) | Filled with garbage values | |||||||||||||
file size | 0x5E8 | uint32 | Length of the map in bytes when uncompressed. | ||||||||||||
tag data offset | 0x5EC | uint32 | Offset into the file to the tag data. | ||||||||||||
foot magic | 0x5F0 | char[4] | Always takes the value | ||||||||||||
0x5F4 | pad(524) | Filled with garbage values |
Resource map header
Field | Offset (relative) | Type | Comments | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
type | 0x0 | ResourceMapType: enum32 | The type of data found in this resource map. | ||||||||||||
| |||||||||||||||
paths offset | 0x4 | uint32 | Offset to tag paths. | ||||||||||||
resources offset | 0x8 | uint32 | Offset to resource indices. | ||||||||||||
resource count | 0xC | uint32 | The number of resources contained in this file. |
Tag index header
The tag index/tag data header is the start of where tag data and definitions are loaded directly into memory at runtime at the game's tag address. For most versions of the game, it looks like this:
Field | Offset (relative) | Type | Comments |
---|---|---|---|
tag array pointer | 0x0 | ptr32 | A pointer to the tag array, which lists all tags in the map. |
checksum | 0x4 | uint32 | A CRC32 checksum of the tag files used in the map. |
scenario tag id | 0x8 | uint32 | Unique ID of the scenario tag in the tag array. |
tag count | 0xC | uint32 | The number of tags in the tag array. |
model part count | 0x10 | uint32 | Number of model parts in the map. |
model data file offset | 0x14 | uint32 | File offset to the vertex data. |
model part count pc | 0x18 | uint32 | This is a repeat of the model part count field. |
vertex data size | 0x1C | uint32 | Size of the vertex data in bytes. |
model data size | 0x20 | uint32 | Total size of the model data in bytes. |
magic | 0x24 | char[4] | Typically equal to "tags", or |
Xbox tag index header
The H1X tag index header is slightly different since model data is also in the tag data, so it uses pointers instead of file offsets.
Field | Offset (relative) | Type | Comments |
---|---|---|---|
tag array pointer | 0x0 | ptr32 | A pointer to the tag array, which lists all tags in the map. |
checksum | 0x4 | uint32 | A CRC32 checksum of the tag files used in the map. |
scenario tag id | 0x8 | uint32 | Unique ID of the scenario tag in the tag array. |
tag count | 0xC | uint32 | The number of tags in the tag array. |
model part count | 0x10 | uint32 | Number of model parts in the map. |
vertex data pointer | 0x14 | ptr32 | This is a pointer to the vertex data. |
model part count xbox | 0x18 | uint32 | This is a repeat of the model part count field. |
triangle data pointer | 0x1C | ptr32 | This is a pointer to the triangle data. |
magic | 0x20 | char[4] | Typically equal to "tags", or |
Tag array entry
Each 32-byte element in the tag array contains information about a tag in the map, including a pointer or resource index to the actual tag definition itself.
Field | Offset (relative) | Type | Comments |
---|---|---|---|
primary group id | 0x0 | char[4] | The group ID of the tag's primary tag class. |
secondary group id | 0x4 | char[4] | The group ID of the tag's secondary (parent) tag class. For example, this would be object for device_ |
tertiary group id | 0x8 | char[4] | The group ID of the tag's tertiary (grandparent) tag class. For example, this would be object for device_ |
tag id | 0xC | uint32 | Unique ID of the tag. |
tag path pointer | 0x10 | ptr32<char> | Pointer to the tag path string. This data may unusable in protected maps and is technically not needed at runtime except for debugging purposes. |
tag data ref | 0x14 | ptr32 | This field can be interpreted as either a |
is external | 0x18 | uint32 | If the value is |
Acknowledgements
Thanks to the following individuals for their research or contributions to this topic:
- gbMichelle (How maps are loaded into memory)
- hellux (Testing and documenting tag space on Xbox)
- Jakey (Tag data limit info)
- Kavawuvi (Documenting the map file structure, base addresses, and Invader code for reference)
- Masterz1337 (Context on OpenSauce capabilities)
- WaeV (Diagrams and overview of the map file structure.)