CHAPTER 1 · CODE
CODE IN FLIGHT
Linked flat, Sid Meier's Colonization needs 484,241 bytes of code in memory before it allocates its first game buffer. Instead it asks DOS for about 168 KB — and never asks for more. The missing 361,168 bytes stay inside the EXE file on disk, and are paged into RAM at run time, call by call, by a tiny operating system the linker itself put there.
Why a far call can fly
Real mode has no memory-management unit, no page faults, no loader in the loop after
launch. DOS's EXEC reads an MZ executable's declared pages into one block,
applies the header's relocation table — a list of every location in the image that holds
a segment number — and jumps in. From that instant the program owns its bytes; nothing in
the machine will ever move code on its behalf.
The loophole is the far call itself. CALL FAR is five bytes —
9A, then a 16-bit offset, then a 16-bit segment — and that destination
segment is just data sitting in the instruction stream. Anything that can rewrite those
bytes, or route them through one level of indirection, can move the destination code
anywhere it likes between calls. Overlays are that observation industrialised: the linker
gathers functions into units, replaces every cross-unit far call with something it
controls, and ships a small runtime that loads units over one another on demand.
Software-defined paging, on a CPU with no paging hardware.
Both games also exploit a second, quieter loophole: DOS loads only what the MZ header
declares. CIV.EXE is 304,512 bytes on disk but declares 178,111; the
126,401 bytes beyond that boundary are its overlay store. viceroy.exe is
493,470 bytes and shows DOS a 123,073-byte load module — 73% of the file is
invisible to the operating system that launches it.
1991: the interrupt is the linker
Civilization uses the scheme Microsoft built into LINK: INT 3Fh
overlays. At link time, every far call into an overlaid unit is replaced by a
five-byte trap — CD 3F, then an overlay number and an offset. Our scan of
CIV.EXE finds 168 CD 3F sites, 147 of them carrying overlay
numbers 1–23; at file offset 1F8C the bytes read
CD 3F 07 12 00 — interrupt 3Fh, overlay 7, offset 0012h. The
handler behind vector 3Fh swaps the target overlay into a single resident buffer (loading
it from the EXE via ordinary DOS file reads) and jumps to the offset. One overlay area,
one overlay resident at a time, every dispatch through an interrupt.
The overlay store is plainer than you might expect: scanning paragraph boundaries
finds 23 embedded MZ images after the root, each 512-aligned, with
e_ovno numbered 1–23 in order, sized 1,514 to 10,666 bytes — 121,720 bytes
in all, and the last one ends exactly at end-of-file. The division of labour is visible
in the relocation tables: the root declares zero relocations, while
every overlay carries its own private table of 34–246 entries, applied when it is
paged in. Each overlay's header CS:IP field reads the identical
2058:2E83 — we read that as a fixed pointer into the root's
overlay manager rather than a real entry point. The resident root itself is a
177,599-byte image plus 60,368 bytes of MinAlloc: 237,967 bytes, about 232 KB,
before drivers and buffers — and beyond it, only the largest overlay's ~10.4 KB
ever needs to be in memory at once.
The manager Microsoft shipped came with stock error messages. The ones MicroProse added to them have more personality:
; CIV.EXE, $-terminated string block at file offset 29B4Fh Allocated 1Mb of space???? MS-DOS lied to us about how much memory was available Filename not found Not enough memory to load file Overlay load failed for some reason Overlay has overrun allocated memory Unable to shrink overlay memory allocation Error releasing overlay memory ; and the stock Microsoft set at 2A076h–2A0EAh Overlay not found Overlay Manager stack overflow Cannot load overlay: too many open files Please enter new program spec:
The quirky lines sit directly alongside the stock ones, which we read — flagged as
inference, not fact — as MicroProse customising Microsoft's overlay manager rather than
using it untouched. Next to PATH= and CIV.EXE in the string
table sits an environment-variable name, CVDEBUG: the manager re-opens its
own EXE by name to read overlays, and appears to have shipped with a debug switch.
1994: the linker ships an operating system
Three years later, Colonization's viceroy.exe was linked with
RTLink/Plus, from Pocket Soft, Inc. of Houston, Texas — a third-party
DOS linker whose selling point was what it called dynamic overlays: a module's
code "divided into pages to be brought into and out of memory on a least recently used
basis". That is not an overlay tree; that is demand paging with LRU replacement —
virtual memory, implemented in a linker, for real-mode code with no MMU in the loop.
The same product was bundled with CA-Clipper 5.x, so by 1994 every Clipper 5 business
application ran on the identical runtime.
The machinery inside viceroy.exe decomposes cleanly under a byte-level walk. After the 123,073-byte load module and 15 bytes of padding, a chain of page records starts paragraph-aligned at file offset 204D0:
; RTLink page record — 31 of these, chained from file offset 204D0h
+0 u16 total record size, paragraphs ; next record at +size×16
+2 u16 header size, paragraphs ; fixup table, paragraph-padded
+4 u32 2BCh = 700 ; constant in all 31 pages — meaning unknown
+8 u32 fixup count N ; Σ across all pages = 8,452
+12 u32 0 ; in all pages
+16 N × { u16 offset, u16 paragraph } ; MZ-style fixups, page-relative
... code — (size − header) paragraphs ; Σ = 325,936 bytes of code
The walk yields 31 pages totalling 361,168 bytes (325,936 of code, 35,232 of header), and ends at 493,472 — two bytes past end-of-file, the last record's paragraph rounding overshooting the truncated tail. Every fixup in every page targets its own page's code area: pages are relocated locally, at load time, by the runtime — 8,452 acts of relocation DOS never sees.
Calls get into those pages through a thunk farm: 1,024 hand-packed call gates occupying 12,322 bytes at file offsets 1A490–1D4B2 (three padding holes of 16–20 bytes; seven more gates live inside the VM manager itself, 1,031 in all). Each gate is a far call into segment 10F7 — which is also the EXE's entry segment; the program starts life inside its own pager — followed by a far-jump parameter block:
; the two gate shapes (1,024 packed into 12,322 bytes at 1A490h)
paged (659): 9A AB 0D F7 10 CALL FAR 10F7:0DAB ; into the VM manager
EA oo oo 00 00 JMP FAR 0000:oooo ; segment 0000 — resolved at run time
0D 00 page number (13) ; 376 of the 659 carry one extra word
root (365): 9A 91 0D F7 10 CALL FAR 10F7:0D91
EA oo oo ss ss JMP FAR ssss:oooo ; segment patched by DOS at load
And here is the proof of the whole design, sitting in the file header. viceroy.exe's MZ relocation table has 2,254 entries. Exactly 365 of 365 root-gate segment fields appear in it — they point at resident code, so DOS must patch them at load. Exactly 0 of 659 paged-gate segment fields do — they are zero placeholders, resolved again and again at run time as pages come and go. Load-time linking and run-time linking, mechanically separated, visible in a relocation table. The 659 paged gates cover all 31 pages; there is no dead page in the file.
The runtime earns the name "VM manager". It probes for an EMS driver
(EMMXXXX0, at file offset 1337A) and uses
expanded memory as a page cache when present; its pool of resident pages is sized from
free conventional memory at startup and grows with it; watched live under our emulator,
a first load is a plain seek-and-read of raw page bytes from the EXE (staged through the
top block Chapter 2 describes), while an evicted page's contents are FAB-compressed into
an in-memory cache so a re-load can decompress from RAM instead of touching the disk.
Its error table speaks its vocabulary fluently:
; viceroy.exe — the RTLink VM manager's error table (selected; evm0001–evm0022)
evm0001 Cannot find VM file "
evm0003 Read error on VM file "
evm0008 Out of conventional memory.
evm0009 EMM error: error = xx, function = xxxx.
evm0010 XMM error: error = xx, function = xx.
evm0012 Unvectored call to resident symbol calls a Virtual Vector.
evm0015 Vectoring error: interrupt handler calls a Virtual Page.
evm0018 Reference to a Virtual Page after $$RTL_TERMINATE.
evm0020 Illegal alteration of EMS mapping state.
evm0022 Smart vectoring failed. BP-chain destroyed.
Vectored call to Page xxxxH (page address xxxx:xxxx)
occurs near map address xxxx:xxxx. ; crash post-mortem, keyed to the linker map
Note what those messages police: interrupt handlers must not live in pages
(evm0015); nothing may touch a page after shutdown
(evm0018); nobody may fiddle the EMS mapping behind the manager's back
(evm0020). And "smart vectoring" (evm0022) is the stack
surgery that makes eviction safe: the manager pushes an intermediate return address —
and repairs the BP chain — so that when a paged function returns, control passes back
through the pager, which can re-load whatever the caller lived in. Structurally, pages
hold no raw pointers into other pages: every one of their inter-unit calls goes back
through the root's gate farm, and their fixups are all page-local. Evicting a page can
therefore never strand a live pointer — the only references to it are gates the manager
owns.
Thirty-one pages of a game
The pages themselves have a shape. Two giants carry the fattest subsystems — page 2 is the largest raw record at 32,528 bytes, page 13 the largest code payload at 29,520 bytes (the number that sizes the resident page buffer) — while page 24 is a 752-byte sliver. Header overhead tracks the fixup count, not the code size; and how callable a page is has little to do with how big it is: 15,696-byte page 23 exposes 40 entry gates, while 5,008-byte page 17 exposes exactly one.
Two schemes, side by side
Set the two mechanisms next to each other and the generational jump is stark. Microsoft's scheme dispatches every overlaid call through an interrupt into a single overlay area — tree-structured, one unit resident per area, no replacement policy worth the name. RTLink dispatches through patchable gates into a pool — page-granular, LRU, several pages co-resident, misses paged in and hits jumping direct. One is a clever trick; the other is an operating-system service smuggled into a linker.
A decade in flight
MicroProse did not arrive at this by accident; overlays were house culture. In 1989,
F-15 Strike Eagle II shipped a hand-rolled scheme: driver executables loaded
with DOS EXEC subfunction 4B03 — "load into pre-allocated
memory, but don't execute" — placed by a loader that read the chosen driver names from a
communication buffer at 0000:04F0 (the BIOS inter-application
communication area) and then patched a table of five-byte far-jump stubs, opcode
EA, so the game called its drivers through fixed slots. Microsoft C 5.1
already offered INT 3Fh overlays; MicroProse rolled their own anyway.
Civilization, 1991, adopted the Microsoft convention — and its 4B03 sibling lived on:
Colonization still loads its sound driver, asound.col, with the same
subfunction, into a block it then shrinks from ~498 KB to ~49 KB. Colonization's main
executable, 1994, graduated to RTLink/Plus — and joined a crowd. Pocket Soft's runtime
also carried MicroProse's MADS adventures (Rex Nebular and the Cosmic Gender Bender,
1992; Return of the Phantom, 1993; Dragonsphere, 1994 — the same engine family whose
madsdev.lib turns up, named, in the CodeView debug block MicroProse shipped
inside Colonization's own opening.exe), Darklands (1992, its stub table confirmed by the
Darklands Restoration Project at 1B465–1B52D),
later Legend Entertainment titles, and — via the CA-Clipper 5.x bundle — a vast
population of DOS business software. The games industry did not wait for hardware
paging. It linked its own.
Sources for this chapter — byte-level: viceroy.exe page-chain walk, thunk census and MZ-reloc proof; CIV.EXE INT 3Fh stub scan, embedded-overlay sweep and string blocks (fresh 2026-07-18 analysis). Runtime: live overlay-load and eviction traces under mddosem. Community: dreammaster's rtlink_decode notes, Microsoft KB Q31994/Q40408/Q63269, RBIL INT 3F, the neuviemeporte F-15 SE2 teardown, the Darklands Restoration Project, and the Viva Clipper linker glossary — full credits in How We Know.