The Secret World of 640 Kb
Back in the old days of retro computers (we're talking late 80s, early 90s), it didn't matter how many sticks of RAM you bought for your machine. The REAL problem was how to free up the same 640 Kb of conventional memory that limited all our machines. That's just how the computer architecture was designed. Sure, you needed XMS or extended memory, which nowadays we all take for granted - it's the only memory we know about now - but our modern machines aren't bottlenecked by DOS the way they used to be.
If all this sounds out of context, don't be alarmed. It will become clear in a moment. What should be very visceral and real to you is the anxiety and frustration you feel when you try to run Descent or System Shock or Falcon 3 or Wing Commander or any number of other games and you just get some cryptic error complaining about memory, even though you've got an entire 8MB sitting there doing nothing. Know what I mean? Even those of you trying to do work on your machines like AutoCAD or God forbid you try to get the internet running (which is totally doable by the way), will have seen these dreaded memory errors before. As tempting as it might be, you can't just think of memory as a simple number ("I have 8MB and this game only needs 500kb. What's going on?"). The truth is, all these errors are complaining about not being able to get enough from a certain small portion of your total memory. I'm going to show you how this all works and how to get rid of these errors for good.
DOS Has Been Around for a While!
To be honest my history knowledge gets fuzzy this far back but DOS at least has been around since well into the early 80s and the 8086 the future Intel chip architectures were based upon has been around even earlier than that, as early as the 70s so way back then somebody decided that it would be convenient to place certain hardware ROMs in specific places in memory, which then formed the footprint in memory we know as conventional memory or the first 640kb. The problem was the technology at the time. You only had so many 'bits' to work with and memory addressing could only count so high with those bits so it was impractical to use anything more than 1 MB of memory with these simple CPUs. Adding more to that was fine later on and as we got into the 286 and 386 architecture we definitely needed it, but the heritage of the 8086 architecture and how DOS utilized those lower memory blocks remained. It would take decades before Windows 98 would finally dispel with DOS dependencies altogether and dynamically manage all the memory for you. Windows 95 may seem similar but in many ways it was just another extension of DOS, like Windows 3.1 was and lots of software was still tied to those pesky DOS limitations.
Breaking it Down
So now that we've reminisced in the past that most of us never experienced, here's the deal.
All the RAM in your old computer is like a big stack of bricks, starting at the ground and going up. You start at 0 and go up to let's say... 8 MB. Doesn't matter how much total RAM you have though, since everybody has the same 640Kb at the bottom of the stack. Now this stack is divided up into 16kb layers. Each layer is addressed using hexadecimal, which may sound spooky but it just means instead of counting 10, 20, 30kb, etc. you're counting 16, 32, 48kb, etc and using a base-16 nomenclature to label those rows so you'll get a row like A000 - A3FF. Don't worry too much about it. It's more important to think of the whole thing visually as a simple stack of bricks with 16 bricks per layer and 40 layers to make up your 640k.
Now when you fire up your computer, the first most basic element that takes control is the BIOS, which is a chip built into the motherboard and it talks to all the computer's hardware to see what's there and how to set it up for use. Only after it's queried and set up everything can you get a command prompt and do stuff. So it looks around, finds your video card, finds your sound card, your NIC, your SCSI drive, 387 coprocessor, CD-ROM and whatever else is connected and attempts to set it up. In order to set it up so it works, some devices will need to copy a piece of code that's part of the device itself into the computer's main memory, so that it can be accessed quickly by the system. This is called shadowing. The BIOS shadows its own code, the video card shadows its code, the NIC sometimes and possibly other devices too. All these ROMs have to be told where to go in memory and by default, some will just get dumped into the first 640k.
On top of that, you will want to execute certain TSRs, which are little programs that need to run in the background and do important things. For example, if you want DOS or Windows to be able to use your network card you may have to load a packet driver. If you want your video games like Duke Nukem 3D to run at high resolution you may need to load a VESA driver. These aren't strictly mandatory but at the discretion of the user you may need these programs to run certain programs or get certain performance from them. You can tell these files to run at boot time using your autoexec.bat file but they also get dumped into your 640kb of memory.
After the computer is finished booting to DOS, you can type "mem" and hit enter to see how much conventional memory you have left. The upshot is that whatever program you want to execute, has to have a file size that is smaller than the remaining free conventional memory. Since you start with 640k, that's the maximum size any program can ever be. There are some tricks to circumvent this but it's beyond the scope of this article and most of the time you won't need it anyway. The idea is just to get your free memory back up in the neighborhood of 600'ish so you can run just about anything you want.
So What's the Fix? How Do I Get My Games Running Again?
Alright. We know we need memory for all these ROMs and TSRs and stuff. But we also need the lowest 640k to run our applications so where are we going to put these files and how? Unfortunately you can't just tell the computer to put them in extended memory. However, you CAN tell the computer to put them in what's calld UPPER and HIGH memory. This is kind of a second area of special memory that makes up just a bit more than the first 1MB of RAM. So you have your lower 640k, then comes 384k of UPPER memory to make 1024 (1MB) and then there's a tiny piece on top of that called HIGH memory. It's complicated to explain why that small piece is there and it has to do with how memory addresses wrap around to the next line and small accumulated bits of memory that result from that convention. Suffice it to say you have a few extra hundred kb of memory you can cram things into, so they don't occupy the lower 640kb that you need. There's no penalty to this either because if the space isn't used, you don't get it back later so fill it as much as you can. Use it or lose it!
Yeah.. now might be a good time to explain what that little message says about your game needing EMS memory.
Some games (and possible some programs) need more that 640kb of memory to fit themselves into but that isn't strictly possible so Microsoft added a feature to their memory management utility, EMM386, which designates a portion of upper memory as EMS memory (among other things), which just tacks onto the 640kb limit to make it a bit bigger. Wing Commander for example needs EMS memory. So does Falcon 3. Not all games need it though. If they don't, you're just wasting memory assigning EMS so you might want to have a boot disk that copies your config/autoexec files but with EMS added, just for those games.
To enable EMS, you just have to tell EMM386 to block off 64kb of memory in the upper memory area as the "pageframe" and give it a starting location. You will lose 64kb of upper memory but you'll be able to run those games that need EMS. We'll get to the execution specifics shortly. For now just know that as precious as RAM is on retro PCs, you might not be able to do everything you want with 1 boot sequence so you might just want to make 2: 1 on your hard drive for normal games and apps and another on a floppy just for EMS stuff like Wing Commander. That way when you want to play those games, just put the floppy in and boot normally. All your settings will return to normal when you remove the disk and reboot so it's really convenient and modular this way.
Using HIGH Memory
Remember that little piece of memory just above 1MB called HIGH memory? Well only certain programs can use that space but DOS is one of them. When DOS is running and you're typing away, doing your business, you're interacting with an interpreter that needs to be loaded in RAM, just like everything else. You might have guessed right now that this RAM is located in your conventional memory BUT with 2 lines of code in your CONFIG.SYS, you can put DOS in the HIGH memory space. To edit your config.sys file, start from the C:\ prompt and type EDIT CONFIG.SYS. Once in the editor, go to the top of the file and type:
and you're done. DOS will now load into high memory and you'll free up the space it was using in lower memory. BAM!!!
Using UPPER Memory
Upper memory is that 384k directly on top of the lower 640k. It's reserved by the system mostly for ROMs. Some ROMs go there automatically while some others get dumped in conventional memory. Therefore, your UPPER memory will likely start out looking like a mine field at first. You'll have segments that are already filled, with free space scattered around it. The art of memory management is to cleverly defragment this area and put all the free space together and then fill it in the right order with the right stuff so that it all gets pack nice and tight, freeing up even more space. Again, it sounds worse than it actually is.
DOS comes with a nifty tool called MSD.EXE. When you install DOS you can access it from any folder because it's in your DOS path so just type MSD and hit enter. You'll find yourself in a big blue screen with menus. Hit M for memory and you'll see something that looks kind of like a Tetris game pop up... because that's all memory management is... playing Tetris ;) Now might be a good time to whip out your cell phone and take a picture of this thing. You'll need to consult it later. What you're looking at is the stack of bricks I was talking about earlier, but only those above 640k, which is the 384k of UPPER memory. The legend tells you which pieces are available, used for RAM, etc. Each layer of "bricks" will have a starting address on the left and an end address on the right, in HEX. Those addresses are what you need to know when you move the pieces around later. By taking a picture of your memory map, you will be able to reference those addresses to say, "Take this and move it there", etc.
Setting Up Your BIOS
Now that you have a preliminary idea of what your memory map looks like you'll want to go into your BIOS and see if you can optimize what the BIOS is putting into your upper memory. Reboot the machine and usually the Delete key must be pressed at the very start to get you into your BIOS but if not, watch the screen because it will tell you what key to press.
Once inside the BIOS you'll be at some sort of menu screen and usually the 2nd menu says something like BIOS Features or something like that. Try the first 3 menus anyway and look for the word SHADOW. If you see that, you have the right menu. Now there should be several lines where you can enable or disable shadowing at various HEX addresses and they will be listed there. Here's the takeaway. Unless a piece of hardware you've installed requires you to shadow its ROM, best to assume that you don't need to shadow it. There are only 2 things that you should ever shadow. The first is your Video BIOS. Usually you'll get a speed boost from doing that. The second is your system BIOS for the same reason. Those should be identified by name and you should turn them on if they aren't already. If they aren't named as Video and System, you're looking for 32k between C000-C7FF for the VGA card and 64k worth of memory between F000-FFFF. Every other address that is shadowed should be disabled. What this is going to do is free up every part of your memory map that was labelled as ROM or RAM, except those segments required by the video card and the BIOS. If you look at your memory map picture and see a row that is blocked off as ROM/RAM, you should see a corresponding line in your BIOS where shadowing was enabled, which is why we're disabling it now.
After exiting the BIOS the machine will reboot. Go into MSD again and update the picture you took of your memory map.
Now that your BIOS is set up for optimal memory usage, it's time to enable your UPPER memory for use. Up until now you could see the upper memory area but not use it. This step will make MSD display a capital F (free) or capital U (unavailable) in every block of memory reserved as a UMB. It's now time to go back to your CONFIG.SYS and adjust the lines as follows:
To summarize, HIMEM.SYS will give you access to HIGH memory and DOS=HIGH will put DOS there, which we did earlier. DOS=UMB enables upper memory blocks and EMM386.EXE is the program we need to run in order to fine tune how the UPPER memory is allocated. The I=B000-B7FF and I=BC00-BFFF is a little trick we use to steal 48k of extra memory from an area normally reserved for monochrome displays. If your system has a monochrome display you can't do this so delete that part but normally this is not applicable so we can use it. One other reason you may not be able to do this is if there's currently a device loading ROM/RAM there. Again, if it doesn't work just undo the offending piece of code.
Now if you look in MSD again, you'll notice those extra 3 rows we just included and they may even have already been partially populated but the task at hand now is to fill all these UMBs without leaving spaces. Take another picture for your reference.
Move Your Device Resources Around
The best way to use all your UPPER memory is to make the memory area "contiguous", which is to say, put all the free space in 1 monolithic block instead of having it interrupted by used segments. Now, everybody's machine is different but yours will undoubtedly have some cards in it like sound or network or whatever. They may be plug-n-play but probably at least some aren't and in that case the resources are controlled by jumpers. You can consult my other 386 building article on that. Either way, your cards may be reserving certain addresses that fall in the UPPER memory area so they can put their ROMs there. Either a software utility (in the case of PnP) or the jumpers on the board should tell you what addresses are being reserved for the ROMs. You will need to determine these addresses by pulling the cards and looking at the jumpers or entering the software setup. You're looking for the same type of HEX code as those used in MSD. Consult your picture for clues as to where the devices are putting the ROMs. If you see a match, for example a row in MSD that's designated RAM/ROM, which is also called out on the jumpers on your card, then what you can do is change the jumpers to move the address in memory. You do this so that the row of memory occupied doesn't chop your free space into pieces. If you have a bunch of free space, then a full line, then more free space, try to change the address up or down until the free space is all together. Sometimes this is not possible but if it is, that's what you'll want to do with each card in your computer. As I said... playing Tetris.
Now once you find a home for all the RAMs and ROMs that your devices need to use, you'll want to check MSD again, make sure those segments are unavailable where you expect them to be, take another picture and then go back to your CONFIG.SYS and EXCLUDE those areas from being used as UMBs. To do this you use the X= switch which is similar to I= in the examples above. I means include and X means exclude. Use the same address convention and the picture you took of MSD to exclude the memory segments containing device ROM/RAM. Do it like this:
I should mention a couple of tips at this point. If for any reason your system throws an error during boot as a result of the changes you've made in your CONFIG.SYS, simply undo the changes 1 at a time until the problem goes away. The second tip is to check your addresses carefully. You are not allowed to overlap ranges. Say for example you want to include an entire 64k area of memory but then you put a line right after that telling EMM386 to exclude a 16k segment in the middle of that first range. The correct way to do that is to include the lower segment up to the start of the 16k segment you wish to exclude, then put a line to exclude that 16k segment, then add a 3rd line to include the segment on top of it. That way you have no overlapping/conflicting commands telling EMM386 to include and exclude the same segment.
Fill Your UMBs With Stuff
Now that you've told EMM386 what parts of UPPER memory to use, and you've made the space contiguous, it's finally time to start filling it.
You do this by modifying your CONFIG.SYS and AUTOEXEC.BAT files so that anything that gets loaded into memory actually goes into UPPER memory first until that's totally filled. Now, take 1 last look at your memory map picture. Others will tell you to load things in order of biggest to smallest but this isn't always the best. Depending on how your map looks, you may have an unavoidable segment of free UMBs at the bottom that could not be made contiguous (there's used space on top of it) so if that segment is only 20k for example you can only fit a 20k program into it. If you load things from biggest to smallest, that segment may get skipped and later filled with small programs. Better for you order your CONFIG/AUTOEXEC lines according to what your free space demands. In MSD if you hit ALT from the main menu, go to Utilities and then Memory Block Display, you can see the size of everything currently loaded in memory. This actually tells you how much space things are taking up so you can decide how to order them. If you see something that takes up 19k and your lowest free UMB segment is 20k, put that device first in your CONFIG/AUTOEXEC. After you've done that, if you then have a huge contiguous UMB segment, THEN you can start loading stuff biggest to smallest.
As for how to put stuff in the UPPER memory area, it's really simple. For your CONFIG.SYS, just find every instance of something that says DEVICE= and change it to DEVICEHIGH=
An important note for the CONFIG.SYS is that you CANNOT put DEVICEHIGH in front of HIMEM or EMM386 and these must come before anything else you mark as DEVICEHIGH. So make sure that HIMEM and EMM386 are at the top of your list and that they are not set to high.
For your AUTOEXEC.BAT, find every line where a path points to some file like an EXE or a COM file or something like that (ex. C:\MSCDEX.EXE /D:MSCD001) and put LH in front of it (ex. LH C:\MSCDEX.EXE /D:MSCD001). LH stands for Load High. Don't put LH in front of every line in the AUTOEXEC. Some lines will say STACKS or PATH or SET or things like that. You're only putting the LH in front of "files" that are being loaded, just as you might execute a file from the DOS prompt.
Now reboot and type mem at the C prompt and you should be able to see that your upper memory is quite a bit bigger than it used to be and that it is mostly filled. You should also note that your conventional memory has more free space than before. This is exactly the result you want.
Go back into MSD and have a look at all the Us and Fs. There shouldn't be that many Fs and they should not be scattered all over the place. If they are, you have to play with the ordering of the lines in your CONFIG/AUTOEXEC again OR you made a mistake earlier on in allocating your UMBs and need to consolidate those better. Otherwise you should be good to go.
For EMS Users
Now for you guys who need EMS, there is 1 additional step that complicates things. You need to sacrifice 64k of contiguous space in the UPPER memory area for something called a PAGEFRAME, which is where the EMS goes. You can still use the rest of your UMBs, you just won't have as many. On the plus side, you can run your EMS apps without getting an error. This is why it's good to make a second boot disk just for this setup.
To tell EMM386 to give you EMS, you need to add the RAM switch and another M switch at the end. This says, "I want EMS" and "I want it to start at this location in memory". Your CONFIG.SYS line will look something like this:
In order to know which M number to use, consult this chart:
1---> C000 8---> DC00 2---> C400 9---> E000 3---> C800 10--> 8000 4---> CC00 11--> 8400 5---> D000 12--> 8800 6---> D400 13--> 8C00 7---> D800 14--> 9000
So M9 starts at E000 and goes 4 rows up from there (64k). You will want to look in MSD first, so you can find 4 contiguous rows of free UMBs. The whole 4 rows have to be free and I recommend sticking this block at the very top or the very bottom of your UMB space so you don't fragment the rest of your UMBs. Once you do this, MSD will put a bunch of Ps in those rows to denote PAGEFRAME. That means it worked and you should be able to run your games or whatever it was that needed EMS.
It's Game Time!
So now that you've gone through all these steps, a simple mem command at the C prompt should indicate that you've moved a significant amount of memory from the Conventional category to the Upper category. This means more memory that programs are actually looking for to run and a better likelihood that they actually will run. If you've done a superb job you can get almost all your conventional memory freed up, well above the 600k mark, which is enough to run anything. Even if you're in the 500s though, most programs should run without complaint. Usually an un-optimized system is just a hair away from being able to run a program that is throwing an error so even if you mess up or don't have the discipline to waste hours on memory management, you can probably stumble your way through this article and fix the problem quite easily. The most demanding game I have found so far is Falcon 3, which likes to have a 387 co-processor, more than 600k of conventional memory and EMS on top of that... quite a tall list but if you're sufficiently motivated to make software like this run, it is totally doable and all you need is the info provided here and some Tetris skills ;)