RAM and the Address Bus

Memory is often called RAM, for random-access memory. "Random access" means you can read any byte directly, in any order, just by giving its address. That's unlike a tape, where you have to wind past everything in between. In this chapter we'll answer three questions: how does an address pick out a byte? How is a bit physically stored? And why does it take so long?

Addresses and the address bus

The processor and memory are connected by three groups of wires:

• The address bus carries which byte the processor wants, as a binary number.
• The data bus carries the byte itself, in or out.
• A few control wires say whether this is a read or a write, and when.

Inside the memory, a circuit called an address decoder turns the address into exactly one "select" signal, connecting just that one byte to the data bus. It's built from AND gates we already know. For 3 address wires there are 8 possible addresses, so there are 8 AND gates, each one wired to recognize exactly one pattern of 1s and 0s.

Each extra address wire doubles the number of patterns, and so doubles how much memory you can reach. With n wires you can address 2ⁿ bytes. This is why memory sizes are always powers of two, and why a "kilobyte" in computing traditionally means 1,024 bytes (2¹⁰), not 1,000.

The IBM PC's megabyte

The 8088 had 20 address wires, so it could reach exactly 1 MB. But its registers were only 16 bits wide, and 16 bits can only count to 65,535. How do you build a 20-bit address from 16-bit registers? Intel's answer was segments: the address is written as two 16-bit numbers, segment:offset, and the processor computes segment × 16 + offset. Multiplying by 16 is just shifting left by one hex digit:

  B800:0000   →   B800 × 16 + 0000   =   B8000      (text on the screen, Chapter 2)
  FFFF:0000   →   FFFF × 16 + 0000   =   FFFF0      (the first instruction, Chapter 4)

IBM then divided that megabyte between memory chips and other hardware. Not every address leads to RAM: some lead to the video card's memory, and some to ROM chips holding the BIOS. This layout is called the memory map:

That's the origin of the famous 640 KB limit. In 1981, when the base PC shipped with 16 KB, 640 KB seemed enormous. A decade later, programs were bumping into it, and DOS users spent years juggling "extended" and "expanded" memory schemes to get around it.

Bytes, words and byte order

Memory is addressed one byte at a time, but numbers are usually bigger than a byte. A 32-bit number takes four consecutive addresses. That raises a surprisingly contentious question: which byte goes at the lowest address?

• Little-endian: the lowest byte (the "little end") goes first. This is what x86, RISC-V and nearly all ARM systems do. We've already seen it in Chapters 5 and 7: mov eax, 5 is B8 05 00 00 00.
• Big-endian: the highest byte goes first, the way we write numbers on paper. Many older processors worked this way, and it's still the standard order for numbers sent over the internet.

The names come from Gulliver's Travels, where two nations go to war over which end of a boiled egg to break first. A computer scientist borrowed them in 1980 to poke fun at how fiercely engineers argued about byte order.

Inside a DRAM chip

In Chapter 3 we built memory from loops of gates: flip-flops for registers, and six-transistor SRAM cells for caches. Both are fast, but they take up a lot of room on a chip. Main memory needs billions of bits, cheaply. So it uses a much simpler trick, called dynamic RAM, or DRAM.

A DRAM cell is just one transistor and one tiny capacitor. A capacitor is like a microscopic bucket for electric charge: full means 1, empty means 0. The transistor is the bucket's tap, opened when the cell is selected. That's six times fewer transistors than an SRAM cell, which is why DRAM is so dense and cheap.

The catch is that the bucket leaks. Within a fraction of a second, a full cell drains until a 1 can no longer be told apart from a 0. So the memory must constantly refresh itself: read every row and write it back at full strength, over and over, before the charge leaks too far. That's why it's called dynamic memory: it only remembers as long as it keeps being refreshed.

In the demo, things are slowed down enormously so you can watch. A real DRAM chip refreshes every row every 32 or 64 milliseconds, thousands of rows at a time, and the processor barely notices. There's another subtlety: reading a DRAM cell drains it, so every read has to be followed by writing the value back. Memory chips do this automatically.

Rows, columns and the row buffer

Inside each chip, the cells are arranged in a grid of rows and columns. And the chip can't read a single cell on its own. To read anything, it first opens a whole row: every cell in that row is read at once into a row buffer, a line of fast sensing circuits along the edge of the grid. Then the requested column is picked out of the buffer.

Once a row is open, reading other bytes from the same row is quick. But to read from a different row, the chip has to close the current row (writing it back) and open the new one, and that takes time.

Reading in order is about twice as fast here, because consecutive addresses live in the same row. Real memory controllers exploit this hard, and so do programs that walk through memory in order. Keep this lesson in mind: where your data sits in memory matters, not just how much there is.

The memory wall

Here's the problem that dominates modern computer design. Over the last 40 years, processors have become thousands of times faster. Memory has become enormously bigger, and much better at delivering lots of data at once. But the time to fetch one particular byte, called the latency, has barely improved at all.

• On the IBM PC, one memory access took 4 clock ticks of the 8088, about 840 nanoseconds. The memory chips of the day, at around 200 ns, had time to spare.
• Today, a trip from the processor to DRAM and back takes something like 80–100 nanoseconds. At 5 GHz, that's around 400–500 clock ticks in which the processor could have done useful work.

Meanwhile, the amount of data memory can deliver per second, its bandwidth, has grown enormously. A single module of modern DDR5-6400 memory moves 6,400 million transfers of 8 bytes each second: about 51 GB/s. ("DDR" stands for double data rate: data moves on both the rising and the falling edge of the clock.)

So memory is like a very wide pipe that's also very long. If you ask for one byte and wait, you wait a long time. If you ask for lots of nearby bytes at once, they arrive quickly. Processors deal with this "memory wall" by keeping copies of recently used data in small, fast SRAM memories right on the chip. Those are caches, and they're the subject of the next chapter.

Meanwhile, inside a real CPU

Byte order shows up whenever a program stores a number and then reads part of it back. Both of our real ISAs are little-endian:

; x86-64
mov  dword [rdi], 0x12345678    ; memory at rdi: 78 56 34 12
mov  al, [rdi]                  ; al = 0x78: the lowest byte sits at the lowest address

# RISC-V
li   t0, 0x12345678
sw   t0, 0(a0)                  # memory at a0: 78 56 34 12
lbu  t1, 0(a0)                  # t1 = 0x78

Both ISAs can load and store 1, 2, 4 or 8 bytes at a time. Ideally, a 4-byte number sits at an address divisible by 4, and an 8-byte number at an address divisible by 8; this is called being aligned. x86 handles misaligned accesses in hardware, at a small cost. On RISC-V, a misaligned access may be slow, or may even need help from software, so compilers take care to keep data aligned.

Try it yourself

1. How many address wires do you need to address 64 KB? What about 4 GB?
   Show answer

   16 wires for 64 KB (2¹⁶ = 65,536), and 32 wires for 4 GB (2³² = 4,294,967,296).
2. On the 8088, which physical address does 1234:0010 refer to?
   Show answer

   1234 × 16 = 12340, plus 0010 = 12350 (hex).
3. A little-endian processor stores the 32-bit number 0xCAFEBABE at address 200. What byte is at each of addresses 200 to 203?
   Show answer

   200: BE, 201: BA, 202: FE, 203: CA. The lowest byte comes first.
4. In the row buffer demo, reading 16 bytes in order took 266 ns. Where does that number come from?
   Show answer

   The first read opens row 0: 14 + 14 = 28 ns. The next 7 bytes are row hits: 7 × 14 = 98 ns. Byte 8 is in row 1: close, open and read, 42 ns. Then 7 more hits: 98 ns. Total: 28 + 98 + 42 + 98 = 266 ns.
5. Why does DRAM need refreshing, while SRAM and flip-flops don't?
   Show answer

   SRAM and flip-flops store a bit in a feedback loop of gates that actively holds the value as long as there's power. DRAM stores it as charge in a capacitor, with nothing actively holding it, and the charge slowly leaks away.

What we learned

• The processor talks to memory over an address bus, a data bus and control wires.
• An address decoder (AND gates) selects exactly one location. n address wires reach 2ⁿ bytes.
• The 8088 built 20-bit addresses from segment:offset pairs, and the IBM PC's memory map left 640 KB for programs.
• Multi-byte numbers are stored little-endian (x86, RISC-V) or big-endian.
• DRAM stores each bit as charge in a capacitor. It's dense and cheap, but it leaks and must be refreshed.
• DRAM opens whole rows at a time, so nearby accesses are faster than scattered ones.
• Memory latency hasn't kept up with processors: hundreds of clock ticks per trip. This "memory wall" is why caches exist.

In the next chapter we'll build a cache: a small, fast memory that keeps copies of recently used data close to the processor. You'll see how it decides what to keep, why it fetches 64 bytes when you asked for one, and why some programs run ten times faster than others that do the same work.