Down to the Metal

Part VI · Chapter 16

Framebuffers: From Text Mode to Pixels

Everything you see on a screen, every letter, photo and window, comes from a block of ordinary memory. The program writes numbers into it, and a circuit reads them out, over and over, dozens of times a second, and turns them into light. In this chapter we look at that memory, from the IBM PC's text mode to today's 4K displays.

We've already met it twice. In Chapter 8, the PC's memory map had a region set aside for video at 0xB8000. In Chapter 11, we saw that storing a byte there makes a letter appear on the screen. That region is a framebuffer: memory whose contents are the picture. Change the memory, and the screen changes.

Text mode: characters and colors

The IBM PC's color graphics card (CGA) normally ran in text mode: the screen was a grid of 80 columns by 25 rows of characters. Each character cell takes exactly two bytes of memory:

So the whole screen is 80 × 25 × 2 = 4,000 bytes, starting at 0xB8000 with the top-left corner and going row by row. The video card does the drawing: for each cell, it looks up the shape of the character in a font stored in its own ROM, 8 × 8 dots on the CGA, and paints it in the chosen colors.

This is the top-left corner of an 80 × 25 screen. Click any cell to see its two bytes, then type, pick a character, or change its colors. The blinking cursor at the bottom is just a _ with its blink bit set.

Text mode was wonderfully economical. Writing one letter meant storing two bytes, and the 16 KB of memory on the CGA card held four complete screens. It's also why DOS-era programs all have that blocky, colorful look: they could only arrange characters on a grid.

From characters to pixels

For pictures, the card could switch to a graphics mode, where every dot on the screen, every pixel, gets its own little piece of memory. How big a piece depends on how many colors you want:

Paint some pixels, then switch between the modes and watch both the picture and the bytes. Click a pixel to see exactly which byte (and which bits) hold it.

The trade-off is memory. The CGA's 320 × 200 pixels at 2 bits each need 16,000 bytes, just under its 16 KB. The VGA's 320 × 200 at one byte per pixel need 64,000 bytes, which fitted neatly in the 64 KB window at address 0xA0000, and made it the favorite mode of a whole generation of PC games. A 4K screen today has 3840 × 2160 pixels at 4 bytes each: 33 million bytes for a single picture.

Finding a pixel is simple arithmetic. If each row of the screen takes pitch bytes, the pixel at column x and row y lives at:

address = framebuffer + y × pitch + x × (bytes per pixel)

Painting the screen, 60 times a second

A screen doesn't remember anything by itself. The picture has to be sent to it again and again, from the framebuffer, many times a second. This is called scanout, and it always happens in the same order: row by row, from the top of the screen to the bottom, and each row from left to right.

That order comes from the old cathode-ray tube (CRT) monitors. Inside, a beam of electrons was swept across the glass, lighting up glowing dots as it went. It drew one row, flew back to the left (the horizontal blank), drew the next row, and when it reached the bottom, flew back up to the top (the vertical blank). The PC's monitor drew 60 frames a second, about 15,700 rows every second. Today's flat panels have no beam, but HDMI and DisplayPort cables still send the pixels in exactly that order, with the blank gaps still in place.

That makes the display controller one of the hungriest readers of memory in the computer. A 4K screen at 60 frames a second needs 3840 × 2160 × 4 × 60 bytes: about 2 GB of memory read every second, just to keep the picture on the screen.

Tearing and double buffering

Scanout causes a problem for anything that moves. Drawing a new picture takes time, and the display keeps reading the framebuffer while the program is changing it. If the program replaces the picture while the display is halfway down the screen, the top half of that frame shows the old picture and the bottom half the new one. A moving object appears sliced, a glitch called tearing.

A bar slides across the screen. The program draws a new picture whenever it's ready, without paying attention to the display. Watch the white line, the row being sent to the screen right now. Then switch to double buffering.

The cure is double buffering. The program draws each new picture into a second, hidden framebuffer, the back buffer, while the display shows the front one. When the new picture is finished, the two are swapped. No pixels are copied: the display controller is simply told to start reading from the other buffer. Doing the swap only during the vertical blank, called waiting for vsync, guarantees that each frame shows one complete picture.

Waiting has its own cost: if a picture isn't ready in time, the old one is shown again, and motion stutters. That's why modern monitors can do the opposite and wait for the computer: with variable refresh rate (FreeSync, G-Sync, Adaptive-Sync), the screen starts a new frame whenever a new picture is ready.

Meanwhile, inside a real CPU

Putting a white-on-blue letter A in the top-left corner of a text screen takes one store:

mov     word [0xB8000], 0x1F41  ; 66 c7 04 25 00 80 0b 00 41 1f

Look at the last two bytes of the instruction: 41 1f. x86 is little-endian (Chapter 8), so the low byte 0x41, the letter A, lands at 0xB8000, and the attribute 0x1F, white on blue, at 0xB8001. Exactly the order text mode wants.

Drawing one pixel in a 32-bit framebuffer is the formula from above:

; x86-64: rdi = framebuffer, esi = x, edx = y, ecx = color 0x00RRGGBB
imul    edx, edx, 15360         ; y × pitch (3840 pixels × 4 bytes)
lea     rax, [rdi + rdx]        ; start of row y
mov     [rax + rsi*4], ecx      ; x × 4 bytes in: store the color
# RISC-V: a0 = framebuffer, a1 = x, a2 = y, a3 = color
li      t0, 15360               # the pitch (becomes two instructions)
mul     t1, a2, t0              # y × pitch
add     t1, a0, t1
slli    t2, a1, 2               # x × 4
add     t1, t1, t2
sw      a3, 0(t1)               # store the color

Notice how x86's addressing can multiply rsi by 4 inside the store itself, while RISC-V uses separate simple instructions. And the number 15,360 is too big for a single RISC-V li, so the assembler quietly turns it into two instructions.

That pitch isn't made up. Linux still offers a simple framebuffer for its text console, and on the computer this book was written on it reports:

$ cat /sys/class/graphics/fb0/{name,virtual_size,bits_per_pixel,stride}
i915drmfb               ← provided by the Intel graphics driver
3840,2160               ← a 4K screen
32                      ← bits per pixel
15360                   ← bytes per row: 3840 × 4

Try it yourself

  1. In 80-column text mode, at what address are the two bytes for row 2, column 5? (Both start at 0.)
    Show answerCell number 2 × 80 + 5 = 165, and each cell takes 2 bytes: 165 × 2 = 330 = 0x14A. So the character is at 0xB814A and its attribute at 0xB814B.
  2. What does the attribute byte 0x4E mean?
    Show answer0x4E = 0100 1110. Bit 7 is 0: no blinking. Bits 6–4 are 100 = 4: a red background. Bits 3–0 are 1110 = 14: yellow characters.
  3. How many bytes does a 640 × 200 screen need at 1 bit per pixel?
    Show answer640 × 200 = 128,000 pixels, at 8 pixels per byte: 16,000 bytes. Exactly the same memory as 320 × 200 at 2 bits per pixel: twice the pixels, half the colors.
  4. At 32 bits per pixel, how much memory does a 1920 × 1080 picture need? How much must be read per second at 60 frames per second?
    Show answer1920 × 1080 × 4 = 8,294,400 bytes, about 8 MB. At 60 frames per second: about 500 MB every second, just for scanout.
  5. With double buffering, why swap the buffers during the vertical blank and not immediately?
    Show answerIf you swap while the display is halfway down the screen, the rest of that frame comes from the new buffer: the same tear as before. During the vertical blank, no visible row is being sent, so every frame is read entirely from one buffer.

What we learned

Filling 8 million pixels, 60 times a second, with 3D scenes, video and smooth animations is far too much work for an ordinary processor. In the next chapter we meet the specialist built for it: the GPU, thousands of small cores running little programs called shaders.