Week 11.b CS 5600 03/30 2022 On the board ------------ 1. Last time 2. mmap 3. I/O architecture 4. device drivers ------------------------------------------------------- Admin: --Lab2 cheating detection results --confess before Midnight today --only several hours left --for those who haven't confessed, please expect to receive emails from us tomorrow --Lab3 challenge implementation --you have to implement the kv data structure and synchronization yourself --VM review session --revisit walking page tables --go through the examples in HW6 --------- 1. Last time -- page faults -- two types: not present & violating permissions -- page fault handler (another case of OS configures and hardware executes) -- very versatile -- the most classic usage: overcommitting memory 2. mmap --a syscall; a cool way to bring some ideas together. --recall some syscalls: fd = open(pathname, mode) write(fd, buf, sz) read(fd, buf, sz) --we've learned fds before, but what's an fd in kernel? --indexes into a table maintained by the kernel on behalf of the process --syscall: [see slides] void* mmap(void* addr, size_t len, int prot, int flags, int fd, off_t offset); --means, roughly, "map the specified open file (fd) into a region of my virtual memory (close to addr, or at a kernel-selected place if addr is 0), and return a pointer to it" NOTE: the "disk image" here is the file we've mmap()'ed, not the process's usual backing store. The idea is that mmap() lets the programmer "inject" pages from a regular file on disk into the process's backing store (which would otherwise be part of a swap file). --after this, loads and stores to addr[x] are equivalent to reading and writing to the file at offset+x. --why is this cool? - example: mmap enables copying a file to stdout without transferring data to user space [see handout from last time, week11a] Question: mmap vs. normal read/write, which is faster? by how much? [Answer: --mmap is faster --"by how much" depends on the file size and the underlying machine. --for ~1G file on my machine (M1 Macbook Air), mmap is 13.5x faster. ] NOTE: the process never itself dereferences a pointer to memory containing file data. NOTE: this saves two sets of memory-to-memory copies (kernel-to-user, user-to-kernel), versus the "naive" solution of read()ing into a buffer in user space, and then write()ing [Also, a well-tuned buffer cache manages which file pages are kept in RAM, rather than leaving the app developer to have to explicitly try to manage that (and potentially have the OS page replacement algorithm underneath make conflicting decisions).] - other examples: - reading big files. map the whole thing, rely on the paging mechanism to bring the needed pieces into memory as necessary - shared data structures, when flag is MAP_SHARED - file-based data structures: - load data from file, update it, write it back - this is implemented entirely with loads/stores Question: how does the OS ensure that it's only writing back modified pages? --how's mmap implemented?! (answer: through virtual memory, with the VA being addr [or whatever the kernel selects] and the PA being what? answer: the physical address storing the given page in the kernel's buffer cache). --have to deal with eviction from buffer cache, so kernel will need a data structure that maps from: Phys page --> {list of (proc,va) pairs} note that the kernel needs this data structure anyway: when a page is evicted from RAM, the kernel needs to be able to invalidate the given virtual address in the page table(s) of the process(es) that have the page mapped. 3. I/O architecture general: [draw picture: CPU, Mem, I/O, connected to BUS] --lots of details. --fun to play with. --registers that do different things when read vs. written. [draw some registers in devices, with status and data] CPU/device interaction (can think of this as kernel/device interaction, since user-level processes classically do not interact with devices directly. A. Mechanics of communication (a) explicit I/O instructions [sometimes called port I/O, port-mapped IO (PMIO)] instructions: outb, inb, outw, inw operands: IO address space (separate from memory address space) examples: (i) reading keyboard input. --I/O ports (PS/2): IO Port Access Type Purpose 0x60 Read/Write Data Port 0x64 Read Status Register 0x64 Write Command Register [https://wiki.osdev.org/%228042%22_PS/2_Controller] --keyboard keycode --handout uses "Scan Code Set 1" --differentiate "pressed" and "released" by the most significant bit: 0x1E: A pressed 0x9E: A released (notice: the difference is 0x80, namely binary: b10000000) Question: how many keys can be mapped to the key code? [answer: 128, because a key code is 1B, indicating 256 (2^8) key code; and each key has pressed and released; making it 128 keys.] --more keys? a mode change: --two bytes received, starting with 0xE0 --a lot of multimedia keys, like volume up/down --see "Scan Code Set 1" for more details (link below) [https://wiki.osdev.org/PS/2_Keyboard#Scan_Code_Set_1] an implementation of reading a key stroke: keyboard_readc(); [see handout] (ii) setting blinking cursor. see handout console_show_cursor(); (b) memory-mapped I/O physical address space is mostly ordinary RAM low-memory addresses (<1MB), sometimes called "DOS compatibility memory", actually refer to other things. You as a programmer read/write from these addresses using loads and stores. But they aren't "real" loads and stores to memory. They turn into other things: read device registers, send instructions, read/write device memory, etc. --interface is the same as interface to memory (load/store) --but does not behave like memory + Reads and writes can have "side effects" + Read results can change due to external events Example: writing to VGA or CGA memory makes things appear on the screen. See handout (last panel): console_putc() To avoid confusion: this is not the same thing as virtual memory. this is talking about the *physical* address. --> is this an abstraction that the OS provides to others or an abstraction that the hardware is providing to the OS? [the latter] Question: the memory layout seems different from what we saw on "where is OS live". [notice that this layout is PYSICAL memory address!] [if you're interested, check out these slides: https://opensecuritytraining.info/IntroBIOS_files/Day1_00_Advanced%20x86%20-%20BIOS%20and%20SMM%20Internals%20-%20Motivation.pdf] (c) interrupts [we've seen interrupts many time starting from "trapping to kernel" to "scheduling"] (d) through memory: both CPU and the device see the same memory, so they can use shared memory to communicate. --> usually, synchronization between CPU and device requires lock-free techniques, plus device-specific contracts ("I will not overwrite memory until you set a bit in one of my registers telling me to do so.") --> as usual, need to read the manual B. Polling vs. interrupts [next time] [Acknowledgments: Mike Walfish, David Mazieres, Mike Dahlin, Brad Karp]