Lecture #3

Our Story So Far (with a small smattering of new stuff)

• Operating systems function at lower levels of abstraction

  • closer to the metal
  • performance is critical

• Kernel operates in supervisor mode

  • context switching
    • application programs make system calls to request specific services
    • "timer interrupts" give kernel its share of CPU time
      • allows kernel to attend to "housekeeping" tasks & schedule other processes (timeslice)
      • prevents any userland process from "hogging" the CPU
    • hardware interrupts (devices)

• init (process #1): system startup

  • secondary function: reap zombie orphans

• Kernel concerns

  • process management
  • filesystems
  • device drivers
  • networking
  • memory management
  • accounting
  • protection/security
  • error detection

• Process: (essentially) running program

  • executable file
  • argv
  • pre-opened files
  • exit status
    • 0 = success/true
    • non-zero = failure/false

• Conventions for file descriptors

  • 0: standard input (bound to cin in C++)
  • 1: standard output (cout)
  • 2: standard error (cerr)

• File descriptors are used for low-level operations (read/write array of bytes)

  • C/C++ & other languages provide standard libraries for formatting & buffering which
  • a C FILE* or C++ iostream are language-level objects that wrap a file descriptor
    • actual I/O operations are performed internally using the system calls

• System call/trap

  • arguments are ints & pointers
  • arguments & return value are passed in registers
  • integer return value
    • <0: fail
    • =0: ok (if no other meaningful result)
    • ≥0: result (e.g. fd, pid)

• Global variable errno hold integer error code (raw info)

  • perror (standard library function) prints meaningful error message

• Unix/Linux/Posix: separation between process creation & program invocation

• pid_t pid = fork() clones process (not to be confused with Linux clone system call)

  • <0: failure (see errno)
  • =0: child process
  • >0: parent process (return result is pid of child)

• Child process inherits same open files as parents

• Parent is responsible for collecting child exit status: wait/waitpid

• Process tree

  • init is root
    • inherits/adopts orphan processes (not grandparent)
      • orphan process: its parent died first

• Fork is useful for distributing workload

  • assignment 1

• But, what if you want to run a different program

  • stay tuned...

• Pipe: original IPC (inter-process communication) mechanism

  • producer-consumer
    • producer writes data to memory buffer in kernel
      • when buffer is full, writing is blocked
    • consumer reads data from buffer
      • when buffer is empty, reading is blocked
        • unless the non-blocking I/O flag is set

• Pipe is unidirectional: one end writes; other end reads

  • for bidirectional communication, use 2 pipes
    • but watch out for deadlocks

• Pipe instance is a file descriptor (2 descriptors, one for write, other for read)

  • uses same write/read system calls as regular file
    • program generally don't need to care where the output is going to or input is coming from
  • file descriptor is object (in C++ sense): method depends on the object's type
    • but implemented in C
    • uses table of function points (like C++ vtbl)

• From programming POV (point-of-view), pipe is just like an ordinary file

  • additional system call to set non-blocking flag if you need to change the behavior for a particular application

Pipes (cont.)

• pipe system call returns a pair of open file descriptors for producer (writer) & consumer (reader)

  • each fd is one-way (end-points)

• Single process can't really use pipe descriptors

  • read empty pipe would block, then you're stuck
  • similarly, write full would block, then you're stuck
  • there is a way around this using threads, but there are better ways to do inter-thread communication

• File descriptor is index into a kernel data structure (array element)

  • table is per-process (each process has fds 0, 1, 2, ...
  • so how do you get a pipe across two processes

• Recall: files stay open across fork

  • therefore, two processes that share a pipe must have a common ancestor that created the pipe

• Typically one process closes the reader and keeps the writer while the other does the opposite

• Patterns of usage TODO: diagram

• Shell syntax: vertical bar

  • e.g. do_this | do_that
  • pipe symbol (vertical bar) used elsewhere as or operator

• Shell only supports pipelines with standard input/standard output

• system call allows more complex patterns
  • assignment 1

"Everything Is a File"

• Basic file ops (read bytes, write bytes): good abstraction for many things

  • ordinary file
  • tty (teletype): dumb terminal (keyboard & screen)
    • teletype: email version 2 (version 1 was telegraph)
  • pipe (original IPC mechanism)
  • network connection
  • audio: microphone & speakers
  • video: camera
  • blinking lights
  • pseudo devices: /dev/null, /dev/zero, /dev/random
  • other devices

• Standard input (fd 0) & standard output (fd 1) are pre-opened by parent process

  • parent is usually the shell
  • you have to go out of your way to explicity open any other file

• Standard input is not the keyboard; standard ouput is not the screen

  • defaults for interactive program

• Unless you have a good reason otherwise, your programs will be simpler & more flexible if you use standard input & standard output

  • design principle: separate policy and mechanism
  • design principle: early vs. late binding

• Shell syntax for redirecting input/output

  • < standard input from file
  • > standard output to file
  • >> standard output append file
  • 2> standard error to file
  • | pipe standard output of command 1 into standard input of command 2
  • ...

Process Management (cont.)

• fork allows a program to divide work arcorss multiple processes, each running the same program

  • pipe is just one IPC mechanism
    • we'll look at others later in the quarter

• But what if you want to run a different program?

  • excve (plus several standard library wrappers that let you marshal arguments differently)

• exec* replaces current running program in process wtih new program, new argv

  • same process, new program
  • if call to exec is successful, it does not return (you can't go home again)

• Open files (file descriptors) stay open across exec

  • unless you set the file's close-on-exec flag)

Shell

• Why separate calls for fork & exec?

  • allows shell & other processes to do some housekeeping after fork, before exec
  • sometimes you want to fork but not exec
    • assignment 1
  • sometimes you want to exec without fork
    • restart/replace init process

• We now have enough information to construct a rudimentary shell

  1. read line of input
  2. string processing to get name of executable file
  3. more string processing to get argument vector (argv)
  4. additional string processing to get stdin/stdout/stderr
  5. call fork
  6. parent process calls wait (or wait_pid)
  7. if necessary, child process closes fd 0, 1, 2, and reopens as required
  8. child process constructs argv
  9. child process calls exec
  10. new program executes, calls exit when done
  11. wait call in parent returns with child exit status
  12. lather, rince, repeat

TODO: diagram

• producer-consumer pipeline (simplified) TODO: diagram

Assignment 1: Convex Hull

TODO: diagram

• Convex hull problem

  • important (seminal) problemn in computational geometry
  • much-studied, many algorithms

• Graham scan algorithm

  • sort points by angle wrt lowest, leftmost point (lowest point, leftmost if tie-breaker needed for lowest)
  • tricks to avoid trigonometry (trig is O(1) but expensive: large constant), divide-by-zero
  • Graham scan: construct stack of points, removing points that make right turns
    • did we mention that this only works in the plane

• Performance analysis:

  • find min: O(N)
  • sort: O(N log N)
  • graham scan: O(N)
  • total: O(N log N)

TODO: diagrams

• Divide & conquer
  • given 2 convex hulls, find convex hull of union

TODO: diagrams

• Divide & conquer algorithm

  • divide points into 2 sets
  • separately compute hulls of both sets
  • find upper chain of upper hull (O(N))
  • merge with lower hull (O(N))
  • Graham scan of merged sets (O(N))
  • recursion (divide in half): O(lg N)
  • ==> total running time O(N lg N)

• The assignment is to use fork to compute the halves in separate processes

  • concurrent independent operations
  • exploit modern multicore architectures: 2 (or more) cores running at the same time
  • child process must communicate results back to parent (pipe)
  • skeleton does everything except mutiprocess part
    • don't need to worry about the algorithm, just the systems programming part