CSS 343: Notes from Lecture 3 (DRAFT)

Administrivia

• is everybody having fun yet?
• assignment 1
  • dropbox for assignment 1 coming soon...
  • complete parts 2 & 3 by Thursday (original due date) using the mock allocator.
    • mocking or stubbing out modules lets you test client modules in isolation
      • this works because you're providing an alternate implementation for the same interface
      • a practical benefit of abstraction
    • may also let you test the mocked out module by giving you a baseline to compare behavior with
    • a mock could be a simpler (less efficient) implementation or a partial implementation, or a routine that returns pre-canned results for a finite set of input values
      • whatever suits the needs of the moment
  • complete part 1 by Tuesday
  • if your humble instructor has the opportunity to teach this course again, the rough edges will get sanded down
    • thank-you for beta-testing this course
  • OK to use cin/cout instead of fgets/printf
    • using unfamiliar I/O routines doesn't advance the learning objectives of the assignment
  • the sample Run script was overkill and slightly buggy (sorry!)
    • for the purposes of the assignment, all that is required is the command lines that you would type at the console to compile and run the program
  • .cc is the same thing as .cpp
    • also same as .cxx (hipster programmer says 'x' is a sideways '+') and .C
    • the extension does not matter; use whatever's comfortable
      • as long as it matches your Run script
    • consistency is good: use the prevailing standard of your shop
    • your instructor speaks C++ with an accent

Assignment 1

• assignment 1 is about jumping across layers of abstraction
  1. application
  2. library
  3. low-level system service
• we play games with (break) the abstractions in several ways:
  • we treat memory in the allocator as a large char array, a linked list of free blocks, and blocks of allocated memory
  • we intentionally write past the end of a struct
  • the interface of the linked-list library is (intentionally) crippled
    • missing functionality (locate function) has to move into the application layer
• our allocator mimics the functionality of C library functions malloc() and free()
  • in fact our mock allocator is a thin wrapper around the library functions
  • the allocator returns a pointer to memory, but it is not initialized in any way, so the caller has to cast the pointer to the desired type and do whatever initialization is required
  • C++ operators new and delete perform memory allocation/deallocation and invoke the appropriate constructor/destructor function
    • new returns a pointer to a well-formed (i.e. properly initialized) object automagically

Running Off the End of a Structure

question: what does this do:

#include <iostream>
int main() {
   int i;
   int a[10];
   for (i = 0; i <= 10; ++i) {
      a[i] = 0;
   }
   cout << "initialization complete\n";
   return 0;
}
	

• the program is buggy: 11 of the 10 array elements are initialized
  • intuitively, we are writing past the end of the array and we don't know what is supposed to be there that we are scribbling over
  • this is a classic off-by-one or fencepost error
  • C/C++ lets you do this because, in the name of efficiency, it does not check array subscripts at run time
    • C was designed to compete against assembly-language programming
    • assembly doesn't even have the concept of an array: you have to do the pointer arithmetic yourself
    • C exposes the pointer arithmetic too
    • this is yet another reason why C-style arrays are evil in C++
      • generally better to use std::vector
    • criticizing unchecked array subscripts begs the question of whether there is, in fact, a universally good way to signal an array subscript error
      • C++ allows you to create your own array-like class that check subscripts and deals with out-of-bounds errors in some way that works for your particular needs
  • this kind of nonsense is why people wind up writing books with titles like Enough Rope to Shoot Yourself in the Foot: Rules for C and C++ Programming
• the language standard says a program like this gives undefined behavior
  • "undefined" means absolutely anything can happen
    • hence the term nasal demons
  • a program with undefined behavior may behave differently with with the same compiler on the same operating system when compiled with different flags (e.g. debug vs. release build)
    • debug flags typically specify low optimization and enhanced symbols and line number data in the object file (may not use memory & registers efficiently)
    • release builds tend to be highly optimized (may use memory differently, e.g. the loop variable may be kept in a register and never written to memory)
    this means you can have something that seems to work fine during develoment but breaks in the wild
• this kind of buffer overrun bug has been the root cause of enormous damage, including financial damage and loss of life
  • among other things the original internet worm (not named after your humble instructor) exploited a buffer overrun in the fingerd server
  • the (deprecated) standard C library function gets() is particularly unsafe because it exposes the bug to malicious input
• actual, observed behavior (with different system/compiler/flags combinations) includes:
  • error/warning message is issued during compilation
    • depends on the specific compile-time analyses performed by the compiler
    • this particular case may be detectable, but the general case is not (cf. the halting problem)
  • program appears to work as intended
    • the memory location beyond the end of the array is not used for anything important
    • this might be due to alignment requirements, or numerous other reasons
  • program crashes with a segmentation fault
    • the program writes to an address that is protected by the operating system
  • program goes into an infinite loop and never prints out the message
    • that's particularly surprising until you realize that a[10] aliases i
    • this can happen even though i is declared first
    • it can also happen if i is declared second
    • depends on the implementation
• this program differs from Assignment 1 because, in Assignment 1, we are intentionally writing past the end of our struct into memory specifically reserved for this purpose
  • this idiom is legal and accepted (in some vulgar circles), but ugly
  • don't ever do this unless you have a damned good reason
    • there do exist good reasons (e.g. serialized data for networking protocols) but if you think you have a good reason, think twice and then think again

Interlude: A Random Musing About the Payload

Inside our list structure, we defined a string for the word data and an integer counter as part of the list node structure. For a library, this is not very general. Word and count are not a fundamental part of the list algorithms. There are several approaches of variable reasonableness to solve the problem (all of which have been applied):

1. copy the list code and substitute the payload field(s) using your favorite text editor
2. write a program (in any programming language, but some languages are easier than others) that will read input and write output like this:

   for each line of input
     if line does not contain "$$PAYLOAD"
       print line unchanged
     else
       print the part before "$$PAYLOAD"
       print the payload definition
       print the part after "$$PAYLOAD"
   	  

3. use the C/C++ #define mechanism (but this is not fundamentally different different than case 1)
4. use the preprocessor #include mechanism and some compiler flags to select a file that contains some definition for a Payload class
5. use C++ inheritance
6. use C++ templates

   template <typename T> struct Node {
      T* payload;  // or "T payload;" depending on your resource management
      Node<T>* Next;
   };
   	  

C++ templates are the preferred mechanism for solving exactly this sort of problem.

More About Sequential Processing

• "processing stuff sequentially" is a higher-level abstraction for linked lists
  • this is a low level of abstraction:

    struct Node {
      //payload
      Node* next;
    };
    	      

  • this is a far, far preferable design:

    class Node {
    public:
      //payload
      Node* next() {return next_;}
    private:
      Node* next_;
    };
    	      

  • In the former design, the client code must step through the list using idom like this: current = current->next;
  • In the latter case, the idiom is: current = current->next();
  • The syntactic difference between the two is is miniscule but the semantic difference is huge: in the later case, you can swap out the implementation entirely without altering the client code
  • Alternate implementation:

    class Node {
    public:
      //payload
      Node* next() {return this+1;}
    private:
      static Node[MAX_NODE_COUNT];
    };
    	      

• sequential operations:
  • is_empty
  • first
  • last
  • next
  • previous
  • insert_first
  • insert_last
  • insert_before
  • insert_after
  • remove
  • remove_before
  • remove_after
  • remove_first
  • remove_last
• asymptotic performance of each operation depends on the particular implementation (typically either O(1) or O(N) where N is the number of elements in the list)
• common implementations include:
  • singly-linked linked list
  • doubly-linked linked list
  • arrays
• special cases of sequential processing (may be implemented as adaptors over general-purpose routines):
  • insert last, remove first: queue (or LIFO: Last In First Out)
  • insert last, remove last: stack (or FIFO: First In First Out)
  • insert first or last, remove first or last: deque
• sequential processing is so important that the latest C++ standard introduced a range-based for statement
• in practice, use the C++ std::list or std::vector routines
• C++ uses operator overloading to introduce the concept of iterators that mimics the idiom of pointer arithmetic

  for (it = container.begin(); it != container.end() ; ++it) {
    // process *it
  }
  	  

  this is high level of abstraction because ++ does not mean what you think it means
  • abusing operator overload for fun and profit

Nonsequential Processing

• list operations that are nonsequential but could be implemented in O(N) time:
  • find total number of elements (could be O(1) if running tally is kept)
  • find max/min element (could be O(1) if data is sorted, but sorting is O(N log N)
  • find nth (not "Nth") element (O(1) nth operation is practically the definition of an array)
  • find some element by its value
• there's a pattern here: all operations involve finding something

Interlude: (Mis-)Use of a Phone Book

• given a name, finding a number in the phone is an O(log N) operation and only takes a few moments, even for the NYC telephone book
• given a phone number and finding its name is O(N) operation and is intractable even for the East-Side-of-Lake-Washington telephone book
• finding the name for a phone number only takes a few moments if you already know the page and column number—the difference is scale or the actual value of N

Dictionaries

• finding stuff by its value is an extremely common operation—so common, in fact, that we have many names for the concept:
  • symbol table
  • lookup table
  • table
  • associative array
  • map
  • set
  • relational database
  • hash (that's a Perlism, but we'll get to hashing later in the quarter)
• the abstraction is a (key, value) pair
• operations:
  • insert
  • lookup
  • for_each_element (maybe)
  • for_each_element_in_order (maybe)
• the abstraction is a (key, value) pair
• dictionaries are distinct from arrays: arrays are about finding the nth element; with dictionaries you don't know n for the item you're seeking
• dictionaries may be implemented using a linked list (we did this for assignment 1 at a low level of abstraction)
  • linked list give O(N) performance
    • not great performance, but probably ok if
      • N is small, or
      • just doing a very small number of lookup operations
• can we do better than O(N)?
  • obviously, since we just did so with the phone book
  • sort the data, and use binary search algorithm
    • but sorting is O(N log N) which is much greater O(N), so where is the savings?
      • data might arrive already in sorted order so you get it for free
      • sorting might happen "offline" when you don't care about responsiveness
      • a large number, say O(N), of lookups would be O(N log N) with sorting versus O(N**2) unsorted
  • of course, sorting and binary search require an ordering predicate
• reverse index: separate sorted arrays for each key

  data by name		original (raw) data		data by value
  name ref 0 bambam 5 1 barney 1 2 betty 3 3 fred 0 4 pebbles 4 5 wilma 3		name value 0 fred 42 1 barney 68 2 wilma 33 3 betty 24 4 pebbles 18 5 bambam 54		value ref 0 18 4 1 24 3 2 33 2 3 42 0 4 54 5 5 68 1

  • this may lead to screwy nested array subscripting like this: value = data[ by_name[i].ref ].value
• the big problem with sorting & binary search is that it is inflexible
  • insert/delete is expensive (O(N))
  • data must fit into an array: need to know N before you start
  • often requires ofline (batch) processing
• one way to look at binary search: binary search tree (BST) with implicit structure

Trees and Binary Trees

A binary search tree (BST) is a binary tree with additional properties, so let's look at trees in general first. Trees express hierarchical relationships:

• a node may have zero or more children and is the parent of its children
• a parent link (equivalent to the previous link in a doubly-linked list) is handy for some algorithms, but is frequently unnecessary
  • most tree algorithms are recursive and the chain from the root is stored in the call stack of the recursive function calls
• the root is a distinguished node that has no parent
• a node with no children is a leaf
• a node that is not the root and not a leaf is an interior node
• for every node, there is a path to it from the root via the parent-child relationship
  • a descendant is a child or the descendant of a child
  • an ancestor is a parent or the parent of an ancestor
• the height of a node is the length of the longest path downward from the node to a descendant leaf
• the depth of a node is the length of the longest path from the root to the node

In a binary tree, a node has up to two children which we shall call left and right

• an interior node that only has one child is a knee (the child may be left or right)
  • but this does not appear to be common terminology
    • maybe because it doesn't mesh with the genealogy theme
• traversal: visit every node (recursive algorithms)
  • preorder: process current node, then visit left child, then visit right child
  • inorder: visit left child, then current process node, then visit right child
  • postorder: visit left child, visit right child, then process current node
• additional info at the wiki page

Binary Search Tree

A BST is a binary tree and an ordering relation where, for each node, every node in its left subtree is less than the node and every node in the right subtree is greater than the node

• if node has a left child, immediate predecessor is rightmost left decendant
• otherwise, sucessor is first ancestor whose child on the path is on a right branch
• if node has a right child, immediate successor is leftmost right descendant
• otherwise, predecessor is first ancestor whose child on the path is on a left branch
• if successor/predecessor is a descendant, it must be a leaf or knee (otherwise, it would have a right/left child which is closer to the ancestor)
• an in-order traversal visits the nodes in sorted order
• insert/delete/find operations are O(log n) if the tree is reasonably balanced (average case)
• insert at knee or leaf
• delete: three cases
  1. leaf node: just delete
  2. node has one child: replace node with its single child
  3. node has two children: replace with immediate predecessor or sucessor (either of which must be a leaf or knee) and reduce to previous case
• nth operation is O(N) without preprocessing
• nth can be found in O(log N) time with suitable preprocessing
  • each node stores the number of children it has (which is updated in O(log N) time during insert/delete
  • recursive function must pass down a count-so-far value

Problem with Binary Search Tree

• Trees can become unbalanced
  • an degenerate tree is a linked list with O(N) search performance
  • common cases (input already sorted or reverse sorted) generate degenerate trees (similar to Quicksort problem)
  
• In practice, randomizing the input order gives good average performance
• a rotation transformation may be applied to any BST to obtain another valid BST (with the same data)
• a right rotation shortens the left subtree and lengthens the right subtree (by one node each)
• similarly, a left rotation shortens the right subtree and lengthens the left subtree
• we can balance a BST if we are suitably clever about which node pairs we chose to rotate
• various tree-balancing techniques establish a notion of balance and make appropriate rotations during insert and delete to maintain the balance invariant
  • rebalancing may give a performance penalty for insert/delete in average case (same O(log N) asymptotic performance, but larger constant)
  • rebalancing may in some cases be more efficient as it avoids O(N) insert/delete in the degenerate case
  • the tradeoff is guaranteed faster lookup time (possibly asymptotically faster, i.e. O(log N) vs. O(N)