Lecture #13

Administrivia

• lab 3 posted

• will try to get assignment 3 out by end of week

• no progress grading assignment 1

• still accepting assignment 2 until Wednesday

Our Story So Far

• filesystem as data structure

• file control block (aka inode)

  • file metadata (creation time/ownership/size/...)

• multiple filesystem types

• directory: "special file"

  • Unix: just mapping between name and inode

• hard vs. soft (symbolic) link

• superblock

  • where are the inodes?
  • free/allocated blocks
  • root inode

• disk allocation

  • linked list (serial data)
  • contiguous block (random access, but fragmentation may be a problem)

FAT Filesystem

• FAT: File Allocation Table

  • MS DOS (pre-Windows)

• table at beginning of filesystem, one entry per block

  • block# is array index
  • contents of table entry: pointer to table entry (index/block#) of next block in file
  • linked list containing only "next" pointer, but next == array index == block#

• pros & cons

  • pros:
    • no overhead in data blocks
    • faster random access (than linked blocks)
  • cons:
    • multiple disk seeks to read links
      • unless table is cached
      • thumb drive: no seeking

• FAT16

  • 16 bits (2 bytes): 64k address range
  • 32k block size
  • 4 blocks (128 kB) for table
  • maximum disk size: 64k * 32k = 2 GB

• FAT32

  • 28 bits of 32: 256M address range
  • 32k block size
  • maximum disk size 2TB (overhead)

Linux Filesystem

ext2

| block size | max file size | max filesystem |
|------------|---------------|----------------|
| 1 kB       | 16 GB         | 4 TB           |
| 2 Kb       | 256 GB        | 8 TB           |
| 4 Kb       | 2 TB          | 16 TB          |
| 8 Kb       | 2 TB          | 32 TB          |

• many (older) userland programs cannot handle files larger that 2 GB because they use 32-bit file positions

• ext4

  • up to 1 exabyte (2⁶⁰, approx. 10¹⁸)

Indexed File Allocation

• index block: pointers to data block

TODO: diagram

• index block(s): per file

  • unlike FAT

• index block size

  • too small: cannot support larger files
  • too large: waste disk disk space

• solution: multi-level indexed allocation

Unix Inode Indexing

• file metadata

  • mode (permission bits)
  • owner/group id
  • timestamps (atime, ctime, mtime)
  • size, block count
  • block pointers
    • 12 direct block pointers
    • 1 pointer to index block (single indirection)
    • 1 pointer to double indirection
    • 1 pointer to triple indirection

• index structure fast for small files, but supports large files

• Calculations:

  • 32-byte block addresses with 4 KB block size
    • 128 block pointers per block

TODO: diagram

Journaling Filesystem

• journal: transaction log
  • if filesystem crashes, recover incomplete operations from journal

Free Space Management

• bit vector
  • pro:
    • easy to find contiguous space
  • con:
    • expensive use of memory

• linked list

  • pro:
    • space efficient
      • use free space to hold free space list
  • con:
    • hard to find contiguous memory (fragmentation)

• essentially: a memory management problem

RAID

• RAID: Redundant Array of Inexpensive Disks

• various schema:

  • RAID 0
    • even split into "stripes"
    • no parity info: no fault tolerance
      • failure of one drive loses all the data
    • benefit: speed
    • large "logical" volumes
  • RAID 1
    • mirrors
    • no parity info
    • duplicate writes: slower
    • faster reading: duplicate "channels"
  • RAID 2/3
    • not used
  • RAID 4
    • block-level striping
    • dedicated parity disk (Reed Solomon)
    • writing bottleneck
  • RAID 5
    • block-level striping
    • distributed parity
    • tolerates failure of single drive
      • lost data can be reconstituted from parity info
    • faster read performance (than raid4)
  • RAID 6
    • RAID 5 + additional parity block
    • tolerates 2 concurrent drive failures
    • no performance penalty for read
    • slower writes due to additional parity block

Error Correction/Detection

• Reed Solomon Algorithm

  • used in practice for error recovery
  • but we'll look at older, simpler schemes

• parity: even/odd

  • detect single-bit error
  • but you don't know which bit is wrong
    • on communication channels, resend message
  • 2-bit error will not be detected

• Hamming Code (7,4): use 7 bits to encode 4-bit "message"

  • 3 check bits: parity of overlapping subsets
  • can correct any single-bit error
  • can detect any 2-bit error

• method

  • number the bits from 1 (rare case where numbering from 1 works better than from 0)
  • place a parity bit where bit# is a power of 2
  • each parity bit is responsible for bits that have 1-bit in corresponding bit of its index

example (7,4):

| bit 7 | bit 6 | bit 5 | bit 4 | bit 3 | bit 2 | bit 1 |
|------ |-------|-------|-------|-------|-------|-------|
| 111   | 110   | 101   | 100   | 011   | 010   | 001   |
|------ |-------|-------|-------|-------|-------|-------|
| a     | b     | c     | p3    | d     | p2    | p1    |

• data bits checked by parity bits: unique data-bit error will be caught by at least 2 parity-bit errors and for each data bit, the parity-bit combination is unique

  • a: p1, p2, p3
  • b: p2, p3
  • c: p1, p3
  • d: p1, p2

• parity bits check data bits: unique parity-bit error means it cannot be a data-bit error because each data bit is covered by at least 2 parity bits

  • p1: a, c, d
  • p2: a, b, d
  • p3: a, b, c

• conclusion:

  • single-bit error in data bits will cause unique parity error from which we can determine the error bit
  • single bit parity error will be detected because other parity bits will be correct
  • two-bit error will be detected, but we won't know which bits are affected