CSS 430
Program 1: System Calls and Shell
Due date: See the syllabus
This series of programming assignments is a step-by-step implementation of an OS simulator in Java. As you might imagine, significant effort was expended in the comprehensive design of the individual assignments and the way they are linked together. The goal is to enhance your understanding of key concepts by giving you an opportunity to learn by doing, rather than being limited to learning by reading or listening (though it is hoped these will also enhance your understanding). It would be very difficult to significantly change the specifications of these assignments every quarter the course is offered. Thus, solutions developed by former CSS430 students will likely be similar to solutions you will develop for these assignments. However, learning by copying will not help enhance your understanding, and submitting someone else's work as your own would be a violation of the academic integrity expected of all members of the UWB community.
In cultivating a collaborative learning environment, it is expected that you will consult various resources as you seek to understand each assignment, and as you iteratively develop and test your solutions to each one. However, all the solutions - programs, reports and other materials - that you submit for this or any other class you take at UWB must be your own. The instructor expects that students will appropriately conduct themselves throughout the course. However, there have been instances in the past where students have not maintained academic integrity in this course, and they have been reported to the Vice Chancellor's office in accordance with the UWB Student Conduct Code. All solutions submitted by former css430 students in all sections have been archived, and can be easily accessed by software plagiarism detection applications.
The three goals of this assignment are for you to
This section explains the behavior and the language syntax of the Unix (Linux) shell.
The shell is a command interpreter that provides Unix users with an interface to the underlying operating system. It interprets user commands and executes them as independent processes. The shell also allows users to code an interpretive script using simple programming constructs such as if, while and for. With shell scripting, users can coordinate a collection of their jobs.
The behavior of the shell simply repeats:
Examples of prompts include a host name followed by a % symbol
uw1-320-00%or a dollar sign
$
There are a variety of different shells that can be run on a Linux system, including the Bourne shell (sh), C-shell (csh), Korn shell (ksh) and Bourne again shell (bash). You can check which shell is running on a Linux system by issuing the command
echo $SHELLIf you run this in the Linux Lab machine, you'll see that it will return
/bin/bash[i.e., the bash shell is the default shell for the Linux Lab machines, and the one we will use as the reference shell for this course]
To execute each command, the shell follows these steps:
For example, assume that your current working directory (CWD) includes the a.out executable file and you have typed the following line input from your keyboard:
uw1-320-00% ./a.out a b c[Note: using "./" before "a.out" in case "." (CWD) is not in the PATH list ... which it should not be, due to potential security problems]
After reading this command line, the shell process duplicates itself, and this duplicated process executes a.out with a, b and c as its arguments.
The shell has several builtin commands in addition to the commands that are stored externally as executable programs. Many of the built-ins have some effect on the state of the shell. For example, the cd command changes the current working directory of the shell:
uw1-320-00% cd public_htmlchanges the shell's current working directory to public_html (which may be reflected by "public_html" now being included in the shell prompt).
The shell can receive two or more commands at a time from a keyboard input. The symbols ';' and '&' are used as delimiters to separate each individual command. The ';' symbol specifies sequential processing of commands; the '&' symbol specifies parallel processing of commands. More specifically:
uw1-320-00% who & ls & datethe shell executes who, ls, and date concurrently. Note that if the last command does not have an delimiter, the shell assumes that it is implicitly delimited by ';'. In the above example, the shell waits for the completion of date before displaying the system prompt again. However, if '&' is appended to the end of the line
uw1-320-00% who & ls & date &the shell spawns processes for all three commands and immediately displays the system prompt (try both variations, and see for yourself).
Now that we have learned a little more about the behavior of the shell, we can specify its behavior more precisely as follows:
One of the shell's interesting features is I/O redirection. Unless specified otherwise, input to a command run from by shell is read from the keyboard, i.e., the default standard input source is the keyboard. Likewise, the standard output destination for a command run by the shell is the monitor, unless otherwise specified. The '<' symbol on the command line specifies a file from which input to a command will be read, and the '>' symbol specifies a file to which output will be written. Each command can have zero, one or both redirection specifications. For example, when the following command is input to the shell
uw1-320-00% a.out < file1 > file2the shell spawns a process to execute the program a.out, but the input to a.out is read from file1 rather than the keyboard, and the output from a.out is written to file2 rather than the monitor.
Another useful feature is pipelines. Pipelines connect the output of one process to the input of another process using the '|' (vertical bar) as a delimiter.
uw1-320-00% command1 | command2 | command3connects command1's standard output to command2's standard input, and also connects command2's standard output to command3's standard input. Any given command line can have 0 or more pipes.
As a specific example,
uw1-320-00% who | wc -lfirst executes the who command that prints out a list of current users. This output is not displayed directly on the monitor, but is rather passed - or piped - to wc's standard input. wc (the wordcount program) is executed with its -l option. It reads and counts each line (one per current user) and writes a single line containing that count on standard output.
Through a series of assignments, you will implement and/or enhance some portions of ThreadOS. ThreadOS loads Java programs that have been derived from the Java Thread class, manages them as user processes, and provides them with some basic operating system services. Those services include spawning threads, terminating threads, disk operations, and even standard input/output. ThreadOS receives all service requests as a form of interrupt from each user thread, handles them, and returns a status value to the interrupting thread.
Component | Java/Class | Description |
Boot | Boot.java | invokes a BOOT system call to have Kernel initialize its internal data, power on Disk, start the Scheduler thread, and finally spawn the Loader thread |
Kernel | Kernel.java | receives an interrupt, services it if possible, otherwise forwards the request to Scheduler or Disk, and returns a completion status |
Disk | Disk.java | simulates a slow disk device composed of 1000 blocks, each containing 512 bytes. The disk is divided into 10 tracks, each of which thus includes 100 blocks. The disk has three commands: read, write, and sync, as detailed in assignments 3, 4, and 5 |
Scheduler | Scheduler.java, TCB.java | receives a Thread object that Kernel instantiated upon receiving an EXEC system call, allocates a new Thread Control Block (TCB) to this thread, enqueues the TCB into its ready queue, and schedules its execution in a round robin fashion |
SysLib | SysLib.java | provides user threads with a collection of system calls, which it converts into corresponding interrupts passed to Kernel |
To start ThreadOS, simply type:
uw1-320-00% java Boot ThreadOS ver 1.0: Type ? for help threadOS: a new thread (thread=Thread[Thread-3,2,main] tid=0 pid=-1) -->Boot initializes Kernel data, powers on Disk and starts Scheduler. Note: the first time you run ThreadOS, it will take ~30 seconds to initialize the simulated DISK; the message "default format( 64 )" will appear at the start of this initialization, and a message "Superblock synchronized" will appear when it is completed. The sample output below shows this sequence (in which the user immediately terminates ThreadOS after initialization) nestled between two calls to the Linux date command.
uw1-320-00% date; java Boot; date Sat Oct 8 11:14:41 PDT 2011 threadOS ver 1.0: threadOS: DISK created default format( 64 ) Superblock synchronized Type ? for help threadOS: a new thread (thread=Thread[Thread-3,2,main] tid=0 pid=-1) -->q q Superblock synchronized Sat Oct 8 11:15:09 PDT 2011 uw1-320-00%After initialization, ThreadOS launches Loader, which then carries out one of the following commands:
? | prints out its usage |
l user_program | starts user_program as an independent user thread and waits for its termination |
q | synchronizes disk data and terminates ThreadOS |
Note that Loader is not a shell. It simply launches and waits for the completion of a user program (which, in the case of Shell.class, may behave as a shell). From ThreadOS' point of view, there is no distinction between Loader and the other user programs.
A user program run by ThreadOS must be an instance of a Java thread. Java threads are execution entities concurrently running in a Java application. They maintain their own stacks and program counter but share static variables in their application. At least one thread, (i.e., the main thread) is automatically instantiated when an application starts a main function. Threads other than the main thread can be dynamically created in the similar way to instantiate a new class object using new. Once their start method is called, they keep executing their own run method independently from the calling function such as main. Java threads can be defined as a subclass of the Thread class.
The following Java thread repeatedly writes copies of word (given in args[0]) to standard output, waiting for loop milliseconds (given in args[1]) between printing each copy [note: in an earlier version, loop was used to specify the number of dummy iterations of a busy wait for loop between printing each copy; code from earlier version is shown in comments].
public class PingPong extends Thread { // NB: "extends Thread" private String word; private int loop; public PingPong( String[] args ) { word = args[0]; loop = Integer.parseInt( args[1] ); } public void run( ) { for ( int j = 0; j < 100; j++ ) { // substituting SysLib.cout() for System.out.println() // System.out.print( word + " " ); SysLib.cout( word + " " ); // substituting SysLib.sleep() for busy wait for loop // for ( int i = 0; i < loop; i++ ) ; SysLib.sleep( loop ); } SysLib.cout( "\n" ); SysLib.exit( ); } }If you write the following main function:
public class TestPingPong { public static void main( String[] args ) { String[] testargs = new String[2]; testargs[0] = "ping"; testargs[1] = "10"; new PingPong( testargs ).start( ); testargs[0] = "PING"; testargs[1] = "90"; new PingPong( testargs ).start( ); } }it will instantiate two PingPong threads, one printing out "ping" every 10 milliseconds and the other printing out "PING" every 90 milliseconds.
Just as the Java Virtual Machine searches for and runs a main method defined in whatever class file it is given as input, the ThreadOS will search for and run a run method defined in whatever class file is loaded via the Loader (l command). Note that any Java class defined as a subclass (extension) of Thread must define a run method.
ThreadOS Loader takes care of the thread-instantiating task typically invoked by the main function, automatically instantiating a new thread object and invoking its run method.
Once you invoke ThreadOS, the Loader waits for a l command, for example
l PingPong ping 100This command instructs Loader to load the PingPong class (in the byte-compiled PingPong.class file) into memory, instantiate its thread object (create an instance of PingPong, which is an extension of Thread), pass a String array including ping and 100 as arguments to this thread object, invoke its run method and wait for its termination. Note that general Java threads can receive any type of and any number of arguments, however ThreadOS only works with Java classes that have a String array (possibly with zero elements) as their argument.
Java itself provides various classes and methods that invoke real OS system calls such as System.out.println and sleep. Since ThreadOS is an operating systems simulator, user programs running on ThreadOS are prohibited from using any real OS system calls. Prohibited classes include but are not limited to the following packages:
Instead, user programs are provided with SysLib, a system library of ThreadOS-unique of system calls including standard I/O, disk access, and thread control. For example,
System.out.print( word + " " );should not be used; instead, the equivalent ThreadOS SysLib method should be substituted:
SysLib.cout( word + " " );
While Java threads can be terminated upon a simple return from their run method, ThreadOS needs an explicit system call to terminate the current user thread.
SysLib.exit( );This is because thread termination is a part of thread control, and is thus one of the ThreadOS services.
The following Java class uses SysLib methods to print "Hello world" to standard output and then terminate:
public class HelloWorld extends Thread { public HelloWorld() { } public void run() { SysLib.cout( "Hello, world\n" ); // using SysLib.cout vs System.out.println SysLib.exit(); // don't forget to explicitly exit return; } }
As a slightly more interesting example, we can revisit and revise the PingPong class definition to work with SysLib. Since the original definition of the class includes an infinitive loop (while (true)), we will revise it so that this sample code safely terminates the invoked thread and resumes Loader.
public class PingPong extends Thread { private String word; private int loop; public PingPong( String[] args ) { word = args[0]; loop = Integer.parseInt( args[1] ); } public void run( ) { for ( int j = 0; j < 100; j++ ) { // for vs. while SysLib.cout( word + " " ); for (int i = 0; i < loop; i++ ) ; // busy wait } SysLib.cout( "\n" ); SysLib.exit( ); } }[The source code for PingPong.java can be found on the uw1-320-lab network under /usr/apps/CSS430/ThreadOS]
ThreadOS Kernel receives requests from each user thread as interrupts to it. Such an interrupt is performed by calling:
Kernel.interrupt( int interruptRequestVector, int trapNumber, int parameter, Object args );where:
Since this interrupt method is fairly low-level, ThreadOS provides a system library, called SysLib that includes several important system call functions as shown below. (Unless otherwise mentioned, each of these functions returns 0 on success or -1 on error.)
In addition to those system calls, the system library includes several utility functions. One of them is:
[The source code for SysLib.java can be found on the uw1-320-lab network under /usr/apps/CSS430/ThreadOS]
Code a C++ program, named processes.cpp that receives one argument, (i.e., argv[1]) upon its invocation and searches how many processes whose name is given in argv[1] are running on the system where your program has been invoked. Specifically, your program should demonstrate the same behavior as:
ps -A | grep argv[1] | wc -lwhen a string is passed in as an argument. For example, if your executable is named processes, then the following two commands should generate the exact same output:
ps -A | grep sh | wc -l ./processes sh
Implement the program using only the following system calls, i.e., do not use other system calls not listed below [links are to man pages for the system calls on die.net]:
In addition to the links included for each command above, you can also use the man command from the shell prompt line. Wolf Holzman has posted a nice overview of pipe, fork, exec and related topics. Note: Use only the Linux system calls listed above for this assignment. Do not use the system system call.
You may find the following series of simple programs, derived from the example on the Linux pipe(2) man page, helpful:
The following series of sample programs illustrates small variations in using fork, dup2 and execlp to emulate the behavior of
ps -A | tr a-z A-ZA short sleep period is used to help highlight the differences when the parent vs. the child executes the command that reads from vs. writes to the pipe. Some of these examples are derived from information contained in CPS590 - Del's Pipes Lecture Notes (via Darren Korman).
Imitate how the shell performs ps -A | grep argv[1] | wc -l. In other words, your parent process spawns a child that spawns a grand-child that spawns a great-grand-child. Each process should execute a different command as follows (note the order):
Process | Command | Stdin | Stdout |
Parent | wait for Child | no change | no change |
Child | wc -l | redirected from a Grand-child's stdout | no change |
Grand-child | grep argv[1] | redirected from a Great-grand-child's stdout | redirected to a Child's stdin |
Great-grand-child | ps -A | no change | redirected to a Grand-child's stdin |
uw1-320-00% java Boot ThreadOS ver 1.0: Type ? for help -->l MyShell l MyShell threadOS: a new thread (thread=Thread[Thread-6,2,main] tid=1 pid=0) shell[1]% TestProg1 & TestProg2 & ... A concurrent execution of TestProg1 and TestProg2 ... shell[2]% TestProg1 ; TestProg2 ... A sequential execution of TestProg1 and TestProg2 ... shell[3]% exit -->q uw1-320-00%
Once MyShell is invoked, it should print out a command prompt that contains the word "shell" and then the command number (starting at 1) in square brackets, e.g.,
shell[1]%
When a user types multiple commands, each delimited by '&' or ';', MyShell should execute each of them as an independent child thread with a SysLib.exec() system call. Note that the symbol '&' denotes concurrent execution, while the symbol ';' denotes sequential execution. Thus, when encountering a delimiter ';', MyShell needs to call SysLib.join() system call(s) in order to wait for this child thread to be terminated. Since SysLib.join() may return the ID of any child thread that has been previously terminated, you have to repeat calling SysLib.join() until it returns the exact ID of the child thread that you are waiting for.
You do not need to implement standard I/O redirection or pipes. You do not need to provide shell variables and programming constructs, either. The only the required functionality of MyShell is handling an arbitrary number of commands in a line. You may assume that commands, arguments, and even delimiters are separated by arbitrary amounts of spaces or tabs.
To test your MyShell, use the PingPong class that is defined in the same directory as ThreadOS. Your test should be
shell[1]% PingPong abc 100 & PingPong xyz 100 & PingPong 123 100 & shell[2]% PingPong abc 100 ; PingPong xyz 100 ; PingPong 123 100 ;There is a byte-compiled version of a ThreadOS shell (Shell.class) included in the ThreadOS directory that you can use to see what kind of output should be generated by MyShell.
ThreadOS can be found in the following directory on the uw1-320-lab network:
/usr/apps/CSS430/ThreadOS/
To use ThreadOS and develop and test your program,
mkdir ~/ThreadOS
cp /usr/apps/CSS430/ThreadOS/*.class ~/ThreadOS
[joemcc@uw1-320-17 ThreadOS]$ java Boot threadOS ver 1.0: Type ? for help threadOS: a new thread (thread=Thread[Thread-3,2,main] tid=0 pid=-1) -->l Shell l Shell threadOS: a new thread (thread=Thread[Thread-5,2,main] tid=1 pid=0) shell[1]% HelloWorld HelloWorld threadOS: a new thread (thread=Thread[Thread-7,2,main] tid=2 pid=1) Hello, world shell[2]% PingPong abc 100 & PingPong xyz 100 & PingPong 123 100 & PingPong threadOS: a new thread (thread=Thread[Thread-9,2,main] tid=3 pid=1) PingPong threadOS: a new thread (thread=Thread[Thread-11,2,main] tid=4 pid=1) PingPong threadOS: a new thread (thread=Thread[Thread-13,2,main] tid=5 pid=1) shell[3]% abc abc abc abc abc abc abc abc abc abc xyz abc xyz abc xyz abc xyz abc xyz abc xyz abc xyz abc xyz abc xyz abc xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz abc 123 xyz 123 xyz 123 xyz 123 xyz 123 xyz 123 xyz 123 xyz 123 xyz 123 xyz 123 xyz 123 123 123 123 123 123 123 123 123 123 shell[3]% PingPong abc 100 ; PingPong xyz 100 ; PingPong 123 100 PingPong threadOS: a new thread (thread=Thread[Thread-15,2,main] tid=6 pid=1) abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc PingPong threadOS: a new thread (thread=Thread[Thread-17,2,main] tid=7 pid=1) xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz xyz PingPong threadOS: a new thread (thread=Thread[Thread-19,2,main] tid=8 pid=1) 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 shell[4]% exit exit -->q q Superblock synchronized [joemcc@uw1-320-17 ThreadOS]$
Do not try to compile the ThreadOS source code. Some Java source files cannot be accessed (you will be writing your own Java code to substitute for the byte-compiled classes in future assignments.)
Hints:
ps -A | grep sh | wc -l processes sh ps -A | grep mingetty | wc -l processes mingetty ps -A | grep gnome | wc -l processes gnomeTo generage the output file, copy the shell script testprog1.sh into the directory containing your executable. If you have chosen to use a name other than processes for your executable, you will need to edit the PROG1EXE variable in that file. Run the script, redirecting its output to a file:
./testprog1.sh > processes.outHere is an example of what one might find in processes.out:
Running ps -A | grep sh | wc -l 19 Running ./processes sh 19 Done Running ps -A | grep mingetty | wc -l 6 Running ./processes mingetty 6 Done Running ps -A | grep gnome | wc -l 9 Running ./processes gnome 9 DoneYour mileage - or counts - may vary.
gradeguide1.txt is the grade guide that will be for used for assignment 1.
CSS430 Operating Systems Assignments Home Page is the general guide to ensure that your assignments adhere to proper procedures. This may be modified in the near future, but will serve as a good approximation.
A page with frequently asked questions (and answers) represents the collective wisdom of previous offerings of CSS430. This, too, may be updated in the near future, but should be consulted before posting questions on the General Discussion forum or emailing the professor.