For the final project, you will implement a virtual file system to work with your minithreads package. We have provided a block-based disk interface in disk.h, which simulates a disk by translating block reads and writes to accesses to a single file.
You should implement a hierarchical, Unix-like file system on top of the disk emulator that supports the following operations:
minifile_t minifile_creat(char *filename);
minifile_t minifile_open(char *filename, char *mode);
int minifile_read(minifile_t file, char *data, int maxlen);
int minifile_write(minifile_t file, char *data, int len);
int minifile_close(minifile_t file);
int minifile_unlink(char *filename);
int minifile_mkdir(char *dirname);
int minifile_rmdir(char *dirname);
int minifile_stat(char *path);
int minifile_cd(char *path);
char **minifile_ls(char *path);
To get an idea of what these functions should do, look them up (omitting the "minifile_" prefix) via man pages on a Unix system. The only exception is that our minifile_open takes arguments like the fopen call, instead of open. Don't worry about reporting detailed error codes when something goes wrong, returning -1 from the function (or some other appropriate error value) is good enough. Your file system should support variable-sized files via a Unix-like inode mechanism, and reuse of blocks from unlinked (deleted) files. It is vital that your file system has concurrency control, so that it can cope with simultaneous accesses by multiple threads.
If you stick to the interface above, you should be able to compile and run the shell program included with this version of the code. You should then be able to create files and directories from the command line.
The disk simulator is relatively straightforward. To initialize the disk, use the disk_initialize() function. You can specify the disk initialization behavior by changing the following global variables: use_existing_disk, disk_name, disk_flags and disk_size. To create a disk, set use_existing_disk to 0, and specify the disk_name, disk_size and disk_flags. To start an existing disk, set use_existing_disk to 1 and specify the disk_name. Upon starting up the disk, disk_size and disk_flags will be set appropriately.
Since you are not asked to support mount points, there should be only one file-system in your implementation. This unique file-system should reside on a virtual disk that uses the file MINIFILESYSTEM in the current directory. Make sure you write a C program, called mkfs.exe, that creates an empty file-system in this virtual disk. This initial file-system created by mkfs.exe should contain only one directory (the root directory) with no entries.
Before sending requests to the disk, the disk must already have been initialized using disk_initialize(), and the interrupt handler installed. Once these steps have been completed, disk requests can be issued through the disk_send_request() function. The format of the requests is shown in disk.h. When requests complete, the disk controller signals the completion by raising an interrupt. As with previous assignments, you need to write an interrupt handler that will handle these interrupts appropriately.
Recall from the course discussion that the disk controller may reorder your requests in any arbitrary order. In fact, an efficient controller will reorder requests quite aggressively. Consequently, if you have a series of blocks that need to be written with a well-defined order (e.g. block A before B before C), then you must, in your file-system code, make sure that you do not issue request B before the request for A has been completed.
The disk driver code (disk.[ch]) sends a disk interrupt after a read/write request is processed. The disk interrupt is processed by the disk handler that receives an argument of disk_interrupt_arg_t type. The disk handler is installed using the install_disk_handler function.
Note that not all of the functions are file operations. For instance, the concept of the "current working directory" is not a global abstraction that applies to the filesystem, but a piece of local state kept with each process (minithread). Similarly, minifile_pwd() returns the path to the current working directory associated with the calling thread. Your file system should NOT have any such state as global variables shared across independent minithreads.
Applications may access the file system concurrently. For this project, you will implement UNIX semantics for concurrent accesses. You will allow multiple applications to open the same file for writing concurrently and to issue concurrent write requests to the same file. You will invoke the end-to-end argument and say that multiple applications performing concurrent accesses to the disk need to coordinate among themselves if they want to retain application-level guarantees of atomicity, and that the FS is not the appropriate subsystem in which to enforce synchronization constraints. Therefore, your FS can leave aside any guarantee of integrity of file contents when the file is accessed concurrently. Specifically, if thread A writes blocks 0 with data A0 and block 1 with data A1, and thread B writes the same blocks with data B0 and B1, the file may end up with "A0, A1", or "A0, B1" or "B0, A1" or "B0, B1", all depending on who performed the last write.
This approach is quite simple to implement. But you must ensure that your code guarantees the integrity of the filesystem. For instance, concurrent accesses should not cause your FS to generate orphaned data blocks (as would happen when a naive write() implementation that overwrites an existing inode is used concurrently by multiple threads).
Similarly, your FS must define a reasonable set of semantics when multiple applications concurrently access the file and delete or rename it at the same time. For example, suppose a thread is writing to a file when another thread unlinks that file. Once again, you will follow UNIX semantics in your implementation. The file is made unreachable (i.e. it is taken out of the directory hierarchy) at the time the unlink is issued. However, the inode and the file's data blocks are not placed back on the free list until all applications that have an open file descriptor to that file close their descriptors. This permits applications to continue working on a file which has been deleted, and permits concurrent applications to make forward progress. It can be implemented with a simple reference count of open handles to each file. Make sure that the last person who closes the deleted file places the (now-deleted) blocks back on the free list. Any changes made to the file are lost if the file got deleted while it was being modified.
Use the first block from the disk as the superblock followed by the disk blocks that contain the inodes (about 10 percent of the disk) and the rest for data blocks.
The superblock contains global information about the file system. You can store in the superblock things like the first free inode (if the free inodes are organized in a linked list) the first free data block (more about this latter), the pointer to the root inode (the entry point in the file-system), the total number of inodes and blocks, the overall size of the file-system and any other thing you consider useful. Also in the first 4 bytes put a predefined number (called magic number) that helps you determine if you have a legitimate file system on a disk or not.
Inodes should contain information about the file or directory they represent. Any relevant information about the file except the name should be kept in it's inode. This might include: type (file or directory), size, NEXT_FREE_INODE (to maintain the list of free inodes), etc. Also the inode contains information about which of the data blocks are used by the file. You have to be able to address 11 direct blocks and one indirect block (that contains pointers to other disk blocks). You can find more information about direct and indirect blocks in your textbook.
You do not have to pack as many inodes as will fit into a data block. For simplicity and ease of debugging, you may want to initially place each inode into a separate block. This wastes space, but once you have the basic system working, you can modify your filesystem to pack multiple inodes into a single disk block.
Directories are slightly special files that contain the mapping between file or directory names and inode numbers (or pointers). The only differences between a directory and a file are: directories cannot be deleted with unlink and can be deleted only if they are empty, they have a fixed format (for example you can use an ASCII representation with the inode number and the file on the same line and separated by a tab or a binary representation; just make sure the file name can be at least 256 characters) and they have the type set to DIRECTORY. You can perform linear search to find the inode number that corresponds to a particular file.
In addition, you should not limit the size of a directory to 1 data block, as this would unnecessarily restrict the number of entries that can be accomodated. Your directories should be able to hold any number of entries.
You will also need to submit an mkfs utility. The purpose of mkfs is to create a new disk and to format it for use with your file system. This utility is crucial in your submission as we will not be able to test any file operations without first being able to create and format a disk. Your mkfs utility should take a single argument from the command line that specifies the size (in number of disk blocks) of the new disk to create. The disk name to create this under is specified in the global variable disk_name.
While not required for submission, it is highly recommended that you also develop a fsck utility. The purpose of fsck is to check the integrity of your file system. A good fsck utility should check the references on all inodes and directory data blocks, as well as verify that there are no orphaned blocks or inodes. It is a good idea to develop the fsck utility first so that you are clear of your own disk's expected layout before you begin writing the minifile API. In addition, the fsck utility does not require substantial synchronization and can be written fairly quickly.
Make sure to test your code extensively. Simple sequential tasks, such as creating files, creating directories, removing directories, etc. should be easy. But you should also test your code with concurrent accesses as well as failure cases, e.g. five threads are concurrently writing to the disk when a system crash occurs (someone presses control-c). The file system should not be left in an inconsistent state. You can set the failure rates to non-zero values to have the disk controller experience such errors occasionally, just like a real disk.
Make a backup copy of your code from the previous project, download the code from CMS, and merge.
We've included a simple command shell in shell.c, which you can link against your minifile implementation. It should enable you to test your code from the command line.
Use CMS to submit your project. You should include a README file with your name, your partner's name, and anything you think we should know. This includes documenting any broken functionality.
Make sure to include all code.
If you need help with any part of the assignment, come for office hours. As usual, we are always here to help.