firstworks Programming with Rudiments using the file class

Using the file class

- Introduction
- Creating a File
- Removing a File
- Opening and Closing Files
- File Properties
- Read Access
- Write Access
- Random Access
- File Locks
- Truncating a File
- Buffered File Access
- Optimizing File Access
- Temporary Files
- Links
- Fifos
- Named Pipes
- Convenience Methods

### Introduction

The file class provides methods for creating and accessing files.

The filedescriptor class provides methods for generic input and output to file descriptors. The file class inherits from filedescriptor and adds methods that are specific to files. However, when using the file class, you will likely make heavy use of filedescriptor methods such as read() and write().

### Creating a File

The following example demonstrates the different ways to create a file.

Note that when creating a file, the file permissions must be supplied. Rudiments supports unix-style file permissions, even on Windows. See the permissions class for more information.

> #include <rudiments/file.h>
> #include <rudiments/permissions.h>
> #include <rudiments/stdio.h>
> int main(int argc, const char **argv) {
> file f;
> // Create a file using the create() method, with read-write
> // permissions for everyone.
> if (f.create("testfile1",permissions::evalPermString("rw-rw-rw-"))) {
> stdoutput.write("testfile1 created\n");
> } else {
> stdoutput.write("failed to create testfile1\n");
> }
> // Create a file using the open() method and O_CREAT flag,
> // with read-write permissions for everyone.
> if (f.open("testfile2",O_CREAT,
> permissions::evalPermString("rw-rw-rw-"))) {
> stdoutput.write("testfile2 created\n");
> } else {
> stdoutput.write("failed to create testfile2\n");
> }
> // An attempt to create a file that already exists will just open
> // the file.
> if (f.create("testfile1",permissions::evalPermString("rw-rw-rw-"))) {
> stdoutput.write("testfile1 opened\n");
> } else {
> stdoutput.write("failed to open testfile1\n");
> }
> // If O_EXCL and O_CREAT are used together, then an attempt to create a
> // file that already exists will fail.
> if (f.open("testfile2",O_CREAT|O_EXCL,
> permissions::evalPermString("rw-rw-rw-"))) {
> stdoutput.write("testfile2 opened\n");
> } else {
> stdoutput.write("failed to open testfile2 "
> "(this is the correct behavior)\n");
> }
> }

### Removing a File

The file class provides a static method for removing a file.

> #include <rudiments/file.h>
> #include <rudiments/stdio.h>
> int main(int argc, const char **argv) {
> // remove testfile1
> if (file::remove("testfile1")) {
> stdoutput.write("testfile1 removed\n");
> } else {
> stdoutput.write("failed to remove testfile1\n");
> }
> // remove testfile2
> if (file::remove("testfile2")) {
> stdoutput.write("testfile2 removed\n");
> } else {
> stdoutput.write("failed to remove testfile2\n");
> }
> }

### Opening a File

To open a file, call one of the open() methods.

The open() methods take the filename to open and a set of flags.

Flags may include one or more of the following flags, or'ed together.

- O_RDONLY - Open the file in read-only mode.
- O_WRONLY - Open the file in write-only mode.
- O_RDWR - Open the file in write-only mode.
- O_APPEND - Set the position to the end of the file.
- O_TRUNC - Truncate the file. Requires O_WRONLY or O_RDWR.
- O_CREAT - Creates the file if it doesn't exist. See O_EXCL.
- O_EXCL - Requires O_CREAT. Causes O_CREAT to fail if the file already exists. Without O_EXCL, O_CREAT will succeed, and just open the file, if the file already exists.

Many platforms support additional, platform-specific flags.

There are two varieties of the open() method. They both take filename and flags. The second also takes permissions. If the flags include O_CREAT then the permissions are used when creating the file. If the flags don't include O_CREAT then the permissions are ignored.

Files are closed when the instance of the file class is deleted, or when the same instance is used to open another file, but they can also be closed manually using the close() method.

The following example demonstrates various ways to open and close a file.

> #include <rudiments/file.h>
> #include <rudiments/permissions.h>
> #include <rudiments/stdio.h>
> int main(int argc, const char **argv) {
> file f;
> // Create a file, or open it for write if it already exists.
> if (f.open("testfile1",O_WRONLY|O_CREAT,
> permissions::evalPermString("rw-rw-rw-"))) {
> stdoutput.write("created/opened testfile1\n");
> } else {
> stdoutput.write("failed to create/open testfile1\n");
> }
> // Attempt to create a file, and fail if it already exists.
> // (the previously open file will be closed automatically)
> if (f.open("testfile1",O_WRONLY|O_CREAT|O_EXCL,
> permissions::evalPermString("rw-rw-rw-"))) {
> stdoutput.write("created/opened testfile1\n");
> } else {
> stdoutput.write("failed to create/open testfile1 "
> "(this is the correct behavior)\n");
> }
> // Open a file for read, starting at the beginning of the file.
> // (the previously open file will be closed automatically)
> if (f.open("testfile1",O_RDONLY)) {
> stdoutput.write("opened testfile1 for read\n");
> } else {
> stdoutput.write("failed to open testfile1 for read\n");
> }
> // Open a file for write, starting at the beginning of the file.
> // (the previously open file will be closed automatically)
> if (f.open("testfile1",O_WRONLY)) {
> stdoutput.write("opened testfile1 for write\n");
> } else {
> stdoutput.write("failed to open testfile1 for write\n");
> }
> // You can also close the file manually.
> f.close();
> // Open a file for read and write,
> // starting at the beginning of the file.
> if (f.open("testfile1",O_RDWR)) {
> stdoutput.write("opened testfile1 for read/write\n");
> } else {
> stdoutput.write("failed to open testfile1 for read/write\n");
> }
> // Close the file manually.
> f.close();
> // Open a file for write, starting at the end of the file.
> if (f.open("testfile1",O_WRONLY|O_APPEND)) {
> stdoutput.write("opened testfile1 for append\n");
> } else {
> stdoutput.write("failed to open testfile1 for append\n");
> }
> // Close the file manually.
> f.close();
> // Open a file for write,
> // first removing the current contents of the file.
> if (f.open("testfile1",O_WRONLY|O_TRUNC)) {
> stdoutput.write("opened testfile1 for append\n");
> } else {
> stdoutput.write("failed to open testfile1 for append\n");
> }
> // when main() exits, f will be deleted and the file will be closed.
> }

### File Properties

The file class provides methods for getting various file properties including permissions, ownership, size, type, and access and modification times.

The following example opens a file and prints out various file properties. Note that other classes must be used to convert the codes, ids and raw times to human-readable form.

Note that this program depends on the existence of testfile1, so you'll have to touch it, or run one of the programs above to create it.

> #include <rudiments/file.h>
> #include <rudiments/permissions.h>
> #include <rudiments/userentry.h>
> #include <rudiments/groupentry.h>
> #include <rudiments/datetime.h>
> #include <rudiments/stdio.h>
> int main(int argc, const char **argv) {
> // open the file
> file f;
> if (f.open("testfile1",O_RDONLY)) {
> // print out various file properties...
> // permissions
> mode_t perms=f.getPermissions();
> char *permstring=permissions::evalPermOctal(perms);
> stdoutput.printf("Permissions: %s\n",permstring);
> delete[] permstring;
> // owner user/group
> uid_t user=f.getOwnerUserId();
> userentry ou;
> ou.initialize(user);
> stdoutput.printf("Owner User: %s\n",ou.getName());
> gid_t group=f.getOwnerGroupId();
> groupentry og;
> og.initialize(group);
> stdoutput.printf("Owner Group: %s\n",og.getName());
> // sizes
> stdoutput.printf("File Size: %lld\n",
> f.getSize());
> stdoutput.printf("Block Size: %lld\n",
> f.getBlockSize());
> stdoutput.printf("Block Count: %lld\n",
> f.getBlockCount());
> // file type
> stdoutput.printf("Is Socket: %s\n",
> (f.isSocket())?"yes":"no");
> stdoutput.printf("Is Symbolic Link: %s\n",
> (f.isSymbolicLink())?"yes":"no");
> stdoutput.printf("Is Regular File: %s\n",
> (f.isRegularFile())?"yes":"no");
> stdoutput.printf("Is Block Device: %s\n",
> (f.isBlockDevice())?"yes":"no");
> stdoutput.printf("Is Directory: %s\n",
> (f.isDirectory())?"yes":"no");
> stdoutput.printf("Is Character Device: %s\n",
> (f.isCharacterDevice())?"yes":"no");
> stdoutput.printf("Is Fifo: %s\n",
> (f.isFifo())?"yes":"no");
> // access/modification times
> time_t access=f.getLastAccessTime();
> datetime da;
> da.initialize(access);
> stdoutput.printf("Last Access: %s\n",da.getString());
> time_t mod=f.getLastModificationTime();
> datetime dm;
> dm.initialize(mod);
> stdoutput.printf("Last Modification: %s\n",dm.getString());
> } else {
> stdoutput.write("failed to open testfile1\n");
> }
> }

There are 3 other methods worth mentioning as well:

getCurrentProperties()
  By default, the file class fetches file properties when open() or create() are called. If a property is changed, then the old value will still be reflected when the method to get it is called, unless this method is called first.
dontGetCurrentPropertiesOnOpen()
  By default, the file class fetches file properties when open() or create() are called. If this method is called, then file properties will not be fetched unless getCurrentProperties() is called manually.
getCurrentPropertiesOnOpen()
  Returns the class to its default behavior of fetching file properties when open() or create() are called.

### Read Access

The filedescriptor class (which the file class inherits from) provides read() methods to read data from files and file descriptors.

Methods are provided for reading all primitive types directly. This example opens a file in read-only mode and reads primitive data types from it.

> #include <rudiments/file.h>
> int main(int argc, const char **argv) {
> // open the file
> file f;
> f.open("testfile",O_RDONLY);
> // read a bool
> bool b;
> f.read(&b);
> // read various characters
> char c;
> f.read(&c);
> unsigned char uc;
> f.read(&uc);
> // read various integers
> uint16_t ui16;
> f.read(&ui16);
> uint32_t ui32;
> f.read(&ui32);
> uint64_t ui64;
> f.read(&ui64);
> int16_t i16;
> f.read(&i16);
> int32_t i32;
> f.read(&i32);
> int64_t i64;
> f.read(&i64);
> // read various floats
> float fl;
> f.read(&fl);
> double db;
> f.read(&db);
> }

Methods are also provided for reading arbitrary data into buffers. This example approximates the unix "cat" or Windows "type" utility. It opens a file in read-only mode, reads it in chunks, and prints out each chunk.

> #include <rudiments/file.h>
> #include <rudiments/stdio.h>
> int main(int argc, const char **argv) {
> file f;
> char buffer[1024];
> // for each file specified on the command line...
> for (int32_t i=1; i<argc; i++) {
> // open the file
> if (!f.open(argv[i],O_RDONLY)) {
> continue;
> }
> // read chunks from the file and print each chunk..
> ssize_t bytesread=0;
> do {
> // attempt to read 1024 bytes into the buffer
> bytesread=f.read(buffer,1024);
> // bytesread will be the number of bytes that were
> // actually read, 0 at EOF, or a negative number
> // if an error occurred
> if (bytesread>0) {
> // print the buffer
> stdoutput.write(buffer,bytesread);
> }
> // exit if we read fewer than 1024 bytes
> } while (bytesread==1024);
> }
> }

A read() method is also provided that reads until it encounters a specified terminator. This example reads a file, one line at a time, and prints out each line.

> #include <rudiments/file.h>
> #include <rudiments/stdio.h>
> int main(int argc, const char **argv) {
> file f;
> // for each file specified on the command line...
> for (int32_t i=1; i<argc; i++) {
> // open the file
> if (!f.open(argv[i],O_RDONLY)) {
> continue;
> }
> // read lines from the file and print each line...
> ssize_t bytesread=0;
> do {
> // attempt to read a line
> char *line=NULL;
> bytesread=f.read(&line,"\n");
> // bytesread will be the number of bytes that were
> // actually read, 0 at EOF, or a negative number
> // if an error occurred
> if (bytesread>0) {
> // print the line
> stdoutput.write(line);
> }
> // clean up
> delete[] line;
> // exit on eof or error
> } while (bytesread>0);
> }
> }

### Write Access

The filedescriptor class (which the file class inherits from) provides write() methods to write data to files and file descriptors.

Methods are provided for writing primitive types as well as blocks of character or binary data. This example creates a file in write-only mode and writes various types of data to it.

> #include <rudiments/file.h>
> #include <rudiments/permissions.h>
> int main(int argc, const char **argv) {
> // open/create the file
> file f;
> f.open("testfile",O_WRONLY|O_CREAT,
> permissions::evalPermString("rw-rw-rw-"));
> // write a bool
> bool b=true;
> f.write(b);
> // write various characters
> char c='a';
> f.write(c);
> unsigned char uc='a';
> f.write(uc);
> // write various integers
> uint16_t ui16=16;
> f.write(ui16);
> uint32_t ui32=32;
> f.write(ui32);
> uint64_t ui64=64;
> f.write(ui64);
> int16_t i16=-16;
> f.write(i16);
> int32_t i32=-32;
> f.write(i32);
> int64_t i64=-64;
> f.write(i64);
> // write various floats
> float fl=1.234;
> f.write(fl);
> double db=1.234;
> f.write(db);
> // write some text
> const char *text="hello there";
> f.write(text);
> // write the first 5 bytes of the same text
> f.write(text,5);
> // write some binary data
> unsigned char binary[]={1,2,3,4,5,6,7,8,9,0};
> f.write(binary,sizeof(binary));
> // write arbitary binary data
> uint64_t data[]={12345,67890,12345,67890};
> f.write((void *)data,sizeof(data));
> }

### Random Access

It is possible to open a file for both reading and writing, write to it, jump around, overwrite parts, jump around some more, and read from it.

This example illustrates random file access.

> #include <rudiments/file.h>
> #include <rudiments/permissions.h>
> #include <rudiments/stdio.h>
> int main(int argc, const char **argv) {
> // open the file
> file f;
> f.open("testfile",O_RDWR|O_CREAT,
> permissions::evalPermString("rw-rw-rw-"));
> // write 4 fixed length records to the file,
> // each consiting of two 10-character fields
> f.write(" ");
> f.write(" ");
> f.write(" ");
> f.write(" ");
> // go to the first record
> f.setPositionRelativeToBeginning(0);
> // overwrite the first record
> f.write("goodbye friends ");
> // go to the last record
> f.setPositionRelativeToEnd(-20);
> f.write("hello there ");
> // go to the third record
> f.setPositionRelativeToBeginning(40);
> f.write("bye folks ");
> // go to the second record
> f.setPositionRelativeToCurrent(-40);
> f.write("hi guys ");
> // print the records in reverse order
> char record[20];
> for (off64_t i=1; i<=4; i++) {
> f.setPositionRelativeToEnd(-20*i);
> f.read(record,20);
> stdoutput.write(record,20);
> stdoutput.write('\n');
> }
> }

### File Locks

...

> int main(int argc, const char **argv) {
> // FIXME: example...
> }

### Truncating a File

If you want to erase the current contents of a file, there are two options. The first is to open the file using the O_TRUNC flag. But, if you want to truncate the file after you've already opened it, you can use the truncate() method.

> #include <rudiments/file.h>
> #include <rudiments/permissions.h>
> int main(int argc, const char **argv) {
> // open/create a file
> file f;
> f.open("testfile",O_WRONLY|O_CREAT,
> permissions::evalPermString("rw-rw-rw-"));
> // write some text to it
> f.write("this is some text\n");
> // truncate the file
> f.truncate();
> // write some different text to it
> f.write("this is some different text\n");
> }

### Buffered File Access

If your program does a lot of reads and writes, then you probably want to buffer them. There are two main reasons for this:

- Accessing storage is much slower than accessing memory.
- System calls are generally slower than library calls.

Both file systems and physical storage are typically organized in blocks. Each block is some number of bytes. A read or write of the entire block costs the same as reading or writing a single byte. If you use buffers that are the same size as the file system's block size, then reading or writing the entire buffer takes only marginally more time than reading or writing a single byte.

Also, even if the kernel or disk does some kind of buffering, each read or write still requires a system call. In effect, the program tells the kernel to do the read or write, hands control over to the kernel, waits for it to do it, and then waits for it to hand control back. The overhead of the switching involved usually makes the whole process slower than if the program read or wrote directly from a buffer under its own control.

Buffering always helps writes and usually helps reads. How much is platform-specific.

The filesystem class makes buffered I/O simple. There are only three methods involved:

setReadBufferSize()
  Sets the size of the buffer used when reading.
setWriteBufferSize()
  Sets the size of the buffer used when writing.
flushWriteBuffer()
  Makes sure that any data that is buffered but hasn't been written to storage, actually gets written to storage.

Reading is completely transparent. The read buffer is filled during the first read, and filled again when all bytes have been read from it.

Writing is nearly transparent. Bytes are written to the buffer. When the buffer is full, it is written to storage. However, when the program is done writing, it should call flushWriteBuffer() to make sure that any data still in the buffer is written to storage.

The example below illustrates buffered I/O.

> #include <rudiments/file.h>
> #include <rudiments/permissions.h>
> #include <rudiments/datetime.h>
> #include <rudiments/stdio.h>
> int main(int argc, const char **argv) {
> file f;
> uint32_t i;
> uint32_t j;
> char c;
> // open/create the file
> f.open("testfile",O_WRONLY|O_CREAT|O_TRUNC,
> permissions::evalPermString("rw-rw-rw-"));
> // write 1mb of characters to the file, unbuffered
> stdoutput.write("writing unbuffered...\n");
> for (i=0; i<1024*1024; i++) {
> f.write('a');
> }
> stdoutput.write("done\n");
> // truncate the file
> f.truncate();
> // write 1mb of characters to the file, buffered
> stdoutput.write("writing buffered...\n");
> f.setWriteBufferSize(4096);
> for (i=0; i<1024*1024; i++) {
> f.write('a');
> }
> f.flushWriteBuffer(-1,-1);
> stdoutput.write("done\n");
> // read 1mb of characters from the file, unbuffered (10 times)
> stdoutput.write("reading unbuffered...\n");
> for (i=0; i<10; i++) {
> f.setPositionRelativeToBeginning(0);
> for (j=0; j<1024*1024; j++) {
> f.read(&c);
> }
> }
> stdoutput.write("done\n");
> // read 1mb of characters from the file, buffered (10 times)
> stdoutput.write("reading buffered...\n");
> for (i=0; i<10; i++) {
> f.setPositionRelativeToBeginning(0);
> for (j=0; j<1024*1024; j++) {
> f.read(&c);
> }
> }
> stdoutput.write("done\n");
> }

### Optimizing File Access

...

> int main(int argc, const char **argv) {
> // FIXME: example...
> }

### Temporary Files

...

> int main(int argc, const char **argv) {
> // FIXME: example...
> }

### Links

...

> int main(int argc, const char **argv) {
> // FIXME: example...
> }

### Fifos

...

> int main(int argc, const char **argv) {
> // FIXME: example...
> }

### Named Pipes

...

> int main(int argc, const char **argv) {
> // FIXME: example...
> }

### Convenience Methods

...

> int main(int argc, const char **argv) {
> // FIXME: example...
> }