Introduction

Behold the fileone of the first things that any reasonably pragmatic programmer reaches for in a programming language's toolbox. Because processing external files is a very real, tangible task, the quality of file-processing interfaces is a good way to assess the practicality of a programming tool.

As the recipes in this chapter attest, Python shines in this task. Files in Python are supported in a variety of layers: from the built-in open function (a synonym for the standard file object type), to specialized tools in standard library modules such as os, to third-party utilities available on the Web. All told, Python's arsenal of file tools provides several powerful ways to access files in your scripts.

File Basics

In Python, a file object is an instance of built-in type file. The built-in function open creates and returns a file object. The first argument, a string, specifies the file's path (i.e., the filename preceded by an optional directory path). The second argument to open, also a string, specifies the mode in which to open the file. For example:

input = open('data', 'r')
output = open('/tmp/spam', 'w')

open accepts a file path in which directories and files are separated by slash characters (/), regardless of the proclivities of the underlying operating system. On systems that don't use slashes, you can use a backslash character (\) instead, but there's no real reason to do so. Backslashes are harder to fit nicely in string literals, since you have to double them up or use "raw" strings. If the file path argument does not include the file's directory name, the file is assumed to reside in the current working directory (which is a disjoint concept from the Python module search path).

For the mode argument, use 'r' to read the file in text mode; this is the default value and is commonly omitted, so that open is called with just one argument. Other common modes are 'rb' to read the file in binary mode, 'w' to create and write to the file in text mode, and 'wb' to create and write to the file in binary mode. A variant of 'r' that is sometimes precious is 'rU', which tells Python to read the file in text mode with "universal newlines": mode 'rU' can read text files independently of the line-termination convention the files are using, be it the Unix way, the Windows way, or even the (old) Mac way. (Mac OS X today is a Unix for all intents and purposes, but releases of Mac OS 9 and earlier, just a few years ago, were quite different.)

The distinction between text mode and binary mode is important on non-Unix-like platforms because of the line-termination characters used on these systems. When you open a file in binary mode, Python knows that it doesn't need to worry about line-termination characters; it just moves bytes between the file and in-memory strings without any kind of translation. When you open a file in text mode on a non-Unix-like system, however, Python knows it must translate between the '\n' line-termination characters used in strings and whatever the current platform uses in the file itself. All of your Python code can always rely on '\n' as the line-termination character, as long as you properly indicate text or binary mode when you open the file.

Once you have a file object, you perform all file I/O by calling methods of this object, as we'll discuss in a moment. When you're done with the file, you should finish by calling the close method on the object, to close the connection to the file:

input.close( )

In short scripts, people often omit this step, as Python automatically closes the file when a file object is reclaimed during garbage collection (which in mainstream Python means the file is closed just about at once, although other important Python implementations, such as Jython and IronPython, have other, more relaxed garbage-collection strategies). Nevertheless, it is good programming practice to close your files as soon as possible, and it is especially a good idea in larger programs, which otherwise may be at more risk of having excessive numbers of uselessly open files lying about. Note that TRy/finally is particularly well suited to ensuring that a file gets closed, even when a function terminates due to an uncaught exception.

To write to a file, use the write method:

output.write(s)

where s is a string. Think of s as a string of characters if output is open for text-mode writing, and as a string of bytes if output is open for binary-mode writing. Files have other writing-related methods, such as flush, to send any data being buffered, and writelines, to write a sequence of strings in a single call. However, write is by far the most commonly used method.

Reading from a file is more common tha