Python Compatibility

Bcfg2 maintains compatibility with a wide range of Python versions – currently 2.4 through 3.2 This requires some (often significant) compatibility interfaces. This page documents the compatibility library, Bcfg2.Compat, and its contents.

Note that, due to limitations in Sphinx (the Bcfg2 documentation tool), this documentation is not automatically generated. Compat.py should always be considered the authoritative source.

There are several basic types of symbols found in Bcfg2.Compat:

  • Symbols whose names have changed, e.g., MutableMapping.
  • Symbols whose modules have changed names, e.g., urljoin/urlparse
  • Symbols that have been added or removed, e.g., any/all, reduce, walk_packages
  • Symbols that need some significant workaround to function identically in all versions, e.g., b64encode/b64decode.
  • Helper functions and classes for compatibility, e.g., CmpMixin.

Using Bcfg2.Compat

To use the compatibility libraries, simply import them as such:

from Bcfg2.Compat import StringIO, all

The individual symbol import is preferred over just importing Bcfg2.Compat as a whole, since in the future we will be able to remove some items from the library and this makes that process easier. A wildcard import is definitely discouraged.

Bcfg2.Compat symbols

Bcfg2.Compat defines the following symbols:

Py3K compatibility imports

The following symbols are imported to provide compatibility with Python 3. In cases where the newer symbols has also been backported to Python 2, the older symbol will be used unless otherwise noted. This is to ensure that functions or modules with radically different behavior (e.g., input()) do not cause unexpected side-effects.

Name Python 2 Python 3
urljoin urlparse.urljoin() urllib.parse.urljoin()
urlparse urlparse.urlparse() urllib.parse.urlparse()
urlretrieve urllib.urlretrieve() urllib.request.urlretrieve()
HTTPBasicAuthHandler urllib2.HTTPBasicAuthHandler urllib.request.HTTPBasicAuthHandler
HTTPPasswordMgrWithDefaultRealm urllib2.HTTPPasswordMgrWithDefaultRealm urllib.request.HTTPPasswordMgrWithDefaultRealm
build_opener urllib2.build_opener() urllib.request.build_opener()
install_opener urllib2.install_opener() urllib.request.install_opener()
urlopen urllib2.urlopen() urllib.request.urlopen()
HTTPError urllib2.HTTPError urllib.error.HTTPError
URLError urllib2.URLError urllib.error.URLError
StringIO cStringIO.StringIO() io.StringIO
ConfigParser ConfigParser configparser
cPickle cPickle pickle
Queue Queue.Queue queue.Queue
Empty Queue.Empty queue.Empty
Full Queue.Full queue.Full
xmlrpclib xmlrpclib xmlrpc.client
SimpleXMLRPCServer SimpleXMLRPCServer xmlrpc.server
SocketServer SocketServer socketserver
httplib httplib http.client
input raw_input() input()
reduce reduce() functools.reduce()
long long() int()
cmp cmp() Not implemented

Python 2.4 compatibility

The following symbols are imported or defined to provide compatibility with Python 2.4 (and occasionally 2.5). Be sure to read the notes below, since some of these implementations may be feature-incomplete.

Name Python 2.4 Python 2.5+
formatdate email.Utils.formatdate() email.utils.formatdate()
walk_packages Not implemented pkgutil.walk_packages() (2.6+)
any Not implemented any()
all Not implemented all()
wraps Not implemented functools.wraps()
md5 md5.md5() hashlib.md5
MutableMapping UserDict.DictMixin collections.MutableMapping (2.6+)
literal_eval eval() ast.literal_eval() (2.6+)

walk_packages

The walk_packages implementation for Python 2.5 is feature-complete. The implementation for Python 2.4 is not. Differences:

  • Requires a full path, not a path relative to something in sys.path. Anywhere we care about that shouldn’t be an issue.
  • The first element of each tuple is None instead of an importer object.

wraps

The wraps implementation for Python 2.4 is a no-op. It does not attempt to copy the docstring or other details from the original function to the wrapped function.

md5

hashlib is available for Python 2.4, but it is not part of the standard base. If it is installed, it will be used. If you are doing something fancy with MD5 sums, you may need to determine which object is in use, since they are not equivalent. For the majority of simple cases – finding the MD5 sum of a string – they are equivalent enough.

MutableMapping

collections.MutableMapping provides a subset of the functionality of UserDict.DictMixin; that is, any object that is written to work with MutableMapping will also work with DictMixin, so you should write classes with MutableMapping in mind.

collections.MutableMapping is available in Python 2.6+, and will be used if available.

literal_eval

ast.literal_eval() is a safe version of eval() that will only allow delaration of literal strings, ints, list, dicts, etc. This was introduced in Python 2.6, and as such Python 2.4 uses the plain-old eval().

Other Symbols

The following functions, classes, and symbols are provided for other miscellaneous reasons.

u_str

Bcfg2.Compat.u_str(string, encoding=None)[source]

print to file compatibility

b64encode/b64decode

In Python 2, base64.b64encode() and base64.b64decode() expect strings and return strings. In Python 3, they expect bytes and return bytes. For Python 3, Bcfg2.Compat provides b64encode and b64encode that transparently encode strings into bytes, then decode bytes into strings, so that those functions can be used identically to their use in Python 2.

CmpMixin

class Bcfg2.Compat.CmpMixin[source]

Bases: object

In Py3K, object.__cmp__() is no longer magical, so this mixin can be used to define the rich comparison operators from __cmp__ – i.e., it makes __cmp__ magical again.

unicode

In Py3k, the unicode() class is not defined, because all strings are unicode. Bcfg2.Compat defines unicode as equivalent to str() in Python 3.

oct_mode

Bcfg2.Compat.oct_mode(mode)[source]

Convert a decimal number describing a POSIX permissions mode to a string giving the octal mode. In Python 2, this is a synonym for oct(), but in Python 3 the octal format has changed to 0o000, which cannot be used as an octal permissions mode, so we need to strip the ‘o’ from the output. I.e., this function acts like the Python 2 oct() regardless of what version of Python is in use.

Parameters:mode (int) – The decimal mode to convert to octal
Returns:string