12. Data Persistence

The modules described in this chapter support storing Python data in a persistent form on disk. The pickle and marshal modules can turn many Python data types into a stream of bytes and then recreate the objects from the bytes. The various DBM-related modules support a family of hash-based file formats that store a mapping of strings to other strings.

The list of modules described in this chapter is:

• 12.1. pickle — Python object serialization
  • 12.1.1. Relationship to other Python modules
    • 12.1.1.1. Comparison with marshal
    • 12.1.1.2. Comparison with json
  • 12.1.2. Data stream format
  • 12.1.3. Module Interface
    • HIGHEST_PROTOCOL
    • DEFAULT_PROTOCOL
    • dump()
    • dumps()
    • load()
    • loads()
    • PickleError
    • PicklingError
    • UnpicklingError
    • Pickler
      • Pickler.dump()
      • Pickler.persistent_id()
      • Pickler.dispatch_table
      • Pickler.fast
    • Unpickler
      • Unpickler.load()
      • Unpickler.persistent_load()
      • Unpickler.find_class()
  • 12.1.4. What can be pickled and unpickled?
  • 12.1.5. Pickling Class Instances
    • object.__getnewargs_ex__()
    • object.__getnewargs__()
    • object.__getstate__()
    • object.__setstate__()
    • object.__reduce__()
    • object.__reduce_ex__()
    • 12.1.5.1. Persistence of External Objects
    • 12.1.5.2. Dispatch Tables
    • 12.1.5.3. Handling Stateful Objects
  • 12.1.6. Restricting Globals
  • 12.1.7. Performance
  • 12.1.8. Examples
• 12.2. copyreg — Register pickle support functions
  • constructor()
  • pickle()
  • 12.2.1. Example
• 12.3. shelve — Python object persistence
  • open()
  • Shelf.sync()
  • Shelf.close()
  • 12.3.1. Restrictions
    • Shelf
    • BsdDbShelf
    • DbfilenameShelf
  • 12.3.2. Example
• 12.4. marshal — Internal Python object serialization
  • dump()
  • load()
  • dumps()
  • loads()
  • version
• 12.5. dbm — Interfaces to Unix “databases”
  • error
  • whichdb()
  • open()
  • 12.5.1. dbm.gnu — GNU’s reinterpretation of dbm
    • error
    • open()
    • gdbm.firstkey()
    • gdbm.nextkey()
    • gdbm.reorganize()
    • gdbm.sync()
    • gdbm.close()
  • 12.5.2. dbm.ndbm — Interface based on ndbm
    • error
    • library
    • open()
    • ndbm.close()
  • 12.5.3. dbm.dumb — Portable DBM implementation
    • error
    • open()
    • dumbdbm.sync()
    • dumbdbm.close()
• 12.6. sqlite3 — DB-API 2.0 interface for SQLite databases
  • 12.6.1. Module functions and constants
    • version
    • version_info
    • sqlite_version
    • sqlite_version_info
    • PARSE_DECLTYPES
    • PARSE_COLNAMES
    • connect()
    • register_converter()
    • register_adapter()
    • complete_statement()
    • enable_callback_tracebacks()
  • 12.6.2. Connection Objects
    • Connection
      • Connection.isolation_level
      • Connection.in_transaction
      • Connection.cursor()
      • Connection.commit()
      • Connection.rollback()
      • Connection.close()
      • Connection.execute()
      • Connection.executemany()
      • Connection.executescript()
      • Connection.create_function()
      • Connection.create_aggregate()
      • Connection.create_collation()
      • Connection.interrupt()
      • Connection.set_authorizer()
      • Connection.set_progress_handler()
      • Connection.set_trace_callback()
      • Connection.enable_load_extension()
      • Connection.load_extension()
      • Connection.row_factory
      • Connection.text_factory
      • Connection.total_changes
      • Connection.iterdump()
  • 12.6.3. Cursor Objects
    • Cursor
      • Cursor.execute()
      • Cursor.executemany()
      • Cursor.executescript()
      • Cursor.fetchone()
      • Cursor.fetchmany()
      • Cursor.fetchall()
      • Cursor.close()
      • Cursor.rowcount
      • Cursor.lastrowid
      • Cursor.arraysize
      • Cursor.description
      • Cursor.connection
  • 12.6.4. Row Objects
    • Row
      • Row.keys()
  • 12.6.5. Exceptions
    • Warning
    • Error
    • DatabaseError
    • IntegrityError
    • ProgrammingError
    • OperationalError
    • NotSupportedError
  • 12.6.6. SQLite and Python types
    • 12.6.6.1. Introduction
    • 12.6.6.2. Using adapters to store additional Python types in SQLite databases
      • 12.6.6.2.1. Letting your object adapt itself
      • 12.6.6.2.2. Registering an adapter callable
    • 12.6.6.3. Converting SQLite values to custom Python types
    • 12.6.6.4. Default adapters and converters
  • 12.6.7. Controlling Transactions
  • 12.6.8. Using sqlite3 efficiently
    • 12.6.8.1. Using shortcut methods
    • 12.6.8.2. Accessing columns by name instead of by index
    • 12.6.8.3. Using the connection as a context manager
  • 12.6.9. Common issues
    • 12.6.9.1. Multithreading