The Berkeley DB Package: Interface Changes introduced in DB 2.2.0
Description: DB: A database programmatic toolkit.
Keywords: embedded,database,programmatic,toolkit,b+tree,btree,hash,hashing,transaction,transactions,locking,logging,access method,access methods

### Interface Changes introduced in DB 2.2.0:

1. In DB 2.2.0, the handles returned by the DB subsystems, for example, the db_appinit() function's DB_ENV, and the db_open() function's DB, are free-threaded. If you specify the DB_THREAD flag to each subsystem, or globally using db_appinit(), you can then use the returned handle concurrently in any number of threads.

   There are some important caveats for using DB handles concurrently in multiple threads:

   1. The DB_THREAD flag must be specified for all subsystems either explicitly by calling the subsystems open() function, or via the db_appinit() function. Setting the DB_THREAD flag inconsistently may result in database corruption.

   2. Spinlocks must have be implemented for the compiler/architecture combination. Attempting to specify the DB_THREAD flag will fail with an EINVAL error if spinlocks are not available.

   3. Only a single thread may call the close function for a returned database or subsystem handle. See db_open(3) and the appropriate subsystem manual pages for more information.

   4. Either the DB_DBT_MALLOC or DB_DBT_USERMEM flags must be set in a DBT used for key or data retrieval. See db_open(3) for more information.

   5. The DB_CURRENT, DB_NEXT and DB_PREV flags to the log_get() function may not be used by a free-threaded handle. If such calls are necessary, a thread should explicitly create a unique DB_LOG handle by calling log_open(). See db_log(3) for more information.

   6. Each database operation (that is, any call to a function underlying the handles returned by db_open() and db_cursor()) is normally performed on behalf of a locker associated with the handle. If, within a single thread of control, multiple calls on behalf of the same locker are desired, then transactions must be used. See db(3) for more information.

   7. Transactions may not span threads, that is, each transaction must begin and end in the same thread, and each transaction may only be used by a single thread.

   See db(3) and db_appinit(3) for more information on using DB in the context of threads.

2. We've added spinlock support for OSF/1, HP, Solaris and UTS4, in all cases for the native compiler/architecture combination. In the case of Solaris, this means that all DB applications must be loaded using the -lthread library.

3. We've added a C++ API for DB. There are not yet any manual pages for the C++ API, and the interface is going to change! Please feel free to browse and send us comments, but please remember that the C++ interface is going to be different in the next release.

   The C++ files and directories in the source distribution are:

   db/cxx/ | C++ API
   db/examples_cxx/ | The example programs recoded in C++.
   db/include/cxx_int.h | Internal C++ include file.
   db/include/db_cxx.h | External C++ include file.

   C++ support is automatically built on Win32. To configure it under UNIX, specify --enable-cxx as a configuration argument (see the file db/build.unix/README for more information).

4. The functionality previously embodied in some of the support utilities (db_archive(1), db_checkpoint(1), db_deadlock(1), and db_recover(1)) is now available through the DB API as well.

   ### Utility | ### Underlying API support:
   db_archive(1) | Log archival. See the log_archive() function in db_log(3)
   db_checkpoint(1) | Transaction checkpoint. See the txn_checkpoint() function in db_txn(3).
   db_deadlock(1) | Deadlock detection. See the lock_detect() function in db_lock(3).
   db_recover(1) | Database recovery. See the DB_RECOVER and DB_RECOVER_FATAL flags for the db_appinit() function in db_appinit(3).

5. We've added a new default hashing function, __ham_func5(), written by Glenn Fowler, Landon Curt Noll and Phong Vo, and integrated into DB by Ariel Faigon of SGI. We've also deleted the previous supplied function __ham_func1().

   This change is NOT transparent to applications. We incremented the hash access method database version number, and the new hash function will only be used in newly created databases, which means that applications written using version DB 2.2.0 and greater will be able to share databases with applications written using previous versions of DB with a major number of 2.

   However, we now use the __ham_func5() hash function internally, in the log and lock subsystems, which means that applications written using version DB 2.2.0 and greater will NOT be able to share database environments, or read log files, written using previous versions of DB.

6. The interfaces that DB uses to export statistics have been enhanced and, in one case, modified:

   1. DB 2.2 exports statistical information via the DB handle for the access methods. Currently, the only access method for which this information is available is B+tree. See the dbp->db_stat() function in the db_open(3) manual page for more information.

   2. DB 2.2 exports statistical information for the transaction region via a new function, txn_stat(). See db_txn(3) for more information.

   3. The db_stat(1) utility has two new options. The -d flag permits users to display the access method statistics. The -t flag permits users to display the transaction region statistics.

   4. The interface for shared memory buffer pool statistics in DB 2.2 has been revised to make it consistent with the interfaces provided for the transaction region and access methods. This change is NOT transparent to applications. See db_mpool(3) for more information.

7. The interface to the shared memory buffer pool has been extended in DB 2.2 to permit applications to control the maximum size of read-only files that will be mapped into the application's address space instead of being read through the memory pool cache. See the DB_NOMMAP flag to the memp_open() function and the mp_mmapsize field in the DB_ENV structure, as described in db_mpool(3).

8. The interface to the transaction subsystem has been extended in DB 2.2 to permit applications to specify that the log is not to be synchronously flushed on transaction commit. This potentially provides a significant performance improvement for applications that do not require database durability. See the DB_TXN_NOSYNC flag to the txn_open() function, as described in db_txn(3).

9. There have been several changes to the process of creating a DB environment:

   1. By default, when a lock is unavailable to a DB thread (or process), the thread/process is put to sleep for a period of time, permitting other threads/processes to run. This may not be optimal in the presence of multiple threads in a single process. The DB_ENV structure has been extended to permit applications to specify a ``yield'' function, which is called when a DB thread has requested a lock which is unavailable. See db_appinit(3) for more information.

   2. In previous versions of DB, the DB_CREATE flag was implied by calling the db_appinit() function, that is, initializing the environment implied that the application wanted to create the environment if it did not already exist. This version of DB no longer supports this semantic, and the DB_CREATE must be explicitly specified to db_appinit() if the application wants to create the DB environment. This change is NOT transparent to applications.

   3. The flags that may be specified when creating the DB environment have been extended in DB 2.2 to allow the specification of the DB_MPOOL_PRIVATE flag, which was previously supported only by the underlying memory pool subsystem.

      The flags that may be specified when creating the DB environment have been extended in DB 2.2 to provide new functionality: the list of new flags includes DB_NOMMAP, DB_THREAD and DB_TXN_NOSYNC.

   4. The DB_DATA_DIR configuration argument to the db_appinit() function is now additive, permitting applications to specify multiple directories in which to search for database files. If multiple paths are specified, created data files will always be created in the first directory specified.

   5. The db_errbuf field of the DB_ENV structure has been deleted from the current release. In its place, we have added the db_errcall field, which specifies a function which is called with the information previously found in the db_errbuf buffer. This change is NOT transparent to applications.

   6. The default temporary file location list (used when no DB_TMP_DIR configuration argument was specified) has been extended to include any directory specified by the TempFolder environment variable, if it exists.

   In all cases, see db_appinit(3) for more information.

10. There have been several changes to the logging subsystem:

    1. The log_get() and log_put() functions now support the standard DBT flags described by the db_open(3) manual page.

    2. The interface to the log_flush() function has been extended to flush the entire log if a NULL LSN is specified. See db_log(3) for more information.

    3. The interface to the log_file() function has been changed in DB 2.2 to eliminate the need for the library to return allocated memory that may never be freed. This change is NOT transparent to applications. See db_log(3) for more information.

    4. The db_checkpoint(1) and db_deadlock(1) utilities have a new option, -L, in DB 2.2 to optionally log their process ID to a file. If they exit gracefully, or if they receive a SIGINT signal, the log file is removed before they exit.

11. There have been a couple of changes to the source code layout other than those specified above:

    1. Operating system specific functionality is separated out in DB 2.2 into a separate subdirectory in the source code, db/os.

    2. All of the include files in DB 2.2 have been moved into a single subdirectory, db/include, as part of the work to port DB to MacOS.