CHAPTER 1 INTRODUCTION TO BTRIEVE

Btrieve is a key-indexed record management system designed
for high-performance data handling and improved programming
productivity.  Btrieve allows your application to retrieve,
insert, update, or delete records either by key value or by
sequential or random access methods.


Client-Server Design

Btrieve products include server-based Btrieve (also
called NetWare Btrieveª) and client-based Btrieve.  In
most cases, applications written for client-based
Btrieve can run on server-based Btrieve and vice versa.

Server-based Btrieve is provided with the NetWare
operating system and includes the following:

o Server-based Btrieve Record Manager, which runs at the
  server and manages data I/O with the file system.

o Btrieve communications programs, which handle incoming
  requests from a remote source and which can route requests
  from a server-based application to a copy of the Record
  Manager running on a remote server.

o Btrieve Requesters, which run at the workstation and
  handle data I/O between the workstation and the server.
  The Requesters allow applications running at the
  workstation to communicate transparently with the Record
  Manager.  Btrieve Requesters are available for DOS, OS/2,
  Windows, and UnixWareª workstations.

o Btrieve utilities, which provide setup, rebuild, monitor,
  maintenance, and recovery programs for Btrieve users and
  files.

o This manual, which documents using Btrieve in a NetWare
  environment.

Client-based Btrieve is available only as part of a
Btrieve Developer's Kit (DOS, OS/2, or Windows), which
must be purchased separately.  Each Btrieve
Developer's Kit includes the following:

o Client-based Record Manager for the applicable
  environment (DOS, OS/2, or Windows).  The client-based
  Record Manager executes all processing on the workstation.
  It accesses all files through operating system calls.  The
  operating system calls are either executed locally (for
  local files) or redirected to the server (for files on the
  server).

o Btrieve utilities, which provide setup, rebuild (Btrieve
  v5.x to v6.x files), monitor, maintenance, and recovery
  programs for Btrieve users and files.

o An installation and operation manual, which documents
  using Btrieve in the applicable environment (DOS, OS/2, or
  Windows).

o Btrieve Programmer's Manual, which documents the
  Application Programming Interface (API) and the language
  interfaces that allow Btrieve to be called from various
  programming languages.


Btrieve Features

The following sections introduce some of the features
that make Btrieve a uniquely powerful record
management system.  Index MaintenanceBtrieve
automatically creates and maintains file indexes as
records are inserted, updated, and deleted.  In
addition to automatic index maintenance, Btrieve
supports the following index features:

o Up to 119 key segments per file

o Adding or dropping any index after a file has been created

o Numerous data types for key values:  integer, float,
  date, time, decimal, money, logical, numeric, bfloat,
  string, lstring, zstring, unsigned binary, autoincrement,
  and sign trailing separate

o Numerous key attributes:  linked/repeating duplicatable,
  duplicatable/nonduplicatable, supplemental,
  modifiable/nonmodifiable, segmented/nonsegmented,
  descending/ascending sorting,
  case-sensitive/case-insensitive sorting, alternate
  collating sequence, null
  (any-segment/all-segment)/non-null


File Specifications

Btrieve offers these file specifications :

o File sizes up to 4 billion bytes (4 gigabytes)

o Number of records limited only by the size limit of the
  file

o Consistent file definition and management routines
  independent of the operating environment

o Consistent file structures


Memory Management

The cache is an area of memory Btrieve reserves for
buffering the pages that it reads.  When your
application requests a record, Btrieve first checks
the cache to see if the page containing that record is
already in memory.  If so, Btrieve transfers the
record from the cache to your application's data
buffer.  If the page is not in the cache, Btrieve
reads the page from the disk into a cache buffer
before transferring the requested record to your
application.

If every cache buffer is full when Btrieve needs to
transfer a new page into memory, a least-recently-used
(LRU) algorithm determines which page in the cache
Btrieve should overwrite.  The LRU algorithm reduces
processing time by keeping the most recently
referenced pages in memory.

Generally, a larger cache improves performance because
it allows more pages to be in memory at a given time.
Btrieve allows you to specify the amount of memory to
reserve for the I/O cache buffers.  To determine this
amount of memory, consider your application memory
requirements (if your application is an NLM), the
total amount of memory installed on your server, and
the combined size of all files your application will
access.


Data Integrity

The following Btrieve features will let you support
concurrent access while ensuring the integrity of your
files in a multiuser environment:

o Single-record and multiple-record locks.

o Concurrent and exclusive transactions.  (See "Concurrent
  Transactions.")

o Deadlock detection (in a server-based environment).

o Shadow paging, which entails making changes to a copy of
  a page's image rather than changing the original page when
  inserting, updating, or deleting a record.  (See "Shadow
  Paging.")

o Logging feature, which records (in a log file) any
  changes made to a designated file.

o Roll Forward utility, which uses log files maintained by
  Btrieve's logging feature to recover data corrupted by a
  system or server failure.


Security Controls

Btrieve provides the following capabilities for
enhancing data security in a network environment:

o Assigning owner names to filesu Specifying dynamic
  encryption and decryption of data

o Providing NetWare file-level security (provided through
  the Btrieve Requesters)


Enhancements to Btrieve

Btrieve v6.x provides several new features and
performance enhancements that support the requirements
of today's powerful database management systems.  The
following sections describe these enhancements.The
first section describes the enhancements that apply
only to Btrieve v6.1.  The second section describes
enhancements that apply to Btrieve v6.x (that is, to
both v6.0 and v6.1).


Btrieve v6.1 Enhancements

This section describes enhancements to Btrieve that
apply to v6.1.


Support for Operating on a Portion of a Record: Chunks

Btrieve v6.1 allows you to operate on portions of a
record, called chunks, rather than on the entire
record.  This enhancement provides new chunk
operations that work on any file conforming to the
Btrieve v6.x file format.


New Operations to Support Records Larger than 64 KB

Through the use of the Get Direct/Chunk (23) and Update
Chunk (53) operations, Btrieve v6.1 supports records
larger than 64 KB.  Applications can use chunk
operations to build records larger than 64 KB in any
Btrieve v6.x file that allows variable-length records.


New File Structure to Support Very Long Records

Btrieve v6.1 also allows an application to create
Btrieve files that contain structures called
Variable-tail Allocation Tables (VATs).  VATs speed
Btrieve's access to the data residing at large offsets
in very long records.  VATs also significantly reduze
the size of buffers needed by Btrieve to process
records in files that use data compression.


Multiple Alternate Collating Sequences

Btrieve v6.1 allows you to specify a separate alternate
collating sequence (ACS) for each key in a file.
Btrieve files with multiple keys are no longer
restricted to having only one ACS.


Locale-Specific Collating Sequences

Btrieve v6.1 allows you to instruct Btrieve to build an
ACS that is sensitive to the specified locale's
character sorting order.  This ability allows Btrieve
to sort according to a character set specified by a
particular country ID and code page.


Index Balancing

When an index page becomes full, Btrieve automatically
creates a new index page and splits the values in the
full page between the two pages.  Btrieve v6.1 now
offers the option of using index balancing instead.

When you use index balancing, Btrieve looks for
available space in other index pages associated with
the same key each time an index page becomes full.
Btrieve then rotates values from the full page into
the pages that have space available.  Index balancing
increases index page utilization, results in fewer
pages, and produces an even distribution of keys among
nodes on the same level.


Performing Reads While Creating an Index

With Btrieve v6.1, you can perform reads while a Create
Index operation (13) is executing.  Previous versions
of Btrieve locked the entire file when executing a
Create Index operation.


New Data Type

Btrieve v6.1 allows you to specify a new data type for
keys called sign trailing separate (STS).  STS is a
COBOL data type.  Basically a numeric data type, it is
represented as an ASCII string that is right-justified
and padded with zeros.


Percentage Operations

Btrieve v6.1 provides two new operations that a
window-oriented application can use for implementing
scroll bars.The Find Percentage (45) operation finds
the position of a record either relative to a key path
or as the recordÕs physical location within the file.
The position is expressed as a percentage value.

The Get By Percentage (44) operation retrieves a record by
that recordÕs position in the Btrieve file, where the
position is based on a percentage value you supply when you
call the operation.  You can specify whether the position is
relative to a specified key path, or represents the
record's actual physical location in the file.


No Currency Change

Btrieve v6.1 provides a No Currency Change option on
inserts and updates.  In previous versions of Btrieve,
positioning was reestablished based on the key value of the
inserted or updated record.


New Ability to Specify Repeating- or Linked-Duplicatable Keys
on Create Operations

On the Create (14) and Create Index (31) operations,
Btrieve v6.1 allows you to specify a key as either a
repeating-duplicatable key or a linked-duplicatable
key.


Improved Btrieve Requesters

The Btrieve DOS and OS/2 Requesters now support MAP
ROOT drives and NetWare file-level security.  Since
the Windows Requester requires the DOS Requester,
Windows users can also take advantage of these
features.  When opening files in a NetWare v3.11
environment, Btrieve Requesters provide enhanced
performance by reducing the bindery access for each
file and reducing network traffic in general.

The Btrieve v6.1 Requesters have three new features:

o Automatic support for double-byte character environments

o Support for the NetWare Runtime serialized NetWare
  operating system

o Optional compression of data prior to network
  transmission

NOTE:  Btrieve now provides a requester for the
UnixWare environment.  For information about this new
UnixWare Requester, please refer to the Readme file
that accompanies this release.


Btrieve 6.x Enhancements

This section describes ehancements that apply to
Btrieve v6.x (that is, to Btrieve v6.0 and Btrieve
v6.1).


New File Format

When creating files, Btrieve v6.x uses a new file
format that allows faster data access than was possible
with previous Btrieve versions.  This format,
introduced in Btrieve v6.0 and modified slightly in
Btrieve v6.1, is responsible for many of the
enhancements and new features available with these
releases.

Btrieve v6.0 operates on any file created with Btrieve
v6.1 unless that file uses v6.1 features that altered
the Btrieve file format (such as multiple ACSs and
VATs).

Pre-6.x versions of Btrieve cannot open files that
have a v6.x format.  However, Btrieve v6.x can open
files created with earlier versions of Btrieve.  When
Btrieve v6.x opens files from earlier versions, it
does not convert the files to the v6.x format.

The Create Btrieve Files in Pre v6.x Format
configuration option is useful if you must use
newly-created files with versions of Btrieve prior to
v6.0.


Online Backups

Through a feature called continuous operation, Btrieve
v6.x now lets you back up Btrieve files while they are
open and in use.  This feature is important for
applications that conduct transactions 24 hours a
day.When you enable the continuous operation feature,
Btrieve opens each original file in read-only,
shareable mode to allow backup utilities to access the
file's static image.  Any changes to the original file
that occur during the backup are stored in a temporary
file called a delta file.  When the backup ends,
Btrieve automatically updates the original file with
the changes from the delta file and then deletes the
temporary delta file.


Concurrent Transactions

Concurrent transactions allow one or more applications
to run multiple transactions simultaneously for the
same Btrieve file (if the file uses the Btrieve v6.x
format).Versions of Btrieve prior to v6.0 support only
one type of transaction:  exclusive.  When an
application reads, updates, inserts, or deletes a
record from a file in an exclusive transaction,
Btrieve locks the entire file for the duration of the
transaction.  This type of locking is known as
file-level transaction locking.

Once a file is locked in an exclusive transaction,
other users can still read the file, provided they are
not involved in exclusive transactions and are not
attempting to lock the file themselves.  They cannot,
however, make any changes to the file.

If the file uses the Btrieve v6.x format, other users
will not see changes that occur during the transaction
until that transaction ends.  (If the file uses the
Btrieve v5.x file format, other users can see such
changes.)

In addition to supporting exclusive transactions,
Btrieve v6.x supports a new type of transaction:
concurrent.  When a Btrieve v6.x file is included in a
concurrent transaction, modifications cause Btrieve to
lock only the page (or pages, if the record is
variable length) that contains the record, as well as
its associated index pages.

This allows other users to modify or include the same
file in their own concurrent transaction, as long as
no concurrent has already locked the pages that
contain the records to be modified (or any affected
index pages).

Concurrent transactions have the following additional
features:

o Locked pages remain locked for the duration of the
  transaction.

o If the transaction simply reads a record, Btrieve does
  not lock the corresponding page (unless a read lock is
  applied).

o Other users can read the data on the locked pages;
  however, they cannot lock the pages (by updating or
  applying an explicit read lock).

o Other users cannot see the changes to a file in a
  transaction until the transaction ends.  (That way, if a
  system failure occurs before the transaction completes,
  other users will not have read false data--that is, data
  that will be rolled back.)

NOTE:  For compatibility, Btrieve v6.x continues to
support exclusive transactions, but with one
difference from previous versions:  if the exclusive
transaction affects Btrieve v6.x files, other users
cannot see the changes to those files until the
transaction ends.


Shadow Paging

With Btrieve file versions prior to v6.0, Btrieve uses
pre-imaging to protect files from corruption in case
of a system failure.  Before updating a file, Btrieve
creates a temporary pre-image file.  This file
contains the pages to be updated from the original
file.  Btrieve then performs the update on the
original file.  If the system fails during the update,
Btrieve can restore the original file using the
pre-image file.

A new Btrieve v6.x page handling technique called
shadow paging has replaced pre-imaging.  When a user
needs to change a page (either inside or outside a
transaction), Btrieve creates a shadow page- a virtual
copy of the original page within the same Btrieve
file.  Btrieve then makes the changes to the shadow
page instead of the original.  When the changes are
committed (either when the operation is complete or the
transaction ends), Btrieve designates the shadow page
as the current page, and the original page becomes
available for reuse.  If a system failure occurs
before the changes are committed, Btrieve drops the
shadow page, and the current page remains in its
original condition.

With shadow paging, each user works with a virtual
copy of the page.  Consequently, two users can access
the same logical page, and neither user sees the other
user's changes until the operation or transaction is
complete.  Also, shadow paging enhances reliability
because the original file is always valid and
internally consistent.


New Caching Algorithms

Btrieve v6.x provides new caching algorithms that
improve memory management for concurrent users.  These
new algorithms include hashing search methods for
improved access, concurrent sharing of a single cahe,
and use of existing cache across operations.


Better Usage of Large Data Files

Btrieve v6.x provides faster access and more efficient
use of large data files.  With Btrieve v6.x, you can
create additional indexes for large data files more
quickly than in previous versions, and new merge sorts
take advantage of whatever cache is available.


Up to 119 Key Segments

Btrieve v6.x supports up to 119 key segments in files
with a page size of 4,096 bytes.  The maximum number
of key segments you can define for a file depends on
the file's page size.  Versions of Btrieve prior to
v6.0 supported a maximum of 24 key segments.


Adding and Dropping Any Index

Btrieve v6.x supports adding and dropping any index.
(In versions prior to Btrieve v6.0, you could add and
drop only supplemental indexes.) Also, you can drop
indexes without renumbering the remaining indexes.


Specific Key Numbers Allowed When Creating a File or Index

Btrieve v6.x allows an application to assign specific
key numbers when creating a v6.x format file with
indexes, or when creating an index for a preexisting
v6.x file.


Optional Renumbering of Keys

With Btrieve v6.x, the Drop Index operation (32)
allows an application to specify whether to renumber
the remaining keys when an index is dropped from a
Btrieve file.  (Note that the file must use the
Btrieve v6.x format.)


Enhanced Support for Case-Insensitive Keys

With Btrieve v6.x, you can create case-insensitive keys
without using an alternate collating sequence.  When
creating a file, you can use the key specifications to
specify case-insensitive keys.  With case-insensitive
keys, the letter a is sorted the same as the letter A.


Enhancement to autoincrement Key

In Btrieve v6.x, you can initialize the value of a
field in every record of a file to zero and later add
an index of type autoincrement.  This feature allows
you to prepare for an autoincrement key without
actually building the index until it is needed.


Reserved Space for Duplicate Pointers

Btrieve v6.x allows reserving duplicate pointers on
data records for linked duplicate keys.  When creating
a Btrieve v6.x file, an application can reserve space
in the data records for extra, unused duplicate
pointers.  Later, when an application adds an index
that allows duplicate values, Btrieve stores pointers
to those duplicate values in the reserved space
(unless the Repeating Duplicates key attribute has
been specified).


Key-Only File Modification

With Btrieve v6.x, you can update and delete records in
key-only files.  In versions of Btrieve prior to v6.0,
you could only insert records.


New Stat Option

A new option in the Stat operation (15) allows an
application to obtain additional information such as a
file's Btrieve version and the number of unused
duplicate pointers.  This feature also works with
files created with previous versions of Btrieve.


Locking in Extended Operations

Unlike previous versions of Btrieve, v6.x allows an
application to use locks on the extended operations:
Get Next Extended (36), Get Previous Extended (37),
Step Next Extended (38), and Step Previous Extended
(39).


Support for Referential Integrity

Btrieve v6.x supports the use of referential integrity
(RI) constraints created through Novell's NetWare SQL
relational data access system.  RI ensures that
dependent data stays synchronized throughout the
database.  No Btrieve operations currently exist to
manipulate RI constraints directly; you must use
NetWare SQL.