List:General Discussion« Previous MessageNext Message »
From:Paul McCullagh Date:August 8 2006 8:38am
Subject:PBXT version 0.9.6 has been released
View as plain text  
Hi All,

PBXT version 0.9.6 has been released and can be downloaded from 
http://www.primebase.com/xt.

Most of the mysql-test-run scripts now run correctly with PBXT as the 
default storage engine. This version has also been compiled and tested 
on a multi-processor, 64-bit machine. In addition, all bugs reported so 
far have been fixed. For further information on this release, please 
read my blog, http://pbxt.blogspot.com, and the associated comments. 
Details of all changes made are given in the release notes below.

PrimeBase XT (PBXT) is a new transactional database engine for MySQL. 
It uses multi-version concurrency control (MVCC) and has been designed 
for web-based applications and other high concurrency environments. The 
Beta release of PBXT is planned for September 2006 and will include 
support for the 5.1 storage engine API. The current version is based on 
MySQL 4.1.16 and runs on Mac OS X and Linux.

Any questions, comments and bug reports can be sent directly to me. 
Thanks for your support!

Best regards,

Paul McCullagh
SNAP Innovation GmbH


PBXT Release Notes
==================

------- 0.9.6 - 2006-08-05

RN35: ++++ NOTE: This version is incompatible to older versions of PBXT 
++++.

The disk format of tables and log files has changed slightly in this 
version. As a result, files created by older versions cannot be opened 
by version 0.9.6. An error will be generated. If you have data wish to 
preserve, first start the older version of XT and convert all tables to 
MyISAM. The stop the server and removed all transaction log file (files 
of the form xtlog-*.xt). Then start the new version and convert tables 
back to XT.

RN34: Implemented READ COMMITTED transaction mode. XT now supports READ 
COMMITTED and SERIALIZABLE transaction modes. NOTE: if the mode is set 
to REPEATABLE READ, SERIALIZABLE is used. If the mode is set to READ 
UNCOMMITTED READ COMMITTED is used.

RN33: The implementation of AUTO_INCREMENT on a paritial index is 
non-standard. A unique value is generated without regard to the value 
of the index prefix. For example, assume we have the following table: 
CREATE TABLE t1 (c1 CHAR(10) not null, c2 INT not null AUTO_INCREMENT, 
PRIMARY KEY(c1, c2));

With the following contents:   c1 c2
                                A  8
                                B  1

After executing the following statement: insert into t1 (c1) values 
('B');

This is the result using PBXT: c1 c2
                                A  8
                                B  1
                                B  9

The standard result would be:  c1 c2
                                A  8
                                B  1
                                B  2

RN32: PBXT does not permit access to multiple databases within a single 
transaction. For example:

begin;
update database_1.t1 set a=10;
update database_2.t2 set d=10;
commit;

In this case the following error is returned: 1015: Can't lock file 
(errno: -1)

RN31: The implementation of COUNT(*) has changed. For efficiency, rows 
are not counted. The information is taken from the header of the record 
(.xtr) files. This information is only 100% accurate after transaction 
cleanup has completed. Which basically means, only when PBXT is idle. 
ANALYZE TABLE waits for all background activity to stop, so the 
statement may be executed before a COUNT(*) to ensure an accurate 
result. NOTE: Other then waiting for background processes, ANALYSE 
TABLE is not implemented.

RN30: Two concurrency bugs have been fixed: a shared lock was used 
instead of an exclusive lock when deleting from a transaction list, the 
transaction segment semaphore was not initialized. XT now runs 
correctly in a multi-processor environment. The test used was sysbench 
on a dual-process, dual-core, AMD 64-bit machine running SUSE Linux 
10.0.

RN29: PBXT compiles and runs on under 64-bit Lunix. [RESOLVED: 0009]

RN28: ./mysql-test-run --force --mysqld=--default-storage-engine=pbxt 
will now execute most tests successfully. Changes to the tests and the 
result have been documented in 
http://www.primebase.com/xt/download/pbxt-test-run-changes.txt. 
[RESOLVED: 0004, 0019]

RN27: Fixed a bug that caused the server to crash if when using tables 
locks and transactions. For example: LOCK TABLES, BEGIN, COMMIT, 
SELECT. This sequence now returns an error. The correct sequence is:

LOCK TABLES, BEGIN, COMMIT, UNLOCK TABLES, SELECT
or
LOCK TABLES, BEGIN, COMMIT, BEGIN, SELECT COMMIT, UNLOCK TABLES

RN26: Fixed a concurrency problem which caused a number of threads to 
hang during the sysbench test - see RN30 above (bug reported by Vadim).

RN25: Fixed a bug that caused the server to hang when ha_pbxt::create() 
and ha_pbxt::ha_open() where given different, but equivalent paths for 
a particular table.

RN24: Fixed bug in the indexing of blob columns, for example: create 
table t1(name_id int, name blob, INDEX name_idx (name(5)));

RN23: When a duplicate key error occurs in auto-commit mode, the 
transaction is now rolled back.

RN22: Fixed incorrect duplicate key error. In the case of a unique key 
which allows NULLs, duplicates are allowed if the inserted key contains 
a NULL. For example:

create table t1 (id int not null, str char(10), unique(str));
insert into t1 values (1, null),(2, null),(3, "foo"),(4, "bar");

RN21: PBXT now returns the correct error code on duplicate key: 1062 
instead of 1022.

RN19: Implemented AUTO_INCREMENT on partial keys. However, the XT 
implementation is non-standard. Increment of partial index works, but 
the ID generated is incremented like a non-partial index. For example:

create table t1 (c1 char(10) not null, c2 int not null auto_increment, 
primary key(c1, c2));
select * from t1;
c1 c2
A  8
B  1

insert into t1 (c1) values ('B');
select * from t1;
c1 c2
A  8
B  1
B  9

The standard result would be:
c1 c2
A  8
B  1
B  2

RN18: Implemented TRUNCATE TABLE and DELETE FROM <table>; (i.e. a 
DELETE without WHERE clause). Previously DELETE FROM <table>; did not 
cause an error, but no rows where deleted (TRUNCATE TABLE returned an 
error). [RESOLVED: 0012, 0022]

RN17: Implemented CREATE TABLE (...) auto_increment=<value>;

------- 0.9.51 - 2006-07-06

RN16: Fixed crash which could occur when creating the first table in a 
database (bug reported by Hakan).

------- 0.9.5 - 2006-07-03

RN15: This version concludes the re-structuring of the PBXT 
implementation. I have made a number of major changes, including:

- All files except the transaction logs are now associated with a 
particular table. All table related files begin with the name of the 
table. The extension indicates the function.

- I have merged the handle and the fixed length row data for 
performance reasons.

- Only the variable size component of a row is stored in the data log 
files. As a result the data logs can now be considered as a type of 
"overflow" area.

- Memory mapped files are no longer used because it is not possible to 
flush changes to the disk.

RN14: File names have the following forms:

[table-name]-[table-id].xtr - These files contains the table row 
pointers. Each row pointer occupies 8 bytes and refers to a list of 
records. The file name also contains the table ID. This is a unique 
number which is used internally by XT to identify the table.

[table-name].xtd - This file contains the fixed length data of a table. 
Each data item includes a handle and a record. The handle references a 
record in the data log file if the table contains variable length 
records.

[table-name].xti - This file contains the index data of the table.

[table-name]-[log-id].xtl - This is a data log file. It contains the 
variable length data of the table. A table may have any number of data 
log files, each with a unique ID.

xtlog-[log-id].xt - These files are the transaction logs. Log entries 
that specify updates reference a data file record. Each active thread 
has its own transaction log in order to avoid contension.

RN13: Fixed the bug "Hang on DROP DATABASE". [RESOLVED: 0016]

RN12: PBXT currently only supports the "Serializable" transaction 
isolation level. This is the highest isolation level possible and 
includes the "repeatable-read" functionality [RESOLVED: 0015]. This is 
implemented by giving every transaction a snapshot of the database at 
the point when the transaction is started.

If the transaction tries to update a record that was updated by some 
other transaction after the snapshot was taken, a locked error is 
returned. A deadlock can occur if 2 transactions update the same record 
in a different order. PBXT can detect all deadlocks.

RN11: I have implemented write buffering on the table data files. 
[RESOLVED: 0013]

RN10: The unique constraint (UNIQUE INDEX/PRIMARY KEY) is now checked 
correctly. [RESOLVED: 0008]

RN9: I have implemented a conventional B-tree algorithm for the indices 
(instead of the Lehman and Yoa B*-link tree). Although this reduces 
concurrency it improves the performance of queries significantly 
because of the simplicity of the algorithm. Deletion is also 
implemented in a very simple manner. [RESOLVED: 0007]

RN8: PBXT now has only 2 caches [RESOLVED: 0006]:

The Index Cache (pbxt_index_cache_size): This is the amount of memory 
the PBXT storage engine uses to cache index data and row pointers. This 
is all the data in the files with the extensions '.xti' and '.xtr'. 
This cache is managed in blocks of 2K.

The Record Cache (pbxt_record_cache_size): This is the amount of memory 
the PBXT storage engine uses to cache table row data (handles and 
records). This is all the data in the files with the extension '.xtd'.

The size of the caches are determined by the values of the system 
variables pbxt_index_cache_size and pbxt_row_cache_size. By default 
these values are set to 32MB.

RN7: Auto-increment is now implemented in memory. This is done by doing 
a MAX() select when a table is first opened to get the high value. 
After that, then high value is incremented in memory on INSERT. On 
UPDATE (or INSERT) the value in memory is adjusted if necessary. This 
method also makes it possible for rows to be inserted simultaneously on 
the same table. [RESOLVED: 0005, 0014]

RN6: ./run-all-tests --create-options=TYPE=PBXT succeeds. [RESOLVED: 
0004]

RN5: Using sql-bench and my own Java based test I have confirmed that 
PBXT behaves correctly during multi-threaded access. [PARTIALY 
RESOLVED: 0002]

RN4: Load/Stability test. Using sql-bench I have tested PBXT under load 
over a long period of time. [PARTIALY RESOLVED: 0001]

------- 0.9.2 - 2006-04-01

RN3: Fixed a bug that cause the error "-6: Handle is out of range: 
[0:0]".

RN2: Implemented SET, ENUM and YEAR data types.

RN1: Fixed a bug in the error reporting when a table is created with a 
datatype that is not supported. [RESOLVED: 0011]

Thread
PBXT version 0.9.6 has been releasedPaul McCullagh8 Aug