We are still fine-tuning our basic database code. It is a gradual process since it requires testing, and making design decisions. This is pretty much the last programming we need to do, outside of the GUI (graphic user interface). Our contractor is making slow progress on that.
We use a B-tree to store the disk location of every record. It is a popular design that is similar to the older NeoAccess code. However, we make each node much wider, thanks to the generous amounts of RAM available on modern hardware. It’s also faster that way: these days, a 64K read takes much less time than 16 4K reads.
Neo had a complicated system to keep track of empty spaces in the database (necessary to keep the file from becoming huge). It was buggy, so in 2003 we replaced it with our own File Manager. The new system performed well but took up a big chunk of memory, which sometimes caused quits or file corruption when there was not enough RAM to load it. In theory we could have rebuilt the FileManager from the NeoAccess records, so corrupted files could be fixed. Unfortunately, the Neo code was so complicated that we never could make that happen.
In Goldenseal Pro we now use multiple File Managers, so each keeps track of records in just one sector of the file (32,000 records). They have a relatively small memory footprint, so they’ll be quicker and safer.
In the new database, the location of every record is stored in two different places, and now it uses code that is easy to understand. That means we can cross-validate them, and fix any errors. Our own company file actually has a few corrupted records from the early 2000s, so it will be the perfect test platform for developing a self-repairing database.
The classic B-tree requires ‘branch balancing’ code to keep it in shape, and part of the complexity in NeoAccess was that it kept moving things around to balance the tree. We add ID numbers sequentially, and most users don’t delete many records. Our initial design takes advantage of those conditions, but we are still researching other ways we can make it better.
Dennis Kolva
Programming Director
Turtle Creek Software
