Goldenseal Pro Progress Report #6 (June 24)

The new database code in our Goldenseal Pro software now works.  A few details are still not finished, but it looks up records from the company file, and saves them.  That’s 95% of what a database needs to do.  The indexing design is much simpler than the old NeoAccess code, so it will be easier to debug and maintain, and probably will run faster.  There were a few serious design flaws in the old code, and they are now gone.

Our contractor is still working on the GUI (graphic user interface), but the Mac code now shows a data entry window, and the Windows code is making progress.  It requires a lot of prep work to get that first window showing, and we hope that progress will be faster after the basics are finished.

The next thing on deck for our staff is to write a converter that will read existing Goldenseal files, and convert them to the new file format.  We will still use the older NeoAccess code to read files created in version 4.93 and earlier.  However, we are eager to get rid of NeoAccess entirely, so we probably will only support the conversion process for a year or so.  Latecomers will need to do a two-step conversion.

Dennis Kolva
Programming Director
Turtle Creek Software

Farewell to NeoAccess

When we started work on the Goldenseal software back in 1993 or so, we licensed an object database library called NeoAccess to manage how records are stored on the hard drive.  It performed well, but as we neared completion we started to discover serious bugs.  Unfortunately the Neo developers disappeared, and never fixed them.  It was too late to start over, and there was no good alternative to replace it.

So, we rewrote their index class in 1999, and fixed an awful bug that corrupted the entire file if you deleted more than 16 records.  In 2002 we did another rewrite, and added more changes to make it more reliable.

Since then we have gradually improved or replaced many parts of NeoAccess.  Unfortunately, the main part of the code was just too complicated to change.  It worked, but was extremely complicated, and almost impossible to debug or improve.

A few weeks ago we started on a major refactoring of all the NeoAccess code.  We stripped out about half of it, but when we got to the hard parts we decided it would be easier to just start over, rather than fix the rest.

We looked into SQL and other relational database libraries, but soon decided it will be better to stick with an object database.  We checked a few open-source object database libraries, but none are very solid.

So, last week we started a complete redesign and rewrite, based on the parts of NeoAccess that worked well.  It only took a week to create the new database classes, and we now have a brand-new, working database.  It still needs a few weeks to polish and debug, but it will end up being much more reliable than what we had before.

The new database allows 64-bit addresses, so the max file size increases from 4 gigabytes to 9 exabytes.  Big number names jump by 1000x at a time, and go kilo/mega/giga/tera/peta/exa.  That means it will support yuuuge files.  A few users have pasted in many pictures and come close to the 4-gig limit, but now that won’t be a problem.

We’ve redesigned the indexes that keep track of where every record is located. That info is now stored in two different places, so we can recover more easily if data is corrupted.  NeoAccess scattered the indexes all over the file, but we will keep them near the beginning of the file so they are less likely to be lost or corrupted.

The new code also uses much larger memory blocks. Modern systems can move one 64K chunk much faster than 64 1Ks. That means the new code will be quicker, and also easier to debug.

Generally, the new code is simpler, more compact, and much easier to understand.  C++ has come a long ways, and we were often able to replace 20 lines of code with just 1 or 2.

One thing that will help us in the redesign is to see some basic file data from people who have been using Goldenseal for a long time.  If you have a few minutes, please do the following:

1   Open your company file.
2   Choose Security from the View menu, and choose Database Report from the submenu.
3  Click the Save as Text button, and save a file.
4   Attach the file to an email and send it to support@turtlesoft.com
5   The data is anonymous. It only tells us the number of records in each class, which will help us design the best index sizes for typical users.

Thanks,

Dennis Kolva
Programming Director
Turtle Creek Software