When we started work on the Goldenseal software back in 1993 or so, we licensed an object database library called NeoAccess to manage how records are stored on the hard drive. It performed well, but as we neared completion we started to discover serious bugs. Unfortunately the Neo developers disappeared, and never fixed them. It was too late to start over, and there was no good alternative to replace it.
So, we rewrote their index class in 1999, and fixed an awful bug that corrupted the entire file if you deleted more than 16 records. In 2002 we did another rewrite, and added more changes to make it more reliable.
Since then we have gradually improved or replaced many parts of NeoAccess. Unfortunately, the main part of the code was just too complicated to change. It worked, but was extremely complicated, and almost impossible to debug or improve.
A few weeks ago we started on a major refactoring of all the NeoAccess code. We stripped out about half of it, but when we got to the hard parts we decided it would be easier to just start over, rather than fix the rest.
We looked into SQL and other relational database libraries, but soon decided it will be better to stick with an object database. We checked a few open-source object database libraries, but none are very solid.
So, last week we started a complete redesign and rewrite, based on the parts of NeoAccess that worked well. It only took a week to create the new database classes, and we now have a brand-new, working database. It still needs a few weeks to polish and debug, but it will end up being much more reliable than what we had before.
The new database allows 64-bit addresses, so the max file size increases from 4 gigabytes to 9 exabytes. Big number names jump by 1000x at a time, and go kilo/mega/giga/tera/peta/exa. That means it will support yuuuge files. A few users have pasted in many pictures and come close to the 4-gig limit, but now that won’t be a problem.
We’ve redesigned the indexes that keep track of where every record is located. That info is now stored in two different places, so we can recover more easily if data is corrupted. NeoAccess scattered the indexes all over the file, but we will keep them near the beginning of the file so they are less likely to be lost or corrupted.
The new code also uses much larger memory blocks. Modern systems can move one 64K chunk much faster than 64 1Ks. That means the new code will be quicker, and also easier to debug.
Generally, the new code is simpler, more compact, and much easier to understand. C++ has come a long ways, and we were often able to replace 20 lines of code with just 1 or 2.
One thing that will help us in the redesign is to see some basic file data from people who have been using Goldenseal for a long time. If you have a few minutes, please do the following:
1 Open your company file.
2 Choose Security from the View menu, and choose Database Report from the submenu.
3 Click the Save as Text button, and save a file.
4 Attach the file to an email and send it to support@turtlesoft.com
5 The data is anonymous. It only tells us the number of records in each class, which will help us design the best index sizes for typical users.
Thanks,
Dennis Kolva
Programming Director
Turtle Creek Software
