When we first started work on Goldenseal accounting software, 1 megabyte of RAM was the norm. Apps could only use a couple 100K of that. A few bytes here and there was a big deal, so programmers used the smallest possible memory sizes. Colors came from a palette with 256 choices (1 byte). Fonts used a 2 byte FontID (max 16,000 fonts). Pictures and other resources also used 2 byte IDs.
Larger data was packed tight to conserve space. Records inside a Goldenseal file have data that’s as dense as possible. Our old layout formats are similar.
Binary data like that is touchy. When reading or writing records, being off by just one byte will wreck havoc.
These days, memory is cheap, whether in RAM or storage. There is no reason to scrimp on bytes. Since text is easier to read, data storage and programming often use that instead of binary bytes. Modern system resources have names and file paths, rather than ID numbers. Fonts have names. Lists often use “hash tables”, based on text instead of a number.
A few years ago we shifted most of our resources to text files. The code finds them with a file name rather than an ID number. We also are half-done converting screen layouts from binary to text.
In theory, we could do the same with estimating and accounting records. The company file could be one huge text file. Switching over would not be hard: we already have code to export and import from text. It’s how we saved sample data during early development. Plain text sure would be easier to debug.
Unfortunately, text would also be easy for anyone else to debug. Just open the file with a text editor, search for your name, give yourself a raise, and save changes. Any other fraud would be just as easy. The current binary format is not invincible, but it takes much more effort to hack it. We haven’t heard of any cases where that happened.
On top of that, text is bigger. It’s not bullet-proof, either. Lose a tab character separating values (or add a new one), and the data after it still gets trashed.
To keep binary data reliable in Goldenseal, we’ve added all sorts of fail-safes over the years. It makes programming easier now. If we make a change and forget something, it soon gives an error message that tells us exactly which code line had the problem. Likewise, if a file has damaged data, it will complain. And there are extra markers inside the file, now, to help with forensics.
Goldenseal has run the TurtleSoft business for 22 years. In that time, our company file developed a couple of data errors. There’s a corrupted NeoAccess index that lost access to a block of sales breakdowns. It happened about 15 years ago. The converter found most of them via DB_FileManager, but one record was corrupted, and a few are gone forever. There also was a Contact Log record with a bad length marker in some text. That zapped all the data after the error. It was a 1993 phone call, imported from FileMaker in 1999 or 2000. We were able to fix that record by re-saving it within Goldenseal before the import.
As our staff gets closer to completion, we’ll post a beta version so users can convert their current Goldenseal files. If the import reports problems, there will be time to fix them.
It will be interesting to see how well other files have done. Hard drives and SSDs have grown more reliable over the years, so most “bit rot” probably will be in older records.
Dennis Kolva
Programming Director
TurtleSoft.com
