Database Reliability (Nov 20)

TurtleSoft started out in 1987 with MacNail, a set of Excel templates for construction estimating. Later we added accounting and scheduling features. The MacNail software was extremely popular, and it still has a few hundred die-hard users. Unfortunately, we could only do so much with Excel macros and spreadsheets. MacNail was complicated to use, and difficult to support.

In 1989 we released a second estimating program, built atop Apple’s HyperCard. It had a much nicer human interface, but severe limitations on the back end. Among other problems, HyperCard introduced us to “race conditions”. That’s where the app has intermittent bugs, depending on which message path completes first (unpredictably). They are extremely hard to track down.

In the early 90s, Microsoft replaced the Excel macro language with Visual Basic, and Apple halted development on HyperCard. It was clearly time for us to move on.

For the next software generation, we looked at many database programs to build upon: FileMaker, Omnis, Double Helix, Sirius Developer, Prograph. Each was better in some ways than what we had already, but in some ways each was worse. Rather than build a mediocre app on top of some other software, we finally decided to write a “real” application in C++. Coding from the ground up takes longer, but allows full control. As it happens, it also prevented us from riding a platform to its death, since all but FileMaker are now long gone.

We were too inexperienced to write our own database code, so we tested several C++ database libraries, and settled on one called NeoAccess. It was quite popular in the mid 1990s. The database performed well, and wasn’t too difficult to build upon. Unfortunately, when Goldenseal was almost completed, we discovered that NeoAccess had serious bugs. Files eventually became corrupted, and all data was lost. The developers at NeoLogic never fixed the bugs, and eventually stopped answering emails. 

AOL 4.0 and Netscape Communicator also used NeoAccess. AOL abandoned it in their version 5.0. Netscape retired their software prematurely (and was soon swallowed by AOL). Dozens of other companies went through similar drama, thanks to an unreliable database engine.

We also considered abandoning NeoAccess, but the alternatives were just as buggy. So we put about 2 programmer-years into rewriting their code to make it more reliable. We learned a lot in the process, which is why we decided to build our own database for Goldenseal Pro. You might say that bugs that you create yourself are much easier to fix, than ones made by other people.

The first step in writing reliable software is to make the code maintainable. That means simple logic, clear comments, and easily readable code that anyone can understand. NeoAccess was completely the opposite, and that was its primary flaw. To fix it, we stepped through their code hundreds of times, and gradually rewrote it to make more sense. A few of the bugs were simple logic errors that finally stood out when the code was less cryptic. Some of the bugs were in code that was so convoluted that we just gave up, and rewrote it entirely.

BTW most open-source code is equally unreadable. Whenever you hear in the news about a major security breach, the root cause was probably some important open-source code that nobody understands. If other programmers can’t read it, then they can’t fix it or improve it.

There are more specific methods we are using to make the database more reliable in Goldenseal Pro. NeoAccess taught us a lot about database design, both from its good parts and its bad. I’ll cover them in more detail, in a future post.

Meanwhile, the past couple weeks we have been working on the new database code for Goldenseal Pro. It is mostly working, but probably still needs another week to finish.

Dennis Kolva
Programming Director


Author: Dennis Kolva

Programming Director for Turtle Creek Software. Design & planning of accounting and estimating software.