Database & Cache (May 27)

Folks are still discussing exceptions on the Qt developer mailing lists. The new accounting software uses them to show useful error messages and prevent crashes. Ditto for the current Goldenseal.

Right now, exceptions don’t work properly on newer Macs with ARM chips. However, it’s looking better for a possible fix. The details will take a while to figure out.

One issue that came up in the talk is how to terminate cleanly: especially if Apple’s code causes the ARM problem. That probably would be impossible to work around or fix. It’d be like trying to nudge an ocean liner.

Nobody wants their data to be corrupted if their accounting program suddenly quits. Fortunately, we remove that risk for all but the rarest of conditions.

As you fill in fields for a record, it’s only stored in the GUI objects on the screen. Hit Save, and the data moves to a temporary record, stored in RAM. The app then posts changes to other records: anywhere from a few to 30+. Those also sit temporarily in RAM.

After posting is finished, all data writes to disk at the same time. It’s called a commit. The process only takes milliseconds on an SSD, or a fractional second with a hard drive. Modern hardware is great! In the early days of Goldenseal, commits took several seconds.

The only way to screw up the database is to exit in the middle of the commit process. The code there is very careful, so the only real risk is a grid failure or cat-power-plug-swat at the exact wrong time. Everything stays in sync if there’s an exit, anytime else. Unexpected program termination does lose unsaved changes, but what remains is fine.

To manage temporary records, we use a cache. It’s a list of all records in temporary storage, with a bit about their status. How long to keep them there is a design decision. If the cache gets too big, you may run out of RAM memory. Dump records too quickly, and things slow down. Disk access is much, much slower than RAM, so the cache tries to guess at what might be used again soon.

During testing, we’ve seen a few crashes from records that were deleted, but stayed in the cache. Such dangling pointers are impossible to sanity-check, and hard to fix. We need to track down where/when they were deleted, and why the cache wasn’t told about it. The crashes are rare and random, which makes them even harder to diagnose. Problems are so much easier when they happen every time you do something.

We just discovered that deleted bank transactions don’t clean up properly. It’s because we switched where the list of currently-found records is stored. Fixing that may also fix the crashes, but we’ll see.

Meanwhile, breakdowns work pretty well for Material Purchases and Sales. We’ve moved on to Estimates. Those and payroll are by far the most complicated parts of our software.

Dennis Kolva
Programming Director
TurtleSoft.com

 

Author: Dennis Kolva

Programming Director for Turtle Creek Software. Design & planning of accounting and estimating software.