Database & Cache (May 27)

Folks are still discussing exceptions on the Qt developer mailing lists. The new accounting software uses them to show useful error messages and prevent crashes. Ditto for the current Goldenseal.

Right now, exceptions don’t work properly on newer Macs with ARM chips. However, it’s looking better for a possible fix. The details will take a while to figure out.

One issue that came up in the talk is how to terminate cleanly: especially if Apple’s code causes the ARM problem. That probably would be impossible to work around or fix. It’d be like trying to nudge an ocean liner.

Nobody wants their data to be corrupted if their accounting program suddenly quits. Fortunately, we remove that risk for all but the rarest of conditions.

As you fill in fields for a record, it’s only stored in the GUI objects on the screen. Hit Save, and the data moves to a temporary record, stored in RAM. The app then posts changes to other records: anywhere from a few to 30+. Those also sit temporarily in RAM.

After posting is finished, all data writes to disk at the same time. It’s called a commit. The process only takes milliseconds on an SSD, or a fractional second with a hard drive. Modern hardware is great! In the early days of Goldenseal, commits took several seconds.

The only way to screw up the database is to exit in the middle of the commit process. The code there is very careful, so the only real risk is a grid failure or cat-power-plug-swat at the exact wrong time. Everything stays in sync if there’s an exit, anytime else. Unexpected program termination does lose unsaved changes, but what remains is fine.

To manage temporary records, we use a cache. It’s a list of all records in temporary storage, with a bit about their status. How long to keep them there is a design decision. If the cache gets too big, you may run out of RAM memory. Dump records too quickly, and things slow down. Disk access is much, much slower than RAM, so the cache tries to guess at what might be used again soon.

During testing, we’ve seen a few crashes from records that were deleted, but stayed in the cache. Such dangling pointers are impossible to sanity-check, and hard to fix. We need to track down where/when they were deleted, and why the cache wasn’t told about it. The crashes are rare and random, which makes them even harder to diagnose. Problems are so much easier when they happen every time you do something.

We just discovered that deleted bank transactions don’t clean up properly. It’s because we switched where the list of currently-found records is stored. Fixing that may also fix the crashes, but we’ll see.

Meanwhile, breakdowns work pretty well for Material Purchases and Sales. We’ve moved on to Estimates. Those and payroll are by far the most complicated parts of our software.

Dennis Kolva
Programming Director
TurtleSoft.com

 

Breakdown Tables (May 15)

Progress on the new accounting software accelerated this past week, thanks to a rainy spell. That will continue, since most of the garden is planted now. It has 6 different varieties of broccoli, to see which does best in this new location. Also a dozen types of tomatoes, scattered all over the property. If they all do well there will be way too much produce this summer. Maybe TurtleSoft should include a free heirloom tomato with every purchase!

Plenty of bugs are still turning up in the new estimating/accounting app, especially in breakdown tables. The basics work, but there are many small details that still need work. Breakdowns also took a few years to get right in Goldenseal. They are small lingering bugs, even after twenty years.

The new app still uses the original tables from Goldenseal. They no longer draw on the screen or interact with mouse and keyboard: that is handled with a Qt table and cell fields. But they contain all the code that does math or links cells (e.g. category and subcategory). Getting the two tables classes to talk with each other can sometimes be tricky, but it’s safer and easier than a total rewrite.

We will get breakdowns in Material Purchases and Sales to work perfectly, then move on to Estimates. Those have by far the most quirks. After that, I think the new app will be ready for its first early release.

It will be easy to test Estimates, since I plan to build a tool shed soon. The last time I did any new exterior framing was 30 years ago. Not much swinging of hammer these days, it’ll all be screws and lithium.

Some day we need to update the stock assemblies in the construction starter files. Construction has changed a lot since those were made.

Dennis Kolva
Programming Director
TurtleSoft.com

 

Qt Bugs (May 6)

TurtleSoft uses a framework called Qt to build the new accounting software. It’s pretty good, though it does have plenty of quirks. We’ve worked around most of them, but at least two are unsolved despite our efforts.

The first is just ugly: if you use the tab key to enter text into multiple fields, the flashing vertical edit bars won’t go away. There can be 6 or 8 of them blinking. So far we’ve tried a dozen different ways to remove them, but no luck. Last week we decided to file a Qt bug report on it, but one already was in the system. The ticket was from us, almost a year ago. A Qt person rated it as “somewhat important”. Otherwise there has been no action.

The other bug is more serious, related to error handling.

Goldenseal stays reliable and keeps improving because we “sanity check” anything that might crash. If the test fails, it shows a message with source code file and line number. Getting that info saves 90% of the work to fix bugs.

After the error message, Goldenseal throws a C++ exception. It’s a “get me out of here” that evades the code that will crash. Instead, the exception moves outward until it hits a catch. If no catch, the app terminates.

We catch almost all exceptions in the event loop. The app goes back to waiting for the next action from keyboard or mouse, just like normal. 99% of the time, it means you can keep working just fine. That one thing with the bug still won’t work, but everything else is OK.

TurtleSoft has built our estimating/accounting app atop five other frameworks. All of them had some way to catch exceptions in the event loop. Qt does it for Windows and for Intel Macs, but the catch does not work on newer ARM Macs with a M1/M2/M3 chip. Instead, the app exits abruptly. Any unsaved changes are lost. It’s nasty for users, and also for bug fixers.

We posted the problem on the Qt forum, but the main Qt spokesman there said if exceptions work on Intel it’s just an accident. They are not supposed to work at all. We put in a bug report, but it was closed as “invalid” the next day. Apparently Qt decided long ago to not handle exceptions in most of their code. Period.

Sadly, the new app will be less reliable than we would like. The response of the Qt team is also very discouraging. Bugs do happen, but it’s worse when the problem will never be fixed.

Alas.

Dennis Kolva
Programming Director
TurtleSoft.com