Category: Goldenseal news

A while back I mentioned how easy it was to set up the Goldenseal Pro outline view. It replaces the menus at the top, and does an excellent job of organizing things. I assumed that NSOutlineView probably was so reliable because it also runs the Macintosh Finder.

Unfortunately, I spoke too soon about that. This week we opened a few items, and the outline view stopped working. Clicking did nothing.

Usually our staff is very methodical. We add new code, make sure it works, and then commit it to the source control system. Nearly always, the process works well. Stuff improves, step by step. Unfortunately, a few times a year we miss something subtle, or change something with unintended consequences. It becomes a bug that rears its head weeks or months later. This is one of those times.

The first thing we did was check the weekly full-project backups, to see when it broke. Recent backups also had the problem. The June 2 backup worked OK. The June 9 backup didn’t. Most of the changes that week were related to bank transactions, and unlikely to affect the outline view. That week we did add a new NSSplitView to make the window resizable. So we removed that, but it didn’t help.

Next step was to rebuild from source control, to narrow it down to a shorter time range. June 7 was OK. June 8 wasn’t. Most of the changes that day were just accounting stuff, but we also started to add “weak” references. It’s an Apple thing that prevents memory leaks. That seemed like the smoking gun.

To test it, we set some breakpoints. Sure enough, the outline table was deleted randomly, after running a while. In earlier builds it stuck around. It didn’t work because it wasn’t there.

Apples ARC (automatic reference counting) is hard to debug. In C++ you can set a breakpoint right where an object is deleted, to see what killed it. Objective-C with ARC does the deletion some random time later. That makes it very difficult to find the cause of death.

Our staff spent a couple days stepping through code and reading more about ARC. It’s more complicated than we first thought. One website describes ARC as “kind of like a Japanese B-horror movie”. That seems accurate. We spent several weeks in April and May, struggling with a similar problem in the tab view.

Today we got too frustrated, and gave up for a while. It’s time to get back to more productive programming.

The Apple developer list is often helpful for odd problems, so we just posted a message there. This is the kind of enigma that may be solved in a dream at 3 AM, days or weeks later. Worst case, we get one support incident with Apple per year. It may be time to use it.

Dennis Kolva
Programming Director
TurtleSoft.com

 

Goldenseal stores many different types of business data. There are customers, suppliers, employees, projects, purchases, estimates, sales, payroll and bank transactions. Then another 40 or 50 special items like change orders and equipment hours. Then another 70-odd small details like sales tax rates and payroll tax tables.

Navigating around in that sea of information is challenging. Goldenseal makes it easier by providing may ways to see linked info via a click or menu choice.  For example, a material purchase has quick links to the supplier account and job account, plus the bank transaction that paid for it. Project accounts have buttons to view job costs, contacts and other details. Pretty much everything has a quick link to something else that’s related.

The current Goldenseal version has separate windows for each type of data. When you click a link, it opens another window for it. Most of the windows are about half the size of a laptop screen, or 1/6 of a big desktop screen. It’s easy to end up with 20 or 30 Goldenseal windows scattered about. Not the ideal way to manage a virtual desktop. It’s kinda like strewing papers across your actual desktop.

Goldenseal Pro uses an outline and tabs to organize things into a single window. You also can open separate windows just like the old style, but you probably don’t need to. The new interface is the virtual equivalent of putting your paperwork into folders and binders. Tidier. Quicker access.

With the new interface, the question is: how best to show linked records. There are five choices:

* Open a new window, just like the current version.
* Open a smaller, simpler window, with just one record and no fancy controls.
* Open a temporary panel that drops down in front of the current record.
* Add a tab for the record, and bring it to the front.
* Add a tab, and leave it somewhere behind the current record.

The past couple weeks, our staff has been experimenting with all of them. We are still deciding which are the most useful and the least annoying.

Actually, there’s a 6th choice. For a while, we showed breakdowns in a slide-out drawer on the right side. That would also be a great way to view linked records. Related and connected. Drawers used to be a standard Apple interface feature, but they are deprecated. Goldenseal Pro used them for a while despite that, but ran into bugs we couldn’t fix. An attached window may still be the ideal solution for linked records (and breakdowns), but it won’t happen for Goldenseal Pro 1.0. It will be complicated to program it from scratch, and the deadline for 64-bit conversions is coming soon.

This sort of interface design needs a few weeks of real-world testing and usage. Link display may be something that deserves a Preferences option, or a way to control how it looks via a modifier key.

Dennis Kolva
Programming Director
TurtleSoft.com

 

 

Progress on Goldenseal for Mac has been slow but steady. Most of the basics are finished, but there are many small details still left to test and fix. We are testing it with TurtleSoft data, but the software is not good enough for daily use, yet.

Our staff is also still working on the shopping cart for SmartKnives. We bought many vintage Swiss Army knives and Leatherman tools from airport confiscations in the years after 9/11. Now it’s time to sell them to collectors. Setting up the e-commerce has taken longer than expected. Unfortunately, it will continue to consume time between now and year end.

In the short run, it’s going to delay completion on the first release of Goldenseal Pro for Mac. However, programming is hard, and very few people can keep it up full-time. Taking breaks to do inventory counts and photography is a good break. A chance for ideas to percolate.

In the long run, the extra income will allow us to pay subcontractors for the Windows version and  multi-user code. Our staff has less experience in those areas, and there are plenty of contractors able to do the work. It will take time to ramp up software sales, and this is something to fill in the gaps.

We came close to launching SmartKnives in 2015. The plan then was to subcontract the entire upgrade, so it was something to do while we waited. That was before we discovered how few people actually know how to write 64-bit software for Macintosh. It was long before we gave up on contractors, and tackled the Pro update with our own staff. That’s when we discovered the reasons why competent Cocoa developers are so rare.

In the mid 1990s we talked with some Venture Capital folks, considering a quicker and more massive launch for the original Goldenseal. One of their requirements was sufficient “barriers to entry”. Essentially, it means you need patents or some other protection to make sure there won’t be too much competition. They saw QuickBooks looming, and thought it would eat our lunch and only leave crumbs. It did.

We have struggled with Cocoa the past few years. But what keeps us going is the fact that it’s difficult for everyone. Cocoa is just about the only way to write Mac software that will run on Catalina. QT is an alternative, but that also is no easy road.

When we finally finish this thing, it will put us behind that barrier to entry.

Dennis Kolva
Programming Director
TurtleSoft.com

 

Our staff is back at work on Goldenseal Pro. The first thing we faced was a new problem while editing list items. The OK and Cancel buttons used to work fine, but suddenly they didn’t.  It turned into a journey down the rabbit hole of Cocoa memory management.

Memory is a general problem for all programmers. Apps need to use many small chunks of RAM: thousands, sometimes millions of them. If you don’t delete them when done, it’s a memory leak. Leaks cause the app to gradually use up more and more memory space. Eventually it will get slow. Pile up enough leaks, and it will crash. Web browsers often do that.

Preventing leaks is not easy. Delete something too early, and the program will fail when you access a long-gone bit of memory. Usually the app crashes from that, but sometimes it reads corrupted data and causes a hard-to-fix bug. Wait too long, and there’s no way to go back to tidy up.

The C and C++ programming languages are considered “hard” because they require programmers to manage raw memory.  On the other hand, most newer programming languages do the memory management for you. Usually that involves periodic garbage collection: automatically deleting objects from memory, when they are no longer needed.

Apple’s Cocoa framework started out by requiring programmers to manage their own memory, just like C++. It then added its own memory manager in 2010. Rather than garbage collection, Cocoa uses something called ARC (automatic reference counting).

We turned on ARC when we first started programming Goldenseal Pro for Mac. Sometime in the past year or two it was accidentally turned it off, possibly during an Xcode update. The setting is buried in a list of 200+ other build settings, so it’s easy to overlook.

Removing ARC caused no noticeable problems, since the app was never used long enough to show leaks. Finally we noticed the setting this March, and turned ARC back on. That broke the buttons, and a few other things.

Our staff has worked with ARC long enough now that I feel confident in saying it’s the worst of both worlds. Yet another complaint about Cocoa.

It’s possible to write code in Python, Java, Javascript or other garbage-collected languages, and simply not worry about memory. It’s all taken care of for you. During run time there may be weird multi-second delays as the app cleans up memory, but it’s a small price to pay for convenience.

It’s also possible to use C or C++, and manage memory reliably. Many tricks exist to make it easier. We used them for the original Goldenseal, and it almost never leaks or crashes.  Recent versions of C++ provide even better tools for managing memory. We use them in any new code we write.

With its in-between approach, ARC is definitely not mindless. It makes you decide whether to make connections “strong” or “weak”. It sometimes deletes things unexpectedly. It also is hard to debug. There are tricks to help track down memory problems, but nothing as simple as in C++.

Despite all that, the buttons are working again. Then for a while, the app crashed when editing a second list item. That is fixed too. Both were general “object lifetime” problems caused by ARC. They also applied to code in other places, so all those are also fixed now. Sometimes it’s one step backwards, two steps forward.

Dennis Kolva
Programming Director
TurtleSoft.com

 

 

Our staff took a short break from Goldenseal Pro, the past couple weeks. Instead, we’ve been finishing up the shopping cart for SmartKnives.com.  It’s almost done. A contractor did most of the work, but we needed to import inventory data, and tweak the appearance.

As app developers, it has been interesting to visit the world of e-commerce and website development.  Things are very different there.

When building an app like Goldenseal Pro, programmers spend most of their time in source code. For us it’s a couple thousand text files, mostly in the C++ programming language. Some of the files are 25 years old. We make life easier by writing clean code, and adding comments to explain what’s going on. When something doesn’t work, there’s a debugger that shows what is happening, line by line. Most of the code is very well polished.

App developers also use libraries, usually written by some big entity like Microsoft or Apple. Some libraries show their source code, and some don’t. It’s always harder to work with code written by other people. It’s even harder when the source code isn’t visible. But, lots of people use the same libraries. For most problems, there usually is an answer on Stack Overflow or a developer forum.

Web development is much crazier.

It’s possible to create simple web pages by just writing some html and css text files. But, to accomplish anything complicated, programmers must use layers upon layers upon layers of other people’s code. Most of it is open-source. Most is in a state of utter chaos.

App developers occasionally make hacks- an ugly bit of code that does the job. It’s the duct tape of the programming world. Eventually we refactor them into good, solid code.

E-commerce code is almost entirely hacks. It’s an empire built entirely of duct tape. No debuggers. Lots of poorly written code that is rarely fixed. Lots of guessing.

It will be good to get back to the comfort zone of Goldenseal Pro source code. We’ll probably appreciate the Cocoa libraries more.

Dennis Kolva
Programming Director
TurtleSoft.com

When we first designed Goldenseal accounting software, banking was heavily regulated. There were clear distinctions between different types of banks, so it made sense to have different types of bank accounts. Then, deregulation and the Internet gradually made it all blurry.

Goldenseal Pro uses a new file format. When you open an old-format file, it automatically converts the data. The process is an opportunity to do other conversions, so it also condenses the original 7 types of bank accounts down to just 3. Investment accounts are special, because they can be job costed. Ditto for Escrows, since they aren’t assets. Everything else is now just a plain old Bank account. We added a popup menu that you can still use to distinguish the old types (plus new banking types that have arisen since then).

Up until now, our staff has done most testing on expense transactions. They are relatively simple, which made them a good starting point. Bank transactions are more complicated. The biggest problem is that every account gets a separate window. That is different from everything else, which just has one class per window. There is a lot of interface code that does something special for bank transactions. It’s working now, but it took a few days to find all the quirks. The old system to display bank transactions was rather clunky, but it’s much better now.

We also revised the indexes so the Reconcile command is much faster. When we used Reconcile for our own bank statements, we discovered that start balances were slightly off. That tracked down to a few dud transactions in the early Aughties. The original Goldenseal ignored them, but Goldenseal Pro didn’t. The change is annoying because it throws off the start balance for all later periods. We’re still deciding how best to handle old, corrupted data like that. Turtlesoft sometimes used beta versions for daily data entry, so it’s possible that other business users won’t run into the same problems.

Bank transactions are probably the third-most difficult thing in Goldenseal. Estimates are more complicated, and Payroll is more complicated yet. We are getting close to fixing all the banking bugs. Then it’s on to the tougher stuff.

Dennis Kolva
Programming Director
TurtleSoft.com

Besides the crashing bug that we fixed recently, there has also been a long-time problem with window layouts. They functioned, but didn’t resize in a pretty way.

Apple has at least 4 different methods to position things on the screen. They started with plain old x and y coordinates, plus a springs and struts model to move things when you resize the window. Then they added constraints, a more complicated version of springs and struts. Finally they added anchors, yet another method for window setup. It’s a confusing jumble.

Last week we decided to finally get it right. But, for a while, everything we tried just made things worse. At one point, switching between tabs caused the whole window to jump around on the screen. Then we discovered setTranslatesAutoResizingMaskIntoContraints, a poorly-documented bit of magic that solves most conflicts between the different systems. After a bit more tweaking, the main window finally works like it should.

In the last post I compared Apple’s Cocoa library to Serius Developer, an unusable platform we tried briefly in the early 90s. Serius was close to fradulent, but I didn’t mean to imply that Cocoa is that way.

I think the main problem is that Apple has developed the Cocoa framework primarily to build their own apps.  Making it work well for other software companies is a relatively low priority. So, Cocoa is great for building apps that are similar to something from Apple. The more you diverge, the harder it gets.

Goldenseal Pro uses NSOutlineView to replace most of the current menu commands. It only took a week to set up, and it has worked great ever since. Apple uses that class for the Finder, so they must have really polished it. It was one of the first things we programmed 2 years ago, and the ease gave us a false sense of optimism.

Getting NSTabView to work properly was much harder. Apple uses it for Preferences, but our tabs are more complicated. Our staff has spent a couple months getting them right.

The NSTableView class that we use for breakdown tables has been a nightmare that consumed almost 3/4 of a year. Apple built a spreadsheet app, but after exploring its package I’m pretty sure that Numbers doesn’t use NSTableView.  With hindsight, we probably should have adapted our existing table code for breakdowns, and ignored NSTableView.

At any rate, our staff has fixed all the big problems. Goldenseal Pro is stable enough that we just started to use it to run our own business. There are plenty of small bugs still lurking, but they don’t take long to fix. Daily use is now daily testing. Eating our own dog food.

When the app stops being frustrating, we will release a public beta version. Best guess: in a month or so.

Dennis Kolva
Programming Director
TurtleSoft.com

 

Goldenseal Pro has a new, single-window appearance. To move between different functions, you click on on tabs rather than separate windows. We’ve used it enough now to really like the change.

Unfortunately, the tab view has had a serious bug: crashing after a few tab switches. It started about a year ago, when we updated to Apple’s newest memory management system. About once a month, we’ve spent a day or two trying to fix it, without success.

Our staff spent most of the past week attacking it again. We finally cracked it Wednesday, with help from folks on the Cocoa Developer Forum. The root cause was some unexpected behavior in Apple’s NSTabViewController class, complicated by the fact that we can’t see Cocoa source code.

We have been using Apple’s Cocoa framework for 2.5 years now. It is probably the most difficult programming environment our staff has ever used. Unfortunately, there are no viable alternatives for building software that runs on future versions of Macintosh OS. We are getting there, but it has been a real struggle.

As an attempt to quantify the difficulty of programming environments, I’ve decided to invent the Serius Scale. It’s a measure of programmer pain.

0 on the Serius Scale is an absolute joy for programmers. Productive days that create useful code, quickly. 10 on the scale is a tool that causes maximum frustration, and never results in anything useable. Right now, I’d rate Cocoa at about a 6, maybe a 7. Apple really could do better.

The scale is based on Serius Developer, a software development tool that came out in 1989. It had an amazing demo that included a built-in spreadsheet. We were desperate for a better platform for our construction estimating and accounting software, so we plunked down $2000 for it.

Unfortunately, we soon discovered that their spreadsheet tool was hard-coded to run the demo, and nothing else. Other features were similar. There was no way to get it to do what we wanted. Serius was a totally useless black box. Its developer later moved on to start a Ponzi-like Internet company, then got into space aliens. So, I’ll rate his software a 10 on the eponymous Serius Scale.

Before that, TurtleSoft started out in 1987 with simple construction estimating templates for Excel. They took less than a year to develop, and sold like hotcakes. That was a fantastic time to be in software.

I’d give early versions of Excel a 1 or 2 on the Serius scale. The macro language had its quirks, but it was pretty decent for its day. Things improved in Excel 2.2 and 3.0, but then went seriously downhill. Excel 4.0 added a bug that randomly changed our code. Then the macro language was replaced with Visual Basic. That’s why we were so desperate to find a better way to produce software.

Back then, we also shipped a few products based on Apple’s HyperCard. It was a fun app that could whip up something functional, very quickly. At the beginning, I probably would have given it a pain-free 0 or 1. Unfortunately, after using it to build real software, we discovered race conditions. HyperCard worked by passing messages around, and they weren’t always received in the same order. It made hard-to-find, intermittent bugs. Apple never fixed them. Instead, HyperCard languished, and was never updated for OS X.  I guess the net result is a high Serius number. It just never lived up to its potential.

After fiddling with FileMaker, Omnis, Double Helix and a half-dozen other platforms, we finally decided to use plain old C++ for Goldenseal. It was built upon the PowerPlant framework that came with Metrowerks CodeWarrior. Basic, but easy to understand. About a 2 on the Serius scale. Metrowerks was a small, competent company. Very pleasant to work with. Life was good again, for a while.

Unfortunately, Metrowerks was bought and strip-mined by Motorola. They sold off the Intel version of CodeWarrior a month before Apple switched from Motorola chips to Intel. A quick end for CodeWarrior and PowerPlant.

Unfortunately, this post has been using “unfortunately” a lot. We would like to spend our time writing code for estimating, accounting, payroll, and other functions that users need. But, looking back, at least 75% or 80% of our programmer days have been spent on OS and hardware issues. Too much time struggling with tools that are high on the Serius Scale.

The good news is, we are making progress. It’s hard, but not impossible.

Dennis Kolva
Programming Director
TurtleSoft.com

Our staff is well into the miscellaneous phase of the Goldenseal Pro project. That means tidying up small details until it’s all shiny.

First thing, we enhanced the database verification reports so they catch a few more potential errors. The reports will make the final testing go much faster. Rather than waiting around to see if file error messages pop up, we can run Verify File a few times a day. It will notice any problems right away.

It’s quite possible there won’t be any more database issues. The data conversion from version 4.96 to version 5.0 already futzes with many thousands of records, and it may already have hit every possible problem. Really, database code is not that huge a task. It’s much simpler than construction estimating or payroll. A database has to locate records on disk, make sure they don’t write on top of each other, and manage gaps (otherwise the file becomes an enormous hunk of Swiss cheese). Most of our database code is just sanity checking, and duplicating info so it’s possible to recover if data is corrupted by the outside world.

Since we are already converting data from old format to new, we decided to simplify bank accounts and transactions in the new accounting software. Goldenseal Pro only has 3 types of bank accounts instead of the current 7. The update process converts any old records, and also converts links between records. As we started using the Reconcile command, one missed link turned up. We fixed it.

Right now, Passwords are on the plate. I personally designed the Goldenseal interface, with little sliders to set the access each employee has to different parts of the data. It looks nice, but it was so non-standard that it took more than a month of programming. Too much time for a minor feature that many companies don’t even use.  In Goldenseal Pro it won’t look quite as nice, but it will use existing code. One more day to finish it, instead of a month.

Our staff started work on the Custom Layouts view more than a year ago. Some things work, but we got stuck on a few things and moved on to other code. Our staff is revisiting it, now that we’re more experienced with Cocoa.  It’s the last major interface element. The first release doesn’t absolutely need Custom Layouts, but it will make Passwords easier to finish.

Dennis Kolva
Programming Director
TurtleSoft.com

The past week, our staff has focused on the database code that runs our new accounting software. First thing was to update the Verify File command to work for Goldenseal Pro. It reads every record, runs a few sanity tests, then reports any errors. When we ran the verify, the Sample Company File showed 4 errors, and our Turtlesoft file had 8.

As it turned out, almost every error was caused by something different.  They were all very obscure problems. Hard to find and hard to figure out, so it took a full week to fix them all.

Along the way, we added a few other commands that test the integrity of the database structures. They helped with the bug fixes. Even better, they’ll show any future file errors before they become serious problems.

Like almost all databases, Goldenseal Pro locates records with a tree structure. Searches in a tree start at the root, then travel up through branches to the leaf nodes: one for each record. Actually, Goldenseal Pro uses a few thousand branches at each node, so it looks more like crabgrass than an actual tree. It’s faster that way.

With such fat nodes, it only takes two levels to handle anything under a few million records. None of our test files are even close to that big, so we temporarily turned one class into a binary tree, with just two records per node.  For 900 records it built a 10-level binary tree, and that tested out fine. So we are now confident that our accounting database can handle at least a few nonillion records. That’s a thousand billion billion billion, or a 1 followed by 30 zeros.

Our current company file has a bad NeoAccess node, which breaks several thousand Sales records from the early 1990s. Fortunately, we found a way to recover all but 8 of them, when converting from the old format to the new.

There are times when it feels insane to be writing our own database code, rather than using something well-established and open-source. So, I recently downloaded the MySQL source code to see what we are missing. It is very old C and C++ code. Hard to read and understand, and probably very hard to debug. Many potential security flaws and time bombs. Its style is similar to the NeoAccess code that we struggled with, partly rewrote, then finally abandoned. However, MySQL has several thousand source files rather than a couple hundred. It’s probably even more convoluted.

I think we made the right choice. It’s nice to have a report that explains every bit within the accounting file.

Dennis Kolva
Programming Director
TurtleSoft.com