Goldenseal Pro- Breakdown Table Redesign (Dec 29)

For software designers, a table is anything that shows a list of items in rows. Sometimes it has multiple columns of data, sometimes just one. For modern Mac OS, tables are made with NSTableView, built into Apple’s Cocoa framework.

In Goldenseal, we use tables for many things. Once we figured out the basics of NSTableView, we got most of them working fine in Goldenseal Pro, also.

The most complicated tables in Goldenseal are breakdown tables. Those show detailed line items in Estimates, Sales, Material Purchases, etc. They work kinda like a spreadsheet, but on steroids. Each row has several different types of data entry cells— text, popup lists, check boxes, padlocks. Some cells can change their type, depending on what’s in other cells.

A year ago, breakdown tables were already showing data, so we were optimistic about them. We just had to add data entry. Click and type, or click and choose from a menu, or click to unlock a padlock.

Alas, our staff has really struggled with that part. We’ve tried many different approaches. Each got some things working OK, but then hit a dead end. Each time we took a break and worked on other things for a while, then tried again.  Each failure made it all the more frustrating.

During holiday break this week, there was a “eureka” moment. Maybe the problem is bigger than just getting table cells to show popup menus. Maybe breakdown tables need a more extreme solution. Maybe the current design is fundamentally wrong.

Breakdown tables were also a struggle in the original Goldenseal. They took over a year of programmer time, with lots of futzing and fiddling. They’ve had the most bugs since then. Users often find them confusing. They still aren’t perfect.

Part of the problem is that the tables are very dense. Each line item only gets a little rectangular strip of screen space, maybe 1/4″ x 7″. It’s not many pixels to work with. Even worse, many table cells pop up a list of choices. That has to happen from a teensy box about 1/4″ x  1″. It’s too small for good interface design. Overall, the crowding makes breakdowns hard to program, and hard for users to understand.

For Goldenseal Pro, we already decided to put breakdown tables into a separate window, rather than in the main layout. It’s better use of screen space. Also, now there is room for useful buttons: you can click the Insert Row button, instead of control-click, click Delete Row instead of control-shift-click. The new setup is much more obvious for beginners.

We will now make the same interface change for data entry into table rows. Instead of typing or list-choosing directly in the row, you’ll double-click and see a data entry window. It will have regular data entry fields in a comfy layout, with helpful buttons.

For our programmers, the change saves a few thousand lines of code, and lots of frustration. No more futzing with rows and columns, and cramped interface design. In fact, breakdowns will have the same interface that we already use for lists, so we can reuse that code. Simpler design, which means fewer bugs.

For beginners, the new interface will be much easier to understand. Useful buttons, and less clutter. Easier for tired data entry staff, as well.

The change probably will be hardest for our long-time users, especially those who started out on the MacNail estimating spreadsheet. Goldenseal breakdowns still look and behave a bit like an Excel spreadsheet. Goldenseal Pro breakdowns will be less so.

Hopefully, the advantages will offset the annoyances, even for users who love Excel.

Dennis Kolva
Programming Director



Goldenseal Pro Progress- Breakdowns & Cocoa (Dec 21)

Last week we finished redoing the basic windows setup in Goldenseal Pro for Mac. This time we built the code with the latest OS version (and the latest version of Apple’s Xcode development tool). Xcode now gives better log messages for improper setup, so we could eliminate the errors, one by one.  Finally, everything was proper.

As a result, Goldenseal Pro now runs fine for all OS versions. The former crashes are gone. All windows are also resizable, now. They used to be stubbornly fixed-size, probably because their constraints weren’t valid.

There is still one more big hurdle to overcome.  Our staff has started on breakdown tables several times, but every time we get stuck. Most things work OK, but the basic interface is just not right. We want to have an editable text field appear promptly when you click in a cell. Unfortunately, that basic task has been extremely challenging to pull off.

What we finally figured out is that Apple’s table class (NSTableView) is designed for Finder-like lists.  To change a file name in the Macintosh Finder, you click in the text, then wait a second, then click again. It works that way so you won’t accidentally rename files when you really just want to double-click to open a file, or click-and-drag to move it.

Unfortunately, click-wait-click is not a good fit for Goldenseal Pro. That behavior is deeply baked into NSTableView.

To solve problems like this in C++ and PowerPlant, we would set breakpoints in the code, and then step through line by line to see how the library works. Eventually that leads to a clear understanding of the mechanics. Then, eventually, we can write code that does the job reliably.

Unfortunately, Apple’s Cocoa library is header-only.  Stepping into it show machine language. It is much, much harder to understand than C or C++ code. Even worse, it doesn’t show any of the comments that good programmers add, to help others understand their code. As a result, Cocoa is pretty much a ‘black box’. We have to try things, and see what they do. Then try other things. Maybe we can eventually understand the code, and maybe not.

Best guess right now is another week until we get the breakdown tables working properly, when you click on a cell. After that it will go much faster. Most of the rest is already programmed.

Dennis Kolva
Programming Director

Goldenseal Pro Progress- Containers & Constraints (Dec 11)

Every computer app needs to position stuff on the screen. Usually it takes four coordinates for each item: horizontal and vertical positions, plus height and width. The whole setup is called a frame, since it defines a rectangle that encloses the field, button, window or whatever.

Numbers are fine for computers, but not great for humans. We programmers would rather use a drawing program to lay out the screen appearance. Then we can see what we’re doing.

Goldenseal includes a simple drawing program in the Custom Layouts command. It lets anyone drag around fields, and change the appearance of data entry windows, printed forms or reports. Our staff used Custom Layouts to set up the stock layouts. Users can customize anything with it.

To set up the rest of the interface for Goldenseal, our staff used Constructor. It was a simple drawing program that came with PowerPlant, the library we used to build the first Goldenseal. Constructor set up the rest of the appearance that Custom Layouts couldn’t handle.

For Goldenseal Pro, we use similar visual drawing environments built into Xcode (for Mac) and Visual Studio (for Windows). They save a lot of time.

Frames are good enough for a simple screen layout. Everything has its size and place. But, what happens when you resize the window? These days, people expect stuff to rearrange, and stay useful whatever the size and shape of the window they are in. Unfortunately, a dynamic layout is much harder to set up than a static one. There are many complications.

One layout tool that every programming library uses is containers. All the contents of a container will move together. As a bonus, containers make programming a bit tidier. PowerPlant called them panes, Apple’s Cocoa framework calls them views, and Microsoft’s MFC calls them windows (in Windows programming, everything is a window). Containers are generally a good idea, but they do add some complexity.

Cocoa also has separate controllers for each view, as part of their MVC architecture (model/view/controller). Another good idea, but it adds even more complexity.  The main window in Goldenseal Pro has 9 different views, each with their own controller. Even worse, we need to swap several different views into the top bar and main record area, depending on what you are doing. It took a few months to get it all working correctly.

The other layout tool found in modern programming libraries is constraints. Basically, they are a way to bind each item to its neighbors, so it knows where to go when its container moves or resizes.

PowerPlant and Constructor had a very simple constraints system, but we rarely used it.

For the Mac version of Goldenseal Pro, it’s required that we use the constraints built into the Cocoa framework. We tried doing without it, but just got errors. Unfortunately, they are a confusing mess.

Apple started out with a ‘struts and springs’ metaphor, pretty much like connecting screen items with paper clips and rubber  bands. Then they replaced it with an auto-layout feature in the drawing view. It works great for some things, but can’t be used everywhere.

In OS 10.7, Apple added a NSLayoutConstraint class.  Every view needs at least four constraints, and each looks something like this in the code:

NSLayoutConstraint *rightConstraint = [NSLayoutConstraint                                                                                constraintWithItem : statusBar
                                                    attribute : NSLayoutAttributeTrailing
                                                    relatedBy : NSLayoutRelationEqual
                                                    toItem : theRecordView
                                                    attribute : NSLayoutAttributeTrailing
                                                    multiplier : 1.0    constant : 0];
Throw in a few hundred of those, and the code gets very cluttered. Finally, in OS 10.11, they added yet another constraint system, with much simpler code. Now each constraint looks like this:
[statusBar.topAnchor constraintEqualToAnchor :
                                                  theRecordView.topAnchor].active = YES;

It’s much more palatable, and less prone to stupid errors.

Over the past two years, we used frames to lay out some things in Goldenseal Pro. For others we used constraints, with some of each type. The screens looked OK, but we were definitely bumbling around and didn’t always do the right thing. Sometimes we accidentally added duplicate constraints. Some conflicted with others. Some were just plain wrong. Those inconsistencies are probably why the code crashes when we build with the latest versions of Mac OS.

This week we are redoing all the windows to use the most recent Cocoa constraint system. It’s going to be all anchors.  Newer versions of Xcode are better at reporting errors, so I think we can finally get it right.

Dennis Kolva
Programming Director



Goldenseal Pro- Mac OS Versions (Nov 30)

For the past two years we have been building Goldenseal Pro on Macs running OS 10.11 (El Capitan). Occasionally we test it on newer OS versions.

A few weeks ago we began to update the actual build computers. That means using newer versions of Xcode (Apple’s software-development app), running on the matching, newer OS. The app also uses newer versions of the SDK (Software Developer’s Kit).

To create Goldenseal Pro, Xcode starts with our source code (about 2500 C++ and Objective-C files). It converts the text into machine language, then adds Apple’s code from the SDK. It then links everything together to make the final app. Altogether, it’s a complicated process. Luckily, once it’s set up we can just change text, then hit the Run command. During productive programming, that happens 50 or 100 times per day.

Apple moves fast. Since 2012, they have released a new OS and SDK every September +/-. Each new release adds a few features, and fixes some old bugs. Unfortunately, new versions also remove a few old/deprecated features, and add new bugs.

Along with the OS, Xcode also evolves. Sometimes the interface that we see changes, as does the compiler which translates source code to machine language. Sometimes bug fixes, sometimes new bugs.

With all the changes in basic frameworks and tools, it’s not surprising to discover problems when we update the build OS.

At present, Goldenseal Pro does well when built with OS 10.12 (Sierra). The main window display has a few cosmetic glitches, but those are caused by layout errors that we are fixing now. Sierra is more fussy, but that’s a good thing.

We also can build with OS 10.13 (High Sierra). Most things work, but the app crashes after switching tabs in the main window. Lots of things break Cocoa, so it’s possible that fixing the minor display issues will also fix the crashes. We’ll see about that soon.

OS 10.14 (Mojave) uses a new file system that isn’t visible to OS 10.12 or earlier. To use it conveniently will require more new hardware. The new Mac Minis are not very attractive for the price, so we are considering options. Besides cash, it’s a matter of desk and shelf space, and what we use for daily business. I personally still like OS 10.8 (Mountain Lion) better than all. It’s the last version that allows files with color bars. Upgrading will be a downgrade for my own personal file management.

Before TurtleSoft ships Goldenseal Pro, we need to make a decision about which SDK to use for the final release. It limits which computers can run the app.

Right now, it looks like the best option will be to require either 10.12 (Sierra) or 10.13 (High Sierra).  Hearing from users will help. Mac OS version updates are free, but at some point they require a hardware update. There’s no way Goldenseal Pro can run on OS 10.8, but 10.11 is doable.  Maybe 10.10.  Even if we use an older SDK, it’s still good that we are building on newer OS versions now. It helps catch bugs, and ensures that Goldenseal Pro will migrate easily to future updates.

A couple weeks ago we were making good progress on breakdown tables. Since then, most days have been consumed by installations and testing. Fortunately, final setup is growing near. It will be nice to get back to regular programming again. Making code do tricks is much more fun than downloads and settings for the 6th or 7th time.

From here on out. TurtleSoft will run entirely on SSDs instead of hard drives. They are zippy.

Dennis Kolva
Programming Director

Goldenseal Pro Progress: NSDrawer -> NSPanel (Nov 16)

We started interface work on Goldenseal Pro for Mac a bit over two years ago. During the first couple months we learned Apple’s Cocoa framework, and checked their sample projects to see how best to design the windows.

One sample project was called Drawer Madness. It had a main window, with smaller windows popping out from the sides. It was implemented with a Cocoa class called NSDrawer. Getting drawers to work only took about 10 lines of code. It seemed an excellent way to display breakdown tables, so that’s how we programmed them.

Right about then, Apple decided to deprecate the NSDrawer class.  Deprecation means they won’t provide any further support or improvements, and plan to remove it completely some day. Sigh.

Breakdown drawers looked really nice, so we decided to stick with them for as long as they still worked.

This week, we started testing with the latest Mac OS version. Unfortunately, after opening a breakdown drawer, Xcode gave a warning that the “next responder” was incorrect. The app then crashed randomly. Xcode didn’t say what caused the problem, but NSDrawer seemed a likely suspect.

So, we replaced the breakdown drawer with a different type of window called an NSPanel. It comes down in front of the main window instead of out from the side. We already use panels for lists and “more info” windows, so the change wasn’t difficult. When it was done, there were no warnings, and no crashes.

We probably could duplicate the drawer appearance with a secondary window and some animation. Unfortunately, it would mean learning new Cocoa stuff, and probably would take at least a month to set up. Right now we really want to finish ASAP. Panels are good enough for now.

The way we display breakdowns is kind of a design trade-off. Goldenseal currently has breakdown tables at the bottom of the main window, with all the other fields. You can switch them on and off with the Breakdown popup button on the left. It makes some windows much taller than others. That’s OK when there are lots of separate windows, but it’s ugly if everything is in one window. These days, most apps use a single window with tabs, and Goldenseal Pro will do the same. It’s less cluttered.

Using a drawer on the side means you can see both the regular Estimate fields and the table, similar to how Goldenseal looks now but wider. It also has space for a few control buttons: Add Row etc. The down side is that the drawer is so wide that it may not fit on a 13″ laptop.

Using a panel is the most efficient use of screen space, it also has room for control buttons, and it reduces clutter to a minimum. But you can’t see the breakdown and the rest of an Estimate at the same time. If you need to jump back and forth it’s slightly annoying.

When our staff moved breakdown tables from drawer to panel, we consolidated the code so it will be easier to move again later. That way it won’t be too hard to add drawer-like windows in a later update, if users really want the option.

Dennis Kolva
Programming Director


Goldenseal Pro- Smart Fields & Tables (Nov 9)

In the original Goldenseal, our staff spent 3 or 4 months programming clairvoyant fields. They are complicated.

Clairvoyant fields (soon to be called smart fields) have three different ways to data enter. You can click on a popup button, type in, or start typing and then select from a scrolling list. They show lists that may come from several different places. They also let you edit list items, or add new ones, or jump to a different record.  Or if they are disabled, you can double-click to jump to that item. It took a lot of fiddling and testing to get everything working smoothly.

Programming breakdown tables took even longer. We spent 2 or 3 programmer-years on those. It was like re-inventing Microsoft Excel, but with special features for estimating and accounting.

Then, even worse, clairvoyant fields act differently when inside a table cell. They need extra coding to handle the quirks.

Last week our staff got so frustrated with Cocoa smart fields, that we went back and tried for the third time to build them from NSComboBox. It’s Apple’s version of a list-choosing field, and it is so-o-o-o close to what we need. This time we set things up so it’s easy to switch back and forth. After a few days of testing and modifying, we gave up for the third time.

The good news is, regular smart/clairvoyant fields now work very well. There probably are some obscure problems still lurking, but they perform as they should in all possible combinations of tabs, clicks and typing.

Breakdown tables are mostly working, but the smart fields inside them still have problems.  Our staff is on that now.

Once those are done, we should be ready to start using Goldenseal Pro to run our own daily business. Though we will be cautious, and keep using old Goldenseal in parallel for a while. It’s almost guaranteed that we will find bugs. That’s why we do it.

Dennis Kolva
Programming Director


Macintosh Hardware (Oct 30)

Apple just announced its second round of new hardware for this Fall. Most of our users are probably interested in the updated MacBook Air, but we were all about the Mac Mini. It hasn’t been updated since 2014, so the new version was long overdue. Really, the Mini stalled out in 2012. The newer ones seem to be slower.

Why do we like the Mini? Well, software developers need large screens. Writing code means looking at scads of text. The more you can see, the better. Beside that, there needs to be room for the app being developed, plus a debugger window. So, it makes sense to get a very large screen, with a separate CPU.

For testing, it also helps to have several machines with different OS versions. It’s nice to still have space for other things on one’s literal desktop. So, the smaller the CPU box, the better.

TurtleSoft switched to Mac Minis, soon after they were released. With a big monitor and a KVM switch, it’s possible to run several machines at once and switch between them. For a long time we stacked the Minis, but lost a couple to overheating. Now they are on individual shelves. Meanwhile, Windows desktops still tend to be huge, but we can pop in cheap laptops when there’s need for more than one.

The new Minis have better specs, but not by much. The underlying problem is that computer hardware is reaching maturity. CPU speeds are not increasing like they used to, and it’s only going to get worse. Silicon chips are already starting to be affected by physical limits like the speed of light and the size of atoms.

Meanwhile, we are still working on breakdown tables.

Dennis Kolva
Programming Director

Goldenseal Pro Progress Report- Oct 16

The two most recent estimating spreadsheets for Goldenseal Pro calculated that it would be finished this month. Unfortunately, the project is still not done.  The math we used was very guessy, so this is not a big surprise.

The best estimate right now is that it will still take another month or two to complete the basic data entry features. Most things are working, but they can still use some polishing. Then, there is still more to do after that. We’ll get a better handle on what remains, after we start using Goldenseal Pro to run our own business.

Our staff started the interface programming a bit over two years ago. If we had just duplicated the previous appearance, the work probably would already be done. However, once the software was stripped down to studs and joists, it was a great opportunity to do more serious remodeling.  The changes probably added six months or a year to the schedule, but the result is a much better software program.

For example, a couple weeks ago we rewrote the former popup buttons. Now you can tab into them, and type instead of clicking. It’s a nifty little improvement, but it took a few days to write the code and test it. Then another couple days to redo all the data entry layouts, so tabbing would enter the former popups in correct order.

We currently are working on breakdown tables again. That is another area that’s getting big changes. The interface is particularly complicated for estimating, project billing and payroll.

Overall, progress has been steady. It will continue to be so.

Dennis Kolva
Programming Director

Goldenseal Pro Popups -> Smart Fields (Oct 6)

Goldenseal currently uses a few popup buttons on the data entry screens. Human interface guidelines for both Mac and Windows consider them as the official way to present a short list of choices. You click on the button, see a list of items, and choose one.

For serious data entry, popup buttons are a pain. Problem is, popups require a mouse click. You can’t access them from the keyboard. It’s not a huge problem for one-handed typists, but it definitely breaks data entry flow for touch typists who use all fingers. Half the time, when you move one hand back to the keyboard you get leubpard or jetbiard instead of keyboard.

Goldenseal uses clairvoyant fields for longer lists. For those, you can tab in from the keyboard, and type the first few letters to choose an item. You can also click the mouse on a popup button. It’s the best of both worlds. Clairvoyant fields are similar to combo boxes, but they don’t let you type in things that are not already on the list. We plan to call them smart fields in Goldenseal Pro.

Replacing popup buttons with smart/clairvoyant fields has been on our to-do list since 2001. This week we finished the transition. The new versions look great and work well. We will still use pure popup buttons in a few places, but not on screens that involve serious data entry.

There was some adventure along the way. First of all, smart fields had somehow lost their popup buttons. We ran some older versions, and the buttons disappeared back in May.  Since then we have been working on other things, and didn’t notice it until now.

Our programming staff saves changes into source control at least once a day. Often, several times per day. That helps in a case like this, since it’s easy to go back and run versions from different dates. Even with a binary search, it meant building and testing nine or ten times. Eventually we hit the last version that worked, and the first one that didn’t. Checking the list of changes between them made it easy to spot the error.

With that out of the way, we started to type into smart fields, and discovered that the pop-down list of items was also missing. So, another binary search in source control. That eventually narrowed down to a small, unrelated change made last February. It had the side effect of zapping the smart field display. An easy fix once we found it.

Finally, we soon discovered that the smart field code was pretty bad. Our staff wrote it back in Fall 2016, and we’ve learned a lot about Cocoa since then.  There are better ways to handle tables than our first efforts.

So, we tossed it completely and rewrote it to current standards. Doing that probably saved time, and it definitely saved mental anguish. In fact, almost everything else written in the first six months has been replaced since then.  It’s an expected part of the learning curve.

Dennis Kolva
Programming Director

Goldenseal Pro- Popups & Technical Debt (Sep 29)

Software developers have to worry about something called technical debt. It’s all the mediocre code and bad designs that happen because someone is too tired, or too rushed, or too inexperienced to do it right. The result is code that is buggy, or hard to maintain. Large projects will always have some technical debt. It’s just a matter of how much.

Construction contractors face their own versions of technical debt. It accrues when equipment isn’t maintained properly, or when stuff is not built quite to spec. Technical debt isn’t a problem right this minute, but it probably will bite you on the a** someday.

In construction, the cure for technical debt is regular equipment maintenance, plus quality control inspections to catch small problems before they snowball into disasters. In the software world, the cure is about the same: testing, training, code reviews. Because software is easy to revise, another debt-reduction tool is refactoring: rewriting old code so it makes more sense. We have been doing a lot of that since we started on Goldenseal Pro.

Tables are a fairly complicated part of Goldenseal, so they have plenty of potential debt. This week we rewrote the way they store data, so it is simpler. There is much less code now to do the same thing.

The clairvoyant fields that show lists of items are another complexity. We already rewrote the ones in data entry layouts (and renamed them to smart fields). Now we are working on the smart fields in tables. They look a little different, and have many complications. For example, the Cost Item column in an estimate breakdown needs to adjust what it shows, depending on three different columns to its left (Cost Area, Category and Subcategory). 

Goldenseal currently uses popup buttons to enter data from lists that don’t ever change. For example, any transaction involved in job costing has a Job Class popup to allocate to overhead or projects. Popup buttons require a mouse click, which can be annoying when doing data entry. For everything else you can type and hit the tab key to move on, but popups require you to use the mouse. Not great for two-handed typists.

In tables, we already use smart fields instead of popups, so it would not be hard to do the same thing for all popup buttons.  We are also working on that now.

Dennis Kolva
Programming Director