Turtlesoft Blog

In the original Goldenseal, our staff spent 3 or 4 months programming clairvoyant fields. They are complicated.

Clairvoyant fields (soon to be called smart fields) have three different ways to data enter. You can click on a popup button, type in, or start typing and then select from a scrolling list. They show lists that may come from several different places. They also let you edit list items, or add new ones, or jump to a different record.  Or if they are disabled, you can double-click to jump to that item. It took a lot of fiddling and testing to get everything working smoothly.

Programming breakdown tables took even longer. We spent 2 or 3 programmer-years on those. It was like re-inventing Microsoft Excel, but with special features for estimating and accounting.

Then, even worse, clairvoyant fields act differently when inside a table cell. They need extra coding to handle the quirks.

Last week our staff got so frustrated with Cocoa smart fields, that we went back and tried for the third time to build them from NSComboBox. It’s Apple’s version of a list-choosing field, and it is so-o-o-o close to what we need. This time we set things up so it’s easy to switch back and forth. After a few days of testing and modifying, we gave up for the third time.

The good news is, regular smart/clairvoyant fields now work very well. There probably are some obscure problems still lurking, but they perform as they should in all possible combinations of tabs, clicks and typing.

Breakdown tables are mostly working, but the smart fields inside them still have problems.  Our staff is on that now.

Once those are done, we should be ready to start using Goldenseal Pro to run our own daily business. Though we will be cautious, and keep using old Goldenseal in parallel for a while. It’s almost guaranteed that we will find bugs. That’s why we do it.

Dennis Kolva
Programming Director
TurtleSoft.com

 

Apple just announced its second round of new hardware for this Fall. Most of our users are probably interested in the updated MacBook Air, but we were all about the Mac Mini. It hasn’t been updated since 2014, so the new version was long overdue. Really, the Mini stalled out in 2012. The newer ones seem to be slower.

Why do we like the Mini? Well, software developers need large screens. Writing code means looking at scads of text. The more you can see, the better. Beside that, there needs to be room for the app being developed, plus a debugger window. So, it makes sense to get a very large screen, with a separate CPU.

For testing, it also helps to have several machines with different OS versions. It’s nice to still have space for other things on one’s literal desktop. So, the smaller the CPU box, the better.

TurtleSoft switched to Mac Minis, soon after they were released. With a big monitor and a KVM switch, it’s possible to run several machines at once and switch between them. For a long time we stacked the Minis, but lost a couple to overheating. Now they are on individual shelves. Meanwhile, Windows desktops still tend to be huge, but we can pop in cheap laptops when there’s need for more than one.

The new Minis have better specs, but not by much. The underlying problem is that computer hardware is reaching maturity. CPU speeds are not increasing like they used to, and it’s only going to get worse. Silicon chips are already starting to be affected by physical limits like the speed of light and the size of atoms.

Meanwhile, we are still working on breakdown tables.

Dennis Kolva
Programming Director
TurtleSoft.com

The two most recent estimating spreadsheets for Goldenseal Pro calculated that it would be finished this month. Unfortunately, the project is still not done.  The math we used was very guessy, so this is not a big surprise.

The best estimate right now is that it will still take another month or two to complete the basic data entry features. Most things are working, but they can still use some polishing. Then, there is still more to do after that. We’ll get a better handle on what remains, after we start using Goldenseal Pro to run our own business.

Our staff started the interface programming a bit over two years ago. If we had just duplicated the previous appearance, the work probably would already be done. However, once the software was stripped down to studs and joists, it was a great opportunity to do more serious remodeling.  The changes probably added six months or a year to the schedule, but the result is a much better software program.

For example, a couple weeks ago we rewrote the former popup buttons. Now you can tab into them, and type instead of clicking. It’s a nifty little improvement, but it took a few days to write the code and test it. Then another couple days to redo all the data entry layouts, so tabbing would enter the former popups in correct order.

We currently are working on breakdown tables again. That is another area that’s getting big changes. The interface is particularly complicated for estimating, project billing and payroll.

Overall, progress has been steady. It will continue to be so.

Dennis Kolva
Programming Director
TurtleSoft.com

Goldenseal currently uses a few popup buttons on the data entry screens. Human interface guidelines for both Mac and Windows consider them as the official way to present a short list of choices. You click on the button, see a list of items, and choose one.

For serious data entry, popup buttons are a pain. Problem is, popups require a mouse click. You can’t access them from the keyboard. It’s not a huge problem for one-handed typists, but it definitely breaks data entry flow for touch typists who use all fingers. Half the time, when you move one hand back to the keyboard you get leubpard or jetbiard instead of keyboard.

Goldenseal uses clairvoyant fields for longer lists. For those, you can tab in from the keyboard, and type the first few letters to choose an item. You can also click the mouse on a popup button. It’s the best of both worlds. Clairvoyant fields are similar to combo boxes, but they don’t let you type in things that are not already on the list. We plan to call them smart fields in Goldenseal Pro.

Replacing popup buttons with smart/clairvoyant fields has been on our to-do list since 2001. This week we finished the transition. The new versions look great and work well. We will still use pure popup buttons in a few places, but not on screens that involve serious data entry.

There was some adventure along the way. First of all, smart fields had somehow lost their popup buttons. We ran some older versions, and the buttons disappeared back in May.  Since then we have been working on other things, and didn’t notice it until now.

Our programming staff saves changes into source control at least once a day. Often, several times per day. That helps in a case like this, since it’s easy to go back and run versions from different dates. Even with a binary search, it meant building and testing nine or ten times. Eventually we hit the last version that worked, and the first one that didn’t. Checking the list of changes between them made it easy to spot the error.

With that out of the way, we started to type into smart fields, and discovered that the pop-down list of items was also missing. So, another binary search in source control. That eventually narrowed down to a small, unrelated change made last February. It had the side effect of zapping the smart field display. An easy fix once we found it.

Finally, we soon discovered that the smart field code was pretty bad. Our staff wrote it back in Fall 2016, and we’ve learned a lot about Cocoa since then.  There are better ways to handle tables than our first efforts.

So, we tossed it completely and rewrote it to current standards. Doing that probably saved time, and it definitely saved mental anguish. In fact, almost everything else written in the first six months has been replaced since then.  It’s an expected part of the learning curve.

Dennis Kolva
Programming Director
TurtleSoft.com

Software developers have to worry about something called technical debt. It’s all the mediocre code and bad designs that happen because someone is too tired, or too rushed, or too inexperienced to do it right. The result is code that is buggy, or hard to maintain. Large projects will always have some technical debt. It’s just a matter of how much.

Construction contractors face their own versions of technical debt. It accrues when equipment isn’t maintained properly, or when stuff is not built quite to spec. Technical debt isn’t a problem right this minute, but it probably will bite you on the a** someday.

In construction, the cure for technical debt is regular equipment maintenance, plus quality control inspections to catch small problems before they snowball into disasters. In the software world, the cure is about the same: testing, training, code reviews. Because software is easy to revise, another debt-reduction tool is refactoring: rewriting old code so it makes more sense. We have been doing a lot of that since we started on Goldenseal Pro.

Tables are a fairly complicated part of Goldenseal, so they have plenty of potential debt. This week we rewrote the way they store data, so it is simpler. There is much less code now to do the same thing.

The clairvoyant fields that show lists of items are another complexity. We already rewrote the ones in data entry layouts (and renamed them to smart fields). Now we are working on the smart fields in tables. They look a little different, and have many complications. For example, the Cost Item column in an estimate breakdown needs to adjust what it shows, depending on three different columns to its left (Cost Area, Category and Subcategory). 

Goldenseal currently uses popup buttons to enter data from lists that don’t ever change. For example, any transaction involved in job costing has a Job Class popup to allocate to overhead or projects. Popup buttons require a mouse click, which can be annoying when doing data entry. For everything else you can type and hit the tab key to move on, but popups require you to use the mouse. Not great for two-handed typists.

In tables, we already use smart fields instead of popups, so it would not be hard to do the same thing for all popup buttons.  We are also working on that now.

Dennis Kolva
Programming Director
TurtleSoft.com

Tables are an important part of the Goldenseal interface. They are spreadsheet-like rectangles with a list of details, arranged into rows and columns. Many accounting transactions include breakdown tables. There are similar tables in some types of accounts, and many of the setup lists. Almost all reports use a table. So do the action commands like Pay Bills and Write Payroll.

When we started on Goldenseal back in the mid-90s, computers only had 1 to 4 megabytes of RAM. A construction estimate might have many hundreds of rows in its breakdown table. Multiply that by 18 columns, and a table could take up a significant chunk of available memory.

We built Goldenseal using the CodeWarrior compiler. It included an LTable class in its framework, to manage the rows and columns in a table. LTable was designed to be very frugal with RAM. It did not include much code to manage table appearance or behavior. As a result, Goldenseal has thousands of code lines to handle all the nitty-gritties of tables.

This week, we got far enough along on breakdown tables in the Mac update, to realize that the raw table code is due for a major overhaul.

For one thing, we now have plenty of RAM available. No need to be frugal, so we can redesign table data storage so it’s easier to understand and maintain. On the Mac, we also need to do less. NSTableView (the Cocoa table class for Mac) is already a full-fledged table maker. It handles most of the interface, so our code only needs to store data, and handle the business-specific details. More than half our existing code is unnecessary on the Mac.

We started to delete stuff, but then reverted. It seemed wise to double-check the Windows version, first. Since February our staff has only worked on the Mac version, so there were many changes to sync before we could run the Windows version again.

It turns out that Windows does not have a table class  in their MFC framework. That was surprising, since Excel was the main reason that Windows succeeded, and it is just one big table. There are some open-source table libraries out there, but when we checked them, none seemed worth adopting.

Fortunately, we already have code to draw and manage tables. For the Mac, we will bypass it, but for Windows we’ll still use it. What we can still do for both platforms is improve the basic code that manages tables.

Goldenseal uses object-oriented programming, but for some reasons our tables did not use objects for rows, columns and cells. NSTableView has them, and it makes everything much easier to understand.  Our staff is about half-way done with rehab work, to add them.

Dennis Kolva
Programming Director
TurtleSoft.com

 

In the past couple weeks, our staff has been working on bank transactions, the Find command, and breakdown tables.  No major snags. Progress has been steady. It’s mostly just a constant process of testing, finding bugs, and fixing them. Rinse, repeat.

When writing new software, you might say that the code starts out as 100% bugs. Nothing works at all. Then, it gradually shrinks down to near 0%. For a major upgrade like this one, maybe it starts at 60% bugs.

Similar to construction projects, software has its phases as the structure gradually approaches completion. Most things move in spurts, often with months between them.

For example, the current version of Goldenseal accounting software uses four different types of bank transactions to handle checkbooks, cash, loans and credit cards. We decided to combine them into one. There are currently five different layouts for checks, payments, deposits, and transfers in/out. Those will also combine into one. The change is good for the long term. There will be less code to contain bugs, and a simpler interface for users. In the short term it means extra work.

Our staff finished the database changes for bank transactions more than two years ago.

Bank transactions are unusual, because each window shows transactions for just one bank account. Everything else groups all accounts together. That means some fancy coding for the menus in Goldenseal. We’ll still keep that setup for Goldenseal Pro, since most people work with one bank account at a time. But the accounts will now show individually in the outline view instead of in pull-down menus. We finished that work more than a year ago.

Current users have data that they’ll want to keep using. That means we need to convert it to the new format for Goldenseal Pro.  It’s complicated because expenses and income transactions link to the bank transaction that paid for them, and they also need converting. We finished that last Winter.

Goldenseal currently uses a special window layout for banking, but we wanted to simplify that. We moved fields around and got the new windows working last week.

The basics now work for bank transactions, but some bugs probably still lurk. We will meet and conquer them as we finish banking reports and the Reconcile command, and start using it with our own data.

A similar spurty development process is happening for the Find command, breakdown tables, and every other aspect of Goldenseal Pro.

Dennis Kolva
Programming Director
TurtleSoft.com

 

As mentioned in the last post, we recently switched to a newer memory management system (ARC). That caused a crash after opening list windows. Switching back fixed it temporarily, but we want to use the newer system. Unfortunately, there weren’t many clues to help solve the problem.

One good approach for mystery errors is to eliminate details, and make the setup as simple as possible. We gradually removed everything from the window, but even when completely empty, it still crashed. A brand new empty window also crashed. That was good progress, actually, since it eliminated many possible culprits.

Many of the accounting screens have More Info buttons that show a smaller window that contains more fields. It uses very similar code, and it still works fine. We copied that window, but it crashed. Then we copied all the code that creates it, and it worked OK. Yesterday we changed that working version bit by bit, until it crashed. Then went back a step, and finally got lists working again. It’s not 100% clear why it was breaking, but at least we know approximately how to fix the error, if it happens again.

We use C++ for most of our programming. It’s a programming language with a reputation for being very hard. One reason for that is because the programmer needs to manage memory. It requires care and skill to avoid crashes and memory leaks.

Many newer languages are ‘easier’, thanks to built-in memory management. The Cocoa library also does that with ARC.  It sounds great in theory, to have software that manages memory for you.

I think the best analogy here is that C++ is like a regular car or truck with a stick shift. Hard to learn, but reliable once you learn it. It does what you expect. Cocoa, Java, and other modern languages are like a self-driving car. It’s way easier, and practically lets you snooze while on the road. But occasionally the system won’t see a stationary bus, and you’ll drive into it at full speed. That’s pretty much what happened, the past couple weeks.

I think we will stick with ARC, because it is working fine otherwise. Maybe we won’t hit many more stationary buses.  Or we will get good at spotting buses on the horizon.

The Cocoa library for Macintosh can be extremely frustrating, but sometimes it works like a dream. For example, Interface Preferences has a couple of popup buttons that show a lists of fonts. It took about 30 lines of code to get it working in the original Goldenseal. We did a quick search, and copy/pasted two lines of code. It now shows a standard font dialog. Cool!

Meanwhile, we are going through each data entry window, and fixing the occasional problem or missing feature. For example, bank transactions currently shows a running total on the left side of the window. Goldenseal Pro uses that space for the outline view, so it can’t go there. However, we have a bottom bar similar to Excel’s, and it’s the perfect place for that kind of text. It has been empty for quite a while, but now it shows useful text. We also added a built-in progress bar to replace the current window.

Running a construction business is complicated. Estimating and accounting are complicated. That makes Goldenseal a complicated program. There are many quirks to get working again. The good news is that most of them are easy to convert. They are close to instant gratification.

Dennis Kolva
Programming Director
TurtleSoft.com

 

 

Last week’s switch to ARC (Automatic Reference Counting) seemed to go without a hitch. Our staff was cruising along, finishing up the Preferences window and a few other details. Then, we went to edit a list on Wednesday, and the app crashed. It never did that before.  Xcode usually shows a ‘stack trace’ that explains what went wrong, but this one only had the program entry point. That’s a very hard crash.

A while back I mentioned that the completion date for Goldenseal Pro depends mostly on how many curve balls the Cocoa framework throws at us. Each is different, but here is how we are handling this particular problem.

When there are no clues, the first thing we do is to step through the code to see exactly where it goes wrong. The process can be tedious, but usually it ferrets out an obvious error. This one turned out to be tougher. Sometimes the window would crash immediately. Sometimes it would open and stick around for a few seconds, then crash. Sometimes it would wait until we clicked something. There was no smoking gun in the code.

That delay made it seem like it might be caused by ARC deleting something prematurely. Sure enough, turning off ARC made it work OK again.

Apple has several diagnostic tools that can help with memory errors, including something called NSZombie. We tried them, and discovered that the window itself was being deleted too soon. Very odd. Nothing in the code should have done that. We stepped through the code many times, but didn’t find a smoking gun.

98% of the time, when something doesn’t work we can figure out why, and make a fix that makes sense. Then there is the other 2%, where our staff is totally baffled, and has no idea what’s going on. This is one of those.

One approach that sometimes helps is to go back to older versions and see if they also break. Finding the date when things go bad really narrows down the possible culprits. Unfortunately, older copies also work fine without ARC, but break with it.

Next step was to start deleting things from the window. That usually narrows the problem down to one specific control. Except it didn’t. The app crashed, even with everything removed.  Very odd. Along the way, there was an error warning for a button that was deleted last year. Also odd. A search in the code didn’t turn up any mention of it, but Xcode thought it was still there. That suggests some sort of file corruption in Xcode’s layout data. Why it would only break under ARC, we have no idea.

The list window was the very first Cocoa we worked on, almost two years ago. We made many mistakes and changes, so it is the most likely file to be corrupted. The solution is to just build it again from scratch. That’s on the agenda for today. We will add a few things at a time and test it frequently.

In general, the goal is to have very few changes made between working and failing versions. Then it’s easier to see what’s wrong.

Meanwhile, if we turn off ARC, lists work fine. If we turn it on, everything else still works fine. So we added some code to make it easy to switch ARC on and off. Now it just requires one line of code, instead of 100 changes. That way we can proceed productively, despite this curve ball.

If the rebuild doesn’t fix it, this may be the kind of problem where it’s best to just give up for a while. Sometimes an answer appears out of the blue, weeks or months later. Or, sometimes we accidentally fix it when we fix something else.

To be continued…

Dennis Kolva
Programming Director
TurtleSoft.com

 

Our staff spent the past couple days updating Goldenseal Pro to use ARC (Automatic Reference Counting).  It’s a nifty feature that is Cocoa’s third generation of memory management.  We actually thought it was already turned on by default. But, while checking some other compiler settings this week, we discovered it wasn’t.

Memory is a big deal to programmers. As Goldenseal runs, we create thousands of objects. Each is a chunk of memory that stores some data and does tricks. We give each object a name, then the operating system decides where it will live in RAM.

In C++, we need to decide when each object dies. If we kill it too soon and try to use it after it’s dead, Goldenseal will crash. If we forget to kill it, there will be a memory leak. Accumulate too many leaks, and the app will consume more and more RAM. Eventually the program will get very slow. It may even consume all available memory and crash.

Fortunately, modern C++ has many tools for managing memory, to make sure each object is deleted properly. They help Goldenseal to leak very little, and almost never crash. Some other software is not so careful. For example, most web browsers are very leaky. Use them all day, and they will gradually use up many megabytes or even gigabytes of memory.

Cocoa used to require manual memory management, similar to C++ (but without the modern tools to help). In the mid-2000s it added garbage collection (also a feature of Java and many other programming languages). Garbage collection prevents leaks by periodically checking every object, and killing it if it’s no longer being used. That prevents leaks, but also can be annoying. If you ever have an app suddenly stop dead for several seconds, its garbage collector probably is “emptying the trash”.

Apple added ARC in 2011. It automatically kills each object when nothing else uses it any more. For example, dozens of objects die whenever you close a window. It makes memory one less thing to worry about.

For ARC to work, we had to add a __bridge keyword, every time we convert between C++ and Objective-C.  The compiler colors it bright green. It’s actually kinda nice to have it in the code. The conversions are risky, and anything that makes them more obvious is a good thing.

In general, our staff spends a lot of time looking at brightly colored text. Source code is a huge mass of comments, objects, functions and programming commands. It can be overwhelming. A bit of color coding really helps.

Dennis Kolva
Programming Director
TurtleSoft.com