Xojos bug tracking system

1

Recently, we invested a significant amount of VC funding to conduct a thorough analysis of our core business operations. We engaged a well-regarded national consulting firm with expertise in software development to assist in reviewing our efforts. Since we had some “use it or lose it” funds available in the budget, we asked them to also analyze the current “bug” situation in Xojo.

The only reliable data they could access was the issue database. To my knowledge, there are no additional internal tools Xojo uses to track the broader bug landscape. For this analysis, we specifically focused on open issues and did not include feature requests.

Below is a summary of their findings. I share this with the genuine intention of being helpful, as Xojo’s success directly contributes to ours.

=========

This report examines Xojo’s bug tracking system, focusing exclusively on non-feature request issues, and provides actionable insights to enhance software quality. While it’s often said, “All software has bugs,” treating this notion as an excuse for inefficiency is naive and detrimental. Bugs are an expected part of software development, but a mature process and a disciplined approach can minimize their impact and maintain customer trust.

Key Findings

  1. Volume of Non-Feature Request Bugs• Xojo: 2,377 open “non-feature request” bugs.
    • Industry Benchmark: Medium-sized software companies typically have 1,000-1,500 open bugs.
    • Assessment: Xojo’s backlog is well above average, signaling higher-than-acceptable technical debt. The industry reality acknowledges bugs, but this volume suggests inefficiency, not inevitability.
  2. Aging of Bugs• Xojo:
    • Over 2 Years Old: 56.6% (1,346 bugs).
    • 1-2 Years Old: 21.8% (518 bugs).
    • Less than 90 Days Old: 7.3% (174 bugs).
    • Industry Benchmark:
    • Over 2 Years Old: 20-30%.
    • Less than 90 Days Old: 20-30%.
    • Assessment: Bugs lingering for over two years reflect poor prioritization. While all software may have bugs, sustained neglect only erodes product quality.
  3. Average Bug Age• Xojo: 1,021 days (2.8 years).
    • Industry Benchmark: 400-600 days (1.1-1.6 years).
    • Assessment: Nearly double the benchmark, Xojo’s average bug age highlights significant room for improvement.
  4. Bug Update Frequency• Xojo: Average time between updates is 732 days (2 years), with 1,563 bugs receiving multiple updates without resolution.
    • Industry Benchmark: Updates typically occur every 60-90 days with steady progress.
    • Assessment: The extended update intervals and lack of closure suggest systemic inefficiencies.
  5. Oldest Non-Feature Bug• Xojo: 5,791 days (15.86 years, created January 24, 2009).
    • Industry Benchmark: Rarely exceeds 5 years.
    • Assessment: Allowing a bug to persist for over 15 years is not reflective of the claim that “all software has bugs” but rather of inadequate resource allocation.

Debunking the Myth: “All Software Has Bugs”

While true in a literal sense, this phrase often becomes a shield for complacency. The presence of bugs is not an excuse for tolerating inefficiencies. Industry leaders demonstrate that with the right processes, bugs can be managed proactively and resolved efficiently. Xojo’s current practices can be optimized to ensure that bugs are not just identified but systematically addressed in a timely manner.

Strategic Recommendations

  1. Immediate Actions• Prioritize Older Bugs:
    • Dedicate resources to resolving the 1,346 bugs over two years old. Focus first on those with the greatest customer impact.
    • Establish a Triage Process:
    • Categorize bugs by severity and customer impact for targeted action.
  2. Mid-Term Optimization• Create a Dedicated Resolution Team:
    • Assign a team to systematically address long-standing issues using agile principles.
    • Enhance Accountability:
    • Ensure all bugs are assigned with clear ownership and deadlines for resolution.
  3. Long-Term Process Improvements• Automate Tracking:
    • Adopt real-time bug tracking tools to provide visibility and accountability.
    • Set KPIs for Bug Resolution:
    • Target an average resolution time of 400-600 days and resolve 70% of new bugs within 90 days.
    • Balance Resources:
    • Allocate sufficient focus to bug resolution alongside new feature development.

Projected Outcomes

Through disciplined execution, Xojo can transform its bug management approach:
• Reduce the backlog of aged bugs, restoring product stability.
• Shorten resolution times to align with industry standards.
• Build customer trust by demonstrating a commitment to quality.

Conclusion

The sentiment that “All software has bugs” is not a justification for inaction but a call to excel in managing them. By addressing inefficiencies, Xojo can exceed industry benchmarks, deliver a more reliable product, and elevate customer satisfaction.

4 Likes
2

At least that text won’t just disappear. Thank you.

Pretty funny that we already knew all that. Nice to see we’re all not insane though.

THIS is why I dropped Xojo. Begged and begged.

PHP/MySQL/HTML/JS/CSS doesn’t have this issue.

2 Likes
3

Dead.Horse

1 Like
4

This is quite inaccurate…

I’ll fix this:

Assign a team, not consisting of the CEO to systematically address long-standing issues.

1 Like
5

Some interesting info from the past:

1 Like
6

@jay You had to know you’d get this kind of push back and deflection

I think the problem here is the assumption that you can statistically determine success from bug reports. I am unconvinced that is possible.

The point of your original post wasnt “is the business a success”
It was “Is their bug tracking and fixing strategy successful”
Has nothing to do with supporting windows linux pi etc

Their PUBLICLY available data suggests their existing strategy is not working
THAT is what Geoff missed

1 Like
7

I don’t think he missed that. I believe that it is his strategy. He can live with it as long as he gets new customers. If that ends up, Xojo ends up. Simple.

1 Like
8

The 3 D’s ?

Deflect. Defer. Disregard ?

1 Like
9

Na that’s not the point. To follow the wishes of the pro users he would have to spend much money. Tp spare this money it is a nice way not to do this but add features on features. This leads directly to a simple situation: adding features makes the product attractive for users which are new to the product and want to be able to program cross platform. So you have a system you can provide without any problem for all platforms. People believe this and with simple projects it works also. When and if this user will need real professional features and hits the borders: it is a user you will loose in this situation. But the new ones you get.

I am not believing that this system will run further 20 years but I believe that it will run 5 further years. And when it ends up: it ends up. That’s it. There will be the day with a big bang and this galactic feature collection will implode. The users will not have a product anymore, there will not be any fix for any bug and the end is there. For him it makes no difference. The best would be: when he is retiring and can sell this consortium of bugs before for really much money. That’s life. And I can even understand why he is doing that. There is no market anymore for this product in a professional market. People using flutter/dart for mobile projects. They are using one of the javascript solutions for their eb projects or PHP like Hal or even Java with Vaadin.

For Desktop the people are using more and more C#, Java, C++, Kotlin. Where should now be the place for a big stream of Xojo users? there is non. And to speak truths: there was even none. That exactly is the problem. It was never a top 50 product. And it had never even one percent of the users the worst of top 50 has.

And also there will be no chance for open sourcing or anything else. Even if many people believing in that. If somebody would first time build it and get how long it takes to build and in parallel would build Netbeans, Eclipse or IntelliJIdea he would immediately get: it is out of scope for todays times.

10

Ah I see what you’re getting at

I had tracked the influx of reports (not specifically bug reports) for some time
What I did not track was the lag time between report & update/comment or resolution
With public reports that is possible and I assume thats what these guys did

Its how I came up with the numbers I reported in

https://www.great-white-software.com/blog/2021/11/14/numbers-3/
https://www.great-white-software.com/blog/2022/04/06/numbers-2022/
https://www.great-white-software.com/blog/2022/05/21/numbers-2022-05/
https://www.great-white-software.com/blog/2022/06/08/numbers-after-the-change/
https://www.great-white-software.com/blog/2023/07/19/one-year-later/
https://www.great-white-software.com/blog/2024/01/02/one-year-on/

The upshot an average of 10+ new reports per day (of all kinds)
Lots unreviewed (nearly 400)
Android reports were in need of review (> 30% of total)

Even if only 50% pop those reports were bug reports they collect 400+ in a cycle and fix fewer
So you end up with this

Screenshot-2024-01-02-at-7.05.08-AM.png
Screenshot-2024-01-02-at-7.05.08-AM.png768×429 33.1 KB

In a way I’m surprised Jay brought this up with Geoff yet again

Geoff has repeatedly demonstrated either a lack of understanding, or lack of concern, and this conversation is a great example
Jay NARROWLY titled it the way he did

Analysis of Xojo’s Bug Tracking System

Not “how xojo measures success etc”
but thats what Geoff took away from the critique
As apparently have others

When you try to get them to look at the sky and they look at the ground and tell you its all pebbly and there’s no blue in it then wtf do you expect ?

:man_shrugging:

2 Likes
11

Complete waste of time, in my opinion, because you know Geoff will refute everything and refuse to change.

2 Likes
12

I tend to agree

The recommendations have all be gone through
Change the release cycle from bugs & features in every release to
Bug fixes
Features
Bug fixes
Features
Geoff doesnt want to do this as it means slower release of features
but the same biotches & moans about the current process have existed since the outset
That should tell them something but apparently they’re willing to ignore that and endure the arrows yearly

More developers (thats ALWAY been true)
Even their own devs have said that one
Always rejected with various justifications

Just fix the PROCESS and prioritize BUGS over FEATURES
Its just a matter of time before most everyone runs into those bugs so the longer they sit the more people they hit

whatever
Geoff knows better than we do what we want
Literally has said that

So aint holding my breath
I’m sure @jay has heard all this

1 Like
13

That thread on the Xojo forums is hilarious.

People are begging to get bugs fixed and Geoff doesn’t respond to @jay. Instead he replies to Eric along with an irrelevant blog post.

It’s a huge FU to anyone stuck with a bug that Geoff just doesn’t care to fix. How could ANYONE use a product like that?

Then later Geoff says people will vote with their wallet. “Xojo is not for Pros” really seems to mean, Pros want bugs fixed and that’s expensive to Xojo.

image
image818×189 20.7 KB

That tells me Xojo should not be used for anything that should be mostly bug free. Bugs are not important to Geoff, based on that insane quote.

It’s nuts.

And then the Xojo koolaide drinkers chime in talking about everything but the bugs.

:exploding_head:

3 Likes
14

It all comes down to:

Software engineers would much rather work on new features than fix bugs.

My spouse ran a volunteer program restoring natural areas and had the same problem: huge turnout for new planting events, tiny turnout for subsequent weeding events. Most of those “restored” areas have gone back to their original state.

15

Reflecting, Geoff said vote with wallets is the only thing that matters.

Stop renewing.

5 Likes
16

Geoff, and supporters, have missed the fact that the problem is, as far as my experience goes, not the software they use to track bugs and not how they pick what goes into any release

Its the process they dont have that creates bugs that get released that then go unfixed

Fix that and they’d correct a LOT of things
There would be fewer bugs to fix in the first place

3 Likes
17

This

2 Likes
18

I’m curious… what other products would Geoff say are the direct competitors to Xojo?

1 Like
19

Indeed. Not having requirements before coding begins, not having code reviews, not having unit tests, not having proper regression testing, not having a proper QA team and relying on external testers (i.e. the prerelease list), to name a few things, all lead to bugs getting into the wild.

5 Likes
20

Xojo has THE BEST new user experience.

But the bugs kill the deal.