Table of Contents
A cautionary tale of navigating modern development
Search the Internet for “Open Source” and the number of hits is astonishing. The fact that open source has been accepted into the business mainstream is indisputable, as IBM’s purchase of Red Hat for US$34 billion proves. Or maybe it’s the rise of more or less open source Linux in its various distros, especially in servers, in the cloud, and in a large number of other devices that has fueled this movement. Certainly, businesses and governments have recognized that being able to change and adapt source code themselves is a benefit. The entire Internet was and still is largely developed using open source, and so it would seem that not only is open source here to stay, the future of computer science in part depends on the wise use of this software.
But what’s missing? Why do companies that cut costs in a relentless drive for efficiency still choose proprietary or commercial software? What are the benefits of using open source, and what are the pitfalls and concerns? What is missing if you, as a software architect, a designer, a developer, or engineer choose to use open source software?
I want to offer answers to these questions using my own personal experience as Chief Technical Officer of an innovative start up company that faced serious funding and schedule pressures (as most do). Since free and open source (F.O.S.S.) tools have become mainstream, there are best practices to take advantage of open source and limited commercial versions, while solving some of the more unnerving challenges developers and businesses face today with open source.
In 2007 I decided to convert my hardware and software design services company into a product company to deal with the great recession and to obtain venture funding. Our new company decided to focus on telehealth for chronic disease patients, and included sensor devices, an innovative tablet platform built from an open source hardware reference design which we extended significantly.
We also created a complete electronic health record/electronic medical record web-based Clinician Access System to be able to set thresholds for the incoming telemetry to trigger alarms, and to provide a database for the billion or so bits of tiny data coming in from as many as a hundred million potential patients. The system was developed with a mixture of PHP, open source MySQL, and other technologies – some home grown, some open source.
Like all good development shops, we were constantly driven by process improvement, seeking ways to reduce our workload, and especially driven by a need to ship the product, start selling systems and licenses, and reach profitability. In order to meet schedule demands, we had to innovate, and we had to be smart about what we invented, and what we borrowed.
We all know that the open source world has come a long way and has become an entirely mainstream alternative for a lot of projects, and even some real products (especially those using GCC, Linux, and so on). But would there have been a better approach, perhaps a hybrid approach, that would deliver the benefits of open source (faster prototyping and demos) but with the robustness and technical support of a commercial offering?
These days, hybrid models are popular. For example, hybrid Cloud Computing that includes some servers inside a company (or a private Cloud) combined with a public Cloud make an enormous amount of business and technical sense, so much so that companies such as IBM, Dell, and HPE have embraced this as their primary Cloud selling strategy. Is there an analogue here for the web and mobile developer?
Open source licenses can be tricky
Learning to deal with various types of open source licenses was a challenge, from GPLv2, GPLv2, to Apache and BSD. Because we modified the design of our hardware reference platform (which was an open source project), we knew that any changes we made to adapt the various open source bits had to be submitted to the various open source projects, which was always a concern, especially for our investors
Our software stack had Open Boot firmware, Angstrom Linux (later we changed to Android), a mixture of open source and proprietary device drivers, our own proprietary sensor engine to control various medical equipment devices, and we used a browser on the tablet platform for our look and feel (perhaps one of the first uses of browser technology in an Internet of Things realm to control the user interface). We had a mixture of open source and home-grown UI widgets. We had to invent a way to keep track of which source repository had which code, and who was entitled to our modifications. With the release of GPLv3, some of our open source concerns would have been alleviated, but the fact still remains that different kinds of projects in GitHub or other open source repositories use(d) different licenses. These days there exists GPLv3, MIT, copyleft, and others. Not every project uses the permissive MIT license.
What we would do differently: lessons learned
Implications for raising capital
The concern about which code we owed back to the open source community, and which source was proprietary (if not a trade secret), affected our ability to raise capital, and was not an easy task during this time. Investors want to invest in companies that have clear competitive advantages, such as a unique business model and, for a software intensive company, software intellectual property (IP). But using open source meant giving up some of our inventions. We spent a considerable amount of time trying to work around this, but the fact remained: our unique software used some interesting modifications to Linux device drivers, and that had to be contributed back to the community – where our competitors could see what we invented. That’s why using a hybrid model made sense. Today this concern is reduced by an MIT license attached to a project, but of course not every open source code base uses the MIT license. The fact of the matter is, you must keep track of all licenses.
We found that one of the best uses of open source was in creating a working Proof of Concept (POC). To better explain to investors our concept and our unique innovations, a presentation deck was really no substitute for a POC that showed how the system would work. This POC allowed potential investors to kick the tires and learn how our solution, based somewhat on open source but with a substantial amount of our own software intellectual property, was best suited to take advantage of the coming telehealth and telemedicine explosion. If investors believed that we could pull this off with very few employees by leveraging open source, we believed we had an advantage – so long as we protected the inventions, the intellectual property we were developing.
By promising them that we would re-write our “secret sauce” (software innovations) and create a protected source environment for those, we mollified the anxieties of our potential investors. We also created the potential to derive licensing revenue from our intellectual property. There is no way a company can or should license open source, because who would pay for something that could be obtained completely free? Naturally we had to organize our source code to do this.
In general, the fewer sources of code, the easier it is to keep track of to whom you owe code back to. Carefully annotating your code is wise in case someone in the open source world insisted that we contribute back to the repository.
Open source giveth, open source taketh away: support and documentation falter
We ran into difficulties with some of our open source software used to perform video encoding and decoding for video conferencing. Unfortunately, our processor vendor couldn’t help us: they were relying on the open source community to provide basic functionality and technical support for multimedia, and at that time the community’s code was almost non-functional. The open source multimedia framework got us about 80% of the way to completion, but then just dropped us off the face of the Earth. We couldn’t seem to make it all work in the way our microprocessor vendor said it should, and technical support was by people uninterested in our particular application of the technology.
Asking questions on various forums didn’t get answers very quickly, if at all (such as how we program the process to do 30 frames per second at VGA resolution with good performance). This is another perverse example of the 80/20 rule: you get to a certain point in 20% of the time with 80% of the functionality, but the rest of the 20% takes 80% of the time!
About documentation: the comments in the source code didn’t really cut the mustard. Documentation during those years was slim to non-existent, and of course there was no one answering a phone call with a technical support request. If I had used a commercial vendor I could have relied on the excellent documentation, product examples, and technical support instead of waiting (sometimes forever) for answers to my most vexing questions. There were literally weeks that we didn’t get answers from the open source communities, and while this has really improved over the years since we created this enterprise-level system, being able to pick up a phone and get answers quickly would have saved a lot of time.
Examples and Sample Code
Using open source developer tools such as GNU GCC worked well, but of course there were no example or sample programs to adopt for our application code. We tried doing all the work on the back-end server and used an embedded web browser to display the screens, but this hampered our ability to handle hundreds of thousands of transactions; the state of the art at the time simply didn’t allow for React or Angular responsiveness. These days, this technique of using a browser on the client side of a high-end embedded tablet device, or better yet progressive web applications, would be the right approach. We were just ahead of our time, often a lethal position in engineering!
The user interface design we created was innovative, especially for elderly chronic disease patients, but it took a very, very long time to design and code all of the widgets. From the Clinician Access System perspective, medical professionals in the chronic disease market are used to a certain look and feel for their graphs, their charts, and the data. Because we were accumulating a very large amount of small bits of data (medical telemetry and other measurements), it was important to be able summarize data for quick viewing and understanding by clinicians. It would have been much smarter to have used a product like Ext JS, ExtReact, or ExtAngular, and would have saved us many hours (months) of design, coding and testing.
Sencha has extended the open source frameworks of React and Angular into areas that can be readily leveraged by developers. For example, ExtReact and ExtAngular products provide over 115+ user interface components each and were created to work with React and Angular.
Getting started with prototyping using free downloads is a great way to quickly experiment with technologies. All of Sencha’s products offer free trial versions, and recently Sencha came out with the Ext JS Community Edition, which makes a lot of sense for prototyping as well. Had I done this with Waldo Health, I would have been able to show my investors real progress and the vision of the products and services. I could have then easily switched to the fully supported, commercial/professional grade version and not lost compatibility. Commercial companies like Sencha offer superior technical support, professional services, documentation, and consistent testing against new versions of browsers and new versions of Android and iOS.
How to leverage Ext JS Community Edition
Would I do it all over again? Yes, but I would likely choose better technologies backed by a real company for technical support, documentation, examples, training videos, and so on. The hybrid model of using open source for non-differentiating technologies combined with the lessons learned here would have likely resulted in a more successful company.
Chris Alfano says
Where is it you could find Sencha experts to staff your team with without training them yourself to replace those PHP developers that trained themselves and are all over the place?
This missing variable here is the cost of building experience. In an open technology ecosystem, the cost of experience is shared by the developer themselves and an array of customers. It is worth a lot more to a developer and a much lower risk to invest in building their own skills in an open platform.
This is Sencha’s biggest struggle. Enterprises have figured out they’re on their own finding and training talent, and even when they succeed in doing that they’re still left with beginners building their first complex project. Disasters abound. The only demand for Sencha expertise these days in maintaining messes the first generation of those built. I *want* to drum up more demand and capacity but am hamstrung by the core SDK’s encumberments.
Without an open model for the lower level end of Sencha’s technology, it will never make sense for developers to invest their own time in mastering it, and there will never be a rich talent pool. Make your money building and supplying the higher-level process support tools that you make an excellent case about businesses needing, and make the framework/SDKs themselves something there’s a credible case for developers to invest their time in learning.
Sencha’s frameworks still have concrete technological advantages over alternatives, that won’t last forever. The window for building a strong developer user base is closing quickly
Chris Alfano says
Microsoft has figured this out with their new approach to .NET Core and Visual Studio Code. People with business on the line and big projects to ship still buy their tools and services on top of that, and thanks to that move their ecosystem is growing instead of collapsing.
I implore you to follow their lead! Not because I want free Sencha, but because I want to see the demand for Sencha solutions and the market of talent grow.
Robert Henniger says
Very good and valid point Chris
I agree. Mr Weiss says: “It would have been much smarter to have used a product like Ext JS, ExtReact, or ExtAngular, and would have saved us many hours (months) of design, coding and testing” which can only be true if you already have experienced EXTJS developers. If not, the time it would take for them to master it would offset any time saved in the first project. The time savings come only in the next EXTJS project.
Alan Weiss says
I guess what I meant was that we had to write our own UI widgets and components, and that took a huge amount of time and frankly a lot of trial and error. In health care, being able to visually scroll through data values with a sliding window, able to zoom in and out, is critical to tracking patient health trends, and being able to even correlate different trend lines. Sencha has a UI component called Navigator available in Ext JS, ExtReact, and ExtAngular that does exactly this. It would have saved me time and frustration if I had used Navigator, that’s for sure.
Sorry, there is no component named “Navigator”. I assume you mean Ext.navigation.View then I’d also say it’s not a very complex component…
Alan Weiss says
Thank you for your thoughts, Chris. I guess you’ve heard about our new Ext JS Community Edition, right? It’s a start and we need to do more.
Chris Alfano says
I have heard about it, and no it is not at all start. Suggesting that it is demonstrates a fundamentally flawed view that open source is a free trial for commercial use. Please reexamine this and try to understand that open source is not a form of trial.
My open source projects are the only place I still get to use ExtJS, and I’m stuck using an old release and fixing bugs in the framework myself that have already been fixed. That does not signal a platform worth learning and investing in to _anyone_. Instead of being an onramp for your community and source of more comprehensive real-world examples than you could churn out in-house, you’re forcing migrating or forking to be my only choices
An actual open source project cannot make use of any framework release since the last GPL release. No amount of trial anythings make it an option. It’s in your own terms. Actual open source projects using your framework are the most valuable assets you could cultivate to grow the value of this acquisition and you’re disarming your own advocates with this continued active neglect. People who want to pirate the framework for unpaid commerical use can do so easily enough without you making active GPL releases, you’re only harming people who comply with licenses and are trying to build your mindshare
I do hear that thr demand in Chicago for Ext JS projects is non existent…
A person who never used Sencha products and doesn’t know their limitations, says it’s a good idea to use the products… Everyone should definitely follow the advice.
Alan Weiss says
Sorry, what I meant is that using commercially-supported, tested, and cross-platform UI components would have been better than trying to invent the wheel ourselves. Yes indeed there is a learning curve, you’re right.
It can be but there are two sides to that coin in that rolling your own framework can be liberating and energizing. Like it was game changing for this company: https://medium.com/kenshoos-engineering-blog/should-you-migrate-fea360077aaf?_branch_match_id=475323378214671584
Richard Styles says
Not quite sure what to make of this post. It seems to suggest open source is good, but have to be cautious of different open source licences. That we should pay for proprietary software but only with a handful of vendors?
Has anyone read the Sencha licences and the requirements needed to maintain the support?
The mention about PHP seems to suggest Mr Weiss used it back in PHP4/PHP5 days. To suggest the ecosystem hasn’t evolved to include enterprise testing tools is laughable. Thanks to composer PHP7 (and soon 8) has several frameworks which include these free testing and debug tools Symfony, Laravel frameworks to name a couple.
Free and Open source tools are very useful, It’s far easier to find a developer who has freely tinkered or worked with one. Who have been able to develop and hone their skills without needing to pay for a licence after ’30 days’ or jump through complicated hoops.
Thank you, very much editor.
“Asking questions on various forums didn’t get answers very quickly, if at all …”
I hope Mr Weiss sees the irony in this statement. If he doesn’t, he has never visited the Sencha forums or used the “paid” support portal.
Alan Weiss says
I know. We’re going to do better. We need to do better.You can call me “Alan” by the way. My kids do (they’re grown though).
Doug Bieber says
Interesting. From having used open-source in the past with much waste of time and frustration I’ve adopted the one hour rule. If I can’t get the damned thing to work in one hour it’s not worth it.
Open-source has come a long way in the past 20 years. It’s curious how Sencha has to tight-rope walk the legalities and competition. This blog is appreciated.