In part 1 of this article, we concluded that successful development of OpenFOAM demands the resolution of issues where the software does not meet reasonable expectations of users; and that agile development, involving collaboration between users and developers to resolve issues, is the best strategy for development of OpenFOAM. In December 2014, agile development of OpenFOAM emerged following the public distribution of the development line, OpenFOAM-dev, when it quickly attracted new contributers to OpenFOAM and stimulated activity on the OpenFOAM Issue Tracking system. The benefits of greater user participation became apparent, and by getting more users involved the project could “maximize the number of person-hours thrown at debugging and development”.1 Users need encouragement to participate because, as homo economicus, most only do so out of self-interest. The best form of encouragement, to promote user participation, is rewarding issue reports with timely resolution.
Unresolved Issues
The number of unresolved issues is a critical metric that drives greater user participation. The following graph shows the number of unresolved issues taken from the Issue Tracking system, plotted as a 3-month running average to smooth the data. The coloured bands indicate the (group) company employing Henry Weller, the creator of OpenFOAM, who still resolves the vast majority of OpenFOAM issues today. The release dates of major versions of OpenFOAM are marked on the graph.
Note: data corrected on 2017-04-18
The change in number of unresolved issues following the release of a particular version of OpenFOAM is a strong indicator of the health of that version. The highest rate of increase occurred following the release of version 2.2.0, a notoriously unreliable version which required an unprecedented release of a second “bug-fix” version 2.2.2 in October 2013. Version 2.3.0 was not much better, doing little to halt the steady increase in unresolved issues.
Public Development Line
Following the public release of OpenFOAM-dev in December 2014, Weller relentlessly resolved a large number of issues, supported by a growing number of enthusiasts, resulting in the release of version 2.4.0 in May 2015. It was a significant improvement over the previous version, not only because of the reduction in outstanding issues and that an increase in issues following release was very brief, but it also included changes to make OpenFOAM easier to use. Subsequent releases of version 3.0.0 and version 4.0 were not followed by an increase in outstanding issues. This apparent improvement in reliability was despite the fact that they included some major new functionality and redesign of major components. The rewrite of post-processing in OpenFOAM v4 is a good example of a remarkable development that resolves issues of design, usability and maintainability for the benefit of users and developers alike. Agile development has arguably taken OpenFOAM from its lowest point, at v2.3.0, to a new level in v4.0 — in just 2½ years.
Issue Resolution Time
The resolution time for issues is the other critical metric to drive user participation; the hope and enthusiasm generated by reporting an issue rapidly diminishes when there is no resolution in sight. The following graphs show the percentage of issues resolved in 1 day and 1 week respectively, plotted as a 1-year weighted-average to smooth the data, with the same colour bands as before.
The graphs show the number of issues resolved in 1 day has risen from 30% at the time of OpenFOAM v2.3.0 to over 70% for OpenFOAM v4.0. Resolution within 1 week was consistently below 50% for v2.3.0 but has risen to almost 90% for OpenFOAM v4.0. The improvement in issue resolution time must surely result in better quality software, supported by greater user participation.
Software Maturity
In 2014, while experiencing the problems of OpenFOAM v2.3.0, we were guided by a popular Software Development Maturity Model. The model provides a table containing activities relating to software maturity, which are categorized into 3 levels from the lowest level, 1, to level 3 which describes a fully mature software organization. In 2014 it was concluded that the OpenFOAM project was at level 2 and there was no clear path to level 3.
Today, OpenFOAM conforms to the majority of descriptions in level 3. In particular, OpenFOAM can certainly be described as always releasable quality, with OpenFOAM-dev continuously tested and building successfully. Code changes in the OpenFOAM Git repository are linked to issues in the Tracking system. The build of OpenFOAM-dev was recently automated and packaged for Ubuntu to eliminate a significant barrier of code compilation to its users, to encourage greater participation. The descriptions of shared, collective ownership of code, code style compliance, continuous integration and metrics all hold up and the entire approach to project management is “agile” and “efficient” and provides the “ability to be opportunistic” and “re-prioritize requirements efficiently”. Agile development has yielded a fully mature software organization behind OpenFOAM.
Summary
- Agile development of OpenFOAM emerged following the public distribution of the development line in December 2014.
- It relies on user participation, which is encouraged by rewarding users for reporting issues with timely resolution.
- OpenFOAM was in poor health at version 2.3.0 with unresolved issues increasing rapidly and long resolution times.
- Today, unresolved issues are under control, with ~70% of issues resolved in 1 day and ~90% resolved in 1 week.
- Agile development transformed OpenFOAM to a new level of quality and maturity (always releasable) by version 4.0.
References
- E.S. Raymond (2000), The Cathedral and the Bazaar.