From Continuous Software Engineering Reference Ontologies to the Integration of Data for Data-Driven Software Development

Abstract

Context: Software organizations face several challenges, such as the need for faster deliveries, frequent changes in requirements, lower tolerance to failures, and the need to adapt to contemporary business models. Agile practices have allowed organizations to shorten development cycles and increase customer collaboration. However, this has not been enough. Organizations should evolve to continuous and data-driven development in a continuous software engineering approach. Continuous Software Engineering (CSE) consists of a set of practices and tools that support a holistic view of software development with the purpose of making it faster, iterative, integrated, continuous, and aligned with the business. Software organizations often use different applications to support CSE (e.g., project management tools, source repositories, and quality assessment tools). These applications store useful data to enable a data-driven software development process. However, data items often remain spread in different applications, each adopting different data and behavioral models, posing a barrier to integrated data usage. As a consequence, data-driven software development is uncommon, missing valuable opportunities for product and process improvement as well as new business opportunities identification. Objective: Considering the need to enable data-driven software development in the CSE context, we aim to provide an ontology-based approach that can aid in: identifying the organization’s information needs, retrieving data from applications, and providing integrated data that meets the information needs. Method: By following the Design Science paradigm and organizing experimental studies as learning iterations, we developed the Immigrant approach, which contains three components: California (a System-Thinking-based process), Zeppelin (a CSE diagnostic instrument, which helps identify the organization information needs), and The Band (an ontology-based integration solution that semantically integrates data from applications and, thus, provides integrated data to support data-driven software development). The Band is based on Continuum, an ontology (sub)network developed in this work to address CSE aspects (particularly, agile development, continuous integration, and continuous deployment) and that is used as a reference model to build software artifacts in the integration architecture. Results: Studies performed in software organizations evaluated each component separately. Results demonstrate California and Zeppelin’s usefulness and show that the integrated solution (The Band) contributed to improving estimates, provided data that helped allocate teams, manage team productivity and project performance, and allowed to identify and fix problems in the software process execution. The complete proposal Immigrant was evaluated in a case study. As a result, it was possible to identify problems related to the allocation of tasks, role overload, and code quality. Conclusion: The results obtained so far suggest that Immigrant is a useful approach to enable data-driven software development in CSE.