‡ Software Production Engineering Lab (LIPS),

Since reuse costs are often higher than expected, several approaches to development with reuse have been proposed. Among the most interesting is the idea of frameworks: sets of single components collaborating according to predefined cooperation rules. This approach can have a great impact on the reusability of parts of a software system, but it is not an easy task to give a quantitative measure of this reusability. The existing reusability metrics can be used, but the results are not fully satisfactory, since they do not take into account the peculiar characteristics of a framework and the particular way it is reused (a mix between black box reuse and white box reuse). For these reasons, a set of criteria adapting the traditional reusability metrics is proposed. These criteria try to evaluate the four main factors affecting the reusability of a framework: portability, adaptability, understandability, and confidence.

In 1994, Barry Boehm presented an impressive study of reuse costs in NASA projects. As a result, he showed the relation between the percentage of changes to a single component in order to make it reusable and the costs of these changes relative to the development from scratch (figure 1). The diagram puts into evidence at least two intimidating factors. First, reuse of a single component without any change has its costs: a software engineer is requested to identify the reusable asset, to understand its interface requirements and to determinate how it can be properly plugged into the host system’s architecture. Second, only few changes (12%) to the component raise the reuse costs to 55% of the costs of development from scratch. This negative effect is not only due to the changes, but also to the required understanding of the internal structure of the component. Even if Boehm’s study was conducted in NASA’s software engineering lab, the results would be similar in any other technical or business domain.

The bottom line is that simply using an object-oriented language – which supports adaptation of existing software through proper elegant mechanisms – instead of a traditional or non object-oriented one does not lead to dramatic reusability improvements. In this respect, it is worth noting that in the few cases in which single component reuse can be successful, the market has sometimes selected alternatives to the pure object-oriented approach. An example are container components and algorithms, which are reused without adaptation and which do not require a substantial understanding effort by the reuser, simply because the external interface provided is part of the common knowledge of every programmer. In fact, the Standard C++ Template Library (STL) supplies both of them as generic templates, rather than using object-oriented capabilities.

To achieve a substantial improvement of the reusability level, a completely different development approach is needed. For this reason, most successful object-oriented projects are shaped upon the idea of framework engineering for reuse. Frameworks are single collaborating components with predefined cooperation among them. The architecture of the framework is explicitly designed so that some components can be replaced with others or refined to meet new requirements.

Whereas a single component is usually thought as the solution to a very specific and finalized need, a framework is the answer to a potentially large set of domain complex requirements.

Whereas adapting a single component to a new situation usually requires a complete understanding of its internal structure, adapting a framework means substituting some clearly identified components with new ones, built by refinement of existing base classes.

Whereas programs built on top of single reusable components reuse their source code, systems built on top of a framework exploit source code and architecture reuse.

For all these reasons, framework engineering seems the most promising approach to software reuse. With frameworks, a single effort in the search for reusable asset can discover the answer to a large set of domain requirements. The reuser is not requested to embrace all the knowledge hidden in the framework architecture, usually a set of complex interaction among domain and software-specific objects; rather he must only handle the limited complexity of the predefined adaptation points, often called hot-spots. The extension of the framework is done programming by difference (using subclassing) on the basis of the provided components.

Patterns are the software-engineering concept most often coupled with frameworks. Object-oriented design patterns are structures for capturing and expressing design experience in terms of practical solutions to recurrent problems[3]. Descriptions of design patterns nearly always comprehend an indication of the problem which they address, of the context inside whose scope the pattern solve the stated problem, of the fundamental trade-offs which shape the pattern’s structure and of examples of the pattern application. Differently from frameworks, they do not define a complete system since they only solve a specific kind of design problem. Moreover, they are not bound to a particular programming language as they simply expresses design experience preserving the idea behind a given design subset.

Patterns are discovered in most of well-designed software. In this respect, frameworks are not exceptions: design patterns are usually applied in building the architecture of frameworks and in shaping the adaptation mechanisms in the hot-spots. In particular, the recurrent structures found in hot-spots are usually classifiable into one of seven combinations called meta-patterns[6]. Meta-patterns have a simple structure, consisting basically of template methods defined on the basis of smaller hook methods. Hooks parameterize the behavior of the template, so that overriding the hooks (by means of subclassing) a reuser can easily adapt the template to changed requirements. A detailed explanation of the intent, organization and practical use of meta-patterns can be found in [6] or [7].

At a general level, meta-patterns are important because they describe the fundamental idea behind a specific adaptation mechanism situated in a framework hot-spot. Software engineers familiar with the generic organization of a given meta-pattern easily understand its specific instantiation in a particular framework and as a consequence the adaptation of the hot-spot is made simpler. In other words, they provide a fundamental ontology summarizing in a concise and general view the important features of several adaptation mechanisms.

Unfortunately, compared to normal ad-hoc development, framework production requires an enormous development effort. The costs are usually significantly higher than the development of a specific application. In this respect, frameworks are long-term investments that produce an evident pay-off only if they are instantiated in several applications in the same domain. Moreover, framework technology is not yet mature: there exists some unsolved problems regarding framework interoperability, language bindings and poor tool support. The right choice of the structure of the hot-spots provided by the framework to the reusers is another fundamental issue, which has not been completely explored. On the other hand, only framework reuse provides large-grain domain-specific components with the size of an architectural subsystem, which can be assembled to build complex applications.

In this context the role of reusability measurement is fundamental. The assessment of the reusability of a framework is the basis for whatever economic evaluation of a corporate reuse strategy. It provides the foundations for the analysis of the activities that are going on and the identification of corrective or improvement actions. Without a reusability assessment only a post-mortem analysis of development with reuse (for instance by means of productivity records) is feasible. Lacking fundamental information, it is not possible, at least at the beginning, to formally distinguish between the alternatives of development from scratch and with reuse. Moreover, the framework engineering team is not able to identify clear evolution directions to foster the possibility of reuse of the collection of components.

In the light of the peculiar features of object-oriented frameworks we analyzed the existing approaches for software reusability. What turned out, however, is that the existing proposals for measuring software reusability are not applicable to our context, because they express features which are mainly applicable to single component reuse. The main problems are:

• An excessive interest in the entire structure instead of a focus on the adaptation points of the software component. Most reusability models are thought for general reusable assets, expressing the view that during design with reuse a developer must consider, understand and change the global interconnection of components. This kind of white-box reuse can hardly be defined as economic. As already examined, frameworks, instead, provide adaptation points which permit low-cost local modifications to meet new requirements. Despite this fact, the existing reusability assessments do not explicitly take into account hot-spots. Examples are the software reusability assessment proposed by Prieto-Diaz and Freeman in their paper on faceted classification [8], which considers program size of the entire structure, and the NATO Standard for Software Reuse Procedures[9], which contains the global cyclomatic complexity (as defined by McCabe[5]).

• The application of metrics especially to code instead of design. The design of a framework (indeed, of every large subsystem) is much more important that its actual code. A rich deal of information is evident in the design and disappears hidden by implementation details in the code. The basic mechanisms providing extensibility must be built during design rather than developed during the implementation. Moreover, a reusability assessment at the design level allows structuring the project of the software asset in an optimum way before starting coding. This is important because in every situation the planning of future reuse activities at an earlier stage means more cost savings. Several metrics commonly used in reusability assessment, such as size expressed in lines of code, are not suitable for an assessment at the design level, simply because no lines of code are present. Other functionality measures, such as function points, can instead be used during design. With the definition of the Unified Modeling Language (UML)[1], a standard-de-facto exists for software design, so that a general description of design-level metrics can practically exploit common glossary and terminology.

• A low interest in patterns, one of the fundamental concept in framework construction. In opposition to most of the existing reusability models, a framework reusability assessment should explicitly consider hot-spots, template and hooks in the definition of metrics or qualitative evaluation. Moreover, as design patterns are usually applied in defining the architecture of a framework, a reusability assessment tailored to those particular software assets should take also into account the documentation of the patterns. Notwithstanding, no existing reusability model explicitly assigns them a value. The scarce care for this aspect has several drawbacks. The structure of patterns found in most hot-spots and the interconnections of templates and hooks with other classes of the framework is not considered, even if these elements have a direct impact on the coupling a reuser has to manage during the adaptation of the framework. A careful application of patterns in the framework design may substantially lower the adaptation costs needed to reuse the asset, but this is not explicitly taken into account.

According to the REBOOT reusability model[4] the main factors affecting the reusability of a software component can be derived from an analysis of the activities undertaken during software reuse. These activities are:

• Porting the component to a new environment. The first important factor is portability, that is the ease with which software can be transferred from one computer system or environment to another. This concept is directly linked with the independence the reusable component has in respect to the surrounding environment, from the operating system to the used libraries and external programs. It is worth noting that in the specialization of the REBOOT reusability model for software design the portability factor is eliminated, perhaps for the consideration that a design can usually be implemented with a variety of lower level programming languages and tools. On the other hand, we are convinced that the design of a system built with reuse in mind should explicitly consider portability problems, so we include portability as a major factor affecting reusability.

• Adapting the component to the specific functional requirements of the new system. To do this a reuser needs high adaptability. Adaptability is the ease with which a component can be adapted to fulfil a requirement that differs from that for which it was originally constructed. As framework development is aimed at lowering the costs of integration of a subsystem into a new system with slightly different functional requirements, adaptability is perhaps the most important factor in our view. In this respect, frameworks should score higher values than other reusable software assets as regards adaptability.

• Understanding the functionality of the component, to decide whether it meets the new functional requirements and to tune it to the new requirements. To do this successfully we need high understandability. This concept is linked to the effort needed by a developer for recognizing the logical concept behind a software asset and its applicability. As such, this is more a matter of documentation than good software design. The only parts of the framework design affected by this factor are the hot-spots: the mechanisms behind the adaptation points must be clearly stated (with a wealth of examples, scenarios and documentation) and with a relatively low local complexity.
• Ensuring that the reused component does not introduce any extra, uncontrolled risks to the system’s environment. For this we need high confidence, defined as the probability that the software component performs its defined purpose satisfactorily and without failure over a specified period of time when applied in an environment different from the one for which it was originally constructed and/or certified. Usually the impact of this factor on the reusability of a framework can only be evaluated after several instantiations of the framework itself.

On the basis of this fundamental decomposition, it is possible to define a set of more specific criteria allowing the evaluation of framework reusability. Criteria make the discussion nearer to the work of software engineers, linking design and implementation issues to the general factors discussed above which are usually stated at the superior project manager level. At the same time, this specialization step permits both an easier assessment, since criteria are more directly coupled with design elements and source code structures, and a way for developers to modify their work towards an improved reusability of their products, since criteria can also be used for selecting design and implementation alternatives during software production. For this reason, this paragraph will set and discuss a small set of criteria for each identified high level factor. All these criteria take explicitly into account specific frameworks’ features, making a reusability assessment defined on their foundation especially targeted to object-oriented frameworks.

Portability, the first important factor, is directly associated to the framework’s dependencies from external accessible functional subsystems (including in this list the operating systems and the libraries). In general, to achieve a high portability, the number of dependencies of the framework on the surrounding environment should be low. External dependencies obviously include technical issues like particular system calls (e.g. thread and process management), access to databases or object-oriented repositories, loading and use of external libraries. Even non-technical issues, on the other hand, can raise limited portability problem (e.g. a particular look-and-feel for a user interface or a particular style of programming). More specifically, it has been shown[3] that a major software composition problem is due to architectural mismatches. They center around the assumptions a subsystem does on the overall structure or dynamics of the environment. Typical assumptions are about the nature of the surrounding environments, the connection and time constraints among them, the global architecture and the construction process. All these dependencies should be carefully shown and have an impact on reusability. Moreover, framework developers can manage the portability problem exactly like every other adaptation problem. Hot-spots and meta-patterns provide the possibility for transforming the complex problem of porting in the simpler problem of providing to the framework new modules, specific for the new target environment. Instead of simply counting the number of external dependencies as a qualitative measure of the portability of a framework, a better strategy should so take into account the adaptation mechanisms built in the framework in order to deal with portability problems.

The adaptability of the framework depends from two different specific criteria: generality and modularity. Generality is defined, for an object-oriented framework, as the capability of meeting the requirements of different applications in which the reusable component could be instantiated. This is translated into the qualitative evaluation of the domain coverage supplied by the possibility of adaptation through the provided hot-spots. The adaptation mechanisms placed in the framework hot-spots should be freely tunable in order to follow all domain variable features. Modularity measures how well the developers have managed to split the framework design into disjoint parts having clear and non-overlapping roles and functionality. A judgement on this point is useful during design with reuse especially if made upon the hot-spots providing the needed domain-specific adaptation mechanisms. In other words, a reuser should not have to worry about the modularity of the entire structure, since hot-spots should be sufficient to customize the framework to new needs. What is interesting is instead the modularity of the provided adaptation points. In this respect, our view counters the normal approaches in which modularity is measured on the global structure.

Complexity and documentation level are the criteria associated to the understandability factor. Complexity is linked with the effort needed for understanding the relationships between the framework’s parts. REBOOT distinguishes different complexity aspects, structural complexity versus inheritance complexity. For object-oriented frameworks the situation is somehow simpler, as the effort of the developers during framework reuse is targeted to the hook classes identified in the hot-spots. Due to their peculiarity, the classical complexity metrics (for example McCabe cyclomatic number[5]) lose a great part of their interest when applied to hooks. Instead, programmers building code upon a given framework consider as a fundamental feature the easiness of use of the provided interfaces. As clients of the framework, they simply look for a contract (defined by the interfaces of hook classes) which asks little and guarantees a lot. Documentation Level is a qualitative evaluation on the documentation (design documents, user manuals, and adaptation examples) supplied to the reusers. This qualitative evaluation has been defined on the basis of the subjective feelings of developers collected by proper interviews.

Confidence is associated to a single criterion, framework design maturity or simply maturity. Information about maturity comes from the number of faults detected during framework design reviews and from the requests for changes or new features addressed to the framework developer team. In particular, every issue raised by reusers about problems of integration of the reusable framework in their products is important to give an assessment about this aspect.

Frameworks can be very useful in increasing the productivity of software development with reuse. Nonetheless their development is very costly, and the decision of building a framework has to be taken very carefully. For this reason, it is very important to be able to estimate the degree of reusability of a framework not only with post-mortem analysis, and a set of ad hoc reusability metrics can be essential in this decision process. None of the existing reusability metrics has been thought with the frameworks as specific object. This means that these metrics do not always give the right weight to characteristics that play different roles in frameworks and in single components, leading to results that are often contradictory.

In this paper, we proposed a set of criteria for a frameworks reusability metrics based upon the same factors that affect the reusability of a framework according to REBOOT (portability, adaptability, understandability, and confidence). Even if the factors are the same, the approach is different because of the different perspective centered on frameworks that takes into account the particular nature of these entities. A set of reusability metrics based on the proposed conceptual model is currently applied in a major software company in Italy for assessing framework reusability. The results of this assessment will be related to the qualitative judgement of reusers and to the productivity levels observed during development with reuse, in order to confirm the validity of the approach.

1. Fowler M., UML Distilled, Addison Wesley Longman, 1997
2. Gamma et al., Design Patterns, Addison-Wesley, 1994
3. Garlan D., Allen R., Ockerbloom J., Architectural Mismatch or Why it’s hard to build systems out of existing parts, Proceedings of the Seventeenth International Conference on Software Engineering, Seattle WA, April 1995
4. Karlsson E., Software Reuse; A Holistic Approach, John Wiley & Sons, 1995
5. McCabe T.J., A Complexity Measure, IEEE Transactions on Software Engineering, Vol. 8E-2, No. 4, December 1976
6. Pree W., Meta Patterns – A Means For Capturing the Essentials of Reusable Object-Oriented Design, Proceedings of the 8^th European Conterence on Object-Oriented Programming, ECOOP’94, Bologna, Italy, July 1994
7. Pree W., Frameworks Patterns, SIGS Books & Multimedia, 1996
8. Prieto-Diaz R. and Freeman P., Classifying Software for Reusability, IEEE Software, Vol. 4, No.1, January 1987
9. STARS, Respository Guidelines for the Software Technology for Adaptable, Reliable Systems (STARS) Program, CDRL Sequence Number 0460, 15 March 1989