Final Report Summary - CGM (Creativity and Goal Modeling)
Requirements Engineering (RE) focuses on creating methods, languages, and tools to enable the effective capture of system requirements. Existing approaches treat the RE process as essentially a problem scoping and understanding activity, focusing on the "engineering" of correct requirements. Effective requirement elicitation and analysis should focus instead on problem discovery and problem solving, potentially facilitated by existing and well-established creativity techniques from business and the social sciences. Although creativity techniques may be the key to generating ideas which evolve into novel requirements and innovative systems, they lack structured ways to capture requirements, and do not explicitly capture criteria to select amongst alternative or competing requirements. Such methods do not contain support for systematic analysis, important for quality trade-off analysis (e.g. speed vs. cost) or property checking (e.g. safety) as part of RE analysis. At the moment, there is no tool-supported way to incorporate creative output into a system development process.
The objectives of the project centered on exploiting the synergies between structured goal modeling and creativity techniques to enhance the discovery of requirements which are creative, structured and rationalized. Specific objectives included supporting the discovery of novel RE ideas and requirements, and the integration of existing technical solutions (e.g. services, apps). The project aimed to support grounding creative ideas produced by creativity techniques in stakeholder objectives, capturing ideas in a structured and graphical form, complementing the existing, less-structured techniques (e.g. storyboards, mock-ups). I intended to apply existing RE goal model analysis techniques in a creative context, facilitating the selection amongst alternative creative and familiar requirements. It was my intention that existing RE approaches (e.g. goal models, goal analysis) should be better aligned with business interests, specifically interest in creativity and innovation. Finally, developed approaches were intended to be validated.
In the two-year project, I followed a Design Science methodology, guiding the design of a practical tool and method using scientific principles. I performed several iterations of the design cycle, starting with three rounds of exploratory studies, then two rounds of formative studies in tandem with tool and method development, and finally an initial summative study, beginning to quantitatively test the hypotheses. The in-lab studies involved undergraduate and graduate students at City University London, the University of Trento in Italy, the University of Toronto, and researchers visiting an international RE conference in Ottawa. Many of the participants had industrial design experience. Participants underwent linked goal model and creativity activities, with the researcher observing and coding output. In total, I conducted six design cycles, with more than 50 participants.
The results gave insight into how structured goal modeling and creativity can be used effectively in one exploratory process. The majority of participants reported that the combination of creativity techniques and goal modeling was promising. Observations showed that users needed much methodological guidance – these observations were used to add embedded guidance into the resulting tool. All creativity techniques applied were useful in varying contexts, with different groups preferring different techniques. Given standard technology, i.e. a shared monitor and mouse, I observed optimum group size to be two to three. With more advanced technology (distributed tooling, smart boards), this limit could be increased.
The concrete outcome of the project was the Creative Leaf tool and associated method, see Figure 1 for a screenshot. Creative Leaf is a simple, web-based tool which supports standard goal modeling, and both divergent and convergent creativity activities. Readers are encouraged to try out the tool (works best in Chrome): http://creativeleaf.city.ac.uk/. Creativity is supported via a creativity palette, containing buttons which open embedded implementations of five established divergent creativity activities. Two of these activities (CRUISE and BrightSparks) are implemented using existing creativity services produced as a result of the Collage European project. Each divergent activity (except for Brainstorming) takes as input some elements of the goal model (goals, tasks, actors). Ideas generated in the tool are added to the goal model canvas as digital post-its (yellow boxes), which can be linked directly to goal model elements using typical goal model relations. Two further activities help with idea prioritization and refinement. The tool supports goal model analysis via the Hover Evaluate feature, evaluating the satisfaction of goals, qualities, and tasks in the model when certain ideas are implemented. The suggested methodology is embedded in the tool via a series of optional unobtrusive prompts. Optional data tracking is used to collect statistical information on tool use, facilitating usage analysis.
By evaluating the effectiveness of our tool and process, I was able to make several key findings:
• Idea generation: all participants in our studies were able to come up with ideas using available versions of our process or tooling, some of which were creative.
• Completeness: the less-than creative ideas discovered were still useful, filling in the gaps of the goal modeling, increasing the completeness of the more typical, but still essential requirements.
• Prioritization: I observed that users were able to prioritize their ideas, making some use of the results of goal model analysis in order to support their prioritizations.
• Idea Refinement: users were able to take their best ideas and refine them into elements of their goal model, but this task took effort.
• Divergent vs. Convergent Creativity: divergent creativity, the generation of ideas inspired by and linked to the model, was far easier for participants than convergent creativity, incorporating ideas into the model.
To summarize, we found our tool and method supported a traditional RE elicitation process, improving completeness, but had a “creative twist”, allowing the potential for creative idea discovery.
In future cycles I plan to extend the tool to support more effective use of model analysis as part of convergent creativity, making direct prioritization suggestions based on analysis results. Support should be added for transformation from ideas to model elements, mining our study data for common patterns. I am currently developing industrial contacts to evaluate the tool in practice, completing the design cycle.
The availability of our web-based tool, method, and supporting documentation makes both goal modeling and creativity techniques easily accessible. This has wide potential impact on both industrial parties who are interested in creativity, but may not have the person-power to invest in a wide-scale project, and RE academics, who are now able to easily apply creativity techniques to a common RE modeling language. Tool output – goal models with enhanced completeness and creative content – can be transformed into further downstream system artefacts (e.g. textual requirements, tabular formats, design models, formal representations) and facilitated application of a wide range of available RE techniques (e.g. traceability techniques, quality analysis, model checking). In this way, creative content can be incorporated into a system development cycle, supporting the integration of creativity into real systems, increasing the chance of successful and innovative system development.
Project website:
https://sites.google.com/site/creativitygm4re/
The objectives of the project centered on exploiting the synergies between structured goal modeling and creativity techniques to enhance the discovery of requirements which are creative, structured and rationalized. Specific objectives included supporting the discovery of novel RE ideas and requirements, and the integration of existing technical solutions (e.g. services, apps). The project aimed to support grounding creative ideas produced by creativity techniques in stakeholder objectives, capturing ideas in a structured and graphical form, complementing the existing, less-structured techniques (e.g. storyboards, mock-ups). I intended to apply existing RE goal model analysis techniques in a creative context, facilitating the selection amongst alternative creative and familiar requirements. It was my intention that existing RE approaches (e.g. goal models, goal analysis) should be better aligned with business interests, specifically interest in creativity and innovation. Finally, developed approaches were intended to be validated.
In the two-year project, I followed a Design Science methodology, guiding the design of a practical tool and method using scientific principles. I performed several iterations of the design cycle, starting with three rounds of exploratory studies, then two rounds of formative studies in tandem with tool and method development, and finally an initial summative study, beginning to quantitatively test the hypotheses. The in-lab studies involved undergraduate and graduate students at City University London, the University of Trento in Italy, the University of Toronto, and researchers visiting an international RE conference in Ottawa. Many of the participants had industrial design experience. Participants underwent linked goal model and creativity activities, with the researcher observing and coding output. In total, I conducted six design cycles, with more than 50 participants.
The results gave insight into how structured goal modeling and creativity can be used effectively in one exploratory process. The majority of participants reported that the combination of creativity techniques and goal modeling was promising. Observations showed that users needed much methodological guidance – these observations were used to add embedded guidance into the resulting tool. All creativity techniques applied were useful in varying contexts, with different groups preferring different techniques. Given standard technology, i.e. a shared monitor and mouse, I observed optimum group size to be two to three. With more advanced technology (distributed tooling, smart boards), this limit could be increased.
The concrete outcome of the project was the Creative Leaf tool and associated method, see Figure 1 for a screenshot. Creative Leaf is a simple, web-based tool which supports standard goal modeling, and both divergent and convergent creativity activities. Readers are encouraged to try out the tool (works best in Chrome): http://creativeleaf.city.ac.uk/. Creativity is supported via a creativity palette, containing buttons which open embedded implementations of five established divergent creativity activities. Two of these activities (CRUISE and BrightSparks) are implemented using existing creativity services produced as a result of the Collage European project. Each divergent activity (except for Brainstorming) takes as input some elements of the goal model (goals, tasks, actors). Ideas generated in the tool are added to the goal model canvas as digital post-its (yellow boxes), which can be linked directly to goal model elements using typical goal model relations. Two further activities help with idea prioritization and refinement. The tool supports goal model analysis via the Hover Evaluate feature, evaluating the satisfaction of goals, qualities, and tasks in the model when certain ideas are implemented. The suggested methodology is embedded in the tool via a series of optional unobtrusive prompts. Optional data tracking is used to collect statistical information on tool use, facilitating usage analysis.
By evaluating the effectiveness of our tool and process, I was able to make several key findings:
• Idea generation: all participants in our studies were able to come up with ideas using available versions of our process or tooling, some of which were creative.
• Completeness: the less-than creative ideas discovered were still useful, filling in the gaps of the goal modeling, increasing the completeness of the more typical, but still essential requirements.
• Prioritization: I observed that users were able to prioritize their ideas, making some use of the results of goal model analysis in order to support their prioritizations.
• Idea Refinement: users were able to take their best ideas and refine them into elements of their goal model, but this task took effort.
• Divergent vs. Convergent Creativity: divergent creativity, the generation of ideas inspired by and linked to the model, was far easier for participants than convergent creativity, incorporating ideas into the model.
To summarize, we found our tool and method supported a traditional RE elicitation process, improving completeness, but had a “creative twist”, allowing the potential for creative idea discovery.
In future cycles I plan to extend the tool to support more effective use of model analysis as part of convergent creativity, making direct prioritization suggestions based on analysis results. Support should be added for transformation from ideas to model elements, mining our study data for common patterns. I am currently developing industrial contacts to evaluate the tool in practice, completing the design cycle.
The availability of our web-based tool, method, and supporting documentation makes both goal modeling and creativity techniques easily accessible. This has wide potential impact on both industrial parties who are interested in creativity, but may not have the person-power to invest in a wide-scale project, and RE academics, who are now able to easily apply creativity techniques to a common RE modeling language. Tool output – goal models with enhanced completeness and creative content – can be transformed into further downstream system artefacts (e.g. textual requirements, tabular formats, design models, formal representations) and facilitated application of a wide range of available RE techniques (e.g. traceability techniques, quality analysis, model checking). In this way, creative content can be incorporated into a system development cycle, supporting the integration of creativity into real systems, increasing the chance of successful and innovative system development.
Project website:
https://sites.google.com/site/creativitygm4re/