I’ve not been using this blog much, but I thought I’d give some bullet-point updates on my PhDs progress for anyone who happens to stumble across it:

- Got a paper accepted at Requirements Engineering Conference 2011 entitled Early Failure Prediction in Feature Request Management Systems. It covers the meat of my thesis and won a best paper award.

- I’ve done a research Internship at the National Institute of Informatics in Tokyo. Japan is awesome. Published a paper on the work I did there at the Adaptive and Intelligent Systems Conference 2011 entitled Utility-Based Self-Adaption with Environment Specific Quality Models. More to come on that research later. Hopefully at SEAMS’12.

- Published an article in Requirements Quarterly 57 entitled Upfront Analysis – The Payoff.

- Writing up my thesis now, scheduled around 6 months to finish it.

That’s all for now folks!

http://sre-research.cs.ucl.ac.uk/TransferReport.pdf

The date is set for the 17th August and between now and then I hope to put a paper together for ICSE. I’ll base it mainly on the report, but will need to do a few additional experiments. Deadline is the 20th August, so I best get writing!

You can find it at http://sre-research.cs.ucl.ac.uk/Intueri/Intueri.html. The idea with this tool is to enable developers and project managers in distributed software development projects to do the following:

- Extract existing data from a feature request management forums.

- Use this data to build failure prediction models.

- Extract further data from forums, and predict where failures are likely to occur.

- Project managers can use these predictions to focus their efforts on ‘problematic discussions’, can developers can use them when taking decisions to ascertain the likelihood of a future failure occurring.

Please refer to http://sre-research.cs.ucl.ac.uk/TransferReport.pdf to find about a little more about what I mean by a failure and the details of this process. I’ll try and get a post up soon summarising these concepts and the benefits provided by the tool.

Intueri should be definitely be classed as ‘under development’ at the moment, but you can have a little play around with it to get an idea of what it might do. Right now you can generate a machine learning kernel from raw feature request discussion data. For example, if you open the CodeFaults project, which has all positive and negative instances of features that have been included in the Firefox project and have later resulted in product faults, you can then click on the ‘generate kernel’ button to create a variety of different kernels. These can then be given to a classifier, such as those available in the Weka tool, and used to build failure prediction models.

PS. The old tool, ARET (Asychronous Requirements Elaboration Tool), which I was using to explore all kinds of ideas before settling on this one is still available at http://sre-research.cs.ucl.ac.uk/ARET/ARET.html. At some future date I’ll write a post about the concepts explored in this tool.

For the past month or so I’ve been working on my Transfer Report take 2, and the research is looking solid. So, to update you all on what going on here’s an abstract:

The shift towards distributed software development and the highly collaborative nature of requirements engineering has resulted in a sharp increase in the use of web-based communications and tools to collaborate on requirements tasks. Specialised feature request management forums are widely used for collaboration between developers and stakeholders on these activities. Standard requirements pitfalls and the challenges introduced by distributed collaboration often lead developers often make decisions that lead to failures in a feature request’s life cycle. Developers, for example, often decide to implement a feature that later results in software product faults, or they decide to reject a feature that is later found to be of value and included in the product.
It would be of significant benefit if the likelihood of a failure occurring was known as these decisions are made. This would focus the attention of project managers responsible for spotting and rectifying these decisions, who’s time is a scare and valuable resource, and would flag cases that might otherwise have been missed, thus avoiding the spiralling costs of fixing the problem as time goes on. Further, developers could consult the likelihood of a failure occurring when considering whether to take decisions and use this as an indicator of whether to reconsider the decision or encourage more activity before taking it.
We propose an automated approach for predicting the likelihood of a decision leading to failure in feature request discussions that uses classification models trained on data and metrics mined from feature request forums. A prototype tool we have developed, Intueri, automates and implements the approach.
Using Intueri we are able to successfully predict reversals of a developer’s decision to reject a feature in the Firefox project with 91% accuracy, 0.82 recall and 0.24 precision. The experiments reveal insights into aspects of discussions that affect the probability of this failure occurring. Prediction models suggest, for example, that discussions where few stakeholders’ opinions are expressed before a developer rejects a feature often lead to a reversal of this decision.
We propose extended research that will vary the projects and metrics mined, the types of failure predicted and classification algorithms used in this approach. Research aims are to further improve the accuracy of our predictions, show how effective such an approach can be in a wider context, and gain more insight into the factors that effect the likelihood of failures occurring.

I’m still in the process of re-working the report, but you can take a look at it if you’re interested at:

http://sre-research.cs.ucl.ac.uk/TransferReport.pdf

So without further delay, here’s a quick update:

My research has taken the direction of ‘Improving Stakeholder Involvement in Global Requirements Engineering’. I’ve included a short abstract of a paper that I’m working on at the end of this post to explain this a little further. You can also take a look at these Slides of a 20min presentation I gave at a PhD Workshop a couple of weeks ago.

I’ve also been slaving away on the code for tool over the past few months to demonstrate my work. It’s currently still work in progress and being used to explore concepts, but you can take a look at it over at http://sre-research.cs.ucl.ac.uk/ARET/ARET.html . I’ll try and get a post up soon explaining a little more about what you can do with it.

Here’s that abstract:

Software engineering activities in large-scale distributed projects are often carried out using web-based communications and technologies. Support for Requirements Engineering in these environments, however, fails to fully address the problems relating to stakeholder involvement introduced by distributed collaboration.
In this paper we propose a tool-supported framework and approach to help an RE team make more informed decisions with respect to stakeholders’ opinions. Goal modelling and design rationale elements underpin our model, into which previous and continuing requirements related discussions are extracted from the ticking systems used by stakeholders to provide feedback. This re-modelling of existing discussions is done to overcome the difficulties a requirements engineer has in accessing relevant stakeholder contributions when making decisions.
We have developed a web-based tool, ARET, to demonstrate this concept and are currently using it to explore and validate the benefits of our approach.

Below is the abstract of a paper I’ve starting writing currently entitled “Support for Distributed Requirements Elaboration”. Hopefully the taster will entice those of you interested in the subject to leaf over the viva report and perhaps provide feedback if you’re up to it.

Requirements engineering activities in large-scale distributed projects are often carried out using web-based communications and technologies. Support for these activities, however, is limited and fails to fully address the problems introduced by asynchronous collaboration.
In this paper we propose a tool-supported framework for asynchronous requirements elaboration that combines goal-modelling techniques with a critique-based review mechanism. Developers drive forward a goal model’s elaboration by annotating its elements with issues. The framework then provides guidance and support for the suggestion and selection of resolutions to these issues.

In other news, I took over the role of the publicity officer for the Requirements Engineering Specialist Group for the BCS (RESG) today. Main responsibilities are care-taking the website and and keeping interested parties updated on forthcoming events. I’m very much looking forward to working with the group.

They serve as a pretty good snapshot of what I’m doing at the moment, and it includes a nice example that can be followed to clarify things.

I’ve mostly been working on a document, entitled  requirements-and-design, which is still work in progress. Please take a look at it if you wish to find out a little more about what I’m doing at the moment. I’ve also updated the About section of this site to reflect the changes in direction I’ve made.

Below follows the motivation section taken from the document I’ve linked to above, which provides an idea of the direction I am moving in:

Software development using a collaborative online environment to house all communications and work-in-progress has proven to be a success, as projects like Mozilla’s Firefox and Ubuntu have shown. Little has been done, however, to facilitate the requirements management process in these environments. Moreover, the environment itself presents many opportunities that are yet to be exploited, such as supporting communication between users and allowing multiple users to collaboratively develop requirements models.

A case study on the Firefox project in the first year VIVA document unveiled problems with requirements management process in collaborative environments that need to be addressed. The key problem, for which this document outlines a solution, is that of agreeing on requirements elaborations when multiple actors are involved in discussions. Currently, requirements are developed through free-for-all discussions in a Bugzilla ticket, which is comparable to a forum. These discussions often escalate into arguments that are pathological, unproductive and can take a long time to reach conclusion. A solution to this problem would aid those working with requirements in a collaborative environment to hold arguments that are more constructive and lead to solutions in a shorter amount of time.

It is proposed to solve this problem by supporting multi-party requirements elaboration in collaborative environments by allowing users to build and annotate goal-models in tandem with their discussions. The solution will not only make the structure of an argument clearer, but will also suggest to users the subsequent ‘moves’ available to them that are likely to make the argument progress. Furthermore, tracing back over the steps of these arguments will give a clear view of how a requirements model developed, thus providing ‘free’ rationale.

This will tackle an area missing from requirements literature, namely that of aiding with the process of collaborative requirements elaboration. A large volume of literature exists on the elaboration of requirements models, such conflict identification and resolution (B Boehm & H In 1999; A van Lamsweerde et al. 1998) and heuristics for elaboration (Landtsheer 2007). This work, however, mainly focuses on offline elaboration and does not provide much to aid multiple users with conflicting opinions. Work does exist that attempts to solve this problem (Robinson 1990; B. Boehm et al. 1995) but requires a single analyst to take the conflicting opinions into consideration and seek resolutions, rather guiding the actual stakeholders through the development and evolution of the models. Involving the stakeholders directly conforms to the current state of practice observed in the case studies. Moreover, it also holds possibilities for better solutions, as it is the stakeholders themselves who have the detailed level of knowledge on their objectives and opinions needed to conduct negotiations. The tactic employed in previous works of modelling this knowledge is not only time consuming, but also may not encode all the information needed. Guiding multiple stakeholders’ moves when arguing about requirements model elaborations in real time would alleviate this issue.

Addressing this problem present great benefit to those working with requirements in all kinds of collaborative software development environments. This not only includes Open Source projects, but also those of a commercial nature where a development team is spread across multiple sites, as is more and more frequently the case for software development projects. The next section begins with a detailed description of these benefits.

A framework and set of techniques will be developed to aid requirements conflict management in collaborative environments, such that that used by the Firefox 3.0 development project. Actors will be able to collaboratively annotate small scale goal-graphs with conflict management discussions as they progress. These techniques will aim to provide more rapid and preferable conflict resolutions with a greater degree of user satisfaction.

These techniques now need to be developed, and to do this I will be concentrating on the following three areas: 

• Working on a concise syntax and set of semantics for the annotations, or mark-ups, needed to annotated a goal graph with conflict management discussions. The case studies contained in the Report have already explored some representations, which will be further developed.
• A set of locutions will need to be defined, detailing actions that actors can make on the model. A schema suggested by Finkelstein[1] provides a set locutions designed for offline work between two actors – an author and a reviewer of a requirements specification. This set will need to be elaborated for a collaborative environment with many actors in different roles.
• Many actors with different roles and levels of authority will participate in requirements conflict management, which need to be represented. Work will therefore begin on listing and defining the actor types and their roles.

[1] FINKELSTEIN, A. (1992) A scheme for review, annotation and correction of specifications. Instructional Science, 21, 183-198.

Management of conflicts in requirements engineering has been shown to be critical to a projects success. Collaborative development environments are particularly well suited to requirements engineering due to its highly collaborative nature, yet little work has been done to use the collaborative setting to aid with requirements conflict management.

The case study examined in this report shows examples of how conflicts are currently managed in the Firefox 3.0 development environment. Developers using this environment face a set of problems for which new solutions are in need of development. These problems give rise to the PhD thesis contribution – A framework and set of techniques for structuring requirements conflict argumentation in a collaborative environment.

The case study concludes that the following points hold possibilities for improving the way conflicts are managed in the Firefox 3.0 collaborative environment:

• Improved support for argumentation
• Improved user modelling
• Improved rationale tracing
• Exploitation of dependency links between requirements

The resulting thesis proposal is to aid requirements conflict argumentation in collaborative environments by allowing users to annotate a small scale goal graph with their discussions. Support for user modelling, dependency links and rationale will also need to be included in the framework and techniques developed.

The document provides both the thesis proposal, and an insightful case study into requirements conflict management in the Firefox 3.0 project. Please do comment – Click