Dr. Dobb's Digest December 2009
As you may have heard, we have been quietly planning a revolution with the goal of "re-founding" software engineering as a rigorous discipline. We recognize that the natural tendency in our field is to perturb systems minimally into approximate correctness, but this path cannot be sustained any longer if we are to support the computing industry and help it meet the demands of society. We need to restart on a solid basis, taking advantage of all that has been learned in software engineering theory and practice over the past five decades.
The initiative started with two articles published in Dr Dobb's -- Methods Need Theory and Why We Need a Theory for Software Engineering -- that analyzed and deplored the fragmentation of software engineering and, in particular, of the methodology scene.
To address this issue, to bring together the best and the brightest, we have founded a community, the Software Engineering Method and Theory (SEMAT) community. The community supports the following Call for Action.
Software engineering is gravely hampered today by immature practices. Specific problems include:
We support a process to re-found software engineering based on a solid theory, proven principles, and best practices that:
The following people have signed the Call for Action statement. This list is dynamic; the up-to-date list is on the SEMAT web site.
All signatories are world-class experts who have individually made significant contributions to the software engineering discipline and of the development of today's best practices. They are distributed around the world; their collective expertise extends across computer science and software engineering, covering advanced programming techniques, modeling languages, Agile as well as traditional software processes, a wide spectrum of practices (human, technical, business models, user interaction), tools, frameworks, and many more areas. Together we represent a large part of the software development community.
Q: The goals are lofty, but they are only goals. Before going public, shouldn't you have waited until you had some actual solutions to show?
A: Lots of people have excellent ideas, but no one holds the whole truth. The only way to solve a problem of this magnitude is through a community effort that identifies the problem precisely and leads to widely agreed solutions. That's why we are starting with a statement of objectives and encouraging comments from a wide range of experts.
Q: Why the emphasis on signatories?
A: The people who have agreed to sign the Call for Action have, each in his or her own way, had an effect on the world of software development. Their participation is essential to the visibility of the project, in particular with large companies, and to its success.
Q: There is little visible consensus among these signatories. How will you get them to agree on a common approach?
A: By its very nature this is indeed a group made of people with strong views. But they realize the importance of coming to a kernel of widely-agreed elements, solid enough to address the core problems and extensible enough to accommodate the diversity of requirements, usage contexts and technologies. The first step is to develop a joint vision statement that will serve as a blueprint for the rest of the effort.
Q: Now that there's a Call for Action signed by an impressive group of people, will the real work start?
A: Yes, the hard part lies ahead. The hardest part will be to get everyone moving forward without compromising ourselves to death, and without anyone giving up because his or her pet idea is not the key driver. We think that with perseverance, creativity and diplomacy we will be successful.
Q: There are plenty of approaches to software engineering discipline and methodologies. We don't need a revolution, we need wide acceptance of my theory!
A: A number of people have superb theories, sometimes backed by successful experiments; no doubt yours is one of them. Unfortunately, no single approach has garnered broad acceptance in industry and academia. Like other engineering fields, our's needs a common framework with which everyone agrees. We hope you will contribute your ideas.
Q: How is SEMAT different from other earlier efforts, such as, the IEEE Computer Society SWEBOK (Software Engineering Body of Knowledge) or the various standard curricula in software engineering?
A: A crucial difference in the SEMAT initiative from previous efforts, such as SWEBOK, is the central focus on underlying theory. While SWEBOK and the various IEEE curricula reflect the state of the art, those constitute purely pragmatic admixtures of ideas, constructs, techniques, and disciplines along with current practice and popular opinion. SEMAT is focused on a kernel of strong theoretical underpinnings. That's not to say that SWEBOK and the like aren't valuable, and indeed valuable inputs to the SEMAT initiative. Only that we need a theory!
In the long run, software engineering needs to become grounded in theory and based on evidence, criteria met by only a small fraction of what passes for software engineering as taught and as practiced today. In order to rise to some such standard of evidence and theory, SEMAT will need to be winnowing down rather than gathering in. While this work will certainly draw on SWEBOK and other recognized sources, rather than expanding and including, as must be the agenda for a body of knowledge or a curriculum, much effort may have to be in narrowing and excluding. In the process, it is hoped to identify precisely some of those areas in most critical need of attention, either to test and prove theory or to build explanation that encapsulates evidence and experience.
Q: The call for action talks about "a kernel of widely-agreed elements, extensible for specific uses". This sounds very much like a unified methodology. Is it?
A: No, we are not looking for a single methodology. But we do seek to identify "universals" of software development: universal problems and universally recognized solution elements. For example, every project has a process and products, and these products must always include, in the end, executables. Every project must do some form of analysis, of design, of implementation, of verification. Other universals include roles: analyst, developer, QA engineer. The SEMAT effort should identify these universals, both in the problem space and in the solution space. This will make it easier not only to recognize solved problems and reuse recognized solutions (theories and practices), but also to compose solutions and, where none exist, develop new ones.
Q: Why would we succeed where so many have failed?
A: We believe that the industry is tired of fads, wary of empty talk about methods and process, and ready for a change We also see that the chasm between industry and academia is being questioned, with (for example) many practices that originated in industry having found their way into standard university curricula, and some advanced research finding its way into actual projects (initially in mission-critical systems, but with potential for much wider spreading).
For these hopes to be realized, the community -- industry and research -- must join forces. While the amount of work that remains is huge, the momentum already created by the SEMAT initiative, over the course of just a few weeks, shows that this prospect is achievable. We hope you will share our enthusiasm.
Take a look at www.semat.org. But you can do more than visiting! First, you can become a supporter by clicking "sign up" on the site. You can help with ideas for the vision statement on which we are now working. You can contribute to the blog on the site, and participate in the discussions. Right now, you can spread the word. This initiative needs to be widely supported if it is going to have any impact on the industry. We need to do a good job, but when we don't you can correct us. As the vision statement and further material becomes available, you can help develop training material for industry and universities. There will be many more ways to participate.
To be successful the SEMAT effort will require:
The longer we wait, the longer we perpetuate the impression by the rest of the world that software is too often brittle. Let's stop wasting time, and get to work.