A SYSTEMS DEVELOPMENT LIFE-CYCLE METHODOLOGY FOR AGENT-BASED MODEL DEVELOPMENT

R. L. SIPE, New Science Partners, Houston , Texas

ABSTRACT

Systems Development Life Cycle (SDLC) Methodologies provide a means of planning and executing a wide variety of software development efforts, ranging from traditional custom application development to rapid application development using prototyping.

Consultants use SDLC's in the commercial delivery of their services to demonstrate to their clients their ability to plan and execute the development with proper controls on scope and cost. As the development of Agent Based Models (ABM) takes its place in the list of capabilities of custom software development companies, a variation on the traditional SDLC will be needed to control the process. This paper presents a methodology for the development of an Agent Based Model, defining the phases, activities and deliverables of the process, and provides some insights and comments on favorable circumstances for the application of an ABM and the nature of the dialogue between the developer and the client.

Keywords: agent based modeling, systems development life cycle methodology, SDLC, project management.

INTRODUCTON

All of us at this conference believe in the future of Agent Based Modeling (ABM). If not as a paradigm shift in the way we see human systems of all sorts that will lead to a revolution in the way we understand and manage our way in the world, then at least as the latest generation of modeling techniques, more efficient and effective than the last.

If we are right, then as it relates to their application in business, over the next few years we will increase our knowledge of how to create these models, refine the capabilities and sophistication of the models and what they can tell us, and move out of the research and development structure of our current experimental efforts and into the world of standard business systems, in which embracing the concepts and utilizing the capabilities of ABM's will have become the standard, not the exception.

Along the way the size and complexity of the projects that create the models will increase with the size and complexity of the objectives of the modeling, and we will find ourselves (indeed, we already find ourselves) facing a problem of structure and organization that is the precise parallel of the issues that have always faced computer application development projects, namely, how to manage and control the systems development life cycle (SDLC).

This paper presents an SDLC for ABM development, in the hope that such an effort will contribute to the ongoing dialogue on the issue amongst the modelers. That dialogue will hopefully lead to a heightened awareness and understanding of the fundamentals of large scale application development projects, and, perhaps more importantly, of the standards and practices already in place in corporate America on the subject that, if we are lucky, we will have to deal with as the capabilities and popularity of the discipline gradually work their way into the “best practices” of critical decision makers in business.

THE BUSINESS CASE FOR AN SDLC

I began my personal participation in Complexity Science based systems development projects as liaison from Ernst & Young's consulting practice to Biosgroup, Stu Kauffman's company. Stu is fond of comparing the behavior of a de-regulating industry to the Cambrian explosion in the fossil record, citing the parallels in the proliferation of candidate solutions to the new paradigm, and the gradual winnowing away of the suboptimal to a steady state of winning solutions. Like the early efforts in a de-regulating industry, we are in the early stages of investigation of the nature and capability of ABM's, and in our ability to build them effectively and efficiently. We are operating in “research and development” mode, in which the project to build the model is as much a research effort into the feasibility of the application of the science as it is a systems development project. Our customer, to date, has been Geoffrey Moore's visionary, of “Crossing the Chasm” fame. Whether a corporate finance executive frustrated with his inability to set effective risk management policy, or the hard bitten operational manager faced with multi-million dollar scheduling decisions on a daily basis, the visionary sees the world as it could be, and is willing to invest in our research just in case we can improve his bottom line. To him, being used to big bets, the risk of the investment is justified by the potential of the payoff. My first big project with Biosgroup was the development of an ABM for a natural gas pipeline; one that modeled the pricing mechanisms of natural gas at a physical pricing point in Louisiana called the Henry Hub. The funding came from the marketing department, to whom the prospect of a strategic advantage on the subject was well worth the investment.

In the introduction to “It's Alive”, Christopher Meyer and Stan Davis, while introducing the two factors they think will make business so, say “Our information systems themselves will become adaptive – otherwise, our businesses cannot be. By the end of the decade, business management, information systems, and biological concepts and technologies will converge around a common view of how change happens.” This theme of seeing the organization as a living, breathing being, rather than a Newtonian clock work, as you are well aware, has been gathering momentum for some time. From Senge's learning organization to Margaret Wheatley's “zeitgeist of interconnectedness” the notion of harnessing the power of an ABM to model the behavior of the organization is well formed in the minds of the visionaries. As usual, the concept is fully articulated far in advance of the reality. Whether ABM's will indeed fulfill their promise as the thinking engines of a whole new generation of systems that are able to adapt themselves to the environment, or will go the way of Artificial Intelligence remains to be seen. I believe that the effectiveness and efficiency of the development projects, themselves, can be a very positive factor in the ongoing success of the effort, lending credibility to ABM development in the eyes of our “investors”, those who would risk their software budgets on our efforts. And that depends, in part, on the method.

In “Crossing the Chasm”, Geoffrey Moore acquaints us with a model of how information systems based products are born with a model he names the “Technology Adoption Life Cycle”. In this model, “Innovators” and “Early Adopters” will put up with the fits and starts of a research and development approach, in which one of the legitimate outcomes is “It didn't work”, in order to get what they want. They are the means by which a software product is birthed but do not constitute a large enough market to sustain success. In order to reach a larger market, the entrepreneur, armed with his product and the innovator's endorsement as a happy first customer, must cross Geoffrey's Chasm, and do battle with a very different sort of a buyer, the “Pragmatist”. Wrapped in the cloak of corporate custom and heavy with the responsibility of “due diligence” the pragmatist is the gate keeper of corporate funding. He wants to see your work plan. He wants to talk to a happy customer. He wants it on time and within budget, and he wants to know exactly what “it” is. Even in the earlier stages of development, still working for the visionary on a research and development basis, the visionary is often “shadowed” by the corporate technology control network, busy insuring standardization and inter-operability of all corporate software products and services.

It is through this portal that the funding for the realization of ABM modeling lies. As we get better and better at modeling, as our track record improves, as we enjoy mass exposure for early, significant successes, as our object libraries fill up with standard customer behavior entities and blind auction modules, as our interface engines evolve, combine and connect and our software moves from custom, one of a kind, stand alone modules to integrated, corporate wide real time systems, one of the enabling technologies of this success will be the methods and standards which govern the development process.

Further, because the project in question is one which attempts to do something that no one has ever done, the customer has reason to be nervous about its outcome, and will require even more handholding during the process than is usual.

So, the business case for an ABM SDLC is simply the need to put what we want to do in a format that is recognizable by those who have to pass judgment on it (and often stand responsible if it is out of spec), to keep them informed during the process, and to give them the comfort that we are about our business in a disciplined way and aren't going to embarrass them with that thing most feared, a “problem” project that is over time or budget or worse, ineffective in it's results.

THE TECHNICAL CASE FOR AN SDLC

In my thirty years as a practitioner of application development, I have participated in countless methodology training sessions and seminars and have been a principal in the development and deployment of methodologies from four full generations of thought on the subject. In all those years I still come back to 1975's “The Mythical Man Month” by Frederick P. Brooks, Jr. as the best book ever written on the subject of large scale systems development projects. Now out in a 20 th Anniversary edition, with four new chapters, it still reminds me of what I once knew, but had forgotten.

The heart of much of the technical miscommunication, when there is one, between the pragmatist buyer and the model developer can be found in Mr. Brooks' discussion on the “programming systems product” on the very first page. He identifies a “Program”, “complete in itself, ready to be run by the author on the system on which it was developed”, and three escalations of that state, first a “Programming Product”, which is “a program that can be run, tested, repaired, and extended by anybody”. Improving “Program” in another direction, his second state is a “Programming System”, “a collection of interacting programs, coordinated in function and disciplined in format, so that the assemblage constitutes an entire facility for large tasks”. The happy marriage of “Program Product” and “Program System” finally, is the third state, “Programming System Product”, which the author notes costs nine times as much to develop as the program.

It is my observation that the developers of ABM's are often writing “Programs”, and the customers are often expecting “Programming System Products”, and the lack of some fundamental best practices within the project sometimes delays the discovery of this miscommunication until it has escalated into a major problem.

It's ok to just be writing a program. As a matter of fact, as we shall see in the methodology, that's exactly what must be done in the prototyping phase, to prove the concept before real money is laid down on its development. The point is that early and effective descriptions of the deliverables are much less likely to be misinterpreted by the customer, and are one of the primary objectives of a good method.

Because of the parallel between the methodology needs of an ABM development project and the methodology needs of a traditional systems development project, it is fair to say that the list of problem areas in the latter all apply to the former. Project planning, work plan development, project team training, scheduling, requirements gathering, module design and testing, data architecture development, training and implementation, all are traditional areas of focus, and deserve the attention of the ABM developer. The ABM development project, just like its traditional counterpart, is susceptible to delays and missed deadlines. While these project execution issues are items of interest and concern to the ABM developer, focused on perfecting the application of the science, they are items of ominous portent to the pragmatist project controller, focused on controlling the scope of the project.

In addition, where the developers of a new Accounts Receivable system start the project with a common understanding of what one of those does, the ABM development project must add the burden of the conceptual development of how the model will work in the real world, what it will do, and how that will be better, to the already lengthy list of things that must be attended to during the project. This puts a particular emphasis on a skill set that is usually a minor sidebar in a traditional project, facilitation. One does not need to get a group of subject matter experts together to discover the fundamental concepts behind Accounts Receivable; they have been well documented for 400 years or more. You do have to do that, however, if you are building an ABM of the pricing mechanism of the Henry Hub, because no one has ever done that before. In my pipeline project example, it took several intensive group sessions with the executives of the company to get closure on the fundamentals of the design of that system, far longer than would have been necessary in a traditional development project. Each executive had his own opinion, forged in the fire of his personal experience, about a matter that was central to the success of the company. Identifying the agent environment, the agents and their behaviors became a war of wills within the executive camp, with implications that went far beyond the model, itself. Had we not prepared the customer for this eventuality, they might very well have taken it as evidence that the model was not feasible rather than a natural consequence of the creative process.

Variances in the work program like this are not a problem, if you have anticipated them, and prepared the customer for the effort, they only become a problem if you leave your customer to compare the project to their only frame of reference, the traditional project, and don't point out to them the ways in which an ABM development differs.

So, all we are saying from a technical point of view is that an ABM model development project has all of the old issues, plus some new ones, and that a key to our ongoing success is educating our customer as to what those differences are, keeping him informed of the status of the project, and doing a good job of scheduling and executing the tasks that are necessary to get the job done.

THE METHODOLOGY

Origins

The need for methodology to control the ABM development process emerged immediately with the first few major assignments in my involvement with the discipline. It was not, and is not, an effort to control the creative process of the developer, but to align that process with the expectations of the customer, a customer who is used to doing business a certain way as it relates to controlling software projects. The fact that a good methodology is all about collecting and using the best practices of the discipline to make development more efficient and effective is a beneficial, but secondary, objective.

My interest in the development and formalization of a methodology has increased in recent years because of the changing nature of the customer. Successes in winning key assignments with very sophisticated objectives and big budgets in my current work with NuTech (www.nutechsolutions.com), Biosgroup's successor, with each assignment being more integrated with the business (and therefore the “line” organization) than the last, has brought the need for this sort of a formalized method out of the “nice to have” category and into the “must have” category. Because of this growing need, over the last year I have formalized a methodology for a systems development project that is based on the application of Complexity Science. The following is a level 1 process model for this method. Because the assignments often employ a variety of techniques drawn from the science, the method is not specific to ABM development, although many of the applications of it have, in fact, been ABM development projects.

In the spring of 2002, as I was launching New Science Partners, I attended “Capturing Business Complexity with Agent-Based Modeling and Simulation: Useful, Usable and Used Techniques” at Argonne National Laboratory, and saw Michael North's excellent presentation entitled, “The ABMS Paradigm” in which Michael presented a “high level visual roadmap of ABMS development and use”. With Michael's permission I have included some of the features of his paradigm, particularly with respect to his steps in the attribution of model behavior, which help bring my generic model down to a specific application of ABM development. Here is an excerpt from Michael's presentation which presents an overview of his method.

An analysis of the components of the two told me, in part:

•  That my entire first phase was an expansion of Michael's first step,

•  That we were in general agreement about the steps in model development, but his model provided a clearer explanation of the entity identification and attribution steps, and of the creation of the global environment in which the entities would operate,

•  That his “Use Phase” was, in general, the same as my “Model Application Phase”, but that mine extended into the first level of results feedback from the initial application of the model,

•  That his model had some of the flavor of “build a capability, then see what business problems we can solve with it” whereas mine had more of a flavor of “prove to me up front you can make this thing work and maybe I'll give you the money to try”, which caused me to move some of the intent of Michael's “Experimental Design” step, the first one in his ABMS Use Phase, all the way up to my Prototype phase,

•  That my “Integrate Model into Production” phase was now in need of detailing because it was where the stand alone models finally find their way into the corporate mainstream, and that it was outside of the scope of Michael's method.

Also, because of my experiences with the actual application of this method, I have broken the model development phase into two activities, “Baseline Model Development” and “Model Refinement”, which is a concession to an orderly management of the process that the business community can understand and help us execute. I wanted the basic operation of the model to be in hand before the sophisticated work of detailed attribution got under way, and this two step approach accomplished that objective.

Additionally, this exercise led me to the realization that there were several different potential definitions of a successful project, depending on what the model was being asked to do, and who the model would be operated by, which led me to Attachment 4, which is an identification of five different levels of deliverable. Proof of Concept, Baseline Model and Refined Model are all required to deliver an “Expert Interpretation” system, but additional work is required to escalate that deliverable to “Decision Support” or “Integrated Production” level.

An Overview of the Five Phases

The following is an overview of the five phases of the SDLC, which are governed throughout the project by a set of activities which manage the project, control change and maintain the client relationship.

Phase 1 – Proof of Concept, includes the initial identification of a “business value proposition” in which the modeler and the customer conspire together to find an application for the science in business, describe that application and what it's potential value is to the business. The model developer then creates a prototype, the purpose of which is to test the feasibility of the objective, and come to a conclusion as to whether or not the application of the science has a reasonable chance of achieving the business value expected.

Phase 2 – Basic Model Development defines the architecture of the model, including any development tools that will be used in its creation, and builds the initial model. Since the identification of, and attribution of behavior to, the agents in the model is such a fundamental feature in the process, phase 2 attempts only to “stub in” the agents, describe the landscape of their environment, implement their basic interaction with that environment and each other, and “baseline” the model for further development. Because of this approach, it is easier for the business community to see if we have the fundamental relationships right, before the model is clouded with subtle and sophisticated interactions.

Phase 3, Model Refinement, asks for the participation of the business community to help us bring the attribution of behavior of the agents up to a “significant” level, meaning that we can begin to see “real world like” interactions in the execution of the model. This phase, in my experience, is where the model must gain the confidence of the business community. It is iterative and intensive, and, when successful, leads to a growing confidence in the customer that the model will have real value in the business.

Phase 4, Model Application, is always the first mode of operation, regardless of the ultimate plans for more sophisticated implementations in the future. In the hands of subject matter experts in the customer's organization, the model is loaded with production data, the model attributes are set, the model is iterated and the results analyzed for business implications. Feedback loops lead back to Model Refinement for errors/enhancements to the model, and an investigation of the business consequences of applying the model's results completes the phase.

Phase 5, Model Integration. Phase 4 may be the end of development, in that a stand alone module run by a specialized subset of the business community may have been the objective all along, or Phase 4 may be a proving ground for further escalation of the use of the tool to include automatic interfaces or a wider user community. In either of these cases, Model Integration is the process of aligning the model with the production system standards in place in the organization, and the modifications required to make the model, in Mr. Brook's words, a “programming systems product”.

Project Management

Project Management is an ongoing effort throughout the project to control the SDLC process in a business like manner, thereby increasing our probability of success, and, just as importantly, keeping those who control our funding happy that we are doing so.

It consists of three components:

Project Management – The sum total of our efforts to plan the project, develop schedules for the execution of the project, staff those schedules with our own and the customers' personnel, gage and report progress on a regular basis, and make adjustments to the project plan for any number of unanticipated events and distractions that will threaten it during its life.

Change Control – A formal system in which the excellent ideas about what else we could do that everyone will have during the project are duly recorded for posterity, but only included in the project with the specific permission of the customer, fully knowledgeable of the cost and time implications of the inclusion, thereby preserving the relationship between the cost of the project and the scope of the project.

Customer Relationship Management – Since ABM development is not Accounts Receivable development, particular attention must be paid to maintaining a clear, open and frequent line of communication with the customer, from funding authority, through project management, down to the actual members of the project team, frequent, frank and informative communication of the project status is necessary in order to keep everyone on the same page.

THE PROJECT ORGANIZATION

The following is a typical project organization chart for an application development project. The need for a formal process and data integration role grows with the number of detail teams running in parallel. In a small project, the project manager usually performs both of these roles, but in a larger one, they are required to keep the design/development teams coordinated.

Design Session Facilitation

As I mentioned earlier, because of the exploratory nature of the ABM development project, facilitation is far more important here than it usually is. It is an art, in itself, and supported by much literature. A good facilitator is worth his weight in gold when toiling away all day to extract the fundamentals of the business out of a room full of headstrong department heads, each with a hidden agenda that usually doesn't have anything to do with the project. Further, it is not a skill that is widely practiced, particularly in the development community.

Because the attribution of the agents in the model can be an iterative process, the busy customer may tire of repeated requests for dialogue on the subject, or worse yet, stop returning our phone calls (he has a railroad to run, you know). It is incumbent upon us to use his time wisely, extract as much information as possible at each session, consolidate and remember the results of session 1 and make sure session 2 moves us forward and doesn't just go over session 1 again. A solid approach to facilitation will insure a happy customer and get closure on the critical information gathering tasks in the design phase.

SUMMARY

Obviously, what I have presented here is the framework of a methodology, far short of the level of detail and depth of content that a formal SDLC contains, be it an internal standard at a major corporation, or a standards package offered for sale by a major software organization.

My motivation for developing it is that the lack of its existence has on occasion caused difficulty and miscommunication between an eager customer and a talented developer who happened to be working on an application of ABM which promised to be a real advancement of the science. It seems that the larger and more important the opportunity, the more likely that issues would arise between the customers technical community and the developer, taking energy away from the objective, and sometimes dampening the momentum of the effort.

Understanding and being able to present our processes in the native language of the customer, so that he is comfortable with our approach and can focus on more important things, is just good business, particularly when that business is the furtherance of a discipline that all of us believe will become a very important one in the years to come.

Finally, many of you may well have begun your own efforts in methodology, and I welcome your comments, and offer to you this model as a contribution to that effort. I continue to evolve and expand this methodology; currently I am working on deliverable definitions and best practices for the project management components, and already have detailed process models of the phases in hand.

The best use of a methodology, in my opinion, is not as a cook book, in which every step must be observed, but as a library, from which those things that are important to the effort at hand are extracted. If nothing more, use this SDLC as a check list, ask yourself if you have done the task, and if not, why not? And it doesn't have to be a huge tome to be effective, either, a simple memo to your customer citing your understanding of an important issue may well uncover fundamental misconceptions, easily corrected in the short term, much more difficult to unravel on the day of delivery of the model.

REFERENCES

Brooks, Frederick P., 1975, “The Mythical Man Month”, Addison Wesley, p. 4.

Kauffman, S., 1995, “At Home In The Universe”, Oxford University Press, pp. 199-203.

Meyer, Christopher and Davis, Stan, 2003, “It's Alive”, Crown Business, p. x.

Moore, Geoffrey A., 1991, “Crossing The Chasm”, Harper-Business, pp. 34-38, pp. 9-13.

North, Michael, 2002, “The ABMS Paradigm”, Proceedings from “Capturing Business Complexity with Agent-Based Modeling and Simulation: Useful, Usable and Used Techniques”.

Senge, Peter M., 1990, “The Fifth Discipline”, Currency Doubleday, p. 14.

Wheatley, Margaret J., 1999, “Leadership And The New Science”, Berrett-Koehler, p. 158.

Corresponding author address: Rod L. Sipe, New Science Partners, 4119 Shady Springs Drive , Seabrook , Texas , 77586 .

As described by Kauffman (1995).

As described by Moore (1991).

As described by Meyer (2003).

As described by Singe (1990).

As described by Wheatley (1999).

As described by Moore (1991).

As described by Brooks (1975).

As presented by North (2002).

As described by Brooks (1975).