Att-e-en-tion! To System Design

What we are paying attention to shapes what we perceive and pay attention to. And paying attention, requires attention. The system, thought of, observed and measured, reasoned about, designed as, a system, needs intentional attention. Attention that competes for bandwidth with the order of the day — delivery of working code (with tests, please). We need to discover on purpose (embracing happy accident along the way), design on purpose (embracing iteration, discovery, accident), and… redesign on purpose (throughout the lifecycle). Using the cheapest design medium that will expose the uncertainties and stress the design (fragment, mechanism, etc.) under consideration sufficiently for the (extraordinary!) moment we’re at. Which may be code. But don’t assume so, when thought experiments, models, sketch-prototypes, cardboard mockups, role-plays, etc. will serve the moment. Yes, yes, “code wins arguments,” but you have to know how to set up your arguments.

The system takes a different kind of attention. Thinking at the interaction between capabilities (and value the system delivers) and system properties (impacting users individually, like usability and performance, and in aggregate, like scalability and availability; and impacting operations and the business, …) and the structures and mechanisms that deliver them, takes a vantage point that is able to move fluidly and adeptly at different scales or scopes (zoom levels, if you like) and across views (frames, cross sections, selective screening, …). But it also takes time, which takes will — the organizational will, and the personal will, to do it. When there is code to be written, and pressure to deliver. Increasing. If we build ourselves into a blind-alley of “technical debt” …which I have come to think of as “decision debt” — where we put off making a decision and the non-decision decision (do nothing, or keeping doing the old thing) or a reactive quick-fix decision gets coupled into the system. Some of these will be of little consequence. But there will be those where the “payback time” comes with stiff interest — and penalties! (Those are the ones an architect should have sniffed out as architecturally significant — and if he didn’t, he will next time! Experience factors. Puns allowed. We’re among friends.)

But wait. We can refactor. Continuously refactor. That’d do it. Well, yes. And no. Imperious Red Queen voice: “I need an architect here… Architect?” “It depends.” What did the Red Queen tell Alice? “it takes all the running you can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that!” Say what?

Someone needs to be thinking strategically about the system. Not alone; working with the team, but having responsibility for the system so that attention is drawn to the system, attending to structural integrity in the small, sure, but in the large, too. Instrumenting the system — making code health and operational health visible. Anticipating the market and problems in the code, keeping tabs on how the system is doing relative to the design envelope for its various system properties. Finding the weak spots in the design. And doing “model storming” “out loud” “in pairs” or small groups. And all that agile stuff. Just in the cheapest medium for the moment, because we need to explore options quick and dirty. Hack them — with sketches/models, with mockups, with just-enough prototypes. Enough early. And enough along the way so that we course correct when we need to. So that we anticipate enough. Put in infrastructure in good time. Avoid getting coupled to assumptions that cement the project into expectations that are honed for too narrow a customer need/group. And suchness.

Comments

Connections

James Burke (Connections, 2007), commenting on the Gutenberg printing press, observed “the easier it is to communicate, the faster change happens.” And “faster change” characterizes our experience of the times we’re in!

The “rising tide that lifts all boats” is readily associated with digital technology which has not only seen more than two billion people connected to the info-verse that is the Web, but has created ever more sophisticated building blocks for new generations of innovation. We have much of the world’s knowledge resources at our fingertips, huge constructive capacity available freely in open source software and increasingly in open source designs and hardware and advancing electronics building blocks. This kind of widely and freely, or very cheaply, available high leverage resource base really adds impetus to and underscores that “all boats” in a way that is unprecedented in history.

With each new tier of capability that is moved into technology, we’re advancing basic and applied science and launching wave after wave of “creative destruction” as products and even industries are continually re-imagined, further enabling their users to connect, to express, to learn, to build, to delegate expertise to systems that amplify what we can achieve.

We have moved ever more expertise into our tools, advancing not only efficiency but giving us enhanced, or even unprecedented, capabilities. For example, digital cameras, post-processing apps, and photo sharing have put capabilities in the hands of amateurs that even a generation ago took the focus of careers to build. Moreover, imaging technology has enabled us to see where we could not. Neuroscience, for example, has advanced at an exciting pace, owing much to the imaging capabilities that have been assembled into the machine that extends the reach of man’s ability to see and comprehend.

We are redefining manufacturing with ever advancing automation, moving ever more work to ever more skilled robots, and using visualization, simulation and 3D printers to speed prototyping and reduce the cost of very small lot size manufacturing, and bringing the ability to create with advanced components and materials into the reach even of home-based workshops.

With connection between and visibility into other cultures, and the knowledge and capability spaces of other fields, we are seeing a tremendous “Renaissance” in every field, with influence across previous “islands” of specialty.

Even so, humankind has taken something of a knock to its collective confidence these past several years, what with:

  • global recession, and struggling national economies
  • climate change

Not only has the vulnerability of interconnected global systems snuck up on our general consciousness, but our human fallibility is becoming all the more clear. Add to that wave after wave of digital transformation that is reshaping entire industries. The peril of change is constantly before us with a litany of fallen and stumbling giants ever resounding through the formal and social media: Borders, Kodak, RIM, Nokia, the list goes on. To be sure, some recover themselves, as GM did. While others manage to reinvent themselves, morphing to fit and even shape changing contexts, as IBM has done.

Still, “the topple rate–which tracks the rate at which companies change rank–has more than doubled, suggesting that winning companies have a tough time keeping their leadership position for long.” — Irving Wladawsky-Berger, Why People Are Gaining Power Over Organizations in the Age of IT, April 23, 2012 [quoting shift index]

All this change and uncertainty, taken together with a sense of inadequacy in the face of complexity and turmoil, and John Gall’s title for his delightful keynote/essay “How to use conscious purpose without wrecking everything” (2012) strikes a chord! How indeed? If we think of business strategy as organizational intentionality, and architecture as design to achieve more the outcomes we intend, this is something we ought to have some interest in understanding. What are the forces and dynamics at play? And more to the point, what is to be done, that we might adapt and thrive, taking advantage of the possibilities scientific discovery and innovation opens up, rather than falling to the waves of change? And how does this relate to architecture? And system development?

Comments

Visual Architecting

The Visual Architecting Process (VAP) supports the architect in leading architectural decision making and the exploration of options, the iterative resolving of critical uncertainties and challenges, and the evolution of the system.

The process is illustrated here and here, emphasizing the visual support for system conceptualization, design and reflection. We regard process as “scaffolding” (it supports us and enables us to build more complex systems, but the effort and focus is on building the system) and we use just as much as we need to fit the design and organizational challenges of our system/organization and the maturity stage of our system. In other words, it is a “just enough” highly iterative and agile process.

We advocate identifying and prioritizing, exploring and addressing opportunities, uncertainties, challenges and risks with the cheapest medium that will serve those we judge most important to address. If we can learn with an exercise of the imagination, with a sketch or model, with role play, with a (paper) mock-up, with a focused prototype, or a spike, then we do that. We recognize uncertainty, and deliberately seek to discover value and resolve direction.

Visual Architecting supports:
Visual Architecting
and

  • Creating architectures that are good, right and successful, where good: technically sound; right: meets stakeholders goals and fit context and purpose; and successful: actually delivers strategic outcomes.
  • Translating business strategy into technical strategy and leading the responsive implementation of that strategy
  • Applying guiding principles like: this extraordinary moment principle; the minimalist architecture principle; and the connect the dots principle.
  • Being agile. Creating options. Reflecting. Responding. Factoring and refactoring.
  • A commitment to objectivity.

Given a “just enough” (e.g., willing to work with as sketchy and cheap a decision medium as does the job well enough) and “minimalist” (focusing on decisions necessary to strategic outcomes) orientation: We seek to make the context and rationale for our decisions explicit, to identify the important side-effects and consequences and dependencies of these decisions, as well as alternatives noting why we make the decision we make. We seek to understand and compensate for cognitive biases and other sources of decision error.

Some further discussions of aspects of VAP:

Comments

Conceptual Architecture: How

How: Some Comments on Creating and Evolving the Conceptual Architecture

During early system conceptualization, we start to envision the form or shape of the system, its boundaries and interactions, its primary elements and their interactions. The sketchy shapes of these elements, expressed mainly in terms of their responsibilities, analogies, and drawing on patterns and experience, take a rough form, allowing us to investigate the system concept. Then, as the system concept (identity, value propositions and capabilities) takes shape, so too does the Conceptual Architecture.

In designing complex systems, separation of concerns is a key strategy for gaining intellectual traction and enabling ourselves to reason about important design decisions while considering cross-cutting concerns that are part and parcel of the decomposition-composition dual of systems design and development. The concerns focused on at this point are related to giving shape to the system paying attention to user-facing capabilities and developer-facing properties of the system. We seek to bring intentionality and experience to design to achieve more the system properties we want, recognizing that ultimately system properties are emergent and we will need to sense and respond and adapt as we learn more from the system in context.

The architectural elements of software systems (that is, elements significant enough to the system to draw out and deal with in architectural design) are constructs of inventive human thought. We mold and shape them to accomplish a purpose, subject to constraints and forces. We leverage analogies to draw experience and knowledge from other domains, including mechanisms of nature (e.g., swarms), man-made (e.g., mechanical mechanisms like hubs), and social constructs (e.g., social and economic mechanisms like brokers).

The Conceptual Architecture Diagram is a lightweight and yet powerful tool for sketch-prototyping the system structure, and for rendering the structure of an existing system, to explore adaptations to it. Thus it allows us to explore the system from its earliest conception through evolution. As we experiment on paper, it is cheap to postulate various alternatives and explore their ramifications.
Early Conceptual Architecture exploration (depicted figuratively)
Later Conceptual Architecture elaboration and documentation

As system capabilities are defined, the associated responsibilities are allocated to “chunks” or components of the system, and these responsibilities are an important thinking tool for architectural designers who need to make software abstractions cognitively malleable and communicable. With something so simple as elements each with their list of responsibilities and relationships, we can experiment with different factorings and hence different abstractions or elements each with a different basis for cohesion of responsibilities. Further, it becomes possible to assess:

  • clarity of responsibilities
  • whether responsibilities are missing
  • components for cohesion of responsibilities
  • the system shape for balanced distribution of responsibilities

Architectural components may play different roles in the system, so their responsibilities are related within role-scopes. That is to say, for example, a component may play a minor role in system health monitoring and an active role with respect to some other system capability (e.g., providing shopping cart functionality). 

In highly dynamic systems, as software intensive systems are, structure and dynamics are designed together — if we focus on structure we impact dynamics, and vice versa. When we focus on dynamics — on behavior and “how it works” and properties that emerge from the structures as they perform their function and interact — our design decisions have to be expressed in terms of the structure for the structure houses, if you like, the implied responsibilities and interfaces.

The conceptualization of a product isn’t limited to the conceptualization of its use, but the product in use. We’re designing what the product is capable of in conjunction with what the product is and how it is built. A car isn’t a faster carriage, and although early cars had aspects of a carriage, the engine was quite different than a horse, and this translated to mechanisms to make the wheels move (from passive to active). In other words, when we conceptualize a new kind of product we iterate across designing capabilities and the mechanisms and structures that deliver those capabilities, and the product changes the ecosystem (a gasoline powered car raises the need for places to replenish fuel) so to make the product successful we also (impact the) redesign (of) key aspects of the ecosystem or larger system of systems into which our system fits, interacts and in important ways shapes and is shaped by.

“In most people’s vocabularies, design means veneer. It’s interior decorating. It’s the fabric of the curtains and the sofa. But to me, nothing could be further from the meaning of design. Design is the fundamental soul of a man-made creation that ends up expressing itself in successive outer layers of the product or service. … The essence of the iMac is to be the finest possible consumer computer in which each element plays together” — Steve Jobs

Conceptual Architecture grants us passage between the from-the-outside and the from-the-inside views of the system, allowing us to design interactively across the concept of the system and the internal concepts that allow that conception to take form. And it allows us to think about the elements and how they play together. More specifically, it is the conception and narration of the organization of the parts or elements of the system and its key, or architecturally significant, mechanisms — as the system is conceived, developed and evolved. More pivotally, it is an agent of consilience, helping us to bridge various divides in system design.

Some related resources and ideas:

Comments

Conceptual Architecture: Why

Why: Motivation for the Conceptual Architecture View

Conceptual Architecture is a key medium for describing the “big picture” and essential design ideas of our system, helping others to more rapidly comprehend a complex system, how it is composed and its critical mechanisms or interworking to achieve some key internal system capability essential to sustaining itself. The Conceptual Architecture Diagram serves as a high level map of our system, providing for navigation around complex systems, location of responsibilities, and identification of dependencies.

But Conceptual Architecture is also a focus of essential design work! As we conceive of the system structure, we’re tackling questions of decomposition and modularity, with the dual of composition and emergent properties. We do so to make the system organizationally and cognitively tractable, bringing order to the system. We might think of this as a kind of “social order” if we see its elements as agents of system responsibilities, working with cohorts to deliver system capabilities and in this interaction effecting emergent properties. But even if we don’t go that far, we are at least bringing a kind of mechanical order, complete with hierarchically composed sub-orders, to the system.

Modular systems enable us to:

  • simplify the design with cohesive units of responsibility
  • partition realms of uncertainty and experiment, separating them from more stable and understood areas of the system;
  • separate areas of the system that change for the same reason (for example, dependencies on underlying technology), from other areas of the system that have different change impulses;
  • create areas of focus for investments in highly tuned performance or extreme robustness and resilience;
  • separate elements for reuse or leverage; and
  • create units of accountability. (Whether we organize development primarily by features or by components, it is helpful to have clear component ownership so that the structural health of the component is an explicit charter.)
  • Early on, we’re dealing with our conceits of formative system elements — on paper. We’re crafting lo-fidelity sketch-prototypes of the system structure, and iterating across the conception of the system and our discovery and exploration of how we might conceive of, arrange and build the system. Because it is cheap to do, relying more on how generative our imagination is leveraging the grist of experience, patterns, capabilities in other systems and frameworks, and so forth, we can explore various alternatives, discovering elements and relationships, illuminating the system conception or possibilities we might offer in terms of system capabilities. We can play with different factorings of responsibilities into capabilities and onto elements. And we can evaluate our postulated alternatives with respect to coverage or completeness as well as balance, harmony, conceptual integrity and consistency, in addition to the achievement of desired system outcomes. All of which allows for a much more active, adaptive, creative exploration of the system concept and structural organization before the direction is channeled and cast in shaping, anchoring expectations and code.

    We don’t claim to be able to so well conceive of the system in its ultimate form that we go into system development with a perfect and complete design. We do claim that we can begin the evolutionary adaptation process with a more plastic and malleable medium than a growing mass of code. And then we can support code evolution with, again, a more malleable medium for exploring architectural-level refactoring and adaptations — changing sketches and more formal models to explore the impact of ideas before we incur the cost of making the changes in the code base. All the while maintaining greater intellectual traction over a complex system, because we have cognitive aids to system understanding and work with system constructs in their own terms, rather than always and only in terms of code which contains details that obfuscate.

    Some related reading:

  • Decisions, Concerns …(re-)Defining Software Architecture
  • Why Do We Need Software Architecture?
  • What architecture is about
  • Comments

    Conceptual Architecture: What

    What is Conceptual Architecture?

    “Conceptual Architecture” is the conceptual view(s) of the architecture of a system. It describes at a broad brushstrokes conceptual level the significant design ideas of the system. In particular, this view includes diagrams and text which identify, explicate, rationalize and contextualize the key structures and mechanisms of the system, and the relationships among them.

    Conceptual Architecture expresses the key architectural elements and their relationships that give “shape” to the system. Each of these architectural elements, or abstractions, play a critical role in the system which is signified by their responsibility assignments. They collaborate and interact, and these relationships are a significant design consideration not only because interaction points (analogous to articulation points in physical systems) and “seams” in the system are non-trivial, but because interactions give rise to synergy that makes the system more than a simple aggregate of executing parts. The parts, their arrangement and concert, cohere within and give expression to the system “gestalt.”

    Conceptual Architecture Diagram

    The Conceptual Architecture Diagram renders the formative architectural abstractions (named boxes) and their interrelationships (lines). For complex systems, there may be a set of such diagrams, exploring the (de)composition of more complex, architecturally interesting architectural elements.

    Component-Responsibility-Collaborator-Rationale (CRC-R) Descriptions

    The Conceptual Architecture also identifies the responsibilities of each of the architectural elements. We advocate doing this on Component-Responsibility-Collaborator-Rationale (CRC-R) Descriptions* for each (conceptual architecture) component, so that the reasoning behind component design choices is captured along with the responsibilities (connecting the dots to desired outcomes, strategic decisions, lessons from experience and grounding assumptions, etc.), and dependencies and other relationships are also recorded. It is a simple and “just enough” format that matches the spirit and intent of Conceptual Architecture, supporting conception, communication and evolution of the key design ideas, including the fundamental organization, of the system.

    Mechanism Sketches

    Beyond the (set of) Conceptual Architecture Diagram(s) and CRC-R descriptions, architectural mechanisms (i.e., mechanisms of architectural significance) are sketched (in diagrams and descriptions) to conceive and convey their essential design nature — the design intent, contextual assumptions, structure, and “how it works” dynamic considerations. The capabilities of the system emerge from the inter- and intra-working of the parts of the system, and mechanisms allow us to focus on specific processes within the system, conceptually (at this point) designing architecturally significant functions of parts of the system.

    Architecturally significant mechanisms are those that have more diffuse or systemic impact or are make-or-break important to system outcomes. Many of the patterns in our field’s literature formulate tried-and-true mechanism designs addressing very specific system capabilities (often internally focused at system sustaining and structural integrity concerns because these are common across systems irrespective of their specific user-facing functionality). A system has functions just like a body has functions. And many of these are internal system sustaining functions that have to do with continuity rather than serving any immediate external demand being made of the system.

    Together, the expressions of the Conceptual Architecture form a conceptual framework within which we conceive of, reason about, communicate and share, extend and reify and evolve the key design ideas of the system.

    References and More Information

  • Conceptual Architecture [Draft for Review], Ruth Malan
  • Conceptual Architecture and example Conceptual Architecture and Why Bother with Concepts, by Doug Newdick
  • Our draft chapter on Conceptual Architecture (the book has been reconceived and is being rewritten)
  • Key to conceptual architecture is the notion of abstraction. I need to rework it, but this piece on Software Abstractions gives an introduction.
  • Logical Architecture is the actionable set of architectural design descriptions or specifications, and includes interface specification.
  • Context: see Visual Architecting Poster and Core Activities Chart.
  • Conway’s Law and architecture of responsibility
  • The influence here also includes the RDD (responsibility driven design) work of Rebecca Wirfs-Brock, as well as work we were doing on Team Fusion under the leadership of Derek Coleman.
  • Comments

    The Art of Change: Fractal and Emergent

    Our “The Art of Change: Fractal and Emergent” Executive Report covers
    1. a model of change, showing how the vectors of change are different at different points in the lifecycle, so that agility means different things, depending on where in the lifecycle the product-market is
    2. a discussion of how the meaning of business and the meaning of design are shifting
    3. Jeff Bezos notion of fractal strategy, leveraging it to illustrate how fractal strategy enables intrinsic agility
    4. positioning IT as a leading player on a strategic stage where relationships and business intelligence are key drivers of innovation and agility
    5. the tandem role of strategy and architecture in an agile business and the implications for architects
    6. a fractal notion of leadership, in a business that relies on fractal strategy and tandem architecture to combine intentional goal-seeking with emergent responsiveness

    Business strategy and its tandem architecture creates coherence of purpose and concert among the many socio-technical systems, the many smaller pools of action and influence, within an organization, so that bigger, more ambitious, impactful things get done. While embracing emergence or extemporaneous dynamic responsiveness, we also note that strategic differentiation takes intentional focus to align inspired, creative, inventive thought and action so that many contributions of mind, will and hands build the systems that create and sustain competitive distinction in the market.

    We borrowed Jeff Bezos’ image of strategy happening fractally at Amazon, and put words to what is done, varyingly, in organizations. The important thing about creating this image of fractal strategy and tandem architecture though, is that it gives us a way to have the conversation about the relationship between strategy and architecture. Why? Because there is inconsistent understanding of the role of strategy, let alone architecture!

    In some organizations, strategy is ignored or derided — they claim there is “no strategy,” and that is treated as a point of cultural pride. A point of cultural pride. Hmm, that sounds like identity, which is a key part of strategy.  So strategy in the organization is fractal, with an independent “cowboy” (shoot first and aim after) culture set as the unifier at the corporate level, and other elements of strategy pushed out to the business elements. But as soon as that company wants to achieve something more coherent across its businesses, it finds itself needing to work strategically and architecturally to create a shared intent and the relationship platform for enabling that coherence. So, whether “dynamic, organic, fractal strategy” enters their parlance, allowing them to explicitly talk about intentional and emergent strategy or not, they have to get more intentional if they want to do those bigger things that require concert to make them more the way they would like them to be (Herbert Simon’s wonderful way of defining and motivating design).  

    The impetus for writing this report, was an increasing rumbling around the future of IT and EA. Well, of course we know IT and EA has a healthy prognosis. Still, many choose to see IT as a cost-center — one that encumbers with a mish-mash of entangled, brittle systems, and expensive tastes in technology frills that can’t be afforded in lean times, at that. So it is worth articulating the counter-position, don’t you think? Anyway, that’s a key message — articulating the role of IT and architects (product, system and enterprise) in sensing, catalyzing and responding to change.  So the report makes points like:       

    “Many of the business capabilities that IT supports and enables have to do with building and maintaining relationships and their information spaces to run the business and create strategic advantage. … 

    Relationships, both formal (with codified transactions) and informal (with dynamic, even ad hoc, interactions), are enabled through high connectivity. In Connections, James Burke, commenting on the Gutenberg printing press, observed “the easier it is to communicate, the faster change happens.” Alternately put, new ideas come about through conversations, and conversations through relationships, and increasingly these are digitally enabled and/or enhanced.”  …

    “When we recognize that this is a world where organizations increasingly compete on and for relationships, perception, and fidelity, and on information leverage, the strategic role of IT jumps into sharp relief. Place this in a context of change, and IT finds itself with a leading role on the strategic stage. Whether it is playing the role of the proverbial bad guy responsible for runaway costs and change encumbrance or a partner in a landscape-defining dance of change depends very much on how well IT is integrated into strategic decision making — at various levels in a fractal approach to strategy setting.”

    and

    “Complexity is a key driver of architecture. That is to say, as complexity increases, so does the need for architecture. It is not that we want complexity to go away, for value comes hand in hand with complexity. Instead, we want to harness complexity and, as it were, to tame it so that it serves rather than obfuscates and subverts the value we are creating.”

    and

    “The role of architects in an agile enterprise, therefore, includes taming the transmogrifying mess created by responsiveness, dynamic learning, and accommodation, even while leading with intentionality to innovatively envisage, build, evolve, and sustain systems and their explicit, enabling and constraining architecture decision sets.”

    – Ruth Malan and Dana Bredemeyer, The Art of Change: Fractal and Emergent, Cutter Consortium Enterprise Architecture Executive Report, Vol. 13, No. 5, 2010.

    We hope that the Report persuades managers and architects that there is an important relationship between architecture and strategy, and that relationship doesn’t have its foundation entirely in the business side, nor entirely in the technical side — but rather in a partnership where strategy and architecture work together collaboratively. That is, they inform and are informed by each other, enhance and are enhanced by, lead and are led by each other. And I hope that the paper unfolds the salient topics in an accessible manner — accessible across the languages of business and technology — to motivate and enable that dynamic tandem relationship.

    The Art of Change: Fractal and Emergent is the first in a two-part series, and focuses on the what and the why. Part Two, The Art of Change: To Lead is To See, To Frame, To Draw focuses on the how. We hope that you find the Fractal and Emergent paper, with its focus on agility through fractal strategy and tandem architecture, inspiring and useful. If so, you can play a role in Part Two, helping us improve it by becoming a reviewer or simply by providing encouragement.

    You can download a complimentary copy of The Art of Change: Fractal and Emergent at http://www.cutter.com/offers/artofchange.html.

    Comments

    Why Getting Past “But…” is Important

    You’ve probably read Getting to Yes and heard of Getting Past No, so why Getting Past “But”? Well, because “but…” is insidious, making it harder to get past than an outright “no.” The person who says “yes, but…” is ostensibly aligning with you. Ostensibly agreeing but for this teensy caveat—this objection that is a showstopper! It can be resistance in a subtle guise, seeming passive yet inherently active—the kind of action that is actively rationalized non-action. Or it can be genuine goodwill—indicating a real desire to orient with you, and active intellectual, creative engagement. The trouble, though, is that “but” can become a barrier. We need the attitude that looks beyond “but.” If we look only to “but,” only to the objections, the reasons why not, we stop there. We need to look to what we want to accomplish, then figure out how to get there from here. We need to look beyond “but” to get past “but.” Yes, this is the stuff of “kindling the collective mind,” engaging others in seeing how we would like things to be, then engaging their creativity in resolving how to get there.

    I’m told “I agree with you, architects should play an active role in requirements, but reality in my organization is that the structure and process doesn’t allow that.” Yes, that reality is hard to change. And it will not, so long as the one person who could begin to make the change, the person who sees that the change is needed, doesn’t start to lead the change! First, to see the need, then to help others see a better future, then to enroll them in creating that better future.

    This by way of an architect’s signature on an email I received this evening:

    “Reasonable people adapt themselves to the world.  Unreasonable people attempt to adapt the world to themselves.  All progress, therefore, depends on unreasonable people.”

    George Bernard Shaw

    Yes, change can be hard, and it can take a long time to even get to the point where people recognize the need for change. It took Madison five years to get the parties to the table to create change. Hopefully it can take us less time to restructure the status quo in software development. Our Getting Past “But” paper is one resource you have to help shift perception and expectation. The Agile Movement also helps underscore the importance of multi-disciplinary teams. If your organization’s approach to scale and complexity is to do just enough requirements and design upfront (for example, to spin off concurrent teams with enough context), you can still leverage the learning that the Agile Movement has embraced—multi-functional teams, iteration and stakeholder participation allow more concurrency to happen earlier, with better outcomes.

    Comments

    Scaling Agile with VAP: Getting Past “But…”

    Our Getting Past “But…” executive report covers two essential areas:

    1. innovation, the circles of innovation model, the innovation process, and what all this means for architects.

    2. scaling agile development projects with VAP (emphasizing just enough design upfront or JEDUF).

    These map roughly to the first and second halves of the report, though we encourage those interested primarily in the material in the second half to also read the first half. What follows is a brief outline of the contribution of these two parts of the Getting Past “But…” paper.

    Innovation and Architects

    Innovation is rampant, and clearly companies big and small are innovating apace. At the same time, many companies are disappointed with the return on their innovation investments. Industry incumbents are adapted to the status quo and defend their inertial tendencies with “but..”: but we aren’t chartered to do that, but that’s too risky, but our customers aren’t asking for that, but that would cannibalize our market. Focusing on immediate releases and incremental improvements, thwarts competitive landscape reshaping innovation which then tends to come from outside, often from start-ups.

    Getting past “but…” takes a shift in attitude. It is true that this shift is fostered by empowerment, with an innovation culture established by top management. Google is the prototypical example there. It is also true that the shift can start with the individual. The creation of masking tape (the 3M story starts around minute 21 of Scott Berkun’s presentation) is just one example where the inventor used his own initiative and limited budget to fly a skunkwords project under the radar. We need to recognize that generally there isn’t a shortage of ideas. And, in aggregate, there isn’t a shortage of willingness to take risks—witness the number of failures, including startups. What we need are more effective ways to get good ideas on the table, sift for those that make good business sense because they create high customer value that can be used to build differentiated identity and strategic advantage in the industry, put them through early, quick and cheap failure/improvement cycles, and get more and more talented peoples’ heads in the game to build the system.

    Our Getting Past “But”: Finding Opportunity and Making IT Happen report speaks to role and process changes that empower design teams to create a new competitive basis through differentiating innovations. You can download a complimentary copy from Cutter Consortium at http://www.cutter.com/offers/findopportunity.html.

    In this paper, we take the position that architects need to be part of, if not lead, the innovation team. The architect’s role is to help the business identify opportunities to create value through capabilities that technology brings to the table. This leverages the unique perspective of the architect into technology and the organization’s technical capabilities, but it also leverages the architect’s unique skills in system thinking and modeling.

    VAP and Scaling Agile

    VAP (the Visual Architecting Process) is all about being agile even when the complexity of the system, and the organizational unit(s) building it, demands just enough upfront designfor example, to launch concurrent agile teams. What VAP emphasizes and enables is parallelizing the requirements and architecture iterations, with intensive stakeholder involvement as pertinent to the quick cycles. VAP can be applied during coding cycles, but for complex systems, early VAP cycles use the cheapest possible artifacts (e.g., models and prototypes) for learning quickly about stakeholder value and architectural challenge. This concurrency, together with the principle of stakeholder involvement (including but not limited to end users), is a major value and contribution of VAP.

    We write about JEDUF and agile architecture in the Getting past But paper. The content of the report is so important to the conversations we’re having, to the challenges of organization after organization as the hype around agile pushes larger and larger projects to experiment with agile development. As we do so, we need to leverage all the lessons of our histories creating complex systems, as well as the lessons and values of agile development, to adapt a process that works for concurrent development of complex systems.

    References

    This synopsis derives from writing in Ruth Malan’s (almost daily) architecture journal.

    Ruth Malan and Dana Bredemeyer,  “Getting Past “But”: Finding Opportunity and Making It Happen.”  Enterprise Architecture Executive Report, Cutter Consortium, August 2008 You can download a complimentary copy from http://www.cutter.com/offers/findopportunity.html.

    Comments

    Conway’s Law

    The Wikipedia community describes Conway’s Law like this; I paraphrase it like this: if the architecture of the system and the architecture of the organization are at odds, the architecture of the organization wins. The organizational divides are going to drive the true seams in the system.

    The architecture of the system gets cemented in the forms of the teams that develop it. The system decomposition drives team ownership. Then the organizational lines of communication become reflected in the interfaces, with cleaner, better preserved interfaces along the lines where organizational distance increases. In small, co-located teams, short-cuts can be taken to optimize within the team. But each short-cut that introduces a dependency is like rebar in concrete–structurally efficient, but rigid. If the environment changes, demanding new lines to be drawn, the cost becomes clear. The architecture is hard to adapt.

    One could say this is part of the innovator’s dilemma. Sustaining innovations, that is, incremental improvements within the cast of the architecture, are what the organization is adapted to be good at. But when a breakthrough innovation demands a new architecture, a new organization (unencumbered by power trees that grew up around the old architecture) tends to be more fleet in bringing the innovation to market.

    Another implication of Conway’s Law is that if we have managers deciding on teams (what they’ll do, who will be on them, and how they will relate), and deciding which services will be built, by which teams, we implicitly have managers deciding on the system architecture. They determine system chunks (services or components) and capabilities by deciding who will build what.

    Conway’s Law also kicks in if we take an initial guess at the system decomposition (a first-cut conceptual architecture), allocate subsystems to teams, and sally forth–the team boundaries will tend to become boundaries within the system. Anything else will be a feat of architectural heroics; hard to accomplish, when architectural heroics have to compete with schedule heroics driven by the steady beat of integration clocks. Yet, architecture is where we address cross-cutting concerns, or at least those that needs-must be addressed with a system perspective so that when it comes time to compose the system it will have the properties stakeholders care about, rather than emergent properties that may or may not suffice.

    Roles are defined by their responsibilities and associated decisions. Architect is a role. Any person may play one or more roles. That is, the architect role may be shared among a group of people (as in many agile project teams), or one person may hold more than one role (as in many small teams, especially in startups).  This may be overt and declared. And it may be the result of decisions that are actually effected. If management decisions determine the architecture of the system, they are in effect its architects. If developers determine the architectural decisions, they are in effect its architects.

    “Duh!” you might well be saying. Yes, a lot of what is absolutely common sense when it is put plainly, is so obtuse in the face of perplexifying reality. Simply witness all the heated arguments and misunderstandings you get around the topic of the role of the architect.

    But what does it mean? Architecture needs to happen across the interfaces, and this also means across the system/organization interfaces. It means that system architects (who we call architects) and business/organization architects (who we call managers) should not work as if one has no impact on the other.

    Other references:

    Why do we need software architecture (how architecture serves organizational and technical purposes)

    Who needs carrots anyway? (about the iron triangle and the interface between management and architects)

    Comments

    « Previous entries Next Page » Next Page »