8 reasons not to leave architecture to the end of a project

Sometimes I see architecture left until the end of a project. These 8 reasons are the reasons I normally try to push for it earlier.

There have been times in the past where I have been pulled in at the end of a project and told that the project has implemented a technical solution and there’s a requirement by the customer for it to be documented and when that happens I get a little sad inside. In large projects one person never has all the information. This blog is from personal experience from working at different companies over the last 25+ years.

Its worth mentioning at this point that design doesn’t all have to be done at once in a waterfall fashion it can be agile. I will talk about this in later blogs.

Documenting architecture always has value, but much of the value you would get from an architecture practice has gone if you start doing architecture at the end of a project. The bigger the project the bigger the problem. Lets get started:

1 – It takes more effort doing it at the end.

When you do architecture at the end of the project you have to take time to understand everything that’s happened. This includes understanding how requirements have been realized and what decisions have been made

As you start to understand the scope and concepts behind a solution questions arise. I might go to one person and ask the answer – they typically wont know. or may have part of the story. They lead me to other resources and it could take me hours of time just to find out why a single decision has been made.

If a team had been working together on a single architecture throughout a project you don’t burn that investigative time trying to understand things. It’s not always obvious what necessitated a decision, so its important to track decisions in a single place.

2 – Its expensive

Time is money – exploring the rationale behind decisions and understanding whats happened at the end means having to engage lots of people. If an architect has a meeting with three technical specialists for an hour that’s 4 hours of work time. When they don’t have the answers to hand – they have to write emails, contact other people, and come back to the architect – that’s also burned time. If the decisions had been put into an architecture as they were made, in some projects its possible to save hundreds of hours.

The same idea applies with things like requirements realization. If you haven’t shown exactly how each requirement has been realized in architecture you will have to investigate each one. If you are in a project with a 1000 requirements and spend 15 minutes investigating each requirement then potentially you would have to spend 250 hrs just figuring this out (more, if you consider you need to engage other people).

Map the requirements at the beginning of the project and make sure you create some kind of requirements realization within a project. Keep a log of all the decisions made, together with the activities taken around each requirements together in one place. This extra effort will save time and money as a project moves on and resources come in and out of projects. It will kill the need for new architects (or other people) having to “investigate” to understand status at the end of a project and will reduce the number of meetings you need during the project.

3 – Its hard to build well under time constraints

At the end of a project there are normally time pressures because people want to shut things down.When I am having to investigate the architecture designs and their rationales, I need to take time. If organizing a meeting to understand how we realized one requirement takes an hour, and may not yield the result – other meetings will need to be scheduled in. In a situation when you are given a week to deliver a design near the end of a project – that time disappears very fast if you need to have multiple meetings and rely on peoples availability. This puts deadlines are at risk.

If we are building the architecture throughout a project there is no rush at the end of a project.

4 – Information gets lost

If decisions are made and not recorded in an architecture information very easily gets lost. In large projects many people are involved – sometimes resources get replaced over time and we lose information on why decisions were made. Sometimes that information is recorded but we don’t have access to it. If you have a large project spanning 6 months, that is running a project meeting every morning where they make decisions that’s potentially 120 sets of meeting minutes – assuming those minutes are kept properly and all the information on decisions is available you still need to sift through it. Of course some people make all their decisions in email, sometimes people leave the company, some also forget or don’t write things down.. In these situations you have real problems.

Keeping architecture concerns and decisions documented in a work log as part of your architecture design process throughout the project mitigates this risk. Having that log available on a collaborative platform is essential. if you have to take a conversation in email – paste that to the collaborative log too, so everyone knows everything that happens and there are no hidden caches of knowledge.

5 – You find things that are missed

When you take a structured approach to analyzing requirements and creating a requirements realization view for your solution, things don’t get missed, because an architect is guiding work. This is one reason why we do architecture design in the first place – to make sure a solution is fit for purpose. I’ve often seen functional requirements well considered by a technical team but non functional requirements missed. A technical team tends to focus on things like getting a server up and showing a tangible result.

If you come to the end of a project and suddenly realize the technical solution doesn’t meet your requirements, you are in a bad position. It’s too late to rebuild or change technologies. You may end up having to compromise quality just to get the project ended, or investing a significant amount of time and money re engineering or rebuilding a solution.

The earlier you do a requirements analysis the earlier you find gaps. The hard and sometimes boring requirements analysis part at the beginning of a project is essential. You must make sure that every requirement is realized in a view. At this point you should be also ensuring every requirement has an owner that is managing it. The project managers can then more effectively chase up project status without having to call lots of expensive meetings.

6 – You’ve lost the benefit of concerns management.

There are two ways you can approach things. You can put out a fire in a burning house, or you can stop the fire from happening in the first place, by taking away the matches from the idiot that likes to start fires.

When you are doing architecture properly you are identifying and managing risks before they happen by applying thought and designing things to stop issues from arising. A design uses the shared experience of all those involved to avoid problems rather than the experience of individual resources. Of course you can also reuse designs, but that’s another story.

On the flip side, you can leave implementation of projects to technical resources. and manage concerns as they happen. When this happens you have an escalation culture. You normally end up with managers and a lot of emergency meetings to handle problems, often after the project is implemented, leaving a sour taste in the mouth of both operations and the customer. This is very expensive because its a lot of burned meeting time because you are handling the after effects of a fire happening; which also includes a level of damage control. If a house has caught fire several times during a project the damage has already been done. I’ve seen before entire platforms have to be rebuilt because technical teams have rushed to implement a solution without realizing the requirements or dependencies of other systems around them. I have also seen the same problems reoccur.

Concerns cannot be managed at the end of a project because you cannot go back in time to fix them. The only thing you can do that’s cost effective is manage them as they happen or suffer the cost of potentially needing to redo things.

7 – It ends up frustrating people

When you have the end of a project in sight you want to cross the finish line, the project don’t want to have to deal with an architect coming in at the last minute and finding holes in their work. It frustrates everyone, architect included. If you have left architecture to the end the chances are the budgets are depleted and people are exhausted through overwork because the plan that was executed wasn’t designed and work wasn’t properly estimated. The last thing anyone wants at that point is to realize a technical implementation is not fit for purpose when they cant do anything about it – thats demoralizing.

8 – It results in lower quality

When you consider everything above you will realize at the end of a project to get architecture documentation out, you are going to have to compromise. This for me personally is very hard, because I always want to deliver the best. It may be that the documentation will end up at a high level, or it may be that some concerns will just have to be accepted rather than mitigated. It might be that we need to make assumptions about why things were done, or that we don’t get to model our solution because there just wasn’t enough time. The team may have made costly mistakes during the project. At this point you have ran out of time to address all the concerns.

Its very likely that the quality of architecture will be lower if you used the time at the end of a project just to document architecture, rather than following an architecture practice during a project. Its also likely that because of this approach there will be issues arising once a project goes into service.

If you have viewed architecture as a documentation exercise at the end of a project the chances are that what is implemented doesn’t address all of your own internal non functional requirements, the customers requirements, or other stakeholder needs – because the time at the end of the project was focused on describing what has been put in place rather than what really needs to be put in place.

Architecture and security non functional requirements normally exist to ensure we meet the standards our companies promise, they are designed to protect the company, to mitigate risks risks. They also may address the needs of internal stakeholders and may have been put in place to ensure we learn from the mistakes that our predecessors have made. . If we haven’t designed a solution to meet the needs of all of our stakeholders, then we may well see the same issues reoccurring time and time again.

There’s no worse feeling in the world than having to present forward an architecture that you cant justify or isn’t as good as you would like it to be, because of things that were outside of your control. In such cases I think its always important to be honest, to present the architecture, and also the concerns that you have on it at the end of a project so they can be addressed by other architects or your quality team.

To raise quality we also need to raise awareness. architecture may be difficult and laborious at times, but as I have mentioned in this blog, people need to know the benefits of not leaving it to the end.

ArchiMate – Looking Beyond Diagrams

When looking at ArchiMate diagrams its easy to miss the fact they are part of a model, and not just pictures. This blog addresses the difference and demonstrates some basics around models.

Last week someone was commenting one of the views I demonstrated in a blog, and pointed out something that didn’t make full sense in the diagram but had been addressed in the view documentation. Its very easy to forget that we are not talking about pictures when we speak of ArchiMate models. In the blog my ArchiMate views are just images. I wanted to show some practical examples and discuss this.For anyone that doesn’t actually do modelling, but interacts with those of us that do, this blog is for you.

Before I get started its worth mentioning that a view and a viewpoint of architecture is a concept that extends beyond the modelling language. We tend to talk about views visually, but the view taken from a specific viewpoint could expressed in writing if we wished it to be, but you would miss the benefits of modelling elements and relationships with a tool.

Diagrams

The dictionary definition of a diagram is:

“a graphic design that explains rather than represents especiallya drawing that shows arrangement and relations (as of parts)”

Of course there are many dictionaries but they all follow a similar pattern.Technically Figure 1, is both a view and a diagram. On this website its a diagram, which is a representation of an architecture view. the actual full architecture view has a lot more information in that is represented in figure 1.

Figure 1 – An Organization View / Diagram

Of course we can just use something like Paint or Visio to create a view like above – and it has a lot of value, but in those tools are for visual representation, and not modelling.

Documenting Models

I previously addressed Documenting An Architecture Model – but I wanted to give a more complete example in a modelling tool.

Figure 1 is a fairly self explanatory made up view which gives a level of information by itself. But it doesn’t tell us everything. I will show some of the things that differentiate pictures and diagrams from an architecture view in the rest of this blog.

The first thing to show is that views can have their own documentation – Figure 2 shows this – I have selected a view and you can see its documentation in the field below.

Figure 2 – View documentation

Of course if this was a real view I might also have provided links in the documentation to the organization chart. Although the view documentation is short here, it helps set the context for the view, and does provide some valuable information.

Individual elements or relationships can also be documented:

Figure 3 – documentation for an element

As well as elements having documentation relationships can:

Figure 4 – Documenting a relationship

The relationship type gives you an understanding of what kind of relationship the elements have but it doesn’t tell you why. That’s something I will typically do in documentation. This is particularly important when it comes to documenting less defined relationships, such as associations.

The point I am making is just looking at the diagram does not give you a complete picture. Some tools will allow you to auto generate documentation:

Figure 5 – Word documentation

Metadata

Many tools allow you to connect all kinds of metadata to elements & relationships, and allow varying degrees of customization in the example below you can see a references link I added:

Figure 6 – Properties

Its very tempting to add lots of metadata to a model because you can, but i tend to think carefully about the value of any kind of information I would want to maintain.

Some tools such as Enterprise Studio, allow some pretty advanced analytics depending on the information you store with the model.

Navigating the model

The main thing that sets a modelling tool apart from a visual tool like PowerPoint and Visio is the fact its a modelling tool. We can navigate each element and see its relationships and the different views it appears in. The tool stores the elements and relationships as a fully relational model

Figure 7 – Navigating an element

Auto generating views

Lets say for example a stakeholder came to me and asked me to tell them a little bit more about what an enterprise architect is – With some tools I can automatically generate views. Take a look at a view I generated literally in 5 seconds:

Figure 8 – An Auto Generated View

In previous blogs I have used the enterprise architect role, and because the modelling software knows all the roles and relationships between each element in a model, and the rules for modelling in archimate we can very quickly create something meaningful.

The more work we do with our models, the more value they can bring us.

Architecture Views

An architecture view is a view into a model. Figure 9 shows our auto generated view but on the left hand side you can see in the model browser that our model package contains many elements that are not on this view (because we created this view to only show actor cooperation’s to the enterprise architect element)

Figure 9

The model package contains many elements, views, and relationships. You see them on the model browser. You could theoretically take all those elements and put them onto a view:

Figure 10 – A fairly small model.

Theres too much to remain readable on a normal screen – and it doesn’t make sense to show our architecture in a single view. This is why we break down architecture into multiple views, created for people interested in its different aspects.

Figure 11 – A view

To put in simple then. a view is just a small part of a model we are communicating, As you can see, the view in Figure 1, would be part of something much bigger.

A Viewpoint

So we can see that models have lots of elements and relationships, and different people have needs to see different things. View Points define which parts of an architecture we look at. A viewpoint definition tells us which types of elements we are interested in, and which types of relationships are allowed between them. There are many standard viewpoints defined in places like TOGAF and the Archimate Manual.

Figure 8 was an automatically generated Actor Cooperation View. It was generated from a Archimate 2.1 Viewpoint. The viewpoint definition tells exactly what elements and relationships are allowed within the view, so our tool could find those within the model and generate a view from the viewpoint definition.

Derived Relationships

As your models become well developed benefits start to come from automatically derived relationships. Not every relationship we use is created by hand. You can find more information on them in the Archimate Manual

Summing It Up

As your model grows the value we can gain from it also grows. For a model to realize its full potential requires architects to actually use it, have competency around it and have a level of discipline. When we achieve that modelling can be used to save time in understanding your enterprise and understanding the impact of change across the different IT aspects. At this point it can become an invaluable tool for management.

7 Reasons I like ArchiMate

Anyone who knows me, knows I use ArchiMate a lot. I know its not perfect, but here are 7 Reasons why its my modelling language of preference.

I spoke about it briefly before in Architecture Languages and Frameworks. To get the full value from ArchiMate you really need to understand it. These are not ordered in any particular order; I am interested what other people think. Here goes:

1. It is a standard

It’s a widely used well documented vendor independent standard. Every element and relationship type is explicitly defined and as such enables improved communication between people across enterprise boundaries

2. It helps focus on value

You can easily define reusable viewpoints that are exactly what different stakeholders need and develop your catalog of viewpoints over time. For Example, if a Product manager decides they want a conceptual product architecture viewpoint, just to help everyone consistently understand products and services aligned to the technologies they use, we can define exactly which elements are allowed, and which relationships. This gives you consistent information that that stakeholder needs every time.

3. It covers many aspects of architecture

It can be used to model most things, some more naturally than others – It enables us to connect the physical world with technology, application, business, implementation & migration and the motivations behind them. I don’t know of any other modelling language that does that in such a balanced way. It covers all the IT aspects.

4. Its easy to draw

I am not a talented artist – But in a meeting with a white board, I can stretch to drawing some stick figures. Its easy to draw elements nicely with a whiteboard pen, and there’s nothing there more complex to draw than box and a few lines. (But not 7 red lines)

5. It aligns nicely to other standards

Its built by The Open Group, Creators of TOGAF and as such aligns with that standard really well. Its also fairly easy to align with the ITIL Architecture Processes.

It doesn’t advocate a way of working.

In addition it connects nicely to ISO 42010, the standard for architecture description. ISO 42010 talks about elements and relationships, and ArchiMate extends those concepts by creating standard element and relationship types that extend the vocabulary.

6. It helps structure your thinking

Because of strict definition of element types and the relationships between them, after becoming familiar with the language I found myself looking at architecture problems in a specific way – breaking down architecture concerns into these modular elements. As a consequence it becomes easier to start to create modular pieces of reusable architecture, which can help us create much more agile architecture, and give us more efficient usage of the resources we have.

7. Fantastic tools exist for modelling ArchiMate

There are many good tools out there for modelling ArchiMate- and when you model, it gives you the opportunity.to build a complex structure which enables us to leverage the benefits of understanding the connection between the layers. I will talk about that further in my next blog – but basically as the model grows so does the value we get from being able to look at it from different standard viewpoints.

ISO Compliance – An Architect’s Perspective

ISO Compliance gets a bad wrap. People roll their eyes, are bored by it, and often don’t see the relevance of it. I wanted to share my perspective on this and address some misconceptions.

International Standards Organisation (ISO) create documents that provide requirements, specifications and guidelines that can be used to ensure materials, products, and processes are fit for purpose.

Although I focus a little on ISO standards here you can apply my thoughts to most standards.

ISO Compliance & internal company process

Its a no-brainer that to be compliant to international standards such as ISO 20000, 9001, and 27001 we need to consider them in our processes. We should have processes for checking compliance and often compliance needs results in changes to our operational processes. Checking compliance basically involves taking a standard, turning it into a set of requirements and then going different exercises of requirements realization, or maybe even scoring your processes against criteria in a similar way to what I described in scoring documentation. My previous blogs on those subjects are just describing one of many approaches you can take.

Its important to ensure company processes validate compliance when they first create their processes but also whenever they change them. You cannot just implement standards and forget about them, because as processes change, so can the level of compliance to standards.

In many cases an ISO compliance certification in one country does not equate to having that certification in all countries. Just because your company may hold an ISO 20000 certification, this does not necessarily mean that it holds that certification in the country you are working in.

Adhering to standards isn’t just the job of a security or quality team. In order to really gain the benefits from ISO standards many different roles and stakeholders need to be considered or involved.

Compliance & Architecture

Its important for architects to be aware of which ISO standards a company states it is compliant to. It has a direct impact on both design and implementation in some cases. Its normal to want to short cut the compliance process. There are a few ways to do this;. One way is to create one set of master requirements that aggregates the requirements from all the relevant standards into a set of non functional requirements (NFRs) Those architecture and security NFRs need to be considered when both designing and implementing solutions, and mechanisms should be put into place to make sure that happens.

As an example – ISO 20000 asks that as part of release management a testing environment exists. An architect should plan for this when building implementation and migration architecture.

It is could be very easy to miss this requirement if architects are not aware of the international standards or company NFRs. These requirements are actually architecture concerns from a security and compliance perspective. Architecture concerns have to be managed in an architecture design.

Just as ISO standards can effect implementation and migration architecture they can also effect core architecture design. GDPR is a hot topic – around security of personal information, but for example ISO 27001 provides standards around all information security management.

Its not just a tickbox excercise

Compliance to standards may seem boring and just a pointless paper exercise and when you view it that way it starts to lose its value. The ISO standards have been put together by groups of smart people, that have developed a set of practices to mitigate risk and avoid pains they may have personally been exposed to.

I have heard from some managers in the past that “You cant expect architects to read a whole ISO standard”. Even if you have good non functional requirements from your quality and security teams I would recommend architects pick up at least one ISO standard and read it all. My personal favorite is ISO 42010 (The International Standard for Architecture Description). Read through an ISO and think about the value it’s recommendations give and the pains you would get from not following each recommendation.

For example, ISO 42010 talks about ensuring architecture concerns are framed by at least one viewpoint. If they were not, then the needs of our stakeholders might be missed. Things might not be managed and potentially a customer may notice it. Maybe even a major incident may happen – there could be a huge cost or risk.

If you start to read through a standard and maybe entertain yourself trying to imagine what kind of disaster may have happened that lead someone to write those sentences you may find yourself with a new appreciation of them.

In thinking about what is written in the different standards rather than blindly checking requirements off a list we can get more value from them, and we can learn from the mistakes of others.

Its an opportunity for group fun!

Those running a security, quality or architecture team wanting to get people engaged in ISO compliance could run a workshop with key stakeholders, and get them a little hands on with the risk management side of things.

If you lay out a set of requirements that have been distilled from an ISO standard, it stands to reason that not meeting those requirements poses a risk. If you are using a simple mechanism to validate compliance you could easily establish standard risks against each and every requirement being missed, and start to estimate impact and cost. Getting stakeholders involved in defining risk gives them a deeper more intimate engagement with risk management, and can also help later when those compliance processes are in use.

Its hard to argue that a risk is not valid, if you are the person that defined the standard risk is the person that violated it.

A message to all architects

If you are designing architecture, you need to be aware of standards you promise to adhere to for your customers. and within your own organization. If you cannot name all the standards your company promises to be compliant to, you need to at least be fully conversant with architecture and security non functional requirements. As architects we have a responsibility to ensure we are managing architecture concerns

Don’t think of ISO compliance as being a boring paper exercise. Think of it as an opportunity to take steps to ensure you have an easier, more relaxing, higher quality life at work.

Scoring Documentation Completion

This blog demonstrates a quick and easy way to score documentation completeness and improving quality which is more accurate than guesstimating,

The Goal

To be able to with a reasonable degree of accuracy estimate how complete documentation is, and to not need to take more that 30 minutes per document to do that. I need a balance between speed and detail.

Its not always the job of an EA to actually do these assessments but as an EA its good to get an understanding of documentation that’s based on an assessment rather than a guess at how complete something is, so I often teach this approach.

Some of the benefits of this approach

In following a structured approach we get

  • Better understanding of expectations. When the document authors understand the things we want to see the chances of getting them increases
  • Easy to visualize status – because we are quantifiable we can easily report status in a number of ways visually.
  • Better trace ability of progress – because again, being quantifiable means we can show improvement over time.
  • More realistic assessment of completion. Normally when i ask someone how complete something is they say 70 or 80% but when you start to look its often not so – people want to naturally please other people and tend to often give higher estimates, or sometimes do not think of the detail involved. This mechanism forces people to look at the work in more detail than if you were to “guesstimate”
  • Raises other quality issues. In doing an assessment like below you are reading and aligning to a set of criteria, which are the things that are important to you. Normally when If run a quick assessment i will come up with half dozen or so comments along side the review.

The End Result

I am looking to get an overall percentage of completion – performing an assessment of something i normally end up with a table which will look a little like this:

Figure 1 – A resultant score of several architectures.

Bear in mind the table is only an example. If I was creating scoring for high level design I may have more detail. You can see across the top I have listed some different architecture documentation. In this example its possible that a project may be composed of several teams working to deliver different things to a customer.

Going down on the left is the breakdown of the things we want to see in the document (Lets call them documentation areas). Each area is given a score of 0 to 5 based on some simple criteria which I will show in a while.

Another important thing to note is this is an estimation of completeness as a percentage and not necessarily work. Some of the things I give points for take longer than others. Reading through an assessment can help estimate the remaining work though.

An Overview Of The Process

The process i use is normally fairly simple. I normally pick a few people that will be assessors and give them an hour of training time with me. We run through a live document together and I explain what each document area means – and this description is normally supported with a template. So for each document we are assessing we normally have one template filled out – and we may go back several times over the course of a project to run the assessments again.

The assessment will happen between the assessor and the document owner. They run through each criteria and agree scores together and make notes to agree improvement areas. It can be done without the document author but i find that the communication between an author and and assessor in a meeting is more personal than just sending a filled in assessment. It also gets better commitment for improvement and enables the assessor to explain each criteria and answer questions.This can also be done as a peer to peer exercise.

I make sure that all people referred to in an assessment (approvers, technical validators) have actually reviewed the work and are aware of its state throughout the process.

Creating An Assessment Template

Its really easy to create a template that can be used for assessing the documentation. It can be done in any format – as a confluence page, as a word document, or as a google form, or Microsoft form for even easier collation.

Minimum Header Information

We always need to know as a minimum:

  • What is assessed – the document name normally – with a version number. I normally keep a copy of the actual document assessed with an assessment, rather than a URL to the current version of a document.
  • Who the assessor was – Just the name of the person who is running the assessment.
  • The document / design author – The person that is actually doing documentation work.
  • When the review was performed – just a date
  • A link back to any previous versions of the same assessment – If you are automating this process for a project in google forms for example the link back to a previous version might not be explicitly stated – you could calculate it if you know what was assessed and when. so if we did three separate reviews of the same thing it would have three different rows of data in your google forms.

The Criteria information

Normally I will expect to see this information each one of the different criteria

  • The criteria – The criteria we are assessing.
  • A short description of the criteria – normally a paragraph just to clarify things
  • The agreed score – using the scoring mechanism below
  • The scored mnemonics – I explain what each letter means in the scoring section.
  • The name of the person that approves the work – has to be a person with approval authority – not normally a project manager. Projects are transitory and end. The architecture normally gets delivered to someone in operations, and they should approve it.
  • The name of the person that technically validated things – normally a technical person who is not the author – its good to get a technically competent person from another team if possible.
  • Notes – any helpful information captured during a review.

In a very large project with multiple architects involved in an assessment I might also add the need for the Architect name, and the individual date You can see part of an example in figure 2:

Figure 2 – A partial example of a template

Scoring Documentation Areas

The Score

I apply a very simple scoring system; the order I show here is the order I assess on. I don’t normally assess something is fully complete until its partially complete. I don’t assess language before something is fully complete, and so on.

You don’t allow people to be their own technical validator or approver.

Figure 3 – The scoring

Short here is a mnemonic letter I use to show easily in assessments what I gave points for. So if something is at least partially addressed a point is given – an extra point is given once its fully addressed.

The Math

The math for calculating percentages in figure 1 is simple, but i will just mention it

  • There are 9 different things to assess and a maximum score of 5 points for each area – giving 45 points total as a possible score.
  • Row 14 is just a sum of the proceeding rows.
  • Row 15 is the percentage calculation – so we are calculating what percentage of 45 is in row 14. For column B that would be =(100*B14)/45

Summing It Up

This has been a quick introduction to how I approach doing quick assessments of documentation status when in large complex projects. Its not perfect but it is fairly flexible and quick and easy to implement. Its saved me many hours of having to do more in depth reviews before teams are ready to do so.

I hope this helps someone out there!