Just Enough Architecture

How much architecture is just enough? Is a common question I get which I will try to address in this blog.

Determining the right level of architecture design can be challenging; its very easy to get caught in a loop of over engineering for some people, and hard to know where expectations start and end for others. So what is the right level?

The High Level Design

The high level design template I created for work runs to 42 pages. Its an extremely long document that was built out of necessity. It has a lot of helping text and example views. It attempts to do a couple of things:

  • Provide a document structure that is ISO42010 compliant
  • Provide a document structure that provides Business, Data, Application, Technology Descriptions, and can be aligned to TOGAF
  • Provide extra spaces for common integrations and considerations we have within our own organization.

It aims to be usable by architects of very different levels of experience. When new architects look at it – it can be scary. Even though I wrote it, I really do not like it. I have discussed my thoughts on documenting architecture in document templates in the past, and I would much rather focus on the views we need to see to address the concerns of the stakeholders than trying to crank something into a rigid template.

Architecture View Points

I previously showed in this blog this diagram of the different types of viewpoints I like to consider as a minimum in a design, and my HLD template covers these, however there are times this can be considered overkill.

HLD Viewpoints
Viewpoint Structures

We determine standard viewpoints in architecture to make sure the concerns of our stakeholders are addressed and communicated. For architecture to make sense, it has to provide value. There are cases where it may not make sense to create all the views shown above.

An Example – Transforming On Premise Services to the Cloud

For example, lets say we have a team that’s been supporting on premise versions of Exchange & SharePoint, and we are creating an architecture as part of a proposition to extend the scope of services they provide to include Office 365 cloud based solutions.

If we are going to consume office 365 as a cloud service it may not make sense to create technology views, given that the technology layer is by Microsoft. It doesn’t provide value to anyone for me to rehash Microsoft’s standard material in ArchiMate.

In the same way, if we are going to be reusing an existing team, with existing processes, it doesn’t make sense to redefine all the processes – we simply do the definition for the things we need to specifically provide. If creating a specific view doesn’t make sense, there’s probably no need to create it. There are cases when its not obvious to your audience why a view doesn’t make sense – and you can document that.

If it doesn’t make sense to do a technology view, you may want to mention the reasons why in the service realization view for example. Sometimes if you just miss something, it will lead to an unnecessary volley of questions that could have been avoided if you had just written a sentence.

We are telling stories

We are trying to tell a story to our stakeholders, and explain an often complex set of elements and their connections. We are also trying to address stakeholder concerns and provide consistent easy to understand architectures. We should structure information in a way that is easy to consume. Sometimes that means changing the way a standard template is, or building custom views to clearly indicate our subject to our stakeholders. I spoke about telling stories in a previous blog on Improving ArchiMate Modelling Quality.

Documenting what we need to

Last week I was designing a solution based on Microsoft Sentinel in Azure. The challenge here is in the way that solutions are built – I could go into great detail about how data sources connect into the analytics engine, how rules are created and processed, and how they trigger playbooks. This is all detailed at some length in various places in Microsoft, and to define everything doesn’t make much sense.

But I did have to define a basic skeleton of azure architecture at an application level, because without it, it wouldn’t be possible to understand my architecture at all in a connected way without experience on Sentinel – which I cannot assume my audience has.

Not missing the essentials

There are some things you always need for an architecture design to be complete – and forgive me if you have heard this before:

An architecture should define and show how it meets the concerns (or requirements) of different stakeholders, and identify and manage related risks.

Without that an architecture is not complete, and will raise problems and there will be consequences in the future. The same thing is true if you do not design to meet the needs of all your stakeholders.

I speak often about things such as the architecture aspects, and architecture principles, and security non functional requirements – the point of these things is always the same – to ensure that we have captured the needs of our stakeholders.

Summing it up

Anyone who focuses on just creating a document is missing the point of architecture practice – its not providing its full value if its only a documentation exercise – its the process of creating a design that mitigates risks and meets requirements that is really providing the benefit – documenting is a natural part of that process, but at the end of the day, personally I do not care how big an architecture is, as long as it provides value to its identified stakeholders. I previously wrote a blog 8 reasons not to leave architecture to the end of a project – which could have also been titled 8 reasons not to think of architecture as a documentation exercise. Architects need to understand our standards, and be able to maintain a balance between what is too much and what is too little. Above all, architects must think.

Stakeholders in Architecture

Architects have many stakeholders in different areas. Even if we need to be customer focused there are still other stakeholder requirements to consider, This blog shows some of the stakeholders I typically consider.

In ArchiMate terms a stakeholder is a person or some group of people who are interested in the outcome of an architecture. They are the people we provide value to. Stakeholders are part of the motivational layer because we normally talk about stakeholders in the context of motivation. This doesn’t mean to say that an actor cannot be a stakeholder. Sometimes I may model a stakeholder in a motivational model and have an actor of the same name. .

When I say we need to consider our stakeholders, for me, this normally means some kind of interaction with them. Considering their needs as an architect is good, but you get more value if you actually ask the stakeholders what they would like to see. We can engage with stakeholders in many ways; the exact technique that’s best often depends on the number of stakeholders we have. You may consider an end user to be a stakeholder, but in an organization with 1000+ end users its not feasible to talk to each one individually – you may introduce surveys, or find team leads that are representative of such a wide audience. Business internal active structure elements can be assigned to the stakeholder element.

Lets look at some of the stakeholders I typically like to see considered in large projects.

Architects

Architects shouldn’t work in complete isolation – if an many architects are working on the same solution which sometimes happens in big bids they should be actively communicating with each other and a lead architect who ensures things are kept together with each other. Different teams may have their own architects; they are often stakeholders in each others architecture. For example, In a larger organization a product may require usage of other products. To provide a web solution we may need a database at the back-end, which could be supplied by another team. If I was creating the full web solution i may have requirements towards the architects in the SQL team and they may have limitations on their standard service. Deviations from standard need to be supported by someone and if I am expecting the SQL team to support something non standard i need to specify what and get an agreement. We provide value to SQL team because we use their service and they provide us value for the same reason.

The Customer

When I ask anyone defining an architecture, they are usually considering the defined needs of the customer. If responding to a request for proposal then its normally fairly easy because requirements are usually fairly clearly stated. Quite often I have seen customer requirements being the only requirements considered when responding to bids, which is bad, as other requirements can impact things.

The Customer Business Owners / Buyers

In product design I don’t think of the customer as a single role – there are several; there’s a need to consider the end users, but also the needs of the team that are buying your product – the business owners. Normally a business unit considering buying a product has to justify an expenditure to their organization. You can facilitate that for example buy having requirements around reporting. Not just delivering standard reports from applications but thinking about your value proposition and how to support it. A simple way of doing that for example, may be showing usage over time in a report, or building something custom so that a customer buying your service can show their management what a good investment is. If your customer wins with their manager, everyone wins.

Your Sales Team

To facilitate a need to sell things a design should consider the needs of your sales team. Sales becomes a much easier proposition if you have solid numbers. Look at how companies like Microsoft do it – when they make a statement like “Azure has shown a 200% growth in the last year” people pay attention. When you have those kinds of metrics and hard data points, products almost sell themselves. To make this happen however somehow that information needs to be captured and tracked. potentially part of an architecture design should cover how this happens so it can be completely automated. This can have a significant impact.

The Operations Team

In a modern organization considering the needs of the operations team in its design can save huge operations costs and radically reduce the number of incidents and escalations. Often, A solution needs to be automated as far as is possible – which includes having needs around monitoring, or even AI Ops. These days we can go a long way towards automating and managing incidents and problems. To have operational efficiency there should be standardized usage of tooling wherever possible, and standardized processes.

In some organizations operations teams try to manage this themselves but the needs of tooling and operations should impact into the overall design of any architecture you are planning to run in continuous services

The Security, Quality, & Risk Teams

Most organizations have requirements around ISO standards such as ISO20k or ISO27k for example. When there are several sets of requirements around risk and security often organizations have their own security non-functional requirements. A common mistake here is to think that because an organization has an ISO certification that any architecture that’s produced is compliant because the standard processes are.

If a company has promised compliance to an international standard steps need to be taken to ensure they are taken care of. In addition to that, companies have their own goals and risks, and to mitigate them they often have security related requirements.

Your Business Team

Product managers need to make profitable products that are attractive – they have goals, and to meet those goals they have targets normally around profitability, or sometimes around operational goals, or budget constraints.

Its easy to miss – everyone assumes there will be cost efficiency because everyone knows we need to make profit – but when you have it as a requirements, and you manage cost efficiency as a requirement – you show how cost optimization happens and it forces you to ask questions which may yield positive results.

Summing it up

The reason i felt a need to write this blog was that I hear a lot about being customer centric in different organizations and focusing on providing top quality services to customers and end users. This i agree with, but i would argue that unless you consider all the stakeholders when designing solutions, you are not achieving that objective. If you build architecture only considering customer requirements, it is much like building a house on quicksand. It could be the best house in the world, but without solid foundations it will crumble.

Architecture Solutions Without Problems

This blog discusses the impact of having solutions documented with no architecture behind it.

Deep thought, a super computer in Douglas Adam’s Hitchhikers Guide To the Galaxy was commissioned to find the ultimate answer to life the universe and everything, and after 7.5 million years managed to do so – the answer being 42. The mice were not impressed; deep thought pointed out that the real problem was that they cannot understand an answer when they didn’t really know what the question was. So they had to build a bigger computer (The Earth). There’s a little bit more about that at the video at the bottom.

In terms of time, that was a fairly costly mistake, and one that I see sometimes repeated in architecture design – it is directly relevant. Some people create Architecture Solutions, but its only really a solution if we can clearly identify what the problem is.

The Earth Mk 2.

Solution Needs

I’ve said it before – A solution needs to Identify stakeholders and ensure their goals and requirements are captured; along with their rationales. There is a need to show in design how the needs of all the stakeholders are met. If that’s not happening methodologically there’s a real risk something will be missed.

For a solution to have properly captured requirements, you need to know where the requirement came from, where it went to, what the requirement was and why it was. You then need to show how its been either realized in the design, or in some cases why it has not been included in a design. If you fail to answer any of those questions you will potentially force someone to have to spend hours tracking down the information later.

If I come into a project which only has a solution document it presents huge problems to myself as an EA – because I can have no confidence that the architects have full control of a situation. I cannot understand a full solution without potentially spending 100’s of meeting hours, in an attempt to build or understand a basic design. That’s effectively trying cover the work of dozens of architects and it really isn’t practical. That’s not really my job so more often or not it ends up in pushing back towards the project.

Some technical specialists with experience will naturally do a fairly good job at planning and implementing a design and just focusing on creating a solution document. They have seen the common challenges and know how to guide projects. Because these guys can seemingly pull solutions out of the air, its easy for people to mistake a technical specialist for a solutions architect. In organisations with poorly defined competency programs I have even seen the roles of tech specialist and architect confused.

The problem is that in this case you are leveraging the expertise of the individual in a way that doesn’t scale very well. It means you need very highly competent and expensive technical specialists. If you are doing this in a large organization the ability to quickly adapt and change as well as the ability to scale is compromised severely – because your architects lack the vocabulary and disciplines needed to talk with each other and properly manage requirements that come from different directions.

Its really painful to become the architecture lead of a project with 20+ architects only to discover that none of them are creating architectures, they are just writing solution descriptions based on discussions. Its hard to lead a team of architects when you don’t have the same vocabulary and in such circumstances is difficult to achieve tangible results. Its also hard to explain to management why architecture is not providing the benefits it should when you have so many architects on paper.

Do you have control?

At this point if you have architect in your job title and you work in a large organization, I would ask these questions:

  • How well do i understand all the developments going on in projects that impact your work?
  • How well do you know which standards your designs my comply to?
  • How compliant is your architecture to the needs of all its stakeholders?
  • Does your architecture design not only show what you need to implement, but how it should be implemented, and how it should be operated by who?

I have met architects whom despite being very competent, smart people, would feel a bit uncomfortable answering all those questions, even if they are routinely building architecture “solutions”. They are still excellent resources with a particular set of skills that provide value; and still have an important part to play in the role of architecture design, but they are not always architects.

Understanding the risk

If your solution doesn’t fully and clearly identify requirements in a way that’s easy to consume and is collected together in a single place, then I would challenge it. Risks are things that may impact your ability to meet your requirements. if you haven’t identified all your requirements clearly, then you have no way of properly understanding or managing risk.

You also have no way of ensuring your test and acceptance criteria are really fit for purpose, and may risk scope creep.

If you have architecture solutions without architecture design I would log at least one risk – that the architecture design may contain un-managed levels of risk and not be fit for purpose.

The Solution

Now I have scoped some of the problems I will point out the obvious solutions – We have to do proper architecture design including requirements and risk analysis preferably following some kind of standard for description such as ISO 42010.

We need to ensure that architects are trained to do architecture, and expected to deliver architecture artifacts. If working in a line organization, for managers who do not understand the architecture roles, I would suggest they should also report to a lead architect. Those resources should be allocated time to do architecture properly, and the lead architect should directly contribute into their goals and job assessments. Failing to do this can denigrate the effectiveness of architecture. I can personally attest to how frustrating that can be.

And for Hitchhikers Guide To The Galaxy Fans

Now I have finished talking about architecture, Here’s a clip from the TV series where the mice are given the answer to the ultimate question. You will hopefully see the comparison between this, and providing a solution without a design.

It’s not a complete architecture unless…

I often get given parts of architecture and am told the architecture is complete. This blog shows the bare minimums I use for assessing completeness.

Here I am laying down some minimums I ask for before I consider architecture to be complete, in terms of the types of things I need to see. In terms of actual subject matter, because the world changes is may be that architecture never stays static.

It can be very frustrating to repeatedly be told an architecture is ready when the basics are missing, so I thought I would lay them down here for all to see.

Although I am speaking from my own perspective, much of what I think, and how I practice architecture aligns to standards such as ISO 42010.

Its worth noting that I will sometimes be a bit relaxed with how I assess architecture depending on who is giving it to me. I don’t want to lay down law so much as provide direction for improvement and avoid having to spend to much time in a cycle of reviews. This blog is written with architects that work for/with me in mind.

It’s not a complete architecture unless it clearly identifies requirements

Requirements should be in some kind of list, saying where the requirement comes from, where it goes to, who is managing it, the priority and why it exists. Often extra information around status is also directly available on a requirements list. It is not enough to say the requirements are in the solution description we sent to the customer.  The solution description is normally a text document, and unless you pull the requirements out of it, its easy to miss some of them. Also the solution descriptions normally are very customer focused. There are a whole bunch of internal requirements solutions normally have to adhere to if we are designing something to be compliant with ISO 27k for example. Have requirements in a list, so we can address them one by one. Take a look at the TOGAF ADM Model – requirements management is at the heart of this model. Understanding needs of stakeholders is key to ISO 42010 as well.

It’s not a complete architecture unless it shows how requirements are met

Every single requirement has to be addressed by a solution or it is a risk or vulnerability; because by definition if you don’t address a requirement, somebodies needs are not being managed. Its OK if an architecture will not address a requirement – but that kind of a decision still needs to be tracked in your design; the polite thing is to agree it with the originator of the requirement or failing that to agree it with someone who has authority to make such decisions.

If you cannot show how requirements are met, the architects coming after you will have a hard time understanding why architecture decisions were made. This can, for example, lead to people innocently making changes at a technical level that breaks how things work from a business one.

ISO 42010 says that architecture concerns should be framed in one or more views.

Collaboration

It’s not a complete architecture unless its been agreed with the stakeholders

Your architecture should be identifying stakeholders – the people it provides value to. It stands to reason that means that how it provides value is of interest to those stakeholders.  If your architecture doesn’t meet the stakeholder needs, it hasn’t succeeded. This doesn’t mean all stakeholders will always be happy. Quite often architecture may be an agreed compromise. A customer is not the only stakeholder!

ISO 42010 has some great examples of different stakeholders your architecture may want to consider.

It’s not a complete architecture unless it addresses the appropriate aspects.

I speak often about covering IT Aspects. Having a brilliant diagram of how different system components fit together is almost never enough.

We have responsabilities to follow regulation, and those fancy technical designs are often going to be operated by someone who didn’t design the solution. This means people need to understand how the technologies work. How we operate things, sell them, and implement them all need to be considered as part of design and communicated.

When this isn’t done as part of an architecture design you end up quite often with something that isn’t scalable. Sometimes what will happen is the talented technical designer will hold the hands of the people that will support things and for a while things will run OK. Over time, people change in the different roles, and what was once laid out clearly becomes less clear as the IT equivalent of Chinese whispers happens.

Of course because you haven’t built and implemented a consistent design people may do things in different ways, leading to cost overhead, a loss in scale-ability, trace-ability and a diminished ability to do any kind of real optimization or effective cost reduction.

It’s not a complete architecture unless its been documented as one contiguous solution

If you do not have all the parts of a solution documented together in one place, or at least tightly connected to each other, its very easy to lose things, and to lose track of your solution.  That doesn’t mean everything is in one word document. If you are using word to describe your solution then its OK to refer to other documents containing information you rely on – such as the requirements. If we do this though,  we have to make sure that a) everyone reading a document actually has access to all the associated links, b) that information across the different information sources is properly versioned.

When you don’t properly maintain versions then you can end up in a situation where a solution refers to a requirements Excel, and someone else developing requirements may make changes that your solution doesn’t address, leading to a great deal of confusion for anyone reading your work. A good architect will be aware of the company documentation standards, and will be also designing solutions that cover whichever modelling standards are in place.

It’s also not OK to make decisions in meetings and not document them, or only documenting them in a set of meeting minutes. Its so very easy in those cases for a decision made in a meeting half a year ago to just disappear.

ISO 42010 makes mention of tracking key decisions in a log. I find that keeping things together relating to a particular work stream makes a lot of sense. It doesn’t have to be in depth – just knowing the date, the person or people working on something – and what they achieved – can save a lot of time when new people are on-boarded or when questions are raised during escalations.

Anyone that’s read my blogs or knows me, knows that I prefer to have my architecture be in a model, rather than documents.

Summing It Up…

If you have a good technical solution – that’s great. I do not argue that a technical diagram detailing how servers connect to different networks is not a valid part of architecture. It is. There’s also an argument that architecture always exists whether it is documented or not. Take a look at my blog on Designing Architecture Through Document Templates if you are interested in that..

The point i really want to make is that in order to have an architecture solution, you need to have clearly identified & scoped the problem, and show exactly how it is that you are keeping promises to your stakeholders, Not doing this will more often or not lead to extra cost later on down the road.

Applying Information Classification To Systems

I wanted to share a simple way I learned of practically applying information classification to IT systems.

I’ve spoken already about Information Security Thinking – this blog is more about ways to practically apply information security to networks and devices. This isn’t the scheme I use in my current job, but one I periodically find myself talking about.

Classifying information

Normally there are variations on a theme for information classification that may go something along the lines of this:

  • Public – Anyone can see, no real restrictions
  • Internal – Limited to people within the company
  • Confidential – Information has a controlled audience of stated members
  • Secret – is the information that could cause significant harm.

This can of course be customized, but as a rule of thumb most people agree we want to keep the information classification structure as simple as we can.

Establishing Controls & Control Goals

Each Information classification is expressed as a control goal – I made up a random example below; its not complete, I modeled this just as an example and to show how we can model this..

Figure 1 – Control Goals

The reason I show the influence relations between the classifications is because we effectively ranked information classification, and layer controls on top of each other. What I mean is if we had confidential information (Layer 3) it was expected that you would apply the confidential controls, but also the internal controls (Layer 2), and the public ones (Layer 1). Secret classified information would effectively necessitate the application of all the controls from the lower levels.

In this way its fairly easy to see how much control we need to apply depending on information classifications.

Applying Information Classification

Effectively once doing this you can designate an information classification to a network, or to a device on the network. Like this:

Figure 2 – Assigning Control Goals

In the example above I simply showed control goals associated with networks. In Figure 2, Effectively I have created Public, Internal and Secret security domains. In the real world I may map the different networks against the different security domains, as in larger organizations there are normally many networks. I simplified for demo purposes.

All devices and networks have an information classification. The rule is, that you are only allowed to have information up to the level of its information classification in any given system. For example a web page from public would be allowed to reside on either internal or secret systems, and its normal to put mechanisms in place to ensure that information coming into systems does carry an information classification, and also that the information classification is allowed on that system or network.

If a lower security classified server is on a higher security network it must adhere to the rules of the higher classification. For example, a web server might have been classified Public – meaning anyone can have access to the information. If we put a web server on a secret network, that’s OK – but it needs to adhere to the rules of the secret network – which in our example might mean that this public web server may not be allowed to have access to the internet.

What happens when the rules are violated?

If for example someone was to put secret information on an internal classified network, a couple of actions may be taken, for example.

  1. That information can be removed, and in depending of the seriousness of the infraction disciplinary action may be taken.
  2. Because a network has to provide a level of protection for the highest level of classified information on it, you could reclassify the network & the servers on it and apply the necessary security controls.

The actual rules should be considered carefully and included as part of a security policy.

Some Security principles

Following the discussion above these are some basic security principles:

  • All networks and devices are designated an information classification.
  • The information classification of a device may not be greater than that of the networks it is connected to.
  • All networks and devices must implement the security controls related to their corresponding information security classification.
  • All devices on a network must also implement the controls of the network they are on.

So whats good about this?

There are a few reasons I like this way of doing things.

  1. It makes the connect between the information we store and the controls we need to secure it.
  2. It can be easily represented in a few architecture views, and is a simple scheme that’s easy to understand, teach and communicate. Less is more.
  3. Because you are applying controls at a network and device level its very easy to see at a quick glance where security is, and gives simple clear rules that help us protect information
  4. It enables IT people to be focused on the protection of information rather than the protection of technology.

Summing it up

When trying to apply security to large organizations the security landscape can get really complicated fast. Its not uncommon to have comprehensive security rules that have grown in complexity and are not followed. This isn’t always because people do not want to follow security rules, its more often because there are already many demands on people & they do not always understand the value of applying security and don’t prioritize it

In designing any kind of security its important to get a balance between ease of use, and effectiveness, because if security policies and rules are too restrictive people will look for and find workarounds.