Architecture building blocks and ArchiMate

ArchiMate isn’t only about drawing views and relational models. Understanding the language fully helps how we shape our architecture.

In a meeting last week there was a fairly lively debate about architecture design. In that meeting we had some people wanting to comment the semantics of the ArchiMate language (the more analytical people). Some people were just wanting to understand the relationships between the applications. Quite a lively conversation ensued, and want to share my thoughts on this.

Architecture Tells A Story

I have spoken about Storytelling in previous blogs. In ArchiMate terms, a view of architecture is created for a purpose for a specific audience – its defined in the viewpoint definition. So its important we address the intentions in the viewpoint definitions.

Following ArchiMate Standards Help Structure Architecture

Using the right elements for the right things is important when it comes to wanting to generate specific views. For example If someone mistakenly stated “ISO42010” as a strategic capability rather than “ISO42010 Compliance” as a requirement, then if I want to automatically generate a view of our strategic capabilities, I am going to have things on there that don’t make sense. If many elements have incorrect definitions or relationships between them then the usage of the model as a whole is greatly diminished.

Understanding the elements and the metamodel fully is key

Understanding and using the full metamodel properly gives so much more than just defining elements and relationships. Services are a fantastic example. Rather than building one massive view of interconnected application or technology components, breaking things down into smaller services gives us opportunity for reuse. This can be done at the business, application and technology level. Building a service is like building a Lego block. You can think of the block as being a service – and the interfaces to that service being the connectors like this:

Figure 1 – Lego Web Service

We can build our services in several layers of course; either stacking services, Business on Application, which sits on Technology – Alternatively our Lego block here could also contain elements from other layers. It could be argued with the example above that the interfaces for directory services or databases shouldn’t be there – because they are part of the internal make up of the building block, but this is one of the key things we need to decide when building architecture – what are the logical blocks that make the most sense for reuse, what is internal, what is external. We are building architecture building blocks. Within our services of course there could also be sub services within these building blocks – so one building block can be made of others. I previously wrote a little on Services In ArchiMate that goes into a little more detail on the subject of Services.

I have already spoken extensively on the benefits of having an architecture model rather than word documents, and an ArchiMate relational model making it possible understand the implication of change, but in understanding how the elements and relationships in the metamodel come together to create a service base thinking we truly start to see some real power.

Putting it all together

At the end of the day, we want to be able to do this:

Figure 2: Building blocks

We start to build standard architecture building blocks, with standard interfaces. connecting into standard services we unlock a myriad of advantages – We can can more efficiently reuse our resources, and we can put our services together in interesting new ways. With Lego the focus isn’t on the blocks – its what you can build with them – the same holds true in architecture.

Summing it up

How we model something and follow the language of ArchiMate is important. We don’t just want to understand the elements, or only have a relational model, we also want to ensure what we are doing is building architecture building blocks that make sense, and make it easy for us to have reuse.

Practicing architecture in this way may mean a learning curve for architects but its key for making efficient use of a companies resources, and maximizing the benefits to business.

Getting The Most Out Of An EA Tool

At work we use Bizzdesign’s Enterprise Studio (BES). I wanted to talk a little about getting the most out of an EA tool like this, if you want to get to a more advanced level of architecture model maturity.

Yes its true, I like Enterprise Studio, but not because of any affiliation with the company; more because its a powerful tool, and after several years of working with it, I am always finding more. Other tools have some similar capabilities, and strengths and weaknesses. I am talking about a tool I currently use, and a little about some of the things we need in order to reach a more advanced level of architecture maturity.

Basic Admin

Its worth mentioning that with different tools there are different access methods. With BES, at a basic level you can maintain your access to the repository by managing users and groups in HoriZZon (their web access portal). Its also worth mentioning, if you have your own servers you can also build an environment that integrates into your organizations active directory and control things via groups, eliminating the need for much of the manual grunt work around administration, and their licensing mechanisms are fairly integrated. This becomes important as tool usage grows.

Basic Modelling

It was lunchtime. I was getting hungry.

As a tool, one of the reasons we decided to use BES in our organization was that it was easy to jump into, whilst also having a fair level of complexity underneath. Whilst i tend to use it most for modelling in ArchiMate, it also covers some other standards I use, like BPMN. At a real basic level, we have a managed repository, with the ability to check out and check in changes, and maintain a level of version control. This I would expect of any tool that has a number of EAs that need to collaborate. Basic functionality needs to be able to manage elements and the relationships between them, offering functionality to navigate the object model and create new views from that – take a look at Archimate – Looking Beyond Diagrams. The better tools out there enable you to use smart connection mechanisms where you can draw relationships, and then pick the correct relation from a list of possible relations between the two elements you selected. This is especially important when you have junior architects – it saves time if they don’t have to chose from every relationship in the language.

Considering Stakeholders

A BES architecture repository can be exposed out to stakeholders of architecture via a web browser using Horizzon. It’s really important to try and get stakeholders engaged, and to do that sometimes we hide ArchiMate, by changing graphics and simplifying things.

How simple model or represent architecture depends on who our stakeholders are for a particular view. Getting stakeholders connected to what we model is an important first step in getting them to start to see the value of the work we do. Being able to traverse the model gives a level of power by itself. Horizzon gives a lot of power and directly connects to the repository for up to date information, Some tools allow HTML outputs, which is also handy.

Appending Basic Information to a Model

We can easily import information into a model a whole bunch of different ways – starting with the basic updating and creation of elements and relations. You can within BES you can use new multiple and easily just copy and paste things from excel for quick addition or updating of elements and their metadata

Its possible to apply a level of automation to that. We can create connections to Excel workbooks, SQL, ServiceNow, and automatically refresh parts of the model package or update them, and we can do mapping from different data sources into the model These are things that take a bit of knowledge and getting used to – but on the most part, you don’t need to be a developer. Basic information can be pulled into the model quite easily if you think a little about how you are formatting it as part of the project that you are working on.

Working With Metrics & Avoiding The Metamodel

I have a love/hate relationship with the meta-model editor, and as a policy where I work we don’t edit it. Some tools work with meta-models easier than others, and working with the meta-model editor that BES has provided lead to some fairly painful mistakes. Its a very flexible and powerful thing, which with a bit of experience can be used, but I don’t often find I need to customize it. In older versions of the tool every time the tool needed upgrading, the meta-model would need redoing – but a lot of new features have come along that make it significantly easier now. As a rule though I tell people not to touch it because when you don’t know what you are doing you can break things in spectacular ways.

When I want to append information to an element these days, mostly I add metrics to the element that needs to contain information. From there its easy to create colour displays or play with labeling to your hearts desire. There are some restrictions in this technique.

Scripting

BES has a really powerful scripting language that sits behind it; you can use it to do all manner of things including creating views, tables of information, modifying the model or colouring the existing model. You can create custom scripts to keep metrics up to date.

The more information you get into the repository the more value it brings. Its good to be able to traverse the portfolio in a number of different ways and represent different things on the same model..

Workflows

Something really cool is workflows – I haven’t used these as much as I would like. Being able to create a workflow in BPMN and then apply it to a model package so we can have approval workflows for example is very cool. We can also use this for getting stakeholders to update specific properties, so we can update our information from the browser without even having to open a modelling tool

API Integration

Here the possibilities start to get a bit insane. I was thinking the other day about building a C# app to pull architecture concerns out of Jira and automatically create/update views in BES. But you could do so many things. What about building your org charts from scanning your active directory? Or pulling a project roadmap directly from a tool like SharePoint? What if we can model our financials? Currently the BES API supports data enrichment; between the API, scripting and other methods I mention today, an imaginative developer can achieve can achieve a lot of cool things.

Creating these integrations have benefits in a number of directions – when you start to expose different information sources, you can not only build models that are correct and business relevant. Doing this sort of work also teaches us a lot about the data consistency and hygiene of our various information sources

Summing It Up

When you start to get to the point where all your different organization sources are being pulled into a repository we can start to connect and traverse them in exciting and interesting ways.

If you are working in an organization that does everything in Excel, PowerPoint and Word, and doesn’t leverage the benefits of technologies like Power BI, I would stop and look at your Enterprise Architecture. Chances are you are spending a lot of time in meetings, and are spending a lot of time looking for the right people and information, rather than focusing on business. Automation doesn’t make people redundant – it enables them to stop doing trivial work so they can focus on things that are important.

It takes time, money, and some developers to truly leverage the best value out of a Tool like BES. The problem I have always seen is a struggle to put together a business case for it. We can save a huge amount of man hours and communication overhead, but calculating how much is hard before you have taken the time to make the connections and make your organization more traceable through its EA.

Tools like BES are pretty expensive to own. We can also spend a significant amount of time and money customizing them. I would suggest that the value of such a tool is unlocked by how you use it – Its worth spending some time in training. The tool is nothing without a good EA behind it, with a bit of imagination, and a little bit of time and money from committed stakeholders. Do that, and you won’t be thinking about your tool spend, because you will be using your tool to unlock your existing information sources and reusing them in creating new views in exciting new ways, rather than just manually modelling things from unmanaged stale data sources like PowerPoint.

Requirements in Architecture

This blog talks about requirements in the context of architecture design – including their importance and some bare minimum needs.

If you have read ISO42010, the ISO standard for architecture description, you will realize how important requirements are. They are how we define the needs of our different stakeholders.

Similarly, if you are using TOGAF, you will see requirements management right in the middle of TOGAF ADM. Generally speaking i tend to think we can manage requirements at different levels, depending on the context of what we need to do, and who we are doing it with.

High Level Requirements with Management

When discussing requirements with senior management I am thinking at a really high level; where we dont want a million requirements with lots of information captured with them. I tend to think in terms of user stories. Those meetings are not normally disciplined meetings where we strictly define “As someone i want to do something to achieve some value” – Although at the end of the meeting i want to have enough information to be able to write those statements – I will typically ask the questions like:

  • Who needs to get value from this?
  • What is it that we need?
  • What value does that bring?

…In taking that approach in a more discursive meeting you have to be very careful – i find its really easy to get into long meetings where you discuss whats needed, but loose track of what the value is that’s brought.

Defining the next level of detail – Capturing requirements

The user stories are great for talking with management, but they are not really enough to be actionable by themselves. We need to refine our requirements into something that is actionable, and clear for anyone working with a design. Its not unusual in a big project to have many high level meetings, and have many stakeholders that need to get value – at this point we need something we can practically work with in design. I need to capture requirements in one place- be it Confluence, Excel, or in a formal modelling or database tool. If its not in one place, its very easy to lose requirements.

There are many things you can ask for but again speaking of minimums to look at as part of an initial architecture definition phase:

  • What is the source of the requirement? Who did it come from? A specific team, person, project or role?
  • What is the destination of the requirement? Is this a requirement coming towards us, or is it a requirement we will need another team to manage? For example if we have a website, the database back end may be run by a completely different team, which we may have the requirement for.
  • What is the actual requirement? The definition of the requirement – it should be as clearly stated as possible and not ambiguous – someone is going to have to validate the requirement has been met. Saying “we need a fast database” is subjective – saying “we need 5000IOPS from the database storage” is a lot better defined – and significantly reduces the risk of miscommunication
  • What is the rationale of the requirement? The reason behind the requirement – It can be simply stated such as “ISO27k Compliance”, “Customer requirement”, “Needed to by the quality team”. Its very often skipped, but is very important because requirements are read by many people from many teams, not all with the same understanding of things. Solutions & Requirements can live for years and its easy to forget why you do something. If you come back to an architecture design with a need to change it later you will be thankful for this – it saves you having to spend countless hours understanding why we do things.
  • What is the priority of the requirement? Commonly I use MoSCoW for this (Must Have, Should Have, Could have, Won’t have). In any given release some things are more important than others.
  • How do I identify requirements? Its worth mentioning that requirement IDs are a good idea to identify right from the start because the exact wording of a requirement is sometimes refined and its easier to speak of specific IDs in general.

Not describing any of the things above will only leave issues later.

Managing requirements once they are defined

During an architecture design project we need to work with each of the requirements, to ensure that they are met – and to show how. Requirements Realization is a whole topic by itself. I spoke about this in my blog Assessing solution requirements – There’s a whole lot more i would like to say on it should I get the time; some of the things i say in that blog are covered in here, but the focus is different. – but the basic things we need to capture are:

  • How the requirement is realized – is it by a specific tool, a specific business process, a specific automation? This is one of those things that are often discussed in meetings, but then sometimes not captured in design – if you ever came back to a project after a year and tried to figure out why someone had made a technology because you want to change something, you may feel pain if this isn’t done. It can lead to huge investigations and many meetings with people which in some cases i have seen thousands of hours being burned. In the absolute worse case you may actually lose this information entirely because people have lost of forgotten.
  • The status of the requirement – different requirements can be in different states – the basic statuses of requirements i would say are – Proposed, In Progress, Implemented, Verified, Deleted, Differed, Rejected.
  • Who’s working with the requirement – on a big project with more than one architect its good to name which architect is being responsible for managing the requirement and ensuring its properly realized (if need be).
  • Activity Log against the requirement – sometimes the whole architecture design process will have a work log and in there you can show what is needed – but sometimes is better to have a log of activities against each requirement – nothing big, just date, whats happened, and who has done it. this way you can keep track of things in a complex project.

Summing it up….

As i said, these are really minimums – I have seen many variations on the above, but have found if you haven’t considered the things I have written about above there is almost certainly going to be an overhead. at some point of time someone will have to run meetings, or investigations to find out what or why something is a certain way, or someone may do something stupid simply because they are not aware of the relevance of a specific piece of architecture. Without properly defining requirements we simply cannot ensure a technical design is fit for purpose.

There are lots of things to consider. but properly managing requirements will radically improve both quality in architecture, and success in project management.

Just Enough Architecture

How much architecture is just enough? Is a common question I get which I will try to address in this blog.

Determining the right level of architecture design can be challenging; its very easy to get caught in a loop of over engineering for some people, and hard to know where expectations start and end for others. So what is the right level?

The High Level Design

The high level design template I created for work runs to 42 pages. Its an extremely long document that was built out of necessity. It has a lot of helping text and example views. It attempts to do a couple of things:

  • Provide a document structure that is ISO42010 compliant
  • Provide a document structure that provides Business, Data, Application, Technology Descriptions, and can be aligned to TOGAF
  • Provide extra spaces for common integrations and considerations we have within our own organization.

It aims to be usable by architects of very different levels of experience. When new architects look at it – it can be scary. Even though I wrote it, I really do not like it. I have discussed my thoughts on documenting architecture in document templates in the past, and I would much rather focus on the views we need to see to address the concerns of the stakeholders than trying to crank something into a rigid template.

Architecture View Points

I previously showed in this blog this diagram of the different types of viewpoints I like to consider as a minimum in a design, and my HLD template covers these, however there are times this can be considered overkill.

HLD Viewpoints
Viewpoint Structures

We determine standard viewpoints in architecture to make sure the concerns of our stakeholders are addressed and communicated. For architecture to make sense, it has to provide value. There are cases where it may not make sense to create all the views shown above.

An Example – Transforming On Premise Services to the Cloud

For example, lets say we have a team that’s been supporting on premise versions of Exchange & SharePoint, and we are creating an architecture as part of a proposition to extend the scope of services they provide to include Office 365 cloud based solutions.

If we are going to consume office 365 as a cloud service it may not make sense to create technology views, given that the technology layer is by Microsoft. It doesn’t provide value to anyone for me to rehash Microsoft’s standard material in ArchiMate.

In the same way, if we are going to be reusing an existing team, with existing processes, it doesn’t make sense to redefine all the processes – we simply do the definition for the things we need to specifically provide. If creating a specific view doesn’t make sense, there’s probably no need to create it. There are cases when its not obvious to your audience why a view doesn’t make sense – and you can document that.

If it doesn’t make sense to do a technology view, you may want to mention the reasons why in the service realization view for example. Sometimes if you just miss something, it will lead to an unnecessary volley of questions that could have been avoided if you had just written a sentence.

We are telling stories

We are trying to tell a story to our stakeholders, and explain an often complex set of elements and their connections. We are also trying to address stakeholder concerns and provide consistent easy to understand architectures. We should structure information in a way that is easy to consume. Sometimes that means changing the way a standard template is, or building custom views to clearly indicate our subject to our stakeholders. I spoke about telling stories in a previous blog on Improving ArchiMate Modelling Quality.

Documenting what we need to

Last week I was designing a solution based on Microsoft Sentinel in Azure. The challenge here is in the way that solutions are built – I could go into great detail about how data sources connect into the analytics engine, how rules are created and processed, and how they trigger playbooks. This is all detailed at some length in various places in Microsoft, and to define everything doesn’t make much sense.

But I did have to define a basic skeleton of azure architecture at an application level, because without it, it wouldn’t be possible to understand my architecture at all in a connected way without experience on Sentinel – which I cannot assume my audience has.

Not missing the essentials

There are some things you always need for an architecture design to be complete – and forgive me if you have heard this before:

An architecture should define and show how it meets the concerns (or requirements) of different stakeholders, and identify and manage related risks.

Without that an architecture is not complete, and will raise problems and there will be consequences in the future. The same thing is true if you do not design to meet the needs of all your stakeholders.

I speak often about things such as the architecture aspects, and architecture principles, and security non functional requirements – the point of these things is always the same – to ensure that we have captured the needs of our stakeholders.

Summing it up

Anyone who focuses on just creating a document is missing the point of architecture practice – its not providing its full value if its only a documentation exercise – its the process of creating a design that mitigates risks and meets requirements that is really providing the benefit – documenting is a natural part of that process, but at the end of the day, personally I do not care how big an architecture is, as long as it provides value to its identified stakeholders. I previously wrote a blog 8 reasons not to leave architecture to the end of a project – which could have also been titled 8 reasons not to think of architecture as a documentation exercise. Architects need to understand our standards, and be able to maintain a balance between what is too much and what is too little. Above all, architects must think.

Stakeholders in Architecture

Architects have many stakeholders in different areas. Even if we need to be customer focused there are still other stakeholder requirements to consider, This blog shows some of the stakeholders I typically consider.

In ArchiMate terms a stakeholder is a person or some group of people who are interested in the outcome of an architecture. They are the people we provide value to. Stakeholders are part of the motivational layer because we normally talk about stakeholders in the context of motivation. This doesn’t mean to say that an actor cannot be a stakeholder. Sometimes I may model a stakeholder in a motivational model and have an actor of the same name. .

When I say we need to consider our stakeholders, for me, this normally means some kind of interaction with them. Considering their needs as an architect is good, but you get more value if you actually ask the stakeholders what they would like to see. We can engage with stakeholders in many ways; the exact technique that’s best often depends on the number of stakeholders we have. You may consider an end user to be a stakeholder, but in an organization with 1000+ end users its not feasible to talk to each one individually – you may introduce surveys, or find team leads that are representative of such a wide audience. Business internal active structure elements can be assigned to the stakeholder element.

Lets look at some of the stakeholders I typically like to see considered in large projects.

Architects

Architects shouldn’t work in complete isolation – if an many architects are working on the same solution which sometimes happens in big bids they should be actively communicating with each other and a lead architect who ensures things are kept together with each other. Different teams may have their own architects; they are often stakeholders in each others architecture. For example, In a larger organization a product may require usage of other products. To provide a web solution we may need a database at the back-end, which could be supplied by another team. If I was creating the full web solution i may have requirements towards the architects in the SQL team and they may have limitations on their standard service. Deviations from standard need to be supported by someone and if I am expecting the SQL team to support something non standard i need to specify what and get an agreement. We provide value to SQL team because we use their service and they provide us value for the same reason.

The Customer

When I ask anyone defining an architecture, they are usually considering the defined needs of the customer. If responding to a request for proposal then its normally fairly easy because requirements are usually fairly clearly stated. Quite often I have seen customer requirements being the only requirements considered when responding to bids, which is bad, as other requirements can impact things.

The Customer Business Owners / Buyers

In product design I don’t think of the customer as a single role – there are several; there’s a need to consider the end users, but also the needs of the team that are buying your product – the business owners. Normally a business unit considering buying a product has to justify an expenditure to their organization. You can facilitate that for example buy having requirements around reporting. Not just delivering standard reports from applications but thinking about your value proposition and how to support it. A simple way of doing that for example, may be showing usage over time in a report, or building something custom so that a customer buying your service can show their management what a good investment is. If your customer wins with their manager, everyone wins.

Your Sales Team

To facilitate a need to sell things a design should consider the needs of your sales team. Sales becomes a much easier proposition if you have solid numbers. Look at how companies like Microsoft do it – when they make a statement like “Azure has shown a 200% growth in the last year” people pay attention. When you have those kinds of metrics and hard data points, products almost sell themselves. To make this happen however somehow that information needs to be captured and tracked. potentially part of an architecture design should cover how this happens so it can be completely automated. This can have a significant impact.

The Operations Team

In a modern organization considering the needs of the operations team in its design can save huge operations costs and radically reduce the number of incidents and escalations. Often, A solution needs to be automated as far as is possible – which includes having needs around monitoring, or even AI Ops. These days we can go a long way towards automating and managing incidents and problems. To have operational efficiency there should be standardized usage of tooling wherever possible, and standardized processes.

In some organizations operations teams try to manage this themselves but the needs of tooling and operations should impact into the overall design of any architecture you are planning to run in continuous services

The Security, Quality, & Risk Teams

Most organizations have requirements around ISO standards such as ISO20k or ISO27k for example. When there are several sets of requirements around risk and security often organizations have their own security non-functional requirements. A common mistake here is to think that because an organization has an ISO certification that any architecture that’s produced is compliant because the standard processes are.

If a company has promised compliance to an international standard steps need to be taken to ensure they are taken care of. In addition to that, companies have their own goals and risks, and to mitigate them they often have security related requirements.

Your Business Team

Product managers need to make profitable products that are attractive – they have goals, and to meet those goals they have targets normally around profitability, or sometimes around operational goals, or budget constraints.

Its easy to miss – everyone assumes there will be cost efficiency because everyone knows we need to make profit – but when you have it as a requirements, and you manage cost efficiency as a requirement – you show how cost optimization happens and it forces you to ask questions which may yield positive results.

Summing it up

The reason i felt a need to write this blog was that I hear a lot about being customer centric in different organizations and focusing on providing top quality services to customers and end users. This i agree with, but i would argue that unless you consider all the stakeholders when designing solutions, you are not achieving that objective. If you build architecture only considering customer requirements, it is much like building a house on quicksand. It could be the best house in the world, but without solid foundations it will crumble.