ISO Compliance – An Architect’s Perspective

ISO Compliance gets a bad wrap. People roll their eyes, are bored by it, and often don’t see the relevance of it. I wanted to share my perspective on this and address some misconceptions.

International Standards Organisation (ISO) create documents that provide requirements, specifications and guidelines that can be used to ensure materials, products, and processes are fit for purpose.

Although I focus a little on ISO standards here you can apply my thoughts to most standards.

ISO Compliance & internal company process

Its a no-brainer that to be compliant to international standards such as ISO 20000, 9001, and 27001 we need to consider them in our processes. We should have processes for checking compliance and often compliance needs results in changes to our operational processes. Checking compliance basically involves taking a standard, turning it into a set of requirements and then going different exercises of requirements realization, or maybe even scoring your processes against criteria in a similar way to what I described in scoring documentation. My previous blogs on those subjects are just describing one of many approaches you can take.

Its important to ensure company processes validate compliance when they first create their processes but also whenever they change them. You cannot just implement standards and forget about them, because as processes change, so can the level of compliance to standards.

In many cases an ISO compliance certification in one country does not equate to having that certification in all countries. Just because your company may hold an ISO 20000 certification, this does not necessarily mean that it holds that certification in the country you are working in.

Adhering to standards isn’t just the job of a security or quality team. In order to really gain the benefits from ISO standards many different roles and stakeholders need to be considered or involved.

Compliance & Architecture

Its important for architects to be aware of which ISO standards a company states it is compliant to. It has a direct impact on both design and implementation in some cases. Its normal to want to short cut the compliance process. There are a few ways to do this;. One way is to create one set of master requirements that aggregates the requirements from all the relevant standards into a set of non functional requirements (NFRs) Those architecture and security NFRs need to be considered when both designing and implementing solutions, and mechanisms should be put into place to make sure that happens.

As an example – ISO 20000 asks that as part of release management a testing environment exists. An architect should plan for this when building implementation and migration architecture.

It is could be very easy to miss this requirement if architects are not aware of the international standards or company NFRs. These requirements are actually architecture concerns from a security and compliance perspective. Architecture concerns have to be managed in an architecture design.

Just as ISO standards can effect implementation and migration architecture they can also effect core architecture design. GDPR is a hot topic – around security of personal information, but for example ISO 27001 provides standards around all information security management.

Its not just a tickbox excercise

Compliance to standards may seem boring and just a pointless paper exercise and when you view it that way it starts to lose its value. The ISO standards have been put together by groups of smart people, that have developed a set of practices to mitigate risk and avoid pains they may have personally been exposed to.

I have heard from some managers in the past that “You cant expect architects to read a whole ISO standard”. Even if you have good non functional requirements from your quality and security teams I would recommend architects pick up at least one ISO standard and read it all. My personal favorite is ISO 42010 (The International Standard for Architecture Description). Read through an ISO and think about the value it’s recommendations give and the pains you would get from not following each recommendation.

For example, ISO 42010 talks about ensuring architecture concerns are framed by at least one viewpoint. If they were not, then the needs of our stakeholders might be missed. Things might not be managed and potentially a customer may notice it. Maybe even a major incident may happen – there could be a huge cost or risk.

If you start to read through a standard and maybe entertain yourself trying to imagine what kind of disaster may have happened that lead someone to write those sentences you may find yourself with a new appreciation of them.

In thinking about what is written in the different standards rather than blindly checking requirements off a list we can get more value from them, and we can learn from the mistakes of others.

Its an opportunity for group fun!

Those running a security, quality or architecture team wanting to get people engaged in ISO compliance could run a workshop with key stakeholders, and get them a little hands on with the risk management side of things.

If you lay out a set of requirements that have been distilled from an ISO standard, it stands to reason that not meeting those requirements poses a risk. If you are using a simple mechanism to validate compliance you could easily establish standard risks against each and every requirement being missed, and start to estimate impact and cost. Getting stakeholders involved in defining risk gives them a deeper more intimate engagement with risk management, and can also help later when those compliance processes are in use.

Its hard to argue that a risk is not valid, if you are the person that defined the standard risk is the person that violated it.

A message to all architects

If you are designing architecture, you need to be aware of standards you promise to adhere to for your customers. and within your own organization. If you cannot name all the standards your company promises to be compliant to, you need to at least be fully conversant with architecture and security non functional requirements. As architects we have a responsibility to ensure we are managing architecture concerns

Don’t think of ISO compliance as being a boring paper exercise. Think of it as an opportunity to take steps to ensure you have an easier, more relaxing, higher quality life at work.

Scoring Documentation Completion

This blog demonstrates a quick and easy way to score documentation completeness and improving quality which is more accurate than guesstimating,

The Goal

To be able to with a reasonable degree of accuracy estimate how complete documentation is, and to not need to take more that 30 minutes per document to do that. I need a balance between speed and detail.

Its not always the job of an EA to actually do these assessments but as an EA its good to get an understanding of documentation that’s based on an assessment rather than a guess at how complete something is, so I often teach this approach.

Some of the benefits of this approach

In following a structured approach we get

  • Better understanding of expectations. When the document authors understand the things we want to see the chances of getting them increases
  • Easy to visualize status – because we are quantifiable we can easily report status in a number of ways visually.
  • Better trace ability of progress – because again, being quantifiable means we can show improvement over time.
  • More realistic assessment of completion. Normally when i ask someone how complete something is they say 70 or 80% but when you start to look its often not so – people want to naturally please other people and tend to often give higher estimates, or sometimes do not think of the detail involved. This mechanism forces people to look at the work in more detail than if you were to “guesstimate”
  • Raises other quality issues. In doing an assessment like below you are reading and aligning to a set of criteria, which are the things that are important to you. Normally when If run a quick assessment i will come up with half dozen or so comments along side the review.

The End Result

I am looking to get an overall percentage of completion – performing an assessment of something i normally end up with a table which will look a little like this:

Figure 1 – A resultant score of several architectures.

Bear in mind the table is only an example. If I was creating scoring for high level design I may have more detail. You can see across the top I have listed some different architecture documentation. In this example its possible that a project may be composed of several teams working to deliver different things to a customer.

Going down on the left is the breakdown of the things we want to see in the document (Lets call them documentation areas). Each area is given a score of 0 to 5 based on some simple criteria which I will show in a while.

Another important thing to note is this is an estimation of completeness as a percentage and not necessarily work. Some of the things I give points for take longer than others. Reading through an assessment can help estimate the remaining work though.

An Overview Of The Process

The process i use is normally fairly simple. I normally pick a few people that will be assessors and give them an hour of training time with me. We run through a live document together and I explain what each document area means – and this description is normally supported with a template. So for each document we are assessing we normally have one template filled out – and we may go back several times over the course of a project to run the assessments again.

The assessment will happen between the assessor and the document owner. They run through each criteria and agree scores together and make notes to agree improvement areas. It can be done without the document author but i find that the communication between an author and and assessor in a meeting is more personal than just sending a filled in assessment. It also gets better commitment for improvement and enables the assessor to explain each criteria and answer questions.This can also be done as a peer to peer exercise.

I make sure that all people referred to in an assessment (approvers, technical validators) have actually reviewed the work and are aware of its state throughout the process.

Creating An Assessment Template

Its really easy to create a template that can be used for assessing the documentation. It can be done in any format – as a confluence page, as a word document, or as a google form, or Microsoft form for even easier collation.

Minimum Header Information

We always need to know as a minimum:

  • What is assessed – the document name normally – with a version number. I normally keep a copy of the actual document assessed with an assessment, rather than a URL to the current version of a document.
  • Who the assessor was – Just the name of the person who is running the assessment.
  • The document / design author – The person that is actually doing documentation work.
  • When the review was performed – just a date
  • A link back to any previous versions of the same assessment – If you are automating this process for a project in google forms for example the link back to a previous version might not be explicitly stated – you could calculate it if you know what was assessed and when. so if we did three separate reviews of the same thing it would have three different rows of data in your google forms.

The Criteria information

Normally I will expect to see this information each one of the different criteria

  • The criteria – The criteria we are assessing.
  • A short description of the criteria – normally a paragraph just to clarify things
  • The agreed score – using the scoring mechanism below
  • The scored mnemonics – I explain what each letter means in the scoring section.
  • The name of the person that approves the work – has to be a person with approval authority – not normally a project manager. Projects are transitory and end. The architecture normally gets delivered to someone in operations, and they should approve it.
  • The name of the person that technically validated things – normally a technical person who is not the author – its good to get a technically competent person from another team if possible.
  • Notes – any helpful information captured during a review.

In a very large project with multiple architects involved in an assessment I might also add the need for the Architect name, and the individual date You can see part of an example in figure 2:

Figure 2 – A partial example of a template

Scoring Documentation Areas

The Score

I apply a very simple scoring system; the order I show here is the order I assess on. I don’t normally assess something is fully complete until its partially complete. I don’t assess language before something is fully complete, and so on.

You don’t allow people to be their own technical validator or approver.

Figure 3 – The scoring

Short here is a mnemonic letter I use to show easily in assessments what I gave points for. So if something is at least partially addressed a point is given – an extra point is given once its fully addressed.

The Math

The math for calculating percentages in figure 1 is simple, but i will just mention it

  • There are 9 different things to assess and a maximum score of 5 points for each area – giving 45 points total as a possible score.
  • Row 14 is just a sum of the proceeding rows.
  • Row 15 is the percentage calculation – so we are calculating what percentage of 45 is in row 14. For column B that would be =(100*B14)/45

Summing It Up

This has been a quick introduction to how I approach doing quick assessments of documentation status when in large complex projects. Its not perfect but it is fairly flexible and quick and easy to implement. Its saved me many hours of having to do more in depth reviews before teams are ready to do so.

I hope this helps someone out there!