Systems Simplicity vs. Complexity
Beware of the microservices, if you don’t need them
At the very beginning of a new project or when maintaining an existing software there is a point where you have to decide which kind of system architecture you’d like to choose. Whether it’s suitable for the projects use case to build a monolithic system, a self contained system or a microservice.
Influenced from the steadily growing serverless hype many decisions are made towards the microservice approach. One reason for this is that serverless is omnipresent at conferences and in articles, the other is that the microservices approach is promising to get a production ready MVP in a very short period of time, additionally solving many problems that can occur when maintaining a monolithic system.
There are many good arguments for microservices, however there are some pitfalls that you should be aware of when choosing this direction. This article will give you an idea of what to keep in mind when choosing microservices from a broader perspective.
Inner complexity vs. outer complexity
One of the biggest advantages with microservices is the reduction of a system’s complexity. One service is responsible for one particular functionality and the business logic complexity within that service is limited to that functionality, which makes contribution to the service’s codebase much easier because onboarding new members is therefore easier. Nevertheless, the system’s complexity is only shifted to the outer boundaries. Every contributor has to know about several outer limitations like runtime, timeout restrictions, communication to other systems, failure handling, etc. Depending on the microservice, the outer boundaries are crucial to understand when working within a microservice ecosystem. Otherwise you won’t get the approache’s benefits like resiliency.
„When you use microservices you have to work on automated deployment, monitoring, dealing with failure, eventual consistency, and other factors that a distributed system introduces.“ — Martin Fowler
To summarize it: Nothing comes without a price and you’ll always be in the situation to decide which trade off is the most suitable for your project.
Is your organization capable of handling microservices?
Before starting to think about possible architectural styles, one of the most important things to know is whether your organization is generally capable of handling the chosen architecture. You can answer this question with the help of several indicators.
1. Remember Conway’s Law
Conway’s law always strikes back when you ignore it. 1968 Melvin E. Conway found out and published that systems designed by an organization will always reflect their path of communication.
“Organizations which design systems […] are constrained to produce designs which are copies of the communication structures of these organizations.” — Melvin E. Conway
Hence if you have several teams crossing each other’s boundaries and sharing some responsibilities, you won’t be able to develop decoupled systems. You might not detect negative effects at the beginning, but as the systems grow you will discover a tremendous overhead of communication, if you don’t manage to define clear boundaries between the teams. In the end it prevents you from getting a faster time to market and autonomous releases. At its worst this can lead to a blaming culture between your teams, where the team members won’t take over responsibility for the system they develop.
2. Review your current organizational structure
Choosing microservices is suitable for nearly every use case, presupposed you are aware of both the complexity and Conway’s law. However the microservice architectural patterns are predestinated for developing software with multiple teams. Microservices enable you to have autonomous release cycles and a minimum of communication overhead to other teams. If you are the only team, you won’t benefit from any of these advantages because you literally don’t have to communicate with other teams a lot or at least you are the only ones deploying software into production. One could argue microservices enable you to have independent scaling capabilities for specific parts of your system. However you should ask yourself if the system you develop really needs to scale indefinitely. If that’s not the case, you probably won’t get the benefits of microservices, on the contrary, you might end up developing a distributed monolith because of the lack of distributed teams and therefore (clear) boundaries.
3. Know your team’s capabilities
As already mentioned, the outer complexity of a project is increasing when choosing microservices. This has an impact on the required knowhow inside the teams to take over the full responsibility of the product. There is not only the need for business knowhow but also, at least partially, knowledge about runtime specifics (in case of a serverless runtime: timeout restrictions, ephemeral storage, etc.), networking and service to service communication (eg. queues, busses, topic). On top, every microservice needs to be assigned to a team which then owns the service and can be held responsible. Having umpteen microservices means having umpteen boundaries. If they are not logically assigned to the right teams, you might discover hidden communication complexity across teams, unless you have enough engineering teams so each of them can be responsible for a small amount of services only.
4. About splitting the monolith
A common scenario for introducing microservices is the gradual split of a monolith. At that point I want you to be aware that moving directly from monolith to microservices is risky (ref. MonolithFirst — Martin Fowler (2015)). However, there are many examples where companies tried starting over by outsourcing parts of a domain of an existing monolith by implementing microservices directly. Unfortunately this tends to run into a distributed monolith when cutting more domains from the existing monolith. Apart from the architectural perspective, you should always track whether your current organizational structure is capable of handling more microservices. Especially when you are maintaining multiple microservices with different frameworks and deployment pipelines, you will sooner or later reach the point where you exceed your team’s cognitive capabilities and they’ll no longer be able to maintain their services in an adequate way.
A more safe approach could be to move from monolith to modulith and then if necessary over to microservices at the cost of time. If you are starting from scratch, implementing a deployment monolith first could prevent you from running into a distributed monolith. With a clear decision towards a monolith, at least you can start implementing and still have the flexibility to move over to microservices or just stick with a mix of both moduliths and microservices later.
Recap
Microservices are still en vogue, at least reinforced through serverless capabilities in public clouds. At the end you have to keep in mind that neither microservices nor serverless are a silver bullet. Nothing comes without a price and the disadvantages can be harmful to your organization. Building software is not restricted to software departments only, it’s the organization’s culture having an influence on the architectural capabilities and therefore on the systems itself. It is crucial to be aware of your decision’s side effects when changing your system’s architecture and/or landscape. Developing a distributed system demands proper organizational evolution.
You can reach me out via:
- Twitter: @pascal_euhus
