1.5 Adaptable architecture

One of the most complex problems in software development is balancing foresight with practicality. On the other hand, you need foresight to determine what features users might require in the future. You plan the architecture of the software with that in mind. However, you can’t develop software that will do everything. It will be immensely complex, and you will fail to build it. You need to balance your foresight with practicality.

Practicality dictates that you begin with a core problem domain. Understanding business goals, user needs, and pain points and distilling core requirements for the software from them is a solid starting point. Then, you evaluate these requirements against the time-tested 80/20 rule. This rule assures that twenty percent of the architecture will deliver eighty percent of the functionality, providing a strong foundation for your development. You always initiate development with this in mind and then iterate on the next twenty percent of the architecture.

Most modern applications can be effectively built using a basic web application stack, such as Ruby on Rails or Java Spring backend with a Single Page Application (SPA) in the front end. These technologies can cater to most of your needs for a long time. When you identify the need for analytics, you can quickly adapt by adding an API to feed Microsoft Power BI or another analytics platform. Remember, these technologies are common because they are adaptable to various problems, providing security in your choices.

Keep an eye on analysis paralysis, where the development team attempts to over-analyze without making any progress on delivering. A good tip is to timebox all design efforts. Say you design the next part of the system in one day. Remember that you can refine your previous plans later, and if you didn’t build a heavy vendor lock-in, you can even change technologies if deemed necessary. Modular design, APIs, and abstraction layers are your friends as an architect.

However, please don’t abstract everything so far away that it will be hard to maintain your software. Abstraction, APIs, and where to enable modularity are expensive design choices. The trick is to identify what parts of the system are most likely to change due to evolving business requirements, technology trends, regulatory requirements, or shifts in the vendor landscape. For example, user interfaces and data analytics functions often change, but the overall business logic behind the core system rarely does.

Not all APIs are created equal. An API should always be easy for other vendors and teams to use. You need standard technology such as REST/JSON, extensive API documentation, and an API design that exposes business services instead of internal technology choices. Ideally, you should test the usability of an API instead of just accepting it from any team or vendor. Only then can you guarantee that the API will offer you adaptability later when you need it.

Beware of claims about standardized APIs and the false security around the idea. The standard may not be detailed, supported by many vendors, or just a marketing term. Again, test the APIs and see if the API is critical for your system design. For example, “this product implements standardized WEBDAV API for clients” is just plain marketing. The base RFC concerning WEBDAV is so complex that no software has implemented more than half of the specification. While there’s a minimal set that works, it’s not that standard. Test with your clients whether the implementation works.

You may implement a microservice architecture, where each service is responsible for a specific business capability and can be developed, deployed, and scaled independently. When implemented correctly, it will create APIs that are nearly guaranteed to be effortless to consume. Microservices architecture is much more complex to implement and requires more work than it seems. Designing good APIs from scratch requires experienced and mature teams to manage products that can stand independently.

Here’s a simplified example of how API design for microservices can fail. You are building a case management system. It is agreed upon that all cases have several documents. The system has an API where users can access the documents via cases like /case/123/documents/456. Later in the development, the product owner states documents should not be tied into cases because not every incoming document leads to creating a permanent case in the system. The implemented API requires that documents are always under existing cases, and the required API change would be disruptive.

It’s crucial to measure all parts of your design. How well does each part align with long-term business goals? How expensive would it be to replace a particular technology? What part of the design will accrue technical debt the fastest? By focusing on making these parts of your design the most adaptable, you empower yourself to handle future changes effectively.

You are working on a system that assists sales managers in calculating how much pipe engineering projects cost. The top-level management says that that company will likely purchase another company doing scaffolding projects and move to that market. Those projects are estimated differently. Instead of hard-coding the estimation calculations, you make that implementation part extensible. Sure, it calculates pipe engineering now; end users may add as many project types as they wish with their calculating logic.

The mantra of scrum is that every sprint should produce functional software to be tested by the end users. OK, but where and when do you do the architecting? Design and documentation sprints are against the principles of scrum. The correct answer is that you can fit enough architecting into every sprint. You start with high-level architecture and implement what you need for the sprint. It may not be everything required, but you add to it incrementally over time. Software developers love working on that stuff in every sprint. In other words, architecture design evolves as a continuous process.

The only correct measurement overall is whether your project will be able to deliver thanks to the practical foresight of the architecture design. While it can not be reliably directly measured, a few good metrics are available. Some that, in combination, can be used are code complexity, code churn, lead time to changes or new features, and the business impact of the released versions ranked by the business leaders. Lead time to changes or new features is the time it takes from identifying a need for a change or a new feature to implementing it. This metric is important because it can indicate how adaptable your architecture is to changing business needs. Keep in mind your architecture is good only if it enables delivery.

Confronted with a significant challenge from the feature pipeline, you should be aware of the tools at your disposal. You can use spikes and timebox experiments to explore options and validate your ideas. Document only what’s necessary to enable pivoting, which would require rewriting all the documentation. Ensure you gather feedback for your designs from the development team and the significant decisions from the business leaders. Reiterating with good feedback makes the design much more effortless than designing in a vacuum.

Knowing how your peers and vendors solve similar business needs in their projects is a good idea. Experiences from different business fields also give you insight into how their systems had to evolve. While the business side looks typically different, the software serving business needs is more generic, and similar concepts are transferable. While researching, you should also study the pitfalls and issues others run into so you can avoid them.

Some technologies scale poorly up. When you have small amounts of transactions, everything seems fine. When the business grows, you might run into issues. Most scalability issues scale exponentially, meaning things work, and then they grind to a halt. You can attempt to salvage the situation by throwing in obscene server resources. However, that is not what you want. You should have known earlier, selected another technology, or contained the issue by architecture design. That’s why you always test the performance past your initial assumptions.

As an architect, you are not an oracle. However, your environment knows its pain points and the immediate needs. Working with others, you can also figure out the volatile parts of your architectural design that require extra adaptability. You should design architecture well enough, one piece at a time and iteratively. In agile development, there’s a lot of room for architectural design. You have the whole team to work on those issues. The architect’s role is not to design in a vacuum but to shepherd the process, ensuring that the bigger picture of business needs is always on top of everyone’s minds. The key is to keep the team focused on the business goals and to guide the design process to serve those goals best.