Software engineering has been facing a crisis for over 50 years! first identified at the 1968 NATO Conference. Edsger Dijkstra described this crisis in his Turing Award speech! noting that the exponential growth of computing power has made programming increasingly complex and unwieldy.
Over time! we’ve created more code! more abstractions! and a growing number of systems that! far from becoming simpler! have become increasingly difficult to maintain and modernize. The software industry has prioritized creating new solutions without adequately addressing legacy software management! leading to a backlog of old systems that require significant effort to understand! modify! or replace.
The legacy software problem affects not only technology companies but all industries that rely on technology. This book proposes a solution: rethinking software engineering as an efficient and systematic decision-making process! rather than one based on manual code reading.
The concept of documented development allows us to transform the way we explore systems! streamlining decision-making and enabling a clearer and more precise understanding of software architecture and operation.
Why talk about decision-making?
As developers! we spend more than 50% of our time understanding the systems we work on before we can make any changes or improvements. We analyze the impact of modifications! investigate bugs! and design new features. However! we rarely reflect on the decision-making process itself.
In most cases! we focus on evaluating country email list whether a decision was correct! but not on analyzing whether the way we made that decision was the best. To optimize our work! it’s essential to improve the decision-making process itself.
The decision-making process in software engineering
In general terms! the decision-making process in software follows these steps:
- We assess the problem: Before taking any action! we must understand the context of the problem.
- We explore the system: We investigate the components of the system and how they interact with each other.
- We talk to other people or the system: We gather information through documentation! customer information! peers! or direct testing in the code.
- We share information: We organize what we have learned so we can make informed decisions.
- We synthesize the information obtained: We draw conclusions from the analyzed data.
- We use development tools: We interact with the system through tools that allow us to verify and validate our hypotheses.
However! in practice! this process is often highly manual and relies on monolithic tools that are not optimized for system exploration.
Problems in current decision-making
In most development teams! decision-making faces significant obstacles that impact work efficiency and quality. Some of the main problems include:
- Reliance on monolithic tools: Most current development tools are designed for generic use and don’t adapt to the specific needs of each context. This limits teams’ ability to effectively visualize key data and make informed decisions.
- Unstructured exploration: Most developers analyze systems through manual inspection and code reading without a defined methodology! example of application of design thinking making the process inefficient and error-prone.
- Inaccurate architectural diagrams: Architectural diagrams are often generated manually! which introduces bias and interpretation errors. Furthermore! over time! they often become outdated and no longer accurately reflect the system’s actual structure.
- Lack of optimization in code comprehension: It is assumed that the only way to understand a system is by reading large volumes of code! when in reality there are more efficient alternative approaches for exploring and visualizing information.
Documented development
To address this decision-making and documentation process! documented development (Rewilding Software Engineering) proposes a new approach to software decision-making. Instead of relying on monolithic tools and manual processes! it adopts an approach based on the development of specialized micro-tools.
How does documented development work?
- Creating custom microtools: Instead of relying on a single! general-purpose tool! small tools are created that are designed to answer specific questions about the system.
- Live Modeling: Interactive views of the system are generated! eliminating the need for manual diagrams.
- Structured scanning: Optimizes the way code is analyzed! reducing the need for manual reading.
- Decision-making optimization: Streamlines the process of evaluating and modifying systems! improving the efficiency of the development team.
Case study: Optimizing a data pipeline
Let’s take the case of a systems team that caseno data spent years trying to improve the performance of a critical data pipeline for the organization! which a former colleague had left unprogrammed and undocumented. However! despite an investment of thousands of euros and hundreds of man-hours of effort! the results remained unchanged.
By applying a documented development methodology to assess! understand! and document! they were able to identify and resolve fundamental problems in the system architecture in just two months. They discovered an unknown external service that was affecting performance! visualized the amount of unnecessary data generated! and optimized the data transformation processes. The methodology enabled a 600x improvement in efficiency compared to the traditional approach.