How can we collaborate in a shared reality while creating software systems?

And why we should never stop learning new things

I’ve been grappling with this question for some time now. And it turns out to be quite complex. Lots of influencing factors, dependencies and subquestions. Luckily, complexity triggers me. Specifically my need to learn new things.

I can probably blame my father for this curious habit, or need to learn new things. Some might call it utterly annoying, others will call it a blessing. Ever since I can remember I was being told that I should never stop learning, and that learning in itself is a better reward than any other (tangible) things you may be offered. (Yep — wise words…)

My background in social science, persistent curiosity and my technical working environment make an excellent combination to celebrate this need. I get to see complexity in many different forms every day, and context appears to be crucial in every single situation.

Having a shared sense of reality can make all the difference.

I’ve come to believe that we tend to mistake organizational problems for technical issues. In fact, I believe that almost every technical issue is a consequence of organizational issues. So while we are busy running around putting out fires and treating the symptoms of technical issues, we should probably focus more on the real cause of our problems — organizational challenges.

Looking at a ‘technical’ issue from a more ‘social’ perspective can offer refreshing insights. Understanding the technical issues I’m facing in my daily work often has a lot to do with understanding underlying social dynamics. And that’s how I spend most of my days trying to balance socio-technical complexity and making sense of the related socio-technical challenges. Trust me, there’s no such thing as a purely technical problem.

Adapt in order to stay relevant

A long, long time ago (in 1974), Lehman stated in his renowned paper on the Laws of Software Evolution that “Software needs to be adapted in order to stay relevant”. Not a lot has changed since then. The same goes for organizations. If you want to realize a successful digital transformation, you’ll have to be adaptable as an organization. Effective collaboration is crucial in this process, including the way you work. How you work together depends on what phase your product is in. Adopting a way of working ‘just because it’s the new standard’ is not a good idea. You have to determine what way of working is most effective both for your team and the system you are building.

We tend to mainly focus on the quality of the systems we build, but we should also put a strong(er) emphasis on the quality of ‘the system of work’ — the people building the system. Great architectures are not just about software, but also about the people that create it.

This means we have to learn new things, new skills and new ways of thinking. Both in- and outside our comfort zones. Especially in this so-called IT-world, we can explore this continuous learning path. Coding is a skill, sure. (I am learning that the hard way…) A skill that we can get better at, that we can master. The same goes for how we collaborate as a team.

Let me introduce 3 social dynamics that — in my opinion — are heavily affecting both your systems and ‘systems of work’.

Feedback loops

To be more specific: properly working, closed feedback loops. As opposed to feedback chaos — the more common version of feedback structures where every possible stakeholder is included in one massive ‘loop’. In one of my previous articles, I mentioned that every architecture that we design, every line of code that we write is a reflection of human behavior and interaction. The key, when it comes to feedback loops, is to keep them short and map them to your organizatioanl structures. Having autonomous teams will make your live easier, as they can create their own context, boundaries, shared language and mindset in which they can work. Don’t strive to control the feedback chaos, but rather aim for a manageable situation. This will also boost my next point — having a shared reality within a team (or feedback loop), and organization.

A shared sense of reality

‘Value congruence’, as the social scientists like to call it. In short: knowing why you’re doing what you’re doing. Meaning that everyone involved has a shared understanding of goals, vision, the business problem and strategic solution and the ‘Definition of Done’. When teams have common standards, metrics, and techniques, they are more productive.

Lots of teams might be familiar with the anti-patterns of value congruence: everyone heading in different directions without any form of alignment whatsoever (“What exactly are we doing here?”), or being stuck in meetings all day because of an overkill on alignment (“How many meetings do you think we can fit into one day?”). That’s what we don’t want.

Then how do we get to that shared sense of reality? A first thing to consider is domains; Are we going to organize our teams around technical domains or business domains? Both have advantages, but when it comes to creating a shared sense of reality, I’d prefer business domains. Yes, there are trade-offs, but I believe that it’s easier to establish a shared sense of reality within a business domain than within a technical domain. When we organize or structure teams around business domains, there will be a much better fit with the organization and its goals and vision than when we’re only talking about technical boundaries. It will help smoothen the conversations between IT and the business, and it will help create autonomous teams.

Clarity around knowledge networks

Lots of team members own a lot of tacit knowledge; knowledge in interaction that comes from experience and lives in peoples head. It’s ‘know how’, as opposed to explicit ‘know-what’ knowledge. Tacit knowledge is hard to codify and transfer. But what happens when these team members leave and documentation was not one of their highest priorities?

It’s very valuable to know what knowledge exists in a team or organisation and how that knowledge is distributed. This will also tell you a lot about the dependencies between teams and team members. Mapping these dependencies between teams and team members will tell you a lot about organisational structures and effectiveness. Are your teams depending too much on a few people? Are there too many dependencies between teams for them to be autonomous? These dependencies will be reflected in your code and systems as well. It’s highly useful to map organisational and technical dependencies and see how they affect each other. Dependencies in code increase coupling making code fragile and harder to change, but the same goes for dependencies in teams.

That’s cool and all, but can we measure that?

Yes we can! There are these underestimated goldmines that contain lots of behavioral data that are often overlooked when it comes to visualizing and analyzing organizational structures.

…. Version control history, source code and ticketing systems can tell you a whole lot about these three social dynamics that are affecting your architectures and IT landscapes. These sources contain extremely valuable data about organizational structures and their relation to technical architectures. Basically, you need to collect data around the following 3 aspects to determine whether or not your teams and architecture are aligned. In other words: if your system and ‘system of work’ are structured in an optimal way :

Team structure
Your software architecture
Commit history

This will provide you with all sorts of relevant data that will allow you to answer questions around 3 quality aspects — alignment, autonomy and knowledge distribution, structured around two main themes:

1. Component / Architectural information

Which systems are taking up most of our time and resources?
Is there any relation between the time/effort we’re spending on a component and its technical quality?
Which components can be classified bottlenecks?
How are we doing on planning?

2. Behavioural information

Who worked on which component(s) when, and what changes did he/she commit?
Are there any knowledge monopolies when it comes to certain components/technologies?
Are we depending too much on certain key players? What happens when those people leave?
Is our team structure — as intended — matching reality? Which developers/teams are collaborating a lot and does that make sense? Are our teams really that autonomous?
Are we developing in depth or in breadth? Are we learning mostly about business domains or technical domains?
How full-stack are we actually?

And these are just a few of the questions you can answer with this data from version control history. That’s pretty exciting, right? All hugely relevant. All hugely interesting. All contributing to the three social dynamics I mentioned above. The more you play around with this data, the more valuable insight you’ll get out of it; the more new things you’ll learn.

If I can give you one piece of advice, it would be to always have a curious mind. Explore different topics, skills and methods — and don’t be afraid to ask questions. Yes, even if it feels like you’re the least brilliant person in the room. That’s how you’ll master new skills, knowledge and learn new things. And that, paraphrasing a wise man, is the best reward you can possibly get.

***

A huge thanks to my go-to person — Pepijn van de Kamp — who’s my go-to person and one of my main sources of new knowledge and ideas. Plus, a brilliant coach that actually seems to like being flooded with questions and always challenges me to surprise myself.