Developer Control Planes: A Community Practice Engineer’s Point of View

This article was originally posted on Ambassador.

Apostolis Apostolidis (better known as Toli) (@apostolis09), Principal Practice Engineer at UK-based car-purchasing platform, cinch, joined Ambassador Labs’s Daniel Bryant in the latest Ambassador Labs podcast. Toli discussed the culture of software engineering, how best to engineer platforms to help teams achieve their goals, and how to set up and benefit from communities of practice.

Here are some of the key takeaways from their conversation:

• Consider how to improve the craft of software engineering

Knowledge loss is a downside to the way many development teams are organized. Discussion on the possible need for internal developer relations functions led Toli to conclude, “One of the biggest problems we have, in Team Topologies speak, is that we have a lot of stream-aligned teams, but minimal platform teams. This leads to overindexing on streams, and each team focuses on their individual domains, and the different disciplines are not talking to each other. You lose critical knowledge-sharing. The way teams are often set up, people are not building cross-functional relationships with each other, but we want people to learn together and from each other. In questioning, “How do we continuously improve our craft?”, we need to consider how teams work together (and ensure that they do) and look to informal communities of practice.”

• Create and communication around communities of practice

Communities of practice might be a new concept for many, although Toli adopted the idea and introduced it to his organizations several years ago. As defined by the Harvard Business Review, communities of practice are “groups of people informally bound together by shared expertise and passion for a joint enterprise.” Bringing different people from different roles together to discuss shared interests in an informal but regular way to focus on practice yields benefits.

“With time, your community of practice matures. You go from generating interest, getting people to join, meeting at a regular cadence to beginning to see cross-functional benefits. You aren’t just talking about things once your mature curve accelerates. Instead you may be doing a project together. For example, one project might be about improving CI/CD across the org. How can we look at the backend template and recommend, as a community, tangible changes that will improve the CI/CD mechanism? And further, how can we improve and share knowledge around this? This involves not just the community but extending it to communicate to a wider audience, for example in GitHub Discussions or a Confluence page,” Toli explains. “Facilitation is a particularly useful skillset. It enables communities, makes them inclusive, helps to move things along, encourages people to participate. Facilitation techniques are a massive bonus for use in fostering communities of practice.”

• Foster cross-org relationships and learning in a fast-scaling environment

Once the cinch team recognized that the silos they faced were in part due to its rapid scale-up process, it became more critical than ever to enable learning and technical knowledge sharing at the organizational level. This involved both team structure and technology and tooling choices.

Toli explains how, at least for some time, teams were designed to ensure that each team had a dedicated person who wore several hats, for example, a software engineer who had an extra focus on infrastructure, CI/CD, SRE, and so on. Having this dedicated resource with more of a full life cycle perspective helped the teams build software faster and better. “The idea,” Toli continues, “is to introduce enablers, or many coaches, who help teams scale.”

Technology, too, is a consideration for fast-scaling organizations. As Toli describes, the pace of growth will throw enough of a learning curve into the mix without adding the burden of too many unfamiliar technologies. In his example, cinch already had a website with a backend hosted in a Kubernetes cluster. Despite taking time to learn Kubernetes and Helm charts and concepts and tooling, the team realized that if they wanted to build a team around Kubernetes, they had to understand it completely and make it stable. They made the decision to move away from Kubernetes to serverless in order to reduce the learning curve and cognitive load.

• Reduce developer cognitive load and get closer to customers

Most organizations grapple with the problem of saddling developers with too much cognitive load. At cinch, they found the cognitive load of adopting and using Kubernetes to be too high. Moving to serverless, the team was able to shift focus from infrastructure to observability and monitoring.

“By leaving containers in the past, we gained cognitive space that we could redirect to observability, which had the bonus of letting us get closer to the customer. A year and a half later, rather than becoming a Kubernetes expert, I became immersed in observability and monitoring using DataDog, and this helped us understand business metrics and understand how customers were using the website rather than metrics around Kubernetes clusters.”

Listen to Toli and Daniel’s full discussion below.