AgileHow-to GuidesHow-to Managing And ControlingLeadership & Team ManagementTopics

The Hard Truth About Scalability: What To Know Before Scaling Up

By 04/09/2019 September 16th, 2019 No Comments

Visualize your work and manage projects with ease.
Try Hive, the productivity platform for fast-moving teams.

Our friend and supporter

Brace yourself, because I’m about to tell you the hard truth about scalability.

We’ll start by getting real about scaling in business:

Most scaling efforts fail.

So, what’s the secret sauce? How do you scale a team successfully, and why is it so hard?

If you have been in the software development business long enough, you’ve most likely experienced this nightmare:

  1. The customer is promised lots of features in an unrealistically short time-frame (that nobody has the guts to challenge).
  2. Later, when the deadline is dangerously close, someone in a position of power decides to throw more teams at the problem in order to meet the deadline.
  3. Things start to fall apart.

Sound familiar?

In this article, I’ll explain what you need to do, in order to avoid that nightmare situation again (or, if you have been lucky enough not to experience it, to avoid it altogether).

My insight in this article comes from scaling software development teams, but you can apply the fundamental theory to scaling a business or growing a team in any industry. You will find this useful even if you’re adopting one of the out-of-the-box scaling frameworks—Large Scale Scrum (LeSS), [email protected], SAFe.

This is because I focus on several important aspects of scalability that are never mentioned in those methodologies—the fundamental truth about the nature of scaling up that often gets forgotten or ignored.

In the following sections, I’ll introduce:

  1. The Fundamental Law Of Scaling—explains the tradeoffs involved with scaling up.
  2. The Prerequisites For Scaling—without these, scaling up will only make things worse.
  3. The Limits Of Scaling—an approach to find out if your project has already reached its limit.
  4. Considerations On Teams Structure—is it better to have feature teams, component teams, or what?
  5. Two Paradoxes Of Scalability—these sum up why most scaling up efforts fail.

Scalability - featured image

The Fundamental Law of Scaling

The Fundamental Law Of Scaling is an empirical observation of what happens in projects when they scale up:

Scaling up amplifies the bad and makes the good more difficult.

I came up with the wording, but I’m neither the only, nor the first, to come up with the insight.

Let’s see what the Law means in practice with some scalability examples.

1. Example: Communication and Scalability

The first and most obvious one is communication. It is a well-known fact that when the number of people in a project increases, so does the number of communication channels. If communication channels are not good enough to start with, scaling up will make the problem worse.

On the other hand, if communication channels are good to start with, scaling up will introduce some challenges due not only to the increased number of people, but also due to their distribution. Often new teams are created in different locations, forcing changes in communication channels—for example, going from face-to-face meetings to video conferencing or by replacing a whiteboard with an electronic tool.

2. Example: Continuous Integration/Continuous Delivery and Scalability

We also see the Fundamental Law of Scaling in Continuous Integration (CI) and continuous delivery (CD) pipelines. When there is only one team, things are quite straightforward.

Moving from one to two teams, the CI and CD systems become more complicated—usually, when there are n teams, the number of CI pipelines tends to be n+1 (one pipeline per team, plus one for integration). This has obvious repercussions on provisioning of environments, and on the coordination among teams.

3. Example: Feedback Loops and Scalability

A final example: feedback loops. Larger projects have longer feedback loops. Therefore, refining the system in an iterative manner becomes much more difficult, increasing the chances of building the wrong product.

These are just three examples of some of the challenges introduced by scaling up. I’m sure you will be able to come up with many more.

Let’s assume you are willing to accept the tradeoffs involved. The next thing you need is for your project to satisfy some important prerequisites to prove out your scalability.

10 Prerequisites for Scaling

Whether or not you’ve got a scalable business, project, or team depends on several prerequisites listed below. If you fulfill these prerequisites, then you have high scalability. On the flipside, if any of them is not satisfied, the chances of failure increase dramatically.

1. The Presence Of Clear, Shared Goals

This should be a given, but in my experience, the absence of clear, shared, goals is extremely common. Without these, the teams will pull in different directions, making it more difficult to deliver something useful.

2. The Use Of Appropriate Metrics

The bigger the project the more important it is to be able to measure the impact of the decisions taken. For instance, did the addition of a new team improve the ability of the project to deliver? Are we pushing the teams so hard that quality is dropping? Are teams working well together, or are they interfering with each other?

3. A Suitable Architecture

If the “shape” of the system doesn’t match the teams structure, adding more teams will only make things worse. This is a direct effect of Conway’s Law, a well known empirical observation that has been proven in practice.

4. Availability Of Managerial And Technical Skills

Issues due to lack of skills will have an amplified negative effect as the project scales up.

5. Good Communication Channels

The more people involved, the more communication takes place. Successful scaling needs streamlined communication.

6. Good Prioritisation And Planning

Lack of appropriate prioritisation and planning, in my experience, is a very common factor in project failures. In particular, when there is more than one team involved, prioritisation and planning activities have to make provisions to mitigate the effects of delays and lack of synchronisation.

7. Good Requirements Elicitation And Management

This goes hand-in-hand with prioritisation and planning. Also, the effect of wrong, misunderstood, or unnecessary requirements will be amplified as the number of teams grows.

8. High Quality Work

The bigger the project, the bigger the negative effect of bad quality. In some pathological (and common) scenarios, some bugs may become features, as the feedback loops to fix them can be so long that other teams may have designed their workarounds to neutralise them.

9. Availability Of Resources

More teams need more desks, computers, tools, and environments.

10. Ruthless Automation

Any manual activity is a bottleneck whose effects become worse with the number of people and teams. A (very common) case in point: manual testing activities to ensure no regressions have been introduced before a production release. In my experience that is one of the biggest bottlenecks in any projects, but especially in projects with more than one team.

Watch Out For Premature Scaling

Scalability is at the top of mind for startups and serial entrepreneurs—and they’ve got a lesson or two to teach us about scaling prematurely.

If you undertake a scaling effort before you’ve really met the prerequisites for successfully scaling a business, team, or organization, you get what the startup community calls “premature scaling”.

A simple definition of premature scaling from serial entrepreneur Jim Pitkow:

“Premature scaling: growing in anticipation of demand instead of demand driven growth.”

Indeed, scaling properly takes time—and true scalability is driven by a real need for scale due to demand, not by artificially imposed notion of simply getting bigger and doing more things faster.

The startup community offers a couple of lessons we can take back to our teams and projects when when think about whether or not it’s time to scale. Here are a few stats from the Startup Genome Report Extra on Premature Scaling:

  • Startups that scale properly take 76% longer to scale to their team size than startups that scale prematurely.
  • 74% of high growth internet startups fail due to premature scaling.
  • Startups that scale properly grow about 20x faster than startups that scale prematurely.

Scalability - infographic

Are You Really Ready to Scale?

If your project is well run, and satisfies the prerequisites above, your next question should be:

“How many teams can contribute productively?”

We will answer this one in the next section.

What’s The Limit Of Scalability? Brooks’ And Amdhal’s Laws

It is a well known fact that adding more people to a software project is not necessarily going to increase productivity.

Actually, in most cases, adding more people will cause a severe productivity drop.

Fred Brooks made this observation in his Mythical Man Month book, which became known as Brooks’s Law:

“Adding manpower to a late software project makes it later […] The number of months of a project depends upon its sequential constraints. The maximum number of men depends upon the number of independent subtasks. From these two quantities one can derive schedules using fewer men and more months[…]. One cannot, however, get workable schedules using more men and fewer months.”

The latter half of the quote above is, in my opinion, the most interesting. In fact, it’s basically the plain language version of Amdhal’s Law—a formula that gives the maximum theoretical speed-up of a concurrent system:

Amdhal’s Law

Speedup = 1 / (s + p / n )

Where:

s = percentage of serial tasks

p = percentage of parallel tasks

s + p = 1

n = number of processors (teams, in our case)

In fact, a project with multiple teams is a kind of concurrent system where each team is a processing unit. In practice, Amdhal’s Law states that the maximum amount of concurrent work is limited by the amount of serial work that needs to be done.

In plain words, this means that the things that needs to be done in a particular order will place restrictions on the amount of stuff that teams can work on all at the same time.

What Is Serial Work In A Software Development Context?

You may be wondering what kind of work is serial work that affects software teams.

Here are a few examples (I’m sure you can come up with many more):

  • Any work on integration and deployment pipelines
  • Merging software changes in code shared by different teams
  • Any kind of synchronisation—for example, a team that depends on deliverables from another one before continuing their work
  • Waiting for the provisioning of resources

Applying Amdahl’s Law To Scaling Teams

The graph below displays the implications of Amdhal’s Law when scaling up the number of teams on productivity.

We can see that productivity will scale up in sub-linear fashion with the increase in the number of teams—until a peak, after which it will decrease again.

Scalability - Illustation of Amdahl Law

An illustration of Amdahl’s law: Deliverables throughput increases with the number of teams, but only until a certain point—and it’s not usually what managers hope.

If you only remember one thing from this article, this is the one:

Increasing your number of teams can increase throughput—but only until a certain point.

So, What’s The Ideal Number Of Teams For A Given Project?

Unfortunately, there are no equations to calculate the ideal number of teams for a given project in advance. The only way is to use a set of project metrics to measure things like productivity and quality, and to see what happens when teams are added or removed. My recommendation is to start small with one team of 3-6 people, and scale up only when and if necessary.

At some point, you may find out that you’ve already reached the peak number of teams, but your project will still be unable to meet its commitments. That’s where your prioritisation and communication skills come in handy, as you may have to have some tough conversations with your customers.

Considerations On Team Structure: Feature or Component Teams?

When adding a new team to the project, how are they supposed to work? To start, ask yourself these two questions:

  1. Will they be in charge of developing customer visible features end to end, or by changing something in the system to achieve their goal?
  2. Will they be in charge of working on a specific component providing part of some user visible functionality?

If you said yes to #1, the team will be a feature team.

If you said yes to #2, it will be a component team.

Tips For Choosing The Right Team Structure

Choosing the appropriate team structure and team organization is very important, but it is not easy. Currently, feature teams seem to be the preferred option in most situations. However, that choice is often based on habit or routine instead of data.

The reality is that “it depends”. More specifically, it depends on the architecture of the system. Software projects tend to follow what is known as Conway’s Law—an empirical observation by Mel Conway:

“Organizations which design systems…are constrained to produce designs which are copies of the communication structures of these organizations”

In other terms, the “shape” of the teams structure has to match the “shape” of the system, or the project will experience some serious problems.

This is the reason why I always recommend managers to involve senior architects and technical leads in teams structure decisions—otherwise, managers are really doing system design without realising it (and without the appropriate skills). That, in turn, can greatly increase the risk of project failure.

Two Paradoxes Of Scalability

In my experience as a consultant with a number of large scale projects, I’ve come to the conclusion that, in most cases, projects scale for the wrong reasons.

I’ve invented two paradoxes that sum up what I’ve seen in practice:

First Paradox Of Scaling

Most projects are scaled up because they don’t fulfill the prerequisites for scaling

Second Paradox Of Scaling

Projects fulfilling the prerequisites for scaling have a lesser need to scale

the 2 paradoxes of scaling

The first paradox says that, in most situations, projects are proceeding slowly simply because they are not well managed.

The second one says that well-managed projects will have a lesser need for scale simply because they are well managed.

In fact, the most productive projects I’ve seen had only one or two small teams and fulfilled all the prerequisites for scaling. Of all the large scale ones I’ve been involved with, I haven’t seen a single one where all the prerequisites for scaling were satisfied. In fact, all of them had plenty of problems in fulfilling any of the prerequisites.

If you are involved in a project that is experiencing issues and delays, the best recommendation I can give is to scale down the mess, before considering scaling up the project.

Takeaways To Scale Your Teams The Right Way

Scaling software teams is not easy, and needs to be done with extreme care. Before you go, some final thoughts on finding the most effective way to scale a software development team:

  1. If you work on satisfying the prerequisites for scalability, you may find that you keep it small and avoid all the issues that come with a bigger size.
  2. On the other hand, if your project is already a large and challenged one, then focus on scaling down the mess first.

Here are the first steps I suggest you take after reading this article:

  1. Find out what is happening on the ground
  2. Implement some metrics to be able to assess the current situation, as well as the quality of the decisions made.

And as always, please comment, share, and get involved with the conversation—what do you think leaders and managers should know before scaling up?

Giovanni Asproni

Giovanni Asproni

I am an experienced Software Architect, Programmer, Coach, and Consultant with experience in many domains, including, among others, investment banking, telecommunications, oil and gas exploration and travel security. I have experience in providing consulting and advice to projects of all sizes: from small ones carried out by small co-located teams up to projects involving hundreds of developers split in geographically distributed teams.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.