< Return to Video

Spotify Engineering Culture - Part 1

  • 0:06 - 0:07
    One of the big success factors here
  • 0:07 - 0:10
    at Spotify is our Agile Engineering Culture.
  • 0:10 - 0:12
    Culture tends to be invisible.
  • 0:12 - 0:13
    We don't notice it
  • 0:13 - 0:14
    because it's there all the time.
  • 0:14 - 0:15
    It's kind of like the air we breathe.
  • 0:15 - 0:17
    But, if everyone understands the culture,
  • 0:17 - 0:19
    we're more likely to be able to keep it ..
  • 0:19 - 0:21
    .. and even strengthen it as we grow.
  • 0:21 - 0:23
    So, that's the purpose of this video.
  • 0:23 - 0:24
    When our first music player
  • 0:24 - 0:25
    was launched in 2008.
  • 0:25 - 0:27
    We were pretty much just a scrum company.
  • 0:27 - 0:29
    Scrum is a well established
  • 0:29 - 0:30
    agile development approach ..
  • 0:30 - 0:32
    and it gave us a nice team-based culture.
  • 0:32 - 0:34
    However, a few years later,
  • 0:34 - 0:35
    we had grown into a bunch of teams
  • 0:35 - 0:37
    and found that some of
  • 0:37 - 0:38
    standard scrum practices
  • 0:38 - 0:39
    were actually getting into the way.
  • 0:39 - 0:41
    So, we decided to make all these optional.
  • 0:41 - 0:43
    Rules are good start, but then
  • 0:43 - 0:45
    break them when needed.
  • 0:45 - 0:46
    We decided Agile matters
  • 0:46 - 0:47
    more than Scrum and ...
  • 0:47 - 0:48
    ... Agile Principles matter
  • 0:48 - 0:50
    more than specific practices.
  • 0:50 - 0:51
    So, we renamed
  • 0:51 - 0:53
    the Scrum Master role to Agile Coach.
  • 0:53 - 0:54
    Because, we wanted Servant Leaders
  • 0:54 - 0:55
    more than a Process Masters.
  • 0:55 - 0:57
    We also started using the term
  • 0:57 - 0:59
    Squad instead of Scrum Team.
  • 0:59 - 1:01
    And our key driving force became Autonomy.
  • 1:01 - 1:03
    So, what is an Autonomous Squad?
  • 1:03 - 1:05
    A Squad is a small cross-functional
  • 1:05 - 1:07
    self-organizing team ..
  • 1:07 - 1:09
    usually less than eight people.
  • 1:09 - 1:10
    They sit together ..
  • 1:10 - 1:12
    and they have ene-to-end responsibility
  • 1:12 - 1:13
    for the stuff they build,
  • 1:13 - 1:14
    design, commit, deploy,
  • 1:14 - 1:16
    maintenance, operations,
  • 1:16 - 1:17
    the whole thing.
  • 1:17 - 1:19
    each Squad has a long-term mission ..
  • 1:19 - 1:20
    such as "Makes Spotify
  • 1:20 - 1:22
    the best place to discover music."
  • 1:22 - 1:23
    or internal stuff like
  • 1:23 - 1:25
    infrastructure for A/B Testing
  • 1:25 - 1:27
    Autonomy basically means that
  • 1:27 - 1:28
    the squad decides what to build,
  • 1:28 - 1:29
    .. how to build it and
  • 1:29 - 1:31
    .. how to work together
  • 1:31 - 1:32
    while doing it?
  • 1:32 - 1:33
    There are of cause
  • 1:33 - 1:34
    some boundaries to this such as
  • 1:34 - 1:36
    the squad mission,
  • 1:36 - 1:37
    the overall product strategy
  • 1:37 - 1:39
    for whatever area they are working on
  • 1:39 - 1:40
    and short-term goals
  • 1:40 - 1:42
    that are re-negotiated every quarter.
  • 1:42 - 1:45
    Our office is optimized for collaboration.
  • 1:45 - 1:46
    Here is a typical squad area.
  • 1:46 - 1:49
    Squad members work together here
  • 1:49 - 1:50
    with adjustable desks
  • 1:50 - 1:52
    and easy access to each other's screens.
  • 1:52 - 1:53
    We gather over here
  • 1:53 - 1:55
    at the lounge for thing like
  • 1:55 - 1:56
    planning session and retrospectives.
  • 1:56 - 1:58
    and back, there is a huddle room
  • 1:58 - 1:59
    for smaller meetings or
  • 1:59 - 2:01
    just get some quite time.
  • 2:01 - 2:02
    Almost all walls are whiteboards.
  • 2:02 - 2:05
    So, why is autonomy so important.
  • 2:05 - 2:07
    Well, because it's motivating
  • 2:07 - 2:08
    and motivated people
  • 2:08 - 2:09
    build better stuff.
  • 2:09 - 2:10
    Also, autonomy makes us fast
  • 2:10 - 2:12
    by letting decisions happened locally
  • 2:12 - 2:14
    in the squad instead of
  • 2:14 - 2:15
    via a bunch of mangers
  • 2:15 - 2:16
    and committees and stuff.
  • 2:16 - 2:17
    It helps us minimize hand-offs
  • 2:17 - 2:18
    and waiting.
  • 2:18 - 2:19
    So, we can scale
  • 2:19 - 2:21
    without block out with dependencies
  • 2:21 - 2:22
    and coordination.
  • 2:22 - 2:23
    Although, each squad has
  • 2:23 - 2:24
    its own mission.
  • 2:24 - 2:25
    They need to be aligned
  • 2:25 - 2:26
    with product strategy,
  • 2:26 - 2:27
    company priorities,
  • 2:27 - 2:28
    and other squads.
  • 2:28 - 2:30
    Basically, be a good citizen
  • 2:30 - 2:31
    in this Spotify eco-system.
  • 2:31 - 2:33
    Spotify's overall mission is
  • 2:33 - 2:33
    more important than
  • 2:33 - 2:35
    any individual squad.
  • 2:35 - 2:36
    So, the key principle is
  • 2:36 - 2:37
    really be autonomous
  • 2:37 - 2:39
    but don't sub-optimized.
  • 2:39 - 2:40
    It's a kind of like a Jazz man.
  • 2:40 - 2:42
    Although, each musician is autonomous
  • 2:42 - 2:44
    and plays his own instrument
  • 2:44 - 2:45
    They listen to each other
  • 2:45 - 2:46
    and focus on the whole song
  • 2:46 - 2:47
    together
  • 2:47 - 2:48
    That's how great
  • 2:48 - 2:49
    music is created.
  • 2:49 - 2:51
    So, our goal is loosely coupled
  • 2:51 - 2:53
    but tightly aligned squads
  • 2:53 - 2:55
    We're not all there yet.
  • 2:55 - 2:56
    But, we have experiment
  • 2:56 - 2:57
    a lot with different ways
  • 2:57 - 2:58
    of getting closer
  • 2:58 - 2:59
    In fact, that applies
  • 2:59 - 3:00
    to most things in this video
  • 3:00 - 3:01
    This culture description is
  • 3:01 - 3:03
    a mix of what we are today
  • 3:03 - 3:05
    and what we are trying to become
  • 3:05 - 3:06
    in the future.
  • 3:06 - 3:07
    Alignment and Autonomy
  • 3:07 - 3:08
    may seem like
  • 3:08 - 3:09
    different end of a scale.
  • 3:09 - 3:11
    as in more automy
  • 3:11 - 3:12
    equals less alignment.
  • 3:12 - 3:14
    However, we think of it more like a
  • 3:14 - 3:16
    two different dimensions.
  • 3:16 - 3:17
    Down here is
  • 3:17 - 3:19
    low alignment and low autonomy,
  • 3:19 - 3:21
    a micro-management culture.
  • 3:21 - 3:22
    No high-level purpose
  • 3:22 - 3:23
    , just shut up and
  • 3:23 - 3:24
    follow orders.
  • 3:24 - 3:25
    Up here is
  • 3:25 - 3:26
    high alignment, but
  • 3:26 - 3:27
    still low autonomy.
  • 3:27 - 3:28
    So, leaders are
  • 3:28 - 3:29
    good at communicating
  • 3:29 - 3:29
    what problem needs
  • 3:29 - 3:30
    to be solved.
  • 3:30 - 3:31
    But, they also tell
  • 3:31 - 3:32
    people how to solve it.
  • 3:32 - 3:35
    High alignment and high autonomy
  • 3:35 - 3:37
    means leaders focus on
  • 3:37 - 3:38
    what problem to solve.
  • 3:38 - 3:39
    But, let the team
  • 3:39 - 3:41
    figure out how to solve it.
  • 3:41 - 3:42
    What about down here then?
  • 3:42 - 3:43
    Low alignment
  • 3:43 - 3:45
    and high autonomy
  • 3:45 - 3:46
    means team do whatever they want
  • 3:46 - 3:48
    and basically all running in
  • 3:48 - 3:49
    different directions.
  • 3:49 - 3:50
    Leaders are hopeless
  • 3:50 - 3:51
    and our product
  • 3:51 - 3:52
    becomes a Frankenstein.
  • 3:52 - 3:53
    We are trying hard
  • 3:53 - 3:54
    to be up here.
  • 3:54 - 3:55
    aligned autonomy
  • 3:55 - 3:57
    and we keep experimenting
  • 3:57 - 3:58
    with different ways
  • 3:58 - 3:59
    of doing them.
  • 3:59 - 4:01
    So, alignment enables autonomy.
  • 4:01 - 4:02
    The stronger alignment we have
  • 4:02 - 4:05
    , the more autonomy we can effort to grant.
  • 4:05 - 4:07
    That means the leader's job
  • 4:07 - 4:08
    is to communicate
  • 4:08 - 4:09
    what problem needs to be solved
  • 4:09 - 4:10
    and why?
  • 4:10 - 4:12
    and the squads collaborate with each other
  • 4:12 - 4:13
    to find the best solution.
  • 4:13 - 4:15
    One consequence of autonomy
  • 4:15 - 4:16
    is that we have
  • 4:16 - 4:18
    very little standardization.
  • 4:18 - 4:19
    When people ask things like
  • 4:19 - 4:21
    which code editor that you use
  • 4:21 - 4:22
    or how do you plan?
  • 4:22 - 4:24
    The answer is mostly
  • 4:24 - 4:25
    depends on which Squad
  • 4:25 - 4:26
    some do Scrum sprints
  • 4:26 - 4:27
    others do Kanban
  • 4:27 - 4:29
    some estimate story
  • 4:29 - 4:30
    and measure velocity
  • 4:30 - 4:31
    others don't
  • 4:31 - 4:31
    it's really up to
  • 4:31 - 4:32
    each squad
  • 4:32 - 4:34
    Instead of formal standards
  • 4:34 - 4:35
    we have strong culture of
  • 4:35 - 4:37
    cross-pollination
  • 4:37 - 4:38
    when enough Squads use
  • 4:38 - 4:39
    a specific practice
  • 4:39 - 4:40
    or tool such as
  • 4:40 - 4:42
    Git. That becomes the path of
  • 4:42 - 4:43
    least resistant
  • 4:43 - 4:45
    and other Squads tend to pick
  • 4:45 - 4:46
    the same tool
  • 4:46 - 4:47
    Squad starts supporting
  • 4:47 - 4:48
    that tool and helping
  • 4:48 - 4:48
    each other
  • 4:48 - 4:49
    and it becomes
  • 4:49 - 4:50
    like a de facto standard.
  • 4:50 - 4:51
    This informal approach
  • 4:51 - 4:53
    gives us a healthy balance
  • 4:53 - 4:55
    between consistency and
  • 4:55 - 4:55
    flexibility.
  • 4:56 - 4:57
    Our architecture is based on
  • 4:57 - 4:59
    over a hundred separated systems,
  • 4:59 - 5:00
    coded and deployed
  • 5:00 - 5:01
    independently.
  • 5:01 - 5:02
    There are plenty of interactions.
  • 5:02 - 5:03
    But, each system focuses on
  • 5:03 - 5:04
    one specific need
  • 5:04 - 5:06
    such as play list management
  • 5:06 - 5:08
    search or monitoring.
  • 5:08 - 5:09
    We tried to keep them
  • 5:09 - 5:10
    small and de-coupled
  • 5:10 - 5:12
    with clear interfaces and protocols.
  • 5:12 - 5:14
    Technically, each system is owned by one Squad.
  • 5:14 - 5:16
    In fact, most Squads own several.
  • 5:16 - 5:17
    But, we have an
  • 5:17 - 5:18
    internal Open-source model
  • 5:18 - 5:20
    and our culture is more about
  • 5:20 - 5:21
    sharing than owning
  • 5:21 - 5:23
    suppose Squad one here needs
  • 5:23 - 5:24
    something done in System B
  • 5:24 - 5:26
    and Squad 2 knows that code best
  • 5:26 - 5:29
    They typically asked Squad 2 to do it.
  • 5:29 - 5:31
    However, if Squad 2 doesn't have time.
  • 5:31 - 5:32
    or they have other priorities.
  • 5:32 - 5:35
    Then, Squad 1 doesn't necessary need to wait
  • 5:35 - 5:36
    We hate waiting.
  • 5:36 - 5:37
    Instead, they're welcome
  • 5:37 - 5:38
    to go ahead edit
  • 5:38 - 5:39
    the code themselves
  • 5:39 - 5:40
    and asked Squad 2
  • 5:40 - 5:41
    to review the changes.
  • 5:41 - 5:43
    So, anyone can edit any code.
  • 5:43 - 5:44
    But, we have a culture of
  • 5:44 - 5:45
    peer code review.
  • 5:45 - 5:47
    This improve quality and
  • 5:47 - 5:49
    more importantly spread knowledge.
  • 5:49 - 5:51
    Over time, we evolved design guidelines
  • 5:51 - 5:53
    , code standards, and other things
  • 5:53 - 5:55
    to reduce engineering friction.
  • 5:55 - 5:56
    But, only when badly needed.
  • 5:56 - 5:57
    So, on a scale from
  • 5:57 - 5:59
    Authoritative to Liberal
  • 5:59 - 6:00
    We're definitely on
  • 6:00 - 6:01
    the Liberal side.
  • 6:01 - 6:02
    Now, none of this would work
  • 6:02 - 6:04
    if it wasn't for the people.
  • 6:05 - 6:06
    We have a really strong culture
  • 6:06 - 6:08
    of mutual respect.
  • 6:08 - 6:09
    I keep hearing comments like
  • 6:09 - 6:11
    my colleagues are awesome.
  • 6:11 - 6:12
    People often give credit to each other
  • 6:12 - 6:13
    for great work and
  • 6:13 - 6:15
    seldom take credit for themselves.
  • 6:15 - 6:16
    Considering how much talent
  • 6:16 - 6:17
    we have here
  • 6:17 - 6:19
    There is surprisingly little ego.
  • 6:20 - 6:21
    One big Ah-ha for new hires
  • 6:21 - 6:22
    is that autonomy
  • 6:22 - 6:24
    is kind of scary at first.
  • 6:24 - 6:25
    You and your Squad mates
  • 6:25 - 6:26
    are expected to find
  • 6:26 - 6:27
    you own solution.
  • 6:27 - 6:28
    No one will tell you
  • 6:28 - 6:29
    what to do.
  • 6:29 - 6:30
    But, it turns out
  • 6:30 - 6:31
    if you asked for help
  • 6:31 - 6:32
    you get lots of it
  • 6:32 - 6:32
    and fast.
  • 6:32 - 6:34
    There is genuine respect
  • 6:34 - 6:35
    for the fact that
  • 6:35 - 6:36
    We're all in this boat together
  • 6:36 - 6:37
    and need to help
  • 6:37 - 6:38
    each others succeed.
  • 6:39 - 6:40
    We focus a lot on motivation.
  • 6:40 - 6:41
    Here is an example ...
  • 6:41 - 6:44
    an actual email from the Head of People Operations
  • 6:44 - 6:45
    Hi Everyone,
  • 6:45 - 6:47
    Our employee satisfaction survey
  • 6:47 - 6:50
    says 91% enjoy working here
  • 6:50 - 6:51
    and 4% don't.
  • 6:51 - 6:52
    Now that may seem like
  • 6:52 - 6:54
    a pretty high satisfaction rate
  • 6:54 - 6:56
    especially considering our growth pain.
  • 6:56 - 6:58
    from 2006 to 2013
  • 6:58 - 6:59
    we've doubled every year.
  • 6:59 - 7:01
    and now have over 1200 people.
  • 7:02 - 7:03
    But, then it continues
  • 7:03 - 7:06
    This is of course not satisfactory
  • 7:06 - 7:07
    and we want to fix it.
  • 7:07 - 7:09
    If you're one of those unhappy 4%,
  • 7:09 - 7:10
    please contact us.
  • 7:10 - 7:12
    We're here for your sake,
  • 7:12 - 7:13
    and nothing else.
  • 7:13 - 7:14
    So, good enough isn't
  • 7:14 - 7:15
    good enough.
  • 7:15 - 7:16
    Half a year later
  • 7:16 - 7:17
    things have improved
  • 7:17 - 7:20
    and satisfaction rate was up to 94%.
  • 7:20 - 7:22
    This strong focus on motivation
  • 7:22 - 7:23
    has helped us build up
  • 7:23 - 7:24
    a pretty good reputation
  • 7:24 - 7:25
    as a workplace.
  • 7:25 - 7:27
    But, we still have plenty of problems
  • 7:27 - 7:28
    to deal with. So, yeah
  • 7:28 - 7:30
    we need to keep improving.
  • 7:30 - 7:32
    Okay, so we have over 50 Squads
  • 7:32 - 7:34
    spread across 4 cities.
  • 7:34 - 7:36
    Some kind of structure is needed.
  • 7:36 - 7:38
    Currently, Squads are grouped
  • 7:38 - 7:39
    into Tribes.
  • 7:39 - 7:39
    A Tribe is
  • 7:39 - 7:41
    a light-weight matrix
  • 7:41 - 7:43
    Each person is member of a Squad
  • 7:43 - 7:44
    as well as a Chapter.
  • 7:44 - 7:46
    The Squad is a primary dimension,
  • 7:46 - 7:47
    focusing on product delivery
  • 7:47 - 7:48
    and quality.
  • 7:48 - 7:50
    While the Capter is a competency area.
  • 7:50 - 7:53
    such as Quality Assitant, Agile Coaching, ..
  • 7:53 - 7:54
    or Web Development.
  • 7:54 - 7:55
    As Squad member,
  • 7:55 - 7:58
    my Chapter lead is my formal line manager
  • 7:58 - 7:59
    is servant leader focusing on
  • 7:59 - 8:01
    coaching and mentoring me as an engineer.
  • 8:01 - 8:04
    So, I can switch Squad without getting a new manager.
  • 8:04 - 8:06
    It's a pretty picture, hah?
  • 8:06 - 8:07
    Except that it's not really true.
  • 8:07 - 8:10
    In reality, the lines aren't nice and straight
  • 8:10 - 8:12
    and things keep changing.
  • 8:12 - 8:13
    Here is a real-life example.
  • 8:13 - 8:14
    from one moment in time
  • 8:14 - 8:15
    for one Tribe.
  • 8:15 - 8:17
    and of cause it's all different by now.
  • 8:17 - 8:18
    and that's ok.
  • 8:18 - 8:20
    The most valuable communication happens
  • 8:20 - 8:22
    informal and un-predictable ways.
  • 8:22 - 8:25
    To support this, we also have Guilds.
  • 8:25 - 8:28
    A Guild is a lightweight community of interest.
  • 8:28 - 8:30
    where people across the whole company gather
  • 8:30 - 8:32
    and share knowledge within a specific area.
  • 8:32 - 8:34
    For example Leadership, Web Development,
  • 8:34 - 8:35
    or Continuous Delivery.
  • 8:35 - 8:38
    Anyone can join or leave the Guild at anytime.
  • 8:38 - 8:39
    Guilds typically have a mailing list,
  • 8:39 - 8:42
    bi-annual un-conferences, and ...
  • 8:42 - 8:43
    other informal communcation methods.
  • 8:44 - 8:46
    Most organizational charts are illusion.
  • 8:46 - 8:48
    So, our main focus is community
  • 8:48 - 8:50
    rather than hierarchical structures.
  • 8:50 - 8:52
    We found that a strong enough community
  • 8:52 - 8:53
    can get away with
  • 8:53 - 8:55
    an informal volatile structure.
  • 8:55 - 8:57
    If you always need to know exactly
  • 8:57 - 8:58
    who is making decisions.
  • 8:58 - 9:00
    You're in the wrong place.
  • 9:00 - 9:03
    One thing that matters a lot for autonomy
  • 9:03 - 9:06
    is how easily get our stuff into Production?
  • 9:06 - 9:08
    If releasing is hard,
  • 9:08 - 9:10
    we'll be tempted to release it seldom
  • 9:10 - 9:11
    to avoid the pain.
  • 9:11 - 9:13
    That means each release is bigger
  • 9:13 - 9:15
    and therefore even harder.
  • 9:15 - 9:16
    It's vicious cycle.
  • 9:16 - 9:17
    But if releasing is easy,
  • 9:17 - 9:19
    we can release often.
  • 9:19 - 9:20
    That means each release is smaller
  • 9:20 - 9:22
    and therefore easier.
  • 9:22 - 9:23
    To stay in this loop
  • 9:23 - 9:25
    and avoid that one.
  • 9:25 - 9:27
    We encourage small and frequent releases
  • 9:27 - 9:29
    and invest heavily in Test Automation
  • 9:29 - 9:31
    and Continuous Delivery infrastructure.
  • 9:31 - 9:34
    Release should be routine not drama.
  • 9:35 - 9:37
    Sometimes, we make big investments
  • 9:37 - 9:38
    to make releasing easier.
  • 9:38 - 9:41
    For example, the original Spotify desktop client
  • 9:41 - 9:43
    was a single monolithic application.
  • 9:43 - 9:44
    In the early days,
  • 9:44 - 9:46
    we've just a handful of developers.
  • 9:46 - 9:47
    That was fine.
  • 9:47 - 9:50
    But, as we grew, this became a huge problem.
  • 9:50 - 9:51
    Dozen of Squads have to synchonize
  • 9:51 - 9:53
    with each other for each release.
  • 9:53 - 9:54
    And it could take months to get
  • 9:54 - 9:56
    a stable version.
  • 9:56 - 9:57
    Instead of creating lots of process
  • 9:57 - 9:59
    and rules and stuff to manage this.
  • 9:59 - 10:00
    We changed the architecture
  • 10:00 - 10:02
    to enable Decupled Releases.
  • 10:02 - 10:04
    Using Chromium embedded framework,
  • 10:04 - 10:07
    the client is now basically a web browser
  • 10:07 - 10:08
    in disguised. Each section is
  • 10:08 - 10:10
    like a frame on the website
  • 10:10 - 10:11
    and Squad can release
  • 10:11 - 10:12
    their own stuff directly.
  • 10:12 - 10:14
    As part of this architectural change,
  • 10:14 - 10:16
    we started seeing each client platform
  • 10:16 - 10:18
    as a client app
  • 10:18 - 10:21
    and evolve three different flavors of Squads.
  • 10:21 - 10:22
    Client App Squads
  • 10:22 - 10:23
    Feature Squads
  • 10:23 - 10:25
    and Infrastructure Squads.
  • 10:25 - 10:27
    A Feature squad focuses on
  • 10:27 - 10:28
    one feature area
  • 10:28 - 10:29
    such as Search.
  • 10:29 - 10:30
    This squad will build, ship,
  • 10:30 - 10:33
    and maintain search related features
  • 10:33 - 10:34
    of our platform.
  • 10:34 - 10:35
    A Client App squad focuses on
  • 10:35 - 10:37
    making release easy
  • 10:37 - 10:39
    on one specific client platform
  • 10:39 - 10:41
    such as desktop, iOS, or Android.
  • 10:41 - 10:43
    An Infrastructure squad focuses on
  • 10:43 - 10:45
    making other squads more effective.
  • 10:45 - 10:47
    They provide tools and routines
  • 10:47 - 10:49
    for thing like continuous delivery
  • 10:49 - 10:51
    A/B Testing, Monitoring, and Operations.
  • 10:52 - 10:54
    Regardless of the current structure,
  • 10:54 - 10:56
    we always strive for a Self-service model.
  • 10:56 - 10:58
    A kind of like a buffet,
  • 10:58 - 10:59
    the restaurant staffs
  • 10:59 - 11:00
    don't serve you directly.
  • 11:00 - 11:02
    They enable you to serve yourself.
  • 11:02 - 11:05
    So, we avoid hand-offs like the plague.
  • 11:05 - 11:07
    For example, an operation squad
  • 11:07 - 11:08
    or client app squad does not
  • 11:08 - 11:10
    put code into production
  • 11:10 - 11:11
    for people.
  • 11:11 - 11:13
    Instead, their job is to make it easy
  • 11:13 - 11:15
    for feature squad to put their own code
  • 11:15 - 11:16
    into production.
  • 11:16 - 11:19
    Despite the self-service model, we sometimes
  • 11:19 - 11:21
    need a bit of sync between squads
  • 11:21 - 11:22
    when doing releases.
  • 11:22 - 11:23
    We manage this using
  • 11:23 - 11:25
    Release Trains and Feature Toggles.
  • 11:25 - 11:28
    Each client app has a release train
  • 11:28 - 11:30
    that departs a regular schedule.
  • 11:30 - 11:32
    Typically every week or every three weeks
  • 11:32 - 11:33
    depends on which client.
  • 11:33 - 11:35
    Just like in physical world
  • 11:35 - 11:37
    if trains depart frequently and reliably,
  • 11:37 - 11:39
    you don't need much up-front planning.
  • 11:39 - 11:41
    Just show up and take the next train.
  • 11:41 - 11:43
    Suppose these three squads
  • 11:43 - 11:44
    are building stuff and
  • 11:44 - 11:46
    when the next release train arrives
  • 11:46 - 11:48
    feature A, B, and C are done
  • 11:48 - 11:50
    while D is still in progress.
  • 11:50 - 11:53
    The release train will include all 4 features,
  • 11:53 - 11:55
    but the unfinished one is hidden
  • 11:55 - 11:56
    using a feature toggle.
  • 11:56 - 11:57
    It may sound weird
  • 11:57 - 11:59
    to release unfinished features
  • 11:59 - 12:00
    and hide them.
  • 12:00 - 12:01
    But, it's nice because
  • 12:01 - 12:03
    it exposes integration problem early
  • 12:03 - 12:05
    and minimize the need for code branches.
  • 12:05 - 12:07
    Un-mearged code hides problems
  • 12:07 - 12:09
    and it's a form of technical debt.
  • 12:09 - 12:11
    Feature toggles let us dynamically
  • 12:11 - 12:12
    show and hide stuff in test
  • 12:12 - 12:14
    as well as production.
  • 12:14 - 12:16
    In addition to hiding unfinished work,
  • 12:16 - 12:18
    we use this to A/B Testing
  • 12:18 - 12:20
    gradually roll-out finished features.
  • 12:20 - 12:22
    All in all our release processes is better
  • 12:22 - 12:23
    than it's use to be.
  • 12:23 - 12:25
    But, we still see plenty of improvement areas
  • 12:25 - 12:27
    so we'll keep experimenting.
  • 12:27 - 12:29
    This may seem like a scary model
  • 12:29 - 12:32
    letting each Squad put their own stuff
  • 12:32 - 12:33
    into production
  • 12:33 - 12:35
    without any formal centralized control.
  • 12:35 - 12:36
    and we do screw up sometimes.
  • 12:36 - 12:38
    But we learned that trust
  • 12:38 - 12:39
    is more important than control.
  • 12:39 - 12:41
    Why would we hire someone
  • 12:41 - 12:42
    who we don't trust.
  • 12:43 - 12:44
    Agile at scale requires
  • 12:44 - 12:46
    trust at scale
  • 12:46 - 12:48
    and that means that no politics.
  • 12:48 - 12:50
    It also means no fear.
  • 12:50 - 12:52
    Fear doesn't just kill trust.
  • 12:52 - 12:53
    It kills innovation
  • 12:53 - 12:55
    because the fear to get punished
  • 12:55 - 12:57
    people won't dare try new things.
  • 12:57 - 12:59
    So, let's talk about failure.
  • 12:59 - 13:01
    Actually, no
  • 13:01 - 13:02
    let's take a break.
  • 13:02 - 13:03
    Get on your feet,
  • 13:03 - 13:04
    get some coffee.
  • 13:04 - 13:06
    Let this stuff sink in for a bit
  • 13:06 - 13:07
    and then come back
  • 13:07 - 13:09
    when you're ready for Part 2. Okay?
Title:
Spotify Engineering Culture - Part 1
Description:

An attempt to describe our engineering culture.

This is a journey in progress, not a journey completed. So the stuff in the video isn't all true for all squads, but it appears to be mostly true for most squads.

(NOTE - Part 2 hasn't been recorded yet. Stay tuned!)

more » « less
Video Language:
English
Duration:
13:12

English subtitles

Revisions