Tips for in-house teams in a free market software culture

For the past 18 months I’ve been running a team at the Financial Times called the “Origami Team”. Our team mission is to:

  • reduce the time the other development teams spend repeating work
  • unify design across the FT digital products

People at the FT seem to really like my team and what we’re building – the adoption of our products is high. This means that sometimes people ask me for pointers on what we’re doing, so I thought I’d write that down.

In the post I’ll cover:

I should point out that I joined this team 2 years ago and a lot of the things I’m going to talk about in this post were already in place when I started, put there by the many brilliant people who worked here before me.

Free market software teams

I’ve had this blog post by my colleague Matt Chadburn bookmarked since I joined the Financial Times 2 years ago. It is a great insight into how the software culture at the FT works. I’ll leave him to describe it succinctly:

The last few projects I’ve worked on have broadly followed the mechanics of free-market economy, where teams are allowed and encouraged to pick the best value tools for the job at hand, be they things developed and supported by internal teams or external to the company.

So, if you’re already running a tooling/SaaS/platforms team in this environment, or about to start, the best piece of advice I have for you is to read that blog post by Matt.

In this post I’ll cover some practical things we actually do, and some fuzzier culture things that the Origami team believes are important.

What is your team really for?

Foundational to being a successful in-house services team in a free market software culture is deciding what to build and why. There are probably a lot of ways to determine this but here is Origami’s.

In the first paragraph, I described our team’s mission as:

  • reduce the time the other development teams spend repeating work
  • unify design across the FT digital products

These are amazing points to work against (I don’t know who came up with them – probably Andrew Betts). They could have been things like “build and maintain services to help team use images” or “maintain a component library” – both of which we do – however the success of our team is not determined by if we built something or not, it’s if we saved people some time or not. This mission allows us to keep quite free about what we’re doing, and it forces us to celebrate what actually matters – not “did we ship a tool” but “did people actually use it and did it save them time”

Having a team mission can be tricky to nail down but it is hugely empowering when you get it right. Examples of good missions could be “reduce number of cyber security incidents” or “reduce AWS spend across the business”.

Data

Once you’ve worked out what your team is for, it’s also probably important to work out how to measure it.

Origami has a couple of key performance indicators which we report on quarterly to the rest of the business. For our web services we report uptime and adoption which we can work out using Pingdom and logs which we send to Splunk. For our front end components system we look at the number of websites the FT has built which have shipped with one or more of our components. This is much trickier to measure.

Having some reasonably rich data about how people are using our stuff is also important for working out what things we could look at deprecating, which in turn, keeps the amount of code we have to maintain down.

Your tools and services are products

Because you’re building a tool to be used by (potentially not that many) other developers, it can be tempting to skip on the design and research. This is very rarely a good idea. In Origami, though we don’t aim to make everything we’ve built as fully finished as a consumer product, we still bring the core ideas and practices for building good products into the process.

This means doing some form of requirements gathering, prototyping, workshops with potential users, user research and iteration. Keep this in proportion with the thing you’re trying to build of course, but even a little CLI tool will double in usefulness if you’ve gotten feedback from at least one potential user before you ship it.

If you come from a consumer products background, where competition with other products is fierce, then this way of working should be obvious. If you’re used to having a closed market around your tools (ie people have to use them, they aren’t allowed to choose an alternative), this might not be something that was important before.

Getting people to use the thing you built

Say you have some tool and services and you’re pretty confident that they’re the right things for the job. They’re well engineered, they’re easy to use because you’ve involved users early on to remove any stumbling blocks, and they address a real problem that the developers at your company have.

That’s fantastic but your job still isn’t done.

Because you’re competing in a free market ecosystem not only do you need to build the right thing, but you also need to provide the following:

  • Feature requests / bug fixes – how do we decide what to work on
  • Support – how do we help people using Origami
  • Documentation – what technical documentation do we provide
  • Comms – how do we let people know about new features, deprecation, system outages

The way the Origami Team (4 software developers) does this is by making sure that, broadly, every person in the team understands that their job is writing software and also participating in everything listed above. It might not make sense for your team to work like this. An alternative approach could be to assign particular responsibilities in your team or hire specialists into these roles.

Team culture

Before I go into the practical aspects of comms, support, documentation and improvements, a quick note on team culture. I was lucky enough to be able to hire everyone into the current team (two external hires and one internal). When we interviewed, I made it clear that as well as writing code, the role in the team would involve talking to other developers a lot. Be that doing support for people who were trying to use our stuff, or responding to issues, or doing code reviews of patches submitted by other people.

I also encourage the team to be sociable with other teams in whichever forms they are happy with. For some of them it’s going to the pub after work, for some it’s playing board games at the FT’s board games night. For me personally (not a heavy drinker and honestly a bit too intimidated to go to the board games night) it’s just saying hello to people in the corridor and following up if people have asked questions to make sure their issue got resolved.

I haven’t hired a team of party animals or extroverts – but I’ve been careful to hire people who are comfortable with communicating with other developers (could be in person, or over Slack or email) and understand it’s part of their job to do that.

As the leader of such a team it’s up to me to help my team to do this work as best they can. This means being flexible about different communication preferences; don’t force people who prefer typing to go talk face to face, but help them if that would be a better option. I also have to do a bit of protecting my team from having to answer live support issues when they’re really busy or just having a bad day.

Bootcamps

A lot of development teams at the FT send their new hires on “bootcamps” which are (usually) week long rotations with other teams. For Origami, bootcamps are incredibly important because they lay the empathetic foundations for future work. Understanding how other teams work, what they’re doing with Origami, and what their time pressures are like, is an important part of working out how to help them.

Bootcamps also allow us to do some covert user research on how teams are using Origami. Sometimes getting into the nitty gritty of how a team is working day-to-day allows us to see things we can improve for them that they would not think to mention to us.

Secondments

Sometimes some work will come up that needs a very close integration with another team’s work. In these cases we will send someone from the Origami team on a secondment with the other team to build the new thing while physically sitting with them. This, like a bootcamp, is a great hack to build empathy with our end users. The end result is also less likely to have problems from not misunderstanding requirements as any questions along the way can be resolved with a quick conversation.

Feature requests / bug fixes

So – team culture tips out the way, here is a list of things we actually do.

Something that the Origami team does which is quite unusual at the FT is we don’t have a backlog and we don’t work in sprints. This works very well for us but we are a team of 4 developers, please don’t take this advice wholesale into your own team without thinking carefully first.

Many of the other development teams at the FT work in sprints and when they raise an issue or a feature that means they want it fixed as soon as possible. Putting their issue into our next sprint would be too slow for them, by which time they’re out of their own sprint. To prevent this, at any one time, the team is either working on some long term strategic play (usually something nobody has directly asked for but we think would help the team or the business decided at our 6 monthly planning away days) or they’re working on fixing a bug or adding a feature. When someone is blocked on something because of a problem with something Origami built, our top priority is to unblock them as quickly as possible.

Support

Good support pays dividends. Not only does it help to get people using your product, it also serves as an opportunity to find out what’s not really working with your product (missing features, bugs, unclear documentation), which means you can fix it for next time. As a team we work hard to keep all support interactions friendly and our response times quick.

In free market software culture, lots of support requests are much healthier than silence. Silence could mean you’ve made a completely flawless tool, but it could also mean it sucks and nobody can be bothered to tell you, which means they’ve found something better to use.

Slack

The FT uses Slack and the Origami team has a dedicated support channel there. People come to this channel with all kinds of issues and suggestions and everyone pitches in to answer questions (sometimes even people who aren’t on the team will help out!)

Keeping our support channel welcoming to newcomers is important to me for practical reasons, and on ethical grounds too. Sometimes this means challenging bad behaviour which requires some confidence and tact but is necessary.

A general measure for if you’re doing the approachable Slack thing well is imagining today is your first day and you’re a bit nervous. You go to the support channel and read some of the backlog. Does that backlog leave you feeling comfortable asking a question in that channel or is it full of people being a bit shirty with each other?

Email

Origami also has a shared email address which can be useful for people who aren’t avid Slack users. At the FT there are a lot of teams who don’t use Slack in the same way product and technology do, email is a good way for those groups to reach us.

Open meetings

Every Friday we have an open meeting that anyone can come to and ask questions. If someone contacts us with a more involved feature request we might ask them to come along to the open meeting so the whole team can hear about it and discuss it.

Workshops

We run a workshop for new starters or people who want a refresher on the team and its projects. The workshop lasts about two hours and we cover what the team does, how to use our projects, and how to get help if you need it. This is often people’s first contact with the team so we try to make sure it’s a positive and engaging session.

Pairing

We do barely any pair programming within the team, but it is invaluable when trying to offer support to someone else who has a problem with an Origami component.

Github issues

All of our repositories are public on Github so sometimes people will raise issues for us. As with all other forms of support request we welcome this and will try to make sure we respond quickly (though I’ll admit, this is one of the least effective ways of getting support from us as we have so many repositories it’s easy to miss things)

Documentation

If doing all of the support things above seems like it might eat a lot of your time then documentation is your friend here. Documentation is part of a product maturation process, where you move from a scrappy MVP to something more serious. For Origami we have some getting started guides on http://origami.ft.com, and every component has its own README which has some documentation that’s specific to that component.

Services like the Image Service and Polyfill.io host their own documentation.

To help keep our documentation consistent we have a (brief) style-guide here: https://github.com/financial-times/ft-origami

Poor documentation erodes confidence in your product which, like poor support, is a push to use one of your competitors.

Communications

We’ve covered what happens when people come to you – support requests, github issues, etc. Sometimes you’ll need to get messages out to people.

Big change comms

In Origami we do some comms if a big change is happening – maybe we’ve deprecated something, or made a significant release that we want people to update to.

One thing we take very seriously is not breaking things for our users unless it is completely necessary, and in the case that we have to, making sure everyone impacted has 6 months to make whatever migration or fix is needed.

When we make a serious breaking change to a web service, someone in the team will lead on the comms, and that includes writing a comms plan (most often this is a spreadsheet so we can track which emails we’ve sent and who has responded).

Our plan is usually something like this:

  • 6 months before the service switch off: do a general broadcast about the new service or change and how to migrate. Include why the new thing is better, and why we’re switching off the old thing. Try to explain the likely impact of not switching over. This type of general post would go on our internal social network (We use Workplace here which is Facebook’s offering for companies), and in our slack channel.
  • Also 6 months before service switch off: Send direct emails to all teams (usually the product owner, or a tech lead) using the old service to say the new service is live and the old one will be shut down in 6 months.
  • 3 months before service switch off: Identify who is still using the old service and send a follow up email. If it’s possible, include the product owner’s name and the names of their impacted projects in the email so they have a wider context for what we’re talking about.
  • 1 month before the service switch off: Again, identify who is still using the old service. If possible, chase them directly. Depending on the severity of not switching to the new thing, we might try a different contact method if we haven’t heard anything back from the team using the old service if they haven’t moved. This is the point at which we’ll also check to see if we can help people with migrating by sending a developer to work with the team.
  • The day of service switch off: Email everyone announcing the switch off, celebrate the cost saving of not having to support the old version of the service anymore.

Some tips for these kinds of messages (be they for Slack, email or Workplace).

  • Keep them informal, brief, and scannable.
  • Put the turn off date in every message in bold.
  • Include what will happen if people don’t switch over. Will they incur an HTTP redirect? Will something break?
  • Include how to get in touch if they have further questions.
  • Tell them you’ll email them again in n months
  • We like to use emojis in the subject line to help the email stand out in people’s inboxes. Some people hate emojis so – YMMV.
  • If the change-over instructions can be put in an email (ie they are short) then do this! If they are more involved, link to somewhere with a more detailed migration guide.
  • Get someone to proofread the email. Believe me, it is deeply embarrassing when you send out an email with a typo, especially when the typo is in the switch over instructions.

Comms around screwing up

Something that will help build trust between you and your users is how you deal with the situation in which one of your products has broken.

I think it’s virtually inevitable that one day your service will go down, or you’ll release a version of your tool which breaks something in a live site that uses it. The difference between regular teams and great teams is how they deal with the issue as it happens and what they do after the fact.

In Origami if someone reports that a release has broken something on a live site or a service has gone down, fixing that takes priority over all other work. While the fix is happening someone will also be in charge of comms. This means making sure the status of the fix is posted to the slack channel as it progresses, and, if things aren’t too hectic, that person will also start writing up the incident post mortem.

Our post mortems have three sections:

  • A summary of the incident. This should include what the problem was, what caused it (if you know), who was impacted, what the fix was, and any lasting issues.
  • A timeline of events. This is a list of relevant events including things like the first report of the incident, the time and point at which something was rolled back, or a patch was released. In the chaos of a live incident it can be really useful to have a timeline of what has been done. In the event that the issue is very difficult to fix or the incident goes on for a long time, having such a timeline will help new people who’ve come in to help.
  • A list of talking points. As you go through a live incident having a place at the bottom of the document to dump questions that come up or things you want to go back and fix later is really useful. This talking points list should be refined and then should form the basis for your post mortem meeting.

We publish our post mortems or send them around. We believe this is really important so everyone can learn from our mistakes. Once the incident has been dealt with we’ll also have a meeting about the incident. Anyone can attend these and we encourage anyone affected to come along. The main focus of this meeting is not to work out whose fault the incident was, but what we can learn from this incident and what we should change to prevent it happening in the future. We loosely follow the principles of blameless post mortems, which John Alspaw of Etsy has written about in detail here: https://codeascraft.com/2012/05/22/blameless-postmortems/

Broad reach comms

One thing we also do for Origami is broadcast messages to remind people we still exist and what we’re up to. These are usually quarterly Workplace posts, but sometimes they’re lightning talks. Developers don’t really care for these kinds of comms, but they’re good for reaching other people in the business who do have an impact on our future but don’t use Origami themselves (eg the finance team, product managers, etc)

Summary

As Matt points out in his original post on free market software teams, in-house teams have many significant advantages over third parties, especially when it comes to access to users. Where internal teams seem to go wrong is not appreciating that the thing they’re building is still a product and so it needs to compete with other products on the market. There isn’t a single one thing (eg “have a workshop”, “hire an advocate”) that you have to do to get an in-house team really competing with an external company, but lots of small things that help set the team up for creating the right products in the right way.

Credits

Thanks to @jakedchampion, @rowanmanning, @commuterjoy, @tekin and @jwheare for the proof reading and edits. ❤️