Earlier this month I was invited as a panelist to the Engine Room conference here at the FT. As I spent last year exploring BDD practices as a product owner with the development teams, I was there to contribute on the automated testing panel. I decided to make it a full day and stayed in the audience for all the five panels. I expected a day of in-depth conversations around technical tools and engineering practices. I came prepared for that I might not understand even half of the details, but hoping to catch up on the latest trends in FT Technology. My expectations were wrong. Most of the discussions centered around what we build and how we manage our product life-cycle and as a product manager I found myself right in the middle of the action!
After a great day at the conference I wrote up my impressions from the panels and from discussions throughout the day. This is a (slightly edited) note I sent to Product Management at the FT. Perhaps there are thoughts you will find useful!
Build – Measure – Learn
The Build, Measure, Learn cycle from Lean startup was a recurring theme. I think it was mentioned or referred to in every talk. There is a strong engineering focus in this area. Tech leads in various functional areas are focused on improving and shortening build – measure – learn cycles. There is a general perception that we build a lot, measure less and learn very little in our product development efforts. We tend to build incrementally, but not iteratively with learning included in the process.
The successful development team quick-starts are good examples of how Technology is measuring and learning to evolve design and architecture from the start of an initiative.
[The Quick-Start is a technique we often use to kick off new product development initiatives at the FT. It is best described as an internal hack event where the team “lock” themselves in a room together with anyone they need in order to hack together as much as they can in a few days! Then they use what they learnt for making recommendations to how to approach the initiative, designing the solution and identifying the budget and resourcing required if we were to build to production quality!]
What can Product Management do? As a product team we can help taking this thinking to the next level. We can make sure that the larger build – measure – learn cycle includes customers and commercial success and give development teams visibility into this to make our whole organisation focus on the bigger picture and goals.
Retire products and features
Who manages the full product lifecycle? Who can decide when something should be removed? I believe that the answer is Product, but there is definitely a gap here. We are not always perceived as the people who follow up on how products and features are performing. Nor are we perceived to take enough responsibility for getting old or underperforming stuff decommissioned. We are placing emphasis on the Retire state in the new product lifecycle. Still, it won’t happen just by proposing and adopting new processes…
There was plenty of discussion around this:
- When do we retire a product or a feature?
- Should we introduce sustain metrics and set thresholds for when things get turned off?
- Do we explore the concept of system apoptosis where all features have a preset lifetime and a product manager must keep demonstrating its value to keep it alive?
- Can we define the cost of increased software system entropy and quantify it for discussions with business stakeholders?
- How do we get better at designing and building things to be removed in the first place? At the moment complex dependencies makes it difficult and expensive to turn things off! We must allow for a component to die without taking the rest of the system with it to the grave.
What can Product Management do? We should take ownership of this and be responsible for tracking performance. We should have the tough discussions with stakeholders. However, we don’t always have a good understanding of the real cost of having features and products live in production though. Technology stakeholders can help us by highlighting the full cost (like added complexity, lost opportunity cost, time spent on maintenance, technical support, training of new staff, ongoing testing, etc).
A question was raised how the development team get more access to Product to influence the investments made into operations and maintenance. We should not leave that unanswered if we want to be seen as the go to people for product.
Build, partner or buy
A point was made that we might only want to build the few things that are absolutely core to our business and buy or partner for anything else. A challenging area without simple answers, but better collaboration between product and technology would be beneficial.
An important point was made that the cost of breaking away from a vendor must be better understood up front. It feels like we still keep sleepwalking into vendor contracts!
What can Product Management do? Play an active role in build or buy discussions and in the vendor selection process. Make sure that we bring comprehensive and accurate requirements to the table where possible to influence the decision. Perhaps encourage a more agile and iterative approach to trials and vendor cooperation where it is applicable.
The silent bang
In technology and product development we are moving away from big bang launches. Many of our stakeholders still want the PR effects of a “proper launch”.
What can Product Management do? Well, this is probably one for product to figure out and test in the organisation. We already have recent success stories where we have experimented with releasing to production continuously and kicked off PR activities once the time was right.
User story value
Can initiative and user story value be added for the team to see up front? Can we assign value points or even cash values to individual user stories? Or can we at least make the rationales behind priorities clear in terms of how work done by a team is expected to add business value?
What can Product Management do? We should definitely answer the questions above to our development teams. End of the day we need the whole team to understand why we are building something and how it is valuable, not only what we expect to see built. This is just us playing our important part in aligning the organisation to our common goals.
Testing beyond finding bugs
Testers and QA are expanding their focus to quality and value, not just testing the code for bugs. The use of behaviour driven development has been a successful way of moving user acceptance testing efforts up front. Test minded team members are now focusing on thinking about scenarios and examples early and can find “bugs” before any code has been written.
Many questions were raised around testing and the risks with frequent and quick releases to production: Are bugs in production ever acceptable? Can we accept that a paying subscriber is experiencing problems when we test our new features? Successful testing is often described as zero bugs in production. Achieving this is nice for our customers, but it can be expensive and time consuming. Are we willing to trade this for reduced time to market, given that risks are managed? Is this a way forward given that we have sufficient monitoring in place, find problems in production fast and the ability to roll-back or fix issues promptly?
What can Product Management do? Good communication, collaboration and agreements between QA and product is required to make the trade-offs. The relationship between product owner and tester is too often neglected. I personally believe that shifting the tester and QA focus to understand and test for end user quality and value is the way to go, but process and organisation need to support this without the negative impact to our customers.
A large part of the budget allocated to development is actually spent on maintenance and operations. It is required just to keep our head above the water, but does not move us forward. This is down from two years ago, but still very high.
What can Product Management do? To start with, spend the time you need to get a good idea of your product. How much of the investment is actually maintenance and support? Do we fully understand the costs and do we work with technology to prioritise cost cutting stories and initiatives in our backlogs?
I heard many positive comments. Someone thought that one of the reasons for the overall good mood of people at the conference was thanks to improvements made in how development teams interact with product managers and how we build products together. There is a perception that things are moving in the right direction. 🙂