Serverless meetup at the FT

The FT hosted it’s second London Serverless meetup on Wednesday 11th October.  Around 60 people from across London came to hear about Serverless in the FT’s Conference Suites.

What’s serverless? Serverless is the aggregation of third party services (e.g. data stores), including ones that run simple business functions (Functions As A Service, known as FaaS).  AWS Lambda is one of the best known of these.  See; Serverless Architectures from Martin Fowler for more. Although it gets it’s name from not involving servers directly, everything runs on third party servers underneath…

At this meetup Yan Cui, Senior Developer at Space Ape Games spoke about “Lambda stories from the trenches“ where he described some of the problems they have faced and gone on to solve running their Mobile games platform on a Serverless Architecture.  Yan is a prolific blogger, check out some of his posts here; https://hackernoon.com/@theburningmonk

Ant Stanley then did a live demo of the new serverless framework (https://arc.codes/) released recently by Brian Le Roux and his team at Begin. A great framework if you’re focused on using serverless for websites or chat bots. Ant also over-ordered far too much pizza and drinks…

We have another Serverless meetup due on 15th November – please sign up here if you are interested in watching this area evolve.

Constructive sloppiness

Tl;dr For hackathons and things, insecure database solutions can save you a big wodge of time and Chrome extensions are a very versatile tool.

The other week, I took part in the FT’s annual internal hackathon. My team and I decided to play with our idea of bringing video-game-style ‘achievements’ into FT.com, with the aim of encouraging exploration and discovery of the site. It was great fun! Check it out:

poliwag screenshot

I got to try out some cringe-inducingly sloppy but highly effective techniques for quickly building rich prototypes. I’m not sure I should really be proud of them but I am.
Continue reading “Constructive sloppiness”

A Privacy Policy and Terms of Service for Polyfill.io

We’ve just published two new legal documents to the hosted version of the polyfill service, Polyfill.io. If you’re hosting your own version of the polyfill service, these documents don’t affect you – they only apply to people using a version of the polyfill service that we host.

The privacy policy outlines what data we collect about requests to Polyfill.io and what we do with that data.

The terms of service document describes what you can expect from us when you use Polyfill.io on your site, and the actions you, as a user, might take that would cause us to revoke your access to Polyfill.io.

Why have we added these documents?

Adding a privacy policy is best practice for service providers. As users of Polyfill.io it is important that you understand what we do with your data.

As for the Terms of Service, Polyfill.io usage has been climbing since we launched it 3 years ago, and is now used by sites around the world. At the FT we both maintain the open source project and host Polyfill.io for free (and Fastly provides free global caching on their CDN). The Terms of Service help ensure we can keep doing this.

What it means for users of Polyfill.io

If you’re a user of Polyfill.io, you should read and understand the Terms of Service and Privacy Policy. Neither of these documents change anything about Polyfill.io or the FT’s behaviour, they just document it.

Contact us

If you have any questions about these documents, you can reach us through the usual channels: https://twitter.com/polyfillio and https://github.com/Financial-Times/polyfill-service

Marshmallows

So, we’ve all heard of The Marshmallow Test, right? This is where children are tested on their ability to resist one marshmallow on the promise of getting two marshmallows later (a level of self-control that even as an adult I find a challenge!).

But what about the Other Marshmallow Test?

This is a game I conceived to illustrate the benefits of limiting work in progress – given as a talk at Agile in the City.

Cracking the WIP – The Other Marshmallow Test

The initial premise is about efficiency – can we complete our work faster (or consume our marshmallows more quickly) when doing it one piece at a time, or by multi-tasking? In itself it’s an interesting question, and in the game the answer often depends on the individuals who have volunteered and how much they enjoy marshmallows. There is so much more to observe when you try this out though, such as the impact on stress levels, managing risk, delivering value etc. You can download the slides here to get the full story.

The best thing about the game is that it shows just how far the concept of limiting work in progress applies to any work environment – this is not just about software! Upon seeing the game played in a lightning talk, a member of our legal team considered whether this could help them deal with the barrage of requests they get from all directions. An invitation to visit his team swiftly followed.

Kanban in the Legal Team

We played the game, talked about flow, and then I left them with Kate Sullivan’s talk about agile adoption within the legal team at Lonely Planet. A couple of weeks later I strolled by to witness the joy of a stand-up around a kanban board.

We talked about the benefits they were experiencing, as well as some of the challenges remaining. They still have things to improve (we all should continuously improve, after all) but they were finding a lot of blockers removed simply by visualising and verbalising them together. You can read more about how they have decided to apply agile in a post written by John Halton (Assistant GC) for Practical Law.

Learning at FT

One of the things I love about working at the FT is seeing teams from across departments learn from each other. Just as our legal team have learnt about agile from technology; our product and tech teams have learned a lot about how to use KPIs from our commercial teams; our editorial teams think more about reader engagement with help from our analytics teams; and here in engineering we continually exchange new lessons with every department we work with.

Removing the Tester Safety Net

approved.eps

Moving to Continuous Delivery and a Quality Focused Process

We’re all familiar with the waterfall approach of software development.  It keeps skill-sets in silos and, from a tester point of view, we were the ones squeezed for time when projects overran.

Adopting agile in the latest Membership Programme incarnation at the Financial Times many years ago started to make a change.  The concept of starting to break work into smaller pieces and working much closer to one unit as a team removed the big bang approach of these problems.  Ultimately they still existed.  Like most development teams our testers were outnumbered by developers, but ultimately had as much if not more to do.  The introduction of automated testing if anything made matters worse.  When you’re new to agile you can struggle to work out where to build automated tests into the process.  We agreed that they needed to be part of the sprint from day one, but this meant we still had split skill-sets – manual and automated testers.  Both were needed to get the work done. Continue reading “Removing the Tester Safety Net”

Splunk HTTP Event Collector: Direct pipe to Splunk

In August 2016 the FT switched from on-premises Splunk to Splunk Cloud (SaaS). Since then we have seen big improvements in the service:

  1. Searches are faster than ever before
  2. Uptime is near 100%
  3. New features and security updates are deployed frequently

One interesting new feature of Splunk Cloud is called HTTP Event Collector (HEC). HEC is an API that enables applications to send data directly to Splunk without having to rely on intermediate forwarder nodes. Token-based authentication and SSL encryption ensures that communication between peers is secure.

HEC supports raw and JSON formatted event payloads. Using JSON formatted payloads enables to batch multiple events into single JSON document which makes data delivery more efficient as multiple events can be delivered within a single HTTP request.

Time before HEC

Before I dive into technical details let’s look at what motivated us to start looking at HEC.

I’m a member of the Integration Engineering team and I’m currently embedded in Universal Publishing (UP) team. The problem that I was recently asked to investigate relates to log delivery to Splunk Cloud. Logs sent from UP clusters took several hours to appear in Splunk. This caused various issues with Splunk dashboards and alerts, and slowed down troubleshooting process as we didn’t have data instantly available in Splunk.

The following screenshot highlights the issue where event that was logged at 7:45am (see Real Timestamp) appears in Splunk 8 hours and 45 minutes later at 4:30pm (see Splunk Timestamp). Continue reading “Splunk HTTP Event Collector: Direct pipe to Splunk”

Tuning Varnish Cache

The FT recently sent me on a Varnish administration course run by Varnish Software; based just around the corner from our London office.

Varnishing the floor.

It was a brilliant two days of learning all about Varnish cache and the VCL language, making good use of The Varnish Book for course material.

Here are some tips on tuning Varnish cache that we discussed during the course. Continue reading “Tuning Varnish Cache”

The case for accessibility

FT.com for everyone. Always.

At the Financial Times we’ve recently released a new version of our website, FT.com. “Next FT”, as we’ve come to know it, is now the default experience for our users, and so far it’s proving to be a great one: It’s faster, it’s nicer, it’s better; a success across the board [1][2]. Yet there’s an aspect of our new site we have largely overlooked: accessibility (a11y).

The new FT.com
The new FT.com

In this post we will explore what web accessibility is, why it’s important, the current state of accessibility at FT.com and the work we’re doing to improve it.

This will be the first of a series of posts that will document our progress on web accessibility at FT.com. Continue reading “The case for accessibility”

The Year of Lightning

Approximately a year has passed since Salesforce announced the new Lightning experience. And what a year for Salesforce! At first I thought ‘this is going to take a while, there’s going to be a learning curve, probably known bugs to deal with’, we tentatively started switching on the New Lightning Experience to play around with the new User Interface. In a short while we tested some visualforce pages embedded in the new Salesforce application. Finally, this summer we made the leap to building the first Lightning components and Lightning application.

Lightning Components framework is a set of out-of-the-box components build on the open source Aura framework. Developers can utilise Aura to build their own custom components and extend framework. The key here is that Lightning Components are client-side based. Lightning Components Framework has an event driven architecture and relies mostly on Javascript on the client side to manage the UI and application data. Hence it is much better performance wise as opposed to Salesforce classic technologies that rely heavily on the server. You can find more information by visiting these links:

Lightning Components Framework: https://developer.salesforce.com/docs/atlas.en-us.lightning.meta/lightning/intro_framework.htm

Open source Aura Framework: http://documentation.auraframework.org/auradocs

the-flashOne of my favourite series as a child was ‘The Flash’. He could miraculously find himself from his home dressed in pyjamas, down the street in front of a shop window within seconds. When I built my first Lightning app this year, the images from ‘The Flash’ running around with the speed of light immediately came to my mind. Three words: fast, simple, beautiful. No wonder they named it Lightning. Continue reading “The Year of Lightning”