Question Page

How do you define and set SLAs with engineers?

I'm currently struggling to define checkout error rates for our e-commerce platform. We're currently at 1.5%. Personally, I think it's too high. However, I have nothing to substantiate my opinion.
Virgilia Kaur Pruthi (she/her)
Virgilia Kaur Pruthi (she/her)
Expedia Group Senior Director of Product, Head of Trust and Safety | Formerly AmazonJanuary 31

I would recommend first building a relationship with your technical lead/engineering counterpart. Have them show you how your e-commerce platform (or product area) works end to end, from the backend perspective. Make sure that you first understand the end to end flow and specifically the systems design (which is critical in any e-comm platform). Once you understand how the customer's journey equates to the systems design, then start looking into each customer interaction with the site and make sure your team is tracking those metrics. You will end up at the checkout rates. If you have a good pulse on SQL or pulling and analyzing data, you could probably do the error rate comparison on your own. If you don't feel comfortable, work with your engineering lead (or data analyst) to dig into those numbers. Build out a report or dashboard that you can look at a regular basis. This will give you the background to ask or share opinions.

...Read More
727 Views
Mani Fazeli
Mani Fazeli
Shopify Director of ProductDecember 14

Service Level Agreements (SLA) are driven by three factors: (1) industry standard expectations by customers, (2) differentiating your product when marketing, (3) direct correlation with improving KPIs.

For checkout, you'll have uptime as an industry standard, but it's insufficient because subsystems of a checkout can malfunction without the checkout process outright failling. You could consider latency or throughput as market differentiators and would need instrumentation on APIs and client response. With payment failures or shipping calculation failures, you would directly impact conversion rates and trust erosion (hurting repeat buying), which are likely KPIs you care about. So your SLAs need to be a combination of measures that account for all of the above, and your engineering counterparts have to see the evidence that these matter in conjunction.

Of the three types, the one that's most difficult to compare objectively is the third. In your question, you mention 1.5% error rates. You could go on a hunt to find evidence that convinces your engineering counterparts that these are elevated vs. competition, or that they're hurting the business. What's more likely to succeed is running A/B tests that attempt to improve error rates and demonstrating a direct correlation with improving a KPI you care about. That's a more timeboxed exercise, and with evidence, you can change hearts and minds. That's what can lead to more rigorous setting of SLAs and investment in rituals to uphold them.

...Read More
2268 Views
Top Product Management Mentors
Poorvi Shrivastav
Poorvi Shrivastav
Meta Senior Director of Product Management
Anton Kravchenko
Anton Kravchenko
Carta Sr. Director of Product Management
Krishna Panicker
Krishna Panicker
Airbase VP Product
Natalia Baryshnikova
Natalia Baryshnikova
Atlassian Head of Product, Enterprise Agility
Marion Nammack
Marion Nammack
Braze Director of Product Management
Ajay Waghray
Ajay Waghray
Udemy Director of Product Management, Consumer Marketplace
Tom Alterman
Tom Alterman
Notable Head of Product
Lizzy Masotta
Lizzy Masotta
Shopify Senior Product Lead
Boris Logvinsky
Boris Logvinsky
Vanta VP Product
Jacqueline Porter
Jacqueline Porter
GitLab Director of Product Management