Boosting
fulfillment capacity
through incentive
ROLE
outcome
team
Josh Griffin | Frontend engineer
Nathan Cozlowski | Data science
Samantha Willis | Ops
Shereen Idani | Product
Sangwoo Kim | Design
With growing oders came the logistical challenge. In order to meet the surge in demand, we needed to get more shoppers to fulfill the orders but weren’t able to do so with tools at hand. As a delivery company, this was critical in that it not only exacerbated our profitability but eroded customers’ trust in us.
Losing profitability
Unfulfilled orders basically mean all orders that are not claimed by shoppers. Our first line of attack when orders don’t get claimed is deployment of promo pay per order to make them more enticing to shoppers. As a result, more unfulfilled orders lead to more promo pay to spare. Especially if that kind of expenditure happens nationwide, it’s obvious that it hurts our profitability significantly.
Erosion of trust
Our customers come to Shipt in a belief that they’ll be able to receive what they need at a given time. If we fail to deliver their orders at the promised time, that erodes trust customers put in us. Lost trust leads to user churn, which is the last thing any business wants.
When we looked at the shopper data, we saw that we had quite a large number of inactive shoppers and the average number of orders shoppers claim per day among active shoppers was lower than desired level. Such finding suggested that there’s still a huge potential within existing shopper pool we can tap in. So our cross functional team decided to deploy incentive program in an attempt to galvanize shoppers.
Setting users up for success
Overview of an incentive requirements and in-app support is called out upon opening shopper app in order to set shoppers up for success from the get go. Illustration adds a bit more visual interest in an otherwise text-heavy screen. I created the illustration myself.
Visualizing Progress
Keeping track of the progress towards incentive is on us. Every time shoppers complete an order that meets the incentive condition, we count and show it in places where it matters.
Keeping users in the loop
Real-time feedback is provided as shoppers complete qualifying orders so they stay informed without extra effort. In addition, shoppers can see which orders were counted and which weren’t along with reasons, because without them they might question as to why their work wasn’t honored.
real-time
regarding incentive progress
Celebrating the completion
Once shoppers complete the last order, we acknowledge the completion and create celebratory moment. Most importantly, shoppers will be briefed on the amount of incentive and when it will arrive.
Our primary goal was getting more orders claimed by incentivizing shoppers with an opportunity to earn bonus. Since a wide array of factors could impact order fulfillment, we needed to be specific about what the success would look like in this project’s context. We came up with 4 sucess metrics.
1. Decrease order reschedule rate
2. Decrease promo pay
3. Increase incentive earners
4. Increase active shoppers
Before jumping into design, I needed to have a big picture of everything that happens from when the incentives ared deployed to the point when shoppers get paid. In doing this, I was able to identify key touch points and area of opportunity.
ideal tracker
In order to optimize tracking experience, we had to consider its potential impact and integration with other parts of shopper experience. From business standpoint, the tracker needed to be impactful enough so that it motivates shoppers to do more orders. From user’s perspective, they need to be able to track progress, while not being distracted from performing primary tasks. With these in mind, I explored a few different tracker concepts, each of which went through feedback session with stakeholders.
❌️ Context is too minimal in order for it to be impactful in terms of incentivizing users to engage more with orders.
❌️ Pushes down important contents below the fold. The weight given to the tracker conflicts with the primary purpose of the page. Positioning the tracker and filter bar in the scrolled view can be tricky.
❌️ Its position conflicts with scrolling, which is single most performed action on the page. This can pose usability challenge.
❌️ Thumb reachability might not be ideal for those who use large phones. Encroaching status bar safe area as a fixed element can be a risky choice.
❌️ The whole section can be easily collapsed, which then defeats the purpose of increasing visibility. Takes up way too much screen real estate.
❌️ Treatment is too light. There is no sense of progression. And as a result it doesn’t serve as an effective motivator. Positive feedback cannot be achieved.
❌️ It’s not clear exactly how much progress you’ve made due to lack of numbers. Also its position conflicts with tap area for expand/collapse.
❌️ It gets in the way of scrolling direclty. Not ideal in terms of usability. Too easy dismissal of the information makes it hard for users to stay motivated.
Besides visual polish and information layout, we needed to think about how the tracker component would work across different locations as it pertains to both order and pay. I decided to land with the version(B) because that strikes the balance between order and pay information the best, which means this treatment would be understood in both order and pay context.
Incentive program might come across a bit overwhelming at a first glance. Shopping a large number of orders, all on time during a short span of time is not an easy task. I figured illustration could soften the tone while also delighting the experience at key moments such as the start and end. We didn’t have brand illustrations at the moment, so I decided to create it myself.
After implementation, we decided to run alpha testing with a set group of users in order to uncover any unfroseen aspects and see that the feature functions as intended in the real environment. Field testing was necessary because it’s an experience that was designed to last for a period of time with financial component that cannot be simulated in a moderated setting. Testing revealed couple of things.
1. Participants perceived this feature as an efficiency booster
2. Participants thought the tracking capability makes their work much easier
3. Participants appreciate the ability to see orders meeting qualification criteria and dynamic counting.
4. Tracker increases motivation to attend to incentives and ultimately drive order claiming (caveat: this is self-reported)
5. There was a strong desire for reducing clutter in the UI.
6. Participants weren’t sure of trackers’ ability to discern qualifying orders from the unqualifying ones
7. Participants wanted to be recognized in a more celebratory fashion upon completing the incentive.
“I love it! This took all the work out or tracking to ensure you get a bonus. I loved clicking on the other page and it tells me what orders qualified.”
This step of alpha test validated our hypothesis for the most part but also shed a light on areas to improve upon. Based on these insights, I went on iterating the design to address those needs.
Reducing clutter down to minimum
We assumed that shoppers would prefer immediacy of information, because it requires less effort to find that. But for incentive tracking, that wasn’t the case. Their primary focus was still browsing orders that fit to their capability. Therefore, tracker needed take a back seat from their point of view.
“Honestly as law breaking as most of us are doing this job while incessantly looking at our phones and driving... We don’t need too much screen crowding.”
Boosting credibility of the tracker
We observed critical issue related to participants’ uncertainty in our trackers’ ability to detect disqualifying orders. Our hypothesis was that it’s better to keep things simple by showing only qualified orders. We were wrong. In the case of incentive tracking, more transparency was better UX as it lends more credibility.
“I do have a question: is the tracker smart? For example, if I was late on an order, does it know to exclude it from the count?.”
To gain their full trust, we changed our qualification history to display both qualified AND unqualified orders along with reason code (why certain orders were considered unqualified).
Recognizing celebratory moment
Majority of shoppers who participated in the alpha testing all said this: confetti, confetti, confetti. The underlying motive was that they wanted to be celebrated once they cross the finish line. Given the time and effort it takes to complete an incentive, I could empathize with this sentiment.
“I so wanted confetti to sprinkle down in my app when I completed. If my worked my tail off to get a bigger incentive, confetti would be nice.”
Although all stakeholders agreed that implementing confetti animation would close the experience loop nicely, we didn’t have time to invest in creating custom animation and QA’ing it. Since it was not functional component of incentive tracking, we decided to parking lot the idea for the later phase. Instead, we adopted bottom sheet with more celebratory illustration than a lightweight toast notification.
On July 31st, 2020, we rolled out incentive tracker to select metros. When reviewing our key metrics at the two groups (one with tracker and the other without tracker), we saw all the KPIs moving in the right direction. Tracker proved its efficacy only within 10 days after the shipment.
1. Major operational benchmarks improved as we saw reschedules dropping 8%, going from 16% to 8% and lates decreasing by 1%.
2. Per-order margin went up as promo pay decreased 22%, which in dollar amounted to spending $34K less.
3. 17% more shoppers earned bonus pay and the number of active shoppers increased 5%.
These numbers as a whole meant improving our operational efficiency in almost all aspects. But the impact had more implications than just the numbers. More shoppers benefitied from the bonus pay, more customers got their orders delivered on time, all of which contribute to trust in our brand. It helped our business so much so quickly that incentive became our nationwide campaign to ramp up our fulfillment capacity.
Product can inspire new ways of working
At the beginning stage of this project, I came to know that incentive programs had been used by Ops team in the past without in-app integration. The way they did it was purely manual, from communications to tracking shoppers’ record. This was possible only because it was targeted towards a small group of users and needed to happen sporadically. So there wasn’t enough case to be made for investing in this area. But when external forces like pandemic changes the landscape completely, you have to reevaluate past practices and come up with new ways of working. That’s when product team got involved into the conversation. We saw the massive opportunity in automating the process by investing in new technology and product integration. In partnernship with Eng and Ops, we proposed to build a new backend service and product features, reimagining how we work. Getting buy-in from leadership for this kind of hefty investment, especially in an urgent time like covid wasn’t easy. But product team’s grand vision of how the upstream investment could bring about efficient internal workflow, enhancing business intelligence, maximizing the impact of incentives, and ultimately saving a ton of operational cost resonated with company leaders. Fast forward to 2021, incentive tracking became one of the most successful projects that came out of the 2020 and already got an endorsement for building the next version. I take pride in the fact that I was part of the initiative that helped the company make an impact-driven decision, departing from its conventional speed-oriented mentality.
Measuring success should involve users
Looking back, one thing I wish I would’ve done differently is including user satisfaction as a part of measuring success. Because this project was derived from the business problem, the focus was on the business metrics. The emphasis around those numbers overshadowed user side stories. As a person whose job is to champion his users, I think I should’ve voiced the need to include users’ voice. This project may have solved imminent business problems but the bottom line is that all the decisions we made along the way influenced users’ product experience. But we are blinded when it comes to user experience. The art of design is to balance business needs and user needs. I’ve learned that such principle should be extended to measuring, not just limited in the design phase, because there will always be a room for improvement and without proper user feedback be it a qualitative or quantitative one, we don’t know where to improve upon.