Pilot Uses "Rocket Science" to Boost Profits
Ken Parent, senior vice president of operations for 310-store Pilot Travel Centers LLC, was taken aback. After 28 years in the petroleum, retail and fast food industries -- 14 of which were with Pilot -- he thought he had a pretty good handle on what types of c-store marketing, merchandising and advertising strategies worked.
Hanging the best-selling candy bars from inside cooler doors, for instance, seemed like a no-brainer. Adding a display point to satisfy customers who like their chocolate cold, and cross merchandising with other best sellers surely led to incremental sales, right?
Enter multivariable testing (MVT), a statistics-based methodology for comparing old and new strategies, and analyzing the results, which provides Pilot with a way to test many different tactics simultaneously. Surprisingly, it turns out the well-used merchandising program of hanging candy bars inside the cooler actually hurt Pilot's store sales and gross profit dollars.
"At the end of the day, it didn't help us one iota," Parent said. "It actually had a slightly negative effect on total sales of our major categories -- and now I have the data to prove it. One Ken Parent belief struck down."
The candy display tactic was just one surprise hit taken by Parent and the team at Pilot Travel Centers as they began using MVT in fall 2008. Unlike traditional c-store testing methods, whereby a number of test stores try out a new program or tactic over a few months, followed by management studying the results before moving on to another variable, MVT provides solid evidence of the success or failure of individual variables -- such as adding product SKUs at the expense of another category's SKUs -- as well as that variable's effect on total store performance, either alone or in combination of dozens of other variables tested at the same time.
Under traditional methods, for example, a retailer may offer $1 off a sandwich and find the promotion increases sales of that sandwich 5 percent. Later, it may find new signage causes a 3-percent uptick in sandwich sales. The retailer won't know, however, what a combination of the two variables will do to sandwich sales -- or what that coupon and signage programs mean for other store categories. MVT allows the retailer to test both variables at once and analyze how much each variable bumps sales, how much the two in tandem increase sales, and their impact on total store sales. Thirty variables may be tested at one time with reliable results analysis in 30 to 60 days --- all thanks to the statistical model that's far beyond most retailers' -- or trade journalists' -- comprehension.
"You know how people say retailing isn't rocket science? In this case, it actually is," said Art Hammer, vice president of QualPro Inc., the firm that set up and analyzes the results of MVT testing for Pilot Travel Centers. (Hammer worked on nuclear weapons programs during the Cold War.) "But the retailer doesn't have to worry about that because we take care of the math. The retail team is free to be as creative as they want to be with the programs they want us to test."
QualPro, which shares a Knoxville, Tenn., home base with Pilot Travel Centers and whose 1,000-company client list includes Meijer, Staples, ExxonMobil and Bridgestone Americas, studies a retailer's historical data, helps brainstorm variables to be tested and rolls out the tests in a statistically valid 30-some stores. It then analyzes the data and presents detailed results to the management team. Fees are based on the complexity of the variables tested and amount of work involved.
Pilot used MVT to test variables in five categories -- coffee, candy, Frito-Lay products, wholesaler-supplied chips and snacks, and cooler drinks -- which altogether account for 25 percent of the chain's sales and 38 percent of gross profit dollars. More than a year into using the methodology, Parent found only 25 percent of the programs and tactics Pilot tested brought about sales and gross profit dollar increases. Another 51 percent of its programs had no benefit; the remaining 24 percent were actually detrimental to sales and gross profit dollars. These results were nearly identical to those of other QualPro clients.
After implementing 10 strategies shown to be effective via MVT, Pilot has seen a 6.5-percent jump in inside store sales -- resulting in an increase of approximately $8 million in gross profit dollars -- at a time when the retail industry experienced 5 percent to 10 percent sales declines. What's more, the team at Pilot now looks at the business in a new way -- they've uncovered inherent flaws in the traditional manage-by-category philosophy most retailers utilize.
"We had been using item-level sales data, and we still do, but using it had been intuitive, and we ended up honing in on one factor or one test at a time," Parent said. "MVT helped us look at various ideas we had for a particular product category, like candy or snacks, and measure the impact of those ideas on the whole category, as well as the whole store. You could find a chip promotion, for example, helped chip sales but adversely affected the entire store's gross profit dollars. There were so many findings that were counter-intuitive."
The largest retailer of diesel fuel in the Unites States has done a good job implementing new efficiency measures, technologies and labor initiatives, Parent said, "but the bulk of those things have occurred and the low-hanging fruit is gone. We needed to play more offense, and that's why we embarked on the MVT process."
One desirable trait of the MVT methodology is it allows the Pilot team to factor out gasoline and diesel traffic when measuring other variables. "This is key because you may have a particular store that is up or down on gasoline or diesel sales and MVT allows you to see beyond that," Parent said. "With the right product, marketing and merchandising activities, the amount of fuel you are selling may have no impact on the success of those activities."
With coffee a key growth category for the retailer, Pilot's team wanted to test the effectiveness of an employee incentive program that awarded each cashier 10 cents for every cup sold. "Doesn't work," Parent said of the effort. "Now who would have thought that wouldn't work? Nobody. It turns out direct rewards given to employees on a near real-time basis if they sell more didn't work. We tested this twice because I didn't believe it. There were so many situations that a retailer will typically go headstrong into that MVT shows don't work."
In another coffee-related test, Pilot typically promoted hot beverages and encouraged store traffic by a "Free on Fridays" campaign, such as giving away a newspaper -- which retails for 50 cents and has a 5-percent margin -- with any size coffee. "The results were abysmal. The promotion had no impact on driving coffee sales."
After implementing some MVT-proven winning strategies in the coffee category -- including improving awareness through new signage, a new cashier plus-selling script and giving away a cinnamon roll with a purchase on Wednesdays, the chain's biggest diesel-sales day of the week -- Pilot saw a category sales increase of more than 4 percent. "Our [rate of] coffee sold per 100 customers keeps going up," Parent said.
Other tactics that scored well under MVT testing include offering bagged confections at two for $2 and reconfiguring the cooler to add bottled water SKUs, upping the total to three full cooler doors.
Parent and his team also used MVT to rethink the meat snacks segment. Doubling the store's 4 linear feet of meat snacks yielded more than $600 per month per store. "Where in the past I may have thought $600 per month doesn't seem like a lot, when you consider these items have a 50-percent margin, and we have 310 stores, it adds up quickly," Parent said, noting MVT made it possible to analyze what a 22-percent increase in meat snack sales meant to other categories' performance as products were reduced in other areas.
In other ways, MVT also helped Pilot trim costs. The travel center operator traditionally places signage at the pumps, changing the message every 30 or 60 days.
"We found that strategy didn't really help us and in some cases hurt us. We should be taking a much more simplistic approach, sticking the message in a few places and not constantly spending the money on changing out fancy point-of-purchase materials with new messages," he said.
"We all have many thoughts and ideas about signage and messages, but ultimately we found much of the signage we thought would be very effective didn't have the impact we thought it would." He added: "We have been able to greatly reduce our monthly advertising budget with that discovery."
Similarly, Pilot tested the power of its billboard strategy, its primary advertising vehicle, by looking at the number of billboards it uses, their placement and the message on them. One variable: whether or not costly LED price displays were effective, and if so, at what locations? "We found we could reduce our $5 million annual billboard budget by more than 15 percent and be more effective with our messaging than we were previously," Parent said.
The most recent MVT testing is related to inventory control and the loyalty program in the chain's $275 million restaurant business. One strategy being tested now is a happy-hour beverage promotion. QualPro also is working with the chain to test the impact of store resets on new products and overall store sales.
The entire process has greatly altered the way category managers look at the business, Parent said. Historically, category managers were compensated on their category's performance. When planogramming and adding new items, managers considered items individually in one product category. MVT has broken down some barriers between categories.
"We found, for instance, a candy promotion helped water sales," he noted. "MVT gives us details like that, which the candy and water category managers would not necessarily come up with. Now, everyone is thinking more in terms of the effect of one action on the entire store's results. A category manager's bonuses are tied not just to their individual categories, but to the entire retail effort winning. When we get into debates about space allocation and new ideas and initiatives, we get consensus around the MVT data in 30 or 60 days and go with the best option -- and everyone wins."
MVT involves tremendous coordination between category managers and operators, he added. "With many factors tested at one time in many different stores, they need to work very closely to ensure each store is configured just right."
As in any c-store company, Pilot had team members who firmly believed in specific programs or tactics as tried-and-true strategies. "MVT showed us those opinions don't always hold water," he said. "The challenge now is to not get caught up in the 'whys' behind the results. Just know the data doesn't lie and it is statistically proven. Sometimes you may not understand what is behind the results. Sometimes good ideas just don't work -- or don't work for us, our sites and our customers."
Hanging the best-selling candy bars from inside cooler doors, for instance, seemed like a no-brainer. Adding a display point to satisfy customers who like their chocolate cold, and cross merchandising with other best sellers surely led to incremental sales, right?
Enter multivariable testing (MVT), a statistics-based methodology for comparing old and new strategies, and analyzing the results, which provides Pilot with a way to test many different tactics simultaneously. Surprisingly, it turns out the well-used merchandising program of hanging candy bars inside the cooler actually hurt Pilot's store sales and gross profit dollars.
"At the end of the day, it didn't help us one iota," Parent said. "It actually had a slightly negative effect on total sales of our major categories -- and now I have the data to prove it. One Ken Parent belief struck down."
The candy display tactic was just one surprise hit taken by Parent and the team at Pilot Travel Centers as they began using MVT in fall 2008. Unlike traditional c-store testing methods, whereby a number of test stores try out a new program or tactic over a few months, followed by management studying the results before moving on to another variable, MVT provides solid evidence of the success or failure of individual variables -- such as adding product SKUs at the expense of another category's SKUs -- as well as that variable's effect on total store performance, either alone or in combination of dozens of other variables tested at the same time.
Under traditional methods, for example, a retailer may offer $1 off a sandwich and find the promotion increases sales of that sandwich 5 percent. Later, it may find new signage causes a 3-percent uptick in sandwich sales. The retailer won't know, however, what a combination of the two variables will do to sandwich sales -- or what that coupon and signage programs mean for other store categories. MVT allows the retailer to test both variables at once and analyze how much each variable bumps sales, how much the two in tandem increase sales, and their impact on total store sales. Thirty variables may be tested at one time with reliable results analysis in 30 to 60 days --- all thanks to the statistical model that's far beyond most retailers' -- or trade journalists' -- comprehension.
"You know how people say retailing isn't rocket science? In this case, it actually is," said Art Hammer, vice president of QualPro Inc., the firm that set up and analyzes the results of MVT testing for Pilot Travel Centers. (Hammer worked on nuclear weapons programs during the Cold War.) "But the retailer doesn't have to worry about that because we take care of the math. The retail team is free to be as creative as they want to be with the programs they want us to test."
QualPro, which shares a Knoxville, Tenn., home base with Pilot Travel Centers and whose 1,000-company client list includes Meijer, Staples, ExxonMobil and Bridgestone Americas, studies a retailer's historical data, helps brainstorm variables to be tested and rolls out the tests in a statistically valid 30-some stores. It then analyzes the data and presents detailed results to the management team. Fees are based on the complexity of the variables tested and amount of work involved.
Pilot used MVT to test variables in five categories -- coffee, candy, Frito-Lay products, wholesaler-supplied chips and snacks, and cooler drinks -- which altogether account for 25 percent of the chain's sales and 38 percent of gross profit dollars. More than a year into using the methodology, Parent found only 25 percent of the programs and tactics Pilot tested brought about sales and gross profit dollar increases. Another 51 percent of its programs had no benefit; the remaining 24 percent were actually detrimental to sales and gross profit dollars. These results were nearly identical to those of other QualPro clients.
After implementing 10 strategies shown to be effective via MVT, Pilot has seen a 6.5-percent jump in inside store sales -- resulting in an increase of approximately $8 million in gross profit dollars -- at a time when the retail industry experienced 5 percent to 10 percent sales declines. What's more, the team at Pilot now looks at the business in a new way -- they've uncovered inherent flaws in the traditional manage-by-category philosophy most retailers utilize.
"We had been using item-level sales data, and we still do, but using it had been intuitive, and we ended up honing in on one factor or one test at a time," Parent said. "MVT helped us look at various ideas we had for a particular product category, like candy or snacks, and measure the impact of those ideas on the whole category, as well as the whole store. You could find a chip promotion, for example, helped chip sales but adversely affected the entire store's gross profit dollars. There were so many findings that were counter-intuitive."
The largest retailer of diesel fuel in the Unites States has done a good job implementing new efficiency measures, technologies and labor initiatives, Parent said, "but the bulk of those things have occurred and the low-hanging fruit is gone. We needed to play more offense, and that's why we embarked on the MVT process."
One desirable trait of the MVT methodology is it allows the Pilot team to factor out gasoline and diesel traffic when measuring other variables. "This is key because you may have a particular store that is up or down on gasoline or diesel sales and MVT allows you to see beyond that," Parent said. "With the right product, marketing and merchandising activities, the amount of fuel you are selling may have no impact on the success of those activities."
With coffee a key growth category for the retailer, Pilot's team wanted to test the effectiveness of an employee incentive program that awarded each cashier 10 cents for every cup sold. "Doesn't work," Parent said of the effort. "Now who would have thought that wouldn't work? Nobody. It turns out direct rewards given to employees on a near real-time basis if they sell more didn't work. We tested this twice because I didn't believe it. There were so many situations that a retailer will typically go headstrong into that MVT shows don't work."
In another coffee-related test, Pilot typically promoted hot beverages and encouraged store traffic by a "Free on Fridays" campaign, such as giving away a newspaper -- which retails for 50 cents and has a 5-percent margin -- with any size coffee. "The results were abysmal. The promotion had no impact on driving coffee sales."
After implementing some MVT-proven winning strategies in the coffee category -- including improving awareness through new signage, a new cashier plus-selling script and giving away a cinnamon roll with a purchase on Wednesdays, the chain's biggest diesel-sales day of the week -- Pilot saw a category sales increase of more than 4 percent. "Our [rate of] coffee sold per 100 customers keeps going up," Parent said.
Other tactics that scored well under MVT testing include offering bagged confections at two for $2 and reconfiguring the cooler to add bottled water SKUs, upping the total to three full cooler doors.
Parent and his team also used MVT to rethink the meat snacks segment. Doubling the store's 4 linear feet of meat snacks yielded more than $600 per month per store. "Where in the past I may have thought $600 per month doesn't seem like a lot, when you consider these items have a 50-percent margin, and we have 310 stores, it adds up quickly," Parent said, noting MVT made it possible to analyze what a 22-percent increase in meat snack sales meant to other categories' performance as products were reduced in other areas.
In other ways, MVT also helped Pilot trim costs. The travel center operator traditionally places signage at the pumps, changing the message every 30 or 60 days.
"We found that strategy didn't really help us and in some cases hurt us. We should be taking a much more simplistic approach, sticking the message in a few places and not constantly spending the money on changing out fancy point-of-purchase materials with new messages," he said.
"We all have many thoughts and ideas about signage and messages, but ultimately we found much of the signage we thought would be very effective didn't have the impact we thought it would." He added: "We have been able to greatly reduce our monthly advertising budget with that discovery."
Similarly, Pilot tested the power of its billboard strategy, its primary advertising vehicle, by looking at the number of billboards it uses, their placement and the message on them. One variable: whether or not costly LED price displays were effective, and if so, at what locations? "We found we could reduce our $5 million annual billboard budget by more than 15 percent and be more effective with our messaging than we were previously," Parent said.
The most recent MVT testing is related to inventory control and the loyalty program in the chain's $275 million restaurant business. One strategy being tested now is a happy-hour beverage promotion. QualPro also is working with the chain to test the impact of store resets on new products and overall store sales.
The entire process has greatly altered the way category managers look at the business, Parent said. Historically, category managers were compensated on their category's performance. When planogramming and adding new items, managers considered items individually in one product category. MVT has broken down some barriers between categories.
"We found, for instance, a candy promotion helped water sales," he noted. "MVT gives us details like that, which the candy and water category managers would not necessarily come up with. Now, everyone is thinking more in terms of the effect of one action on the entire store's results. A category manager's bonuses are tied not just to their individual categories, but to the entire retail effort winning. When we get into debates about space allocation and new ideas and initiatives, we get consensus around the MVT data in 30 or 60 days and go with the best option -- and everyone wins."
MVT involves tremendous coordination between category managers and operators, he added. "With many factors tested at one time in many different stores, they need to work very closely to ensure each store is configured just right."
As in any c-store company, Pilot had team members who firmly believed in specific programs or tactics as tried-and-true strategies. "MVT showed us those opinions don't always hold water," he said. "The challenge now is to not get caught up in the 'whys' behind the results. Just know the data doesn't lie and it is statistically proven. Sometimes you may not understand what is behind the results. Sometimes good ideas just don't work -- or don't work for us, our sites and our customers."