FinOps in Action

Alon Arvatz: Hear you saying happy, happy, having a good time. Uh, and it's really great to hear that. I, I can tell you that for me also as an entrepreneur, it's very important to also enjoy the ride, not only grind all day. And I wanna, is that a philosophy of yours? And do you also try to tr like convey this message to your team as well?

Mike Routier: Absolutely. Yes. Uh, how do I think of this? We do this job wherever you're gonna be plugged in. Right? And we're all in this big ship together. If I think about what, what we really do, and at the end of the day, it's not like we're really saving lives. We're, you know, I think of finops, usually people save money.

Yes. But I just want people to have it just be a nature of empowerment. I don't wanna be this idea of, of a control mechanism. I want to empower them to do what they need to do, but also know that they're doing it at a well, well enough pace. If I go back to the principles of finops, usually we think of right sizing and cloud waste management, all those fun things.

And the idea is how do we just help mitigate that at the, at the front end. If I'm no longer of this idea of labeling folks as doing good or bad, but more of here's like a good way of provisioning in the cloud, I work with teams that do proper guard guardrails. Then we don't really think of it that way.

We just now start thinking of this. What's my footprint? Is it scaling at what we expected? Is it, you know, whatever that conversation is, depending on who that persona is, does it feel right? And so I do a lot of feelings, of course, definitely I'm speaking to this, but that's to me at the end of the day and it's like, oh, now I understand why all how this, all these things connects.

And it, it just bridges this gap of this is my role, this is your role. Now we're in like a happy community and how do we figure this out together?

Alon Arvatz: And you mentioned. Working with, uh, engineering or your platform team in the context of, uh, uh, waste reduction as the waste reduction was tagged as the top priority for finops, uh, practitioners by the latest, uh, state of the finops report.

And I would be happy to double down on that. And I'm curious, how deep do you expect yourself as a finops leader to get into the waste reduction process? How, how you engage into mapping the waste. Into driving engineers to take action. How do you look at it?

Mike Routier: I, I

will say we are definitely decentralized when it comes to finops.

It's not a, a mic show, a choice, so at least I try to get outta the way of that. I, I keep saying this idea of empowerment, that's definitely happening, but we definitely have through I would say, technology leadership idea of just good cloud cost consciousness people would want to do well. It's more of how do I do well to make sure I'm leveraging the right reporting tooling to know.

What opportunity am I now leveraging? That's usually what the conversation is. It's not, Hey, you're misbehaving. Here's a hundred load bouncers that haven't been touched in of a month. Please go clean those up. Yeah. Does that happen? Sure. But then I don't want people having to do that as well. I want automation, so how do I do that?

Well, we do chargebacks, so people are responsible from a budgetary perspective. So of course there's an encouragement there. And then if we earn favorability, I track favorability by cost centers and business units, which almost gamifies it because now people want to be. Who's doing the me the who's generating the most favorability versus an original budget or forecast, and it just automatically gets baked in.

That's not really what I'm trying to do, but it definitely is a side effect of what I've, what what's happened. But really who's gonna make the changes? Who's gonna mitigate the costs? It's really down to the engineering level and it's usually, here's what I'm seeing. I believe these belong to you. We have a decent tag, enough gov, taggy governance to know.

I know who the owners are. Here's a list of things that I'm seeing through our tooling of here's opportunity. You know, the needles in the haystack kit. Does this make sense? Do you recognize these? Oh, I sure do. Maybe we stopped. Maybe they stopped. Instances, didn't realize there was a run rate on the volumes can do we still need that infrastructure?

No. Lemme go clean that up. Boom, that's done. And it just is, it's just a check-in, you know, once a month and I'm gonna cut to the end of this. I don't wanna have it be a human being that always is doing that. There's tooling to do automation. I wanna. Automate the operational side of my job out of the way.

I, I just wanna deal on, I just love strategy and the implementation, so how do I automate that nonsense? So that's where I'm really trying to get to. That's the, that's the good state.

Alon Arvatz: Sonal, you talked in the beginning about uh, uh, the engagement with engineers and how to work with them when you come more from the financial side as a business partner and they come from the technical side. And I wonder how you build trust with engineers where you know that they definitely have a better knowledge than you in the technology side.

Sonal Gupta: Yeah. So our, our goal is, uh, or should be to understand their point of view as well, and not just pushing them to save money all the time, because there might be some constraints, uh, that, uh, that they have, you know, in terms of. Application, different applications, you know, different behavior of the applications.

Some applications are latency sensitive, some applications are critical for the business. Some, some are not. But they still need, um, you know, powerful VMs to run those applications simply because, you know, they, they might have some critical batch, uh, job that runs once in a month, but you still need high capacity of vm.

There is no solid reason for them to downs, downsize the vm, uh, to save costs. So I think from that perspective, we need to, all finops practitioner need to understand their mindset as well, right? And not always push them to save money. That's one. And the another one is like talking the, the same language, um, and, and, uh, showing them the valuable data in order for them to take some decisions.

If you do not give them, uh, the meaningful data, they won't be able to, uh, decide what they actually wanna do because they have some requirements from their end that I need my application to be running on MCE vm, for example, sq dive, which is way too expensive. But that has to be the reason, and in GR engineer engineering teams need to justify that to the finops team.

Then ops team also need to understand from their point of views is what I think, uh, what, what will make or what, at least in my experience, that has made, uh, my life a lot easier because, uh, we often talk, um, uh, talk with each other in the same language, same keyword. So, yeah. And

Alon Arvatz: I'm sure that, uh, for engineers, uh, Enos or cost optimization is not always.

The most fun or the most priorit thing, they would definitely prefer to develop new features and they would definitely be more concerned about very big security risks than us. Do you do anything proactively to get the engineers more motivated or to give more priority to finops and cost optimization tasks?

Sonal Gupta: I think, um, what I basically focus on is providing them the data. Um, I have, uh, some great tools that I use, and what I do is like, I prepare like a weekly report or a monthly report that I published and I have, um, a lot of, you know, different way of communicating things. For example, we are, uh, uh, publishing newsletter.

Every month we have this power BI leadership dashboard wherein we actually, um, um, identify who is doing what. And how they are doing it with their different, you know, maturity levels, crawl, walk, and run. So that is, uh, some sort of a, a great way for us to motivate engineers that, you know, this month for example, you have done X, y, z saving and you have these many recommendation to take action on for next month.

And when you will take those actions, you can see, let's say a, B, c savings and then total savings would be. Let's say something, right? So if you provide them that kind of transparency and data visibility, I think that will definitely, uh, help you push them indirectly and not directly, you know? So just make it more fun, make it more exciting, making, make it more engaged.

I think, uh, you know, that's, that's the key.

Alon Arvatz: Absolutely. And you mentioned some, something interesting. You mentioned the work with a power bi. Now Power BI is part of the Microsoft Suite. And earlier you also mentioned that you're using AWS. I'm curious if you make an effort to do consolidated reporting across all of the clouds that you use or you think it's good enough to do reporting for Azure and reporting to AWS and keep it a lot more simple?

Sonal Gupta: Yeah. Um, so I would say that it was definitely a challenge for the, um. Multi-cloud companies, uh, wherein, uh, they, they had two tools to, to begin with all those cloud native tools from Azure and AWS and they were doing separate reportings. So, so, so that's, um, I would say that was one of the challenge. But now with the help of this focus, which is an open source tool, uh, built by finops foundation.

That's a great way to actually deal with that challenge because focus supports all the, uh, cloud platforms and that gives you a unified billing data, uh, or billing overview or understanding. So that's one way to deal with that. But in, in, uh, in Carlsburg, this Power BI reporting is just to bring this leadership dashboard, uh, into carlsburg or into, uh, Carlsburg finops.

To make sure we bring some sort of gamification and competition among stakeholders that, so only for that we are using Power bi, but for, let's say consolidated billing data, we are using a third party tool and that actually syncs with or uses the APIs from these different cloud platforms. So Azure, AWS, these two cloud platforms.

Um, and, uh, you get the, uh, the uh, different rule-based dimensions on that tool and based on that you can actually slice and dice your cost.

Alon Arvatz: Yes. And you also mentioned that you use it for some kind of gamification where each organization or each leader can measure himself. What are APIs that are you trying to expose to them, that you want them to, to measure themselves, themselves against?

Sonal Gupta: So, uh, one of the biggest KPI that we have added in our leadership dashboard is, of course, the, uh, cost optimization percentage. It's called COP, and then the other one is the budget, uh, variance. So these two are the main KPIs wherein we are tracking the performance of different stakeholders and their functions.

Uh, cost, uh, optimization percentage is nothing, but it is just, uh, an understanding of how much you are saving per month. Then the budget is around budget, how you are doing, uh, when it comes to your budget. So these two KPIs to start with. But definitely as we move further, um, in this leadership dashboard maturity, we'll bring more KPIs in you.

Whatever makes sense to to us.

Alon Arvatz: And do you expect engineering to do cost optimization when they're on budget, or you also only expect them to do that when they pass their budget?

Sonal Gupta: I expect them to do cost optimization in every, uh, stage, uh, and not just when they see that they are under the budget or over the budget, because there is no point doing that or focusing on the optimization when you have already done something wrong.

Um, so if you are, uh, you know, if, if, if you understand, uh, where, uh, sort of where you can, uh, optimize your environment. Uh, with the help of the data, of course, that you get from finops, why not? Why to wait for the report. You can do it, uh, immediately, right? So

Alon Arvatz: I agree. I also think it's a matter of culture, so you need to nurture a culture of people.

Understand that they need to be efficient. And even if you're on budget, if you waste money for no good reason, it's money you could have taken, invest in something else and grow even faster. So why waste it for no good reason? So. I totally agree with, uh, this approach, and it's, uh, really cool that you have these two metrics that you track in, uh, parallel.

Amy Ashby: I mean, looking back, you know, a lot of us are, many, many years in, into this, into this space. And I think, you know, probably the, the number one thing, um, was really thinking that, you know, I was going to come to people with data and they were really gonna care about that data and, and run off and do something about it.

And, you know, it's taken a a lot of time to really, you know. Build more of a culture so that people actually do care. So I think that was, that was really sort of eye-opening. Um, you know, when people aren't really used to, um, having that sort of, as part of their responsibility, it, you know, it's just sort of outta sight out of mind and, you know, why should I, why should I care about these numbers?

Alon Arvatz: I think it's a shock to all, everyone, everyone that gets into pheno, the people don't really care. Like in the end of the day, it's not their money. It's not our money. So it's basically a matter of culture. Uh, do you make an effort to get people to care?

Amy Ashby: I do. And I, I really try to, um, sort of position it more like another metric.

Um. That, you know, there's, there's, you know, just kind of even looking at observability for example, um, you know, when you are developing software, observability is just as important, um, to help make sure that, you know, things are performing and, and to help detect issues. And so I sort of treat costs as maybe any other metric.

And, uh, you know, just sort of try to get that integrated into a more day-to-day view and be something that can help aid, you know, development or performance tuning or, um, you know, just be another one of those indicators that we can, we can see maybe early on or in correlation with other metrics. Um, so that's, that's how I try to approach it.

Not so much of. Trying to come with, uh, you know, you increased by this and really no explanation. I try to, um, help people understand where and how the costs incur so that they're thinking ahead and, and then when they start to do changes or, or different things, then they will naturally reach out and say.

Well, I'm going to do this and, and now that I know that there's a cost impact, now I'm gonna come in and ask you, um, to help me figure out how much of an impact is it going to be.

Alon Arvatz: I think that if we're going 15, 20 years ago, so also quality wasn't a metric that engineers would track. It was the QA department, uh, problem.

Uh, do you predict that 10 years from today, engineers would care about cost and it'll be just one more metric that they. Measure and track alongside, uh, quality and performance.

Amy Ashby: Yeah, I do actually. And I, I really, I really do feel like, you know, we're starting to see tools sort of converge different types of metrics together and really trying to, to put that in the path of an engineer.

And I think cost is just another metric. Um, a lot of times that metric is going to correlate. With others and you'll be able to, to see that correlation and um, or maybe it, it's a, a leading indicator, so you're gonna see that maybe ahead of other metrics sort of spiking. So I really do feel like not only will we see that sort of part of normal everyday life, but um, I think we'll maybe see tooling, maybe center around that more.

Um, you know, surfacing that just like leader C and observability tooling today. I totally agree with you. However, let me challenge you. Uh, I met at, uh, finops X, uh, very experienced, well-known, uh, finops leader, and he, he felt like he wanna give up on trying to change the culture of engineers and he wanna just try to automate everything.

Do you believe that it's actually possible to say, okay, I'll just automate everything and we don't need to care about or worry about Enos culture? I would say that I maybe agree partially. I think we can do both. I think the goal should be automation. The goal should be to, I take as much action as possible without human interaction for some self-healing.

Um, you know, or safe. Changes. But I think that, especially when it comes to forecasting, I don't, I don't think we can really predict without inputs from the business. Um, you know, so I think, you know, tactically, resource wise, based on usage metrics, we can automate a lot. But I don't think we can automate all facets of, of what we do in a finops practice.

So we could. We could, you know, we still need sort of that extra input from the business and teams as to, you know, forward looking and how are we gonna forecast and how are we going to anticipate that, you know, an increase, uh, is not really an anomaly, but it's intentional. Um, so I think there are some things we can automate and some things that will continue to be a practice.

Polly Fabry: It's really, it's putting the responsibility and the accountability on the teams that are spending the money. Right? I mean, it was, that was really what it took to get people to pay attention.

And then once they are paying attention, then the training. Pr, Sean's leadership and the constant reinforcement and the tools and all of the other things that we give them in order to manage these budgets, they're, they're actively engaged with it now because they've got their skin in the game.

Alon Arvatz: So would you say that your secret sauce is actually to do it, uh, top down rather than bottom up?

Polly Fabry: Absolutely. It does not work bottom up. It's, it's very, very hard. It's gotta come as a mandate from the top. Especially because remember, we're, we're in a completely separate organization, so me trying to tell someone in another team how to do their job or what's important or what their priority is, doesn't go over quite as well as when it comes from their leadership team.

Alon Arvatz: And can you add a bit more color to that? What does it mean? Uh, top down, because I know that many finops practitioners actually struggle with that. How to get the executive team engaged, how to get leadership involved. How did you do it? Did you start from the CTO? The CIO? You started from the,

Polly Fabry: we went to the CIDO.

We went right to the tippy top, right? So we went to Tom Peck, who is our CIDO. We kind of brought our case to him. Hey, cloud spend is out of control. Your leadership team doesn't understand their spend. They don't see it at all. So they don't necessarily, I don't wanna say they don't care, but they don't have an incentive or a motivation to care about it.

So we really need them to care. So again, we started with Showback. We started with monthly reports, with meeting with the, with the TLT leadership team so they could see it. And then we, at, at the end of FY 24, we went back and said, Hey listen, we're gonna transfer budget. You guys are gonna own this. You're gonna manage it.

And it was, it came from our CIDO. So there wasn't really any kind of an argument or pushback. And I think, honestly, I think. Now that we're kind of nine, 10 months into it, they're happier that they, you know, they feel like they have a little more control over it, and they have that visibility because the thing is, is when our budgets are off and we have to take from other projects and things like that, it does ultimately impact the entire organization.

So this is a, this is really a great way to help them be accountable for all of our IT spend and it, it does give them the visibility they need.

Alon Arvatz: That's actually a fascinating, I'm also fascinated by the transition it is from show showback to chargeback. So I wonder if that was like an word decision. To start with show because it's either easier and eventually move to chargeback, or you just, you know, you said

Polly Fabry: you gotta start somewhere and it's kind of like spoonfeeding them, right?

They need to understand like, this is what y'all are spending, this is what it looks like. Prasshant did a really good job of meeting one-on-one every month to show these guys, this is what you're spending. Hey, here's opportunities for you guys to save. Did you know that you should get a reserved instance and you can save X amount of dollars or.

You should be in the savings plan. So there's all kinds of, you know, little nuggets that we can feed to them that if they're not seeing that showback, they don't really see the benefit of it. And then once we did the chargeback, those optimization efforts are even more well received because now they've got a vested interest in trying to, to bring those costs down and more in line.

Alon Arvatz: And Prasshant, how did it go meeting with them one by one? It sounds like a pretty tedious work.

Prasshant Sharrma: No, it was a fun, uh, in fact, I. I really, you know, take a passion to talk to people. Uh, initially I didn't get time in their calendars, uh, but whenever I met with them in the lift, you know, I said hello and introduced, uh, myself to them.

And then, then the next meeting, you know, I tried to secure some time with them. So like two after, through two, three attempts, uh, it really, you know, started going in the right direction. As a result of which I made sure that, uh, we should have a customized focus for each of the business tower. And as a result of which, as I mentioned, uh, earlier, we had a monthly cadence.

We did, we, we actually, you know, had set up a process of interviews, so physically going to talk to the people. Whenever we are in office, we make sure I'm actually meeting with the top leaders every day and learning about their challenges and their business goals. So what happened, what evolved as a result of this?

We started a project last year called As Project He Helios. Uh, we identified top five to seven optimization levers. For example, reserved instances, savings plan, ideal databases, auto parking, spot instances, S3 optimization. And we did some kind of categorization. You know, these are our quick wins and if you can help to drive these initiatives, the low hanging fruits.

We can definitely, you know, tie in these results in your cloud bills. So this is how, uh, we started. We had a clear execution plan, what we are going to do, and when the certain things were coming from, flowing from the top, uh, leadership than engineers has started taking action.

Polly Fabry: It's more of a cultural change than anything, right?

I mean, the, the knowledge is there. It's just getting people to use it and to leverage it. And one thing I will say, I mean. It is a lot of work for Prasshant to meet with all these leaders, but building those relationships has been key. And I, I'll give you a good example. Up until, um, Prasshant personally met with our CISO who did not wanna participate in any kind of, uh, gamification or optimization or any of this stuff, he was very, like standoffish, didn't come to anything.

His leadership team did not come to anything. He was in our headquarters here, and Prasshant had an opportunity, I'm not gonna say you cornered him in his office, but he had an opportunity to talk to him. And I will tell you, it's been, it's been a significant turnaround in the engagement we've had from that organization.

So cybersecurity kind of tends to say, everything is a mandate. Do what I do, do what I tell you to do and leave me alone. And so this was a way for them to kind of come to us. They actually updated their forecast and they're participating in reviews. I, I credit all of that to Prasshant actually having a conversation with our CISO and, and, and really kind of convincing him that, Hey, this is the right thing to do.

This is what we wanna do. This works out well for all of us. The more financially viable and and budget we have within our CHDO spend, that benefits all of our organization. Right? Those are projects we can do. That's the innovation work that we wanna do. Um, it gives us the freedom to hire the people we need to hire.

So it, it really does help everybody. And I think getting folks to see that has been a, an amazing thing to watch.

Do you also feel responsible to also provide or find the tooling for engineers to enable them to do everything you're talking about?

Yes. We have few tools. Uh, we have our homegrown tool over here at Sysco and that tool is really effective. We do some kind of gamification for all the teams so that also, you know, everyone on their tool.

And it is actually a very good tool and we are very proud what we built in-house.

Alon Arvatz: And is there also all of fame?

Polly Fabry: There is no, I I, one of the things I will say is our CIDO has been, um, he's been really good about supporting, we try to gamify, right? So our, our technology leadership team, these guys are all insanely competitive with one another.

So no one wants to be like the last horse in the race. So we, uh. We gamify this so that we make it kind of fun, but it's also a competition and it, again, these guys are all very competitive with one another. Nobody wants to be in last place, so it does help push them.

Alon Arvatz: And are there any rewards or ceremonies?

How do you gameify it?

Polly Fabry: Well, someone suggested we get like a light up cloud or something. We haven't yet. But I think that would be cute, right? We get a, a lineup cloud and whoever wins for the ones gets to keep it on their desk or in their office. So,

Prasshant Sharrma: but our CIDO Tom Pack, he highlights this, uh, this is called as Cloud core, and he highlights which team, uh, which team actually won the competition.

Uh, we'll be doing specifically some fin hacks, uh, not this year, but next, next year. So we will see, you know, if the team really understands the finops enablement. The framework, the cultural environment, you know, for, for finops.