Itamar Gilad: the GIST of product discovery

Andrew Skotzko [00:01:02]:
Itamar, my friend, it's so good to see you. Thank you for being here today.

Andrew Skotzko [00:01:10]:
How are you?

Itamar Gilad [00:01:12]:
I'm good. I'm good. It's great to be here. Thank you for inviting me.

Andrew Skotzko [00:01:16]:
Oh, absolutely. I mean, I said this to you on our last conversation. We're not recording, but I mean it. In all seriousness, I've given your book. I think I've gifted your book more than any other book in the last, like, three years. Obviously, your book hasn't been out for three years. That means it really came out of the gate strong in my world. So I'm just excited to finally get you here on the show because I have literally gifted your book to, like, I think every person I've worked with and a few other people.

Andrew Skotzko [00:01:42]:
So it's just great to have you, and thanks for writing a hell of a book.

Itamar Gilad [00:01:45]:
So this is the growth hack for all the authors out there. Get to get to be friends with Andrew, and you're all set.

Andrew Skotzko [00:01:58]:
Exactly. Yeah, I remember you were so generous your time. I'm also working on a book, as you know, and you made a great point of like, yeah, you should make sure you talk about, like, talk to the people who consult on this stuff and who recommend lots of books. And I was like, that is a very good point.

Itamar Gilad [00:02:12]:
This is an often overlooked point that we have these connectors that are very good at recommending your book to other people and consultants, coaches are often these connectors. They know a lot of people, they know a lot of books. So really thinking of these people when you're writing a book is very important in my mind.

Andrew Skotzko [00:02:31]:
Yeah, now you totally landed that message for me last week and again today. And so I'm definitely going to keep that in mind as I'm working on my book. But I want to go ahead. And one thing I wanted to ask you, I've noticed because even before you and I had started speaking in actual live conversations and everything like that, I had read a ton of your material. And one of the things I've noticed about your stuff, relative, at least in my mind, to a lot of the other people out there, you're really good at coming up with catchy frameworks and encapsulating big, complicated ideas into really memorable, pithy frameworks. How do you do it? Because this is such a hack for people who have to communicate a lot like the leaders and listeners of the show. Talk to me about that. How do you do this?

Itamar Gilad [00:03:17]:
First off, I realized just last year, listening to another famous writer, that frameworks are out of fashion. Now you say models, you create models. Pardon me, this is a nicer way to say framework. It's actually the same thing. And I really think it goes back to my origins as an engineer and a product manager. I like to build stuff and now that I started consulting and meeting people, I realized they have pains and they have issues. And so I started writing articles and I trying to divert them to the article and I realized people don't want to necessarily read 1500 words to find a solution to the problem. So I felt I need to do something more graphical.

Itamar Gilad [00:04:03]:
So I need to step back, understand what are kind of the core principles and what can I do to help. And that generated a lot of garbage, frameworks or models and a few that people like. And that's just the way I operate. Just an engineering mindset, I think.

Andrew Skotzko [00:04:22]:
Yeah, no, I love that. But I want to get a little more tactical. And I'm also asking this selfishly because as I said, I'm writing a lot of stuff, but I think I've talked to so many people who are, they're trying to communicate a strategy or a mental model for something about the company. And this is like a real thing. So could you actually tell me a little bit about how do you like, what's your actual process for coming up with one of these? Like I'm wondering, did you know we're going to talk a lot more about the GIST model in a few minutes here. Did that just pop out of your head fully formed as GIST, or was there iterations along the way? Because I feel like people, I see it and I go, wow, it just came out like that. That's amazing. But I'm imagining there might have been a little more to it.

Itamar Gilad [00:04:58]:
Yeah, I'm trying to go back and think about it. That one. I kind of iterated with one of my early clients. I started here in Barcelona, where I live. I realized that I really was into goals because I came from Google and understood how they work and why they're so powerful. And I felt like idea prioritization was very strong, and I saw the connection between the two. And then I realized people really like to do the task management, Kanban, etcetera. If we could put all of these together into one model, that would be helpful.

Itamar Gilad [00:05:40]:
But then I realized the experimentation part is missing. So I felt we need another middle in the middle between ideas and tasks, which I called strides. And I showed this to this guy, he's Spanish. And they said, what the hell is strides? I said, you know, it's like, like, you know, you take a step. I said, so call it a step. And. And that's how it came out. Big goals, ideas, steps and tasks.

Itamar Gilad [00:06:09]:
But the actual framework was kind of buzzing in my mind ever since I worked in Google, when I kind of tried to understand what makes Google different. And you have to, when you think about these large corporations, you have to separate their past. And sometimes that's the golden age and the reality in the day to day. And a lot of googlers maybe are listening to this and saying, yeah, I read your stuff. It doesn't sound like Google today. I know, but if you go back to the DNA, Google was about trying a lot of ideas. If you remember back in the day, they did search, they did advertising, and then they exploded.

Andrew Skotzko [00:06:48]:
There was social and there was all.

Itamar Gilad [00:06:50]:
Yeah, hundreds of things. It looks like the company lost its bearing. It's like trying everything. But there was a strategy behind it. It was, we are about organizing information. What kind of information is there out there that we can organize and make available on the Internet for people? So someone, Paul Buchaith, an engineer, said, I want to import my email into a Google database and make it searchable from a browser. Why not? Go ahead. Someone else said, why don't we organize all the products? So they had something called frugal initially, and then.

Andrew Skotzko [00:07:26]:
Yeah, I remember that.

Itamar Gilad [00:07:27]:
Yeah. And they tried a bunch of stuff, but they had this principle that said, fail fast. Whatever you do, generate the evidence to show us that this is worthwhile. And they had another principle called think big, but start small. They had another principle called let a thousand flowers bloom. And I felt those are beautiful principles. I want to put all of this in a format and so many companies can benefit from them. I want really companies to stop obsessing about developing this very rigorous roadmap with the five perfect ideas, launch across the timeline, and instead embrace this model of let's try a lot of things.

Itamar Gilad [00:08:07]:
Let's start with the clear goals and a strategy, but then let's try a ton of things. You know, like Linus Pauling said, the famous biochemist, if you want to have good ideas, you must have many ideas. And what you need to learn is which ones to throw away, because most of them are not good. So Google basically practiced that. So I realized that's kind of the DNA that made Google successful, at least in my. And I wanted to capture that. And I said, okay, you have to start with goals, but then you need to create this ideas layer to tell people it's a tree for each goal. There couldn't be just one right idea to achieve the goal.

Itamar Gilad [00:08:43]:
There are multiple ones, three. So that's created the top of GIST. And then below that you need to explain to people, you don't build the whole idea in one go. That just bonkers. You don't even do it in two steps. If it's a large idea, just MVP. You know, before we called it a beta, now it's MVP and then launch it. That's also not very conducive to learning.

Itamar Gilad [00:09:08]:
You need to learn much more quickly. So each idea breaks into a smaller set of steps. Steps are like mini projects that build idea a little bit in concept or in code and test it and teach you something. And then you can you vote or dump the idea because most ideas should be dumped. And then those need to be connected to the tasks. And in Google, even the task wasn't a big thing because people just used this very light form of kanban, which was beautiful and much less heavy handed that I see a lot of things today. But when I went into the world and I showed this to people, I said, yeah, but what about agile? What about scrum? What about clarification? What about retrospective? So I connected that layer, too. I think this is a very long winded answer to your question, how? Just came about.

Andrew Skotzko [00:10:02]:
No, no. I mean, I love it because one of the ways, and please correct me if this is not how I should be framing this for people. But often when I've introduced your work and these models to folks who all have varying degrees of prior knowledge, some have been in product work for a long time, some are just learning it right now. One of the ways I framed it is it's kind of like a meta framework that gives you an organizing model for how to do this whole thing. And there's a lot of pieces inside of it. But what I loved about it when I first encountered it was, I was like, oh, perfect. You've so perfectly captured and named something that I and many others had also kind of stumbled our way into. But you explained it, it was like, oh, here's how all this stuff, which is so overwhelming, here's how it all fits together and how you actually do it.

Andrew Skotzko [00:10:54]:
Is that the right way to frame it? Because that's how I've been framing it. And if that's not right, I'd love to be corrected.

Itamar Gilad [00:10:59]:
That's actually much better than I explain it how I explained. So, yes, it's totally a meta framework. If you take each one of the layers, like goals, what you find inside there is not. Anything that I invented is north Star metric, top KPI metrics, trees, value exchange loops. These are like. These are things that existed for 50 years. Some of them are 60. But I do find that people really struggle to put everything together.

Itamar Gilad [00:11:27]:
I meet so many cpos, and they all create their own little mix of everything. And sometimes it's beautiful, and I learn a lot from it, and sometimes they're just lost because they're trying to compound something really complex. And I'm sure you've seen this, too.

Andrew Skotzko [00:11:44]:
So I tried to create it.

Itamar Gilad [00:11:51]:
So I think there's something to say. Here's how it all fits together. But caveat. It's not the only way. It's one way to do it. Just be aware.

Andrew Skotzko [00:12:02]:
Totally. Yeah, absolutely. Yeah. I think that's a great caveat. So let's get into it a little bit more, because, of course, I wanted to have this conversation because I'm a huge fan of the book and your work. But for those who have not yet bought the book, and for that listener, please go fix that. Can you just give us a quick run through about how, like, you kind of just laid it out a little bit in terms of the goals, ideas, steps, and tasks, but if there's maybe a story you could anchor this in to kind of bring it to life a little bit, could you. Could you walk us through, how does this come to life?

Itamar Gilad [00:12:34]:
All right. So it's a bit of a long story, but it's a story I give in the book as well. And that's kind of the first time I kind of stumbled on this model of gist. I was working in Gmail as a product manager.

Andrew Skotzko [00:12:47]:
Okay, and where are we in time? Is this like, what year is this? This is like 2010, something like that.

Itamar Gilad [00:12:52]:
This was 2011, 2012. And we had this overall strategy to increase engagement because Gmail was really freaked out by the fact that messenger and other, you know, ims are taking over all the personal communication and email was left with basically business to consumer communication, which I think is where it is today. And it's a very valid application and people need it for it. But back then we really wanted to engage people, so. And then we ran into this really weird situation. What does it mean, engaging? We can send them a lot of spam or allow a lot of spam coming in. People will open their email more. And that got me thinking.

Itamar Gilad [00:13:37]:
I looked at my own inbox and said, it's full of not spam because spam is filtered out, but what we called basic bacon is messages. You understand why you receive them. You have a relationship with this business, but you don't necessarily want to read them. You know, you have promotions, you have notifications. Back then, Facebook was sending a ton of social notifications and these things really cluttered people's inboxes. And my colleagues at Gmail were very diligent. They were what you call zero inboxers. They would move all of these things, archive them, keep their inbox tidy.

Itamar Gilad [00:14:13]:
And I was super lazy inboxer and I just felt, well, it would be great if something just self organized my inbox and I don't need to deal with this, I'll just get the important stuff. And then I came to my team and to my manager and said, hey, I found figured out engagement. Here's what we need to do to just allow people to engage with the emails they really care about. And they said, oh yeah, very nice, but we already have five features we launched similar to yours. So what makes yours different? And why do you think this is a wide problem? Because consumers don't get that much email on average. And there are all sorts of statistics and reasons why this is not a solid idea. And basically they were pushing back and saying, what's the goal here? Don't start with the idea. Tell us.

Itamar Gilad [00:15:02]:
And that, I think is a good question for every pm to consider when you fall in love with an idea, stepped back and said, what is this going to achieve, actually, except this coolness factor. And we realized that it really is about allowing people like me to engage just with the emails that they care about, but allow them to also find that other email if they want to, not just bury it totally. And we attach some metrics to those so that the classification, whatever we do, people will never miss something that is important to them. We had ways of measuring that, and they will engage with the messages we think are the most important. And then we started brainstorming ideas. It was my idea, which was initially creating something on the side that says, hey, by the way, you only got some notifications, etcetera, but that someone else, a designer, came up with this tabbed inbox model where it's actually, the inbox is segmented into a few tabs.

Andrew Skotzko [00:16:05]:
This is the origin story of the tabbed inbox.

Itamar Gilad [00:16:08]:
It is, yes.

Andrew Skotzko [00:16:10]:
All right. I love it. I use that thing all the time. Please. Thank you for this. I'm so happy this exists. So keep going.

Itamar Gilad [00:16:16]:
Thank you. So we started defining the categories that took a while naming them. That's always challenging. But again, there was a question. How do we know that this is actually a real problem? This is ideas worth pursuing, because this is going to be very transformative for a lot of people. Changing the way the inbox behaves is not a small matter. So we did a bunch of things. We did a quantitative analysis, seeing how people manage their inboxes.

Itamar Gilad [00:16:46]:
We found that most consumers are actually extremely passive. They don't match the inbox. They just let everything pile up. Maybe they read something. Maybe the market is unread. Good luck finding it later. So it was just this massive pile of email. And that kind of shocked a lot of the googlers around me because they, how can people live like this? You need to be a zero inboxer.

Itamar Gilad [00:17:08]:
And I told them, this concept does not.

Andrew Skotzko [00:17:10]:
Were people not using search? Because I, as a Gmail user now, I kind of rely pretty heavily on the tabbed inbox. And then for all those things that I don't know where they are, I just search for it. Is that not what people were doing?

Itamar Gilad [00:17:21]:
It is. And that's the superpower of Gmail. I think that's what saves people. But if they just saw a message, if you just got an important message, chances were that it was buried behind ten other unimportant messages. They really need to work hard to find those messages. And we saw people in the qualitative part of the research, we saw people with 120,000 unread messages in their inbox, for example. This was ridiculous. So we realized this is a much bigger problem than we thought.

Itamar Gilad [00:17:53]:
And we tried a bunch of ideas. One was about teaching people how to move messages to the right place, etcetera. But one idea, when we tested it in a usability test, just exploded. Everyone loved it. And that idea was to move the messages automatically into socials, promotions. And we had another category called notifications back then, and that made out of twelve people that saw these. Ten were just like, when can I have it? This is. This was saving lives.

Itamar Gilad [00:18:26]:
They were smiling. They were happy. The other two didn't like it so much because they already knew how to organize their inbox. They didn't have the problem. And I realized later on, throughout the experimentation, that 15%, this same ratio of ten to 15% of Gmail users already know how to do this, and they usually don't like this feature. So when we launched it, we only launch it to the 85%. But this 15% were all of my colleagues and all of the tech press. Later on, when we launched it, everyone gave us really lukewarm reviews about this change and said, nothing special here.

Itamar Gilad [00:19:03]:
Gmail just did this thing that is unnecessary. But really, the silent majority loved it. Those were steps. Those were kind of. I didn't call them steps at the time. The data analysis, the quality of analysis. And then later on, we did many more experimentation, experiments, and we launched this for longitudinal studies for hundreds and hundreds of people. Dog food.

Itamar Gilad [00:19:26]:
This was a major change in Gmail, so we had to test it really thoroughly. And as part of that, at one point, my manager told me, you know, this is such a transformative change. We can't just do it on the desktop, we have to do it on mobile as well. Because up to this point, it was just a desktop change. And that was a problem because mobile part of it was controlled by Gmail, the iOS app, but the Android app was part of the Android team. And these guys had their own agendas, and we couldn't control what they do. So I came to them and I just showed them the evidence. I said, look, we tested this with this many people, and everyone's just loving it, and it's really a game changer.

Itamar Gilad [00:20:09]:
And that took care of all the convincing I had to do. So it showed me the power of evidence. That's why the book is called evidence guided. It showed me how important it is to convince people with evidence. It really empowers the weak people against the powerful people in the organization. So we did a co launch. It took more than a year to develop. And it launched across all Gmail clients.

Itamar Gilad [00:20:34]:
And it was one of the most successful launches today. It's the. There's more than a billion users in GMO today. That's the first thing they see. But it wouldn't have been successful unless we did it step wise and made a lot of changes in the middle and learned a lot.

Andrew Skotzko [00:20:50]:
No, I love this story, this perfect example. Really quick, I want you to summarize, you just laid out this whole journey, but can you just. For the listener who's still getting their head around the gist framework, can you just say, here was the goal. Here was the g, the I, the s and the t. Which parts go in which buckets?

Itamar Gilad [00:21:06]:
Yeah. So I started at the. I started with an idea, which is very common, by the way, but then I was forced to go back to the goal and ask the question, what are we trying to achieve? And the objective there was to allow casual users, passive users, to engage just with the email they want to engage. And there were a set of key results attached to that. And that goal really focused us moving forward. And then we went back to the eye and we considered a bunch of ideas for that same goal. One was mine, which had a certain Ux and a certain algorithm behind it, turned out not to be the best. But then another idea that was simpler in many ways, but more powerful turned out to generate stronger evidence.

Itamar Gilad [00:21:49]:
And we learned this by doing steps. One step was a data analysis to understand how people evaluate, just to see if there's a reason for any of these ideas. Another was to interview them. Another after that was to do a series of usability tests. And by the way, when we did the first test of this tabbed inbox, there was no tabbed inbox, really. It was just a facade of HTML, a complete sham. One of our designers cooked it up. And when we invited people in, the interviewer would distract them, like, they would sign some sort of release, and he would distract them and ask them questions about their lives and usage of email.

Itamar Gilad [00:22:29]:
And in the background, some of us would start moving the messages to the right tab, just the sender and subject. It was permitted by the, by the release they gave us. And then he said, all right, I want to show you something. Can you look at this? And then they saw their inbox segmented into the tabs by category, and that's what blew their minds. So that's a wizard of Oz test, essentially.

Andrew Skotzko [00:22:54]:
Yep.

Itamar Gilad [00:22:55]:
Super powerful. Really told us a lot. So that's the steps layer. We did a bunch of steps. And we went back and forth on the product and improved it until the last step was the launch itself, which was a released stage launch. And then there were the steps, which was back then I didn't really consider them because my team just was part of the whole thing and jiving. But I realized later on, with a very regimented scrum team, you need to connect this to sprints, you need to connect this to the rituals. And that's the task layer that wasn't that prominent in this origin story, but it is useful in today's world.

Andrew Skotzko [00:23:33]:
Absolutely. No, sorry. Thank you so much for laying that out. I think it's really, really helpful just to. Again, I think you're so good at helping people get a mental model to organize this big, messy process. And so it's really helpful to categorize things like that. So I want to zoom in really quickly on the middle part of gist, the ideas and the steps, because the way I think of it is there's sort of this feedback loop between them, right? You have ideas, you rank them, and we can talk about I scores here in a second. And then you take some steps and then you feed that back into the updated ranking and iterate until something makes sense to really go all the way with.

Andrew Skotzko [00:24:05]:
One of the first models of yours that I was introduced to was the confidence meter. And what I loved about it was it actually put down some kind of external, slightly, somewhat objective framework for how to think about confidence. Talk to me a little bit about ice scores and where they go sideways in practice for people.

Itamar Gilad [00:24:24]:
All right, so I scores stand for impact, confidence and ease. It's just a way to rank ideas by attributing these three attributes to them, usually in the range of zero to ten or zero to five each. Impact is how much is this going to impact our goal. Confidence is how sure are we that we're going to have that impact and ease and ease is about how easy or hard it is to be. And you should think about it. The ten impact is the most impactful idea you can ever imagine. It's going to be a game changer, and ten ease is the easiest idea you can ever implement. The easiest feature, it's like maybe two weeks and it's out there in production and everything aligns back from that, from zero to ten.

Itamar Gilad [00:25:08]:
So it's really tempting for people to collect a lot of ideas, I mean, hundreds, and try to ice core all of them and then manage this huge list of ideas. And then they do it just once per idea. And that's kind of the prioritization, and from that, it leads into a roadmap. And that's a terrible use of ice. I mean, if you do this, don't even bother you doing ice, because most of these initial ice cores are just based on guesswork, and people don't realize that that's a very weak form of evidence, just opinions and judgment. Even if you're the most informed and experienced person, it's really almost impossible to say that this idea will have that impact in six months when we launch it. So what's important is to use initialize to kind of filter out a lot of these ideas and stay with a much smaller working set of candidates, and then do steps and reevaluate ice. That's what I recommend doing.

Itamar Gilad [00:26:09]:
And every time you have new evidence that kind of supports or refutes the assumptions behind the idea, you can elevate or decrease the impact and the ease and give yourself a slightly higher confidence score. That's the way I recommend thinking of ice. So that's the most common mistake, is doing it just once and over, relying on the ice core to construct your roadmap?

Andrew Skotzko [00:26:35]:
We'll go to roadmaps in a second, but I want to actually emphasize one thing and ask a clarifying question. So the part I want to emphasize is the repeated nature, the iterative nature, right. As you said, don't just rank it on an ice score and then go straight to, like, a roadmap. You're throwing out the whole entire point of experimentation. So I just want to underline that Tripoli. Mike, the thing I also wanted to kind of emphasize and ask you to clarify, you said right there at the end, which was that you use the steps to refute or support the assumptions under the ideas, which I think is a really subtle but important distinction between refuting or supporting an idea and the assumptions under it. And I'd love you to unpack that a little bit, because I think there's something really key there.

Itamar Gilad [00:27:16]:
So, yeah, first off, gist is a discovery framework. Multi Kagan gave us this term product discovery. It's really useful. So if you're confused, it's a product discovery method. And one of the contributions of multi Kagan is that it lists these four types of assumptions or risks that exist in every idea. One is the value risk that customers actually don't need this, or there's no desire to have this. The second is feasibility risk. It might be very hard, or we might not have the technology usability risk.

Itamar Gilad [00:27:52]:
It might be very hard for them to use it or it may not fit with their lives. And the fourth is business viability risk. So it might not actually contribute to our business goals or maybe even compete with one of our existing ideas or project, which is a wonderful way to kill an idea. Even if it's a great idea, the VP responsible for that product will kill it for you. You can be sure of it. So that's another risk we need to be aware of and factor for. Under each one of these categories, you can list a bunch of assumptions. This will work.

Itamar Gilad [00:28:26]:
If do people really need that, will people be willing to pay for that? Etcetera. And those are the things you want to test in your experiments, in your steps. So when we did the tabbed inbox, for example, one of the most immediate big risks was whether or not they understand what happened to their messages, why they ended up there. So that wizard of Oz step help us determine that this assumption is verified. People completely got it. It wasn't easy for them to explain.

Andrew Skotzko [00:29:00]:
So that was the usability risk.

Itamar Gilad [00:29:02]:
Yeah, that was a usability risk, exactly. And also the value risk is actually something they desire and they want. Yeah, they absolutely wanted it. But going back to the confidence meter, that didn't give us 100% confidence, because we only asked twelve people and out of them, ten said for sure. Yes, but you can definitely be on the wrong side of statistics. And when you launch this to the whole population, it might turn out that it's not good. So that gave us what I call medium low confidence, which is enough to go to the next step and build a more evolved version of this thing, which we actually started using internally within Google, what's called dogfooding. And that gave us even more confidence and so on and so forth.

Itamar Gilad [00:29:46]:
That's the cycle.

Andrew Skotzko [00:29:47]:
All right, cool. So now I think we need to take with second and touch the third rail here. So I think we need to talk about roadmaps for a minute. It's a topic that is so divisive and all these things, but when we think inside the world of this meta framework around gist and how do you recommend folks think about roadmaps? Where does that fit in all this?

Itamar Gilad [00:30:10]:
Yeah, it's such a central topic. I mean, when I was growing up in product management back in the late nineties, early 2000, that was the core of product management, roadmaps. Everyone was like, where is your roadmap? And still today, for a lot of companies, like having the right roadmap is a sign of success. We nailed it, we did the planning. There's a bunch of problems with those output roadmaps, as we call them, or release roadmaps because they require an awful lot of time upfront to sit and decide. These are the most important ideas. This is what we're going to launch this year. Usually, roadmap used to be a yearly thing in some companies, multi year.

Itamar Gilad [00:30:52]:
So just deciding was a huge political endeavor, let's be honest.

Andrew Skotzko [00:30:57]:
Oh yeah, the annual planning cycles from hell, right?

Itamar Gilad [00:31:01]:
Yeah, exactly. I hated that. And often it's based mostly on opinions and some sparse data and consensus. And, you know, it's not a really good decision mechanism either, although we might feel confident about it. And often people don't like they come out of the decision exhausted. Yeah, we decided on this, but I don't trust this plan so much, this roadmap. Let's go with so. Yeah, so it's a tough process.

Itamar Gilad [00:31:30]:
But then when you look at how good the roadmap is predicting the future, how much are we able to follow the roadmap? It's abysmal. We're always late, we're missing deadlines constantly. We're dropping launches all of a sudden or something pops out of the blue. And we are going to do this for this customer. So we're not very good at following the plan. And even worse, if you look and do a backwards analysis of how well the roadmaps are helping the business, how well they're helping the user experience, it's really hard to show any difference. It's still growing at the same rate. And those must have features turn out to be not that must have.

Itamar Gilad [00:32:10]:
So I think I understand the reasoning behind roadmaps. It gives a sense of certainty. It gives a sense that the product organization is working towards something in some companies, like a contract between the business and the product to keep them accountable. And it's sometimes used as a sales tool towards customers. You know, by Q three, we're going to launch this feature. Exactly. Really impressive. But we really have to ask the meta question, is it worthwhile? Is it helping our business? Is it helping our customers using this process? And also, what's the alternative? And the alternative is what's.

Itamar Gilad [00:32:52]:
There are two alternatives basically on the table, the time of war, and you may enlighten me. And maybe there are others. There's what's called outcomes roadmap, or pure outcomes roadmaps, which is what I'm in support of. And there's the now next later roadmaps, which are kind of middle of the road and are subject to interpretation. They can be outcomes based. They can be output based. They're kind of, what they are trying to do is shorten the horizon. They're saying we're going to commit for the now, the next quarter, and then after that we're less committed.

Itamar Gilad [00:33:24]:
There's the next and the later phases, and for those we're not going to commit, we're going to do the things required to know for sure. And there's parts I like about this and there's parts that I like less. And I think it's really tempting to subvert the now next later into output now, output later, output next and later. So I can talk about, if you want to, the outcomes roadmaps and.

Andrew Skotzko [00:33:55]:
Yeah, please, just to comment on that. Yeah, that basically matches what I see as well. I was trying to sit, I was sitting here trying to think of is there anything else I see out there? And the only other thing I can really, that I've seen other than, you know, let's just call them pure output roadmaps, which we're not such a fan of, then we have outcome roadmaps and then we have now next later, which is somewhere in between. The only other thing I've really seen, I think, is there really isn't, and this is really only, I've really only seen this in early startups is there really is no roadmap and there's just a goal, and we're just, we're just going really hard at the goal and we have an idea of what we're doing. But that's kind of implicitly, like, it's sort of implicitly doing an outcome roadmap anyway. You just haven't laid it out in that structure. But yeah, I'd love to hear you talk about how you think about it in terms of the outcome roadmaps.

Itamar Gilad [00:34:47]:
Yeah, I think you nailed it on the head. I think we should all think like startups, we should start from the end goal. What is the outcome? What is the change we're trying to achieve? And then build work backwards to build a stage plan that leads us there and with some honesty, not just with fiction and hopeful thinking. So first off, let's say, let's take a typical case. It's a mid sized company, a few hundred people. The company has, let's say four main okrs in their yearly goals. I would say take the year, divide it into an actual timeline, and place the okrs on it. Say, when are we going to start working on this goal and when are we going to stop working on it? And at that point we expect to achieve it in full.

Itamar Gilad [00:35:43]:
What does this mean? If we look at the key results, we could put a check mark next to all of them. If we said we will reduce churn from three to 2.5 by that point, that will be true. If we said that we will acquire that many new customers, or shorten the onboarding time, or whatever it is, we want to nail it by that point. So that alone is an interesting exercise. You need to think, not which projects are we going to build and how we rely on headcount, but which achievements, which goals are we going to do and how we're going to allocate resources for those. And it's a bit more tricky for the CFO and the CTO to do this, but I think it's just as reliable as the output roadmap in a sense, which means it will slip too. But at least we're working towards our goals, not towards theoretical bets that will achieve the goals, and that's the key difference. So let's say you place your four or five okrs on a timeline, now you have something to go to the rest of the company and say, everyone coalesce around this.

Itamar Gilad [00:36:54]:
We need to achieve this. Product marketing, product management, product development, everyone. We need to achieve this. Think how you gonna get there. Now we can deconstruct each one of those, because we really want to see what the work that goes into it. So usually there are four stages or steps in the in each goal. The first one is research, exploratory research, which is often the case when we have a goal which we don't understand enough, we don't know how we may achieve it, we don't know what's possible, we don't know what are the key challenges underlying it, or opportunities. So it's worthwhile allotting some time.

Itamar Gilad [00:37:38]:
Says, let's take three weeks to interview customers, to observe the competitors, to look at technology, to do some proof of concepts, and come up with a bunch of ideas. Essentially, that's what we want to generate from the research phase. The next phase, which is slightly overlapping, because every time you find one of these ideas, you can switch into it, is discovery, which is exactly what we covered until now in the gist framework, which is, let's validate which of these nine ideas we came up with actually works. The statistics say only three, by the way, one in three is the best case scenario. So let's do discovery on these ideas, which first means evaluating them using ice, then picking the ones that look most promising, and then starting to do steps on them. Some will throw away or pivot, and some we will double down on. And eventually some of these ideas, let's say three, we will validate enough to the point where we have sufficient confidence to say, no need to validate enough anymore. And we need to move on into delivery, which is the third stage.

Itamar Gilad [00:38:50]:
Every time you find one of these ideas, switch to delivery and then just heads down, build it, production ready, and launch it. But then the outcomes you expect will not materialize immediately. There's a delay, a propagation delay until the customer deploys it, until new cohorts come on, etcetera. So you need to monitor. That's the fourth stage, monitoring. And of course, the number of people you need for each stage is different. In the research phase, you need a few people, and other people can work on other things. In the discovery, you need a bit more.

Itamar Gilad [00:39:26]:
In the delivery, you need a lot of people. And then in monitoring, you need far fewer, so you can do some resource leveling. When you think about this way, if you map out these four stages and time box, each one, even if you don't know which ideas will be the winners, but you know it, you guesstimate it will take three months to build those, and it will take a month and a half to discover which ones. And it will take three weeks to do research, and the propagation delay will take a few more weeks. You have more or less a plan, a roadmap of sorts, which I feel is much more honest than what we have today.

Andrew Skotzko [00:40:07]:
Yeah, no, I really like the way you laid that out, because the question that I was wondering about was, I was thinking, you said a few minutes ago, we should probably all work more like startups. I can imagine so many people who don't work in startups, who work in enterprise and much bigger companies going, yeah, right. That sounds nice. And they're thinking about like, yeah, but then how do I deal with, like, the planning? Right? Because there is budgeting and resourcing and all of this stuff. And it seems like you were just starting to touch on that a little bit with, like, the exploratory research. And I know you have an article called talking about multiple strategic tracks, but I was wondering if you could kind of zoom out, if we could zoom out a level here and talk about how that flows into this.

Itamar Gilad [00:40:47]:
Absolutely. So we talked about goals, but it's really hard to have concrete goals without a mission and a strategy. I mean, there needs to be this overarching framework on top of all of this that says, this is the direction, this is our part in the world. This is what we're trying, the value we're going to create. This is the piece of the territory we're going to call for ourselves. And the strategy for me is a theory of how we'll get there. Long term, broad, and it's ever evolving. It's not like, again, there's a tendency to do planning, like an off sites strategy.

Itamar Gilad [00:41:28]:
The strategy is done for the next three years. Absolutely not. Never works.

Andrew Skotzko [00:41:33]:
God, no.

Itamar Gilad [00:41:35]:
So it's an evolving theory. And the theory usually involves opportunities and challenges we're seeing, and some research, as you said, to better understand those. And then we need to do some sort of strategy discovery as well. Just like we discover products, we need to discover the strategy. So the article you reference is a pattern that I call must multiple strategic tracks, which says, once you have this mission in mind, you might want to explore multiple things in parallel, not just bet on the one thing. You see a bunch of opportunities and you're going forward. Google, again, is a good example. Google.

Itamar Gilad [00:42:15]:
So the opportunity to organize the world's information, that was the mission. Right? But they saw so many different opportunities for information they can organize, for example, video. They realized early on, people are going to upload a lot of what was called back then, user generated content and video is going to be searchable and people really want to find other people's video. They were very visionary in this sense. So they launched. Their first attempt as this strategic track was something called Google Video, which was much like YouTube, but not successful. It failed. Google was always struggling with social.

Itamar Gilad [00:42:57]:
They don't get social, they're data people. And that was true back then, it's still true today, if you ask me. So it failed in most companies, they would deduce that the strategy has failed. But in Google, they just said, this attempt at capturing the opportunity has failed. But the strategy is very valid. We still want to be able to index all the videos in the world. What's the next best idea? And then that idea was, let's acquire the best video startups around. They looked around, they found in San Bruno the startup called YouTube, and they acquired them.

Itamar Gilad [00:43:32]:
And there was some due diligence, of course, to validate. This is a. A good idea, and that was much more successful. So if you look at Amazon, if you look at Netflix, if you look at basically every successful company in that article, I also explain how this applies to Apple and the iPhone. You realize it's almost never like a predetermined strategy where someone, some visionary leader says, this is the product we need to build and this will achieve our goal. They kind of try out many different things. Most of them fail, but they learn from the failure and they combine and eventually they ended up with something that is very, very significant. And that's how I recommend companies to think of strategy as well, at least in product companies.

Itamar Gilad [00:44:19]:
Once you have that product discovery can come into place and delivery and all this good stuff that we talked about, the roadmaps, but it is a very important infrastructure that a lot of companies are missing.

Andrew Skotzko [00:44:32]:
Yeah, I see the same thing. Hence the work. What I'm trying to do, I'm basically, I'll say this for the first time publicly, I'm trying to do for strategy what you did for discovery. I want to write the strategy version of evidence guided. So that's my aspiration. So you set the bar very high, my friend. So I'm curious about that in terms of it's almost taking the same mindset that we do in product discovery, just up a couple levels to strategy. It's saying, hey, we have a mission.

Andrew Skotzko [00:45:01]:
The way I frame it for people is often mission, vision and measures. We have a mission. There's some fundamental thing we're trying to do. There's a vision of what might that look like? Hopefully we have a vision that's concrete and people could actually look at it. That's often shockingly missing. Then we have some high level impact measures. If we are wildly successful, we might have this level of impact in the world and a couple big metrics that represent that. And then you have this gigantic chasm and then sort of resume on the other side of like product discovery and all these things.

Andrew Skotzko [00:45:32]:
And it's this middle zone, this missing middle where strategy lives. And that's what I'm trying to solve. And so the question I love bringing this iterative, evolving theory mindset, evidence guided mindset to the work of strategy because it's so much about, as your, as the subtitle of your book says, in the face of uncertainty, we're navigating through this crazy, uncertain world with evidence. I guess with all that preamble, the question I'm curious about is, when you think about discovery at the strategy level, we have some ideas about how we might do a thing. We think video is a key part of the world's information. We want to organize that. Let's get some people, a little strategy squad to go off, see if there's a there there. When you get into the sort of discovery bit of strategy where you're trying to figure out where there's something worth truly pursuing in product discovery.

Andrew Skotzko [00:46:26]:
How do you think about validating or substantiating things at that level? Like, how does something jump from, like, the strategy idea level to like, yeah, we should go work on this for real?

Itamar Gilad [00:46:38]:
That's a great question. And I think you are in a better position because you're helping more companies at that level. So I'm interested to hear later your thoughts on that. I think it varies by company, and I think there are multiple ways to do it. But here are two canonical ones. At Apple, when they were developing kind of the strategy around the phone, at some point they realized they need to have a phone to protect their income from the iPod. It was really a defensive, a strategic defensive move. What I expected from people is to keep showing them demos, to come up with stuff.

Itamar Gilad [00:47:18]:
And in Apple, it's a very top down kind of company in the sense that the approval and the decisions and the funding comes top down. But people are welcome to bring out ideas and to demo. And the managers have this very distinct taste and knowledge of the market. They're very special type of managers. I don't recommend this as a general system for every company, but in Apple, it works. Especially Steve Jobs was the master of all of this. And so some people came and showed them multitouch, irrespective of phones. They thought it would be a desktop thing initially.

Itamar Gilad [00:47:53]:
And so another team came and demoed an iPod that was gutted and fitted with modems and other things. And they just build a phone with the click interface, with the wheel interface of the iPod. And that was another idea. And they allowed all of these ideas to coexist and to try to develop and come back and show them more evolved versions of them. And then Steve Jobs would kind of say, all right, this is not working. You guys start working on that and rearrange things. So that's one way of doing it. If you take Netflix, for example, back in the two thousands, they realized sending dvd's in envelopes has its limits.

Itamar Gilad [00:48:34]:
We're getting to the point where instant delivery through the Internet is viable. Initially, they thought about downloads, which was kind of the main idea there. But they had a lot of gaping, open questions. How do we monetize this? Is there a social element to it? What's the user experience? If you follow Gibson, Vidal Gibb, he has a whole set of articles that explain that part of Netflix. What they created is set of swim lanes, which were teams that got a little bit of budget, some metrics, exactly as you said, these are your success metrics. One team was about finding what's the best business model. So they tried advertising, they tried pay per view, and they also tried all you can eat, and they found that that's the thing that works. How did they find it before the thing was launched? Through experimentation, through different ways of evaluating data and testing.

Itamar Gilad [00:49:30]:
Another team tried social. Turned out the social element of sharing videos between people was not strong. This swim lane was cut and so on and so forth. And eventually all of this converged into this very beautiful and successful video streaming products in 2007. So absolutely there's different ways to move forward and to discover, and you need to involve real product people like senior, usually real teams, real data analysts, not just managers sitting in the room. You need to actually create kernel team that will develop the idea. And once the idea is mature enough, it will turn into a bigger and bigger project offer. Often the mistake is to start from the big project and say for sure we need this huge SaaS system or this huge video streaming service, 500 people start working on it concurrently.

Itamar Gilad [00:50:26]:
Huge mistake, because this project is doomed to begin with. If you start from a kernel team that's just testing the assumptions and then developing an MVP, and that evolves much higher chances that you will end up with your big strategic move.

Andrew Skotzko [00:50:44]:
I really like that. One of the things that, as you're talking, is becoming very clear to me that you and I are both really emphasizing is the iterative nature of this sort of thing we talked about in product discovery. Hey, don't just slap an ice score on it and throw it on a roadmap. And that's just being linear. The iterative nature of updating your assumptions and getting new evidence and updating your assumptions and that cycle, the cyclical nature of the work is what I'm trying to say, and that's actually very similar to how I see it. I'm still trying to crack the model, not the framework. But one of the things that is very clear to me that I'm stressing heavily is the cyclical nature. I think it's such an anti pattern that we're going to have all the executives go to an offsite and they're going to cook up the 18 month strategy and then we'll come back in 18 months.

Andrew Skotzko [00:51:35]:
It's like, oh God, that is such a recipe for waste. Basically that is just going to, I mean, you might get lucky, but most of the time that's not going to work. And I think we need a better method. And the main one that I'm seeing is the need for this to be an evolving theory, as you said. You know, it's a strategy cycle, it's not a strategy process. That's one and done.

Itamar Gilad [00:51:58]:
Exactly. Gabe explains that they had this rotating meetings with every one of these swim lanes, including him and some senior leaders and the CEO and the team just would explain what they achieved, how are they doing? And then they will adjust the funding for each swim lane based on that. And that's exactly what we're kind of doing with steps. We were kind of evaluating multiple ideas, and then we were kind of pivoting or shutting down or investing less in some ideas and investing more in other ones. So it's exactly the same pattern. You can use the confidence meter as well. It works also for strategy development, but just on a much larger scale. And it takes longer to develop your strategic ideas.

Andrew Skotzko [00:52:44]:
Yeah, that's a really good point, that the steps are a little bit of longer yet. The other thing that I've really seen is the experience of strategy and the fundamental challenges you're dealing with change as the company evolves in stage. So, for example, a lot of early startup folks look at Gib Biddle's series there, which is great, and they can't really, it's hard to relate to if you're like a young startup. It's so big, it's much better suited for a larger, mature, more mature organization. And so there's often this question that I hear people ask of like, well, is strategy even really necessary at the early stages of a startup? Right. And I was like, absolutely it is. But we have a whole generation of people who they heard execution is all that matters, and this sort of thing. And of course, execution matters profoundly.

Andrew Skotzko [00:53:35]:
But I think of it as strategy is a force multiplier on execution, and it sort of sets the ceiling on how much value you can create. And then execution determines how far, how high you actually get. And so when I think about it in that context, it's almost like if we use the multiple strategic tracks idea in those swim lanes, each of those swim lanes typically is almost like a big bet or a pillar. It's often called like a strategic pillar. And to me, a startup is basically just one strategic pillar of like, your whole company is just one big strategic bet. And hopefully it's a good one. But I'm trying to emphasize, I'm trying to make explicit that strategy does matter. It's just, you have a different challenge at the startup stage is does this whole bet make any sense? And hopefully you get to product market fit and there's something there, and you can start to scale.

Andrew Skotzko [00:54:25]:
And then once you reach product market fit and you're scaling now you have a different challenge, which is not so much. Is there anything here? It's now a focusing challenge. Oh, we have so many opportunities to expand this. How do we not drown an opportunity? Okay, that was really long winded, but I'll stop there.

Itamar Gilad [00:54:41]:
No, I agree with you. I think part of the challenge is the word strategy means a lot of different things, and some of these meanings don't apply to startups. Let's think how to strategize, how to partner, how to expand our footprint in emerging markets. You don't have a product yet, so none of this is relevant. And some of them, which are the ones that I think us product people think about, like how do we find the next big thing, the next biggest product market fit? The startup is involved in getting from zero to one, from that to that. And so it's extremely strategic. Everything they're doing is actually inventing the future company and inventing the future product. But it has to be extremely discovery based and iterative.

Itamar Gilad [00:55:24]:
They cannot just be executing on one idea. That's a sure way to burn out a lot of cash unless you're extremely lucky. So I think it applies to them. But I also understand the people that think strategy does not apply from a product perspective.

Andrew Skotzko [00:55:45]:
A startup is almost like if we took a big company like Google and we take Gmail as a big product line there, that was one big strategic bet that became a whole product line and probably its own. I don't know if it's its own division or however it's structurally organized. And it's like, well, a startup's essentially the same thing, just pulled out of that big mothership context. And it's just the act of doing this kind of work is often better done in a startup because many big companies, just by the nature of their large organization and bureaucracy and incentives and all the stuff, it's a lot harder to do this kind of inventive, high strategic work in terms of products anyway. I don't know if that matches what you see, but that's kind of how I think of it.

Itamar Gilad [00:56:29]:
Yeah, totally. And I think actually the inception story of Gmail, you can find Paul Buchai, the engineer who created Gmail, telling this story. He was an investor later on. It's very typical at actually how startups should behave. So Paul was this engineer inside Google, and he just had this idea for a side project where he can import all of his emails into a Google database. He was frustrated with his ISP, there was limited storage and search sucked. He said, ok, I'm just importing my email and now I can search it and read it from. He put on it, use groups or something, that Google groups had this interface.

Itamar Gilad [00:57:08]:
He put it on it. All of a sudden he could search and read his email from anywhere. So he started sharing this with his colleagues inside Google and said, what do you think about this? And everyone was like, yeah, I love it, but I wish it was my email and not yours. And so feature number two was import. I didn't realize this was such an old feature just to allow other people to import. And then people said, I really dig it, but now I would like to reply to some of these messages. So that gave him a pipeline of ideas how to improve. And that's exactly how you work as a startup, too.

Itamar Gilad [00:57:41]:
Don't just go all in building the entire Gmail. Start small and test your assumptions.

Andrew Skotzko [00:57:49]:
Yeah, I love it. I want to start to close out here. And the question that I wanted to wrap up our conversation with is I'm realizing that underneath all of the things that we're talking about here is there are certain skills, but I think even underneath those skills, there are certain mindsets. And coaching mindset is a notoriously difficult thing. And the mindset in particular that I see people struggle with is around uncertainty. Right. Like so much of this work is very uncertain. And I'm curious how you, whether it was when you were at Google or in your own work now, how do you help people get more, if not more comfortable, at least more adept at dealing with all the uncertainty.

Itamar Gilad [00:58:39]:
Yeah, that's. I would say that's the biggest challenge. The adoption is really the hardest. I would separate it between companies that already mentally made the switch or they want it. They realize that's the way they want to develop. They're still struggling with actually going through the motions of doing it. They still. This instinct to just lean into execution and drive for faster launches and is still ingrained in all of us.

Itamar Gilad [00:59:10]:
And then there's the companies that still are kind of on the fence. They're not sure. Companies that completely don't want it usually don't talk to me. So I don't have a solution for them. For the companies who are on the fence, I suggest to do some analysis of their past performance to just see objectively how did past roadmaps work for them, these big launches they were so committed to? What were some of the sources of the failure? It wasn't just execution. Don't let any manager tell you that we failed with this big project because of execution? Mostly it's because there was no need for it, or it was of low value for the company or for the customers, and really asked themselves, how much of their engineering resources that are super expensive are they wasting? If the statistics are right, at least 50% and often much more than that. You're just sending people to build the wrong things, but you're projecting this with this air of certainty, and you're actually just. It's a recipe for waste, as you said before, for the companies who are already there but are struggling.

Itamar Gilad [01:00:18]:
I suggest to change the incentives, move away from all incentives tied to output, try to remove output roadmaps and try to do incremental changes, not to switch from zero to one. All of a sudden we're fully on discovery and everything, but to adopt Kaizen like, you know, the principle of continuous improvement. Last month we managed to do two experiments. Let's try to do four next month. Last month we evaluated just one idea. Next time let's try to evaluate five, etcetera, just up. There's a certain set of metrics that I give in the book about what you should look for. Just try to push yourself, nudge yourself to do more and more until it becomes much more natural to you and you remove a lot of the barriers as you are trying to deal with these challenges.

Andrew Skotzko [01:01:15]:
Which are those metrics? Just so for the listener who wants to go look it up in the.

Itamar Gilad [01:01:18]:
Book, some things that are correlated with better products are evaluating more ideas, getting more ideas to a certain confidence level, being able to say, we got at least five out of ten ideas up to confidence level medium, and only launching if you actually achieved the right confidence level. Testing more, running more experiments. Jeff Bezos is a really big one on this. He always encourages more experiments, getting more experiments that yield learning, because not every experiment is necessarily successful, and then launching more validated ideas. So the ratio of ideas that actually launched and reached a certain confidence level before that should go up and up, and eventually you should launch nothing without confidence. And then the ultimate metric is what outcomes did you achieve with your ideas or what percentage of ideas actually generate the outcomes you expect if you optimize for all of those, you are more likely to create higher impact.

Andrew Skotzko [01:02:30]:
I love that you spell it out like that because it's so helpful for people. Again, amidst all this uncertainty, it's very, very helpful to have something to grab onto that will lead you in the right direction. Thank you for creating that clarity. I think that's a general note I want to say just to close this out is thank you for doing the work you're doing, Itamar. It's really, really helping a lot of people, and I'm so glad that you are doing it. So thank you for what you're doing. Please keep it up. And just in closing out, how can folks listening be helpful to you, and where can people find you online first?

Itamar Gilad [01:03:06]:
Really, thank you for the kind words. And it's humbling to hear that people like you are finding value in my work. People can find my things in my site, itamargilar.com dot, where a lot of the models we talked about you can actually download as free downloads. Caveat. You'll need to sign up for my newsletter, but maybe that's a price to pay and then you can unsubscribe right away. I will not hold you to it. And if you're interested in the book, just go and check evidenceguided.com and you can read all about the book itself. And honestly, I'm looking forward to your book because it sounds like it's going to be very much in line and it's going to be.

Andrew Skotzko [01:03:52]:
Well, you're going to be part of that journey. So I will certainly keep you posted and look forward to getting your input on all the things. But for now, Itamar, thanks so much for what you're doing and thanks for being here. We'll see you out there.

Itamar Gilad [01:04:04]:
Thank you.