Social Dilemma, Metrics & Thoughtfulness
I happened to watch Social Dilemma recently. And it triggered a series of thoughts, on topics I had been pondering upon for some time already. So I wrote. In my private notes. But then I remembered the existence of this newsletter. In fact, to be completely honest, few friends recently b̶u̶l̶l̶i̶e̶d̶ reminded me about the eerie silence here.
So here I am, writing this newsletter. But not before I thought a bit about just why I have not been putting anything out here. It's not like I haven't written much. There's a lot in my Notion. But with the self-imposed expectation of just how good the essay needs to be before it's published, I never published. I have been wanting to write as beautifully as some of my favorite writers. It will take me years of honing the art before I get there. Until then, aiming for excellence is just getting in the way of practice.
In other words, I'm posting below a piece without wracking my brain to edit it 100 times. Just warning ya!
Social Dilemma
Social Dilemma presented the concept that the metrics companies are optimizing their business for is not based on what the user really wants. The documentary basically calls out that companies like Facebook, Google etc. are building products for advertisers, exploiting human attention in the garb of building products for these users. They claim to be improving lives of their users but stop short of solving for the real needs of the user like forming deep, meaningful relationship.
They really are eventually aiming to increase your interaction with advertisers. And this is why while Facebook sends an email notification when a friend tags you in a photo but it does not attach the photo in the email itself. It wants you to go to the app and spend time. Get sucked into that photo and everything else around it, especially the ads.
Side Note: It's a pretty well made documentary, albeit a bit dramatized for the right effect, but largely based on undeniable facts. Highly recommend watching it if you are in the tech industry, building products with mass impact.
Leading through Metrics
As companies become larger, they tend to lead through metrics, the goal being to optimize these metrics. Almost all advertisement driven revenue generation platforms end up optimizing for the time spent on the app as the north star goal.
But metrics are hard to get right. There's so much nuance to right metric tracking and ones you need to optimize for. Let's see an example.
You want to ensure customers get fast responses to their support tickets. So you put a goal on “First Response Time” (FRT). But now you have bad responses being sent very quickly so your FRT is low? Ok, add an average resolution time goal, so that actual time before issue is resolved is optimized for. Check in few weeks later and you have a case of tickets being marked as resolved, in the hope that the customer doesn't reopen the ticket within 3 days, the time window within which they need to reply otherwise their reply creates a new ticket.
You see, each metric creates a new problem.
Some of the smartest folks built the Facebook Like button. And I honestly don't think any of them had an idea of the long term impact that it would cause. Especially when they need to weigh the positive impact (effective way of showing appreciation) and the negative impact (frequent dopamine hits causing it to become a dependency), and even more especially when the the negative impact will take much longer to show.
How on earth do you expect someone to think of long term unintended consequences whose job depends on increasing the time spent on Facebook, who will be measured on their progress against this goal in the next 3-6 months, and significant part of their monetary benefits over years depends on progressing against that specific goal of increasing page views?
When metrics became the primary decision making parameters, without people understanding the unintended consequences of metrics, what you get is Goodhart's Law.
Goodhart’s Law is expressed simply as: “When a measure becomes a target, it ceases to be a good measure.” In other words, when we set one specific goal, people will tend to optimize for that objective regardless of the consequences.
Goodhart’s Law Explained (Source)
When Metrics Aren't Possible
There are a bunch of things that a company also needs, and many do focus on, which are hard to quantify or be forced through metrics. Things like importance of design, the tonality of marketing, kind of culture they want to create etc.
Seriously, think about it. How do you justify creating a great user experience with meticulously designed interface and flows? Everyone intuitively understands the need for good UI/UX. But what's the impact of exceptional design? You can't prove these through some number crunching for a company. Heck, this is still a problem for Razorpay, with many PMs not understanding the importance of high quality design, not just hygiene quality, on long term goals of Razorpay. Strong belief and philosophy drives this right now, with hindsight making things clear.
(I wrote a bit about this in the post, how Razorpay used design as a differentiator)
Many companies do a great job at driving not so easily quantifiable things. And that's because the philosophy is very deep seated in the company. Why is it deep seated? Because the key leaders of the company, and these are almost always the founders, strongly believe in them, they have an understanding of the long term impact of these things, even if they can't prove the impact mathematically on the back of a napkin.
So why isn't this more common? Because this is a rare skill. Most of us suck at understanding and predicting the long term impact of something. We grasp at straws when forced to predict what would be the impact of a decision in a 3 year horizon or a 5 year horizon. Reduce the scope of the decision, put in a more closed setup and accuracy increases. But that's a luxury rarely available for high impact decisions.
Most startups that are successful, have founders who intuitively grasp the importance of things that can't easily be measured. And hence they still focus on these not-easily-quantifiable philosophies, creating something magical in the process. But what happens when these companies start scaling? Can they still depend on the founders to push forward this philosophy?
Side Note: This is also why some of the best founders are part philosophers in disguise like Peter Thiel, Kunal Shah, Naval, Elon Musk.
Thoughtfulness in a company
One of the first casualty of scale in a company is thoughtfulness.
When a company is about <100 people in size, it's quite easy to take thoughtful decisions like the appraisal someone gets, whether you measure leaves or not, whether it is okay to allow a particular person to go on a sabbatical etc. If the founders have the right thinking on these topics, decisions are easy to take, people are happy with the decisions and company has a great culture.
These things become much harder to thoughtfully solve when company size is >500. Coming up with person specific solutions is hard and policies start coming in the way. It gets harder for most companies to allow thoughtfulness and discretion to be applied on individual cases as they scale. Especially for those where thoughtfulness towards employees is not deeply rooted in "Company Operating Philosophy".
Even the companies with founders who are very clear in their thinking of how they want to treat their people, it still becomes an issue because as the company scales, their visibility reduces. Their thinking is implemented by the team and it's very hard to propagate a philosophy. It needs clear and repetitive communication in multiple forms, which takes time. And with so many things contesting for the founder’s attention, this ends up not getting adequate attention.
Here's what Sequoia's founder, Don Valentine, has to say on this topic:
And this is not just restricted to people practices, which is why I also say that thoughtfulness is also the casualty of metrics.
Let's take an example. Should a website put up an exit popup banner or not?
Seems like a simple decision? Let's take a guess where this idea would be coming from. One real world example is that someone in the marketing team has been assigned the goal of driving subscriber count for the company blog. After all, more subscribers = better, right?
An idea being floated is that let's put up a % scroll based popup modal asking the reader to subscribe. Basically ask them to subscribe once they read 70% of the article since they are already hooked by now. In addition, another idea is that if the reader tries to close the window, we can detect this and show a popup modal with some offer which will be available if they subscribe.
All very fair ideas that will very likely increase the subscribe count. But at what cost? Turns out this is quite difficult for people to grasp.
This can have tiny impact on brand every passing day that may add up in the long term. As more and more people see popup modal, the more they start thinking of these as cheap marketing activities. And if premium-ness is a key aspect of the brand, it smacks right against it. But how do you prove it? A company's own metrics really can't in the short term. You can look at learnings of other companies but you don't have any data of your own that will help here.
Let's say the enterprising founder of the company where all this is happening decides to block this decision, explaining the impact on brand in long term. But it's a difficult topic to grasp. Many don't understand it. And while the founder may have tackled this particular problem, there's no guarantee that decision making quality in the team around this has improved.
And when do these things become frequent problems and go unnoticed? When a company hits scale and philosophy got diluted over time because it just did not get adequate attention.
This brings us back to Social Dilemma. Extremely highly used products have unintended consequences with deep impact on society. And only way to avoid the unintended consequences from becoming critical enough to have deep and lasting impact is by us recognising this challenge and solving for it.
In this post I wanted to bring to light what happens inside a company, how some of these things are actually unintended consequences. I’m still sharpening my thoughts on these topics as well as how to best present them. I didn’t want to wait for that to happen before I published some raw thoughts.
As always, if you have any feedback, just hit reply and send it over!
There are many threads in this post that I would like to explore in much more depth in the future. For eg,
What's the alternative to standardised policies for driving the people aspects in a company
Look at how Netflix drives things. Reed Hastings has penned this down recently in his book "No Rules Rules"How do you avoid Goodhart's law hitting you when you least expect it
This article is a good starting pointImportance of being "data informed" vs "data driven" and how do you create such a setup
The first article I share whenever I mention this to anyone is: "If You Want to Be Creative, Don’t Be Data Driven"
Maybe I'll write on these topics some day, if those aforementioned friends bully me again. Over and out.
(To get future posts served directly in your inbox, subscribe to this newsletter)