Disambiguate by creating visibility

When there is a lot that needs to be done, it is common to feel overwhelmed and anxious. From the sheer number of different tasks that remain to the enormous amount of work left to complete a task as a deadline approaches, the uncertainty can become debilitating. In the past months, I have found that creating visibility into work is a great way to counter this. Not only does it reduce the uncertainty in the situation, it can help to create confidence and propel oneself and others towards goal completion.

For instance, on my team, we have had a backlog that’s been getting out of hand the past couple of years. Fixing it, however, always doesn’t make our priority list though we are all concerned that there be dragons in that backlog. Our concern mainly stems from not knowing what the dragons are. Given how busy we all tend to be, no one is able to carve out adequate time to resolve it and so it persists. To combat our analysis paralysis, we created a weekly hour long meeting during which we hoped to slowly chip at it. This ensured that we time boxed how much time we spent and let us feel like we would make progress.

To start off, we built a sense of what was in the backlog: what epics existed, did we actually use them? How long had tickets been opened for? Equipped with all this information i.e. visibility, we no longer thought that the task was daunting and were able to come up with steps to work through over the next weeks to fix the backlog. We started off by calling bankruptcy on tickets older than 2 years and iterated till the backlog got to a good state over about 6 weeks. What seemed like an insurmountable obstacle quickly became manageable through the process. Before we started, we all thought that we’d have to go over every existing ticket and that scared us but the act of creating visibility into what was really going on, enabled us to fix it.

I’ve seen similar situations arise when engineering teams are working on a large project and burning the midnight oil’ to finish it off. There is a short sighted focus on the tasks at hand and a drive to get through them. Sometimes, this is coupled with a lack of insight into the overall health of the project. You know that you are in this situation if no one can tell you how the project is going or when it will complete with high certainty. Additionally, you know this is the case when no project tracking exists or that which does exist is convoluted. Everyone is working extremely hard to complete the goal but they are also very worried and anxious because of no holistic visibility.

In cases like this, it is paramount to fight the urgency around you. In the moment, it feels as though the time needed to create visibility into the project could be better spent burning through the remaining work. While there may be situations where this is true, I think that most times it is better to spend the time creating visibility. The exception to this, being solely when the project truly is almost done.

Having this visibility lets you holistically make decisions for the project which can help track to a faster completion. E.g. It can become clear that some work isn’t necessary for launch. Further, visibility is appreciated not only by the team working on it but by all the stakeholders that care about the project. When they don’t know where a project is, they too get anxious. Imagine an infographic that anyone can quickly glance at and know the status of the project. Either things are real bad and everyone knows so there aren’t any surprises or they are good and everyone’s happy. Irrespective, it beats not knowing.

May 30, 2021

A brief interlude

It’s been a couple of months since I last published on this blog. From my last post, it’s obvious that things got extremely busy and I was attempting to curb my exhaustion and avoid burn out all together. Eventually, I made a decision to pause blogging to make what was on my plate manageable.

Things have progressively gotten better over the past months. I was able to take a week off during which I spent my days relaxing at the beach! It went a long way in helping me get back to a good place. There are many techniques one can take to reduce overall exhaustion but nothing beats a chunk of time spent disconnecting and rejuvenating.

Not all was lost during this period. I did manage to speak at the Data Summit Connect conference this month and you can see the video here. I’ve added talks to the site’s menu and will be using it to share any talks I give. Here’s to a continued manageable plate and to regularly blogging again.

May 29, 2021

Curb your exhaustion

Over the past week, I’ve spoken with several people who are all feeling and dealing with exhaustion at work. I also have had a lot on my plate recently and am currently dealing with managing my load. On top of all this, we are all trying to deal with the pandemic and it is unsurprising that we are feeling exhausted. To cope, I am attempting to curb my exhaustion.

The spectrum of exhaustion can stretch anywhere from starting to burn out all the way to total burnout. I’m no expert on the subject but these are some of the things I’ve heard from others and seen in myself in starting to realize that I may be heading in that direction:

  • Loss of motivation to do work
  • Personal relationships remain unattended
  • Dreading going to work the next day
  • Low energy to do anything else
  • Continually working late into the evenings and weekends

It is important to acknowledge and realize we are all human and have limits. Pushing too hard at something pushes us towards our limits. It is not sustainable to stay operating at that intensity indefinitely. Either you will crash, or you will have to stop or slow down to prevent the crash.

We live in a society (mileage may vary depending on where you live) where putting everything into work is often heralded. Start by Resetting that mentality. It is okay to slow down. While there is no substitute for rest and you should eventually rest, here are some things I’ve done to try manage my current situation and exhaustion:

  • Taking a day off here or there. Mondays or Fridays are best as they get you a longer weekend and more time to recoup
  • Ruthlessly prioritize, stop doing things and communicate that you won’t be doing those things. Personally this helped me a lot. There was a non negligible level of exhaustion coming from thinking about the insane to-do list I had
  • Delegate as much as you can
  • Create a regular exercise routine and stick to it
  • Talk about it with others you trust
  • Turn off anxiety provoking notifications. E.g. Slack reminders, daily to-do list updates
  • Slow down wherever you can
  • Remember you can’t do everything and that’s OK
  • Book a week or a long duration of time off in the future and take that time off

In the end you need to eventually rest so whatever you do, ensure there’s an eventual path to get to rest.

February 27, 2021

Decrease Inertia

Maintaining an exercise routine this past year has been quite difficult. We are all stuck at home and the lines of separation between home, work, exercise, and socializing have become very blurred. There is no longer an office you go to work or a gym that you get to work out in. Instead, we all have to make our homes our offices and our gyms. Creating and sticking to habits in this environment, I’ve found, to be challenging.

Prior to the pandemic, I was an avid exerciser”. I went to the gym about 3 times a week and played both basketball and soccer. Being part of leagues, playing sports and having a gym routine where I exercised either before work or after the work day prior to heading home made it easy to do. Being stuck at home on the other hand has presented challenges trying to exercise and keep a routine.

Similarly, sticking to a work routine has proved trying. The temptation to work more hours than usual due to being stuck at home, or to go back and solve that thing that you weren’t able to but that keeps nagging at the back of head. The delineation between work and home no longer exists.

In reconciling these challenges and creating new habits I should stick to in the pandemic, I realized that the same simple strategy I apply when creating habits that stick as a leader have helped me become better at keeping these habits and that is to reduce inertia.

Decreasing inertia is critical to successfully make a habit stick. In my one on ones, I employ a similar tactic to ensure the variability of one on ones. Besides the regular one on ones, I schedule monthly career goals check in, and quarterly retrospective discussions in place of them in the schedule. By having these on the calendar, when they arrive, both myself and my report know and are reminded to have the necessary discussion.

Compare this to a situation in which the onus to have these discussions is on me. I could be tired that day and forego the conversation because I’m not up for it, or I simply could forget to do it. Even if I’m not up for it, my report is expecting us to have it and so we do.

I applied the same principle to my work and exercise. For work, I use an office space separate from my room in the apartment that I go to every morning. When I’m done with my day, I walk away from that space never removing my laptop from that area. If I’m tempted to keep working, I need to go back to that space. This added inertia has prevented me from doing so. With exercise, I’ve used the same calendar approach as my one on ones. Blocked out time and calender reminders to exercise have been a great forcing function to actually do so.

If you are ever trying to create a new culture or habit on your team or in your personal life, start with the question of what’s the inertia and build a solution that reduces it.

February 20, 2021

Provide requirements not solutions

Being an engineering team that serves internal customers comes with its ups and downs. One of my biggest pet peeves on such teams is the task of customer requirement gathering. While I can’t speak for non-infrastructure, non-internal development teams, this pattern is highly prevalent on internal teams serving other engineers and technical staff at a company.

It usually goes this way. A customer using your systems has additional requirements or needs that they would like met in order to accomplish a goal. The customer proceeds to communicate this to you as something along the lines of System X needs to implement feature Y so that I can do Z. Oh, while at it, you should add feature R and fix bug Q that’s been slowing us down. In the most extreme of cases, you get sent a technical specification detailing features and modifications you should make to the system; diagrams included.

Unfortunately, the biggest challenge you then face, is that you don’t understand what specific problem the customer is trying to solve. You don’t know the exact requirements needed from your system. Sure the customer is presenting a solution to you but they don’t have the same depth or knowledge of the system. They are neither knowledgeable on all the design decisions and trade offs made nor are they typically able to propose a solution that gels with your current ecosystem.

This leaves you in a bit of a bind. It will take a long time to read through the document and understand the proposal. Similarly, it will take significant time to consider the impact of their new feature suggestions on your system. However, you can’t tell them no because they’ll ask why and you don’t know exactly why. You are frustrated that this happened yet again, but eventually suck it up, go through their requests and formulate a response.

Deep down though, you wish they’d tell you what problem they’d like to solve and then give you the opportunity to design a solution that solves it. We are trying to build this new feature and will require data X,Y,Z from our service. We’ve got an SLA to meet and so need at least an RPS of Q. Per our current tests, our current service infrastructure seems unable to support that. What do you propose?. That, right there, is the dream. Having grasped their requirements, you’d go ahead and either propose changes needed to your systems/infrastructure or explain why their request is unfeasible.

Instead, you end up going back and forth on their feature request(s). You discuss the details, and the trade offs. You explain and provide more context on your current system and then rinse and repeat. Finally through this dialogue, you grasp what their needs are and provide a possible solution. However, you are both frustrated, and have both spent a lot of time getting to this point. It could have be better.

Why does this happen? Why do attempts to be helpful end up this way? In most cases, the engineer that made the request is trying to be helpful. But, in their attempts to be helpful, they make things worse. I think that situations like this are very common among collaborating engineering teams because of what I like to call the expert fallacy, but that is more commonly known as the Dunning-Kruger Effect.

In the case of the solution provider, this boils down to a little knowledge leading to overconfidence. The phenomenon is best illustrated in the graph below in the difference between an expert and someone that is moderately acquainted.

Beginners vs Experts

The mismatch between what people in the Young category think they know versus what they really know is huge. As they believe that they know a lot, they don’t hesitate to provide solutions. I think this happens a lot in engineering because we all fundamentally do similar things: we all code, and build software. This makes it easy to assume we have a good handle of other’s systems. A good enough handle to tell them how it should evolve to meet our needs. Unfortunately, the receiving party of our proposed solution is typically an expert of the system. The expert sees all that is unknown and shudders inside.

How then do we prevent this scenario? I think there are two improvements that can be made on both sides. Firstly, to get better at dealing with customers that propose solutions, harness the skill of asking follow up questions. Focus on understanding what the underlying customer need is so you can propose a good solve. Harness your inner product manager and remember articulation of needs is hard.

On the flip side, if you are requesting changes to a system you neither own nor understand the nitty-gritty of, provide requirements for what you’d like and not solutions. Get better at articulating exactly what you need and leave the proposal of how to solve the need to the system owners. This will save the time of everyone involved and foster a good relationship between you all.

No one goes to their doctor and tells them what to do. Why then do we do that to other engineering teams? Because WebMD came along and we all now think we are doctors”…

February 3, 2021

Factfulness

I just finished reading Factfulness and absolutely enjoyed it. If you are looking for a new book to read, I cannot recommend this highly enough, especially in the current state of the world that we are in. The book delves into facts and figures, enlightening you on the reality that the world has been getting, and is much better than you think it actually is. It then goes on, similar to the style of thinking, fast and slow to describe 10 different instincts” or ways our brains work that led us to believe the world is far worse off than it actually is.

What really stood out to me reading this book, was the applicability of the 10 instincts of factfulness to leadership, decision making and building technology. As such, I will start with an overview of the ten instincts in this blog post, covering the first, the gap instinct in detail and then cover the remaining 9 in 3 future posts. I’ll explain the instinct(s) and their relevance to leadership and engineering in each post. The book’s authors hopes are that increased knowledge and awareness of them will help us all avoid making overly dramatic evaluations and I hope these blog posts do as well.

As you read either the book, or these blog posts, keep in mind that we all share these instincts. We are all human and our human nature leads us to use these instincts. They are not a shortcoming but instincts we have evolved to have. However, the world we live in now is far different than it was hundreds of years ago. The instincts that served us well then, can inhibit us now. In understanding these instincts, we can use them to our advantage.

The 10 factfulness instincts are as follows. I’ve added links to the gapminder website which provides details on each:

  1. Gap instinct
  2. Negativity instinct
  3. Straight line instinct
  4. Fear instinct
  5. Size instinct
  6. Generalization instinct
  7. Destiny instinct
  8. Single-perspective instinct
  9. Blame instinct
  10. Urgency instinct

The Gap Instinct

The gap instinct is our tendency to divvy things up into 2 distinct groups, not realizing that most lie in that middle gap. In the book, the authors focus on our gap instinct to bucket the world into the outdated, so-called developed” and developing” countries. The authors then go to show that the majority of the world is actually in the middle and this gap instinct blinds us from both seeing and realizing that.

Reading about the gap instinct, I thought about how often this comes up at work. It really is everywhere! It is the fabled them and us” mentality and is very common among us all. I manage an infrastructure team and a common challenge we face, is supporting the large and diverse group of customers we serve. Given their different and often differing requirements, we eventually have to make trade offs in forming a globalized solution to cater to the different customers. This pleases some and displeases others.

Usually, there are teams that want/have a localized solution they believe solves their problem. They have neither intention nor will they migrate to the global solution. Everyone else (them) just needs to move to their systems. Or, the infrastructure team (them) should leave them be, running their own system. The crux, and what made me chuckle, reading the book is that in most cases this is a small group and the extreme. A lot of customers are typically in the middle, and willing to make migrations provided they are made easier to do. It is however, very easy to, and I personally am guilty of of forgetting about everyone else in between. The heated discussions, and the tug-of-war with these teams not only blind us, they lead us to feeling continually frustrated, reinforcing this view.

It is therefore important that we remind ourselves of this gap. Take a step back and look at the entire picture: How many teams are glad to move, willing to move with some changes, or adamantly against moving. Are you so focused on the extremes that you are missing the middle? Should your strategy change to cater to this larger but often, less vocal group? The answer as always, is that it depends. However, knowing this reality should help you with that decision.

Apply this same methodology to other scenarios at work. In cross team collaboration, are teams bucketed into allies and enemies? Does that capture everyone? Is everyone on the teams in question really that way? Are you viewing your systems as either stable or in dire technical debt? Is this all your systems? Anything in the gap? You get the picture. When faced with two extremes at work, take a step back and ask, what’s in the middle.

January 31, 2021