My main goal for 2016 was to triple my output, primarily by focusing high-level processes, as opposed to e.g. things normally lumped under productivity tricks. By any reasonable measure this was a major success–for example, the team I manage at work saw a 100%-300% increase in per-person productivity depending on how you measure it, and we scaled from 2 to 7 people. This is unusual–per person productivity in Data Science usually goes down when you hire more people, especially when they’ve never held the job before.
I now have a better sense of exactly what I was trying to do, and the major obstacle I had to get past: I needed to think clearly in a space with lots of bad advice. Nothing I did was particularly clever, every optimization I made seems obvious in retrospect. And yet, the world is ripe with opportunities for these trivial improvements: an ocean of bad business advice drowns out the good. Furthermore, this information tends to be presented as many small “nuggets” rather than one cohesive theory, which makes it incredibly difficult to apply to new situations.
Personal Productivity – 0 Dead Weeks
While I didn’t focus too much on my individual productivity this year, I started reaping benefits from some work I’d done in 2015. Historically, it’s been pretty typical for me to have a few weeks a year where I accomplish essentially nothing. This year, the longest dead period I had was two days straight, and even those were pretty rare.
A simple, but important factor is that I started to take Tyrosine last year, and seriously upped my dosage whenever I felt depressed and unable to act. This didn’t have much effect on my mood, but it made me noticeably more able to get things done. I would estimate that I was about 70% as productive in a “down” state as I was in a normal state, which is a huge improvement for me–previously, that looked more like 10%.
From previous jobs, I had a habit of waking up at 7 AM every day, which I’ve started to realize was quite bad for my productivity–no matter how hard I tried, there would always be days where I fell asleep late, and even a small amount of sleep deprivation crushed my productivity. But I’ve had a lot of flexibility at Euclid–some days I’ll arrive at 7, some days at 11–and that means there are almost no days where I have to work while seriously sleep deprived.
Up until 2015, I found it pretty easy to motivate myself to do things that involved learning or directly helping my friends, but hard to motivate myself to get something 100% done. Late last year, I managed to connect my fundamental motivations to directly accomplishing things in the real world, and this has remained pretty stable.
Iterations & Feedback Loops
A substantial amount of my improvement involved learning to really see iterations and feedback loops everywhere, and optimize them. Examples include how people learn, debugging, building a software project, converging on product-market fit, understanding someone’s motivations, and so on. It’s hard to describe this at a high-level without stating something completely content-free, so I’ll talk about several specific cases.
My First Task at Euclid
When I joined Euclid, my very first task was to investigate a bug where a calculation in a long data pipeline produced the wrong answer. There were several things that made this difficult:
- The codebase was primarily in Scala, a language I had never really used before joining Euclid
- I was completely unfamiliar with the Euclid codebase, and the engineer who had written all of the relevant code was no longer at the company
- I had to set up a development environment, and there wasn’t much of a standard process
- The process that produced the incorrect calculation took several hours to run
- Since the bug was simply a wrong number, rather than a process crashing somewhere, there were a lot of different ways it could happen
- The algorithm that resulted in the bug was highly statistical and non-deterministic, so it was difficult to examine intermediate results and find the issue
- And we had to solve the bug within two weeks, or lose a valuable customer
Piece of cake, right? In 2014, I would have tried to solve a similar bug by diving into the codebase and looking for something that appeared to be wrong. Here, I took a different approach:
- First, set up the development environment well enough that I could reliably run the process and produce the bug. This took about 3 days.
- At this point, the process took 6 hours to run, which was far too slow for any sort of iterative experimentation. So I spent some time optimizing it, getting it to run on just the single customer, and ignore parts of the nightly pipeline that weren’t relevant to the problem I was trying to solve. This took another two days, and I got it down to 15 minutes per iteration.
- From there, it was just a matter of trying things: the relatively fast iteration speed made it easy to make a lot of small changes in order to see what affected the final result, and it only took one more day to find the actual issue and fix it. (It turned out that I had to change one number: a “-95” to “-85”.)
There are a few interesting things about this approach: first, most of the work I did in the first 5 days was applicable to other projects–I was able to solve similar bugs in less than a day later. Second, in some sense this was a very low-risk method: once I’d gotten the fast iterations, even if there had been 8 times as many configurations to try, it would only have taken about 3 times as long.
A much simpler example of a change I made was noticing that our tests took 2 hours to run, which slowed down iteration tremendously. Furthermore, it encouraged people to make a small number of large changes as opposed to many small ones, which is pretty bad practice. I went through the tests and which bugs we’d found, and it turned out that the two slowest tests took 95% of the time and had not been responsible for finding a single issue in the last 6 months. So I deleted them, allowing everyone to work faster.
Risk Driven Development
Most iterative loops focus directly on solving some problem. For example, many people plan software projects by figuring out which components need to be built in which order, then trying to build them. Our team took a different approach: we tried to figure out which parts of a project were most uncertain (if any), and then prioritize taking actions which reduced the risk. Only when we’d done all of this did we start actually building a project.
For example, one project’s main risk was that it required us to use a large number of new technologies, many of which were somewhat immature, and which the team had little experience with. Some of these technologies wouldn’t come into play until the later stage components of the project, but we started by building proofs of concept for each one. Out of 9 tools, we found that 5 worked as expected, 2 had issues which forced us to make some substantial architectural changes, and 2 had such severe issues that we had to replace them with alternatives or implement our own versions.
Doing this ahead of time probably cut down at least one third of the time we would have spent on the project–without it, we would have had to make some very substantial changes very late in the game.
It’s worth noting that Agile/Scrum is a special case of risk driven development–they assume that the biggest risk is details of customer requirements and user interfaces, and focus on getting something into the hands of end users in order to get feedback ASAP. This makes a lot of sense in some types of development, but fails badly when requirements are pretty simple to state and implementation is the hard part.
This year, I started my first full-time management job. While I started at Euclid as a Senior Data Scientist, it quickly became clear that the team really needed a manager, and that I was the natural candidate for the role.
One thing that surprised me was how easy the transition was. I had heard that engineering skills generally don’t translate well to management, and that you need to start over–but that wasn’t my experience at all. Managing the team was a systems problem, where the most important step was repeatedly just analyzing a class of problems, figuring out ways to avoid it, and introducing norms or processes that dodged the issues.
Building Trust Between Business & Engineering
In some organizations, Business sees Engineering as a bunch of kids more interested in playing with toys than meeting company goals, while Engineering sees Business as dishonest used car salesmen who don’t care about building a good product and will do anything for a quick buck. This is generally an unproductive state of affairs.
One major issue that results from a lack of trust is that Engineering has a hard time convincing the Business to allocate any time towards technical debt or work that isn’t directly user-facing.
Euclid wasn’t anywhere near this bad, but there was still a substantial gap between the two sides–neither seemed to really understand or appreciate what the other did. So early on, I focused on bridging the gap at least with Data Science, by learning to speak the language of the business, and talk about its high-level goals. Being able to relate specific ways of paying down technical debt to meeting Euclid’s goals for the years made it much easier to get everyone on the same page, and to be able to spend most of our time solving problems rather than playing politics.
One common anti-pattern is for a manager to become the roadblock in a team, by requiring that they approve every major decision: for example, Marissa Mayer famously required that all changes to the Google Homepage UI go through her, resulting in huge lines of people lining up during “office hours” each week to get approvals. I think this is a terrible idea: it trains people to stop thinking for themselves, and to pass responsibility to someone else.
Instead, I try to provide my team with information and norms, and have them make most of the decisions. One example is the UNODIR pattern: having people say “I’m going to do X at Y time unless otherwise directed.” This moves people into a state where they have approval by default, and can feel like they directly own the results of their work.
More generally, my view is that a manager should be a force multiplier of a team, rather than a necessary component. While I try to help wherever I can, any time my input is needed, that’s an opportunity to improve the system.
Most business advice is terrible, and even the advice that isn’t terrible is presented as many small disjoint pieces of information, which makes it hard to adapt to new situations. In order to make real progress, I needed to develop a theory of organizations–a set of many small, cohesive principles that could be combined to produce answers in a variety of situations. A great example of a small theory is Newton’s three laws of motion, which (when combined with a few empirical measurements) allow you to predict the answers to a huge number of phenomena.
I don’t really have a good way of describing my process for coming up with theory, other than “writing and thinking a lot.” But I moved from primarily seeking advice to primarily trying to develop theory, and this was hugely effective–essentially every individual insight above came from me practicing thinking by myself rather than trying to directly apply external knowledge to my situation.
Most of the things I mentioned above seem obvious. There’s nothing brilliant about saying “in projects with high uncertainty, start by reducing uncertainty”–it seems almost content-free. So why did this make such a big difference? Why was Satvik of 2014 not able to do these things?
I think the biggest mistake I made was systematically underrating logic. Thinking works. Historically, I tried to solve business problems primarily by reading and asking others for advice. This year, I primarily tried to think about the problems myself, and develop theories around productivity, organizations, and learning. I find myself repeatedly coming to the same realization in many areas of my life: outside of the hard sciences, the level of knowledge in many domains is quite poor, to the extent that throwing most of it out and reasoning from first principles frequently yields dramatically better results.