Don't Get Eaten

Managing your career in software product development

Navigating a career in software engineering can be a confusing process. Different organisations have different terminologies - SDE 1, SDE 2, Staff Engineer, Team Lead, Director, VP and so on. Some organisations have different career tracks for managers and Individual Contributors….this stuff can get confusing fast. In this post I’m going to give you some simple heuristics for assessing where you are in your career and how to get to the next level, and the one after that, and so on.

The career ladder is actually quite simple and its steps can be broken down into something like this.

• You get told what to do AND how to do it

• You get told what to do

• You get asked how to do it

• You get asked what to do AND how to do it.

• No one tells you anything.

• No one asks you anything.

ie

• You get told what to do AND how to do it - Junior Software Engineer

• You get told what to do - Senior Software Engineer

• You get asked how to do it - Staff Engineer

• You get asked what to do AND how to do it - Director

• You only get given Goals - VP

• You decide the Goals - CXO

• No one asks or tells you anything - God

The mapping is, of course, fuzzy but provides a model for managing your career. Another view would be this

You are somewhere on this map.

At the lowest level of Complexity you deal in solutions for a given set of problems. At the next level you set yourself the right problems to achieve a goal. At the outermost level you get to choose the goals.

In the end the amount of value you create is directly linked to what level of complexity you operate at. The highest level of complexity that we as product builders can aspire to is to build a profitable product - to provide more value to millions of people than it costs us to provide it. Such a business, backed by a strong brand, can be seen as a complex life form. When it obeys your command you are very, very hard to eat. This organism lives on the substrate of entire human civilisation, the Courts, the State and its ability to enforce its Laws, the needs and wants of a large population, the Financial System and complex human society. But in order to get there you have to start from where you are and to establish this simply ask which of the layers seems most like your life. Then, begin to peel back the layers of complexity.

Life Feeds on Life

This might seem like a strange thing to say when discussing one’s career, but it is the underlying principle behind all progress. In our Universe and on our Earth we see the wonder of Creation and there is one thing that is observed commonly, everywhere - that the energy of the sun has been constantly used by Life to become more and more complex. The single-celled organisms who over billions of years became Homo Sapiens did not just grow more complex biologically but at some point the human brain started manipulating abstract concepts and thus began a journey into the infinite complexity of the mind. We didn’t need biological evolution any more. We had memetic evolution which goes much faster. We used it to invent amazing technologies - the wheel, the sail, writing, electricity, the Internet - and we continue to do so, to build new products and services at an astonishing pace. You dear reader, somehow, seem to have gotten caught up in all of this. You might think that your problems have to do with learning a new programming language or with hiring more developers but at the core what you’re engaged in is the infinite battle of Cosmos vs Chaos.

And in all of this battle, we see that one inviolate principle - an organism will only use for food another organism simpler than itself. And so your basic job as a participant in the memetic war against chaos is to feed on things less complex than you and grow ever more complex. This is why the post is called ‘Don’t Get Eaten’.

And the corollary to this is that you have to stop thinking of your career in terms of roles, jobs, brands and so on and see it as going higher and higher up the layers of Complexity that make up our business and our life.

Along the way you will meet three bosses. Your mission is to fight and defeat them. Nay, your mission is to consume them whole and transcend the level at which they operate.

The First Boss

For Software Engineers, the first boss is Computation. To go from being told “what to do and how to do it”, to only being told “what to do” requires you to have consumed the complexity of Computation, to some extent at least. And to reach the next level beyond that you will need to have a fundamental understanding of Computation. You will need to know how to reverse entropy with your program - ie reduce chaos without increasing it somewhere else. Else your computer program itself will fall victim to chaos - the fabled Technical Debt that has brought so many bright ideas to a grinding halt.

There is a lot to study here. Understanding computation through the abstractions provided by computer languages is hard but if you remain rooted in syntax and tooling you will miss the big picture. Go deep. Understand type systems, homoiconicity, SOLID principles, functional programming, the power of TDD, algorithms, data structure and so on. Much of this you probably learned in college but the application of this knowledge brings a different level of understanding. Practice then is how you deepen your understanding of the issues involved. Understand that programs are life forms. They suffer from entropy. Build robust programming habits that blunt the attack of entropy. Understand how programming abstractions help you to do this. Don’t write 3000 lines of code in a single file. And so on.

When you get reasonably good at this you’ll stop being told how to do it and with any luck you’ll be passing on this knowledge to fresher recruits. Some people management and project management complexities will lurk in the background but they only rear their heads at the higher levels. You’re good for now.

But merely getting good at computation does not guarantee anything. To create value in the world you also have to choose the correct problems to solve. There is a layer of turbulence between the layers of Computation and Value Creation that marks an important turning point in your career. This is where you decide if you want to work on formulating better solutions or formulating the problems to which solutions are being sought.   To decide this you will have to decide how much ‘fuzziness’ you want in your life. If you’re good at Computation and get a thrill out of orderliness and elegance of construction, you might want to focus on that. If you like ambiguity and vagueness and get a thrill out of delivering value to users you might want to move one level up in the Complexity ladder and work on problems instead of solutions.

This is the IC/Manager dichotomy. The IC goes deep into the puzzles of computation, making the lever ever longer. To create value, merely having a long lever is not enough. Knowing where to place it is an equally important skill.The Manager track makes you better and better at deciding where to place the lever.

The Second Boss

The Second Boss is Value. Value is always bounded by a context. A glass of water has a different value depending on where it is - the last sip of water in a desert is as valuable as life itself. And at this level one seeks to extract value out of contexts by building stuff that is valuable in that context. When you get very good at building levers you move on to figuring out where to place the lever and what to move with it. The end goal of the system is a critical input into this calculation and so one has to take a step back from the system itself and understand how this system fits into a larger system. Most Product Managers never have to deal with the lower levels of Complexity but for Engineers it is impossible to get here without figuring out the lower levels.

This judgement — where and how much to insist on performance, for example, or how to achieve the same end goal with less effort or less risk — is the stock in trade for the middle levels of your career. Just like you studied how abstractions such as Classes or Functions interact with each other to form a system, you now how to study how the system itself interacts with the fuzzy messiness of the Real World.

Some of the key skills required here are Problem Classification (using tools like the Cynefin Framework), Risk Management, Project Management, RoI Calculations and so on. In addition you will need to be able to build Fast Feedback Loops, use Outside-In Thinking and be able to guide yourself and the project reliably to a target in the given time.

Here you will also deal with increasing Human complexity. Keeping the team aligned and motivated is no longer a problem you can ignore. Additionally, since the projects tend to be longer, you will also have second order organisational complexities that you will not yet have the authority to fix and will have to navigate around.

However by this time you are starting to get to grips with how to extract Value given a context. And this then leads to the Final Boss - Meaning.

The Final Boss

Human Consciousness is the most powerful organising force in the Universe because it and it alone can provide Meaning to it. The Universe is only Chaos. It is the human consciousness which provides it Meaning.

They say that the only job of a CEO is to make sense of the world for the company. Where to focus, what to ignore, which narrative to use to explain the facts — all these are first class concerns of VPs and CTOs.

At the VP Engineering Level, one deals with providing not just Technical Leadership but also with providing Meaning to the Engineering teams. Who are we? What Value do we choose to provide to the world? These might seem like fuzzy concepts but yeah, there is no linter for the Real World.

Everything is a mess and the only question to ask is is it a useful mess? Does it provide Value? Does it have Meaning? Its messiness is a function of it being alive. And it will get messier every day unless you can instil in your team the values of Excellence, of Discipline, of Empathy.

A VP Engineering is tasked with setting and executing strategic vision. It’s like a CTO role but with a more limited scope. The VP runs the full stack from diving meaning from the cryptic clues that reality offers, to translating them into engineering projects, negotiating for resources and executing them in time to stave of the crushing blows of Entropy, which at this level punish mis-judgements with extinction in the worst case. The Blast Radius for a VP can deal permanent damage to large sections of the org and the whole org itself. VPs have to constantly ask themselves the question - What Does This Mean?

The CTO has one job - to negotiate reality with the rest of leadership and this is done by choosing the correct Goals. To choose the correct goals they will have to decide what the Universe means, understand which reality to bend with which lever and in which direction. Business goals, Product goals, Human goals, technical constraints - all must be managed. In order to do this the CTO must understand Computation better than anyone in order to produce the right sized lever, must understand where to place the lever from a distance, must manage risks, must keep the people productive through a vast toolkit of incentives and disincentives. The CTO is the gatekeeper of the culture. What they allow is what becomes the culture of the company.

A wise man once said to me — the only predictor of successful execution is a cohesiveness of thought from top to bottom of the organisation. The CTO is responsible for this cohesiveness of thought in the Technical Organisation and is a stakeholder in such cohesiveness manifesting in the entire organisation.

As you can see, the Final Boss takes no prisoners and provides no clues as to their whereabouts and how to kill. Most CTOs have fought this boss and lost multiple times in their career. But don’t worry - the boss is always there, waiting for you to have another go with an updated mental model, nay an updated visceral understanding of reality. In a technological organisation the CTO is more of a Professional Philosopher than an Engineer.

This is why young ex-CTOs of even failed start-ups are valued in technological organisations. It’s because they have practiced the art of divining Meaning from Chaos.

As Picasso once said - computers are useless. They can only provide answers.

To find useful answers it is important to ask the right questions. Only from these questions and their answers can Meaning emerge.

In Conclusion

With any luck, by now, you’ve formed a mental model for what kinds of work happens at every level of your career ladder. Your mission, should you choose to accept it, is to observe, orient yourself as to what layer of Complexity you’re fighting with and go and conquer it.

And when you get to the end you will realise that there has on this entire journey been only one boss - Meaning - manifesting according to their appropriate forms at each level. Seek out this Boss, let it kill you, over and over. If you finally prevail, the whole world will be yours.