co-located teams in team-room like settings are much more productive
Whenever I've seen this done, the very best employees leave. I am a firm believer that you can train yourself to hop in and out of a reasonable facsimile to the "zone" in much less than 20-30 minutes (I do it all the time)[1]; but the level of distraction of the warroom type environment massively favors your more extroverted engineers. My experience has been the engineers I really want to keep, the ones who solve the really hard problems, are much less likely to be in that group.
In short, you may double your productivity for building that CRUD application, but when you have to solve that really hard problem, you may find you don't even have an engineer who is capable.
1. There is a level of concentration, the "real zone" below it that most development doesn't require. However, when it does, the interruptions become catastrophic.
It depends entirely on the kind of work being done.
If you've got a super-top engineer who can solve the hard problems, then they're probably working on something of their own, and can be off in their own room / at home / whatever.
But in my experience, that kind of "hard", mentally intense, non-collaborative work makes up only a tiny, tiny fraction of commercial software development (although it contributes a great deal to the value). So it's not something to base general development practices around; it's the exception.
As you say -- "double your productivity for building that CRUD application" -- that's exactly the point, that's what most companies need, and why most companies aren't interest in remote working.
When they suddenly need a top-level engineer to solve a hard problem, they hire one who consults for three months, works remotely, and visits once or twice.
When they suddenly need a top-level engineer to solve a hard problem, they hire one who consults for three months, works remotely, and visits once or twice.
You've essentially described every job I've had since college. That's exactly my experience of how companies actually work in practice.
I'm constantly learning, and seeing when things become "possible" with the current state of knowledge. Whenever I see that, on my own, I'll sketch out how the solution would work.
Then, I look around for companies that need a problem I already have a solution for solved, and I pitch my solution to them. Sometimes, I'll take jobs to solve problems I haven't considered before, but that's rarer.
If they like my solution, I set up a contract to implement it for them.
I'd say about 50% of the time I pitch a pre-existing solution, I get work. What's great is I'm essentially picking my own work and I get paid well (because the problems are hard).
The downside is you pretty much always have to be coming up with new solutions to things, and stay very current on technology.
I'm generally into something long before the "early adopters" get there, and I'm gone by the time it hits the mainstream. For example, I was working on a Google Spanner-like database while Google was doing the same.
At the time, no-one thought something like that was possible (outside of Google -- and they didn't tell anyone). I implemented different parts of the solution with three different clients over a two year period, none of whom hired me to create something like Google Spanner, but all of whom needed one part of the whole package solved.
So, in the best interview tradition, what do you read to keep up to date with the state of the art?
I have a couple of times found myself inventing solutions to problems I had in the business, only to see them appear as cutting edge open source at the same or a little later (ie Python deployment tools)
I have always assumed one needs to be working in an area, pushing hard and then finding that "what no-one has an answer to this!?" moment.
I would have trouble believing I could find a solution at the cutting edge, and find the people who needed it, without being knee-deep myself.
As you suspected, I'm developing solutions in the areas I work in specifically (stuff I'm "knee-deep" in).
I read the usual CS papers, plus lots of PhDs thesis. I follow the references in the bibliographies like crazy, and I also look up the writer's on the 'Net and see what other stuff they've done/written.
It feels like detective work, actually: the most-cited papers are frequently not the best. There's tons of researchers working in obscurity but who nevertheless have useful results.
I see my role as taking advances from research and bringing them into practical use.
It feels like detective work, actually: the most-cited papers are frequently not the best. There's tons of researchers working in obscurity but who nevertheless have useful results.
What are some examples of the best research that is obscure?
Sven Havemann's PhD on "Generative Mesh Modelling"[0] is brilliant. At this point, it could be implemented on top of Blender 3Ds newest modelling kernel.
In general, stack languages are great for graphics in both 2 dimensions (PostScript) and 3 dimensions (GML)[1].
If you have experience solving hard problems that helps. I had the good fortune of doing so in graduate school. That lead to a job solving another totally unrelated hard problem, with a technology stack totally unrelated to my graduate work. Once I had two gigs like that, the only people who responded to my resume's were people looking for someone to work with immature technology, or had a very hard problem to solve.
Also once you decide that you want to have a professional career jumping on technology grenades, do not work as an employee. I made that mistake. The nature of grenade jumping is that your are asked to solve a hard problem that needs to be solved in a short time, if you are an employee this amounts to massive hours, (70-90 hour work weeks), of unpaid overtime. Work as a contractor and be sure to bill for every hour. Or be sure to get a very healthy option grant if it is a promising start up. My life has been much happier once I started doing that.
Also be sure to work out, this sort of work takes a severe toll on your health if you do it for 20-30 years.
I've been in more or less the same situation, but for longer periods of time.
I'm curious given your experience. My biggest issue is that I don't know what a 'hard problem' is, I know what comes naturally to me, and what takes me some time to grok, but my experience is that those problems that come natural to me seem to be extremely obtuse to others.
So what types of problems do you solve that allow you to market yourself in such a manor?
I assume you have NDA type structures in place so I expect very general responses.
Most importantly, not every project (nor startup, nor business) needs "the very best" engineers. Not every startup is changing the world, and even among the ones that are I'm sure you can pick out a few that aren't solving massively complex technical challenges that will be discussed in lecture halls for decades. The majority of good startups are just making money (sometimes not even that) until they get acquired.
Secondly, you're assuming that the best developers are going to be more introverted than the developer culture/profession as a whole, which I don't think is accurate.
> Most importantly, not every project (nor startup, nor business) needs "the very best" engineers.
This is highly underrated wisdom.
If you focus on always hiring "Rockstar developers," you'll find yourself in a miserable situation of competition and egotistical conflict. Hire your team, not your developers. Get your team working like clockwork with good systems and good relationships, and your company will be rockstar with whatever people you throw at it. Look at github, stripe, twitter, facebook—you think they only hire the absolute best developers? No, they have a top 2% just like you will—your goal should be to increase the competence of your entire company, not just ignorantly hire the best and fire the worst and hope for the best.
Spot on.
You're right about the other assumptions too (though the parent may not have been that serious about them). IMHO stereotyping introverts or extroverts as being better or worse at certain tasks is a huge --ism to avoid, especially in our field. I see 'extrovertists' and 'introvertists' battle it out all the time with their propaganda and ignorant beliefs about people, and no one wins.
It's far better and more productive to be a humanist and find a good balance. Both extroverts and introverts can be valuable to your team; both extroverts and introverts can be great programmers and workers. You should strive to hire a mix, or stop using it as a metric and naturally get a mix, since these traits like many others generally fall on a normal distribution.
Don't trust your assumptions about yourself, and don't apply them to others. They're probably wrong.
Totally agree - imagine you could hire Linux Torvalds. He is a great programmer but his best work was sending emails to others for 12 years.
This leads to an interesting thought - if you hired Linus and he started spending all his time emailing sarcastic notes to the other developers, how quickly would he get fired for "not focusing on developing code"?
Perhaps the key for CEOs is not to grow culture but to hire people who will grow it for you. You dont get to choose the culture, or even direct it. The funny Finnish bloke will.
You just pay him.
... and also he created the kernel, and a way to manage its development better.
I don't see the problem. People who make cool stuff and solve problems in their spare time are the ones you want working for you. Even if they waste time sometimes going off on tangents, it's probably honing their skills and improving something relevant to your company or their own work abilities. No problem, you want that. Trust them to do the right thing.
Exactly. This is where having good engineering leadership can come in. A "mediocre"* developer can often be coaxed into producing great product with a little support from things like paired programing and code reviews. You'll probably increase his/her value and abilities in the process.
So you need your lead to be awesome. But the rest can just be good people who know how to code.
* this would be someone who may not fully grasp all the concepts and sometimes writes bad code, but is open to learn and listen. There are developers who write bad code and are arrogant/confrontational about their abilities. Those should be identified and fired as quickly as possible. They will sink your ship.
The problem here is that the awesome lead will not stay long in a company where all other people are mediocre. Awesome people want to work with other awesome people. There should be a critical mass of them to keep them from leaving.
This is relatively true, but at a small company (< 5 programmers) some people do like the role of "lead" and will stay with it as long as the people they work with aren't total idiots.
> Secondly, you're assuming that the best developers are going to be more introverted than the developer culture/profession as a whole, which I don't think is accurate.
I don't think that's the assumption at all. My personal experience is, unless you happen to be located where one of the "best" developers is, they simply won't move. So you have your choice of them working remotely, being lucky, or not hiring them.
> Whenever I've seen this done, the very best employees leave.
I took this to mean (a) be relative to the office specifically, not the industry generally, and (b) that those very best will get sick of the required face time, quit, and get a better job (the insinuation being that that better job will likely be majority or entirely telecommuting. I may have misunderstood the intent as I commented pretty much immediately after reading it.
I believe the point was that people detest that working environment, and those with a choice will pull the ripcord, whereas people who can't get another job will stay. I've seen that happen.
I just generalized it to my experience, which is less about introvert/extravert and more about "bad conditions force good people out/prevent good people from being hired"
> ... favors your more extroverted engineers. My experience has been the engineers I really want to keep, the ones who solve the really hard problems, are much less likely to be in that group.
I, too, read "group" as "extroverted engineers," and I don't really see alternative readings.
I don't think extroverted helps in a disruptive team war room. Most "collaboration" is someone interrupting to ask a question they could have asked in email and waited for an answer.
This sort of thing is the thing I'd love to see more evidence about.
Getting interrupted kills my flow. I'm less productive as I get back into flow (although I have techniques now that let me get into flow much quicker - but that's a separate post).
However - is that drop in productivity outweighed by the increase in productivity of the interuptee not having to wait / switch tasks / fumble on and make a mistake / build the wrong thing / etc.
Times when I've been put in a war room were absolutely infuriating for me, but the tighter loops in bringing people in on what you're doing got the "why not just do X" questions answered much faster. As much as I disliked the interruption, I'm more thankful for all the code that didn't get written because we talked it over first.
If I were to estimate, this probably doubled efficacy of our junior devs while halving the (direct) productivity of senior folks. In my cases, it was immediately apparent that we came out ahead: the former outnumbered the latter by several times.
This pretty much matches my experiences. I went from a sceptic to a believer in co-located team rooms after seeing this a few times.
Did you also see the incidence of "bad" interruptions drop as the team got more experienced? That's what I've seen pretty much every time I've worked in this sort of environment.
> If I were to estimate, this probably doubled efficacy of our junior devs while halving the (direct) productivity of senior folks. In my cases, it was immediately apparent that we came out ahead: the former outnumbered the latter by several times.
I really like this. I consider myself an intermediate developer, and can't even count the times a 5-10 minute conversation would have saved me an entire day - or week! - of wasted effort. I think this is easily overlooked because its not totally obvious that interrupting a highly productive, highly skilled developer could result in higher team productivity.
Generally I think about flow in terms of setting up intermittent reward cycles where I need to put in a little bit of effort, but not too much, to get the reward.
So I think about ways I get get back into those cycles quickly. To do that I try and cut up my work into the right size chunks, have a plan, and leave work with an "easy win" so I can get back into the loop again quickly.
Some examples. Hopefully I'm not descending too far into life-hacker wankery here ;-)
* I TDD code, and always try and leave myself with a failing test. If I'm interrupted and don't have a failing test I will deliberately break something or hit undo enough times to get a fail so I have an easy win when I get back.
* I don't track time on task. I do block out time for tasks - and track non-relevant interruptions during that time and see if there are ways to stop 'em happening.
* I run a personal kanban board for stuff so I can keep that continual ping of reward happening during the day.
* I breakdown tasks as they come in so they I can get little reward pings on a regular basis.
* When I have real problems getting into flow I give myself automatic rewards (do 20m on X then you can do 5m of fun on Y). Similar to http://en.wikipedia.org/wiki/Pomodoro_Technique. Once I get started the normal task-achievements often mean I don't actually end up doing Y - it's just a personal hack to get me started.
Basically - fake the reward cycle until the real one takes hold.
The reward technique never works with me. Because, to me, there is not 5m doing something fun. I cannot have something really fun in 5m. So I rather finish all the things in limit time and enjoy the rest of the day, it's the biggest motivation.
My only work solution to get back to the flow quickly is leaving something undone. Depend on which task, it will have different ways.
Btw, I really love the idea of leaving failing test :D
Your use of "the really hard problems" is revealing. Those of us who went to engineering school know exactly what you mean by that, because it's the way we describe the most challenging exam questions, projects, or open questions in the field. We can usually figure out those really hard problems if we put our minds to it.
But for most tech startups, I'm not sure "the really hard problems" are the technical ones, because we are good at technical problems. Perhaps the really hard problems are how to get early traction, how to build something people love, and how to make the business profitable. Some of those hard problems might be easier to solve as a face-to-face team.
I think it depends on how you run the war room. The way I prefer it is no off-topic conversation, no distractions, inside voices, and a lot of respect for your co-workers. Also, you need to provide convenient nearby meeting and lounge spaces so that people who want to do those things can easily do it elsewhere. I think that's a great environment for focus.
It takes some commitment to make it happen. Without everybody committing to a low-distraction workplace, volume is definitely the default. But I think the benefits of a team environment make it worth the effort.
When it comes down to research vs. your subjective experience influenced by your personal preference and opinion, why would I put more stock in the later?
I don't think that's being fair to the parent post's point. They're implying that the research, by focussing only on a productivity metric, may miss out on job satisfaction issues that cause top performers to leave. They're not contradicting the research, just pointing out that the issue is complex.
If that's the point, then I'll agree that the issues is indeed complex. This all highlights the need for more research in an age of better tools for videoconferencing and cloud project/file management.
As a management major (do regret) I was taught that it only works when there are clear definitions and metrics for success, especially routine tasks that require less creativity. I don't see that backed well by modern research, and I wish we had better research.
I also feel the anger people (especially programmers, it seems) feel about not being able to work remotely is misplaced. If it upsets you that they want people to work face-to-face, you're probably not a good fit for the company culture that requires it anyway.
Then how do you explain the many happy teams I see working in exactly that environment, and how do you explain the research?
(Part of the confusion may be that "war room" and "open plan" are not synonyms. The former only includes people actively working on the project. This is not the environment where you have the sales guy yapping on the phone.)
I've worked in the "war-room" style environment, it's okay as long as it's only for work, and not play, because when you have the "game-room" the "work-env" in the same place, productivity goes way way down. It's hard to work when you see your co-workers playing table tennis or foosball, especially at how loud foosball is or tennis is when they laugh etc. I'm not against these things, I just think that companies might consider spending a little more money on an office or atleast put their game-rooms in separate areas, and require a library policy in open-warroom offices.
Introvert programmer here. I find the opposite to be true. In an office situation, I'm much less likely to intrude on someone I need to communicate with, and they are much less likely to intrude on me. In an open environment, I get to overhear conversations that then demand my input. Yes, on occasion, I have been interrupted while attempting to formulate a very complex solution in my head, and yes this was frustrating. However, the benefit of all those overheard misconceptions and subsequent explanations vastly outweighed them.
That is why I firmly believe that given any team, it is best to have them collocated in a large but private room.
However, we are not talking here about the optimal location for any given group of programmers. We are talking about finding an optimal group of programmers without having to exclude those outside a 20 mile radius of your geographical location.
I've seen the opposite. When the team is that close it is difficult to hide your ineptitude. Team members that don't contribute get outed pretty quickly and tend to leave. This is generally viewed positively by those that are core contributors.
Sometimes solving the hard problem means jumping into a teamroom in front of a whiteboard and doing some serious brainstorming / design. That is the one thing that is very hard to do effectively over say skype. Often with remote developers this type of problem is solved by a single person without the input of others. Resulting in a less then optimal solution that only one person is knowledgeable in.
As far as really getting into the "zone". What I do is put my headphones on, pipe some white-noise in and in a minute or two it's no different then working by myself in my home office.
I think the hardest part of product development is deciding which hard problem to solve. A team can often be better than a single very smart person at doing that, by virtue of bringing different perspectives to the conversation.
IMO the best working environment places the team in conversational proximity, but provides an option for individuals to find isolation (either in a dedicated room or at home) when it is necessary to solve a hard "real zone" type problem.
The most productive team I worked with, and one of the most senior teams within an organization of several tens of thousands with a very significant development staff, was approximately 50% remote, depending upon how one measured and when one included some remote developers and operations people (also very senior) who were intermittently present depending upon project needs. (Some of the operations people were often on site but located far enough away that much communication was as if they were remote.)
And its trend was increasingly remote. Members sought this once and as they could, because it increased their effectiveness so much. (And lowered costs and total time demands -- e.g. commuting -- in positions that were already delivering far more than 40 hours / week.)
This team outperformed the other teams by a fair margin. And some of the truly remote members were among the most efficient and effective I've seen.
Throughout my career, I've seen very little evidence that physical presence -- colocation -- makes a significant positive contribution in software development. I am biased, I will say, in that I both prefer and need a quiet work environment. I feel validated or "justified" in this need, for me at least, in that I've consistently been one of the most effective employees in the organizations within which I've worked. (And I've had numerous people in all sorts of roles tell me this.)
I don't mind colocation, if quiet offices are provided. However, cubes or worse are largely de rigueur, these days, and even in such open space environments, I've witnessed not just the destructive nature of noise and interruptions, but depending upon the staff member enormous amounts of time wasted on "socializing" of various (very off-topic and sometimes inane) sorts. Doubly bad in that context, in that it invariably distracts surrounding employees who are trying to work. And it builds a conflict between not betraying teammates and one's own need to have them quiet down or take it elsewhere.
As for who tends to "thrive" in such environments? Those who "multi-task", juggling lots of things shallowly to the point of making many mistakes and oversights. Those who are constantly "interacting", meaning most often that they are interrupting someone else to solicit knowledge that they should have or be able to look up themselves. Those who use the concept of "team" to disperse and deflect individual responsibility.
Granted, I and the people I've described may be a minority. But in my observation and perhaps in staid corporate environments, those who benefit from colocation tend to be those who need to be herded and hand-held. Those who want and are really able to get shit down, don't mind face to face but are greatly frustrated with pointless and distracting face to face.
(As an example of this latter, we got many of our meetings down to 15 to 30 minutes, when they were necessary. People knew and held themselves responsible for knowing what was up, and the meeting could quickly focus on problem solving and setting a specific course.)
Granted, I and the people I've described may be a minority. But in my observation and perhaps in staid corporate environments, those who benefit from colocation tend to be those who need to be herded and hand-held. Those who want and are really able to get shit down, don't mind face to face but are greatly frustrated with pointless and distracting face to face.
I too don't know what the numbers are like. I've seen great teams do great things remotely. I've seen great teams do great things co-located. Some of the very best teams I've seen have been co-located. I've also seen some distributed teams fail miserably. I've seen teams co-locate, try remote working, found it didn't work, and co-locate again. And so on.
I wish I knew the relative spread of how these things work out for people in different contexts and on different projects. I just find it suspicious that remote work is often presented as all upside since:
a) In my experience things are rarely all upside - I always start looking for the downside ;-)
b) For whatever reasons a chunk of the dev world tends to be a tad asocial. I know I can be like that. So when something comes up that supports that tendency I wonder if I may be overlooking problems because I would like something to be true.
I think that some in the development world may be less indiscriminately social.
I'm plenty social. And I'm not a snob -- I can talk with most anyone, about genuine interests (interests on either party's part, not just my own). Or just sit around drawing spirals in the pancake syrup, for a while.
I don't particularly enjoy discussing the latest Hollywood movies and actors ad nauseum, whose existence and which conversation seems to serve significantly to reinforce established, simplistic stereotypes. Especially when the conversation ranges in exactly that direction.
I don't enjoy conversations that "get personal", looking for ways to pick apart those not present (behind their back). Or that look for ways to pick apart those present, in active, emotionally manipulative pursuit of a pecking order.
On the team I described, we socialized, although people were on average fairly reserved. A good part of that socialization reflected people actually paying attention to and remembering what each other were interested in and doing, and what was going on in their lives.
Again, this is just an opinion, but I think some of the "anti-social" attributed to "nerds" and "geeks" is rather an active dis-interest in much of the lowest common denominator of what we consider "social".
(Some, not all. I've met plenty of intellectual assholes, too.)
Another, personal example: I rather enjoy playing sports. I couldn't care less about watching paid professionals doing it. (This does vary somewhat by sport and event. Also, if anything, I tend to find amateur games more interesting; they bear a closer relationship to my life than the over-produced, over-medicated thing that professional sports has become. Upon reflection, maybe this is a somewhat arrogant or avoidant attitude on my part; I received a lot of abuse as a kid for not being "sports-wise" -- something my father wasn't and never passed on to me.)
I've spend too much time and energy on self-doubt, as it is. A lot of places and people have told me that I'm the one who needs to change. They love my work, and how do I do that? But, you know, "We all work in cubes."
If you are outperforming your peers (and, not infrequently, management) and otherwise succeeding, then, well, who has the problem?
The following may take the interpretation a bit far, but: The bully will always insist that it's "your problem".
Whenever I've seen this done, the very best employees leave. I am a firm believer that you can train yourself to hop in and out of a reasonable facsimile to the "zone" in much less than 20-30 minutes (I do it all the time)[1]; but the level of distraction of the warroom type environment massively favors your more extroverted engineers. My experience has been the engineers I really want to keep, the ones who solve the really hard problems, are much less likely to be in that group.
In short, you may double your productivity for building that CRUD application, but when you have to solve that really hard problem, you may find you don't even have an engineer who is capable.
1. There is a level of concentration, the "real zone" below it that most development doesn't require. However, when it does, the interruptions become catastrophic.