Kris Buytaert, whom I’ve had the pleasure of meeting on several occasions at various conferences, recently wrote a blog post on the topic of systemd and devops. He is someone who has the experience of being very much hands-on with actual infrastructure in production (amongst many other talents and skills) — and insight about the various pieces that make up an organization’s infrastructure, from someone who also really understands the cultural aspects of devops, is incredibly valuable to me.
While Kris does dance around one of everybody’s favorite topics (systemd) he specifically avoids turning the post into yet another rant about systemd. His main point: identifying that there is a gap in communication between OS developers and users. It may be lack of empathy, lack of feedback loop, etc. And Kris specifically points out that this is a starting point for discussing how to fix that gap.
Similarly, this blog post from myself is not meant to defend or justify decisions made. Nor is it to point fingers. It is to help build upon the discussion.
Having been the Fedora Project Leader up until fairly recently, I can certainly say that one of the things I worried about was this gap. If we in Fedora, and distributions in general, were becoming silos. Even though the Fedora Project is made up of contributors with skills other than development, I still worried that as a whole, we weren’t often looking outside the window to see how the rest of the world was doing work, to hear about their problems. I did a lot to try and open those windows, and share those stories, especially with the developers — because most of them have never had to manage large numbers of systems at scale. Most of them have never carried a pager that inevitably went off at the WORST POSSIBLE times. (Or dropped one down a toilet. While flushing. I did that. It was totally an accident though. Seriously.)
Bridging that gap is hard. Yes, distros could do a better job of listening to end users. This is where efforts like the OpenStack User Committee (because this issue isn’t limited to just distros) may prove to be incredibly helpful. It’s gathering and formalizing that commentary from users into something that is essentially seen as contribution to a community, in digestable form, rather than yet-another-angry-mail that represents an unknown portion of the population or community. And recognition as an actual contribution that contributes to a larger process, and ultimately success, can carry a lot of weight, particularly in communities that operate partially by meritocracy, as many distributions do.
Would something like this work in distributions? I don’t know. Perhaps it’s worth trying or floating around as an idea.
DevOps and Open Source
We talk about feedback loops a lot in the DevOps community. How to better stay in touch with and listen to and have empathy for end users or teammates. How to “build the right thing,” as the Lean Startup tells us, by showing our work as early as possible. Or, as folks in Open Source might say, “release early, release often” — because it enables transparency about what’s going on, and provides opportunity for earlier feedback and discussion.
I don’t think it’s a coincidence that DevOps and the use of open source projects seem to go hand in hand when it comes to success stories. The shared values of the DevOps and Open Source communities, when it comes to *how we practice our craft and do it best*, are often similar. Transparency. Why we document how and why things happened. Why we “release early, release often.” We all strive for continuous improvement.
You’d think it would seem natural to talk to each other. But we don’t. I think distributions are in some ways very set in their processes – and simply expect that if you want change, you’re going to show up and make it happen. And when it comes to the DevOps folks, the end users of distros and other open source projects, the focus on feedback loops tends to be with their end users — not necessarily their “suppliers.”
Where I go all Deming on y’all
I’m going to take a page from John Willis, aka @botchagalupe, for a moment, and refer you all to W. Edwards Deming and one of his 14 points for management.
“End the practice of awarding business on the basis of a price tag. Instead, minimize total cost. Move towards a single supplier for any one item, on a long-term relationship of loyalty and trust.”
Now, if you’ve read Deming’s “Out of the Crisis,” you’ll know that this is largely related to manufacturing. Suppliers supplying you with products and parts that are screwed up or defective or not meeting your standards can SERIOUSLY screw up your day, as a manufacturer. But when you build a relationship, when there is loyalty and trust between the supplier and the user of those supplies, there is mutual understanding of roadmaps, and needs, and there is empathy and consideration.
In some ways, an open source project is the greatest supplier of all. You can see the incoming quality of code; you can see roadmaps and plans and delivery dates; you can file feature or improvement requests, and sometimes they will be implemented.
But the concept of a relationship builds upon that. (Yes, we all know relationships can be complicated, but generally the rewards are worth it, and this should be no exclusion.) Even Deming pointed out the differences between having “specifications” and knowing the whole story. And in this case: The whole story is — what are your problems? What are you trying to accomplish? What are your goals? What do you wish could be improved — whether in a distro, or in a project.
Those stories are what create empathy – and relationships. Done regularly (you know, like a loop) – those relationships can last. Having stories, understanding the problems, understanding the practices people use on a daily basis gives developers the opportunity to not only develop empathy — but also the opportunity to solve *the right problems*. And not go fixing things that may not actually be broken. And not feeling like the only feedback they ever get is people who are unhappy.
Generally: I think there are improvements to make all around in the closing of the gap, on both sides of the fence. But like many problems, it all boils down to empathy.