The ultimate software development tool

Best practices on project management, issue tracking and support

Category: Uncategorized

Software Development Metrics – Interview with David Nicolete.

 

We’ve interviewed Dave Nicolette, a consultant specializing in improving software development and delivery methods and author of ‘Software Development Metrics’. We dive into what factors to consider when selecting metrics, examples of useful metrics for Waterfall and Agile development teams, as well as common mistakes in applying metrics. He writes about software development and delivery on his blog.

Content and Timings

  • Introduction (0:00)
  • About David (0:21)
  • Factors When Selecting Metrics (3:22)
  • Metrics for Agile Teams (6:37)
  • Metrics for Hybrid Waterfall-Agile Teams (7:37)
  • Optimizing Kanban Processes (8:43)
  • Rolling-up Metrics for Development Management (10:15)
  • Common Mistakes with Metrics (11:47)
  • Recommended Resources (14:30)

Transcript

Introduction

Derrick:
David is a consultant specializing in improving software development and delivery methods. With more than 30 years experience working in IT, his career has spanned both technical and management roles. He regularly speaks at conferences, and is the author of ‘Software Development Metrics’. David, thank you so much for taking your time out of your day to join us. Why don’t you say a bit about yourself?

About David

David:
I’ve been involved with software for a while and I still enjoy it, though I’ve been working as a team coach and organizational coach, technical coach, for a few years. I enjoy that.

Derrick:
Picking up on the book ‘Software Development Metrics’, what made you want to write the book?

David:
It’s an interesting question, because I don’t actually find metrics to be an interesting topic, but I think it is a necessary thing. It’s part of the necessary overhead for delivering. If we can detect emerging delivery risks early, then we can deal with them. If we don’t discover them until late, then we’re just blindsided and projects can fail. It’s important to measure the right things and be sure we’re on track.

Secondly, I think it’s important to measure improvement efforts, because otherwise we know we’re changing things and we know whether they feel good, but we don’t know if they’re real improvements if we can’t quantify that. I noticed several years ago that a lot of managers and team leads and people like that didn’t really know what to measure. I started to take an interest in that, and I started to give some presentations about it, and I was very surprised at the response because quite often it would be standing room only and people wouldn’t want to leave at the end of the session. They had more and more questions. It was as if people really had a thirst for figuring out what to measure and how. I looked at some of the books that were out there and websites that were out there, and they tended to be either theoretical or optimistic.

Derrick:
Metrics for measuring and monitoring software development have been around for decades, but a lot of people still don’t use them effectively. Why do you think that is?

David:
I often see a pattern that when people adopted new process or method, unfamiliar one, they try to use the metrics that are recommended with that process. There are a couple of issues that I see. One is that they may only be using the process in name only, or they’re trying to use it but they’re not used to it yet, and the metrics don’t quite work because they’re not quite doing the process right.

The other issue is that people tend to use the measurements that they’re accustomed to. They’ve always measured in a certain way, now they’re adopting a new process. They keep measuring the same things as before, but now they’re doing the work in a different way. There’s a mismatch between the way the work flows and the way it’s being measured. They have numbers and they rely on the numbers, but the numbers are not telling truth because they don’t line up with the way the work actually flows.

Factors When Selecting Software Development Metrics

Derrick:
Picking the right metrics is paramount. What are some of the factors that we should consider when selecting metrics?

David:
Look at the way work actually flows in your organization and measure that. I came up with a model for that in the course of developing this material which I would look at three factors to try to judge which metrics are appropriate. The first factor is the approach to delivery. The basic idea there is if you try to identify all the risks in advance, all the costs, you identify all the tasks, lay out a master plan, and you follow that plan. That’s what I’m calling traditional.

What I call adaptive is a little different. You define a business capability, you’ve got your customer needs, and you set a direction for moving toward that, and you steer the work according to feedback from your customer, from your stakeholders. You don’t start with a comprehensive plan, you start with a direction and an idea of how to proceed. Then you solicit feedback frequently so you can make course corrections. That’s the first factor I would look at: traditional versus adaptive, and that I think has the biggest impact on which metrics will work.

The second factor to look at is the process model. I don’t have to tell you that there are a million different processes and nobody does anything in a pure way, but if you boil it down I think there’s basically four reference models we can consider, or process models. One is linear, you can imagine what that is. The canonical steps that go through from requirements through support. The next one would be iterative, in which you revisit the requirements multiple times and do something with them. The third one that I identify is time-boxed. It’s really popular nowadays with processes like Scrum and so on. The fourth one is continuous flow. This is becoming popular now with the Kanban method, and it’s also being adapted into Scrum teams quite a lot. We’re really interested in keeping the work moving smoothly.

Now a real process is going to be a hybrid of these, but it’s been my observation that any real process will lean more toward one of those models than the others, and that’ll give us some hints about what kind of metrics will fit that situation. The third thing probably has the least impact on is whether you’re doing discrete projects or continuous delivery. What some people call a continuous beta, or some people just don’t have projects. You have teams organized around product lines or value streams, and they continually support them, call it a stream. Between those two there are some differences in what you can measure. Well I look at those three factors, and based on that you can come up with a pretty good starter set of things to measure, and then you can adapt as you go from there.

Metrics for Agile Teams

Derrick:
Let’s take a couple of example scenarios. If we have an agile team working in short sprints who are struggling to ship a new product, what kind of metrics should they consider to identify areas for improvement?

David:
If they’re using Scrum basically correctly, they could probably depend on the canonical metrics that go with that, like velocity and your burn chart. You might look for hangover, incomplete work at the end of the sprint. You might look for a lot of variation in story size, when you finish a story in one day and the next story takes eight days. When it comes to metrics as such they could use, as I said, velocity and so on, and you can always use lean-based metrics because they’re not really dependent on the process model. What they might consider is looking at cycle times. They could look at the mean cycle times as well as the variation in cycle times and get some hints about where to go for root—cause analysis. Metrics don’t tell you what’s wrong, but they can raise a flag.

Metrics for Hybrid Waterfall-Agile Teams

Derrick:
What about a hybrid waterfall agile team working on a long term project, wanting to know what it’s possible to deliver by a certain date?

David:
To know what’s possible to deliver you can use the usual things like a burn chart, burn up or burn down as you prefer, to see according to their demonstrated delivery, their velocity, I’ll call it that, how much scope they can deliver by a given date. Conversely you could see by what date approximately they could deliver a given amount of scope. It depends on what’s flexible. In this kind of a project, usually neither is flexible, but at least you can get an early warning of delivery risk. If it looks like the trend line is way out of bounds with the plan, well now you’ve got a problem.

One thing that might surprise some people is the idea that agile methods can be used with traditional development. We need to decouple the word “agile” from “adaptive,” because quite often it is used in a traditional context.

Optimizing Kanban Processes

Derrick:
What are some metrics relevant to those working in a bug queue? Say they’re wanting to optimize their working practices to stay on top of incoming bugs.

David:
For that I usually like to use little metrics, mainly cycle time, because you want to be somewhat predictable in your service time so when a bug report comes in people have an idea of when they can expect to see it fixed. How do you do that? Well, you can use empirical information from past performance with fixing bugs, and your mean cycle time will give you approximately how long it takes to fix one.

I like to use the Kanban method for these kind of teams because it defines classes of service. You’ll find that every type of bug doesn’t take the same amount of time to fix. Based on your history, pick out different categories. You can identify the characteristics of particular kinds of bug reports that tend to fall together, and you can track cycle times differently for each of those classes of service. If someone calls in and says, “Well we got this problem.” “Well that looks like that’s in category two. Whatever that means, that typically takes us between four hours and eight hours.” That can give them a little bit of warm fuzzy feeling about when they’re going to see it fixed. I think that using empirical data and tracking cycle time, is the simplest, most practical way toward that workflow.

Rolling-up Metrics for Development Management

Derrick:
What about a CTO who wants to monitor how teams are performing, and ensure code is of high quality? How can metrics be rolled-up for those in more senior positions?

David:
How the teams are performing, you need measurements that are comparable across teams and comparable across projects. The lean-based metrics needed to compare across teams and projects and across development and support, those kinds of things. If you’re tracking throughput, cycle time, there’s another one I haven’t mentioned that I wanted to: process cycle efficiency. If you track those, those roll up nice. Some other metrics don’t roll up so well. Some of the agile metrics, particularly velocity is really different for each team. Percentage of scope complete, that may not roll up very well either.

The other question about code quality, I think that if we can let that be a team responsibility then they can use metrics, but they don’t need to report outward from the team. Usually things that they can get out of static code analysis tools will help them spot potential quality issues, but I wouldn’t share very detailed things like static code analysis kind of stuff and code coverage outside the team, because then team members will feel like they’re going to get judged on that and they’ll start gaming the numbers thinking they’re going to be needed those. Those kind of metrics are really for the team’s own use.

Common Mistakes with Software Development Metrics

Derrick:
What are some mistakes you often see people make when applying software development metrics?

David:
People seem to make a couple of mistakes over and over. The one I think I mentioned earlier, people apply metrics that won’t fit the context. Maybe a company wants to ‘go agile’, and so they start tracking agile metrics, so whatever is recommended. The metrics that are recommended are safe or something like that, but they haven’t really fully adopted these new methods. They’re still in the transition, and so the numbers don’t mean what they’re supposed to mean. For instance, they may track velocity, but they might not have feature teams. They might have component teams, and the work items that those teams complete are not vertical slices of functionality. Whatever they’re tracking as velocity isn’t really velocity. You start getting surprised by delivery issues.

You can also have the opposite situation where teams are working in an agile way, but management is still clinging to traditional metrics. They will demand that the teams report percentage of still complete to date, but you’re doing adaptive development so you don’t have 100 percentage scope defined. You have a direction. You may be 50% complete this week based on what you know of the scope. Next week you might be 40% complete because you’ve learned something, and then the management says “Well what’s going on? You’re going backwards,” and they don’t understand what the numbers mean. What I see happen in that case is it drives the teams away from adaptive development, and causes them to try to get more requirements defined upfront.

The second mistake I see a lot is that people either overlook or underestimate the effects of measurement on behavior. We can be measuring something for some objective reason, but it causes people to behave differently because they’re afraid they’re going to get their performance review will be bad because they didn’t meet their numbers. I think we have to be very conscious of that. We don’t want to drive undesired behaviors because we’re measuring things a certain way. That not only breaks morale, but it also makes the measurements kind of useless too when they’re not real.

Those are really the two main mistakes: that they don’t match up the metrics with their actual process, or they neglect the behavioral effect of the metric.

Recommended Resources

Derrick:
What are some resources you can recommend for those interested in learning more about managing projects and improving processes?

David:
I like this, an older book called ‘Software By Numbers’. That’s a classic, but one of David Anderson’s earlier books is called ‘Agile Management From Software Engineering’. That has a lot of really good information to apply economic thinking to different kinds of process models. He covers things like feature driven development, extreme programming. Another guy whose work I like is Don Reinertsen. He combines really deep expertise in statistics with deep expertise in economics and applies that to software, and can demonstrate mathematically why we don’t want to over-allocate teams. How it slows down the work if you load everybody up to 100% actually slows things down. What’s counter intuitive to a lot of managers is if you load your teams to 70% capacity they’ll actually deliver better throughput, but it’s very hard for a lot of managers to see somebody not busy. It’s really hard for them to get there.

Derrick:
Really appreciate your time today Dave, some great stuff here. Thank you.

David:
Well I enjoyed the conversation, thanks.

The Problems with Open Source (and How to Fix Them) – Interview with Justin Searls

In this interview with Justin Searls, Co-Founder at development agency Test Double, we discuss issues in open source software and what we can do about them. Justin raises problems for both consumers and contributors – like managing hundreds of dependencies to security issues and burnt-out maintainers. We dive into the efforts being made to address these and what you can do too.

Justin discusses issues like this and software development more generally on his blog.

Content and Timings

  • Introduction (0:00)
  • Open Source convenience, long-term fragility (1:33)
  • Who is Auditing Open Source Code? (5:17)
  • The Rollercoaster Ride of Maintaining Open Source Projects (7:38)
  • Recruiting More and Better Contributors (11:18)
  • Improving Communication in Open Source (12:42)
  • How Can We Help? (14:35)

Transcript

Introduction

Derrick:
Justin Searls is co-founder of Test Double. A software development agency based in Columbus, Ohio. Through his work Justin uses and contributes to a number of open source tools, and also speaks at conferences about a range of software development topics, including the talk The Social Coding Contract, which highlights some issues he sees in open source projects.

Justin, Thank you so much for taking time to join us today, really appreciate. Do you have any more to add about yourself?

Justin:
There’s a lot of brokenness around us, and I think the talk that you referenced, The Social Coding Contract, is really just putting a lens on a lot of brokenness in open source. But I’m not just here to shout at the clouds and complain about stuff. I think that by building awareness we can make it better.


“In an open source team you have like a dozen people who all want to be point guard”


Derrick:
So your talk raises a number of issues in open source development, both for consumers dealing with dependency issues, and for maintainers resulting in burnout. But what lead you to want to raise these issues?

Justin:
I think that when you really really distill things down, boil it down to just a kinda core essence. The answer to that question, is that our industry has organized itself around, essentially a lie, and that lie is that faster equals better. Anything we can do to like, faster to build an app, faster to ship into production, devs that slingcode faster than slower devs, or 10X better than … All of our orientation of how, you know technology is sold and described, and glorified in our culture, is all about how like how fast it is how fast people get stuff done. But the overall attention span is so brief that we just tend not to focus on problems.

Derrick:
Lets dive into a couple of these issues, you say that those building with opensource optimize for convenience, but often at the cost of long-term fragility. What do you mean by that, and has this impacted your projects.

Justin:
By fragility there … What I don’t mean is that there is some sort of cabal of open source developers trying to make a mess for you. What I really mean is that when you ship something into production these days, odds are you’re running like a little tiny layer, 10 percent or less of the code that’s getting executed in production is stuff that you wrote. Most of it is a mountain high of application dependencies that you stand on top of. Some directly, some transitively that got pulled in via those other dependencies. It’s a lot of stuff that frankly we don’t understand really well. And that’s fine for getting started, because obviously you gotta be competitive, and if somebody else can get a prototype out the door quickly, you can get really fast feedback. You know?

There’s a lot to be gained from it, we just need to move cautiously to understand, okay so it’s been 3 months, it’s been 6 months. Lets actively look at how these dependencies have been serving us. Have they been a pain to use, have any died or gone out of maintenance. Like what’s the influence that they’ve had on our, the design of our code. Are we really just like writing cookie cutter code to satisfy all the APIs that we are depending on or are we really growing a domain model that very nicely fits like a glove the problem that our application’s trying to solve.

Those are the sorts of things that, you know I would kinda call fragility that’s baked in to this process of looking for something to help get us a quick start. Another facet of this that’s really interesting to me is that coming from the ruby community, where ruby on rails was really huge, or became huge almost a decade ago now. That was one big monolithic framework and that was very difficult for me to start my own framework, because I would push it up, but then I had n plus one queries everywhere. All sorts of like pain of not knowing how to it, a magical little thing. And I could remediate that on one project, and it would take a long time, and everyone would pay the price of learning rails. But on subsequent ones we could at least use that knowledge.

But nowadays I think the trend is so anti-framework, and so pro-modularization, tiny libraries that all do their own thing. We’ve all become effectively framework maintainers. Not to say everyone’s inventing their own, but we’re like curators now of like this manifest of here’s my thirty dependencies, and no one else in the world will have all thirty dependencies at exactly the same versions that you do. Which means that now the onus is on you to make sure that they all work together correctly, and if there is any interplay between the two of them that doesn’t, it is up to each project team who’s tasked with building an application to also be responsible for troubleshooting two potentially divergent dependencies that are stepping on one another.

Derrick:
It’s definitely an interesting perspective, I haven’t thought of that myself, as a curator of frameworks.

Justin:
Yep, and it’s not like small is bad and that big monolithic stuff is good. It’s just these are the source of costs and the responsibilities and the roles that we should be thinking about. That go with maintaining something built that way.

After five to six years of working with a lot of, you know open source technology projects you start to see a lot of patterns, just do basic pattern recognition. Be like oh we’re finding like writing adapters for this third party thing is really really hard, or its API is seeping all over our project. Like how can we guard our self, and push that out, build some scar tissue between us and that dependency, because it has been found to be kinda problematic.

All those sorta good habits I think tend to grow organically, but then can only happen if people have an awareness that all these dependencies leaking all over there stuff lead design problems long term.

Derrick:
We often approach using popular open source libraries with the assumption of security, because anyone can read them, and with trust because they have a number of respected contributors. But you don’t buy that, why is this?

Justin:
The concept that you’re alluding to, is introduced in the book called The Cathedral and the Bazaar, like fifteen years ago, maybe longer. But that was written in a time when, when most people were thinking about open source they’re thinking about really big operating systems, like Windows versus Unix. Like a total black box versus this thing that is like one or two or three major big projects that have tens of thousands of developers looking at really closely. And over the last fifteen years, everything’s inverted. Now GitHub hosts hundreds of thousands open source projects.

I think at some point we probably eclipsed the point where there were literally more libraries being used, than eyeballs looking at the actual source code of other people’s open source. So in theory that would work but the problem is that there is more projects then there are eyeballs.

The other thing that I think effects this is the stack, the open source stack that we stand on gets a little taller every year. Every year we find some new common solution to a well known problem, and we stand on it. And fifteen years ago, twenty years ago, when people were starting to grow thoughts about what open source would mean for security, well you know like openness itself for example, like you know like that’s a pretty cool library, like I’m gonna use that for how I do my network securely. But now it’s such a given that everything depends on OpenSSL, that we … You know it’s mature, its mostly settled down, there’s not a lot of maintainers, it’s not a sexy thing to invest in from a marketing perspective if you’re Facebook or Google. So it mostly just gets no attention.

Even if you are a really big company, and you understand that your openness is all working is critical to your business. You know that it’s like a Mexican standoff where every company in the world, they all depend on it. So it’s certainly like, it’s very important that someone be auditing the security of a fundamental thing like that. But it’s nobodies responsibility to do that, and no one feels that accountability. So I think that’s where we’ve really fallen on our face, where something like Shellshock happens to Bash, and it’s a program almost every developer uses, and lots of different internet of things stuff use. But zero people feel like it’s my job to go on the weekend and read up on, you know Bash to make sure that it is secure.

Derrick:
You describe the life of an open source project maintainer to be something of a roller coaster ride, that all too often ends with the maintainer getting burnt out. Why do you think that this is commonly the case?

Justin:
On the one hand, we kinda glamorize prolific open source authors. There’s maybe hundreds of thousands of open source projects out there but there are a handful of people that are like, Hey when I’m doing some node stuff I’m gonna look for a library that TJ Hallowaychuck posted, because he’s got a lot of them, and his other ones were pretty good, so I’ll use this one too. Of course he up and left node one day, which was problematic for anyone that lived in his stack. We don’t do this intentionally, we don’t seek to live in this celebrity culture, its just when there are so many options out there, we need all the tribal markers for quality that we can find, And that person wrote a good thing, and I like their approach to something else, so I’m gonna use all there stuff. And that’s how that sorta power accrues.

Now, the second phenomena which I think is really interesting, is as personality drives so much open source adoption the asymmetric relationships just fail to scale. Like an illustration that I was thinking of is like imagine that you found a golden lemon, and it could squeeze and infinite amount of juice, so you started an open source lemonade stand. And so you squeeze all the lemonade from that lemon all day long, for anyone who wants it. And the early adopters of your lemonade stand are gonna be like whoa you’re brilliant, this is amazing. Of course they’re gonna get sick of the lemonade and move on eventually. But once the word gets out, even though you’ll have gotten that initial fame and excitement of getting a lot of positive attention; pretty soon the line’s gonna be so long that the upper bound on how much lemonade comes out is your own physical laborious painful labor. And at that point you’re at a crossroads, you have to decide like, and this is analogous to an opensource maintainer. It’s like I don’t get any distinct joy out of this anymore, I’m mostly just doing work for people for free, and they don’t really appreciate it because they’re used to it now.

So do I continue out some sort of misplace sense of duty, or do I just quit and leave people in a lurch. I think that’s how a lot of open source projects slowly atrophy and die.

Derrick:
How do you think we should go about fixing this?

Justin:
I think that the successful long-term sustainable open source projects are ones where a community formed around them in proportion to their growth and success. And I think all parties involved need to be A, recognize that, acknowledge that, and B accept some amount of responsibility other than treating open source like a corporate welfare program. Right, so if I’m a maintainer I acknowledge that now, after writing this talk and thinking about this a lot is like, okay so this project is a hundred stars, like I really shouldn’t be the only owner on this repo or this NPM library, or this ruby gem. Lets pull in a couple other owners because other people are joining and like oh this project is a thousand stars, like lets look at a code of conduct, a governance model, you know some kinda mission statement for what this project’s about, the core tenants.

You know oh it’s at ten thousand stars, like we should probably start to have like a technical committee so I get out of the role of dictator for life. Because it’s now way bigger than what I meant for it to be. So it’s a bit of humility right, like seeding control gradually, to avoid that burn out. But then from the perspective of users, they need to be actively seeking, and accepting like, hey you know I should be contributing back to this, and I should work with my employer to make sure that I have the time to contribute back to the projects that we use, that are interesting to me. So that they have the support that they need, so that they don’t just wither on the vine and die even though hundreds or thousands of applications might continue to depend on them.


“There is never going to be a day where you feel like your thing is ready enough for public consumption… so please just publish”


Derrick:
What else can project maintainers do to recruit more, and better contributors, and to reduce friction for those interested and contributing?

Justin:
I really like the idea of an imagined application that was like a match.com for putting together people that have open source projects that need additional contributors, maybe they have a lot of issues, maybe like I said they’re getting more popular and they recognize this need. With people who are looking to work on open source. Maybe they want the additional visibility or the prominence that comes from having a lot of open source stuff. Maybe they just want something to program on. Maybe they just want to improve their skills.

Whatever the reason, it would be really cool if an application, say used Oauth, or a storage of your various credentials to be like hey you use this library and that library author is looking for people. Or hey you write a lot of node JS and this person’s got a node JS project that’s got this many stars, and he’s desperate for a maintainer. Something to connect these people, because I think that if you look at the network graph of people who are publishing open source, it’s too small for them to try to solve it by you know, simply tweeting out or going to a user group and being like, hey will someone please maintain this. Because usually by the time that they know to ask for help, it’s late enough that that project is no longer appealing, like its probably mostly settled down and the only work left to do is to put up with all of the random people who want basically free tech support.

Those are some ideas, but you know short of that it’s hard, it’s better to try and solve that early than solve that late.

Derrick:
So in problematic projects at work, we use techniques like improving documentation, retrospectives, stand ups, pair programming to overcome some of these problems. Yet you rarely see them applied in open source projects. Do you think there is a role for these, and other ways of improving communication between maintainers, contributors, and users?

Justin:
So like comparing like a product team and a business versus an open source team. It’s almost like in a product team you’ll have role players who specialize in one area or the other. In an open source team you have like a dozen people who all want to be point guard. They don’t necessarily have any incentive or organization to be a role player that slots into just like one particular aspect of the project. Unless there is somebody or some group of people who are smart enough and experienced enough to be able to … Like you just did, carve out all of these individual responsibilities. Like if we don’t do these things problems are gonna happen. Like we have to individually justify because we don’t have some boss, we don’t have a paycheck coming typically. We need to justify for ourselves why these activities are important.

Then some of them are just shit work compared to other stuff. Like everyone wants to be the genius that gets to code a thing, and gets to be famous for building it. But if you look at somebody, maybe they are a great writer, and they are a great teacher and instructor. And if they wrote the documentation pages, then that would be fantastic for the community. But that doesn’t necessarily get the same accolades or glory. And so it’s contingent on everyone else on the team to lift that person up, to point out how great they are and how they’re helping.

That’s what I see on really successful open source teams. Is giving people a reason to fill those roles that isn’t just about money. Because once you put money into it, it might as well be a starupified or productified open source, in kinda you know name but not spirit only project.

Derrick:
What do you think of initiatives like Todo group and Ruby together, and what more do you think companies who use open source components can be doing?

Justin:
I’m friends with Brandon Keepers at Github, who’s done a lot of there work through Todo group, and it sounds like in many ways, its goal is to solve a lot of the systemic problems, or at least address as a bunch of companies realizing like hey none of us individually have an individual interest in being the one to own these shared concerns, like security audits. But we understand whenever there is one we all have a huge business risk to that.

So I love the idea of them coming together, putting a little bit of funding behind it, and seeing where it goes. I don’t think that it’s gonna turn into this thing where it’s developing a lot of standards and process that other people are gonna feel beholden to. You know they might publish some examples like an example code of conduct that you could use, or something like that to help people get started.

But I think its biggest point of value is gonna be how do we solve some of these interweaving problems, that no individual person feels responsible for. When you think organizations like Ruby Together … I don’t very much about Ruby Together per se or properly, but in conversations I’ve had with other people about things like GitTip or efforts to try to put money into projects that are commonly used as support. On one hand like sponsorships like hey we’re Ruby Gems, we need hosting, you know. That is a real clear cost. Where it gets a little muddy is … I think a lot of people have great intrinsic motivators for working on open source, and when you just kind of like put a bounty on like, hey if you get this feature done, or individual sponsorships for like, hey I want this feature, as a company in this open source thing so we’re just gonna fund two months of the developer’s time.

A lot of those are really interesting, and some of them might scale, and some of them might work. But it’s important to know that there’s still this impedance mismatch. The reason anyone starts writing an open source thing isn’t to get paid, its because it scratches an itch or it’s something that they found interesting. There’s a real risk that once you start throwing money at it, whether it’s at the individual or at the specific features and issues. You’re probably going to change the incentive structure, and alter the course of where that project goes.

So I’m a little bit more cautious around the argument that open source work is labor too, and we need to be paying directly for people’s open source labor, through some kind of like patron model. Another example of that that stands out to me, is I’ve got a friend who is a very prolific open source developer, and he could totally start a Kickstarter tomorrow and raise two hundred thousand dollars a year, if he wanted. To just do open source, and he won’t do it because he’s a smart guy, and he know that as soon as he gets however many five dollar donations it takes to get up to that number. Those are gonna act like customers, like if they don’t get the feature that they want or they don’t get their bug fixed, they’re all gonna come knocking down. Then it’s that lemonade stand problem again. Of not only do they expect the free lemonade but it was five dollar lemonade, and they paid for it and they want a refund.

So all of these things sound like solutions, but I don’t think that any of them are ever going to solve the problem of code that’s written for an egalitarian reason or a non-financially motivated reason can’t just be solved by throwing dollars at it.


“Our industry has organized itself around, essentially a lie… that faster equals better”


Derrick:
Can you recommend any resources to those wanting to learn more about running sustainable open source projects, or those wanting to contribute to open source more effectively?

Justin:
So first, when I meet people at conferences or user groups, a lot of them will come up to me, and its like a hallway confession almost. Like hey man I like all your open source stuff and I would totally open source stuff too, I’ve got this thing that I’ve been working on for like five years, but its not ready yet, its not ready to be open sourced yet. I just want to say you know … Okay bye. So like clearly they felt the urge to come and talk to me about there not yet open source thing. But they have this kind of like intention towards contributing.

What I always tell them is that there is never going to be a day where you feel like your thing is ready enough for public consumption, that you’re just gonna flick from private to public and ready the flood gates. First of all, I probably have like 200 or 300 repositories between test double and my personal account. That calculation is expensive, I just default to open sourcing absolutely everything.

You’ve never heard of 99% of my repositories, I guarantee it, because most of them weren’t that useful, maybe didn’t go great, maybe it just didn’t get viral, and a couple others did. The only reason that those ones did get successful, is because I was defaulting to open sourcing everything that I do that doesn’t have like a strict business model in front of it.

So that’s what I tell people about getting started and open sourcing. The first place is like if you just publish everything openly, that’s going to look real good when you have all this stuff to show, even if it’s not perfect.

Oh and don’t worry about people looking at your code and judging you, because a lot of people say they read open source, but like we discussed earlier about security, no one actually reads open source. Like if there’s a bug or something they look at it, but no one’s going to read your code. They’re going to see that you have thirty repositories, and that presumably they work if they have a little green badge at the top.

So please just publish. The other bit of advice about like resources for successful, sustainable, long-term, like once something’s been successful where do you look at. I don’t think that book’s been written, I don’t think that there’s a starter pack for how to build a community. At least not in the open source world, but what I would look at, is look at the non vendor-backed very successful projects. The two that are most relevant to my personal technical experience is Ruby on Rails, and even more so EmberJS. When you look at Ember, like yes there are a handful of agencies, that kinda back it and firmly sponsor it. Like obviously Tilde is the Ember company. But they’re not Oracle, there not Facebook or Google. There a smaller group of developers that just care a lot about making a ten year framework, and putting in place all of the structure that they need to build the community around it. Now look at it, there are Ember meetups, in I think like thirty different cities, we have one here in Columbus.

There’s a very strong affinity and a strong sense of ownership by even people who just use Ember. So they’ve done a great job of leveling people up, encouraging contribution, and encouraging engagement really at every level. So look at everything that they’re doing.

Derrick:
Justin, thank you so much for joining us today.

Justin:
Thanks so much for having me, I really appreciate it.

Anyone watching this, and wants to get in touch with me directly, please don’t be shy, just justin at testdouble dot com, I’d be happy to talk to you.