Issue closed with/without code change

[GeorgLink]

I was looking through our Issue metrics and didn't find specifically what I was looking for:

I want a metric for what percentage of issues resulted in code changes. In GitHub that could be expressed as a reference between an issue and a merged pull request. This metric would be helpful to get a gauge for whether a community is more talkative or more outcome-driven.

Here is a visual for what the data may look like for this metric:

Week	Issues closed with code changed	Issues closed without code changed	Ratio with/total
1	12	88	12%
2	30	20	60%
3	5	15	25%

Should this metric be a filter for Issues Closed?
Is there another related metric?
Is it its own metric?

[valeriocos]

For code changes you mean everything that is in a PR (even PRs containing changes on the documentation)?

Focusing on the issue closed without code changed, they may include more kinds of "talkative" issues, for instance the ones asking about how to use a functionality or invalid bug reports (which don't lead to a PR, but born as potential outcome-driven). For this reason, it would be helpful to define the concept of talkative, wdyt?

Should this metric take into account also references to other issues/PRs? An example could be this one: crossminer/scava-deployment#89 where the issue is linked to a PR (which improved the documentation), or crossminer/scava-deployment#94. In a nutshell, the issue makes a summary referencing other related issues, one of them is the first example.

1. Should this metric be a filter for Issues Closed?
2. Is there another related metric?
3. Is it its own metric?

I bet for 3), although related to issues closed, it uses the information with a different intent (talkative vs outcome-driven)

[jgbarah]

I find this metric you propose as a holy gray, not easy to implement, but very interesting. WRT the metrics we have now, yes, it would be implemented as "Issues_Closed", with the filter for "is source code" set to "touched at least one source code file when fixed". The problem, as @valeriocos states, is how to know if the fix touched code or not.

To begin with, you need to know if there is a fix. That can be done if the project has the policy of labeling commits with the issue they are closing (eg, Closes #23), or if issues are cited in PRs (in a similar way). In some very specific cases (such as when people use Bugzilla for code review), code review tickets are semi-automatically associated to issues (because the project has the policy of citing related issues in Bugzilla, and both code reviews and issues are on the same system). Something similar happens in the case of some projects using Gerrit, which usually link to the issue being fixed with a new patchset (patch proposed for code review).

The second problem, is that you know this only when the bug is fixed. Since at any given moment may issues are still open, and there is (still) no commit fixing them, if you can produce this metric, it is going to be difficult to use for the recent past, because you really don't know how many of the currently open bugs are going to be closed by touching code, until they are actually closed. But this is something that can be a no-problem, depending on what you need your metric for (eg, just knowing the fraction of issues that were closed touching code is rather useful in many cases).

So, in summary: from the "theory" point of view, we have this covered with the "Closed_Issues" plus a strict filter on "fix touched source code". From the practical point of view, it is not that easy to implement (but it is certainly doable).

[sgoggins]

This would require consistent practices on development team. Connecting a change request with an issue is possible, but again, this requires the project to follow the practice making these associations. Until it is automated, as Jesus notes, it is kind of an impractical ambition to do this, AND be consistent, and accurate across repositories.