How to expedite critical issues?

GitHub 🐙
#1

Issue

There is currently no way to highlight issues that should be fixed with high priority.

  • We have a common bug label for Parse Server issues.
  • We have an additional unbreak now label in other repos (Parse Android SDK) for critical bugs.
  • We have a distinct process to expedite security issues, no applicable to non-security bugs.

Example

Issue 6728 breaks the Instagram auth adapter due to API endpoint deprecation. It already generates duplicate entries. This issue has the potential to impact production systems. The deprecation of the API endpoint has been published in 2018 and several times since then until being disabled on June 29, 2020.

Suggestion

  • Define a uniform set of Github issue labels across all repos, with maybe minor modification for special repos like docs if necessary.
  • Introduce bug severity levels in Github issue labels to increase awareness for important issues, expedite fixing these issues and merging associated PRs.
  • Introduce a guideline for team members on when to apply which priority level to prevent inflationary prioritization.

(The following part is continuously updated to show the latest consensus)

Uniform Github issue labels

1. Initial status

  • no label (issue has not yet been look at by a Parse team member)
  • needs more info (more info is needed to classify the issue)

2. Issue classifiers:

  • :bug: bug (impairs a documented feature or a functionality that is not explicitly documented but one would strongly expect the general user to infer from a documented feature)
  • :star: feature (a new feature that does not exist yet)
  • :dna: enhancement (performance improvement or functional enhancement of existing feature)
  • :grey_question:question (not an issue but a question; should be in Community Forum but we can’t easily transfer, so we refer the author to the Community Forum)
  • :wrench: troubleshooting (not an issue but asking a code-level question; should be in StackOverflow; for simple issues we may give a quick tip / solution to the author, in general we refer to StackOverflow to educate the author about where to post code-level questions)
  • :speech_balloon: meta (issues regarding CI and tests or pinned calls for high visibility like “maintainers wanted”; other meta discussions are preferably held in the Community Forum)

2.1 Augmenting labels for bug severity

  • :rotating_light: S1 (catastrophic - blocks development/testing, may impact more than 25% of users, causes data loss, no workaround available, potential chemspill although we forward the issue into our distinct process for security related issues if found to be so)
  • :zap:S2 (serious - major functionality / product severely impaired and a satisfactory workaround doesn’t exist)
  • S3 (normal - blocks non-critical functionality and a work around exists)
  • S4 (small / trivial - minor significance, cosmetic issues, low or no impact to users

2.2 Augmenting labels for any classifier

  • duplicate (duplicate of already reported issue; references and closes the issue to encourage discussion in one place)
  • documentation (requires a change in the docs; since docs tend to be neglected, an distinct label could bring more attention to keeping the docs updated)
  • up for grabs (a bug, enhancement or feature that is not in the works; once a feature is being actively worked on, it is expected to infer that from the comments and the label can be removed; the label will be reapplied after prolonged time of inactivity)
  • on hold (the issue cannot be resolved anytime soon because it depends on a 3rd party or another issue that needs to be resolved first)
  • won’t fix (the issue won’t be fixed, e.g. due to low priority)

Remove all other existing labels (non exhaustive list)

  • discussion, because every thread is essentially a discussion and every discussion is either regarding a bug, an enhancement, a feature or a question.
  • in the works, because we can never be sure whether a feature is still actively being worked on and we rather have more people looking at an open issue than less (see “up for grabs” label); the current status of an issue should be inferred from the last comments, not from a label that is likely to be outdated.
  • good first issue, because it seems to be rare that someone picks up an issue just to make a PR; more often an issue is picked up because someone faces the same roadblock.
  • long term, because it is unclear what it means; fixing a bug is long-term when no-one is working on it; other issues that currently use that label are not parse server issues and would be better placed in the community forum; if an issue cannot be solved because of an external dependency then a distinct “on hold” label makes it more clear.
  • needs investigation, because an issue usually needs investigation and it is a temporary label that needs to be removed at some point anyway.
  • pr submitted, because a PR is referenced in the thread anyway and a submitted PR doesn’t say anything about it’s viability or readiness for merge (sometimes PRs are submitted even before opening an issue and asking for community feedback).

Framework

  • Bug severity (S1-S4) can only be assigned by a “triage team” to ensure proper classification of and attention to critical issues.
  • A bot requires an issue to be classified (as bug, enhancement, feature, etc) after n days of being opened, to mitigate the risk of missing a critical bug.
  • A bot requires an issue that is classified as bug to have a severity level assigned by the triage team within n days, to mitigate the risk of missing a critical bug.
  • A bot that adds the “up for grabs” to “feature” / “enhancement” after n days of inactivity.
  • Issue transfers between repos team members who have write access to the target repo, otherwise by asking @parse-community/core-maintainers in the comments.
  • Adding, editing, deleting labels is restricted by guideline (cannot be restricted by repo permission)
1 Like
New features decisions
#2

This seems like a good idea to me, could be a quick way to check for outstanding important issues before publishing a release.

A guideline would be a good idea - I’ve been thinking about having a doc for new team members (and existing when needed) with some details about what they can and can’t do, best practices etc.

1 Like
#3

could be a quick way to check for outstanding important issues before publishing a release.

Yet another good reason.

I think we can get some inspiration from the Mozilla triage process. It’s overkill for us but a starting point.

  • Who sets the bug priority / severity?
    Some repos have only a small group of committers, so to ensure a bug is triaged in a timely manner and at all, it may make sense to allow anyone to triage at this point.

  • Across all repos
    In the process it may make sense to uniform all labels across all repos and restrict the creation of new labels.

(Moved label suggestions from here to first post for better overview)

Questions:

  • When is enhancing an existing feature not an “enhancement” anymore but a “feature”?
  • How should the “help wanted” label be used?
  • When would we use each of the bug priority labels?
  • In order to transfer issues to the appropriate repo, Parse team members should be able to transfer to any Parse repo, but that is currently not allowed unless they are also committer of that repo; can access rights be changed so that transfer is allowed to any repo?
#4

This could work but I would be more inclined to only allow this for team members who feel comfortable triaging issues and are likely to use the ability. I could setup a team with Triage perms on all key repos. I think there.

As far as I know it wouldn’t be possible to restrict label creation without downgrading perms but we could have a written policy.

I would say the bug severity dictates the priority so not sure about the need for both sets of labels. Unless we want to have flexibility in cases where contributors don’t have the time or ability to fix a bug.

I would say a bug or feature request that the author and team members will not address or not alone, or an PR that the author will not be able to finish - but that does leave room for a fair bit of ambiguity - it may be that it’s not useful enough to keep as a label.

Not without write permission but maybe this could be something given to the ‘triage team’ - perhaps another reason for a higher barrier to entry. Alternatively, members could mention @parse-community/core-maintainers and one of us can do it.

Btw I found a useful document about org perms.

#5

This could work but I would be more inclined to only allow this for team members who feel comfortable triaging issues and are likely to use the ability. I could setup a team with Triage perms on all key repos. I think there.

Sounds good.

I would say the bug severity dictates the priority so not sure about the need for both sets of labels. Unless we want to have flexibility in cases where contributors don’t have the time or ability to fix a bug.

Agree, removed. Severity describes the impact while priority describes the scheduling, but since we don’t schedule (and don’t have a defined release cycle), we can just skip these. I will add a won't fix label to the “Augmenting labels” to account for the removed P5 priority.

“help wanted” label … it may be that it’s not useful enough to keep as a label.

Agree, removed.

Not without write permission but maybe this could be something given to the ‘triage team’ - perhaps another reason for a higher barrier to entry. Alternatively, members could mention @parse-community/core-maintainers and one of us can do it.

Can we add a “transfer” label and the triage team gets a notification when the label is set, to act on it?

#6

Probably but it could be overkill considering I’ve only transferred ~4 issues in months.

#7

Alright, I’ve also had 2-3 transfers in the last month, but I don’t know what the total number of transferred (and should-have-been-transferred) issues across all repos is. Let’s go with your suggestion and we can add a transfer label later if needed.

What would be the process for the proposed change of labels? I think a poll / invitation for feedback for all contributors would make sense, since it affects all contributors. And a required majority core team confirmation since it concerns an essential instrument.

#8

There is no requirement for voting for something like this but I would agree that we should give all team members a chance to feedback and yes core team majority or ideally a unanimous approval.

It might be worth getting Diamond & Davi to give an opinion at this point as they deal with the bulk of incoming issues and then we can put forward a finalised proposal to everyone else but I’m equally happy if you want to put it to the whole team at once.

I also just though that from a functional point of view we may want some kind of bot that prompts team members to categorise issues with details on severity level etc, otherwise I think it could fall through quickly as people may either not bother or label inaccurately.

#9

@dplewis @davimacedo can we get your feedback on the “Suggestion” in the first post?

Good idea, I added that to the proposal in the first post for 2 cases:

  • bot requires issue to be classified after n days of being created
  • bot requires issue to be assigned a severity level if its classified as a bug after n days
#10

I think reviewing the current issue closing policy would also be a good idea.

For example: https://github.com/parse-community/parse-server/issues/6752

I closed the issue after waiting for 3 days for any response, because it seemed to be resoled. I wouldn’t want to close it too early, because the author may have additional questions or even propose a PR to implement the feature. But I also wouldn’t want to leave it open for weeks, cluttering the issue list with an already resolved question.

I think as it currently is, after some time of inactivity, the bot adds a won't fix label which does not correctly reflect the issue status and eventually closes the issue. I have seen this happening with some seemingly resolved issues, but I think manually closing makes more sense as it cleans up the issue list faster.

Should we
a) leave it up to the team member when to close an issue?
b) add a written guideline on when to close an issue, e.g. at least 3 days no further comment?
c) never manually close an issue and wait for the bot - and how long?

#11

I think the way you have been handling it is good. I agree that the won't fix label is bad, I discussed with the core team changing it to stale as the bot would suggest it should be anyway; we agreed upon this and since have changed it on the JS and iOS SDKs - I should also do this on Parse Server and anywhere else but feel free to beat me to it :sweat_smile:

I would go with option a, as I think it works for the most part and having a wait period would add too much overhead for maintainers. Having said that it could be worth having some general written guidance because sometimes issue closing can be dismissive and/or passive aggressive.

Option c could work if we had consistent & timely labelling because we could ask stalebot to avoid certain tags and have confidence that it would work smoothly but I think we would have to wait and see how well labelling works.

#12

I agree. For a guideline, the closing comment could include

  • why an issue has been closed and
  • invite the author to comment so it can be re-opened (unless the issue has been closed because things got off the rails)

For example:

I’m closing this as it seems to be resolved, feel free to comment if you have any questions and we can re-open this issue.

1 Like
#13

Replaced the CI label with a meta label to include pinned calls like “maintainers wanted”.

1 Like
#14

@dplewis, @davimacedo apologies for tagging you again. As @Tom already suggested, your input to refactoring the Github labels and triage process would be much appreciated, before we collect feedback from the broader community. All infos are collected in the first post of this thread.

#15

Hi, guys. Sorry for the delay. I think it makes a lot of sense to refactor our labels. We could also try to re-label and clean the current issues. We can then try to create a board in GitHub at least with the most critical bugs so we can have a clear vision of what it is the most important thing to do next. Overall I agree with your ideas and suggestions in this thread. I think we can summarize everything in the Governance repo? Then we can just make it live and discuss possible improvements when running the process.

#16

Absolutely that makes sense.

For bugs I can see that easily, because they are triaged and assigned a severity level. For feature requests I wouldn’t know how to prioritize them without polling the community.

I think we should invite all members of the team for feedback before publishing them. It’s a change that impacts virtually everyone and being a community project, we should maintain the team culture by including it in such decisions. (The StackOverflow moderator community gave a warning example over the last months of how important it is to consider team dynamics in a community of volunteers.)

#17

Nice. I think it makes sense. Let’s do that. We can draft a PR to the Governance repository and/or Create a new post with a summary of this thread. Then we can invite the others to comment and agree/disagree. @Tom is probably the best one to setup that.

1 Like
#18

After having another look over the proposal I have a few final thoughts…

  1. Not sure about going from ‘feature request’ to ‘feature’, I find that request describes the intent of the author well. Is the idea that ‘feature’ can then encompass tracking issues (for feature PRs) and discussions where the author intents to make a PR? If that is the case could we have two (or more!) separate labels or a couple sub-labels so maybe feature & request or feature & tracking… or something else?

  2. question & troubleshooting - I would push back against ‘trying to help there’ - I think the more we do that the more people expect help on GitHub - but I think it’s ok if you can provide a quick tip/solution to the author, I’m also hoping with some improvement in the issue templates we can decrease this load.

I’m not sure that there is an important distinction to be made here for issues. For me if it’s a request then feature request is a sufficient label whether its a small improvement or a significant new feature. Maybe we could use something more generic like ‘change request’ or ‘work request’ - feel free to come up with something better.

#19

Good point, changed.

I would suggest we start with a basic set and add more distinct “feature” labels as needed.

True that, I will remove “try to help here” from the description and add your “quick tip/solution to the author” and for more in depth issues refer to SO / community forum.

Then maybe the current label “enhancement” and its description is fine. I guess whatever change it is, it could always be considered an “enhancement” in some form, even if it’s cleaning up the code base by removing an unused / obsolete feature. I won’t change anything here, unless there is a suggestion.

#20

@Tom do we have a process to collect feedback from the Parse team regarding the change of labels?