Harmonize GitHub labels and usage

TLDR

This is an attempt to harmonize GitHub labels and their usages across Parse repos. Below are the new labels and usage suggestions, up for discussion. Since this impacts every team member, your feedback is most welcome!

Issue

Currently there is a variety of labels and usages on GitHub across Parse Platform repos:

  • available labels vary across repos
  • no prioritization for bugs according to criticality
  • some labels are de-facto unused and clutter the label list
  • there is no guideline on how to use labels for common understanding
  • some repos use labels also on PRs, others only on issues

This can lead to issues such as:

  • issues may not get the appropriate attention according to their criticality (eg. #6728)
  • labels may be confusing for contributors and followers
  • security issues that should be submitted confidentially may stay published publicly for long periods of time

There is currently no way to highlight issues that should be fixed with high priority:

  • most repos have a common bug label
  • some repos have an additional unbreak now label for critical bugs that seems unused and there is no guidance on when to apply it
  • there is a distinct process to expedite security issues, not applicable to non-security bugs

Suggestion

  • Define a uniform set of GitHub labels across all Parse repos (excluding the docs repo due do its speciality of referencing multiple repos).
  • Introduce bug severity levels in GitHub labels to increase awareness for important issues, expedite fixing these issues and merging associated PRs.
  • Introduce a guideline for team members on when to apply which label to facilitate uniform label usage.

(The following part is continuously updated to show the latest consensus; please see the edit history for any changes incorporated and see the thread for the change reason.)

Uniform GitHub labels

1. Initial status

  • no label (issue has not yet been look at by a Parse team member)
  • needs more info (more info is needed from author to classify the issue)

2. Issue classifiers:

  • :bug: bug (impairs a documented feature or a functionality that is not explicitly documented but one would strongly expect the general user to infer from a documented feature)
  • :rocket: feature (a new feature that does not exist yet)
  • :dna: enhancement (performance improvement or functional enhancement of existing feature)
  • :grey_question:question (not an issue but a question; should be in Community Forum but we can’t easily transfer, so we refer the author to the Community Forum)
  • :wrench: troubleshooting (not an issue but asking a code-level question; should be in StackOverflow; for simple issues we may give a quick tip / solution to the author, in general we refer to StackOverflow to educate the author about where to post code-level questions)
  • :speech_balloon: meta (issues regarding CI and tests or pinned calls for high visibility like “maintainers wanted”; other meta discussions are preferably held in the Community Forum)

2.1 Augmenting labels for bug severity

  • :rotating_light: S1 (catastrophic - blocks development or testing, may impact more than 25% of users, causes data loss, no workaround available, potential chemspill although we forward the issue into our distinct process for security related issues if found to be so)
  • :zap:S2 (serious - major functionality severely impaired and a satisfactory workaround doesn’t exist)
  • S3 (normal - blocks non-critical functionality and a work around exists)
  • S4 (small / trivial - minor significance, cosmetic issues, low or no impact to users

2.2 Augmenting labels for any classifier

  • duplicate (duplicate of already reported issue; references and closes the issue to encourage discussion in one place)
  • documentation (requires a change in the docs; since docs tend to be neglected, a distinct label should bring more attention to keeping the docs updated)
  • up for grabs (a bug, enhancement or feature that is not in the works; once a feature is being actively worked on, it is expected to infer that from the comments and the label can be removed; the label will be reapplied after prolonged time of inactivity)
  • on hold (the issue cannot be resolved anytime soon because it depends on a 3rd party or another issue that needs to be resolved first)
  • won’t fix (the issue won’t be fixed; the reason for this should be clearly stated)

Remove all other existing labels (non exhaustive list)

  • discussion, because every thread is essentially a discussion and every discussion is either regarding a bug, an enhancement, a feature or a question.
  • in the works, because we can never be sure whether a feature is still actively being worked on and we rather have more people looking at an open issue than less (see “up for grabs” label); the current status of an issue should be inferred from the last comments, not from a label that is likely to be outdated.
  • good first issue, because it seems to be rare that someone picks up an issue just to make a PR; more often an issue is picked up because someone faces the same roadblock.
  • long term, because it is unclear what it means; fixing a bug is long-term when no-one is working on it; other issues that currently use that label are not parse server issues and would be better placed in the community forum; if an issue cannot be solved because of an external dependency then a distinct “on hold” label makes it more clear.
  • needs investigation, because an issue usually needs investigation and it is a temporary label that needs to be removed at some point anyway.
  • pr submitted, because a PR is referenced in the thread anyway and a submitted PR doesn’t say anything about it’s viability or readiness for merge (sometimes PRs are submitted even before opening an issue and asking for community feedback).

Framework

  • Bug severity (S1-S4) can only be assigned by a triage team to ensure proper classification of and attention to critical issues. This triage team will be formed from the core team members. The purpose of restricting severity classification is to prevent inflationary severity use and ensure timely detection of critical issues.
  • A bot notifies core team members when an issue is not classified (as bug, enhancement, feature, etc) after 3 days of being opened, to mitigate the risk of missing a critical bug.
  • A bot notifies core team members when an issue that is classified as bug does not have a severity level assigned by the triage team within 2 days, to mitigate the risk of missing a critical bug.
  • A bot adds the up for grabs label to issues classified as feature or enhancement after 30 days of inactivity.
  • Issue transfers between repos happen by team members who have write access to the target repo, otherwise by asking @parse-community/core-maintainers in the comments.
  • Adding, editing, deleting labels is restricted by guideline (cannot be restricted by repo permission).

Governance

  • Labels play a fundamental role in issue severity classification and can have a significant impact on the product, therefore any future changes of the labels or the framework governing their usage requires a majority vote by the core team members.
  • Labels are used (interpreted or applied) by virtually all active team members on almost a daily basis, therefore changes of labels and their usage requires a non-binding feedback opportunity for team members.

The feedback period for the suggested changes above is until August 31, 2020, after which the latest consensus will be presented to the core team members for a final vote. A majority vote by the core team members is required to adopt the changes.

2 Likes

Suggestion for improvement of:

  • A bot notifies core team members when an issue is not classified (as bug , enhancement , feature , etc) after 3 days of being opened, to mitigate the risk of missing a critical bug.

Change to:

  • A bot notifies team members with write access to the repository when an issue is not classified (as bug , enhancement , feature , etc) after 3 days of being opened, to mitigate the risk of missing a critical bug. After 6 days of being opened, the bot notifies the core team members. The duration times double for issues with the needs more info label. A team member can always elevate an issue to the core team to expedite issue classification, regardless of any notification intervals.

Reason:

  • As seen in this issue, we may be waiting for more info from the author to be able to reproduce a bug or classify an issue as a bug. If we pause the bot indefinitely, we may miss critical issues, so the suggested alternative is to extend the period.
  • To avoid flooding the core team with notifications from all repos, a 2 step notification process (repo team members - core team members) may better distribute the work load.

Hi mtrezza,

I like the idea of consolidating the labels for the github repos to have a common understanding throughout the repos.
I’m part of the .NET SDK team and wondered how we may cover additional information about target platforms or frameworks (e.g. Unity, Xamarin, .Net Standard) with labels or if we should at all.
Some of the issues in the .NET SDK repo are related to specific target platforms, although only few really come down to be bugs and most others are questions which should be referred to stack overflow or the community forum I found it helpful to have those issues labeled for categorisation of some sort.

I assume this is kind of a special case with .NET but how may this circumstance be handled with your suggestions, May the teams introduce additional purely informative labels for such things as affected target platforms?

Cheers
Tobias

@TobiasPott Thanks for your feedback.

Sure, we can deviate from the suggested standard set of labels if you think it makes sense for the .NET repo. To find out whether it makes sense I think the gatekeeper questions would be:

  • How does the label improve the quantity / quality of issue handling?
  • How does the label benefit the maintainers / reviewers / authors / developers?

We actually have a similar situation with the Parse iOS SDK which covers iOS, macOS, tvOS, watchOS. @noobs2ninjas, @drdaz, @Tom what are your thoughts on platform labels for the iOS SDK?

I think platform labels could be helpful - I’m not active enough with the iOS SDK to give a thoughtful opinion. Similarly though, I do find with the docs repo that tagging issues & PRs with the guide they correspond to.

I think the difference here is that a docs tag actually points to a particular file in the docs, which is helpful since the docs are not yet consolidated. In iOS SDK / .NET SDK, the sub platforms share a common code base, so I’d assume that an issue usually affects all sub platforms.