Effective Assignment and Assistance to Software Developers and Reviewers
What factors influence the reviewer consignment to pull requests?
Abstruse
Context
When external contributors want to collaborate with an open up-source project, they tin can send a pull request to the project core team. Later on, a cadre team fellow member is assigned to review the pull request. Recently, some techniques for recommending reviewers to analyze pull requests were proposed. Notwithstanding, they replicate previous (not e'er desired) patterns and exercise not provide the rationale behind the recommendation.
Objective
This newspaper aims at understanding the factors that influence on assigning reviewers to pull requests and evaluating the extent of this influence.
Method
To accomplish this goal, we mined association rules from 22,523 pull requests belonging to 3 projects hosted on GitHub. In addition to showing such patterns, we besides nowadays a qualitative analysis that explains why the patterns arose.
Results
In summary our results point that (i) some reviewers always analyze smaller pull requests, with few commits and few files; (ii) some reviewers frequently (up to 58% of the times) clarify pull requests filed by inexperienced requesters; (iii) some reviewers take more than chances (upwardly to 25 times) to analyze pull requests filed by requesters of their acquaintance; and (4) some reviewers have more chances (upwards to twenty times) to analyze pull requests containing files that they take recently changed.
Conclusion
In detail, the results allow for the following conclusions: (i) factors such as number of commits and files in the pull request may influence the reviewers assignment; (ii) factors regarding the requester profile may influence on reviewer allocation; (iii) the social relationship between requester and reviewer exert influence on pull request evaluation, that is, when the reviewer knows the requester, his or her chances of evaluating such contributions may increase; and (iv) factors such as ownership and locality of pull request artifacts are important predictors for the reviewer. Furthermore, nosotros point out that, likewise identifying influence factors related to pull request reviewer, the adopted arroyo immune us to quantify the extent of that influence via support, confidence, and lift metrics.
Introduction
Open-source software development happens in a collaborative and distributed context, which enables development teams to receive external contributions. External developers tin can contribute to the project by coding, documenting, testing, or performing other software engineering science activities.
An emerging epitome employed for the systematization of contributions in open-source projects is named pull request [ane]. According to this prototype, external contributors can make isolated changes over artifacts and then request the integration of their changes into the principal projection repository. A pull request containing either bug fixes or new features, after an analysis carried out past a member of the core team, may be accepted (merged) into the project main repository or rejected (airtight).
Still, the process of assigning a reviewer to analyze a pull request is non always well divers. The decision to review a given pull asking may come directly from the cadre team members with interest in the pull request, potentially incurring in inappropriate assignments. Equally a event, a series of studies arose [2], [three], [iv], [five], [6], [seven], aiming at recommending reviewers to analyze pull requests. In general, these studies employ predictive techniques for suggesting reviewers based on the project history. Nevertheless, these recommendations generally replicate previous (non e'er desired) patterns and work as black boxes–i.e., they practice not uncover the reasoning behind the recommendation. Besides, the quality of the recommendation is tightly related to the called predictive attributes.
Understanding the factors that influence on assigning reviewers to pull requests is crucial to better blueprint recommendation tools and provide adequate support for project managers. These factors may reveal possible conflicts of interest among requesters and reviewers, unbalanced workloads amid reviewers, and regions of the codebase with few experts, representing a risk in the face of turnover. Replicating such issues in the time to come recommendation is non desired, but a naïve implementation of a classifier tin can pb to the reinforcement of these undesirable patterns, which should never accept existed. By knowing these patterns, developers of recommendation tools can add heuristics that prevent them. Besides, comprehending the nature of reviewers allotment for analyzing pull request is a first pace for identifying the aforementioned problems and quantifying how frequently they occur.
In this piece of work, we mined association rules from 22,523 pull requests to identify factors that influence the assignment of pull request reviewers. More specifically, we answer the following research questions:
-
RQ1 -Practice the pull request characteristics influence on the reviewer consignment?
-
RQ2 -Does the requester profile influence on the reviewer assignment?
-
RQ3 -Does the relationship betwixt the requester and the reviewer influence on the reviewer assignment?
-
RQ4 -Does the location of a pull asking influence on the reviewer consignment?
We could unveil the following patterns, amid others: (i) some reviewers always clarify smaller pull requests, with few commits and few files; (ii) some reviewers frequently (up to 58% of the times) analyze pull requests filed past inexperienced requesters; (three) some reviewers have more chances (up to 25 times) to analyze pull requests filed by requesters of their associate; and (4) some reviewers have more chances (up to twenty times) to analyze pull requests containing files that they have recently changed. Too the quantitative analysis using association rules mining, nosotros also run a qualitative analysis, aiming at amend understanding some of the uncovered patterns.
In previous work [eight], [9], we analyzed the factors that influence the acceptance and rejection of pull requests in open up-source projects. Although both previous and this work contribute to understanding the nature of pull requests ameliorate, this work has a completely different goal: understanding the factors that influence on reviewers assignment instead of credence/rejection. Moreover, in previous work, nosotros have besides practical classification algorithms for recommending developers to review pull requests [4]. However, the results of our prediction, similarly to any other recommendation tool for assignment of pull requests, do not present the reasoning behind the recommendation and is subject to the previously discussed problems. Herein, we bear a wider exploratory analysis on reviewers assignment to pull requests. Therefore, this work tin can be seen as complementary to recommendation tools, as our results may be used to improve the accurateness of these recommendation tools further.
The residue of this work is organized every bit follows. Section 2 offers a discussion on pull-based evolution. Next, in Department 3 we present our enquiry method, project corpus, and a general overview of the employed techniques. In Section 4, we testify the obtained results, answer each of the inquiry questions, and discuss the implications of our answers. Adjacent, in Section 5 we discuss the master related work. Finally, Section vi concludes this work by summarizing our man contributions and pointing out some hereafter work.
Section snippets
Background
In open-source projects, the systematization of the contribution process is utterly relevant, and the pull-based model tin can help on such job. In this model, a pull request contains a set of changes to software artifacts, which may be incorporated into the main repository (receiving an "accepted" status) or airtight (receiving a "rejected" status) [one]. This model of contribution is based on a workflow composed of seven steps [ane]: (1) Forking the main repository; (2) Cloning the forked repository;
Materials and methods
In this work, we adopted a data mining technique chosen association rules extraction. This technique can unveil implicit information in the data that could exist ignored should an exploratory analysis exist carried out. The information mining process considers information related to the characteristics of the pull request, requester profile, social relationship between requester and reviewer, and alter location.
Specifically, our study employed the Knowledge Discovery in Databases (KDD) process, equally follows [10]:
Results and give-and-take
In this Department, nosotros discuss the results of our analyses and present answers to the enquiry questions defined in Section iii. Section 4.1 presents a discussion on pull request characteristics that influence the reviewers assignment. Section four.ii brings the discussion on factors related to the requester profile influencing the reviewers consignment. Section 4.3 presents the influence of social factors on the consignment of reviewers to pull requests. And so, Section 4.4 brings the results on how the
Related work
Recently, some studies have explored strategies to recommend reviewers to analyze pull requests. However, none of these studies attempt to depict or explain patterns regarding the factors that influence on the reviewer assignment. Consequently, although related to our work, they are not direct competitors. They tin can do good from having an in-depth picture show of what factors influence on the reviewers consignment, provided by our work.
Yu et al. [ii] proposed a tool to recommend developers to review
Conclusions and hereafter work
In this paper, we presented a report most the factors that influence on the programmer consignment to review pull requests in open-source projects. Our results reveal patterns that may provide insights both to recommendation tool developers and projection managers. These results may help in explain influence factors for unbalanced workloads amongst different reviewers. Moreover, the results may indicate regions of the codebase with few experts, representing a run a risk in the face of turnover.
In
Acquittance
Nosotros would like to thank CNPq (305569/2014-7, 308369/2015-7), CAPES, and FAPERJ (E-26/201.523/2014) for the financial back up.
References (19)
- et al.
Reviewer recommendation for pull-requests in github: what tin can nosotros learn from code review and bug assignment?
Inf. Softw. Technol.
(2016)
- Distributed Git, in: Pro Git, Apress, 2009, pp....
- et al.
Reviewer recommender of pull-r in GitHub
Proceedings 2014 IEEE International Briefing on Software Maintenance and Evolution (ICSME'fourteen)
(2014)
- et al.
Developers assignment for analyzing pull requests
Proceedings of the 30th Annual ACM Symposium on Applied Computing
(2015)
- et al.
Earec: leveraging expertise and potency for pull-request reviewer recommendation in github
Proceedings of the 3rd International Workshop on CrowdSourcing in Software Engineering
(2016)
- et al.
Correct: code reviewer recommendation in github based on cross-projection and technology experience
Proceedings of the 38th International Conference on Software Engineering Companion
(2016)
- et al.
Coredevrec: automatic cadre fellow member recommendation for contribution evaluation
J. Comput. Sci. Technol.
(2015)
- et al.
Credence Factors of Pull Requests in Open up-Source Projects
Proceedings of the 30th ACM/SIGAPP Symposium On Applied Computing, (SAC'15), Salamanca, Kingdom of spain
(2015)
- et al.
Rejection factors of pull requests filed by core team developers in software projects with high credence rates
Proceedings 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA'fifteen)
(2015)
There are more references available in the full text version of this commodity.
Cited by (22)
Recommended manufactures (6)
© 2018 Elsevier B.V. All rights reserved.
Source: https://www.sciencedirect.com/science/article/abs/pii/S0950584917303804