In 2021, when our CTO asked us to improve developer productivity in one of our quarterly retreats, I took a chance to interview some developers. I interviewed 7 developers in our company reporting to other Engineering Managers. I found one of the topics of concern was that the PR reviews are taking a long time (3 to 4 days on average). This waiting period for the code review was demotivating for developers and this trend seems to be growing over time.
Thinking that I know the problem, I investigated the side of PR reviewers. Since I assumed the problem is known, I also assumed a target for reviewers as 1 working day to review the PR.
There were 2 kinds of PR reviewers
Engineering Managers who work on the project directly
Senior developers who work on their projects but review PRs to maintain code standards in the codebases they own.
Both of them had trouble reviewing PRs on the same day.
Senior Developers had very less context about the project and it was difficult to review a PR without taking some time to talk to the PR authors and know some depth in the project.
Managers get pulled into many projects and firefighting during the day, leaving less time to review the PR. I couldn’t interview more reviewers due to a time crunch. So as a PR reviewer myself, I questioned myself on what happens in my mind when I review a PR. I start for PR review only when I know I have ample time to review the PR (free 30 minutes in my calendar). Else, I would go on to look at daily activities in Slack threads.
Upon questioning myself more, I found that I also needed some additional context to do a PR review. The reason being the PRs are longer enough to occupy me for more than 10 minutes easily.
The average no of code changes in a pull request was 200 lines.
Some junior developers assumed 1 project is equal to 1 PR. Thus, sending one big PR by the end of the project. Since an entire project/feature is coming out as one PR, we had to remember a lot of functions and variables and logic while reviewing one PR. It occupied a lot of cognition and gave the sense of difficulty to review. Other senior devs and EMs also agreed to this point.
So, I googled how PR review processes and the size of the PRs in different tech companies. Since this is a common problem in many companies, in a few minutes of searching with different terms I hit a goldmine case study on PR reviews at Google.
I got to know the shocking average time taken for PR review and no of line changes:
Stripe → 10 minutes to 1 hour → 12 lines
Google → 1 to 4 hours → 24 lines
AMD → 10+ hours → 44 lines
The time taken to review grew exponentially with the size of the PR.
Now, let me explain how sending small PRs which is seemingly ineffective, will help solve review times for both developers and reviewers.
For developers, to send small PRs, you are supposed to know how to code for the project before starting it.
one reason for big PR is to start coding without much planning ahead and add up more code as it evolves and send it all in the end for review.
For reviewers, big PRs involved the complete context of the project but small PRs involved only a very small functionality you add to 1 or few functions/methods. The context here is much reduced and so less cognition to understand and review.
I personally was super happy as a reviewer in finding these.
Now, to implement this across the team, we had to educate both stakeholders.
Firstly, developers to have a plan on how many PRs they are going to send for the project and document it. They can send more PRs than the plan if needed but the point is to send small PRs.
To achieve this, devs are supposed to look a little deep to measure how big of the code we are going to write while the planning stage.
Junior devs were struggling to do this easily, so we helped them on how to split the PRs later once they finished coding.
Managers helped to split the PRs in the planning stage
One of our teammates wrote a blog to explain this.
Secondly, all reviewers are asked to comfortably and politely send back the PR if it is too long and ask them to split it.
This helped both devs and reviewers save at least 1 hour per week in PR reviews thus increasing a bit of developer productivity.
This also had nice second-order effects like
Better project estimations
some projects which supposed to take longer than initial assumptions were uncovered and estimated more practically
our internal jokes about project estimations were reduced considerably. 😜
New reviewers emerged
Since the no of line changes is less, seniors can comfortably ask for some PRs to be reviewed by new folks
Fewer comments in the PRs
since PR itself is very small, the PR description and one 2 lines description gave almost all the info needed to review it.
Thanks for reading 🙇.