Developing a tool for automated code review in ServiceNow

At GuideVision, we place great emphasis on excellence in our work implementing solutions based on ServiceNow. Our team is constantly trying to come up with new ways to improve the quality and speed of our delivery. This quest for quality led Filip Vacula to write GuideValidator, a tool that automates aspects of the code review process. Not only did Vacula create the software, but he also used it as the focus of his thesis for his master's degree in applied computer science at the Prague University of Economics and Business this year. As part of the thesis, Vacula wrote a meticulously-researched case study based on GuideValidator, providing an insight behind the scenes at GuideVision. 

GuideValidator is in daily use here at GuideVision, and we believe it’s a very effective tool. In addition, it serves as a good illustration of the technical capacities of the GuideVision team. What follows is a summarised version of Vacula’s master’s thesis. 

By Filip Vacula, Principal ServiceNow Consultant at GuideVision

Technical debt, code review and related concepts of software development were first explored and then applied for the specific case of GuideVision, a technical consultancy with an exclusive focus on the ServiceNow platform. A tool for automated code review was developed as part of the thesis, and after two years of active use was evaluated in a single case study. The study found out that the tool provides value to the company by lowering the time to perform code review, increasing quality of delivered code, speeding up the development by minimizing handoffs between people and has five additional secondary benefits. However, there is potential for increased value if identified areas for improvement are addressed. The most impactful is universal use on all customer projects, which is currently limited due to security policies.

ServiceNow is a cloud platform with extensive customisation possibilities, which makes it a popular platform for uses even beyond its original IT Service Management scope. The possibilities for customisation, however, also open additional opportunities for lower customisation quality.

“Quality is king”

GuideVision’s internal motto is “quality is king.” The focus on quality runs through the entire process of engagement with its customers – starting with precisely and transparently estimated offers to potential customers, continuing through the analytical phase to development and testing. GuideVision is strong at maintaining good relationships with its customers, which usually results in long-term cooperation. When one project finishes, the project team is often kept together in order to start another project with the same customer. Combined with the use of agile scrum methodology on all projects, this means that code is maintained for a long time and is changed frequently. All of these factors contribute to the need of keeping the highest quality standards of the delivered code. GuideVision takes pride in its own extensive list of coding standards and practices, which are all based on past experience.

The demand for ServiceNow consultants is very high, which means that hiring senior ServiceNow consultants is a great challenge. Combined with GuideVision’s goal to grow at least at the same speed as ServiceNow does, it means that most new hires are juniors with no previous ServiceNow experience. The education benefit is especially important to enable and speed up the growth of talent.

To sum up, there are 3 benefits of code review for GuideVision:

• delivering high-quality customisations,
• proving the quality of customisations delivered,
• educating developers.

Implementing GuideValidator

GuideValidator is an extensible tool for performing automated static code review in ServiceNow. Its extensibility lies in the way validation rules are managed, as it is not required to modify implementation of the validation logic to add new rules. The application was entirely developed on the ServiceNow platform, starting from the database structure, through the validation logic and implementation of the validation rules, ending with the user interface.

At the time of starting the production use of GuideValidator in late 2018, GuideVision was a small-to-medium-sized company with around 100 employees. Owing especially to its rapid growth – roughly doubling its count of employees each year – delivery processes were at a low maturity level. While some standards were followed, the processes were never formally documented. This proved to be a challenging proposition for GuideValidator, as it was not immediately clear where to place it in the development process. The biggest question was the order of activities - Internal testing, automated code review using GuideValidator and human code review. The goal was to make the process as efficient as possible, minimizing the time involvement of senior team members performing the human code review and reducing the handoffs between team members wherever possible.

Case Study Findings

GuideValidator was found to decrease the time to perform manual code review by 63 hours per year, if only the time to log findings is considered. It increases the quality of delivered code by finding violations which might go unnoticed due to time pressure or by being inadvertently overlooked. It can be used when manual code review is skipped to ensure at least a base level of quality. An indirect way how code quality was improved is by discussing and formalizing the development process which includes code review. GuideValidator speeds up the development by lowering the number of code review cycles per user story and by minimizing the delays in process.

This finding is confirmed also by interviews with GuideVision team members; the 6 interviewees unanimously agreed that GuideValidator does decrease the time to perform code review. One of the quotes also confirms the hypothesis that when trivial findings are automated, the code reviewer can focus on more complex issues: “you can really focus on the actual code functionality and performance optimizations and you don't have to really solve the code quality from the, for maintaining an coding policy perspective.”

However, this hypothesis is only partially confirmed. It was based on the assumption that code reviewers skip coding standards which are covered by the GuideValidator – that they simply do not check for these. The survey showed that in three out of four cases, code reviewers are carefully checking violations even though they are covered by the tool. Interviewee A explains “it’s a tool and I'm tending not to trust tools and technology, I don't know why, it's my nature. So I would be checking all the same.”

There were 5 additional benefits mentioned in the interviews: Learning coding standards; no human involvement; sanity check for the developer; and reportable outputs. In addition, six different areas for improvement of GuideValidator were identified. The use of the tool with customers who have restrictive security policies was identified as one of the key roadblocks to adoption - the limiting factor in its widespread use on all GuideVision projects.

As the results of the case study show, GuideValidator has achieved the goals it set out to achieve. It measurably decreases the time to perform code review, improves the quality of delivered code and increases development velocity. However, due to the known areas for improvement, the benefits are not realized to their full potential. With additional investment, the reach of the tool can be increased, and with it the total value it brings.

We have also found out that there are aspects which vary greatly between customers – for example their own coding standards, how restrictive they are, whether they care about code quality or not. There are also differences between code reviewers – their motivation to perform code review, whether they like this activity, or if they check the results of GuideValidator.

Automating code review is valuable, especially in the ServiceNow platform. However, it

has to be usable with all customers, has to be easy to use by the developers and provide immediate feedback, otherwise the value it brings will be limited, and economic feasibility of building one’s own solution might not be justifiable.

What comes next

In the Quebec release, ServiceNow introduced Instance Scan, a new functionality targeted at a similar use case as GuideValidator. With the improvements we see in each major version, ServiceNow is putting an effort into making Instance Scan a complete solution for automating code review. Utilizing built-in platform capabilities is typically more efficient than maintaining custom ones, and using Instance Scan will allow for wider project adoption. That’s why GuideVision is currently in the process of transferring the validation rules from GuideValidator to “Checks” in Instance Scan. GuideValidator will then be decommissioned, but its legacy will continue to live with the new technology.

You can read the full thesis at https://vskp.vse.cz/english/84498.

Interested in learning more about the GuideValidator tool and how it can be used for your ServiceNow instances? Feel free to get in touch - we’d love to hear from you.