Report Merging, Diffing and Comparing

Two or more HTML reports can be merged into a single report by verissimo.sh using multiple merge arguments.

The reports are merged by merging their categories, the checks within the same category, and the failures of the same check.

Categories

  • Two categories will be merged if they have the same category name and category description, otherwise both categories will be preserved in the merged report.

  • The merge between two categories is performed by merging their checks, if two checks are equal, they will be merged, otherwise both checks will be preserved in the merged category.

Checks

  • Two checks will be merged if they have the same name, title, description, status (enabled / disabled) and effort level.

  • The merge between two checks is performed by merging their hits, if two failures are equal, they will be merged, otherwise both failures will be present in the merged check.

Failures

  • Two failures are considered equal if they have the same file, line, macro stack, message and severity,

  • The merge between two failures is performed by merging their annotations, the resulting failure will inherit the annotations from both failures.

If the merged reports contain source code in HTML format from using the <include-html-code /> tag in the ruleset, these files will be copied in the merged report, clicking on a failure will jump to the relevant source file.

You can generate a HTML report containing the difference between two HTML reports by using the diff arguments.

The difference between the reports is computed by removing from the first report the failures that are also in the second report, for the same check in the same category.

You can specify the location of the merged, diffed or compared reports by using the flag html_report_location followed by the destination directory.

Note

Failures can be annotated when generating the original reports in order to use that information for filtering in the merged/diffed report.

Use the info argument followed by a “key=value” string when generating the reports. Multiple info arguments can be used if needed.

All failures in the generated report will have the annotation key with the value value, this information will become visible only when the reports will be merged, diffed or compared and will allow you to filter based on value.