...
What is a Kiuwan Audit?
...
...
...
...
...
- Audit Results Page
- Audit Result when using Local Analyzer
- Audit Result when using Kiuwan plugin for Jenkins
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
Other Audit and Checkpoints parameters
Before explaining the logic applied during Audit evaluation, we need to define a couple of concepts and parameters you can control in the definition of Kiuwan Checkpoints and Audits.
Checkpoint Weight
Every checkpoint has an associated Weight that represents the relative weight of the checkpoint in the Audit. The weights you specify translate (automatically) into a percentage contribution to the overall Audit.
For example, if your Audit has 2 checkpoints of equal importance, you should set this value to 1 for both, translating into a 50% contribution for each checkpoint. Now, if you consider that one is 2 times more important than the other, you should set them as 2 and 1 respectively, translating into a 66% and a 33% contribution.
Audit Approval Threshold
In a Kiuwan Audit you can specify an Approval Threshold. This threshold will represent the minimum percentage of checkpoints contribution to consider the Audit as OK. Independently if they are mandatory or not, only the contribution percentage of each checkpoint is taken into account to evaluate this threshold.
Learn how the Audit evaluation logic works in the next section.
Audit evaluation logic
The logic behind of audit evaluation is based on two-steps
- All mandatory checkpoints must be successful. Otherwise, the Audit will FAIL.
- If all the mandatory checkpoints are OK, the sum of successful checkpoint percentage contribution (based on the defined weights) must be higher than the Audit Approval Threshold for the Audit to be OK. Otherwise, it will be FAIL.
Let's see this logic applied to some examples.
Example 1
Audit Approval Threshold = 75%
Checkpoint | Mandatory | Contribution | Result |
A | Yes | 50% | FAIL |
B | No | 30% | OK |
C | No | 20% | OK |
Audit will FAIL. Mandatory checkpoint has failed, therefore Audit result is FAIL.
Example 2
Audit Approval Threshold = 75%
Checkpoint | Mandatory? | Contribution | Result |
A | Yes | 50% | OK |
B | No | 30% | FAIL |
C | No | 20% | OK |
...
...
...
...
...
...
...
...
Audit Management
To access Audit Management module, select "Audits Management" option from the configuration drop down menu.
Icon
Only users with "Manage audits" privilege will be allowed to access Audit module.
You will go directly to the audit summary page for the default selected Audit.
Kiuwan comes with an off-the-shelf Default audit. This audit cannot be modified by end users but can be used in any application. In fact this is the Audit assigned to any new application in Kiuwan by default.
Default audit comes under Shared Audits section in left panel. Any user-defined audit will be under My audits.
Clicking on any audit name will allow you to view/manage it.
Create an Audit
To create a new Audit, click on New link at the end of the Audits list in the left side panel.
You have provide a Name and an optional Description.
Approval Threshold represents the minimum percentage of checkpoints contribution to consider the Audit as OK. After audit execution, this value is used to evaluate if the audit passes or not. Please see Audit evaluation logic above to know how this value is used in audit evaluation.
Click Create Audit to save the new audit and have it available under My audits
Every Audit needs to have at least 1 Checkpoint. Therefore, once you create an audit, the next natural step is to create checkpoints.
Checkpoint management
For any selected Audit, the Checkpoints tab will show all the defined checkpoints in a table.
To facilitate working with checkpoints when you have many, you can filter them by Name, Type or Mandatory status; in the filter panel above the table.
Clicking on the checkpoint Name you can modify the checkpoint details and definition.
In this page, you can also directly modify the weights of the checkpoints. By introducing integers in the Weight text box, Kiuwan will automatically calculate the contribution percentage of every checkpoint in the audit. This way, you can easily fine tune checkpoint contributions without editing every individual checkpoint.
Similarly, you can also make each checkpoint Mandatory or Optional with the Mandatory checkbox directly in the checkpoint list.
The Add Report Section button, allows you to create sections to group checkpoints. When exporting the audit results to a PDF report, these sections are used to group audit results as well. You can define the order of sections and checkpoints using the arrows in the first column of the checkpoints table.TO move checkpoints across sections just use the arrows in the checkpoint until you place it in the section you want. The order defined here is used just to display results, it doesn't affect the Audit evaluation logic explained above.
Create a Checkpoint
To create a new checkpoint, click the Add checkpoint button.
To create a checkpoint you should provide Name and Description.
Weight represents the relative weight of this checkpoint in the Audit.
Every checkpoint has an associated Weight that represents the relative weight of the checkpoint in the Audit. The weights you specify (integer values) will translate (automatically) into a percentage contribution to the overall Audit. Please see Audit evaluation logic to fully understand how this value is used in audit result calculation.
Mandatory checkbox indicates if the checkpoint is Mandatory (checked) or Optional (unchecked).
Maximum threshold indicates the maximun number defects that are allowed. When audit is executed, if the number of defects is higher than this value, the checkpoint will FAIL.
Type combo allows to select between the available checkpoint types. Please see Checkpoint Types for an explanation.
Available checkpoint types:
- Threshold for total maximum number of defects of specific types
Sets a maximum number of defects allowed in the application for the specific defect types selected
- Threshold for maximum new defects of specific types
Sets a maximum number of new defects allowed in the application for the specific defect types selected
- Threshold for total maximum number of defects
Sets a maximum number of defects allowed in the application for the defined languages, categories and priorities
- Threshold for maximum new defects
Sets a maximum number of new defects allowed in the application for the defined languages, categories and priorities
Depending on the selected checkpoint type, you wiil be able to specify that only defects of selected languages, characteristics and priorities will be taken into account when evaluating the checkpoint.
In this example, the checkpoint evaluation will only compare with the threshold those defects of Very High priority belonging to Maintainability or Security in Java or Cobol source files.
In case you prefer that only defects of certain types be considered, click on Search defect types and select the desired rules by searching the Kiuwan rules repository.
After selecting the rules, the checkpoint will check only defects of those types.
Associate an Audit to an Application
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...