Tuskr CLI Import Rules

Purpose

For effective QA analysis, it's advantageous to categorize and structure test cases systematically. In traditional approaches, QA teams often rely on developer collaboration to configure properties, resulting in time-consuming iterations and reduced productivity, leading to potential friction between teams.
The introduction of a rules file addresses this challenge.
With the rules file, test case attributes can be easily configured and organized using pattern matching, eliminating the need for constant developer involvement. This streamlines the process, empowering QA teams to efficiently manage test cases without relying on developer intervention.

Overview

The rules configuration is structured in JSON format, comprising two primary rules: defaults and always, alongside a series of pattern matching rules. These rules define attributes, as well as suite and section properties, for imported test cases.

JSON Structure

defaults: These rules are applied to every test case in the absence of specific properties in the JUnit XML file.
always: Rules applied universally to every test case before the application of pattern matching rules.
Pattern Matching Rules: These rules operate based on predefined patterns, such as setting the Suite to 'Performance' if the class name includes '.performance'.

{
    "defaults": {
      // default properties
    },
    "always": {
      // always applied properties
    },
    // pattern matching rules
  }

Attributes and Placeholders

Within the rules, placeholders enclosed in double curly braces (e.g., {{classname}}, {{name}}) dynamically insert values from the test case XML attributes into the properties of the imported test cases

Handling Custom Fields

Custom fields associated with test case can also be used along with other fields. Custom fields are specified within the customFields property as a JSON object, where each property name is the key of the custom field. These fields can be defined within in both the defaults and always sections of the rules configuration, as well as within pattern matching rules.

Sample Rules JSON with Custom Fields

{
  "defaults": {
    "name": "{{classname}} > {{name}}",
    "testSuite": "Suite 1",
    "automationId": "{{name}}",
    "customFields": {
      "priority": "Low",
    }
  },
  "always": {
    "testSuite": "Suite 2",
    "customFields": {
      "priority": "High",
    }
  },
  "name ~ link": {
    "testSuite": "Suite 3",
    "customFields": {
      "priority": "Critical",
    }
  }
}

Explanation

The execution order of rules is as follows: defaults are applied first to every test case, followed by always rules which can override defaults. Finally, pattern matching rules are evaluated and applied, potentially overriding both defaults and always rules.

For example, if a test case named "LoginLinkTest" is imported, it first receives the default settings (e.g., "Suite 1", priority "Low"). Then, the always rules are applied, changing its suite to "Suite 2" and priority to "High". If the test case's name matches a pattern (e.g., contains "link"), the corresponding pattern matching rules are applied last, which could further change its suite based on the specific conditions defined.

Using Matching Patterns

The property names in pattern matching rules can include matching patterns to conditionally apply rules based on the content of the test case attributes. The supported operators are:

Contains (`~`)

This operator is used to apply a rule if the attribute contains a specific substring.
Example: "name ~ error" would match any test case name that includes the substring "error".

Not Contains (`!~`)

Applies a rule if the attribute does not contain a specific substring.
Example: "name !~ deprecated" would apply to any test case name that does not include "deprecated".

Equals (`=`)

This operator applies a rule if the attribute exactly matches the specified value.
Example: "name = LoginTest" would only match a test case with the name exactly equal to "LoginTest".

Not Equals (`!=`)

Applies a rule if the attribute does not exactly match the specified value.
Example: "name != LogoutTest" would match any test case name that is not exactly "LogoutTest".

Rule Application Example with Multiple Test Cases

This example expands on the previous scenario to include three test cases from a JUnit XML file, each with distinct attributes. This demonstrates how different rules apply under varying conditions.

Test Cases

Test Case 1
- classname: LoginTests
- name: Login with Valid Credentials
Test Case 2
- classname: LoginTests
- name: Login with Invalid Credentials
Test Case 3
- classname: PaymentTests
- name: Payment in Dollars

Rules

{
  "defaults": {
    "name": "{{classname}} > {{name}}",
    "testSuite": "Suite 1",
    "testSuiteSection": "Section 1",
    "automationId": "{{name}}",
    "customFields": {
      "priority": "Low",
      "type": "Functional"
    }
  },
  "always": {
    "testSuite": "Suite 2",
    "testSuiteSection": "Section 2",
    "customFields": {
      "priority": "High"
    }
  },
  "name ~ invalid": {
    "testSuite": "Suite 3",
    "testSuiteSection": "Section 3",
    "customFields": {
      "priority": "Medium"
    }
  },
  "classname ~ PaymentTests": {
    "testSuite": "Suite 4",
    "testSuiteSection": "Section 4",
    "customFields": {
      "priority": "Medium",
      "type": "Non-Functional"
    }
  }
}

Application of Rules

Test Case 1: `Login with Valid Credentials`

After applying defaults, always, and pattern matching rules, this test case ends up in Suite 2, Section 2 with High priority and Functional type

Test Case 2: `Login with Invalid Credentials`

This test case is moved to Suite 3, Section 3, with Medium priority after matching the "name ~ invalid" pattern, demonstrating the override by specific patterns.

Test Case 3: `Payment in Dollars`

By matching the "classname ~ PaymentTests" pattern, this test case is placed in Suite 4, Section 4, with Medium priority and Non-Functional type, illustrating how different attributes trigger different rules.

Conclusion

The rules configuration feature provides a flexible way to customize how test cases are imported into the application. By understanding and utilizing the available operators and placeholders, you can efficiently organize and prioritize your test cases based on their attributes in the JUnit XML files.

Cypress Integration

Junit reference