# Unit Testing

# What

# What is unit testing in the Clinc AI Platform?

On the Clinc AI Platform, you are offered tools for testing part of the functionalities such as classification and SVP as well as the whole AI pipeline i.e. end-to-end testing. In this section, we will dive deeper into the classification unit testing and SVP unit testing.

# Classification Unit Testing

Classification unit testing allows you to add data to intents in a certain AI version to test if the classification transition with which the intent is associated is taken successfully. It is especially useful when an intent is used under multiple transitions.

# SVP Unit Testing

SVP unit testing allows you to add data to competencies which have slots in an AI version to test if the SVP model can extract the values correctly. Comparing to test queries in the query sidebar and full conversation testing, the SVP unit testing is more focused and provides direct insights into the robustness of the SVP model.

# How

How to use the classification unit testing page?

How to interpret classification unit test results?

How to use the SVP unit testing page?

How to interpret SVP unit test results?

# How to use the classification unit testing page?

# Access the Classification Unit Testing Page

Go to the Test tab on the navigation bar.
Create a test suite or choose the Default Test Suite. To learn what is a test suite and how to create it, go to the end-to-end testing page.
Slide to Classification Unit where you will see a list of all the intents inside the AI version.

# Add Utterances to Intents

Click Add Data or for intents you have added utterances before (or if you wish to view a more detailed result), click Edit.

clf_unit_test

Similar to adding training data, put the testing utterances in the Add new utterance field, then choose Add.
Don't forget to Save.

add testing utterance

When you have a sufficient amount of testing data, you can now Run Test.

# Delete Utterances

To Delete utterances, select the ones you wish to delete, a trash icon will appear. After you delete the utterances, click Save.

# Import/Export Test Cases

You can also repurpose the test cases in each intent by using the Export function. It can be downloaded as a .JSON or CSV file.

import export clf data

To Import test cases, the data files need to take the following formats:

JSON

{
    "sentences": [
        "I want two of those",
        "Take that out",
        "Scratch the chicken sandwich"
    ]
}

CSV
Similar to the JSON format, the CSV file shows the key value pair in the form of a single column table.

clf unit test csv

# How to interpret classification unit test results?

The expand button next to each utterance allows you to view at which transition the utterance failed to take. It also shows the actual intent the utterance is classified into, which provides guidance for classifier model training moving forward.

clf unit test result

The drop-down menu on the upper left corner provides a filter for viewing test results. It shows the passing rate for all the transitions with which the intent is associated. Depending on the transition selected, the Passing/Failing status showing below can be different.

clf_unit_test_filter

# How to use the SVP unit testing page?

# Access the SVP Unit Testing Page

Go to the Test tab on the navigation bar.
Create a test suite or choose the Default Test Suite. To learn what is a test suite and how to create it, go to the end-to-end testing page.
Slide to SVP Unit where you will see a list of all the competencies (with slots) inside the AI version.

# Add Testing Utterances

Click Add Data or for competencies you have added utterances before (or if you wish to view a more detailed result), click Edit.

svp_unit_test

Similar to adding training data, put the testing utterances in the Add new utterance field, then choose Add.
Don't forget to Save.

add svp testing utterance

When you have a sufficient amount of test utterances, you can now Run Test.

# Delete Utterances

To Delete utterances, select the ones you wish to delete, a trash icon will appear. After you delete the utterances, click Save.

# Import/Export Test Cases

Similar to classification test cases, you can repurpose the testing cases in each competency by using the Export function. It can be downloaded as a .JSON or CSV file.

import export clf data

To Import test cases, the data files need to take the following formats:

JSON

  {
      "sentence": "Is my next event at the City Center?",
      "svpLabels": [
          {
              "label": "O",
              "word": "Is"
          },
          {
              "label": "O",
              "word": "my"
          },
          {
              "label": "O",
              "word": "next"
          },
          {
              "label": "O",
              "word": "event"
          },
          {
              "label": "O",
              "word": "at"
          },
          {
              "label": "O",
              "word": "the"
          },
          {
              "label": "B-location",
              "word": "City"
          },
          {
              "label": "I-location",
              "word": "Center"
          },
          {
              "label": "O",
              "word": "?"
          }
      ]
  },

CSV
Similar to the JSON format, the CSV file shows the utterances with the words and labels. Tokens that are belong to a slot are labeled B-slot_name or I-slot_name. 'B' stands for 'beginning', 'I' stands for 'inside'. The rest that should not be labeled as slot values are labeled 'O' which stands for 'outside'.

svp unit test csv

# How to interpret SVP unit test results?

The expand button next to each FAILING utterance allows you to view what the slot is. It also shows the expected value and the actual value that is extracted, which provides guidance for SVP model training moving forward.

svp unit test result

On the SVP unit test overview page, the expand button next to each competency provides insights into the number of correct extractions and expected extractions. The difference between those two numbers tells you that there are slot values that are supposed to be extracted but failed to, in other words, false negatives. The false positives tells you that there are words that are not slot values being extracted. The false positives and false negatives together serve as guidance for SVP model improvements. The platform also exposed the F1 score (opens new window) for test precision purposes.

svp_unit_test_filter

Last updated: 10/09/2020

← Query Testing End-to-end Testing →