Contents x
      Monitoring (Classic Flow)
      • 02 Mar 2023
      • 14 Minutes to read
      • Print
      • Dark
        Light
      • PDF
      Contents

      Monitoring (Classic Flow)

      • Updated on 02 Mar 2023
      • 14 Minutes to read
      • Print
      • Dark
        Light
      • PDF
      Article Summary

      Note:
      The Monitoring page is only available to legacy "classic" workflow users. 

      Monitoring Dashboard

      The Monitoring dashboard allows you to monitor the performance of your deployed Model (#1 in the screenshot below). You can also be notified of any changes to the Model's performance by creating Alerts (#2 in the screenshot below). The Monitoring dashboard generates data by pulling information from images predicted via Run Live (or Cloud Inference). Run Live is performed on the Live Prediction tab. 

      Note:
      If the Monitoring tab is empty, see Generate Reports on the Monitoring Tab for help.
      Monitoring Tab 

      Generate Reports on the Monitoring Tab

      When you first open the Monitoring tab, it may be empty. To populate information on the Monitoring tab, go to the Live Prediction tab and select a Device and Deployment Bundle.

      Select a Device and a Deployment Bundle on the "Live Prediction" Tab
      Analyze Reports on the Monitoring TabThe name of the selected device will display above the Detection Rate report. In the screenshot below, the name of the default device is 11589.
      Device Name

      Analyze the Detection Rate Report

      Note:
      The Detection Rate report is generated according to the specified date range.

      The data in the Detection Rate report is separated into categories:

      • Avg. Detection Rate: The average of the total number of images with at least one Class detected over the total number of images analyzed. For example, if the Model detected a Class in 6 out of 8 images, the average would be 75%. (6 ÷ 8 = 0.75)
      • Avg. Confidence: The average of the total Confidence Score over the total number of Classes detected.
      • # Instances Detected: The number of images with at least one Class detected.
      • # Images Predicted: The number of images predicted.
      Detection Rate Report Categories 

      View Data for Specific Dates 

      By default, the Detection Rate report shows the calculations for the last seven days; this report does not show the entire history of detection rates. You can see data relevant to a specific date range by selecting dates from the drop-down menu.

      Note:
      The Detection Rate report can only show up to 14 days of data.

      Select a Timeframe 

      View Graphs 

      You can click each category to see a graph of the relevant data.

      Monitoring Graphs

      Alerts

      Let's say you want to be notified when the number of detections increases substantially. Or, maybe you want to be notified when the  Confidence Score decreases significantly. You can be notified by creating Alerts.

      Receive Notifications by Creating Alerts 

      Alert Types

      Alert TypeDescription
      Average Defect RateBe notified when the Average Detection Rate is at or below the specified number. 
      Average Confidence Be notified when the Average Confidence Rate is at or below the specified number. 
      Volume DecreaseBe notified when the Inference volume goes below the expected range at least 3 times during the provided time window. 
      Inference ErrorBe notified if at least one internal server error is detected during the specified time window when running Inference.

      Note:
      Landing AI will always be notified if an internal server error is detected.

      Alert Settings

      Alert Settings 
      #SettingDescription
      1TitleEnter a short, descriptive name for your Alert. This name will help users understand what the Alert is for.
      2Select DeviceSelect the Device that the Alert should be for.
      3Alarm TypeChoose the type of Alert you want to be notified for. 
      4ThresholdEnter a number between 0 and 1. You will be notified when the specified metric exceeds this number during the set Time Window or Count. 
      5Window Type & Time Window / Count
      • Count Based & Count: Configure the number of times the event should occur to be notified.
      • Time Based & Time Window: Specify the number of minutes that the event should occur at least once to be notified. For example, if you want to receive a notification if an Inference Error should occur in a single day, enter 1440 (or 24 hours). 
      6RecipientsSelect the users to be notified.

      Create Alerts

      1. Go to the Deploy page.
      2. On the Live Prediction tab, select the Model Bundle and Device you want to create an alert for from the drop-down menus.
        Model Bundle and Device
      3. Go to the Monitoring tab.
      4. In the Alerts section, click Manage.
        Manage Alerts 
      5. Click Create Alert.
        Create Alert Button 
      6. The Create Alert pop-up window displays. Enter a short, descriptive name for the Alert in the Title field.
      7. Select the device you want to create an Alert for from the Device drop-down menu.
      8. Select the type of Alarm you want to create from the Alarm Type drop-down menu.
      9. Configure the Alarm settings.
      10. Click Create.
        Create Alerts 

      Edit or Delete Alerts

      Caution:
      Removing an Alert is an irreversible action that will delete it from LandingLens. Consider deactivating an Alert if you plan to use it again in the future.

      To edit or delete an Alert:

      1. Go to the Deploy page.
      2.  On the Live Prediction tab, select the Model Bundle and Device that has the Alert you want to remove from the drop-down menus.
        Select a Deployment Bundle and Device 
      3.  Go to the Monitoring tab.
      4.  Click Manage in the Alerts section.
      5.  Click the vertical ellipses icon (⋮) and select Edit or Removeaccordingly.
        Edit or Remove an Alert 

      Deactivate Alerts

      If you have an Alert that you no longer want, you can deactivate it. You can always reactivate it later.

      To deactivate an Alert:

      1.  Go to the Deploy page.
      2.  On the Live Prediction tab, select the Model Bundle and Device that has the Alert you want to remove from the drop-down menus.
        Select a Deployment Bundle and Device 
      3.  Go to the Monitoring tab.
      4.  Click Manage in the Alerts section.
      5.  Click Deactivate for the Alert you want to turn off.
        Deactivate an AlertTo reactivate an Alert, click Activate.
        Reactivate an Alert
      Was this article helpful?
      What's Next