Monitor the impact of Predict models in Explore Liveboards

If you use Guidewire Predict, you can create Explore Liveboards to visualize and monitor how your predictive models impact business key performance indicators (KPIs). These business impact monitoring (BIM) Liveboards can help you prove the value of the models or discover where they need improvement. For example, to quantify the value of a claims model, you could monitor the average number of claims paid before and after the model was implemented.

Explore has some pre-configured BIM Liveboards, but you can also create your own.

Required applications and services

To create BIM Liveboards, your organization must already have the following Guidewire applications and services in Guidewire Cloud:

Predict, with at least one model deployed and sending scores to Analytics Manager.
Analytics Manager, with a solution enabled to mediate between Predict and an InsuranceSuite core application.
An InsuranceSuite core application, such as ClaimCenter, with feature flags enabled to interact with Analytics Manager. (See the ClaimCenter Configuration Guide.)
Data Studio
Explore

How data flows from Predict to Explore

Data flows from Predict to an Explore BIM Liveboard as shown in the following diagram. This diagram uses ClaimCenter (CC) as the example InsuranceSuite application, but you can also create Liveboards for other core applications.

Collect scores and assessments:
1. One or more Predict models process ClaimCenter data and return scores to Analytics Manager.
2. Analytics Manager writes the scores to ClaimCenter’s assessment summary table. Depending on its configuration, Analytics Manager also assigns meaning to the scores, such as risk-level assessments and recommended actions. This information is also added to the assessment summary table.
3. The assessment summary table flows into Data Studio along with other data from InsuranceSuite databases.
Create a data source: In Data Studio, you join the assessment summary table to a claims dataset, creating a new curated dataset.
Visualize the data: You publish the new curated dataset to Explore, create a model, and use the model to create a Liveboard.

How to configure a Predict impact monitoring Liveboard

Before you begin

You must have at least one Predict model deployed and sending scores to Analytics Manager.
Note the date that the Predict model was deployed.
Become familiar with the purpose of the model and which KPIs it impacts.

Procedure

Create a dataset in Data Studio by joining the assessment summary table with another table that contains InsuranceSuite data.
For best practices, see Creating a useful BIM data source below.
Publish the dataset from Data Studio to Explore.
Add the dataset to Explore and create a model.
Visualize the data and pin it to a Liveboard.
For best practices, see Creating useful Liveboard visualizations below.

Best practices

Creating a useful BIM data source

Creating a useful data source in Data Studio is the key to an effective BIM Liveboard. The Predict model scores and assessments, on their own, are not useful in Explore; You must join them with InsuranceSuite data in order to visualize their impact.

In Data Studio, you’ll find the Predict model scores in the assessment summary table. You must choose a dataset to join to the assessment summary table. The dataset must relate to the model’s content and contain data for KPIs that you want to monitor. For example, the efc_claim_code dataset is useful with claim-level models because it contains cycle time, count, and payment data. You could join the efc_claim_decode dataset to the cc_assessmentsummary table and name the resulting new dataset bim_claim_activities_model_results.

Note that there’s only one assessment summary table for each InsuranceSuite application (such as ClaimCenter), so a single table likely contains scores for multiple models. To specify a model in your Data Studio query, use the TapSubtypeID or Name column. Here are some ways to manage multiple models:

If multiple models are related, join them all into a single new dataset. For example, if three activity-level models are related to the efc_activities_decode dataset, you can join them all into a new bim_claim_activities_model_results dataset. Then, in the Explore Liveboard, create a filter so that users can choose between the three models.

Important: You must create the Liveboard filter because users will see duplicated data when multiple models are included in the source dataset.
If models are unrelated, create a separate data source for a separate BIM Liveboard. For example, if you also have an exposure-level model, join the assessment summary table to efc_exposure_decode to create bim_exposure_model_results.

Creating useful Liveboard visualizations

Here are some tips for visualizing BIM data in a Liveboard:

Before and after: Show the impact of a model by comparing data from before and after it was deployed. Create two Answers, one with data from before and one from after. For example, “Average claim paid before model” and “Average claim paid after model.” Use the model’s deployment date to set timeframes for the Answers. If you don’t know the date, you can find it by looking for the latest row in the dataset that doesn’t have a Predict score.
Alerts: Create a KPI chart for a metric that you want to monitor, then set up an alert. For example, say a segmentation model is supposed to reduce the average days to close a claim by segmenting claims more efficiently. You want to be alerted if the average goes over 20 days because that means the model isn’t working well. So, in the Liveboard, you create an “Average days to close” KPI and set up an alert to be notified if it goes over the threshold of 20 days.
Business-friendly assessments: Use the business-friendly assessment columns, such as recommended actions and risk level. For example, say you expect claims to close quickly if they are recommended for segmentation to a high severity team. To prove that the recommendation works as expected, create a graph showing the average days to close by recommended action.