counterweight/data-jupyter-notebooks
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
data-jupyter-notebooks/flagging_performance_monitoring.ipynb

302 lines
280 KiB
Text
Raw Normal View History

Merged PR 4998: Simple report for flagging analysis Creates a simple single-snapshot report for flagging It shows a business explainer of metrics, the confusion matrix and the main metrics. Related work items: #29284
2025-04-15 11:39:19 +00:00
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# A/B test monitoring\n",
"\n",
"## Initial setup\n",
"This first section just ensures that the connection to DWH works correctly."
]
},
{
"cell_type": "code",
Modify cmap
2025-04-16 08:54:37 +02:00
"execution_count": 1,
Merged PR 4998: Simple report for flagging analysis Creates a simple single-snapshot report for flagging It shows a business explainer of metrics, the confusion matrix and the main metrics. Related work items: #29284
2025-04-15 11:39:19 +00:00
"metadata": {},
"outputs": [],
"source": [
"import pathlib\n",
"import yaml\n",
"import pandas as pd\n",
"from sqlalchemy import create_engine\n",
"import matplotlib.pyplot as plt\n",
"import seaborn as sns\n",
"import numpy as np\n",
"from datetime import date\n",
"\n"
]
},
{
"cell_type": "code",
Modify cmap
2025-04-16 08:54:37 +02:00
"execution_count": 2,
Merged PR 4998: Simple report for flagging analysis Creates a simple single-snapshot report for flagging It shows a business explainer of metrics, the confusion matrix and the main metrics. Related work items: #29284
2025-04-15 11:39:19 +00:00
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"/home/uri/.superhog-dwh/credentials.yml\n"
]
}
],
"source": [
"CREDS_FILEPATH = pathlib.Path.home() / \".superhog-dwh\" / \"credentials.yml\"\n",
"print(CREDS_FILEPATH)"
]
},
{
"cell_type": "code",
Modify cmap
2025-04-16 08:54:37 +02:00
"execution_count": 3,
Merged PR 4998: Simple report for flagging analysis Creates a simple single-snapshot report for flagging It shows a business explainer of metrics, the confusion matrix and the main metrics. Related work items: #29284
2025-04-15 11:39:19 +00:00
"metadata": {},
"outputs": [],
"source": [
"# Prepare connection to DWH\n",
"# Function to read credentials from the YAML file\n",
"def read_credentials(yaml_path: str, env: str = \"prd\"):\n",
" with open(yaml_path, \"r\") as file:\n",
" credentials = yaml.safe_load(file)\n",
" return credentials[\"envs\"][env]\n",
"# Function to create a PostgreSQL connection string\n",
"def create_postgres_engine(creds: dict):\n",
" user = creds[\"user\"]\n",
" password = creds[\"password\"]\n",
" host = creds[\"host\"]\n",
" port = creds[\"port\"]\n",
" database = creds[\"database\"]\n",
" # Create the connection string for SQLAlchemy\n",
" connection_string = f\"postgresql://{user}:{password}@{host}:{port}/{database}\"\n",
" engine = create_engine(connection_string)\n",
" return engine\n",
"# Function to execute a query and return the result as a pandas DataFrame\n",
"def query_to_dataframe(engine, query: str):\n",
" with engine.connect() as connection:\n",
" df = pd.read_sql(query, connection)\n",
" return df\n",
"dwh_creds = read_credentials(yaml_path=CREDS_FILEPATH, env=\"prd\")\n",
"dwh_pg_engine = create_postgres_engine(creds=dwh_creds)"
]
},
{
"cell_type": "code",
Modify cmap
2025-04-16 08:54:37 +02:00
"execution_count": 4,
Merged PR 4998: Simple report for flagging analysis Creates a simple single-snapshot report for flagging It shows a business explainer of metrics, the confusion matrix and the main metrics. Related work items: #29284
2025-04-15 11:39:19 +00:00
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" ?column?\n",
"0 1\n"
]
}
],
"source": [
"# Silly query to test things out\n",
"test_df = query_to_dataframe(engine=dwh_pg_engine, query=\"SELECT 1;\")\n",
"print(test_df.head())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Data Extraction\n",
"In this section we extract the data from the Flagging Performance Analysis within DWH."
]
},
{
"cell_type": "code",
Modify cmap
2025-04-16 08:54:37 +02:00
"execution_count": 5,
Merged PR 4998: Simple report for flagging analysis Creates a simple single-snapshot report for flagging It shows a business explainer of metrics, the confusion matrix and the main metrics. Related work items: #29284
2025-04-15 11:39:19 +00:00
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" flagging_analysis_type count_total count_true_positive \\\n",
Modify cmap
2025-04-16 08:54:37 +02:00
"0 RISK_VS_CLAIM 5987 1 \n",
"1 RISK_VS_SUBMITTED_PAYOUT 5973 1 \n",
Merged PR 4998: Simple report for flagging analysis Creates a simple single-snapshot report for flagging It shows a business explainer of metrics, the confusion matrix and the main metrics. Related work items: #29284
2025-04-15 11:39:19 +00:00
"\n",
" count_true_negative count_false_positive count_false_negative \\\n",
Modify cmap
2025-04-16 08:54:37 +02:00
"0 5460 465 61 \n",
"1 5489 465 18 \n",
Merged PR 4998: Simple report for flagging analysis Creates a simple single-snapshot report for flagging It shows a business explainer of metrics, the confusion matrix and the main metrics. Related work items: #29284
2025-04-15 11:39:19 +00:00
"\n",
" true_positive_score true_negative_score false_positive_score \\\n",
Modify cmap
2025-04-16 08:54:37 +02:00
"0 0.000167 0.911976 0.077668 \n",
"1 0.000167 0.918969 0.077850 \n",
Merged PR 4998: Simple report for flagging analysis Creates a simple single-snapshot report for flagging It shows a business explainer of metrics, the confusion matrix and the main metrics. Related work items: #29284
2025-04-15 11:39:19 +00:00
"\n",
" false_negative_score recall_score precision_score \\\n",
Modify cmap
2025-04-16 08:54:37 +02:00
"0 0.010189 0.016129 0.002146 \n",
"1 0.003014 0.052632 0.002146 \n",
Merged PR 4998: Simple report for flagging analysis Creates a simple single-snapshot report for flagging It shows a business explainer of metrics, the confusion matrix and the main metrics. Related work items: #29284
2025-04-15 11:39:19 +00:00
"\n",
" false_positive_rate_score f1_score f2_score \n",
Modify cmap
2025-04-16 08:54:37 +02:00
"0 0.078481 0.003788 0.007003 \n",
"1 0.078099 0.004124 0.009225 \n"
Merged PR 4998: Simple report for flagging analysis Creates a simple single-snapshot report for flagging It shows a business explainer of metrics, the confusion matrix and the main metrics. Related work items: #29284
2025-04-15 11:39:19 +00:00
]
}
],
"source": [
"# Query to extract data\n",
"data_extraction_query = \"\"\"\n",
"select *\n",
"from intermediate.int_flagging_performance_analysis \n",
"\"\"\"\n",
"\n",
"# Retrieve Data from Query\n",
"df = query_to_dataframe(engine=dwh_pg_engine, query=data_extraction_query)\n",
"print(df.head())"
]
},
{
"cell_type": "code",
Modify cmap
2025-04-16 08:54:37 +02:00
"execution_count": 8,
Merged PR 4998: Simple report for flagging analysis Creates a simple single-snapshot report for flagging It shows a business explainer of metrics, the confusion matrix and the main metrics. Related work items: #29284
2025-04-15 11:39:19 +00:00
"metadata": {},
"outputs": [],
"source": [
"def plot_confusion_matrix_from_df(df, flagging_analysis_type):\n",
"\n",
" # Subset - just retrieve one row depending on the flagging_analysis_type\n",
" row = df[df['flagging_analysis_type'] == flagging_analysis_type].iloc[0]\n",
"\n",
" # Define custom x-axis labels and wording\n",
" if flagging_analysis_type == 'RISK_VS_CLAIM':\n",
" x_labels = ['With Submitted Claim', 'Without Submitted Claim']\n",
" outcome_label = \"submitted claim\"\n",
" elif flagging_analysis_type == 'RISK_VS_SUBMITTED_PAYOUT':\n",
" x_labels = ['With Submitted Payout', 'Without Submitted Payout']\n",
" outcome_label = \"submitted payout\"\n",
" else:\n",
" x_labels = ['Actual Positive', 'Actual Negative'] \n",
" outcome_label = \"outcome\"\n",
"\n",
" # Confusion matrix structure\n",
" cm = np.array([\n",
Modify cmap
2025-04-16 08:54:37 +02:00
" [row['count_true_positive'], row['count_false_positive']],\n",
" [row['count_false_negative'], row['count_true_negative']]\n",
Merged PR 4998: Simple report for flagging analysis Creates a simple single-snapshot report for flagging It shows a business explainer of metrics, the confusion matrix and the main metrics. Related work items: #29284
2025-04-15 11:39:19 +00:00
" ])\n",
"\n",
" # Create annotations for the confusion matrix\n",
Fix FP and FN display
2025-04-15 15:55:31 +02:00
" labels = [['True Positives', 'False Positives'], ['False Negatives', 'True Negatives']]\n",
" counts = [[f\"{v:,}\" for v in [row['count_true_positive'], row['count_false_positive']]],\n",
" [f\"{v:,}\" for v in [row['count_false_negative'], row['count_true_negative']]]]\n",
" percentages = [[f\"{round(100*v,2):,}\" for v in [row['true_positive_score'], row['false_positive_score']]],\n",
" [f\"{round(100*v,2):,}\" for v in [row['false_negative_score'], row['true_negative_score']]]]\n",
Merged PR 4998: Simple report for flagging analysis Creates a simple single-snapshot report for flagging It shows a business explainer of metrics, the confusion matrix and the main metrics. Related work items: #29284
2025-04-15 11:39:19 +00:00
" annot = [[f\"{labels[i][j]}\\n{counts[i][j]} ({percentages[i][j]}%)\" for j in range(2)] for i in range(2)]\n",
"\n",
" # Scores formatted as percentages\n",
" recall = row['recall_score'] * 100\n",
" precision = row['precision_score'] * 100\n",
" f1 = row['f1_score'] * 100\n",
" f2 = row['f2_score'] * 100\n",
"\n",
" # Set up figure and axes manually for precise control\n",
" fig = plt.figure(figsize=(9, 8))\n",
" grid = fig.add_gridspec(nrows=4, height_ratios=[2, 3, 15, 2])\n",
"\n",
" \n",
" ax_main_title = fig.add_subplot(grid[0])\n",
" ax_main_title.axis('off')\n",
" ax_main_title.set_title(f\"Flagged as Risk vs. {outcome_label.title()}\", fontsize=14, weight='bold')\n",
" \n",
" # Business explanation text\n",
" ax_text = fig.add_subplot(grid[1])\n",
" ax_text.axis('off')\n",
" business_text = (\n",
" f\"Flagging performance analysis:\\n\\n\"\n",
" f\"- Of all the bookings we flagged as at Risk, {precision:.2f}% actually turned into a {outcome_label}.\\n\"\n",
" f\"- Of all the bookings that resulted in a {outcome_label}, we correctly flagged {recall:.2f}% of them.\\n\"\n",
" f\"- The pure balance between these two is summarized by a score of {f1:.2f}%.\\n\"\n",
" f\"- If we prioritise better probability of detection of a {outcome_label}, the balanced score is {f2:.2f}%.\\n\"\n",
" )\n",
" ax_text.text(0.0, 0.0, business_text, fontsize=10.5, ha='left', va='bottom', wrap=False, linespacing=1.5)\n",
"\n",
" # Heatmap\n",
" ax_heatmap = fig.add_subplot(grid[2])\n",
" ax_heatmap.set_title(f\"Confusion Matrix – Risk vs. {outcome_label.title()}\", fontsize=12, weight='bold', ha='center', va='center', wrap=False)\n",
"\n",
" cmap = sns.light_palette(\"#318450\", as_cmap=True)\n",
"\n",
" sns.heatmap(cm, annot=annot, fmt='', cmap=cmap, cbar=False,\n",
" xticklabels=x_labels,\n",
" yticklabels=['Flagged as Risk', 'Flagged as No Risk'],\n",
" ax=ax_heatmap,\n",
" linewidths=1.0,\n",
" annot_kws={'fontsize': 10, 'linespacing': 1.2})\n",
" ax_heatmap.set_xlabel(\"Resolution Outcome (Actual)\", fontsize=11, labelpad=10)\n",
" ax_heatmap.set_ylabel(\"Booking Status (Prediction)\", fontsize=11, labelpad=10)\n",
" \n",
" # Make borders visible\n",
" for _, spine in ax_heatmap.spines.items():\n",
" spine.set_visible(True)\n",
"\n",
" # Footer with metrics and date\n",
" ax_footer = fig.add_subplot(grid[3])\n",
" ax_footer.axis('off')\n",
" metrics_text = f\"Total Booking Count: {row['count_total']} | Recall: {recall:.2f}% | Precision: {precision:.2f}% | F1 Score: {f1:.2f}% | F2 Score: {f2:.2f}%\"\n",
" date_text = f\"Generated on {date.today().strftime('%B %d, %Y')}\"\n",
" ax_footer.text(0.5, 0.7, metrics_text, ha='center', fontsize=9)\n",
" ax_footer.text(0.5, 0.1, date_text, ha='center', fontsize=8, color='gray')\n",
"\n",
" plt.tight_layout()\n",
" plt.show()"
]
},
{
"cell_type": "code",
Modify cmap
2025-04-16 08:54:37 +02:00
"execution_count": 9,
Merged PR 4998: Simple report for flagging analysis Creates a simple single-snapshot report for flagging It shows a business explainer of metrics, the confusion matrix and the main metrics. Related work items: #29284
2025-04-15 11:39:19 +00:00
"metadata": {},
"outputs": [
{
"data": {
Modify cmap
2025-04-16 08:54:37 +02:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA2cAAAMWCAYAAAB1Nf4BAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOzdd1wUx/8/8NcBBxxdepViASyIgopiQSwYETU2YgUsETXG3huKJbG3WKKxx4aKYgMrxIIGe4/RgL2BXZD6/v3B7/bLcndwoEby8f18PHjozs7Ozs7uzu7szs5JiIjAGGOMMcYYY+yL0vjSGWCMMcYYY4wxxo0zxhhjjDHGGCsTuHHGGGOMMcYYY2UAN84YY4wxxhhjrAzgxhljjDHGGGOMlQHcOGOMMcYYY4yxMoAbZ4wxxhhjjDFWBnDjjDHGGGOMMcbKAG6cMcYYY4wxxlgZwI0zxtgnk5KSAolEIvzFx8d/6Sx9cQXLY+3atV86O59VfHy8aHtTUlJKlQ4fR5/Hp9o/H+u/un//7XyvXbtWtL6PFRoaKqTl5+f38RlkjH0W3DhjjClV+EZO1V9oaOiXzir7hArfgMr/NDQ0YGRkhGrVquGHH37A7du3v3RW/2ecPn0aXbt2hZOTE3R1daGvrw8HBwfUrl0bvXv3xooVK750Fv816jaA/Pz8/pN10MmTJ/H999+jWrVqMDExgVQqhbm5ORo2bIjJkyfjzp07XzqLjLEvTOtLZ4AxxljZR0R4+/Ytrl27hmvXrmHdunVISEhArVq1hDgVKlTA7NmzhWlTU9MvkdX/lFWrVuH7778HEYnC09PT8eDBA5w9exY7duxAv379vlAOPz1TU1PRcVKhQoUvmJt/x8uXL9GrVy/s2rVLYV5aWhpOnDiBEydOICEh4bO9kfvuu+9QrVo1AICDg8NnWQdj7ONx44wxppbg4GB4e3srhMsv9ux/U/PmzdGiRQu8f/8ehw4dwsmTJwEA7969w7Rp07Bz504hroODA0aMGPGlsvqf8+LFC/z4449Cw8ze3h4dO3aEpaUl3r59i6tXr+KPP/74wrn89IyMjL6q4+T9+/do0aIFzp49K4RZW1ujXbt2KF++PN6+fYvz58/jyJEjnzUfLVu2RMuWLT/rOhhjnwAxxpgSx44dIwDC35o1a4pdJjk5WbTMsWPHhHn//PMPDR48mBo0aED29vakp6dH2traZGtrS61bt6aYmBilab5//57GjBlDDg4OpKOjQ1WqVKFly5bRP//8o3JdRESpqakUHh5OVlZWpKurS15eXrRt2zaF7UpOThYtl5ubS+vXr6fmzZuThYUFSaVSMjc3p1atWtG+ffuU5jE7O5tmzpxJFStWJG1tbXJxcaHIyEjKysoqcRkSEV24cIH69+9PderUIVtbW9LV1SUdHR0qX748de7cmY4fP640D/PnzycfHx8yNjYmTU1NMjU1pSpVqlCPHj1o8+bNaq278D6cPHmyMC8rK4vs7e2Fea6urqJliyrbd+/e0ZQpU6hmzZpkYGBAWlpaZGFhQTVq1KA+ffrQgQMHVOah4L6NiooiTU1NYV6vXr0oNzdX5fZ0795diNu4cWOF+fv37xfma2ho0L1794iI6Pnz5zR8+HCqUqUK6enpkVQqJSsrK6pduzYNHDiQEhMT1SrPouzevVu0nSkpKQpxsrOzKS4uThRW3DHs6OiodP8VXu6ff/6hJUuWUNWqVUlHR4dsbW1p6NCh9ObNG1F6a9asES336tUrGjRoEFlbW5Oenh75+fnRmTNniIjozp071KFDBzIxMSEDAwMKCAigK1euiNJTtX8L5lvZX+PGjWny5MlFxilcHh8+fKDFixdTw4YNqVy5ciSVSsna2po6duxIp06dUrpf3r9/T6NHjyZ7e3uhzlmyZEmxdY4qY8aMES3Xtm1bev/+vUK8hw8f0vLly1WWe0HHjh2jXr16Uc2aNcna2pq0tbVJJpNRhQoVKDQ0lC5fvqyQfkhIiMpzoXA9tX79eqpRowbp6upShQoVaN68eUSUfzxGRkaSk5MTaWtrk5ubG/36669qlQNjTD3cOGOMKfWpG2d79uwp9qZqypQpovSysrKoYcOGSuMGBQWpXNfLly/Jzc1NreUK3silp6dTs2bNiszjsGHDFLb7u+++Uxo3MDCwVI2zxYsXF5kHiUSikFbBGy9lf3Xr1lVr3UU1zoiIatWqJczz9fUVzSuq0eDn51dk/oKDg1XmQb5vd+3aRVKpVAgfMGAA5eXlFbk9R44cETW+Hjx4IJrfo0cPYX6LFi2IiCgjI4NcXV2LzO/o0aPVKs+i7NixQ5Tm7t271VruUzXOCh+f8r/atWtTRkaGsFzhRoKXl5fCMrq6urR7924yNTVVmGdmZkbPnj0T0vu3GmfPnj0jT09PlfE0NDRowYIForIrqs4pXF7qNM6ysrLI0NBQWMba2prevXun1n4uqnE2fPjwIstAW1ubDh06JFpG3caZsv0LgCZOnEht27ZVOu+3335Ta5sYY8Xjbo2MMbXExsYiNTVVITw4OFit7xe0tLTg6ekJb29vWFhYwMjICO/fv8fJkydx7NgxAEBkZCR69+4NOzs7AMDChQtx/PhxIQ0PDw+0bdsWly5dQkxMjMp1TZgwATdv3hSmGzRogCZNmuD48ePYs2ePyuWGDh2Kw4cPAwC0tbXx3XffoVKlSrhy5QqioqJARJg3bx68vLzQtWtXAMD27duxZcsWIY2KFSuic+fOePjwITZs2FBsuSijo6MDHx8feHp6wszMDAYGBnj9+jWOHDmCpKQkEBGGDx+O4OBgyGQyvHv3Dhs3bhSW79ChA2rVqoXXr1/j7t27SEhIKFU+Cnr//j3279+PS5cuCWGdO3dWa9kbN24I39FoaGigZ8+eqFy5MlJTU5GcnKzWNzYHDhxA586dkZ2dDQAYNmwY5s6dW+xyTZo0gZOTE1JSUpCXl4ctW7Zg+PDhAICMjAzRN0BhYWEAgGPHjuGvv/4CAOjq6grH5JMnT3D79u1PUp4A4OnpCYlEInRrbNu2LVxcXODj44NatWqhYcOGqF279icZqU+Zffv2oW3btqhRowYOHDiApKQkAEBSUhJmzZqFSZMmKV3uwoUL6Nu3LwwMDLBkyRJkZ2fjw4cPaNu2LbS0tDBgwABkZWVh1apVAPK/qfrtt98wZsyYIvMzfvx4pKSkYMaMGUJYeHi48E2ag4MDHBwcYGBggGXLluGff/4BAHh7eyM4OFhYRv6tY48ePXDx4kUAgKGhIbp27Qp7e3ucPHkSsbGxyMvLw9ChQ+Ht7Q1fX18AinVOzZo10bp1a1y9ehXR0dFql61cUlIS3r59K0wHBwdDX1+/xOkUpq+vj8aNG6N69eowNTWFTCZDWloa9u3bhxs3biArKws//vgjrl+/XuK0z507h3r16qF58+bYunWrcC5ERkYCABo3boxGjRph5cqVePLkCQBg1qxZ6NWr10dvF2MM4DdnjDGlCj9lV/VX8OlxUW/O5P766y/asmULLV68mObMmUOzZ88mPT09YZn169cLcQu+vXBycqL09HRhXuE3RfJ1ZWdnk4GBgRBev359ysnJIaL8LotNmjRR+pQ9LS2NtLS0hPDVq1eL8j1gwABhXs2aNYXwgIAAIdzY2JjS0tKEedOnTxetS903Z3KXLl2ijRs30sKFC2n27Nk0bdo0UXp//PEHERG9ePFCCDMyMqLMzExROnl5efTPP/+otc7C+1DZn1QqpeHDhyt0J1T1Ruf8+fNCmLu7u8LbrpycHFGXvsJ5GD9+POnq6oqmSyIiIkL0VkBu27ZtQni5cuXow4cPRES0c+dOITwgIEAhvQ8fPii8gSutIUOGFFnWzs7OFBUVJVrmU70569u3rzAvKyuLqlatKsyzt7cX5hV+gzNt2jRhXpcuXUTzZs+eLczz8fERwtu3by+EF1VPqFOHEBE1btxYiBMSEqIw/9KlS6J0jh49KprfqlUrYd63334rhBescypWrCgcE0REffv2VStvBRU8xgDQsmXLil1Grqg3Z0T59dmZM2do7dq1tGDBApo
Merged PR 4998: Simple report for flagging analysis Creates a simple single-snapshot report for flagging It shows a business explainer of metrics, the confusion matrix and the main metrics. Related work items: #29284
2025-04-15 11:39:19 +00:00
"text/plain": [
"<Figure size 900x800 with 4 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
Modify cmap
2025-04-16 08:54:37 +02:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA1sAAAMWCAYAAAAH+py3AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOzdd1wUx/8/8NfBFY7eO4JYAAuioKKIIhaMWIiNWAFNImpM7NHYMGg0YtdYEhN7RYOiwa4gKhp7j4oBe8NekPr+/cHv9styd3AYiZ8k7+fjwePBzc7Ozs7uzu7szs5KiIjAGGOMMcYYY+y90vvQGWCMMcYYY4yxfyNubDHGGGOMMcZYBeDGFmOMMcYYY4xVAG5sMcYYY4wxxlgF4MYWY4wxxhhjjFUAbmwxxhhjjDHGWAXgxhZjjDHGGGOMVQBubDHGGGOMMcZYBeDGFmOMMcYYY4xVAG5sMcY0yszMhEQiEf6Sk5M/dJY+uOLlsXz58g+dnQqVnJwsWt/MzMx3Sof3o4rxvrbPX/VP3b7/1Hwzxv55uLHF2H9AyQszbX+RkZEfOqvsPSp5Qan609PTg6mpKWrVqoUvvvgC6enpHzqr/xpHjx5Fjx494ObmBgMDAxgZGcHFxQX169dHv379sGTJkg+dxb+Nrg2aoKCgf0wdFBkZqfGY0tfXh5WVFZo2bYr58+cjNzf3Q2e1QnFjlTHdST90BhhjjP29iAgvX77ExYsXcfHiRaxYsQIpKSmoV6+eEKdKlSqIi4sTfltaWn6IrP6jLF26FJ9//jmISBT+5s0b3L59GydOnMDmzZvRv3//D5TD98/S0lK0n1SpUuUD5ubDKSwsxJMnT5CamorU1FSsXLkSe/fuhZmZ2YfOGmPsA+PGFmP/QeHh4fDz81MLr1Wr1gfIDfu7tGrVCq1bt8br16+xZ88eHD58GADw6tUrTJ48Gb/++qsQ18XFBSNGjPhQWf3HefLkCb788kuhoeXs7IwuXbrA1tYWL1++xIULF3Dw4MEPnMv3z9TU9D+9n6gams+ePcP69etx/fp1AMCJEycQExOD2bNnf8jsMcb+FxBj7F/vwIEDBED4W7ZsWZnzZGRkiOY5cOCAMO3PP/+kr776ipo0aULOzs5kaGhIcrmcHB0dqV27dpSYmKgxzdevX9Po0aPJxcWFFAoF1ahRgxYtWkR//vmn1mUREWVlZVF0dDTZ2dmRgYEB+fr60saNG9XWKyMjQzRfQUEBrVy5klq1akU2NjYkk8nI2tqa2rZtS7/99pvGPObl5dHUqVOpatWqJJfLyd3dnWJjYyk3N7fcZUhEdPr0aRowYAA1aNCAHB0dycDAgBQKBVWqVIm6detGqampGvMwe/Zs8vf3JzMzM9LX1ydLS0uqUaMG9e7dm9atW6fTsktuw4kTJwrTcnNzydnZWZjm4eEhmre0sn316hVNmjSJ6tatS8bGxiSVSsnGxobq1KlDn376Ke3YsUNrHopv2/j4eNLX1xem9e3blwoKCrSuT69evYS4zZo1U5uelJQkTNfT06ObN28SEdGjR49o+PDhVKNGDTI0NCSZTEZ2dnZUv359GjRoEKWlpelUnqXZunWraD0zMzPV4uTl5dGuXbtEYWXtw66urhq3X8n5/vzzT1qwYAHVrFmTFAoFOTo60tChQ+nFixei9JYtWyaa79mzZzR48GCyt7cnQ0NDCgoKomPHjhER0fXr16lz585kbm5OxsbGFBISQufPnxelp237Fs+3pr9mzZrRxIkTS41Tsjzevn1L8+fPp8DAQLKwsCCZTEb29vbUpUsXOnLkiMbt8vr1a/r666/J2dlZqHMWLFhQZp2jTUREhGi+4rKyssjU1FSY5uLiIkybPn06dezYkapVq0YWFhYklUrJzMyM6tevT5MnT6ZXr14JcZs2bSqk0b17d7U8LFiwQJhuYWFB2dnZwrQnT57QpEmTyNfXl0xNTUkmk5GjoyN9/PHHtHv3brW0im8DV1dX0bS/sm0ZY/+HG1uM/Qe878bWtm3byrxImjRpkii93NxcCgwM1Bi3ffv2Wpf19OlT8vT01Gm+4hdmb968oZYtW5aax2HDhqmt9yeffKIxbmho6Ds1tubPn19qHiQSiVpaJS/oSv41bNhQp2WX1tgiIqpXr54wLSAgQDSttEZAUFBQqfkLDw/XmgfVtt2yZQvJZDIhfODAgVRYWFjq+uzbt0+Ir6enR7dv3xZN7927tzC9devWRESUnZ1NHh4epeb366+/1qk8S7N582ZRmlu3btVpvvfV2Cq5f6r+6tevL7oYL9nY8vX1VZvHwMCAtm7dSpaWlmrTrKys6OHDh0J6f1dj6+HDh+Tj46M1np6eHs2ZM0dUdqXVOSXL6300toiI/Pz8hGkymUwIt7KyKnU9a9euTS9fviSiopsQxbfFkydPRMso3hgbOHCgEH7p0iXRDRRNf1999ZUoLW5sMVbxuBshY/9BO3fuRFZWllp4eHg4XFxcypxfKpXCx8cHfn5+sLGxgampKV6/fo3Dhw/jwIEDAIDY2Fj069cPTk5OAIC5c+ciNTVVSMPb2xsdO3bE2bNnkZiYqHVZ48aNwx9//CH8btKkCZo3b47U1FRs27ZN63xDhw7F3r17AQByuRyffPIJqlWrhvPnzyM+Ph5EhFmzZsHX1xc9evQAAGzatAnr168X0qhatSq6deuGO3fuYNWqVWWWiyYKhQL+/v7w8fGBlZUVjI2N8fz5c+zbtw/Hjx8HEWH48OEIDw+HUqnEq1evsHr1amH+zp07o169enj+/Dlu3LiBlJSUd8pHca9fv0ZSUhLOnj0rhHXr1k2neS9fviy8DK+np4c+ffqgevXqyMrKQkZGhk4vyu/YsQPdunVDXl4eAGDYsGGYOXNmmfM1b94cbm5uyMzMRGFhIdavX4/hw4cDALKzs7FlyxYhblRUFADgwIEDuHLlCgDAwMBA2Cfv37+P9PT091KeAODj4wOJRCJ0I+zYsSPc3d3h7++PevXqITAwEPXr14dEInkvyyvpt99+Q8eOHVGnTh3s2LEDx48fBwAcP34c06dPx4QJEzTOd/r0aXz22WcwNjbGggULkJeXh7dv36Jjx46QSqUYOHAgcnNzsXTpUgDA48eP8fPPP2P06NGl5mfs2LHIzMzEd999J4RFR0cL73S5uLjAxcUFxsbGWLRoEf78808AgJ+fH8LDw4V5VO8K9u7dG2fOnAEAmJiYoEePHnB2dsbhw4exc+dOFBYWYujQofDz80NAQAAA9Tqnbt26aNeuHS5cuICEhASdy1ZXjx8/xtWrV4Xf9vb2wv/Ozs5o3rw5XF1dYWFhASJCRkYGNmzYgNevX+P8+fNYuHAhRo0ahbCwMDg7O+P27dt4+/YtVq1ahS+//BIAcP/+fRw6dEhIV7Wf5+fn4+OPP8bt27cBAPr6+ujduzecnZ2xZcsWXLhwQSiTevXqoU+fPu+8nrpsW8ZYMR+2rccY+zuUvAuu7a/43d3SnmypXLlyhdavX0/z58+nGTNmUFxcHBkaGgrzrFy5Uohb/OmCm5sbvXnzRphW8m6xall5eXlkbGwshDdu3Jjy8/OJqKiLYPPmzTXeBX/8+DFJpVIh/JdffhHle+DAgcK0unXrCuEhISFCuJmZGT1+/FiYNmXKFNGydH2ypXL27FlavXo1zZ07l+Li4mjy5Mmi9A4ePEhERd2AVGGmpqaUk5MjSqewsJD+/PNPnZZZchtq+pPJZDR8+HC17nvanricOnVKCPPy8lJ7GpWfny/qQlcyD2PHjiUDAwPR7/KIiYkR5vX19RXCN27cKIRbWFjQ27dviYjo119/FcJDQkLU0nv79q3aE7J3NWTIkFLLunLlyhQfHy+a53092frss8+Eabm5uVSzZk1hmrOzszCt5JOtyZMnC9O6d+8umhYXFydM8/f3F8I7deokhJdWT+hShxARNWvWTIgTERGhNv3s2bOidPbv3y+a3rZtW2Haxx9/LIQXr3OqVq0q7BNERJ999plOeSupZF0VFxdHcXFxNHbsWKpSpUqpT5GePXtGSUlJtHjxYpo5cybFxcW
Merged PR 4998: Simple report for flagging analysis Creates a simple single-snapshot report for flagging It shows a business explainer of metrics, the confusion matrix and the main metrics. Related work items: #29284
2025-04-15 11:39:19 +00:00
"text/plain": [
"<Figure size 900x800 with 4 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Plot confusion matrix for claim scenario\n",
"plot_confusion_matrix_from_df(df, 'RISK_VS_CLAIM')\n",
"\n",
"# Plot confusion matrix for submitted payout scenario\n",
"plot_confusion_matrix_from_df(df, 'RISK_VS_SUBMITTED_PAYOUT')"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Reference in a new issue Copy permalink