{ "cells": [ { "cell_type": "markdown", "id": "760fb84a", "metadata": {}, "source": [ "# Explanations metrics\n", "\n", "In this notebook, we will explore how to calculate the stability of a local explanation for the Partial Effects explainer. We'll first start creating a classifier and fitting it with the iris data set. Then, we'll define the stability metric function and see how stable are the explanations.\n", "\n", "This example was inspired by the usage of the stability metric in _Plumb, Gregory, Maruan, Al-Shedivat, Ángel Alexander, Cabrera, Adam, Perer, Eric, Xing, and Ameet, Talwalkar. \"Regularizing Black-box Models for Improved Interpretability.\" . Advances in Neural Information Processing Systems (pp. 10526–10536). Curran Associates, Inc., 2020._" ] }, { "cell_type": "code", "execution_count": 1, "id": "dd75b3fc", "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import pandas as pd\n", "\n", "# automatically differentiable implementation of numpy\n", "import jax.numpy as jnp # v0.2.13\n", "\n", "from sklearn import datasets\n", "\n", "from sklearn.model_selection import train_test_split\n", "from IPython.display import display\n", "\n", "import matplotlib.pyplot as plt\n", "\n", "from itea.classification import ITEA_classifier\n", "from itea.regression import ITEA_regressor\n", "from itea.inspection import *\n", "\n", "import warnings\n", "warnings.filterwarnings(action='ignore', module=r'itea')" ] }, { "cell_type": "markdown", "id": "0722b572", "metadata": {}, "source": [ "Let us start by loading the data set and fitting a simple classifier. Latter, the classifier will be used to explain a single instance and the stability of the explanation will be calculated for different sizes of neighborhoods." ] }, { "cell_type": "code", "execution_count": 2, "id": "efbd78e4", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "gen | smallest fitness | mean fitness | highest fitness | remaining time\n", "----------------------------------------------------------------------------\n", " 0 | 0.560000 | 0.848200 | 0.970000 | 0min18sec \n", " 5 | 0.740000 | 0.961800 | 0.980000 | 0min12sec \n", " 10 | 0.790000 | 0.969400 | 0.990000 | 0min13sec \n", " 15 | 0.950000 | 0.982200 | 0.990000 | 0min10sec \n", " 20 | 0.890000 | 0.982200 | 0.990000 | 0min10sec \n", " 25 | 0.860000 | 0.982400 | 0.990000 | 0min11sec \n", " 30 | 0.930000 | 0.981800 | 0.990000 | 0min6sec \n", " 35 | 0.960000 | 0.982600 | 0.990000 | 0min5sec \n", " 40 | 0.940000 | 0.981600 | 0.990000 | 0min3sec \n", " 45 | 0.660000 | 0.973800 | 0.990000 | 0min1sec \n" ] } ], "source": [ "iris_data = datasets.load_iris()\n", "X, y = iris_data['data'], iris_data['target']\n", "labels = iris_data['feature_names']\n", "targets = iris_data['target_names']\n", "\n", "# changing numbers to the class names\n", "y_targets = [targets[yi] for yi in y]\n", "\n", "X_train, X_test, y_train, y_test = train_test_split(\n", " X, y_targets, test_size=0.33, random_state=42)\n", "\n", "# Creating transformation functions for ITEA using jax.numpy\n", "# (so we don't need to analytically calculate its derivatives)\n", "tfuncs = {\n", " 'id' : lambda x: x,\n", " 'sqrt.abs' : lambda x: jnp.sqrt(jnp.abs(x)), \n", " 'log' : jnp.log,\n", " 'exp' : jnp.exp\n", "}\n", "\n", "clf = ITEA_classifier(\n", " gens = 50,\n", " popsize = 50,\n", " max_terms = 2,\n", " expolim = (-1, 1),\n", " verbose = 5,\n", " tfuncs = tfuncs,\n", " labels = labels,\n", " simplify_method = 'simplify_by_var',\n", " random_state = 42,\n", ").fit(X_train, y_train)" ] }, { "cell_type": "markdown", "id": "d66374e5", "metadata": {}, "source": [ "## Local explanations with the Partial Effects\n", "\n", "To calculate the local explanations, we can use the function ``ITExpr_explainer.average_partial_effects``, passing as an argument a single sample we want to explain.\n", "\n", "We can see the explanation both visually or as an array with the importance for each feature.\n", "\n", "Let's start by creating an explainer instance.\n", "\n", "**NOTE**: In the remaining of this notebook, all local explanations will be calculated over the first element of the test set." ] }, { "cell_type": "code", "execution_count": 3, "id": "6eeb7667", "metadata": {}, "outputs": [], "source": [ "explainer = ITExpr_explainer(\n", " itexpr=clf.bestsol_,\n", " tfuncs=tfuncs\n", ").fit(X_train, y_train)" ] }, { "cell_type": "markdown", "id": "c367859c", "metadata": {}, "source": [ "The visual inspection of feature importances can be useful in some cases:\n", "* The features are ordered by the most important to the least important feature;\n", "* The proportions between feature importances allows having a more clear understanding about its magnitudes;\n", "* We can see how much each class is using each feature to make its classification.\n", "\n", "It is important to notice that the prediction of the model is based on the class with the highest probability from the decision function. The decision function is a logit function using the IT expression as its linear model." ] }, { "cell_type": "code", "execution_count": 4, "id": "cdd10d78", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdoAAAEHCAYAAAAJRIdeAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAx0ElEQVR4nO3de3wV1bn/8c8DIaICAlpqJCgoAjEhBBAV72hFRVrbAwq8vAWrHoqint/Bg9aq1PZUPd6wlYqt1rSoFItaFSttRdSKXAyQkBBAxEYgRDGChEi4JHl+f+wh5rJDdkKGAPm+X6/9Ys/MmjXPGjY8e62ZPcvcHREREQlHq+YOQERE5FCmRCsiIhIiJVoREZEQKdGKiIiESIlWREQkREq0IiIiIYpr7gDkwHPJJZf4nDlzmjsMEZGDidW1QT1aqaWoqKi5QxAROWQo0YqIiIRIiVZERCRESrQiIiIhUqIVEREJkRKtiIhIiJRoRUREQqREKyIiEiIlWhERkRCZJn6XmtrEx3vZ7t1NVt8R8a3ZZVC2s7zJ6hQRiSYhMYGN6zc2x6HrfDKUEq3UYmZ+wqTZTVbfZw8NByAlI6XJ6hQRiSY3PZdmymt6BKOIiEhzUKIVEREJkRKtiIhIiJRoRUREQqREKyIiEiIlWhERkRAp0YqIiIRIiVZERCRESrQiIiIhUqIVEREJkRKtiIhIiJRoRUREQqREKyIiEiIlWhERkRAp0YqISIuwfv16hgwZQlJSEsnJyTzxxBO1yrzwwgukpqaSmprKmWeeSXZ2duW2r7/+mpEjR9KnTx+SkpJYsGBBTMdtdKI1sz+Y2SYzy91LmavMbHnw+tDM+jX2eI1lZh3NbHyMZUsaeYy/mVnHKOsnm9nE4H26mR1XZVu+mR0TQ939zeyZxsRVo56+Zpaxr/WIiBys4uLiePTRR1m5ciULFy5k6tSp5OXlVSvTo0cP3nvvPZYvX84999zDTTfdVLnttttu45JLLmHVqlVkZ2eTlJQU03H3pUebAVxST5l/A+e5eyrwC+B3+3C8xuoIxJRoG8vdh7n71/UUSweOq6dMND8FftOI/apx9xwg0cyO39e6REQORgkJCQwYMACA9u3bk5SUREFBQbUyZ555Jp06dQLgjDPOYMOGDQAUFxfz/vvv8+Mf/xiA+Ph4OnbsGNNxG51o3f19YHM9ZT509y3B4kIgsbHH2yPoJU43s3fMbI2Z3Vhl2x1m9lHQg/55sPpB4CQzyzKzh82snZnNNbOlZpZjZpfXc7z/MbNbg/ePm9k7wfsLzez54H1l79TM7jaz1Wb2NtA7WDcSOBV4IYjj8KD6CVXi6BPl2O2BVHfPDpbbmdlzQfnlZjYiWF9iZg+Z2RIze9vMTjOzd83sUzP7QZUq3wBGN+iEi4gcgvLz81m2bBmnn356nWWeffZZLr30UgA+/fRTvvOd7zB27Fj69+/PDTfcwDfffBPTsfbnNdofA29F22BmM4MEVPN1bR11pQKXAYOBe83sODMbCpwMnAakAQPN7FzgTmCtu6e5+x3ADuBH7j4AGAI8ama2l7jfB84J3p8KtDOzNsDZwL9qtGMgkUTWH/gPYBCAu88CMoGrgjhKg12KgjieAiZGOfapQNWh+XuAre7eNxgleCdYfyTwrrsPBLYBvwQuAn4E3F9l/8wqbRERaZFKSkoYMWIEU6ZMoUOHDlHLzJs3j2effZaHHnoIgLKyMpYuXcpPfvITli1bxpFHHsmDDz4Y0/H2S6I1syFEEu2kaNvdfVSQgGq+/lRHla+5e6m7FwHziCTXocFrGbAU6EMk8dYKB/iVmS0H3ga6At/dS/hLiCTt9sBOYAGRBHgONRJtsO5Vd9/u7sXA63upF+CVKsfoHmV7AvBlleXvAVP3LFQZLdgFzAne5wDvufvu4H3VejdRx/C1md1kZplmlllPzCIiB7Ti4mLWrFnDzp07ycnJAWDJkiUALFq0iBEjRjBkyBB+8IMfsGrVKkpKSsjPz6eoqIjCwkL++c9/cv311/Ob3/yGI444gry8PI477ji6dOnC6aefzpIlSxg5ciTz5s2joqKi1nXemuLCbrCZpQLPAJe6+1d1lJlJMMxaw2N1JFuPsmzAA+7+dI26u9coexXwHWCgu+82s3ygbV3xVykzFvgQWE6kJ3wSsDKG2PZmZ/BnOdH/LkprxGZ11L/b3fesr9hTr7tXmFnVetsGddYO2v13BNfQzawhbRAROaB06NChsqfat29fAAYOHIi7M3XqVJKSknjkkUcA6NMnctWuXbt2AKxbt46f/OQnvPDCC5x55pkAnHLKKQCcdNJJrF69moEDBzJ58mQGDx5Mq1atKrfXJdREG9x48wpwjbt/XFc5dx/VwKovN7MHiAyZnk9keLgU+IWZveDuJWbWFdhNZCi1fZV9jwI2BQl0CHBCDMd7n8jQ7vVEeomPAUuqJLeq5TLM7EEi5/b7wJ7EXzOOWKwE/rvK8j+AW4DbAcysU5VebSx6UX0oWkSkxZg/fz7Tp0+nb9++pKWlAfCrX/2KdevWATBu3Djuv/9+vvrqK8aPj9xDGxcXR2ZmZKDvN7/5DVdddRW7du3ixBNP5LnnnovpuI1OtGY2g0iSO8bMNgD3ufuzZjYOwN2nAfcCRwO/DS6Dlrn7qY09ZhWLgTeB44FfuPtGYKOZJQELgmOVAFe7+1ozmx/8DOkt4CHgjWCINAtYFcPx/gXcDSxw92/MbAe1h41x96VB7zwL+KxGmQxgmpmVErm2XC93X2VmR5lZe3ffc+11atCWcuDnfDv8HIshRM6biEiLc/bZZ1O7f1TdM888wzPPRP9FZVpaWmXSbQir76AHGjObDJS4+yPNHcv+YGb/BWxz9336La2ZHQa8B5zt7mX1lPUTJs3el8NV89lDwwFIyUhpsjpFRKLJTc+tN5mGpM6bavVkqAPfU3x7LXdfHA/cWV+SFRGRphX6zVBNzd0nN3cM+5O77wCmN0E9a4A1+x6RiIg0hHq0IiIiIVKiFRERCZESrYiISIiUaEVEREKkRCsiIhIiJVoREZEQKdGKiIiESIlWREQkREq0IiIiIVKiFRERCZESrYiISIiUaEVEREKkRCsiIhKig24+Wglfm/h4L9u9u8nqOyK+NbsMynaWN1mdIiLRJCQmsHH9xuY4dJ3z0R500+RJ+PqlppKZmdncYYiIHBI0dCwiIhIiJVoREZEQKdGKiIiESIlWREQkREq0IiIiIVKiFRERCZESrYiISIiUaEVEREKkRCsiIhIiPYJRaqnrEYx6lOL+1YyPkhORhqvzEYxKtFKLmfkJk2bXWv/ZQ8MBSMlI2d8htUi56bno36fIQaPORKuhYxERkRAp0YqIiIRIiVZERCRESrQiIiIhUqIVEREJkRKtiIhIiJRoRUREQqREKyIiEiIlWhERkRAp0YqIiIRIiVZERCRESrQiIiIhUqIVEREJkRKtiIhIiJRoRQ5Cc+bMoXfv3vTs2ZMHH3yw1vaHH36YtLQ00tLSSElJoXXr1mzevJkdO3Zw2mmn0a9fP5KTk7nvvvuaIXqRlqXBidbMupnZPDNbaWYrzOy2OsodZWZvmFl2UG7svocbLjPLMLORjdhvnJldG2V9dzPLDd6nmdmwKtsmm9nEGOo2M3vHzDo0NK4odb1tZp32tR5pXuXl5dx888289dZb5OXlMWPGDPLy8qqVueOOO8jKyiIrK4sHHniA8847j86dO3PYYYfxzjvvkJ2dTVZWFnPmzGHhwoXN1BKRlqExPdoy4L/dPQk4A7jZzE6JUu5mIM/d+wHnA4+aWXyjIz2Aufs0d/9TPcXSgGH1lIlmGJDt7sWN2Lem6cD4JqhHmtHixYvp2bMnJ554IvHx8YwePZrXXnutzvIzZsxgzJgxAJgZ7dq1A2D37t3s3r0bszrnqxaRJtDgROvuhe6+NHi/DVgJdI1WFGhvkX/F7YDNRJJ0o5nZeWaWFbyWmVn7YP0dZvaRmS03s58H67qb2Soz+2OwfpaZHRFsuzcon2tmv7O9/E9jZl3MbEnwvp+ZuZkdHyyvNbMjqvZOzWxg0ItfQOTLBsEXjPuBUUHso4LqTzGzd83sUzO7tY4QrgIq/xc1s2uD9mSb2fRgXYaZPRWMNHwanKc/BKMOGVXqeh0Y06CTLgecgoICunXrVrmcmJhIQUFB1LLbt29nzpw5jBgxonJdeXk5aWlpdOnShYsuuojTTz899JhFWrJ9ukZrZt2B/sCiKJufBJKAjUAOcJu7V0SpY2aV5Fn1VWsoFpgI3OzuacA5QKmZDQVOBk4j0mscaGbnBuV7A79z91SgmG97c0+6+yB3TwEOB4bX1UZ33wS0DYZuzwEygXPM7ARgk7tvr7HLc8Ct7j64Sh27gHuBme6e5u4zg019gIuD2O8zszZRQjgL2JPok4G7gQuCkYKqw/adgAuA/wLeAB4HkoG+ZpYWxLEFOMzMjq6rvXLgc/da6+r6rvjGG29w1lln0blz58p1rVu3Jisriw0bNrB48WJyc3NDi1VE9iHRmlk74GXg9jqGNS8GsoDjiCTAJ6NdZ3T3UUHyqfmKNhQ7H3gs6P11dPcyYGjwWgYsJZK8Tg7Kr3f3+cH754Gzg/dDzGyRmeUQSU7J9TT3QyIJ71zgV8Gf5wD/qnFOjgriei9YNb2eet90953uXgRsAr4bpUznYOSAINZZQXncfXOVcm945H/gHOALd88JvtisALpXKbeJyN9JNWZ2k5llmllmPTHLfrZkyRIAsrOzKSsro6ysjPz8fPLz8ykqKmLlypW0a9eOLVu2sHbtWkpLS8nLy6OiooJp06YxZsyYyjqWLl1KRUUFeXl5HHbYYaSmpvLKK69QUFBAYWEhRUVF5OfnU1JSwqpVqygrKyM7O7taHHv+zMnJYefOnaxZs4bi4mLWrVvHpk2b2LRpE+vWraO4uJg1a9awc+dOcnJyotaxp02rVq2ipKSksk2FhYUUFBREbdPSpUur1VG1TaWlpaxdu5YtW7aoTWrTfm3T3li0b8f1CXpes4G/u/tjdZR5E3jQ3f8VLL8D3Onui2uUm0mk51nTY9GSrZn1JXLdcgLwPeBG4GN3f7pGue7Ae+5+QrB8QbDPGOAz4FR3X29mkwHcfXIwzDrb3WfVqOtaIgn8QmAwkcSbFZSdHdRRAjxD5HrqnmOmAi+6e4qZpQfHvCXYNhkocfdHguVcYLi759c49hbgaHevCL5gdHH3n9UoUxl30O7ZQW+dmm0KhsFHufsnUc75nvr8hEmza63/7KFIxz8lI6WuXaUJ5abnRu29lpWV0atXL+bOnUvXrl0ZNGgQL774IsnJ1b8vbt26lR49erB+/XqOPPJIAL788kvatGlDx44dKS0tZejQoUyaNInhw+sc1BGR2NR5CbIxdx0b8Cywsq4kG1hHJDFhZt8lkkw/rVmoIT1aMzsp6Kk9RGQItw/wd+D6oIeNmXU1sy7BLseb2Z4h3DHAB0DbYLko2CeWu4zfB64G1gS9xM1Ekv38qoXc/Wtgq5nt6TlfVWXzNqB9DMeqaTVwYvB+LnDlnqFfM+tc515RBH93xwL5jYhDDhBxcXE8+eSTXHzxxSQlJXHllVeSnJzMtGnTmDZtWmW5V199laFDh1YmWYDCwkKGDBlCamoqgwYN4qKLLlKSFQlZXCP2OQu4Bsgxs6xg3U/d/W9mNg4id+ECvwAyguFZAybtGfLcB7eb2RCgHMgD3nL3nWaWBCwIrlOVEEmK5URu1LrOzJ4G1gBPuft2M/s9kSHWfOCj+g7q7vlB3e8Hqz4AEoNrnjWNBf5gZtuJfAnYYx5wZ3DOHmhAm98kctf2J+6+wsz+F3jPzMqJDJenN6CugcDCYMhdDmLDhg1j2LDqN7GPGzeu2nJ6ejrp6enV1qWmprJs2bKwwxORKho1dHwwqDmEerAyswTgT+5+URPU9QTwurvPraecho4PAHUNHYvIAanpho5l/3L3QuD30W4ka4Tc+pKsiIg0rcYMHR8UgpuKDomul7u/1ET1/L4p6hERkdipRysiIhIiJVoREZEQKdGKiIiESIlWREQkREq0IiIiIVKiFRERCZESrYiISIiUaEVEREKkRCsiIhIiJVoREZEQKdGKiIiESIlWREQkREq0IiIiITpk56OVxmsTH+9lu3fXWn9EfGt2GZTtLG+GqFqehMQENq7f2NxhiEhs6pyP9pCdJk8ar19qKpmZmc0dhojIIUFDxyIiIiFSohUREQmREq2IiEiIlGhFRERCpEQrIiISIiVaERGRECnRioiIhEiJVkREJERKtCIiIiHSIxillroewdgqvhUVuyqaIaKWR49fFDno1PkIRiVaqcXM/IRJs2ut/+yh4aRkpDRDRC1Pbnou+rcpclCpM9Fq6FhERCRESrQiIiIhUqIVEREJkRKtiIhIiJRoRUREQqREKyIiEiIlWhERkRAp0YqIiIRIiVZERCRESrQiIiIhUqIVEREJkRKtiIhIiJRoRUREQqREKyIiEiIlWpGDzJw5c+jduzc9e/bkwQcfrLX94YcfJi0tjbS0NFJSUmjdujWbN29m/fr1DBkyhKSkJJKTk3niiSeaIXqRlqdBidbMWpvZMjOrPVlpZPsdZpYVvHLNrNzMOjdNqDHH2NHMxsdYtqQh6/eFmaWZ2bAqy5PNbGIM+5mZvWNmHZoghrfNrNO+1iPNp7y8nJtvvpm33nqLvLw8ZsyYQV5eXrUyd9xxB1lZWWRlZfHAAw9w3nnn0blzZ+Li4nj00UdZuXIlCxcuZOrUqbX2FZGm19Ae7W3Ayro2uvvD7p7m7mnAXcB77r55H+JrjI5ATIl2P0sDhtVXKIphQLa7FzdBDNM5MM+NxGjx4sX07NmTE088kfj4eEaPHs1rr71WZ/kZM2YwZswYABISEhgwYAAA7du3JykpiYKCgv0St0hLFnOiNbNE4DLgmRh3GQPMaExQNY472cymB726NWZ2Y5Vtd5jZR2a23Mx+Hqx+EDgp6FU/bGbtzGyumS01sxwzu7yBx691DDPrbmYrzez3ZrbCzP5hZocH2wYFZRcEx881s3jgfmBUENeooPpTzOxdM/vUzG6tI4SrgMr/Sc3s2qD+bDObHqzLMLOnzGxeUNd5ZvaHIMaMKnW9TuTvRQ5SBQUFdOvWrXI5MTGxzmS5fft25syZw4gRI2pty8/PZ9myZZx++umhxSoiEXENKDsF+B+gfX0FzewI4BLgljq2zwR6R9n0mLv/Kcr6VOAM4EhgmZm9CaQAJwOnAQa8bmbnAncCKUGvGjOLA37k7sVmdgyw0Mxed3ePoR1D6zjGumD9GHe/0cxeAkYAzwPPATe5+4dm9iCAu+8ys3uBU939lqDuyUAfYAiRc7razJ5y9901wjgL+M9gn2TgbuAsdy+qMSzfCbgA+AHwRrDfDcBHZpbm7lnuvsXMDjOzo939q/raLweeaB9bM4ta9o033uCss86ic+fqV29KSkoYMWIEU6ZMoUOHfb4iISL1iKlHa2bDgU3uviTGer8PzK9r2NjdR+0ZYq7xipZkAV5z91J3LwLmEUl8Q4PXMmApkaR1crTwgV+Z2XLgbaAr8N0Y27G3Y/zb3bOC90uA7mbWEWjv7h8G61+sp/433X1n0K5NdcTV2d23Be8vAGYF5alxft8IvjzkAF+4e467VwArgO5Vym0Cjqt5EDO7ycwyzSyznphlP1myZEm1P7Ozszn22GNZtWoVJSUl5Ofns3r1atq3b09BQQFbtmxh7dq1lJaWVl6/PeOMM6rVsWjRIv7jP/6DCy+8kEsvvZS1a9eyZcsWCgoKKCwspKioiPz8fEpKSli1ahVlZWVkZ2dHjScnJ4edO3eyZs0aiouLWbduHZs2bWLTpk2sW7eO4uJi1qxZw86dO8nJyamzTWVlZdXaVFRURGFhYdQ2VVRUsHTp0mp1LF26lIqKCvLy8igtLVWb1KZmadPeWAwdO8zsAeAaoAxoC3QAXnH3q+so/yrwF3ePmmga0qMNen7m7vcFy38CXgbOBT5296drlO8OzHb3lGA5HbgUuNrdd5tZPnC+u+ebWYm7t4sSX4m7tzOzR2M8xkSgHfAEkOXuJwTrU4EX3T0liKNmj7bE3R8JlnOB4e6eX+NYW4Cj3b0iGF7u4u4/q1EmI4hnVpTYKrcFy0uAUe7+Sc12V6nPT5hU+363zx4aTkpGSl27SRPKTc+N2nstKyujV69ezJ07l65duzJo0CBefPFFkpOTq5XbunUrPXr0YP369Rx55JFApDd83XXX0blzZ6ZMmbI/miHSkkQfWiLGHq273+Xuie7eHRgNvLOXJHsUcB5VritGqa+hPdrLzaytmR0NnA98BPwduN7M2gXH7WpmXYBtVB/ePopIb3y3mQ0BToilzYG6jlFXu7YA28zsjGDV6Cqba8YVq9XAicH7ucCVwXnAGnhHt0XGGI8F8hsRhxwA4uLiePLJJ7n44otJSkriyiuvJDk5mWnTpjFt2rTKcq+++ipDhw6tTLIA8+fPZ/r06bzzzjuVP//529/+1hzNEGlRGnKNNiozGwfg7nv+lf8I+Ie7f7OvdVexGHgTOB74hbtvBDaaWRKwILhGVUKk17rWzOYHPcS3gIeAN4Ih0SxgVawHdfd/RDsGUL6X3X4M/N7MvgHeBbYG6+cBd5pZFvBArDEQaff5wCfuvsLM/hd4z8zKiQxppzegroHAQncva8A+coAZNmwYw4ZVv4F93Lhx1ZbT09NJT0+vtu7ss8+O2ksWkXDFNHTcnGoOsR7ozKydu5cE7+8EEtz9tn2oLwH4k7tf1ASxPQG87u5z6ymnoeNmVtfQsYgcsOocOt7nHq3UcpmZ3UXk3H5Gw3qctbh7YfAzog5N8Fva3PqSrIiINK0DPtG6++TmjqEh3H0mMLOJ63ypier5fVPUIyIisdOzjkVEREKkRCsiIhIiJVoREZEQKdGKiIiESIlWREQkREq0IiIiIVKiFRERCZESrYiISIiUaEVEREKkRCsiIhIiJVoREZEQKdGKiIiESIlWREQkRAf8fLSy/7WJj/ey3btrrW8V34qKXRXNEFHLk5CYwMb1G5s7DBGJneajldj1S00lMzOzucMQETkkaOhYREQkREq0IiIiIVKiFRERCZESrYiISIiUaEVEREKkRCsiIhIiJVoREZEQ6Xe0IiKHuN27d7NhwwZ27NjR3KEc9Nq2bUtiYiJt2rSJeR8lWhGRQ9yGDRto37493bt3x6zOBxhJPdydr776ig0bNtCjR4+Y99PQsdSSlbWMNm3jMLMGv47rdlxzhy8iNezYsYOjjz5aSXYfmRlHH310g0cG1KOVWsrLK6AcUjJSGrxvbnpuCBGJyL5Skm0ajTmP6tGKiIiESIlWRKSFSUg8vlGXhup6JSQe32SxZWRksHHjoTVzlYaORURamM8L1nPCpNlNVt9nDw1vsroyMjJISUnhuOMOnfs91KMVEZFQffPNN1x22WX069ePlJQUZs6cyZIlSzjvvPMYOHAgF198MYWFhcyaNYvMzEyuuuoq0tLSKC0tZe7cufTv35++ffty/fXXs3PnTgDuvPNOTjnlFFJTU5k4cSIAb7zxBqeffjr9+/fne9/7Hl988UVzNruSEq2IiIRqzpw5HHfccWRnZ5Obm8sll1zChAkTmDVrFkuWLOH666/n7rvvZuTIkZx66qm88MILZGVlYWakp6czc+ZMcnJyKCsr46mnnmLz5s28+uqrrFixguXLl/Ozn/0MgLPPPpuFCxeybNkyRo8ezf/93/81c8sjNHQsIiKh6tu3LxMnTmTSpEkMHz6cTp06kZuby0UXXQRAeXk5CQkJtfZbvXo1PXr0oFevXgBcd911TJ06lVtuuYW2bdtyww03cNlllzF8eGToesOGDYwaNYrCwkJ27drVoN+6hkk9WhERCVWvXr1YsmQJffv25a677uLll18mOTmZrKwssrKyyMnJ4R//+Eet/dw9an1xcXEsXryYESNG8Ne//pVLLrkEgAkTJnDLLbeQk5PD008/fcA8CUuJVkREQrVx40aOOOIIrr76aiZOnMiiRYv48ssvWbBgARB5ROSKFSsAaN++Pdu2bQOgT58+5Ofn88knnwAwffp0zjvvPEpKSti6dSvDhg1jypQpZGVlAbB161a6du0KwB//+Mf93Mq6aehYRKSFObZrtya9U/jYrt32uj0nJ4c77riDVq1a0aZNG5566ini4uK49dZb2bp1K2VlZdx+++0kJyeTnp7OuHHjOPzww1mwYAHPPfccV1xxBWVlZQwaNIhx48axefNmLr/8cnbs2IG78/jjjwMwefJkrrjiCrp27coZZ5zBv//97yZr476wurrm0nKZmUPjnwylz5TIgWXlypUkJSU1dxiHjDrOZ52PjNLQsYiISIiUaEVEREKkRCsiIhIiJVoREZEQKdHKfjNnzhx69+5Nz549efDBB2ttX7VqFYMHD+awww7jkUceqbbt+uuvp0uXLqSkNPwGLRGR5lRvojWzjmY2y8xWmdlKMxscpUwfM1tgZjvNbGI4oTYtM8sws5Gxrm+C4/20yvvuZhbTxK1mdruZXdsEx7/FzMbuaz2NVV5ezs0338xbb71FXl4eM2bMIC8vr1qZzp078+tf/7ryuaVVpaenM2fOnP0VrohIk4mlR/sEMMfd+wD9gJVRymwGbgUeibJNIn5af5HqzCwOuB54sQmO/wcif0fNYvHixfTs2ZMTTzyR+Ph4Ro8ezWuvvVatTJcuXRg0aBBt2rSptf+5555L586d91e4Ioe07okJTTpNXvfE2o9PDNu9997L22+/3eD93n333cpHNu4ve31ghZl1AM4F0gHcfRewq2Y5d98EbDKzy5oqMDM7j0iSB3DgXHffZmZ3AFcChwGvuvt9ZtYdmAMsAvoDHwPXuvt2M7sX+D5wOPAh8J8e4w89zWwg8BjQDigC0t290MzeDY41BOgI/Njd/2VmRwAZQB8iX0i6AzcDI4HDzSwLWAHcDbQ2s98DZwIFwOXuXlojhAuApe5eFsTTE5gGfAcoB64AugE/B74A0oBXgBzgtqDNP3T3tcG5yDez09x9cSztb0oFBQV06/btj9oTExNZtGjR/g5DRIDPCj7H7+vQZPXZzz9vsrqqcnfcnVatavcJ77///lCOWVNZWRlxcfv2bKf6erQnAl8Cz5nZMjN7xsyObOzBzGymmWVFeUUbGp0I3OzuacA5QKmZDQVOBk4jklQGmtm5QfnewO/cPRUoBsYH659090HunkIk8cT0VcbM2gC/AUa6+0AiPcL/rVIkzt1PA24H7gvWjQe2BDH8AhgI4O53AqXunubuVwVlTwamunsy8DUwIkoYZwFLqiy/EOzTj0iCLgzW9yOSWPsC1wC9gtieASZU2T+TyLnc76J9tzGr8/fdInIImTRpEr/97W8rlydPnsyjjz7Kww8/zKBBg0hNTeW++yL/jebn55OUlMT48eMZMGAA69evJz09nZSUFPr27Vv5FKj09HRmzZoFwEcffcSZZ55Jv379OO2009i2bRs7duxg7Nix9O3bl/79+zNv3rxacW3evJkf/vCHpKamcsYZZ7B8+fLK+G666SaGDh3Ktdfu85W7ehNtHDAAeMrd+wPfAHc29mDuPipINjVff4pSfD7wmJndCnQMenVDg9cyYCmRnuPJQfn17j4/eP88cHbwfoiZLTKzHCI9xOQYw+0NpAD/DHqiPwMSq2x/JfhzCZGeK8Ex/xy0NRdYvpf6/+3uWVHqqCqByBcdzKw90NXdXw3q3+Hu24NyH7l7obvvBNYCe57OnVOj3k1A1NmUzewmM8s0s8y9xByT0tJS1q5dy5YtWygoKKCwsJB27drx8ccfU1JSwqpVq1i3bl1lol2yZEm1P7/44gvKyspYs2YNxcXFrFu3jk2bNlFUVMTu3bspLi5mzZo17Ny5k5ycnKh1ZGdnU1ZWxqpVqygpKSE/P5+ioiIKCwspKChgy5YtrF27ltLSUvLy8qioqGDp0qXV6li6dCkVFRXk5eVFbVNRURH5+fmVbSorKyM7OztqPDk5OezcubNWmzZt2sS6devUJrUp1Da5Ozt27KC8vLxyPtemtmPHDioqKigtLcXd+eabb4DIXLSjR4/mxRdfxN0pLS3lpZdeomPHjqxevZoPPviAxYsXk5mZydtvv015eTmrV6/mmmuuYf78+RQVFbFu3Tpyc3NZuHAhY8eOZfv27bg7u3btorS0lCuvvJJHH32UzMxM3nzzTeLj45kyZQruzqJFi5gxYwbXXnstO3bsoLQ0MnC4fft27r33XlJSUli2bBmTJ0/mmmuuYdeuXZSXl5OZmcnMmTN5/vnna7Vp165dtf6e9mavj2A0s2OBhe7ePVg+B7jT3aMOEZvZZKDE3aNeqzWzmUQSWE2PRUu2ZtYXGEakV/Y94EbgY3d/uka57sB77n5CsHxBsM8Y4DPgVHdfH8SHu082swxgtrvPqlFXBjAbWE2khxzt5q93gYnunmlmxwCZ7t7dzF4Dprj7vKDcUuCmoFyJu7erEu/soJdNcANZO3efXOM4TwDL3D0jGMbPc/fEGmXOD2IZHiW2mtsmAN9195/VbFONOpv8EYxlZWX06tWLuXPn0rVrVwYNGsSLL75IcnLt7z2TJ0+mXbt2tW6Kys/PZ/jw4eTmxnQfmYgEaj4y0MyaeOi4uN5HryYlJTF37ly+/PJLxo8fz+DBg5k1axYdO3YEoKSkhLvuuosLL7yQIUOGVD6neMuWLZx66qkMGzaMyy67jKFDh9KqVSvS09MZPnw4vXv3Zty4ccyfP7/a8X70ox8xYcIELrjgAgDOOeccpk6dyubNm3nkkUeYPXs2/fv35+WXX+bEE08EoFu3buTm5vL4449jZpW97Jqa9BGM7v45sN7M9iTHC4G9p+691xdzj9bMTnL3HHd/iMiQZx/g78D1ZrYnYXU1sy7BLsdXuSN6DPAB0DZYLgr2acjdxKuB7+yp08zamFl9veEPiFw/xsxOITKUu8fuYDi6IVYCPQHcvRjYYGY/DOo/LLgm3BC9gGbJUnFxcTz55JNcfPHFJCUlceWVV5KcnMy0adOYNm0aAJ9//jmJiYk89thj/PKXvyQxMZHi4mIAxowZw+DBg1m9ejWJiYk8++yzzdEMEWmkkSNHMmvWLGbOnMno0aNxd+66667KqfI++eQTfvzjHwNw5JHfXqHs1KkT2dnZnH/++UydOpUbbrihWr3uHvUyVCy34uztklbVGPZVLFd4JwAvmFk88CkwNghmHIC7Twt6vplAB6DCzG4HTgmSQ2PdbmZDiNz0kwe85e47zSwJWBCcjBLg6qDMSuA6M3saWENkuHt7cMNRDpAPfBTrwd19V/Azn1+b2VFEztUUIjcz1eW3wB/NbDmR4e3lwNZg2++A5UEv9+4Yw3gLmF5l+RrgaTO7H9hN5GaohjiLyI1TzWLYsGEMGzas2rpx48ZVvj/22GPZsGFD1H1nzJgRamwiEq7Ro0dz4403UlRUxHvvvUdOTg733HMPV111Fe3ataOgoCDqLw6KioqIj49nxIgRnHTSSaSnp1fb3qdPHzZu3MhHH33EoEGD2LZtG4cffjjnnnsuL7zwAhdccAEff/wx69ato3fv3pVT8wGVZe655x7effddjjnmGDp0aLqe/h71JtrgOuKpUdZPq/L+c6pfv9xn7j6hjvVP8O3dyEDlUGyFu4+LUv5nRK6v1lyfXkf96VXeZxG567pmmfOrvC/i2+ugO4Cr3X2HmZ0EzCUydI27TwImVakmpUodUYfa3f0zM/vKzE529zXuvobIdeaqPgXerSO2d/dsM7P+wIogXhFpwU7oemyT3il8Qtdj6y2TnJzMtm3b6Nq1KwkJCSQkJLBy5UoGD44MRLZr147nn3+e1q1bV9uvoKCAsWPHUlFRAcADDzxQbXt8fDwzZ85kwoQJlJaWcvjhh/P2228zfvx4xo0bR9++fYmLiyMjI4PDDjus2r6TJ09m7NixpKamcsQRR4Q2h+0hMU1ezWuezRhHe2Ae0IbIeP0kd39rH+vsTeS66vv7WM9FwBp3z4+hrKbJEzmEaJq8ptXQa7SHxMTvQfJo9mfzufs2ovT+97HO1USuF+9rPf9sgnBERKSB9KxjERGRECnRioiIhEiJVkREJERKtCIiIiFSohUREQmREq2ISAtzXLfjmnSavOO6RX2E+l5t3LiRkSMbPvX3sGHD+Prrr/daprFT6IXlkPgdrTQt/Y5W5NAS7VnHjfn3XZem/HffFNPSha1Jn3UsIiKyr+qaJi8lJZLsMzIyuOKKK/j+97/P0KFD2b59O1deeSWpqamMGjWK008/nczMyMRi3bt3r5y9KCkpiRtvvJHk5GSGDh1aOTNPfVPo5efnc8455zBgwAAGDBjAhx9+GGr7lWhFRCRUo0ePZubMmZXLL730EoMGDapWZsGCBfzxj3/knXfe4be//S2dOnVi+fLl3HPPPZVT/tW0Zs0abr75ZlasWEHHjh15+eWXq23ftWsXo0aN4oknniA7O5u3336bww8/nC5duvDPf/6TpUuXMnPmTG699damb3QVB3b/XEREDnr9+/dn06ZNbNy4kS+//JJOnTpx/PHHVytz0UUX0blzZwA++OADbrvtNgBSUlJITU2NWm+PHj1IS0sDYODAgeTn51fbvnr1ahISEiqT+p4JA7755htuueUWsrKyaN26NR9//HFTNTUqJVoREQndnmnyPv/8c0aPHl1re9Vp6WK93lt1koDWrVtXDh1XrSfaFHqPP/443/3ud8nOzqaiooK2bdvWKtOUNHQsIiKhGz16NH/+85+ZNWtWvXcbn3322bz00ksA5OXlkZOT06hjVp1CD2Dbtm2UlZWxdetWEhISaNWqFdOnT6e8vLxR9cdKPVoRkRYmITGB3PTcJq2vPjWnyas5zFvV+PHjue6660hNTaV///6kpqZy1FFHNTiuvU2hN2LECP7yl78wZMiQJp3kPRr9vEdqiYtr7RZnlO1s+Le8hMQENq7fGEJUItJYB9s0eeXl5ezevZu2bduydu1aLrzwQj7++GPi4+ObOzSghU6TJ00rLa1/5a30IiL72/bt2xkyZAi7d+/G3XnqqacOmCTbGEq0IiJyQGnfvv0h9WVfN0OJiLQAukzYNBpzHpVoRUQOcW3btuWrr75Sst1H7s5XX33V4J8DaehYROQQl5iYyIYNG/jyyy+bO5SDXtu2bUlMTGzQPkq0IiKHuDZt2tCjR4/mDqPF0tCxiIhIiJRoRUREQqREKyIiEiI9GUpqMbNtwOrmjuMAcQxQ1NxBHCB0LqrT+fiWzgUUufsl0TboZiiJZrW7n9rcQRwIzCxT5yJC56I6nY9v6VzsnYaORUREQqREKyIiEiIlWonmd80dwAFE5+JbOhfV6Xx8S+diL3QzlIiISIjUoxUREQmREq2IiEiIlGhbKDO7xMxWm9knZnZnlO1mZr8Oti83swHNEef+EsP5ON/MtppZVvC6tzniDJuZ/cHMNplZbh3bW9rnor7z0SI+FwBm1s3M5pnZSjNbYWa3RSnToj4fsVKibYHMrDUwFbgUOAUYY2an1Ch2KXBy8LoJeGq/BrkfxXg+AP7l7mnB6/79GuT+kwFE/dF9oMV8LgIZ7P18QMv4XACUAf/t7knAGcDNLfn/jYZQom2ZTgM+cfdP3X0X8Gfg8hplLgf+5BELgY5mlrC/A91PYjkfLYK7vw9s3kuRlvS5iOV8tBjuXujuS4P324CVQNcaxVrU5yNWSrQtU1dgfZXlDdT+BxNLmUNFrG0dbGbZZvaWmSXvn9AOOC3pcxGrFve5MLPuQH9gUY1N+nxEoUcwtkwWZV3N33nFUuZQEUtblwInuHuJmQ0D/kpkeKylaUmfi1i0uM+FmbUDXgZud/fimpuj7NKSPx+AerQt1QagW5XlRGBjI8ocKuptq7sXu3tJ8P5vQBszO2b/hXjAaEmfi3q1tM+FmbUhkmRfcPdXohTR5yMKJdqW6SPgZDPrYWbxwGjg9RplXgeuDe4iPAPY6u6F+zvQ/aTe82Fmx5qZBe9PI/Jv56v9Hmnza0mfi3q1pM9F0M5ngZXu/lgdxfT5iEJDxy2Qu5eZ2S3A34HWwB/cfYWZjQu2TwP+BgwDPgG2A2ObK96wxXg+RgI/MbMyoBQY7YfgY9XMbAZwPnCMmW0A7gPaQMv7XEBM56NFfC4CZwHXADlmlhWs+ylwPLTMz0es9AhGERGREGnoWEREJERKtCIiIiFSohUREQmREq2IiEiIlGhFRERCpEQrIvvMzN4NHsu3v4/bva6ZdfahzvPNLKMp65SWTYlWREQkREq0ItKkgt7tFDP70MxygycmYWZHBvO7fmRmy8ys1gxJZjYzeGbwnuUMMxsR9Fz/ZWZLg9eZUfZNN7MnqyzPNrPzg/dDzWxBsO9fguf1YmYPmlleMHfqI01/NkSUaEUkHEe6+5nAeOAPwbq7gXfcfRAwBHjYzI6ssd+fgVEAweMwLyTytKFNwEXuPiDY/utYAwmePfwz4HvB/pnA/zOzzsCPgGR3TwV+2aiWitRDj2AUkTDMgMh8rmbWwcw6AkOBH5jZxKBMWyKP71tZZb+3gF+b2WFEJlx/391Lzewo4EkzSwPKgV4NiOUM4BRgfvBY4nhgAVAM7ACeMbM3gdmNaahIfZRoRSQMNZ/t6kSmUBvh7qvr3Ml9h5m9C1xMpOc6I9j0X8AXQD8iI3E7ouxeRvVRurbBnwb8093H1NwhGNa+kMhEErcAF+y1VSKNoKFjEQnDnuHfs4nM4LKVyKQNE6rMdtO/jn3/TORh9OcE+wAcBRS6ewWRB9u3jrJfPpBmZq3MrBtwWrB+IXCWmfUMjnuEmfUKrtMeFUxvdzuQ1vjmitRNPVoRCcMWM/sQ6ABcH6z7BTAFWB4k23xgeJR9/wH8CXjd3XcF634LvGxmVwDzgG+i7Dcf+DeQA+QSmZQdd//SzNKBGcGQNESu2W4DXjOztkR6vf/V2MaK7I1m7xGRfRYM96a7e37wfqK7ZzZvVI0T3Kmc7u7pzRuJHCo0dCwiIhIiDR2LSFPIAL4GcPfzmzOQJpAP/LWZY5BDiIaORUREQqShYxERkRAp0YqIiIRIiVZERCRESrQiIiIhUqIVEREJ0f8HWlNRSMHglBEAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "explainer.plot_feature_importances(\n", " X = X_test[0, :].reshape(1, -1),\n", " importance_method = 'pe',\n", " grouping_threshold = 0.0,\n", " target = None,\n", " barh_kw = {'edgecolor' : 'k'},\n", " show = True\n", ")" ] }, { "cell_type": "markdown", "id": "950354b9", "metadata": {}, "source": [ "We can use the ``average_partial_effects`` directly to obtain an array with numeric explanations, making it easy to manipulate the values to assess stabilities." ] }, { "cell_type": "code", "execution_count": 5, "id": "3c3ea1c1", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[6.85112219e-03, 5.00492617e-02, 4.76003440e-02, 1.51608147e-01],\n", " [2.74413002e-04, 3.40860669e-03, 1.31835324e-03, 6.55848291e-03],\n", " [1.03778588e-01, 6.74201575e-01, 6.71034697e-01, 2.10067814e+00]])" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "explainer.average_partial_effects(X_test[0, :].reshape(1, -1))" ] }, { "cell_type": "markdown", "id": "5b951658", "metadata": {}, "source": [ "The function returns an array, where each line corresponds to the importance for each class. As we can see from the bar plot, the total importance for a given feature is the summation of the importance for each class:" ] }, { "cell_type": "code", "execution_count": 6, "id": "54d685d3", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([0.11090412, 0.72765944, 0.71995339, 2.25884477])" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.sum(\n", " explainer.average_partial_effects(X_test[0, :].reshape(1, -1)),\n", " axis=0\n", ")" ] }, { "cell_type": "markdown", "id": "95424fe5", "metadata": {}, "source": [ "## Stability of an explanation\n", "\n", "In the mentioned paper, the authors discuss metrics to evaluate model agnostic explainers.\n", "\n", "The stability measures how much the explanation changes when the feature slightly changes, where lower values are better. Higher values imply that, when the feature being explained changes in a small proportion, the feature importances responds in a much larger proportion, then the explainer is not reliable since it is not locally stable, and cannot be trusted for the given instance.\n", "\n", "Let $f:\\mathbf{x} \\rightarrow y$, with $\\mathbf{x} = \\{x_0, x_1, ..., x_n\\}$ be a model we want to explain, and $g: (f, \\mathbf{x}_i) \\rightarrow \\mathbf{e}$ a explainer that takes a model and a instance to explain and attributes one feature importance for each $x \\in \\mathbf{x}_i$. \n", "\n", "The stability of the explanation is calculated by:\n", "\n", "$$S(f, g, \\mathbf{x}_i, \\mathbf{N}_{\\mathbf{x}_i}) = \\mathbb{E}_{\\mathbf{x}_j \\sim \\mathbf{N}_{\\mathbf{x}_i}} \\left [ || g(\\mathbf{x}_i, f) - g(\\mathbf{x}_j, f)||_{2}^{2} \\right ].$$\n", "\n", "In other words, the stability function evaluates the mean distance between the explanation for the original input and all sampled neighbors." ] }, { "cell_type": "code", "execution_count": 7, "id": "cb609adc", "metadata": {}, "outputs": [], "source": [ "def explain_single(x):\n", " \"\"\"wrapping the process to explain a single instance.\n", " \n", " The input x should be a single observation of shape (n_features, ).\n", " \"\"\"\n", " \n", " return np.sum(\n", " explainer.average_partial_effects(x.reshape(1, -1)),\n", " axis=0\n", " )\n", "\n", "\n", "def norm_p2(vector):\n", " \"\"\"p2 norm of a vector.\n", " \n", " the vector should be an array of shape (n, ).\n", " \"\"\"\n", " \n", " return np.sqrt(np.sum(np.abs(np.power(vector, 2))))\n", "\n", "\n", "def stability(explainer, x, neighborhood):\n", " \"\"\"Stability function.\n", " \n", " Takes as argument an explanation method, a single observation\n", " x of shape (n_features, ), and the neighborhood as a matrix of\n", " shape (n_neighbors, n_features), where each line is a sampled\n", " neighbor and each column is the feature value of the sample.\n", " \n", " Returns the mean squared p2-norm of the difference between the\n", " original explanation and every sampled neighbor.\n", " \"\"\"\n", " \n", " original_explanation = explainer(x)\n", " return np.nanmean([\n", " norm_p2(explainer(nb) - original_explanation)**2\n", " for nb in neighborhood\n", " ])\n", " \n", " \n", "def neighborhood(x, X_train, factor, size=100):\n", " \"\"\"Method to create samples around a given observation x.\n", " \n", " This method uses a multivariate normal distribution to \n", " randomly select feature values. The sigma of the distribution\n", " is calculated over the training data to mimic the original\n", " distributions and a scaling factor is multiplied to \n", " adjust how large will be the neighborhood.\n", " \n", " It is possible to specify the number of generated samples\n", " by setting the size to a different value (default=100).\n", " \n", " Returns a matrix of shape (size, n_features) containing\n", " the sampled neighbors.\n", " \"\"\"\n", " \n", " cov = np.cov(X_train.T)\n", "\n", " return np.random.multivariate_normal(x, cov*factor, size=size)" ] }, { "cell_type": "code", "execution_count": 8, "id": "cf2fc761", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.010499048473435793" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "stability(\n", " explain_single,\n", " X_test[0],\n", " neighborhood(X_test[0], X_train, 0.001)\n", ")" ] }, { "cell_type": "markdown", "id": "e8469106", "metadata": {}, "source": [ "## Increasing the neighborhood to measure stability\n", "\n", "In the original work, the authors used fixed values for creating the neighborhood to evaluate stability.\n", "\n", "I think this can be limiting in some situations. To verify, we will evaluate how stability changes when we increase the size of the neighborhood." ] }, { "cell_type": "code", "execution_count": 9, "id": "83eb27cb", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAegAAAEWCAYAAACtyARlAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAABSj0lEQVR4nO3ddXhc17Xw4d8ascUWo2WZmWQnTuwwQ9M0DZVTyE1TbnpLt19vmZlum7Zp2qTBBpqkYXDAiZlZMoqZeWb298c5I4+kGWkka4TrfZ55LB3YZ58zY63ZLMYYlFJKKTW+OMY6A0oppZTqTwO0UkopNQ5pgFZKKaXGIQ3QSiml1DikAVoppZQahzRAK6WUUuOQBmgVVCKSJyJGREL97P+6iPzF17Ei8pyIfHiU8vk9EakRkYrRuJ59zY+IyFujdb0+1/6jiPy/IKW9UES2BSPtAa55gYiUeP1+QkQuGaG014vI4WGcd6+IfG8k8jBSRORxEblirPOhAqMBWvUjIutE5G0RaRSROhHZKCKr7X0jGlSMMT8wxnzcz74rjTF/D8Z1vYlIDnAXsNAYkx6Ma4wlX8/OGHOHMea7Qbrkd4GfBSntUWeMedMYM280r+kruI/Ql44fAd8/wzTUKNEArXoRkTjgGeC3wHQgC/g20DmW+QqyGUCtMaZqrDMy0YlIBnAh8OQYZ2VY/NX0THRicRhjtgBxIlIw1nlSg9MArfqaC2CMedAY4zLGtBtjXjTG7BGRBcAfgbUi0iIiDQAicrWI7BSRJhEpFpFv+Uj3oyJSJiLlInKXZ6OIfEtE7veVERHZICIf93VdEVktIpXef1BF5AYR2eUnrXgR+YeIVIvISRH5hog47BLJS0Cmnfa9fs6/RkR22dd+W0SW2ttvFpFj9hcbRORKEakQkRT7dyMin7WPqRGRn4qIz/93IvJr+/k1ich2EVnf5zk9Yt9Ds4js9/4jKyJfFZGj9r4DInK9vd3fe9arhCYinxCRIrvG5CkRyfTaZ0TkDhEpFJF6Efm9iIivewAuBXYYYzq8zv+KiJTaeTssIhd73dOjInK/vW+viMwVka+JSJX9LC7zSuc2ETloH3tMRP7LTx76PleH1/OptZ/jdHufp1nlYyJyCnjVx/l9q8993o8fySLykn3s6yIywyud+fa+Ojudm+zttwPvB75sv2dPi8h9QC7wtL3ty/axZ9ufxwYR2S0iF3ilv0FEvi8iG4E2IN/etQG4OpBnp8aYMUZf+up5AXFALfB34Eogsc/+jwBv9dl2AbAE6wvfUqASeLe9Lw8wwINAtH1cNXCJvf9bwP19jg21f98AfHyA6x4ArvT6/QngLj/39Q/g30CsfZ0jwMe88l8ywDNZCVQBZwEhwIeBE0CEvf+fwL1AElAGXON1rgFew6qNyLWv6/OegA/YaYRiVblXAJFez6kDuMrOww+BTV7n3ghk2u/BzUArkDHAs7sX+J7980VAjX2fEVi1J2/0uYdngAT7HqqBK/w8q58Cv/f6fR5QDGR6vcez+tzT5fY9/wM4DvwPEAZ8AjjuldbVwCxAgPOxgs5KX++h/f54PmOfBzYB2fb9/Ql4sM9n7h9Yn88oH/fUk/ZA9+PjvHuBZuA8+7q/9rwP9rWKgdvse19pvweL+r4/vu7J/j0L6//qVfb7fqn9e4rX/59TwCL7GmH29i8Cj4/13xp9Df7SErTqxRjTBKzD+qP1Z6DaLlGlDXDOBmPMXmOM2xizBysYn9/nsG8bY1qNMXuBvwG3jkB2/44V1LBLRJcDD/Q9SERCsILW14wxzcaYE8DPgQ8GeJ1PAH8yxmw2Vq3C37Gq/M+2938KK8htAJ42xjzT5/wfG2PqjDGngF/h596NMfcbY2qNMU5jzM+x/qh7t32+ZYx51hjjAu4Dlnmd+6gxpsx+Dx4GCoE1Ad7f+4F7jDE7jDGdwNewStx5Xsf8yBjTYN/Da8ByP2klYAUlD5d9HwtFJMwYc8IYc9Rr/5vGmBeMMU7gUSDFvlY38BCQJyIJ9j3+xxhz1FheB14E1jO4/wL+xxhTYt/ft4D3Su/q7G/Zn8/2QdIa7H76+o8x5g37uv+D9VxzgGuAE8aYv9nv9w7gMeC9AdyPxweAZ+3PhNsY8xKwDStge9xrjNlvX6Pb3taM9T6pcU4DtOrHGHPQGPMRY0w2sBirZPYrf8eLyFki8ppY1ceNwB1Acp/Dir1+PmmneabuB64VkRjgJqw/9uU+jksGwu3reuchK8DrzADusqsRG+xq4hzsezDGNGAFl8VYgb+vgO5dRO6yq3Ab7WvE0/s5evcwbwMi5XSP9w/J6Sr4Bjsvfd8DfzLxejbGmBaskpj38+l77Rg/adVj1VJ40irCKsF+C6gSkYe8q8+xals82oEa+wuI53c81xKr+WCTXSXcgBWIArnHGcATXs/mIFag9f7SWezrxL4CuJ++etK1n2sd1vOeAZzV5zP1fmAonRRnADf2SWMdkOHr+l5igYYhXEeNEQ3QakDGmENY1W2LPZt8HPYA8BSQY4yJx2rz7NtGmeP1cy5WVfCQsuIjb6XAO8D1WKXh+/ycWwN0Y/1B885DaYDXLga+b4xJ8HpNM8Y8CCAiy4GPYtUc/MbH+YPeu1jtzV/B+qKRaIxJABrp/xz7sds1/wx8Gkiyz93nde5gS9aV4fVsRCQaq6o90OfjbQ92PwYPY8wDxph19jUM8OOhJioiEVglzJ8BafY9PksAzwfr/buyz/sXaX9+erIZaF6GeD897739RXI61vMuBl7vk6cYY8wnB8hP323FwH190og2xvxokPtaAOwe8CbVuKABWvVid1y5S0Sy7d9zsKpkN9mHVALZIhLudVosUGeM6RCRNcD7fCT9/0Rkmogswmp3e3iIWfN1XbDaDr+M1bb9hK8T7RLZI8D3RSTWDmhfxCqBB+LPwB12TYGISLRYHeNiRSTSTufr9n1licidfc7/bxFJtJ/l5/B977GAE6t9N1REvonVHyAQ0Vh/iKvB6kzF6S9U4P/ZeTwA3CYiy+1A+ANgs90UMFQvASvt54KIzBORi+x0O7BKxa6BEvAjHKtquRpwisiVwGUDn9Ljj1jv/Qw7Tykict0w8jCc+7lKrGGL4VjDzzYbY4qx2vTnisgHRSTMfq0Wq1MfWO9Zfp+0+m7z1CBdLiIhIhIpVoe27EFu43zgucDuWI0lDdCqr2aszlCbRaQVKzDvw+q0BFYv1/1AhYjU2NvuBL4jIs3AN7GCYV+vA0XAK8DPjDEvDjFfvq4LVlCeATxhjGkd4PzPYHWcOga8hRWU7gnkwsaYbVjt0L/DqsItwup4BVZnrRJjzP/Z7YwfAL4nInO8kvg3sB3YBfwH+KuPy7yA9UfzCFZ1cweBV7sewKpafwfrj/gSYKPXIf6enef8V4D/h1VCLcfqiHVLINf2kValfT1PAIzAGntbg1VNnor1ZWao6TYDn8X6bNVjfQl8KsDTf20f+6L9Gd2E9RkfjqHezwPA/2JVba/Cqsb23M9lWM+5zE7rx3b6YH1GFtpV10/a234IfMPe9iU70F9nX78a6/Py3wzwd12s+QxajTXcSo1zYkzANTtKjUsichT4L2PMy2Odl75ExABz7LbLKUFEFmJ14Ftj9A/MuCIijwF/NcY8O9Z5UYPTAK0mNBG5AavkMdcY4x7r/PQ1FQO0UmpkTMpZc9TUICIbgIXAB8djcFZKqTOhJWillFJqHApqCVpETmB1OnIBTmOMzv+qlFJKBWA0qrgvNMb06znqS3JyssnLywtydpRSSqnxYfv27TXGmBRf+8ZVG3ReXh7bto3qMrJKKaXUmBGRk/72BXsctMEae7hdrBVa+hGR20Vkm4hsq66uDnJ2lFJKqYkh2AH6XGPMSqxVkT4lIuf1PcAYc7cxpsAYU5CS4rOUr5RSSk05QQ3Qxpgy+98qrBmfAl1dRymllJrSghag7fmKYz0/Y01rty9Y11NKKaUmk2B2EkvDWuLNc50HjDHPB/F6Siml1KQRtABtjDmG14LySimllAqcrmallFJKjUMaoJVSSqlBuN2GHz53kB2n6kftmhqglVJKqUFUNnfwp9ePcaCsadSuqQFaKaWUGkRxXTsAOdOnjdo1NUArpZRSgzhV1wZATmLUqF1TA7RSSik1iOK6NkQgSwO0UkopNX4U17eRHhdJRGjIqF1TA7RSSik1iJK6dnISR6/9GTRAK6WUGoc6ul243Wass9GjuL6N7OmjV70NGqCVUkqNM90uNxf8dAO/fPmI32M2HK7izcLRWaK40+mioqlDS9BKKaWmtt3FDVQ0dfDA5lN0u9z99nd0u/jCw7v4wsO7fe4faWUNHRgzukOsQAO0UkqpceaNwhoAalu7eOVgVb/9z++roL6tm5qWTp/7R9pYDLECDdBKKaXGmTcLq1maHU9aXASPbCvut/+fm0+SlzSN9LhIHtp6Kuj5KfYEaC1BK6WUmqoa27rZXdzABfNSuWFlNhsOV1HZ1NGz/3BFM1tP1PP+s2ZwU0E2rx+ppqS+Lah5Kq5vIzzEQVpcZFCv05cGaKWUUuPGxqM1uA2cNyeZGwtycBv41/aSnv0PbD5JeKiDG1Zlc9PqHAAe2VbiL7kRUVLXTlZiFCEOCep1+tIArZRSatx4s7Ca2IhQluckMDM5mjUzp/PotmKMMbR1OXl8RylXLU5nenQ42YnTWD8nhUe3FeMK4pCs4vo2ske5/Rk0QCullBonjDG8caSGc2YnERpihaebCnI4UdvGluN1PL27jOZOJ+8/e0bPObeuzqG8sYPXjwSvs1hxXduotz+DBmillFLjxLGaVkob2lk/J6Vn21VL0omJCOWRbSX8c/Mp5qbFUDAjsWf/JQvTSI6J4IHN/TuTjYSWTif1bd2jPgYaNEArpZQaJ948Yk08cp5XgJ4WHsq1yzL5965S9pQ08v6zZiByui04LMTBe1dl81qfzmQj5XQPbq3iVkopNUW9WVhDXtI0cpN6l1ZvKsjG6TZEhYVw/cqsfufdsjoHl9vwqI8hWWfq9BhoLUErpZSagrqcbt45VturettjeU4Ca/Km8/6zcomLDOu3Py85mrX5STy0tXjE5+/2lKBztQ1aKaXUVLT9ZD1tXS7Wz0nut09EeOSOtXzjmoV+z79lTQ4l9e28c6x2RPNVUt9OTEQoCdP6fzEINg3QSimlRlVzRzf/dd82fvNKIeWN7YA1vCrUIaydlTSsNC9flE5cZCgPbx28mrusoZ2fvnCIj/xtC62dzgGPLa6zhlh5t3uPltBRv6JSSqkp7cX9lbxgv3718hHOn5vCsZpWVuYmEuujCjsQkWEhvHtFFg9tLaaxrZv4PiVeYwybjtXx97dP8OKBCjw14VtO1HHhvFS/6RbXtzEjKXpYeTpTWoJWSik1qp7fX0FGfCSv//cFfOrC2Rwob+JkbRvnz+vf/jwUNxXk0OV08+Su0n77fvtqEbf+eRObjtdy+3mzePEL5xHiEHacrPebnjGG4rr2MekgBlqCVkqNE21dTm7721ZSYiNYNzuZdXOSyR6jP4wqeNq6nLxxpJpb1+QyIymauy6bx+cunsPukkYWZcadUdqLs+JZlBnHw1uL+fA5eT3bCyub+e2rhVy9JIOf37SMyLAQABZkxLJ9gABd29pFe7drTIZYgQZopdQ4caiimc3H64iJCOWZPeUA5CVN42c3LqMgb/oY506NlNcPV9PpdHPF4vSebaEhDlZ5TT5yJm5encM3/72ffaWNLM6Kx+02fP2JvURHhPLt6xb1BGeAVbmJPLq9BKfL3TNzmbexHGIFWsWtlBonPMNZHr/zHF76wnn877ULKW/s4Nm9FWOcM/+MMfz97RM9eVeDe35/BUnR4awO0peu65ZlER7q6Oks9vC2YraeqOfrVy0gOSai17ErZyTS1uXiUEWzz7R6hlglaYBWSk1hJfVWb97sxCjmpMVy27kzmZMWQ1F1y4hdo6PbRVlD+4ilV9rQzv8+tZ/fv1Y07DSMCd4iD+NNp9PFqweruHRhWtBWhoqfFsaVi9N5clcpxXVt/ODZg5ydP50bV2X3O9ZTat9xync1t/dncixogFZKjQvFdW0kx4QzLfx0y9vslBiKKn2Xbobj168UcsWv3qDL6R6R9PaXNQHw4oFKnK7hpfmLl45w/R820tzRPSJ5Cga329A9zPvz9vbRWpo7nVy+KH3wg8/AzQU5NHc4ueXuTXQ63fzg+iU+h0llJUSRFhfhtx3a12dyNGmAVkqNC9aSfr2rEmenxlDW2DHoWNVAbTpWS1OHk72ljSOSnidA17V2seVE3ZDPd7rc/HPzKXaeauALD+8a8VmwRsq3nt7Piu+8xLef3s/J2tZhp/PCvgpiIkI5Z/bwxjoH6uz8JHKmR1Ha0M5nLpxNfkqMz+NEhFUzEv0HaB+fydGkAVopNS4U17X3W9Jvdqr1h/XoCFRzdzpd7C+1AurWYQRTX/aXNpIzPYqIUAfP7xt6W/n2k/XUtXZxwbwUXj5YxS9fPjIi+fLW0unknB++wmPbS4adxisHq4gMC+H+TSe54Gcb+Pjftw3Y+9kXl9vw4oFKLpqfSkRoyOAnnAGHQ/j0hbM5b24K/3X+rAGPXZmbSEl9u8+FNnx9JkdT0AO0iISIyE4ReSbY11JKTUwut6GsoZ2cPm19ngBdVHXmAXp/WRNddjXtluMjFKDLmliVm8j5c1N4fl/FkEvAL+yvJDzUwe/et5KbCrL57atFPLu3fMj5KBygGeCVg5WUNXbw+w1Fwyqhlze2U9rQzp0XzOKtr1zEpy+czY5T9dz8p3eG9EVn64k66lq7evXeDqabV+fyj4+uITx04DDnGSHQdzy0v8/kaBqNEvTngIOjcB2l1ARV3tiO0236lVZmJEUT6pARCdA7TzUAcNH8VLadqDvj6uTalk4qmjpYlBnPlUvSqWruZGdxQ8DnG2N4YX8F62YnExMRynffvZgVuQnc9chuDpY3BZzO60equfSXb/DygUqf+5/bW4EIHKtu5a2imoDT9dh2wgpcBXmJpMVFctdl83jtSxeQnRjFJ+/fQUVjYEs8vrC/gohQB+fPPbPJSEbawow4IkId/WoETta2+vxMjqagBmgRyQauBv4SzOsopcafkvo2/rChKKBeysV1Vm/ZvuNNw0Ic5CVHj1CAricrIYqrl2TQ1OHk8Bl2PvO0Py/KjOOi+WmEhQjP7wu89HugvInShnYuX5QGQERoCH/6wCriokK5/b5ttHUF1u7+nz1lADy09VS/fa2dTl47XMUtq3NIjgnnH++cCDh/HttO1DEtPISFGacnEYmPCuPuDxXQ3uXkk//cTqfTNWAaxhhe2FfBeXNTiI4YX9NvhIc6WJadwPY+Pbl//9pRwkMcrJvdf/GO0RLsEvSvgC8Dfrv/icjtIrJNRLZVV1cHOTtKqdHyyLYSfvL8YU7UDj5GuKTenhDCx4xNs1NGZqjVzlMNrMhNYM1Mq0rzTNuhPQF6YWYc8VFhnDs7mef2VQQ8bOqF/ZU4BC5ZkNazLTUukl/fsoLiunZ+++rgQ7ecLjcvHagk1CG8driaqj7tqK8drqLT6ea65VncuiaXVw5VcSqA98PbtpP1LM9J6DeRx9y0WH5+0zJ2nmrgW0/tHzCNzcfrKGvs4Iog994erpUzEtlX2khHt/VFY+epeh7bUcJH182cnCVoEbkGqDLGbB/oOGPM3caYAmNMQUrK+Kr6UEoN3xF78ofDFYNX1xbXtyMCGfE+AnRqDCdr285oaFRlUwelDe2syE0kOzGKjPhINp9hO/T+skayEqJImBYOwJWL0ympb+8J3IN5cX8FBXnTSeozecbZ+UncsDKbP79xbMC2ZYCtJ+qpb+vmC5fOxeU2PL6z9xzUz+2tIDkmgtV503n/WTMIEeG+TScCvsfmjm4Oljf5ncntisUZfOrCWTy4pZh/bj7p8xiny813nj5AelzkqLU/D9WqGYl0uwx7Sxtxuw3ffvoAKbERfPqi2WOar2CWoM8F3iUiJ4CHgItE5P4gXk8pNY4cqfIE6MFLvyV1bWTERfrs0DM7NQaX23DiDIb3eNqfV+QmICKszpvO1uN1ZzRJyP6yJhZnna72vXRhOiEO4bkAqrlP1rZyqKKZyxam+dz/9avmEx0Ryjee3DdgHj3turedm8fqvEQe2Vbcc3x7l4tXD1VxxWJrUpD0+EguX5zOw1uLA64+33mqAbeB1Xn+p+H84qXzuGBeCt96ar/Pznf3bTrJgfImvnntwnFXve2xMjcBsHrVP7mrlF3FDXzlivnEjHF+gxagjTFfM8ZkG2PygFuAV40xHwjW9ZRSQ1Pe2D5iE3b01dHt4qRdlXq4MpASdBvZfqoSR6In985T9YSHOHoWY1g9czpVzZ09cy0PVUunk+M1rSzKjO/ZNj06nLNmTg+omvvF/VaHLn8TdiTFRPDVK+ez+Xgdj+/ovzITnO5kdt7cFKaFh3JjQQ7Hqlt7ZsV6/UgV7d0urlqc0XPOR87Jo6nDyZM7ywK6z20n63EIrMj1H6BDHMKvb1lBTuI0Pnn/9l7TnlY1dfCLF4+wfk4yV47T0jNYz3tmcjRvFlbzo+cOsSw7nvesyBrrbOk4aKWmoo5uF5f8/HW/1ZJn6lh1Ky63ITzUwWE/8xx7G2hJv/wUay3eMwvQDSzKiusZf3uW3Q493OFWnl7WfVdfunJxOseqWykcJK8v7K9gYUbcgO2bNxfksDI3gR88e5CGtq5++/eUNFLu1a579ZIMpoWH8MhWa7zzf/ZWMD06vKfNHaBgRiILM+L4+9snAqo92HaijgUZcYOWJOOjwvjzhwvocrn5xD+29Uws8/1nD9LpdPOd6xb7nMlrPFmZm8jGolqqmjv55rWLcARpKtKhGJUAbYzZYIy5ZjSupZQaXGlDO61drhHpHe1LoV29feG8FE7UtvV0vvGl0+misrnD75J+08JDyUqIGnZeu11u9pQ2sCLndClwdkoMCdPChh2g99szkXmXoMEqEYtYM2b5U93cyfZT9Vy2yHf1tofDIXzv3UtoaO/mJy8c7rf/+f0VhDqEixekAhAdEco1SzN4Zk8Zda1dvHqwkssXpffq3CUifOScPA5XNvPQ1mKqmzv9Xr/b5WbnqYaAF7WYlRLD79+3kiOVzXzh4V1sLKrh37vKuOP8fGYmRweUxljyzMt9/YqsEVtZ60xpCVqpKcizYER5gGNYh+pIZTOhDuGKxem43GbA4Fpa344xAy/pNyctJqAAXdXUQXlj78UwDpU309HtZoXdzghW8CuYMX3YPbn3lzWRFB1OWlzvDl6pcZHkJUVzYIBxzC8frMQY/9Xb3hZmxnHbOXk8sPlUrwlMjDE8v6+CtbOSejqpAdxUkENrl4uvPraH1i4XVy3pf413Lc8kOzGKrz2+l9Xff5mzfvAyH713K28V9h4jfbC8ifZu15CC1XlzU/jG1Qt58UAlH/v7VnKmR3HnhWPb0SpQly1K493LM/nalfPHOis9NEArNQWV2qv0jOTKTt6OVLaQlxzNkqx4+3f/1dzFdl4Gqu6dnRLDsZqWQScX+cIju7judxupbTldMtxZbLXJruwTaM6aOZ0TtW1UNZ/+kuJ2G0oDeCb7y5pYmBnns9o2PzmaY9X+O7S9crCS3OnTmJ8eO+h1AL50+TxW5ibwxUd2saekAYDCqhaO17RyWZ8gv2pGIvnJ0bx4oJKEaWGcnd9/zuvIsBBe+Px5PPiJs/nG1Qs4d1YyB8ub+OT923t9udnqNUHJUNx2bh43F+TQ0e3m2+/qvf7yeJYcE8GvbllBalzkWGelhwZopaag0ShBz02LIS8pmvCQgduhPZ2K/FVxg9VRrKPbPWDwNMawt6SRquZOvvLYnp421p2nGkiNjSAzvvcf3tWe8dDHrUBU09LJh/+2hXU/fpWdfpYfBKtK/khlc7/qbY/8lGiO11pt8L4cKGti1YzEgNtkI8NCuPtDBSTHRPDxv2+jvLGdF/ZZs4Nd3qcXuIhwY0EOAJctTCMsxPef+OiIUNbOSuLj6/P5xc3LefATZ+N0G77y2N6e57b9ZJ09JG1oU12KCD94zxJevet8Lpo/cDW+GpgGaKWmoBI70DW2dwc85CZQ7V0uTtW1MTctltAQB7NSYwactau4vo3wEAdpsf5LLp6e3J62bV8qmjpo6nCyOCuOlw9W8Y93rA5wO07V9wyv8rYoM46osBC2HK9ly/E6rv7Nm2w+XkdkaAj3bDzh9zqFlS043aZfBzGP/JQYupxun7UTzR3dlDV29NxPoJJjIvjrh1fT1uXiY/du45k95azMTfRZ2nvvqmzmp8dy65rcgNPPS47mq1fO540j1Ty81RqqtfVEPQXDbIsNcYjfFaRU4DRAKzUFeQePsoaRLUUfrW7BGGumKYD56bEDlqBL6trJSowasNdsIEOtPNf4xtULuXBeCt9/9iBvF9VwsrbN5zChsBAHK2ck8MTOUm798yaiwkJ44s5zeN9ZuTy3t9zvHNP7y6wOYouz/JSg7Q5RvlbgOmpXfc8ZYoAGmJcey29vXcGhiiYOVzb3TBHaV0psBM9//rwBh0b58sGzZ3B2/nS+95+DvHOslurmTr8TlKjRoQFaqSmotKGdDLvKt2+nqjPlCZRz02Lsf2Mpb+ygsb3b5/El9W1kD7JiUMK0cJJjwgMK0PPTY/npjcuIiwzj4//YBlhDaHxZm59EU4eTKxal8/Rn1rEoM54Pr83DZYzfGbf2lzURExHKDD9t5p6So692aM/MYHPSAmt/7uvC+al8612LSI4J56olGYOfMAQOh/DT9y7DbQx33GdNABloD24VHBqglZpiXG5DRWNHT+/c8hEuQR+paiYsRJiRZJUkPZ2h/HUUK64PbM3dWSkD9+Q+XNFMWlyEHcwj+MVNy2jrchHikJ7Oan19fH0+D91+Nr973wpiI8MAyE2axiUL0nhg8ymfw8P2lzWxICPWb4k/OSac2MhQjtX0z2thVQvhoY4zWsLwQ2vz2PL1S8geoNf7cOVMn8bXr1pAU4eTuMjQYZX01cjRAK3UFFPd3Em3y7AyNxERKAuwBG2MCWhyi8LKFmalxPR0UJprB2hf1dytnU7qWrsGHGLlMTvVCtD+8nCoopl56afbhc+bm8J/Xz6PmwqyiQr33ZM4MiyEs/OT+rVP33ZuHvVt3Ty1q/eMWy634WB5k98OYmB1kspPifFbgs5Pju638MRQBXMSjfeflcsVi9K5ZlnmuJisYyobnxOjKqWG5HhNK//ZU8bH1+cPOqzF0xN6ZnI0yTERAa/n+4WHd9Hc4eTPHyoY8A/3kcrmXu2fmfGRxEaE+gzQxQOsYtXXnNQYmjqcVLd0ktqnQ5nT5aaouoVzZ/ceVvSpYY7BXZufxPz0WO7ZeJwbC7IREdxuw09fOExbl4tlOf4DNMCs5GjePlrbb3thVcuQ24ZHm4jwxw+uGutsKLQErdSE9+L+Ct7127f42YtH+pX4fPEE6MyEKDLjIykLIEC/fbSGJ3eV8cqhKh7eVuz3uNZOJyX17cz1qhoVEeb66Sjmbx1oX2anWiVxX9XcJ+zVrrxL0GdCRLjt3DwOVTSz6Vgd7V0u7vznDv74+lFuXZPLtUszBzw/PyWaiqaOnikvAdq6rGej1cYqUBqglZqgXG7DT54/xO33bScvOZq8pGk8sOXUoOeV9QToSDLioygfZGIOYww/fu4QmfGRrMmbzg+fPeh3ikjPHNR9O0HNS4/lcGVzv+rp02OgA6viBt8B2ruD2Ei5bnkWidPC+O2rhdz0p3d44UAF37h6AT+4fvGgVdSejmLHa05Xcx+tGn4PbjU1aYBWagJq6ujmw/ds4Q8bjnLrmhwevWMtH1ybx67iBg4Msh5xaX07cZGhxEaGkR4fOehkJc/tq2B3SSNfuHQuP7xhCR3dbr77zAGfx3o6gs3rEyjnpcXS2N5NZVPvwF5c38a08BASp4UNdsukxUUQFxnKPnsebG+HK5pwCEMeXzyQyLAQ3ndWLm8freVodQt//mABH1+fH9AEI54FPryHWnnGcM9J0wCtAqMBWqkJ6JGtxbxVVMOPb1jCD9+zlMiwEG5YmUV4qIMHtgy8QlVZQztZdpVyZkIkLZ1Omjp8D4Hqdrn56QuHmZsWw3tWZjMrJYY7L5zFU7vLeP1Idb/jCyubiQh1kNunROwJ2H0nLPGsYhVI0BMR1s9N4bXD1f2m/Dxc2UxeUvSITyv50XNn8t5V2Tx6x1ou8bN2sy95SdGI9B5qVVjVQqjjdO92pQajAVqpcaCoqoVDFYOvm+xxpLKZ5JgIbl59eraohGnhXLMkgyd3lvVq++yrtKGdrASrk5VnGkd/Q60e2lrM8ZpWvnLFfELsjmGfvGAW+cnRfOPJvbR39R6GdNjuwR3SpxPZvDRPT+7e91hS3xZQBzGPSxekUd3cyZ4+pejDFc39Su0jISkmgp/duGzAXtu+RIaFkJ0Y1bsEXdnCzORov9NvKtWXflKUGmPGGO7853buvH9HwOccrW5lVkr/ktj7zsqlpdPJ07v9dxazArQVFDPtQO1rqFVrp5Nfv1zImrzpXDQ/tWd7RGgI379+CcV17fz8xcO92pUL7Tm4+0qMDic1NoLDFacDljGG4rq2IY3nvWBeCiEO4aUDp5dzbOtycrKuLSgB+kzkJ/cealVU1azV22pINEArNcb2lzVxpLKFYzWtPZ2mBnO0uoVZPtpbV81IZG5ajN/OYk0d3TR3OMm0A/RAJei/vnWcmpZOvnLl/H5V0GtnJXHrmlz+8tZx7np0N21dVjV5eWNHz7jnvqyOYqdL0PVt3bR2uQLqIOaRMC2c1XmJvHygqmdbYaU1tei8Yc7OFSz5KdEcr2nF7TZ0dFvzk3t6oisVCA3QSp2hTcdq+a/7tlE0wEIOA3lsRwme+PdWUc3ABwN1rV00tHX3zPnsTUR435pc9pQ0+uxM5enBnWXPZJUaG4FDfE/3+dCWU1w4L8XvesDfe/diPn/JHJ7YWcq7f7+R5/dZpdq5foLQvLRYCitb+M+ecr77zAE+fM8WgH7t1YO5dGE6hyubOVVrfZnx9OAedyXolBjau11UNHVwrLoVt9Ee3GpoNEArNUylDe186oEd3HL3Jl7YX8kTO0uHnEa3y81Tu8q4fGE66XGRvFU4eID2tGv6KkEDXL8ym8gwB//c3L8U7VkH2lOCDg1xkBYX2W/BjPLGdsoaOzhvborffIQ4hM9fMpd/fHQNNS1dfPlfe4DTi2T0NT8jjk6nm089sIP7N50kMszBpy6cxbrZyYPccW+XLLCq218+WAlYHcQiwxzjrvPVLPsL1LHq1p4e3P6ejVK+6ExiSg2RMYY/bDjKb18txBj4/CVzeH5fBTtONgw5rTcLq6lt7eKGVdnERIby8sFKXG7Tr5OVt6P2OODZfpbzi48K49qlmTy1q5T/uXoBMRGn/5t7StDZCac7ZllDrXqXoD33EsisV+vnpPDsZ9fz6Qd2UNbQ7nfhi2uWZuAQa07tBRlxhIcOr3wwIymauWkxvHywko+um8nhimbmpMYO+MzGQs+iGTUtVDd3EuIQ8pJHfv5sNXlpCVqpIXqrqIafvnCY8+ak8Mpd5/P5S+ayZuZ0dpc04HIPPle1t8d2lDI9Opzz56awfk4yDW3dPqumvR2tbiEi1NFTCvblljW5tHa5eHF/Ra/tJQ3thIc4SI6J6NmWGR/Vbyz0zlP1RIQ6WJgR2Mxc6fGRPHrHWl790gV+pwGNDAvhPSuzWZaTMOzg7HHJgjQ2H6+jsa2bQxXN47JkmhYXQXR4iFWCrmxhRtI0IkJHdhiYmtw0QCvlQ9/hQ942FtUSFiL86pblPT2QV+Ym0tbl8rtiky+N7d28dKCSa5dmEB7q4Fy7qnewduij1a3MTI4esMS4IieBlNgIXjlY1Wt7WUMHGQmRvYJoRnwkZQ3tvXpj7zhVz5Ks+CEFUhEZ8XHI/lyyMA2X2/D4zhJqWjpHdAaxkSIizEyJ5mh1C4VVzdr+rIZMA7RSffzp9aOs+t5L1LT4ns5y07FalmUnMC38dNXxitwEwApsgXpubzldTjfvWZkNQHJMBAsz4nizsP8EIN6OVVtjjQficAgXz0/l9SPVdDndPdtL69vIjO9d8s5IiKLT6aahzZqspNPpYl9pEyv9dA4bD5ZnJ5AcE8HdbxwDxl8HMY/85BiOVDZzoraNOdqDWw2RBmilvBRVtfDzF4/Q1uViw+H+gbKl08ne0kbWzuq9alLu9GkkRYez81RDwNd6fEcps1KiWZp9ehKM9XOS2X6y3u9EI51Oa7iOrzHQfV2yII2WTidbjtf1bCtr6Ojpwe2RGd97LPSBsia6XG5W5CQEfC+jzeEQLlmQ2lM1P24DdEo0lU2duNxGx0CrIdMArZTN5TZ85bE9RIWHkBwTzquHKvsds/V4HS63YW1+7wAtIqzITQi4BH2qto0tJ+p4z8rsXmOM189JodtlegVVbydr23Ab/z24vZ07O5mIUEdPb+cup5vK5o5+bdcZCb3HQu+wv2SM5xI0WF9AABKmhZEaGzHI0WMj36umYyTnCVdTgwZopWz3vXOC7Sfr+eY1C7l0YRpvHqnpVT0M8M6xWsJDHD6D14rcRI5Vt9LQ1jXotTxDst69IqvX9oK8RCJCHbzpZ7iVpwf3YFXcAFHhIaybncwrhyoxxlDZ1IExvXtww+kStKcn945T9WQlRJEWF9kvzfHk3NnJRIY5mJcWG9Bc3mPBU9MhEth7ppQ3DdBKYS17+JMXDnP+3BTeszKLC+el0tzpZNuJ3iXZd47Wsjw3wWdnKE879M7ihgGvZYzhiZ0lnJ0/vWfKTY/IsBDWzJzutx36mL184Uwfk5T4cvGCNIrr2jlS2UJJnzHQHskxEYQ6pGdd6J0n61lu38t4FhUewrfftYhPXjBrrLPil+d9yp0+bdQ60KnJQwO0mvKMMXz9ib0I8IP3LEFEOHd2MuGhDl49dLoXdGN7N/vLGvtVb3ssy07AIQzaDr31RD0natu4cVWOz/3rZidTWNVChY9lII9WtZARH0l0RGBTGFzsNalHaZ9ZxDwcDiEtLpLyhnYqGjsoa+xgZQDjn8eDm1fncsG81MEPHCPTwkPJTowal8PA1PinAVpNeU/vKefNwhq+euX8nhJtdEQoZ+cn9QrQW47X4Tb06yDmER0Ryrz0OHYO0g796LZiosNDuHJJus/96+dYs3f5Gm51NIAe3N7S4iJZmh3PKwcreyYpyYjvX3WdmRBJWWNHT95XToAS9ETxxw+s4pvXLBzrbKgJSAO0mvJeOlBJelwk7z9rRq/tF81L4VhNK8ftauVNx2qJCHX0VGX7siI3gV2nGvqtV+zR2unkP3vLuXppRq9hWt7mp8eSHBPer5rbGON3FauBXDw/jZ3FDewpaSQ5JsJnVWtGfBTlje3sOFVPeIiDhZmBTVCiBrc4K35IC4Io5aEBWk15u4sbWJGb0G8GrIvmW72EPaXod47WsmpG4oCzQa3MTaS509lrHWBvz+4tp63LxY0Fvqu3wapyXj8nhQ2Hq3tNmFLV3ElLpzOgHtzeLl6QijHw6qHKnnWg+8pIiKSisYPtJ+tZnBWnM14pNQ5ogFZTWm1LJ6fq2ljuY8xvbtI0ZqfG8NqhKhraujhY0cTZftqfPQabsOTR7SXMTI6mYJAhTLeuyaWxvZvHdpT0bOtZJGOIvYEXZcaRER+J2/Rvf/bIjI+i22XYVdwwYdqflZrsNECrKW1PiTXv9TI/k3JcPD+VzcdreeVgFWaA9mePmUnRxEeF+ewodrK2lS3H63jvquxBhwWtzktkaXY892w83lNdfrTaqmofaoAWkZ7OYn1nEfPwtEu7zfgf/6zUVBG0AC0ikSKyRUR2i8h+Efl2sK6l1HDtKm7AIbAkK97n/gvnp9LtMvzqlSNEhYWwLDthwPQcDv8Tljy23Vr3+T0rs3yc2ZuI8LF1MzlW3cqGI1YV+9GqFqLDQ0iLG/qkHBfbk3r4LUF7Db0aqI1dKTV6glmC7gQuMsYsA5YDV4jI2UG8nlJDtqu4gblpsX6HLa2akUhcZCjFde0U5CUGtHjEipxECqtaaOro7tnmdhse21HKutnJZPgpxfZ11ZIM0uMi+etbxwGrijs/JWZYk3Ksm53MHefP4orFvnuOp9sl6Iz4yIDzp5QKrqAFaGPx9JQJs19DW4tPqSAyxrC7pGHAUnFYiIPz5lrDngZrf/ZYOSMBY2D7idOl6LeP1lLa0D5g5zBf1/7wOXlsLKrlYHkTx4bRg9s7ra9eOd9v8E2KDh+0h7pSanQFtQ1aREJEZBdQBbxkjNkczOupya+hrYv1P3mVR7YVn3Fap+raaGjrHnTWrMsWWaXO9XOSA0p3WU4CoQ7htnu3ct5PXuNT/9zBT188TFxkKJctTBtSHt+3JpeosBB+91oRpQ3tQZsuUkT46Y3L+NzFc4OSvlJq6AKbjmiYjDEuYLmIJABPiMhiY8w+72NE5HbgdoDc3NxgZkdNAr986QjFde08t7ecm/yURr/11H5S4yK484LZA6a1y56Sc7B25WuXZjAnNYYFGYGNDY6LDONfnzyHt4/WsK+0kT2lDRTXtfOxdTOHPN1j/LQwbizI5h/vnAQCWyRjuN61LDNoaSulhi6oAdrDGNMgIhuAK4B9ffbdDdwNUFBQoFXgyq/DFc3cv/kUEaEOtp6ox+lyExrSuxKorcvJPzefJDU2MqAAHRUWwtxBlgEUkYCDs8fynIReQ7eaO7r9TkwymNvOncl9m05ijLV8oVJqaghmL+4Uu+SMiEQBlwCHgnU9NbkZY/jOM/uJiQjlG9cspKXTyb6ypn7HbT5WR7fLUNrQTnFd24Bp7ipuYElWfL8gHwyxkWGEOIa34tLM5Ggunp9GiEPIS9IArdRUEcy/TBnAayKyB9iK1Qb9TBCvpyaAoqoW/ueJvThd7sEP9vLigUo2FtXyxUvncoXdJvzO0dp+x71ZWIOnk/M7x/rv9+hyutlf1sSyHN/Dq8ab7717MX/+0CpdEUmpKSSYvbj3GGNWGGOWGmMWG2O+E6xrqYnjbxuP88/Npyis8j0Vpi8d3S6+/5+DzE2L4f1n5ZISG8Gc1BifAfitomrOmZXE9OhwNg0QoA9XNNPldLM8Z2JMypEeH9kz9ahSamrQmcTUqHG7DS8dqASsknSg7tl4nFN1bXzzmkU91dFn5yex7UQd3V4l8YrGDo5UtnDenBTOzp/O5mN1GOO7W8OuYmsI1EQpQSulpp6AArSIfFpEJkZRQ42Y1k4nN/3xHX7x4uERSW93SQNVzZ1A4AG6sa2b371axKUL01jnNcxp7awk2rpcPVN1wunlGdfNSWZtfhKlDe2U1Lf7THdXsbWyU1aCTsqhlBqfAi1BpwNbReQREblChjOVkZpQjDF8/Ym9bDlRx29eLeL1I9WDnwT834ajvPv3G3H5WG7xxQOVhDqE5JgIivys9tTXCwcqaOty8ekLe/fIPmvmdIBe1dhvFVaTFB3OgvS4nklFfLVTg/VlYXlO/LBm5VJKqdEQUIA2xnwDmAP8FfgIUCgiPxCRWUHMmxpDD24p5t+7yvj0hbOZmxbDlx7dTW1L56DnvXGkml3FDbxmL9Ho7cX9FZydn8TS7HiOBliCfm5vOVkJUSzN7l0VnRQTwby02J4A7XYb3iqqZd2cZBwOYXZqDEl+2qGbOro5Wt0y6PhnpZQaSwG3QRurMa/CfjmBROBfIvKTIOVNjZF9pY186+n9nDc3hS9eOpdf37KCxrZuvvb4Xr9tuh6ezl9/e/t4r+1FVS0crW7lskVpzE6N4VhNq89StrfG9m7eKqrhqiXpPku6a2clse1EPV1ON4cqmqlp6WTdbKsaXEQ4Oz+JTcdq++V5b0kjxjDoDGJKKTWWAm2D/qyIbAd+AmwElhhjPgmsAm4IYv7UKGvq6OZTD+xg+rRwfnnTMhwOa5KOL18xjxcPVPLwVv9TbNa3dlHT0klmfCQbi2o5XNHcs8/TOeySBWnMTomhy+mmpH7gccovH6ik22W4akmGz/1n50+nvdvFnpIG3iqyquDXz0k5vX9WEmWNHZzqMx7aM4PYUi1BK6XGsUBL0MnAe4wxlxtjHjXGdAMYY9zANUHLnRp1X31sDyX17fzufStIijm9rOFHz53JubOT+PbTBzhe0+rzXE+78pcun0dkmIO/bTxdin7xQAVLs+PJTIjqma5ysI5iz+0rJzM+steMXN7Ommm1M286VsubhTXMSY3pWZUJYG1+/3bqhrYu/vHOCZZmxxMfFTbg9ZVSaiwFGqBnGmNOem8QkfsAjDEHRzxXakyUNbTz7N4K7rxgFgV503vtcziEn9+4nLAQ4SfP+54QzhNwV+dN5/oV2Tyxs5S61i6qmjrYeaqhZ6GI2SmDB+jmjm7eOFLDFYsz/HbkSowOZ356LBsOV7PleF2vXt4As1JiSI4JZ9Oxup5t3/z3fmpbuvjB9UsGeRpKKTW2Ag3Qi7x/EZEQrOptNYm8bfd4vnKx7yrl9PhILlmQxvaT9T73F1a2EBUWQlZCFLedm0en082DW07x0kGretuzKlT8tDCSYyI4OkBP7lcOVtHlcnP1Ut/rF3usnZXEtpP1dDrd/VabEhHO8mqHfnp3GU/tLuNzF89hcZaOf1ZKjW8DBmgR+ZqINANLRaTJfjVjLR/571HJoRo1bx+tYbpdKvVnSXY8Vc2dVDZ19NtXWNXM7NQYHA5hblos6+ckc987J3l2bzl5SdOY47US0+zU6AFL0M/uLSctLoIVg8z05RlOFRYiPVXe3tbmJ1He2MGW43V848l9LM9J4JMX6OADpdT4N2CANsb80BgTC/zUGBNnv2KNMUnGmK+NUh7VKDDG8M7RWtbmJ+EYYFEHz3An7wlCPIqqWnoF4dvOzaOiqYONRbVctqh3T+zZqTEUVbX47BXe0ulkw5FqrlycMWBewBoPLQIrcxOJjui/WpQngH/iH9vodLr4xU3LRmVxDKWUOlODlaDn2z8+KiIr+75GIX9qlByvaaW8sYO1s/qXQr0tzIjHIbC3pKHX9uaObsobO5jttXTjBXNTmZlsrb7kaX/2mJ0SQ1OHk2ofY6tfPVRFl9Ptt/e2t4Rp4Xzu4jnc4adUPCslmuSYCJo6nPzPVQvITwneespKKTWSBlug9i7gE8DPfewzwEUjniM1Jjztz+fOTh7wuKjwEOamxbKntHcJ2lNdPSf1dPW4wyHcddlcHt5azIrc3lXV3j25U2Mje+17bm85KbERrJoR2Oyyn79krt99IsItq3Morm/jA2fPCCg9pZQaDwYM0MaYT9j/Xjg62VFj5e2jNWTER5KXNG3QY5dkxfPqoSqMMT3V1oU9Abp3CfWapZlcszSzXxqz7eOOVrdyzqzTXwraupy8driKG1flDHv95L6+dPm8EUlHKaVG04ABWkTeM9B+Y8zjI5sdNRbcbqv9+aL5aQHNTb00O55Ht5dQ1tjRs9hEUVUL4aEOcqYPHuAB0uMiiYkI7Tfl5/P7KujodnP10sGrt5VSajIbrIr72gH2GUAD9DhmjOFYTSuzBml3PVTRTH1bN+cM0v7sscSegWtvSUNPgC6sbCY/OTrgUq+IMCulf0/uh7YWk5c0rWcxDKWUmqoGq+K+bbQyokbexqJaPvDXzTz16XMHnNby7aPWMo3nzA4sQM9PjyXUIewpaeQKe8x0YVVLv3bmwcxKjeHtotOzfB2rbmHL8Tq+fMU8XWVKKTXlDVbF/QFjzP0i8kVf+40xvwhOttRI8EwE8s7R2kECdC35ydFkxAe2NnJkWAjz0mPZa3cUa+tyUlLfzk0FOUPK36yUGB7fUUpzRzexkWE8vK2YEIfw3pXZQ0pHKaUmo8EGhEbb/8b6ealxrKyhHYCtJ3zP/AXQ7XKz+VjtoMOr+lqaHc+ekkaMMRytsubm7ttBbDDeHcW6XW4e217CRfNTSY2LHORMpZSa/Aar4v6T/e+3Ryc7aiSV2gF6+8m6Xj2uve0paaS1y9WrJ3UglmQl8OCWYorr2imsslatmpM2zABd1UJFYwc1LV3csnpopXCllJqsAl1uMl9EnhaRahGpEpF/i0h+sDOnzoynBF3f1s3Rat8rUL1jtz8PpwQNsKe0gcKqFkIdwoyk6EHO6m3G9GmEhQhF1S08vPUUaXERnD83ZfATlVJqCgh0zsMHgEeADCATeBR4MFiZUr41tndzqnbgNZS9lTV0sDI3AbBK0b5sLKplQUYc06PDh5SXuWmxhIc42FvSSGFlCzOTowkb4hSaoSEO8pKieauwhtePVHNTQY5Ow6mUUrZA/xqKMeY+Y4zTft2PNcxKjaLvPnOAq37zJrU+psfsq9vlprK5g3Wzk5keHe6zHbqj28X2U/UBD6/yFh7qYEFGLHtKGimqah5y9bbHrJQY9pY24jYMuZOZUkpNZoPNxT1dRKYDr4nIV0UkT0RmiMiXgf+MThYVWJOJbDhcTUunkz9sODro8RWNHRgDWYlRrJqR6HOJyJcOVNLldHPR/NRh5WlJdjx7Sho4VdfG7NTh9Rn0tEOvm50c8CQnSik1FQxWgt4ObANuBv4LeA3YAHwS0DHSo+hgRRM1LZ2kx0Vy3zsnezqA+eNpf85MiKJgRiLHa1qpbu5d8n58RwmZ8ZGszR96CRpgaVYCrV0u3GboPbg9PCXvm7VzmFJK9TLYcpMzjTH59r99X9pJbBS9ccTqzPXHD64C4NcvHxnw+LJGrwCdZ83K5V2Krmru4I3CGt69ImvQJR39WWJ3FIOh9+D2uHxROj+/cVlAK1cppdRUEnCPHBFZLCI3iciHPK9gZkz19mZhNfPTY1mek8AH187gX9tLKLKHN/lS1tABQGZ8FIuz4ggPdbDtxOmOYk/tKsPlNrznDCYFmZMaQ2SYA4fQs6zkUEWGhXDDquwRWxhDKaUmi0CHWf0v8Fv7dSHwE+BdQcyX8tLW5WTbiXrOs4cg3XnBLKaFh/KzF/yXoksb2kmcFkZUeAgRoSEsz05gm1cJ+rEdpSzLSehpAx6O0BAHizLjyUuKJiI0ZNjpKKWU6i/QEvR7gYuBCnt+7mVARNBypXrZfKyOLpeb8+ZYATopJoJPrM/n+f0V7Cpu8HlOeUM7mQmnp+5clZfIvtJG2rtcHChr4mB5EzeszDrjvH37XYv4yXuXnnE6Simlegs0QLcbY9yAU0TigCpA26BHyetHqokMc1CQd3oxio+tn0lSdDg/feGQz3PKGjp6BeiCGYk43YbdJQ08sbOEsBDxuU7zUC3Oiu9p41ZKKTVyAg3Q20QkAfgzVs/uHcCWYGVK9fZmYTVnzUwiMux0NXJMRCgfOSePjUW1PsdFlzW09ywFCbBqhhXcNx2r5YmdZVw4L3XIk5MopZQaPQEFaGPMncaYBmPMH4FLgQ/rUpSjo7ShnaPVrT3tz95W2kH3QHlTr+1NHd00dzrJTDi96ETCtHDmpMZw79snqGnp5IZVumKUUkqNZ4F2EnvF87Mx5oQxZo/3NhU8bxypBuC8Of0Xs1iUGQfA/rLeAdp7DLS3grzpNLR1kzAtjAvnDW9yEqWUUqNjsJnEIu2ZxJJFJNEzs5iI5GHNyT3QuTki8pqIHBSR/SLyuRHM95TxZmE1GfGRPntbJ0wLJyshKvAAbZe437Usk/BQnfNaKaXGswGXm8SaPezzWMF4u9f2ZuD3g5zrBO4yxuwQkVhgu4i8ZIw5MNzMTjVOl5u3Cmu4YnG6z6UiARZmxrG/rLHXtlJ7DHRWnwB9/rwUVucl8qG1eUHJr1JKqZEzWDHqbeAc4Ev2zGHfBvYBr2OtcOWXMabcGLPD/rkZOAic+bieKWRPaSNNHU6f7c8eizLjOF7TSmuns2dbWUM7YSFCSkzvkXDJMRE8esc5ZzT2WSml1OgYrAT9J+ASY8xvReQ84IfAZ4DlwN1Y46MHZVeJrwA2+9h3O3A7QG5ubqD5npSe2l3GxsIaYiJDiYkIZX9ZIyJw7qz+7c8eizLjMQYOVTSxaoY13KmsoZ30+MhhT+GplFJq7A0WoEOMMZ75IW8G7jbGPAY8JiK7ArmAiMQAjwGfN8Y09d1vjLkbK9hTUFAwZZew3F3cwOcf2klMRChuAy12ificWUkkDjAcyrujmHeAzoyP8nuOUkqp8W/QAC0iocYYJ9ZMYrcP4VxEJAwrOP/TGPP48LM5uXU53Xz5X3tIjY3kxS+eR1xkGG63obXLSVTYwFNoZsRHkjgtjP2lp7/7lDV0sGamTh6ilFIT2WBB9kHgdRGpAdqBNwFEZDbQONCJYvVq+itw0BjzixHI66T1hw1FHK5s5p6PFBAXGQaAwyHE2j8PRERYlBnP/nLr7XC5DRVNHb3GQCullJp4Bltu8vvAXcC9wDpjjKcK2oHVFj2Qc4EPAheJyC77ddUZ5nfSOVzRzO9fK+K65ZlcND9tWGksyozjSEUL3S43Vc0duNym3xArpZRSE8ug1dTGmE0+tg28GLF1zFuA9lIagMtt+PJje4iNDON/r1007HQWZcXT5XJTWNlCe7fVdq0BWimlJrZBA7QKnr9tPM7u4gZ+c+uKM5oX+3RHsUYi7DbrvmOglVJKTSwaoMdIY3s3v3zpCBfPT+XapRlnlNbMpGimhYewv6yJ9Hir7TkjXtuglVJqItMAPUYe2HyK1i4XX7xsrt9ZwgLlcAgLMuI4UNaE2xjiIkMD6mCmlFJq/NIJmcdAl9PN3zYeZ93sZBZlxo9Imosy4zhQ3kRJfbu2Pyul1CSgATpIOp0ufvjsQQ5V9Jubhad2l1HV3Mnt5+WP2PUWZcbR0ulk24k6bX9WSqlJQAN0kDy6rYQ/vXGMj927jYa2rp7txhj+/MYx5qfHst7HEpLD5SmJN3U4tQStlFKTgAboIOh0uvj9a0XkJ0dT1dzBXY/sxu22hpC/fqSaw5XNfGJ9/hm3PXubkxZDqD33tgZopZSa+DRAB8EjW4spb+zg29ct4htXL+SVQ1X86Y1jANz9xjHS4yK5dtmAy2kPWURoCHPSYgF0FjGllJoEtBf3CLNKz0cpmJHIutnJrJudzJYTdfz0hUOEhzp4+2gtX7tyPuGhI//daFFmHAfLm7QNWimlJgEtQY+wh7cWU9HUwRcutYZPiQg/vmEpecnRfPeZA8REhHLrWcFZVnN5TgIOgdzp04KSvlJKqdGjAXoEdXRbbc+r8xI5Z1ZSz/aYiFD+7/2riA4P4cPnzOhZEGOk3bw6h6c+vY7UOK3iVkqpiU6ruEfQw1uLqWzq5Jc3Le/XAWxeeiybvn4xMRHBe+RhIQ4WZ43MuGqllFJjSwP0CGnvcvGHDUWsmTmdtV6lZ286u5dSSqlAaRX3CDDG8PUn9lLV3MmXLps3osOnlFJKTU0aoEfA3zae4ImdpXzxkrmsmTl9rLOjlFJqEtAAfYbeOVrL9589yGUL0/jUhbPHOjtKKaUmCQ3QZ6C0oZ1PPbCDvKRp/PymZTgcWrWtlFJqZGiAHqaObhd33Ledbqebuz9UoB3AlFJKjSjtxT1MT+wsZW9pI3/64CpmpcSMdXaUUkpNMlqCHqYndpaSnxLNZQvTxjorSimlJiEN0MNQUt/GluN1XL88S4dUKaWUCgoN0MPw1O4yAK5bnjXGOVFKKTVZaYAeImMMT+wopWBGIrlJuiiFUkqp4NAAPUQHypsorGrhuhVaelZKKRU8GqCH6MmdpYQ6hGuWZIx1VpRSSk1iGqCHwOU2PLW7jAvmpZIYHT7W2VFKKTWJaYAegk3Haqls6uTdKzLHOitKKaUmOQ3QQ/DEzlJiIkK5ZIGOfVZKKRVcGqAD1NHt4vl9FVy5OJ3IsJCxzo5SSqlJTgN0gF4+WElLp5Prtfe2UkqpUaABOkBP7y4jNTaCs/KTxjorSimlpgAN0AFo6ujmtcPVXL00gxBdUlIppdQoCFqAFpF7RKRKRPYF6xqj5aX9lXQ53Vy7THtvK6WUGh3BLEHfC1wRxPRHzdN7yshKiGJFTsJYZ0UppdQUEbQAbYx5A6gLVvqjpa61i7cKa7h2WaauXKWUUmrUjHkbtIjcLiLbRGRbdXX1WGenn+f3VeB0G65dplN7KqWUGj1jHqCNMXcbYwqMMQUpKSljnZ1+nt5dRn5KNAsz4sY6K0oppaaQMQ/Q41lVUwebjtdy7VKt3lZKKTW6NEAP4D97yzEGrd5WSik16oI5zOpB4B1gnoiUiMjHgnWtYHl6dxkLMuKYnRo71llRSik1xYQGK2FjzK3BSns0FNe1seNUA1++Yt5YZ0UppdQUpFXcfjy5sxSAa5bo5CRKKaVGnwZoH9q6nPzt7ROcPzeF3KRpY50dpZRSU5AGaB8e2HyKutYuPnvx7LHOilJKqSlqygboPSUNbDhc1W97R7eLu984xtr8JFbNmD4GOVNKKaWmcID+9tMH+Njft/FmYe/Zy/61vYSq5k4+c5GWnpVSSo2dKRmgO7pd7ClpwG0Md/5zB0erWwDodrn5vw1HWZmbwNpZuu6zUkqpsTMlA/TOUw10uwzfedciwkMcfPzv22hs6+bJnaWUNrTzmYvm6MxhSimlxlTQxkGPZ1tP1CEC71qexYKMOG798yY+9cAOShvaWZQZxwXzxt+c4EoppaaWKVmC3nqijnlpscRHhVGQN50fXL+Et4pqOF7Tymcumq2lZ6WUUmNuypWgnS4320/Wc+Oq7J5tNxbkUNHYwe6SRi5bmD6GuVNKKaUsUy5A7y9roq3LxeqZvYdQfebiOWOUI6WUUqq/KVfFvfVEHQBr8nSMs1JKqfFrygXoLcfrmJE0jdS4yLHOilJKKeXXlArQbrdh64k6LT0rpZQa96ZUgD5a3UJ9W3e/9mellFJqvJlSAXqLtj8rpZSaIKZUgN56vI6U2Ahm6BKSSimlxrkpFaC3HK9jzczpOhGJUkqpcW/KBOiS+jbKGju0elsppdSEMGUCtGf882oN0EoppSaAKROgtxyvJy4ylHnpsWOdFaWUUmpQUyJAG2N4+2gNq/OmE+LQ9mellFLj35QI0EerWzhZ28aF81PHOitKKaVUQKZEgH7pQBUAFy/QAK2UUmpimBIB+pWDlSzOiiMjPmqss6KUUkoFZNIH6NqWTrafqueSBWljnRWllFIqYJM+QL92uBpj0ACtlFJqQpn0AfrlA5Wkx0WyKDNurLOilFJKBWxSB+iObhdvFFZz8YJUnd5TKaXUhDKpA/SmY7W0dbm0elsppdSEM6kD9CsHq4gKC2HtrKSxzopSSik1JJM2QBtjeOVgJevnJBMZFjLW2VFKKaWGJKgBWkSuEJHDIlIkIl8N5rX6OlDeRFljB5cs1OptpZRSE0/QArSIhAC/B64EFgK3isjCYF2vr5cPVCECF+n0nkoppSagYJag1wBFxphjxpgu4CHguiBer5eXD1ayIieB5JiI0bqkUkopNWJCg5h2FlDs9XsJcFbfg0TkduB2gNzc3BG5sMttWJYTz4IMHfuslFJqYgpmgPY18Nj022DM3cDdAAUFBf32D0eIQ/jeu5eMRFJKKaXUmAhmFXcJkOP1ezZQFsTrKaWUUpNGMAP0VmCOiMwUkXDgFuCpIF5PKaWUmjSCVsVtjHGKyKeBF4AQ4B5jzP5gXU8ppZSaTILZBo0x5lng2WBeQymllJqMJu1MYkoppdREpgFaKaWUGoc0QCullFLjkAZopZRSahwSY0ZkbpARISLVwMkRSi4ZqBmhtKYyfY5nTp/hmdNneOb0GY6MkX6OM4wxKb52jKsAPZJEZJsxpmCs8zHR6XM8c/oMz5w+wzOnz3BkjOZz1CpupZRSahzSAK2UUkqNQ5M5QN891hmYJPQ5njl9hmdOn+GZ02c4MkbtOU7aNmillFJqIpvMJWillFJqwtIArZRSSo1DEz5Ai8gVInJYRIpE5Ks+9ouI/Mbev0dEVo5FPsezAJ7h++1nt0dE3haRZWORz/FusOfoddxqEXGJyHtHM38TQSDPUEQuEJFdIrJfRF4f7TyOdwH8f44XkadFZLf9DG8bi3yOZyJyj4hUicg+P/tHJ64YYybsC2sZy6NAPhAO7AYW9jnmKuA5QICzgc1jne/x9ArwGZ4DJNo/X6nPcHjP0eu4V7FWeXvvWOd7PL0C/CwmAAeAXPv31LHO93h6BfgMvw782P45BagDwsc67+PpBZwHrAT2+dk/KnFlopeg1wBFxphjxpgu4CHguj7HXAf8w1g2AQkikjHaGR3HBn2Gxpi3jTH19q+bgOxRzuNEEMhnEeAzwGNA1WhmboII5Bm+D3jcGHMKwBijz7G3QJ6hAWJFRIAYrADtHN1sjm/GmDewnos/oxJXJnqAzgKKvX4vsbcN9ZipbKjP52NY3xxVb4M+RxHJAq4H/jiK+ZpIAvkszgUSRWSDiGwXkQ+NWu4mhkCe4e+ABUAZsBf4nDHGPTrZmzRGJa6EjnSCo0x8bOs7biyQY6aygJ+PiFyIFaDXBTVHE1Mgz/FXwFeMMS6r8KL6COQZhgKrgIuBKOAdEdlkjDkS7MxNEIE8w8uBXcBFwCzgJRF50xjTFOS8TSajElcmeoAuAXK8fs/G+lY41GOmsoCej4gsBf4CXGmMqR2lvE0kgTzHAuAhOzgnA1eJiNMY8+So5HD8C/T/c40xphVoFZE3gGWABmhLIM/wNuBHxmpMLRKR48B8YMvoZHFSGJW4MtGruLcCc0RkpoiEA7cAT/U55ingQ3avu7OBRmNM+WhndBwb9BmKSC7wOPBBLan4NehzNMbMNMbkGWPygH8Bd2pw7iWQ/8//BtaLSKiITAPOAg6Ocj7Hs0Ce4SmsGghEJA2YBxwb1VxOfKMSVyZ0CdoY4xSRTwMvYPVevMcYs19E7rD3/xGrt+xVQBHQhvXtUdkCfIbfBJKAP9ilP6fRVXF6CfA5qgEE8gyNMQdF5HlgD+AG/mKM8TkUZioK8HP4XeBeEdmLVVX7FWOMLkPpRUQeBC4AkkWkBPhfIAxGN67oVJ9KKaXUODTRq7iVUkqpSUkDtFJKKTUOaYBWSimlxiEN0EoppdQ4pAFaKaWUGoc0QCsVJCJiROQ+r99DRaRaRJ4J4NwW+988EXmf1/YCEfnNIOfm+VuFx+uYdw204tZQiMh8e3WpnSIya4jnXiAi54xEPpSabDRAKxU8rcBiEYmyf78UKB1iGnlYC0QAYIzZZoz57JlmzBjzlDHmR2eaju3dwL+NMSuMMUeHeO4FWKulBUxEJvT8DUoFSgO0UsH1HHC1/fOtwIOeHSLyLRH5ktfv+0Qkr8/5P8KaOWuXiHzBLnE+43X+fSLyqogUisgn+l5cRN4UkeVev28UkaUi8hER+Z297V57bdu3ReSY2OtUi4hDRP5grxn8jIg8K33WsBaRq4DPAx8XkdfsbU/aC1nsF5HbvY69QkR2iLUO8Sv2vd4BfMG+v/UiMsPet8f+N9crj7+wr/HjwB+/UhOXfhNVKrgeAr5pB9WlwD3A+iGc/1XgS8aYa8CqEu6zfynWerTRwE4R+U+f/X8BPgJ8XkTmAhHGGF8LzGdgLYIyH2saw38B78EqwS8BUrGm1LzH+yRjzLMi8kegxRjzM3vzR40xdXbNwVYReQyrMPBn4DxjzHERmW4f0+tcEXkaaxm/v4vIR4HfYJXQwVrJ6hJjjCuA56bUhKclaKWCyBizByvI3Yo1PeBI+7cxpt2eqvE1rPWAvT0KXCMiYcBHgXv9pPOkMcZtjDkApNnb1gGP2tsr7PQD8VkR2Y21dngOMAfrS8QbxpjjAMYYf2vtrgUesH++j94rpz2qwVlNJVqCVir4ngJ+htXemuS13UnvL8mRw0i771y9vX43xrSJyEtYC8zfhLWili+dXj9Ln38DZpfwLwHW2tfegHVf4iOvgfA+p3UY5ys1YWkJWqnguwf4jjFmb5/tJ4CVAHaV80wf5zYDsQOkfZ2IRIpIEtYXgK0+jvkLVlXx1gFKrr68Bdxgt0Wn2ekPJh6ot4PzfKySM8A7wPkiMhNARKbb2/ve39tYKzABvN/Og1JTkgZopYLMGFNijPm1j12PAdNFZBfwSXyvabwHcNodq77gY/8W4D9Y1cnfNcb0W5PWGLMdaAL+NsSsP4a17u0+4E/AZqBxkHOeB0JFZA/Wqkmb7DxUA7cDj9vV3w/bxz8NXO/pJAZ8FrjNPv+DwOeGmGelJg1dzUqpCUpEvkXvzln+jssENgDzjTHuIV4jxhjTYpfQtwDn2u3RSqkg0zZopSYxEfkQ8H3gi0MNzrZnRCQBCMcqoWtwVmqUaAlaKaWUGoe0DVoppZQahzRAK6WUUuOQBmillFJqHNIArZRSSo1DGqCVUkqpcej/A17b9X9cLoMqAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "fig, axs = plt.subplots(1, 1, figsize=(8,4))\n", "\n", "factors = np.linspace(0.001, 1, 100)\n", "\n", "axs.plot(\n", " factors,\n", " [stability(\n", " explain_single,\n", " X_test[0],\n", " neighborhood(X_test[0], X_train, factor, size=100)\n", " ) for factor in factors]\n", ")\n", "\n", "axs.set_title(\"Stability of explanation (smaller is better)\")\n", "axs.set_ylabel(\"Stability\")\n", "axs.set_xlabel(\"Multiplying factor\")\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "3a1d7171", "metadata": {}, "source": [ "## The Jaccard Index\n", "\n", "In _Zhengze Zhou, Giles Hooker, Fei Wang. \"S-LIME: Stabilized-LIME for Model Explanation\".\tIn Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '21), August 14--18, 2021, Virtual Event, Singapore_, the authors used the Jaccrd index to evaluate the stability of a function.\n", "\n", "$$J(A, B) = \\frac{|A \\cap B|}{|A \\cup B|}.$$\n", "\n", "This method ignores ordering, and is only concerned if the most important features will always be the same. Values close to 1 means that the algorithm is stable accros different iterations.\n", "\n", "The Jaccard index is the ratio of the intersection over the union of two data sets. In the paper, to evaluate the stability for aa given test instance $t$, they:\n", "1. Execute the explanation method 20 times;\n", "2. Saved the $k$ most important features (with $k=\\{1, 2, 3, 4, 5\\}$);\n", "3. For each value of $k_i$, the $k_i$ most important features for each execution were used to calculate the Jaccard index across all possible pairs (20*19 pairs), then the average was reported.\n", "\n", "Their result look like this:\n", "\n", "| k | Jaccard Index for the method |\n", "|---|:----------------------------:|\n", "| 1 | 1.0 |\n", "| 2 | 0.2 |\n", "| 3 | 0.8 |\n", "| 4 | 0.7 |\n", "| 5 | 0.0 |\n", "\n", "\n", "### Adaptating the Jaccard Index\n", "\n", "In the cell below the Jaccard Index is used to evaluate stability, but since the Partial Effects explanation method is deterministic, the calculation will be performed on the neighborhood of the test point. We can use ``explain_single`` and ``neighborhood`` functions previously declared." ] }, { "cell_type": "code", "execution_count": 10, "id": "154e3d22", "metadata": {}, "outputs": [], "source": [ "def jaccard_index(A, B):\n", " \"\"\"Method to calculate the ratio of the intersection\n", " over the union of two sets. This is known as Jaccard\n", " index and ranges from 0 to 1, measuring how simmilar\n", " the two sets are. A value equals to 1 means that the \n", " sets are identical (remembering that sets does not\n", " have order relations between its elements), and a \n", " value equals to 0 means that they are completely\n", " different. \n", " \n", " Takes as argument two python built-in sets A and B.\n", " \n", " Returns a float number representing the Jaccard Index.\n", " \"\"\"\n", " \n", " return len(A.intersection(B)) / len(A.union(B))\n", "\n", "\n", "def get_k_most_important(explanation, k):\n", " \"\"\"Method that takes an array of explanation and \n", " returns the index of the k most important (highest)\n", " values in the array.\n", " \n", " Takes an array of explanations of shape (n_features, )\n", " and an integer k representing the size of the subset,\n", " k <= len(explanations).\n", " \n", " Returns a python built-in set containing the indexes\n", " of the k highest values.\n", " \"\"\"\n", " \n", " # Reversing the order so its in descending order\n", " order = np.argsort(explanation)[::-1]\n", "\n", " \n", " return set(order[:k])\n", " \n", "\n", "def jaccard_stability(explainer, x, neighborhood, k):\n", " \"\"\"Jaccard adaptation Stability function.\n", " \n", " Takes as argument an explanation method, a single observation\n", " x of shape (n_features, ), the neighborhood as a matrix of\n", " shape (n_neighbors, n_features), and the size of the subset being\n", " considered k\n", " \n", " Returns the mean Jaccard Index between the original sample\n", " and all neighbors, considering how similar the k most important\n", " subset of features between the explanation of the original data\n", " and its neighbors.\n", " \"\"\"\n", " \n", " original_explanation = explainer(x)\n", " original_jaccard_set = get_k_most_important(original_explanation, k)\n", " \n", " return np.nanmean([\n", " jaccard_index(\n", " get_k_most_important(explainer(nb), k),\n", " original_jaccard_set\n", " )\n", " for nb in neighborhood\n", " ])" ] }, { "cell_type": "markdown", "id": "69239668", "metadata": {}, "source": [ "In te cell below we create a table similar to the one reported in the S-LIME paper. For different multiplication factors that adjusts the size of the neighborhood, we can see how stable the explanations are in terms of returning the same most important features.\n", "\n", "Notice that the maximum number of features is 4." ] }, { "cell_type": "code", "execution_count": 11, "id": "6aa92aa3", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
k=1k=2k=3k=4
factor 0.0011.000.7400001.0001.0
factor 0.011.000.6933331.0001.0
factor 0.11.000.7200000.9901.0
factor 1.00.910.6133330.9351.0
\n", "
" ], "text/plain": [ " k=1 k=2 k=3 k=4\n", "factor 0.001 1.00 0.740000 1.000 1.0\n", "factor 0.01 1.00 0.693333 1.000 1.0\n", "factor 0.1 1.00 0.720000 0.990 1.0\n", "factor 1.0 0.91 0.613333 0.935 1.0" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "ks = [1, 2, 3, 4]\n", "fs = [0.001, 0.01, 0.1, 1.0]\n", "\n", "jaccard_df = pd.DataFrame(\n", " data = [[jaccard_stability(\n", " explain_single, X_test[0], neighborhood(X_test[0], X_train, f, size=100), k=k)\n", " for k in ks] for f in fs],\n", " index = [f'factor {f}' for f in fs],\n", " columns = [f'k={k}' for k in ks]\n", ")\n", "\n", "display(jaccard_df)" ] }, { "cell_type": "markdown", "id": "ba26bdeb", "metadata": {}, "source": [ "## (in)fidelity\n", "\n", "If we know the subset of most relevant features, it is expected that the explanation will attribute high values for this features. \n", "\n", "The idea of infidelity is to measure the difference between two terms:\n", "* _i)_ the dot product between a significant perturbation to a given input $x$ we are trying to explain and its explanation, and\n", "* _ii)_ the output observed for the perturbed point.\n", "\n", "$$INFD(\\Phi, \\widehat{f}, \\mathbf{x}) = \\mathbb{E}_{\\mathbf{I} \\sim \\mu_I } \\left [ (\\mathbf{I}^T \\Phi(\\widehat{f}, \\mathbf{x}) - (\\widehat{f}(\\mathbf{x}) - \\widehat{f}(\\mathbf{x} - \\mathbf{I})))^2 \\right ]$$\n", "\n", "Where $\\Phi$ is an importance attribution explainer that takes a model and an instance to explain, $\\widehat{f}$ is our black-box model, and $\\mathbf{I}$ represents perturbations around the point of interest $\\mathbf{x}$.\n", "\n", "The idea is that a good explanation should be robust to changes in the prediction in response to significant perturbations. Infidelity is the expected value of the difference between 2 terms. This metric is useful to see if the explanation is robust to mis-specifications or noise in the given sample.\n", "\n", "A large value of infidelity means that the explainer changes more abruptly when small changes in the input occur, resulting in very different feature importances, and thus is not reliable." ] }, { "cell_type": "code", "execution_count": 12, "id": "26b80700", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "gen | smallest fitness | mean fitness | highest fitness | remaining time\n", "----------------------------------------------------------------------------\n", " 0 | 0.875174 | 1.069199 | 1.153701 | 0min25sec \n", " 10 | 0.785470 | 0.851950 | 0.908672 | 0min25sec \n", " 20 | 0.777389 | inf | inf | 0min20sec \n", " 30 | 0.743367 | 0.768917 | 0.814826 | 0min15sec \n", " 40 | 0.743367 | 0.755422 | 0.808207 | 0min6sec \n" ] } ], "source": [ "housing_data = datasets.fetch_california_housing() \n", "X_reg, y_reg = housing_data['data'], housing_data['target']\n", "labels = housing_data['feature_names']\n", "\n", "X_train_reg, X_test_reg, y_train_reg, y_test_reg = train_test_split(\n", " X_reg, y_reg, test_size=0.33, random_state=42)\n", "\n", "tfuncs = {\n", " 'log' : np.log,\n", " 'sqrt.abs' : lambda x: np.sqrt(np.abs(x)), \n", " 'id' : lambda x: x,\n", " 'sin' : np.sin,\n", " 'cos' : np.cos,\n", " 'exp' : np.exp\n", "}\n", "\n", "tfuncs_dx = {\n", " 'log' : lambda x: 1/x,\n", " 'sqrt.abs' : lambda x: x/( 2*(np.abs(x)**(3/2)) ),\n", " 'id' : lambda x: np.ones_like(x),\n", " 'sin' : np.cos,\n", " 'cos' : lambda x: -np.sin(x),\n", " 'exp' : np.exp,\n", "}\n", "\n", "reg = ITEA_regressor(\n", " gens = 50,\n", " popsize = 50,\n", " max_terms = 5,\n", " expolim = (0, 2),\n", " verbose = 10,\n", " tfuncs = tfuncs,\n", " tfuncs_dx = tfuncs_dx,\n", " labels = labels,\n", " random_state = 42,\n", " simplify_method = None\n", ").fit(X_train_reg, y_train_reg)" ] }, { "cell_type": "code", "execution_count": 13, "id": "d28313ec", "metadata": {}, "outputs": [], "source": [ "explainer_reg = ITExpr_explainer(\n", " itexpr=reg.bestsol_, tfuncs=tfuncs, tfuncs_dx=tfuncs_dx\n", ").fit(X_train_reg, y_train_reg)\n", "\n", "def explain_single_reg(x):\n", " \n", " return np.sum(\n", " explainer_reg.average_partial_effects(x.reshape(1, -1)),\n", " axis=0\n", " )" ] }, { "cell_type": "code", "execution_count": 14, "id": "95b6a41d", "metadata": {}, "outputs": [], "source": [ "def infidelity(explainer, predictor, x, neighborhood):\n", " \n", " original_explanation = explainer(x)\n", " original_prediction = predictor.predict(x.reshape(1, -1))\n", " \n", " return np.mean(np.power(\n", " np.dot((x - neighborhood), np.squeeze(original_explanation)) - \n", " (original_prediction - predictor.predict(neighborhood)), 2))" ] }, { "cell_type": "code", "execution_count": 15, "id": "7c37bf63", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAe4AAAEWCAYAAACg1nQiAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAl1UlEQVR4nO3de7xVdZ3/8ddbRA8oggI6COKhNDVvmGgm2g80JxUVf5Wat2hyYrpMdy/oTIqOzdDUT8FxRmPMkbyFUzaampEKOWmpqJQkGioot+BIcJSUEvv8/ljr4Oaw9znr7LP3OXwP7+fjcR5n3b7f9fnu797nc9Z3rb2WIgIzMzNLwzbdHYCZmZkV58RtZmaWECduMzOzhDhxm5mZJcSJ28zMLCFO3GZmZglx4u6hJP2LpC/n00dLer5guYMkPdpqWR9JP5bULOm/JZ0taVYbdcyR9LcF9xeS9sqnr5f09SLlOkvSZyWtlLRO0sAC239S0i+6IrZqSbpJ0pUFt/2JpAn1jmlLJOkSSTdsAXGMkbS0u+MopyPvJet623Z3AFZ7kgYDnwD2AoiI/wX2KVI2In4jaa2kkyPix/nijwG7AQMjYkO+7NYah01EfKZlWtIY4JaIGFbr/UjqDVwFHBERvy6zvhFYBPQuaW+PEhEndHcM1ZA0h+x9UXXijYh/rl1E9SMpgL0j4oVO1jMZ2CsizqlJYJvXP4dO9ol1jI+4e6ZPAvdFxJtVlr8V+LuS+T2B3/WgJLYb0AD8trsD6WkkdevBQHfv36wrOHH3TCcAP2+ZaT0kJ2mxpPMl/SYf/p4pqaGk/BzgWEnbS7ocuBQ4Ix9WPq/1sLGk4yQ9l9d1LaDSYCR9StICSWsk/VTSnuWCbhmek7QD8BNg93yf6yTtLumN0mFtSYdKasqPoFvXtb2kqZKW5z9T82XvAVpOG6yV9FCZUB4uWb9O0gdK6v123o5Fkk4oWd5f0nclrZC0LG9Hrwrt3EbSJEkvSlot6Q5Ju+TrrpP0g5JtvynpQWXGSFqaD/W+mvfj2RX2sbOke/LXZ00+Paxk/cbTGS39WU3b8rKPSLpa0h+AyUX7Il/X0qavSVqV7+NvKrTpG8DRwLV5v1ybLw9Jn5e0EFiYL5smaYmk1yQ9KenoknomS7oln27My0+Q9Er+uv5Duf3n24+T9HRe7xJlR7MUqUvZKaeb8tf4WeCwNvbT8h78dd7WM/LlJ0map2xU7FFJB5WUuSjvn9clPS/pWEnHA5fwzud3sxGmvOwhkp7Ky84k+8e2ZV3F91IbfVLx9bcaiAj/9LAfoAk4rGR+DLC0ZH4x8DiwO7ALsAD4TKs6XgMOyqcnkw2Ftaz7JPCLfHpQvu3HgN7AV4ANwN/m608FXgD2Izs184/AoyV1BdkwHsBNwJXlYs6X3Qd8tmT+auDfKrwGVwC/AnYFBgOPAv+Ur2vM97tthbKbrc/b/BbwaaAX8FlgOaB8/f8A3wF2yPf5OPB3Fer/ch7bMGD7vNzt+bq+wO/y/R0NvAoMK3lNNpAN828P/B/gj8A+ZV6/gcBH8/r6Af8N/E9JDHNK+qjqtuVlNwBfyPu3Twf7oqVNV5C9f04E3gB2rvDabYy71XvoZ2Tv5T75snPy12Bb4GvA74GG1u/nkr7+T6APcDDwJ2C/CvsfAxxIdtBzELASOLVIXcAU4H/zOPcA5tPqPV6mXXuVzL8PWAW8P++nCWSf5e3JToUtAXYvieXd5T6/ZfazHfAy2We3N9ln+S2qeC+VLKv4+vun8z/dHoB/6tCp2Ydu35L5MWyeuM8pmf9X4PpWdSwDPphPb/LBZ9PE/QngVyXrBCzlnaTwE+C8kvXbkP1h3jOf70jiPgN4JJ/ulf8xOLzCa/AicGLJ/IeBxfl0yx/YjibuF0rm++bb/BXZ0PufKElawJnA7Ar1LwCOLZkfkvfZtvn84cAf8j+mZ7bqxw3ADiXL7gC+3vr1K7PPkcCakvmNf2w707a87CvtvB/b6osxwJutXutVZNcflKtrY9wlywI4pp0Y1gAHt34/l/T1sJJtHwc+XvCzNhW4ukhdwEvA8SXrJtKxxH0d+T88JcueJ/sHbq/8dfsQ2bUZpdtsbG+F/XyQkn/U8mWPVvNeKvL6+6fzPx4q75nWkP1n3Jbfl0y/AezYan0/YG2Bfe1O9p8+AJF9SpeUrN8TmJYP7a0lS0gChhaou7W7gPdKehdwHNAcEY+3EdfLJfMv58s6Y+NrFhFv5JM7krWxN7CipJ3fITvCLGdP4Ecl2y4A3iZLkuRteonsdbqjVdk1EfHHkvmy7ZLUV9J3JL0s6TWy4f8BqjB838m2lfZ3Oe31xerY9PqJcu/H9mwSQz70vkDZ6Zu1QH+y0aFK2vs8tNT7fkmz82HjZuAzZeqtVNcmnxU2fU2K2BP4Wks/5O3ag+wo+wWykZzJwCpJ35dU9P2+O7As/+xuFlsV76VqXn/rACfunuk3wHuqLZx/4LfjnXPBbVlB9sejpaxK58n+UP1dRAwo+ekTEY+2rqiV2GxBxHqyRHY2cC5wcxvll5P9oWsxPF9WxGb7bscSsqPSQSVt3Cki9m9j+xNavSYNEbEMQNLnyYY/lwMXtiq7s7JrAFpUatfXyIZP3x8RO5EdVUGr6w9q1Lb2Xq/O9EVrlfa1cXl+PvUi4HSyIfcBQDMdb3s5twF3A3tERH/g+g7Uu8lnhex16IglwDdavW/6RsTtABFxW0QcRfZaB/DNvFx7/bMCGJp/dsvF1t57aZP66/z6G07cPdV9ZMNn1RoDPBQRfyqw7b3A/pI+ouyK3i+SDbG2uB64WNL+sPFCp9MK1LsSGCipf6vl3yMbnj0FuKWN8rcD/yhpsKRBZBfYtbV9qSbgL8C7imwcESuAWcD/k7STsovP3i2pUh9cD3xD+UV6eYzj8+n3AFeSnSM8F7hQ0shW5S+XtF3+B/IksnOOrfUjG4Jeq+zCt8uKtKUGbSunM33R2kra75d+ZKcUmoBtJV0K7FTl/srV/YeIWC/pcOCsDpS9g+yzsHN+cdcX2tm+dVv/E/hMftQvSTsou1iun6R9JB2j7KK/9WR9/3ZJPY2SKv29/yXZ6/VFSdtK+gjZ6ZrSNrf1XmodZz1ff8OJu6f6HnCipD5Vlj+bLLm0KyJeBU4ju/BmNbA38EjJ+h+R/ef//XyYbT7ZVe/t1fsc2R/8l/Jhwd3z5Y+QJdWnImJxG1VcCcwlG314BngqX1akTW8A3wAeyfd9RIFinyAbpXiW7FTFD8jOXZczjeyobZak18ku3Hp//o/PLcA3I+LXEbGQ7Irgm/M/yJANw64hO2K9leyiwufK7GMq2QVSr+b131+gDbVoWzlV90UZ04CP5Vc3X1Nhm5+SXVvxO7Ih3/W0P5xf1OeAK/J+u5TNT2W05fI8nkVk/wy1NWIE2bD3jPw9eHpEzCW7gPBasn54geyfWMhGaKaQ9ffvyU5lXJKva/nHbrWkp1rvJCL+DHwkr2sN2bUkd5ZsMpW230ut+6Ser7/xzlWj1sNI+mdgVURM7WC5A4HpEfGBdjfuJsq+wnVbbGU3fFAdb0pjZulw4rakSDqM7Ks/e0TE690dT1dy4jYz8FC5JUTSDOAB4MtbW9I2M2vhI24zM7OE+IjbzMwsIUnckH/QoEHR2NjY3WGYmZl1iSeffPLViBhcbl0SibuxsZG5c+d2dxhmZmZdQlLFO+t5qNzMzCwhTtxmZmYJceI2MzNLSBLnuM3MbMvw1ltvsXTpUtavX9/dofQIDQ0NDBs2jN69excu48RtZmaFLV26lH79+tHY2MimDxSzjooIVq9ezdKlSxkxYkThch4qNzOzwtavX8/AgQOdtGtAEgMHDuzw6IUTt5mZdYiTdu1U81o6cZuZmSXE57jNzKxqo6c8xLK1b9asvqED+vDIpGMqrl+7di233XYbn/vc5zpc94knnshtt93GgAEDqoptxx13ZN26dXWJrSOcuLvS1QdC8yvVle0/HL7yTG3jMTPrpGVr32TxlHE1q69x0r1trl+7di3/8R//UTY5vv322/Tq1ati2fvuu6/T8bWlrdhqyYm7KzW/ApObqys7uX9tYzEzS9CkSZN48cUXGTlyJMcddxzjxo3j8ssvZ8iQIcybN49nn32WU089lSVLlrB+/Xq+9KUvMXHiROCd22evW7eOE044gaOOOopHH32UoUOHctddd9GnT59N9rVo0SLOOussNmzYwPHHH79x+bp16xg/fjxr1qzhrbfe4sorr2T8+PGbxXbZZZeV3a7TImKL/zn00EOjR7hsp+4pa2ZWI88+++wm83tedE9N62+vvkWLFsX++++/cX727NnRt2/feOmllzYuW716dUREvPHGG7H//vvHq6++mtW9557R1NQUixYtil69esXTTz8dERGnnXZa3HzzzZvt6+STT44ZM2ZERMS1114bO+ywQ0REvPXWW9Hc3BwREU1NTfHud787/vKXv2wWW6XtWmv9mkZEAHOjQk70xWlmZpa0ww8/fJPvQV9zzTUcfPDBHHHEESxZsoSFCxduVmbEiBGMHDkSgEMPPZTFixdvts0jjzzCmWeeCcC55567cXlEcMkll3DQQQfxoQ99iGXLlrFy5crNyhfdrqM8VG5mZknbYYcdNk7PmTOHBx54gF/+8pf07duXMWPGlP2e9Pbbb79xulevXrz5ZvkL7Mp9XevWW2+lqamJJ598kt69e9PY2Fh2H0W36ygfcZuZWTL69evH66+/XnF9c3MzO++8M3379uW5557jV7/6VdX7Gj16NN///veBLAmX7mPXXXeld+/ezJ49m5dffrlsbJW266y6HnFLGgDcABwABPAp4HlgJtAILAZOj4g19YwjZS1ftVjc0P7VlkW091ULM7OOGDqgT03+NpXW15aBAwcyevRoDjjgAE444QTGjdv0ivbjjz+e66+/noMOOoh99tmHI444oupYpk2bxllnncW0adP46Ec/unH52Wefzcknn8yoUaMYOXIk++67b9nYLrroorLbdZayc+D1IWkG8L8RcYOk7YC+wCXAHyJiiqRJwM4RcVFb9YwaNSrmzp1btzi7zOT+Hb6qvHHSvdlXLaoo22Z9ZmZVWLBgAfvtt193h9GjlHtNJT0ZEaPKbV+3oXJJOwEfBL4LEBF/joi1wHhgRr7ZDODUesVgZmbW09TzHPe7gCbgvyQ9LekGSTsAu0XECoD89651jMHMzKxHqec57m2B9wFfiIjHJE0DJhUtLGkiMBFg+PDh9YmwDtq6/V8156nbO99jZmZbl3om7qXA0oh4LJ//AVniXilpSESskDQEWFWucERMB6ZDdo67jnHWVJu3/5uMzy+bmVmn1G2oPCJ+DyyRtE++6FjgWeBuYEK+bAJwV71iMDMz62nqfQOWLwC35leUvwT8Ddk/C3dIOg94BTitzjGYmZn1GHVN3BExDyh3Ofux9dyvmZl1kc489bCcdp6E2NlHZ06dOpWJEyfSt2/fNrebM2cO3/72t7nnnnsqbjNv3jyWL1/OiSeeWFUs1fItT83MrHqdeephOe08CbGzj86cOnUq55xzTruJu4h58+Yxd+7cLk/cvuWpmZklo/TRmRdccAEA3/rWtzjssMM46KCDuOyyywD44x//yLhx4zj44IM54IADmDlzJtdccw3Lly9n7NixjB07drO677//fvbdd1+OOuoo7rzzzo3LH3/8cY488kgOOeQQjjzySJ5//nn+/Oc/c+mllzJz5kxGjhzJzJkzy25XDz7iNjOzZEyZMoX58+czb948AGbNmsXChQt5/PHHiQhOOeUUHn74YZqamth99925997sK7jNzc3079+fq666itmzZzNo0KBN6l2/fj2f/vSneeihh9hrr70444wzNq7bd999efjhh9l222154IEHuOSSS/jhD3/IFVdcwdy5c7n22msBeO2118puV2tO3GZmlqxZs2Yxa9YsDjnkEADWrVvHwoULOfroozn//PO56KKLOOmkkzj66KPbrOe5555jxIgR7L333gCcc845TJ8+HciS/oQJE1i4cCGSeOutt8rWUXS7zvJQuZmZJSsiuPjii5k3bx7z5s3jhRde4LzzzuM973kPTz75JAceeCAXX3wxV1xxRbt1lXuEJ8DXv/51xo4dy/z58/nxj39c8dGcRbfrLCduMzNLRutHZ374wx/mxhtvZN26dQAsW7aMVatWsXz5cvr27cs555zD+eefz1NPPVW2fIt9992XRYsW8eKLLwJw++23b1zX3NzM0KFDAbjpppsqxlJpu1rzULmZmVWv//B2rwTvcH1taP3ozG9961ssWLCAD3zgAwDsuOOO3HLLLbzwwgtccMEFbLPNNvTu3ZvrrrsOgIkTJ3LCCScwZMgQZs+evbHehoYGpk+fzrhx4xg0aBBHHXUU8+fPB+DCCy9kwoQJXHXVVRxzzDuPRR47dixTpkxh5MiRXHzxxRW3q7W6PtazVlJ6rGebj83szKM5/VhPM9sC+LGetbfFPNbTzMzMas+J28zMLCFO3GZm1iEpnGJNRTWvpS9OS0VnLgBp596/ZmZFNTQ0sHr1agYOHFjx61NWTESwevVqGhoaOlTOiTsVnUm8tbzi08y2asOGDWPp0qU0NTV1dyg9QkNDA8OGDetQGSduMzMrrHfv3owYMaK7w9iq+Ry3mZlZQpy4zczMEuLEbWZmlhAnbjMzs4Q4cZuZmSXEidvMzCwhTtxmZmYJ2eq/xz16ykMsW/tmzeobOqBPzeoyMzNrbatP3MvWvunHXJqZWTI8VG5mZpaQuh5xS1oMvA68DWyIiFGSdgFmAo3AYuD0iFhTzzjMzMx6iq444h4bESMjYlQ+Pwl4MCL2Bh7M583MzKyA7hgqHw/MyKdnAKd2QwxmZmZJqvfFaQHMkhTAdyJiOrBbRKwAiIgVknYtV1DSRGAiwPDhw+scZgdcfSA0v1Jd2f5bUDvMzCxJ9U7coyNieZ6cfybpuaIF8yQ/HWDUqFFRrwA7rPkVmNzc3VGYmdlWqq5D5RGxPP+9CvgRcDiwUtIQgPz3qnrGYGZm1pPULXFL2kFSv5Zp4K+B+cDdwIR8swnAXfWKwczMrKep51D5bsCPJLXs57aIuF/SE8Adks4DXgFOq2MMZmZmPUrdEndEvAQcXGb5auDYeu3XzMysJ/Od08zMzBLixG1mZpYQJ24zM7OEOHGbmZklZKt/rOfWZuiAPjROurem9T0y6Zia1WdmZm1z4t7K1DrJ1vKfADMza5+Hys3MzBLixG1mZpYQJ24zM7OEOHGbmZklxInbzMwsIU7cZmZmCXHiNjMzS4gTt5mZWUKcuM3MzBLixG1mZpYQJ24zM7OEOHGbmZklxInbzMwsIU7cZmZmCXHiNjMzS4gTt5mZWUKcuM3MzBLixG1mZpaQuiduSb0kPS3pnnx+F0k/k7Qw/71zvWMwMzPrKbriiPtLwIKS+UnAgxGxN/BgPm9mZmYF1DVxSxoGjANuKFk8HpiRT88ATq1nDGZmZj1JvY+4pwIXAn8pWbZbRKwAyH/vWq6gpImS5kqa29TUVOcwzczM0lC3xC3pJGBVRDxZTfmImB4RoyJi1ODBg2scnZmZWZq2rWPdo4FTJJ0INAA7SboFWClpSESskDQEWFXHGMzMzHqUuh1xR8TFETEsIhqBjwMPRcQ5wN3AhHyzCcBd9YrBzMysp+mO73FPAY6TtBA4Lp83MzOzAuo5VL5RRMwB5uTTq4Fju2K/ZmZmPY3vnGZmZpYQJ24zM7OEOHGbmZklxInbzMwsIU7cZmZmCSmUuCWNLrLMzMzM6qvoEfe/FVxmZmZmddTm97glfQA4Ehgs6aslq3YCetUzMDMzM9tcezdg2Q7YMd+uX8ny14CP1SsoMzMzK6/NxB0RPwd+LummiHi5i2IyMzOzCore8nR7SdOBxtIyEXFMPYIyMzOz8oom7v8GrgduAN6uXzhmZmbWlqKJe0NEXFfXSMzMzKxdRb8O9mNJn5M0RNIuLT91jczMzMw2U/SIe0L++4KSZQG8q7bhmJmZWVsKJe6IGFHvQMzMzKx9hRK3pE+UWx4R36ttOGZmZtaWokPlh5VMNwDHAk8BTtxmZmZdqOhQ+RdK5yX1B26uS0RmZmZWUbWP9XwD2LuWgZiZmVn7ip7j/jHZVeSQPVxkP+COegVlZmZm5RU9x/3tkukNwMsRsbQO8ZiZmVkbCg2V5w8beY7sCWE7A3+uZ1BmZmZWXqHELel04HHgNOB04DFJfqynmZlZFys6VP4PwGERsQpA0mDgAeAHlQpIagAeBrbP9/ODiLgsv1XqTLInjS0GTo+INdU2wMzMbGtS9KrybVqSdm51gbJ/Ao6JiIOBkcDxko4AJgEPRsTewIP5vJmZmRVQ9Ij7fkk/BW7P588A7murQEQEsC6f7Z3/BDAeGJMvnwHMAS4qHLGZmdlWrM3ELWkvYLeIuEDSR4CjAAG/BG5tr3JJvYAngb2Af4+IxyTtFhErACJihaRdK5SdCEwEGD58eAeaZGZm1nO1N9w9FXgdICLujIivRsRXyI62p7ZXeUS8HREjgWHA4ZIOKBpYREyPiFERMWrw4MFFi5mZmfVo7SXuxoj4TeuFETGX7OKyQiJiLdmQ+PHASklDAPLfqyqXNDMzs1LtJe6GNtb1aaugpMGSBuTTfYAPkX0X/G7eeb73BOCuQpGamZlZu4n7CUmfbr1Q0nlk567bMgSYLek3wBPAzyLiHmAKcJykhcBx+byZmZkV0N5V5V8GfiTpbN5J1KOA7YD/21bBfIj9kDLLV5M9FtTMzMw6qM3EHRErgSMljQVaLiy7NyIeqntkZmZmtpmiz+OeDcyucyxmZmbWjmqfx21mZmbdwInbzMwsIU7cZmZmCXHiNjMzS4gTt5mZWUKcuM3MzBLixG1mZpYQJ24zM7OEOHGbmZklpNCd0yxx/YfD5P7Vl/3KM7WNx8zMqubEvTXoTOKtNuGbmVldeKjczMwsIVvnEffVB0LzKwAsbgAmd6Bs/+H1iMjMzKyQrTNxN78Ck5sBaJx0L4unjOvmgMzMzIrxULmZmVlCnLjNzMwS4sRtZmaWECduMzOzhDhxm5mZJcSJ28zMLCFO3GZmZglx4jYzM0tI3RK3pD0kzZa0QNJvJX0pX76LpJ9JWpj/3rleMZiZmfU09Tzi3gB8LSL2A44APi/pvcAk4MGI2Bt4MJ83MzOzAuqWuCNiRUQ8lU+/DiwAhgLjgRn5ZjOAU+sVg5mZWU/TJee4JTUChwCPAbtFxArIkjuwa4UyEyXNlTS3qampK8I0MzPb4tU9cUvaEfgh8OWIeK1ouYiYHhGjImLU4MGD6xegmZlZQuqauCX1Jkvat0bEnfnilZKG5OuHAKvqGYOZmVlPUrfHekoS8F1gQURcVbLqbmACMCX/fVe9YrAa6D8cJvevuLrN55n3Hw5feaYeUZmZbbXq+Tzu0cC5wDOS5uXLLiFL2HdIOg94BTitjjFYZ7WTeNt8nnkbCd/MzKpTt8QdEb8AVGH1sfXar5mZWU/mO6eZmZklxInbzMwsIU7cZmZmCXHiNjMzS4gTt5mZWUKcuM3MzBLixG1mZpYQJ24zM7OEOHGbmZklxInbzMwsIfW8V7lt7dp5QEm7Zf2AEjOzzThxW/10JvH6ASVmZmV5qNzMzCwhTtxmZmYJ8VC5dcrQAX1onHRvzet8pKY1mpn1HE7c1imPTDqm5nU2TroXGmperZlZj+ChcjMzs4Q4cZuZmSXEidvMzCwhTtxmZmYJ8cVptmXyXdfMzMpy4rYtk++6ZmZWlofKzczMEuLEbWZmlpC6DZVLuhE4CVgVEQfky3YBZgKNwGLg9IhYU68YzABGT3mIZWvfrFl9Qwf0qcuNZ8zMiqjnOe6bgGuB75UsmwQ8GBFTJE3K5y+qYwxmLFv7JounjKtZfbW+xauZWUfUbag8Ih4G/tBq8XhgRj49Azi1Xvs3MzPribr6HPduEbECIP+9a6UNJU2UNFfS3Kampi4L0MzMbEu2xV6cFhHTI2JURIwaPHhwd4djZma2RejqxL1S0hCA/PeqLt6/mZlZ0rr6Bix3AxOAKfnvu7p4/5aAzj7je3HDpheQDR3QpxZhmZltEer5dbDbgTHAIElLgcvIEvYdks4DXgFOq9f+LV2d/qrVZGp6FbmZ2Zakbok7Is6ssOrYeu3TzMysp9tiL04zMzOzzTlxm5mZJcSJ28zMLCFO3GZmZglx4jYzM0uIE7eZmVlCnLjNzMwS4sRtZmaWECduMzOzhDhxm5mZJaSrHzJiVn/9h8Pk/tWX/coztY0ncaOnPMSytW/WtM6hA/p0/p70ZlspJ27reTqTeKtN+D3YsrVv1vyhLZ15+pvZ1s5D5WZmZglx4jYzM0uIh8rNOmjogD5b1FDvL7b/IsP0anWFfU7fLDlO3GYdtMVdVDX5LJjcvHG2cdK9xc9J+5y+WXI8VG5mZpYQH3Gb1crVB0LzK9WV9ZC1mRXkxG1WK82vbDJk3SEesjazgjxUbmZmlhAfcZtZdao9NdB/ODCl5uGYbS2cuM2sOtWeGvBpAbNO8VC5mZlZQpy4zczMEtItQ+WSjgemAb2AGyLCJ7xsy9DZJ4ulpkB7FzcAkyuUrXKfizmrfJ3tWBqDOOpP12y2vO5PG+vMV/06o4u+JliPJ8DVWq37uNZt7son3nV54pbUC/h34DhgKfCEpLsj4tmujsVsM1vbd6kLtLdDd2Kr0T4rGTa5f9lY6n4L2s581a8zuuh6gHo8Aa7Wat3HtW5zV94GuTuGyg8HXoiIlyLiz8D3gfHdEIeZmVlyFBFdu0PpY8DxEfG3+fy5wPsj4u9bbTcRmJjP7gM8X3AXg4Aqn7iwxXKb0uA2paMntsttSkPRNu0ZEYPLreiOc9wqs2yz/x4iYjowvcOVS3MjYlQ1gW2p3KY0uE3p6IntcpvSUIs2dcdQ+VJgj5L5YcDybojDzMwsOd2RuJ8A9pY0QtJ2wMeBu7shDjMzs+R0+VB5RGyQ9PfAT8m+DnZjRPy2hrvo8PB6AtymNLhN6eiJ7XKb0tDpNnX5xWlmZmZWPd85zczMLCFO3GZmZglJMnFLOl7S85JekDSpzHpJuiZf/xtJ7+uOODuqQLvGSGqWNC//ubQ74ixK0o2SVkmaX2F9cv1UoE1J9RGApD0kzZa0QNJvJX2pzDZJ9VXBNqXYVw2SHpf067xdl5fZJrW+KtKm5PoKsjuFSnpa0j1l1lXfTxGR1A/ZBW0vAu8CtgN+Dby31TYnAj8h+874EcBj3R13jdo1Brinu2PtQJs+CLwPmF9hfYr91F6bkuqjPOYhwPvy6X7A71L/TBVsU4p9JWDHfLo38BhwROJ9VaRNyfVVHvdXgdvKxd6ZfkrxiLvILVPHA9+LzK+AAZKGdHWgHdTjbgUbEQ8Df2hjk+T6qUCbkhMRKyLiqXz6dWABMLTVZkn1VcE2JSd//dfls73zn9ZXGKfWV0XalBxJw4BxwA0VNqm6n1JM3EOBJSXzS9n8A1lkmy1N0Zg/kA8p/UTS/l0TWt2k2E9FJNtHkhqBQ8iOekol21dttAkS7Kt8+HUesAr4WUQk31cF2gTp9dVU4ELgLxXWV91PKSbuIrdMLXRb1S1MkZifIrt/7cHAvwH/U++g6izFfmpPsn0kaUfgh8CXI+K11qvLFNni+6qdNiXZVxHxdkSMJLvr5OGSDmi1SXJ9VaBNSfWVpJOAVRHxZFublVlWqJ9STNxFbpma4m1V2405Il5rGVKKiPuA3pIGdV2INZdiP7Up1T6S1Jsswd0aEXeW2SS5vmqvTan2VYuIWAvMAY5vtSq5vmpRqU0J9tVo4BRJi8lOex4j6ZZW21TdTykm7iK3TL0b+ER+1d4RQHNErOjqQDuo3XZJ+itJyqcPJ+u/1V0eae2k2E9tSrGP8ni/CyyIiKsqbJZUXxVpU6J9NVjSgHy6D/Ah4LlWm6XWV+22KbW+ioiLI2JYRDSS/S1/KCLOabVZ1f3UHU8H65SocMtUSZ/J118P3Ed2xd4LwBvA33RXvEUVbNfHgM9K2gC8CXw88ssTt0SSbie7GnSQpKXAZWQXniTbTwXalFQf5UYD5wLP5OcZAS4BhkOyfVWkTSn21RBghqReZMnrjoi4J/G/f0XalGJfbaZW/eRbnpqZmSUkxaFyMzOzrZYTt5mZWUKcuM3MzBLixG1mZpYQJ24zM7OEOHGbJUxSH0k/lzRU0g8KbP+ApJ1L5r+o7Alat0o6RWWeSpdvt67c8nLbSNq9JRZJIyWdWLxFZtYefx3MLGGSPg9sGxHTCm4/ARgWEd/I558DToiIRe2UWxcRO3Z0G0mfBEZFxN8Xic/M2ucjbrO0nQ3cJalR+TPCJX1S0p2S7pe0UNK/lmx/N3Bmvt31ZI+RvVvSV/Jy1+brRkj6paQnJP1T6Q4lXZAv/43KPzu5UdL8/A6AVwBnKHuG8hl5PIPz7bZR9iziLfnWlWZbHCdus0TlifFdEbG4zOqRwBnAgWSJcw+AiFgDbC9pYER8huzeyGMj4upW5acB10XEYcDvS/b518DeZI+hHQkcKumD5eLLH097KTAzIkZGxEzgFrJ/NiC7teWvI+LVjrbdbGvmxG2WrkHA2grrHoyI5ohYDzwL7FmybhWwezt1jwZuz6dvLln+1/nP02RPbNqXLJEXdSPwiXz6U8B/daCsmZHgvcrNbKM3gYYK6/5UMv02m37WG/Ky7Sl3AYyAf4mI7xSKsHWFEUskrZR0DPB+3jn6NrOCfMRtlqh82LuXpErJezP5E5b+CljczqaPkD3VCDZNrj8FPqXsOdfkV7Pv2kY9rwP9Wi27gWzI/I6IeLtg6GaWc+I2S9ss4KgObH8o8KuI2NDOdl8CPi/pCaB/y8KImAXcBvxS0jPAD9g8MZeaDby35eK0fNndwI54mNysKv46mFnCJB0CfDUizi24/TTg7oh4sL6RtRnDKODqiDi6u2IwS5mPuM0SFhFPA7PzZxkXMb+bk/Yk4IfAxd0Vg1nqfMRtZmaWEB9xm5mZJcSJ28zMLCFO3GZmZglx4jYzM0uIE7eZmVlC/j8zXUvoe/036wAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "fig, axs = plt.subplots(1, 1, figsize=(8,4))\n", "\n", "num_samples = 300\n", "\n", "for label, data in zip(['train', 'test'], [X_train_reg, X_test_reg]):\n", " axs.hist(\n", " [infidelity(\n", " explain_single_reg, reg.bestsol_,\n", " x, neighborhood(x, X_train_reg, factor=0.005, size=30)\n", " ) for x in data[:num_samples]],\n", " label=f'{label} data',\n", " histtype='step',\n", " bins=20,\n", " )\n", "\n", "axs.set_title(\"(in)fidelity of the explainer on train and test data\")\n", "axs.set_ylabel(\"Count\")\n", "axs.set_xlabel(\"(in)fidelity\")\n", "axs.legend()\n", "\n", "plt.show()" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.6" } }, "nbformat": 4, "nbformat_minor": 5 }