{
"cells": [
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import matplotlib.pyplot as plt\n",
"plt.style.use('seaborn-whitegrid')\n",
"import pandas as pd\n",
"import numpy as np\n",
"from collections import defaultdict\n",
"\n",
"# Some useful utilities\n",
"\n",
"def laplace_mech(v, sensitivity, epsilon):\n",
" return v + np.random.laplace(loc=0, scale=sensitivity / epsilon)\n",
"\n",
"def gaussian_mech(v, sensitivity, epsilon, delta):\n",
" return v + np.random.normal(loc=0, scale=sensitivity * np.sqrt(2*np.log(1.25/delta)) / epsilon)\n",
"\n",
"def gaussian_mech_vec(v, sensitivity, epsilon, delta):\n",
" return v + np.random.normal(loc=0, scale=sensitivity * np.sqrt(2*np.log(1.25/delta)) / epsilon, size=len(v))\n",
"\n",
"def pct_error(orig, priv):\n",
" return np.abs(orig - priv)/orig * 100.0\n",
"\n",
"def z_clip(xs, b):\n",
" return [min(x, b) for x in xs]\n",
"\n",
"def g_clip(v):\n",
" n = np.linalg.norm(v, ord=2)\n",
" if n > 1:\n",
" return v / n\n",
" else:\n",
" return v"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Machine Learning\n",
"\n",
"```{admonition} Learning Objectives\n",
"After reading this chapter, you will be able to:\n",
"- Describe and implement the basic algorithm for gradient descent\n",
"- Use the Gaussian mechanism to implement differentially private gradient descent\n",
"- Clip gradients to enforce differential privacy for arbitrary loss functions\n",
"- Describe the effect of noise on the training process\n",
"```\n",
"\n",
"In this chapter, we're going to explore building differentially private machine learning classifiers. We'll focus on a kind of *supervised learning* problem: given a set of *labeled training examples* $\\{(x_1, y_1), \\dots, (x_n, y_n)\\}$, in which $x_i$ is called the *feature vector* and $y_i$ is called the *label*, train a *model* $\\theta$ which can *predict* the label for a new feature vector which was not present in the training set. Each $x_i$ is typically a vector of real numbers which describe the features of a training example, and the $y_i$s are drawn from a predefined set of *classes* (usually expressed as integers) that examples can be drawn from. A *binary* classifier has two classes (usually either 1 and 0, or 1 and -1)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Logistic Regression with Scikit-Learn"
]
},
{
"cell_type": "markdown",
"metadata": {
"tags": [
"remove-cell"
]
},
"source": [
"The dataset files you'll need are available here:\n",
"\n",
"- [`adult_processed_x`](https://github.com/uvm-plaid/programming-dp/raw/master/notebooks/adult_processed_x.npy)\n",
"- [`adult_processed_y`](https://github.com/uvm-plaid/programming-dp/raw/master/notebooks/adult_processed_y.npy)"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"X = np.load('adult_processed_x.npy')\n",
"y = np.load('adult_processed_y.npy')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To train a model, we will use some of the data we have available to build a set of *training examples* (as described earlier), but we'll also set aside some of the data as *test examples*. Once we have trained the model, we want to know how well it works on examples that are *not* present in the training set. A model which works well on new examples it hasn't seen before is said to *generalize* well. One which does *not* generalize well is said to have *overfitted* the training data.\n",
"\n",
"To test generalization, we'll use the test examples - we have labels for\n",
"them, so we can test the generalization accuracy of the model by asking\n",
"the model to classify each one, and then comparing the predicted class\n",
"against the actual label from our dataset. We'll split our data into a\n",
"training set containing 80% of the examples, and a testing set containing\n",
"20% of the examples."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"tags": [
"hide-cell"
]
},
"outputs": [
{
"data": {
"text/plain": [
"(9044,)"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"training_size = int(X.shape[0] * 0.8)\n",
"\n",
"X_train = X[:training_size]\n",
"X_test = X[training_size:]\n",
"\n",
"y_train = y[:training_size]\n",
"y_test = y[training_size:]\n",
"\n",
"y_test.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"A simple way to build a binary classifier is with *logistic regression*. The scikit-learn library has a built-in module for performing logistic regression, called `LogisticRegression`, and it's easy to use to build a model using our data."
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"LogisticRegression()"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from sklearn.linear_model import LogisticRegression\n",
"model = LogisticRegression().fit(X_train[:1000],y_train[:1000])\n",
"model"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, we can use the model's `predict` method to predict labels for the test set."
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([-1., -1., -1., ..., -1., -1., -1.])"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model.predict(X_test)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"So, how many test examples does our model get correct? We can compare the\n",
"predicted labels against the actual labels from the dataset; if we divide\n",
"the number of correctly predicted labels by the total number of test\n",
"examples, we can measure the percent of the examples which are correctly\n",
"classified."
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0.8243034055727554"
]
},
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"np.sum(model.predict(X_test) == y_test)/X_test.shape[0]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
" Our model predicts the correct label for 82% of the examples in our test\n",
"set. For this dataset, that's a pretty decent result."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## What is a Model?\n",
"\n",
"What exactly *is* a model? How does it encode the information it uses to make predictions?\n",
"\n",
"There are many different kinds of models, but the ones we'll explore here are *linear models*. For an unlabeled example with a $k$-dimensional feature vector $x_1, \\dots, x_k$, a linear model predicts a label by first calculating the quantity:\n",
"\n",
"\\begin{align}\n",
"w_1 x_1 + \\dots + w_k x_k + bias\n",
"\\end{align}\n",
"\n",
"and then taking the sign of it (i.e. if the quantity above is negative, we predict the label -1; if it's positive, we predict 1).\n",
"\n",
"The model itself, then, can be represented by a vector containing the values $w_1, \\dots, w_k$ and the value for $bias$. The model is said to be linear because the quantity we calculate in predicting a label is a polynomial of degree 1 (i.e. linear). The values $w_1, \\dots, w_k$ are often called the *weights* or *coefficients* of the model, and $bias$ is often called the *bias term* or *intercept*.\n",
"\n",
"This is actually how scikit-learn represents its logistic regression model, too! We can check out the weights of our trained model using the `coef_` attribute of the model:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"scrolled": true,
"tags": [
"hide-output"
]
},
"outputs": [
{
"data": {
"text/plain": [
"(-4.687494808296503,\n",
" array([ 4.78151402e-01, -1.79946560e-01, -3.40197349e-02, 6.84333672e-02,\n",
" -5.91019524e-01, -3.66108816e-01, -1.06083809e+00, -6.69743294e-01,\n",
" -6.36890752e-01, -4.60617087e-01, -5.10608667e-01, -3.97831947e-01,\n",
" -7.57649836e-01, -7.81272015e-01, 6.35777397e-02, 1.18068922e-01,\n",
" 5.10750239e-01, 1.00307253e+00, -1.12279697e-01, 7.38197172e-01,\n",
" -1.18445857e+00, 1.28199942e+00, 1.10337885e-01, -8.97895478e-01,\n",
" 1.50203742e+00, 1.25195060e+00, -5.83470231e-01, -1.31379638e+00,\n",
" -8.89744944e-01, -7.54428949e-01, -5.67430807e-02, 5.14509637e-02,\n",
" -8.09102276e-04, 7.21854816e-01, -8.96542996e-01, -6.94307624e-01,\n",
" -3.23897853e-01, -9.41105845e-01, -1.09690696e+00, 4.72871112e-01,\n",
" 4.76952664e-01, 2.21555807e-01, 5.09558194e-01, -1.29278056e-01,\n",
" -3.74282557e-01, 1.79888307e-03, -7.74198378e-01, -1.05095873e+00,\n",
" -1.90659465e-01, 7.02952290e-01, -5.02465565e-01, -4.51128322e-02,\n",
" -4.88109602e-01, -4.18059124e-01, -2.31600833e-01, -1.17527069e+00,\n",
" -5.10077268e-01, 8.94580565e-01, 6.12561880e-01, -2.84251847e-01,\n",
" -1.27129514e+00, 1.13972192e-01, -4.79608688e-01, -2.56511008e-01,\n",
" -1.84742616e-01, 6.39721203e-01, 8.82889803e-01, 1.58593881e-02,\n",
" 4.91341902e-02, 5.38276653e-02, 4.10230750e-01, -1.65001343e-02,\n",
" 7.53336836e-02, -2.71862104e-01, 5.79575869e-01, 1.69530872e-02,\n",
" 2.56130094e-01, 7.50868301e-01, 7.69366256e-01, 2.66356240e-01,\n",
" -8.56150529e-02, -7.43867191e-01, -5.85226442e-01, -3.65805377e-01,\n",
" -3.50569198e-01, -5.70995791e-01, 2.68340650e-01, 3.62045343e-02,\n",
" 3.74297699e-01, -2.17805065e-01, -9.50762561e-01, -6.53157410e-01,\n",
" -1.22539620e-01, -6.43291000e-01, -4.87931223e-01, 3.02749214e-01,\n",
" -9.45920882e-01, 4.33957129e-01, 2.26093232e+00, 1.02666008e+00,\n",
" 1.99440344e+00, 1.86494843e+01, 2.57234729e+00, 2.74280072e+00]))"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model.intercept_[0], model.coef_[0]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that we'll always have exactly the same number of weights $w_i$ as we have features $x_i$, since we have to multiply each feature by its corresponding weight. That means our model has exactly the same dimensionality as our feature vectors.\n",
"\n",
"Now that we have a way to get the weights and bias term, we can implement our own function to perform prediction:"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0.8447589562140646"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Prediction: take a model (theta) and a single example (xi) and return its predicted label\n",
"def predict(xi, theta, bias=0):\n",
" label = np.sign(xi @ theta + bias)\n",
" return label\n",
"\n",
"np.sum(predict(X_test, model.coef_[0], model.intercept_[0]) == y_test)/X_test.shape[0]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We've made the bias term optional here, because in many cases it's possible to do just as well without it. To make things simpler, we won't bother to train a bias term in our own algorithm."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Training a Model with Gradient Descent\n",
"\n",
"How does the training process actually work? The scikit-learn library has some pretty sophisticated algorithms, but we can do just about as well by implementing a simple one called *gradient descent*. \n",
"\n",
"Most training algorithms for machine learning are defined in terms of a *loss function*, which specifies a way to measure how \"bad\" a model is at prediction. The goal of the training algorithm is to minimize the output of the loss function - a model with low loss will be *good* at prediction. \n",
"\n",
"The machine learning community has developed many different commonly-used loss functions. A simple loss function might return 0 for each correctly predicted example, and 1 for each incorrectly predicted example; when the loss becomes 0, that means we've predicted each example's label correctly. A more commonly used loss function for binary classification is called the *logistic loss*; the logistic loss gives us a measure of \"how far\" we are from predicting the correct label (which is more informative than the simple 0 vs 1 approach).\n",
"\n",
"The logistic loss is implemented by the following Python function:"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [],
"source": [
"# The loss function measures how good our model is. The training goal is to minimize the loss.\n",
"# This is the logistic loss function.\n",
"def loss(theta, xi, yi):\n",
" exponent = - yi * (xi.dot(theta))\n",
" return np.log(1 + np.exp(exponent))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can use the loss function to measure how good a particular model is. Let's try it out with a model whose weights are all zeros. This model isn't likely to work very well, but it's a starting point from which we can train a better one."
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0.6931471805599453"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"theta = np.zeros(X_train.shape[1])\n",
"loss(theta, X_train[0], y_train[0])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We typically measure how good our model is over our entire training set by simply averaging the loss over all of the examples in the training data. In this case, we get *every* example wrong, so the average loss on the whole training set is exactly equal to the loss we calculated above for just one example."
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0.6931471805599453"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"np.mean([loss(theta, x_i, y_i) for x_i, y_i in zip(X_train, y_train)])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Our goal in *training* the model is to *minimize* the loss. So the key question is: how do we modify the model to make the loss smaller? \n",
"\n",
"Gradient descent is an approach that makes the loss smaller by updating the model according to the [*gradient*](https://en.wikipedia.org/wiki/Gradient) of the loss. The gradient is like a multi-dimensional derivative: for a function with multi-dimensional inputs (like our loss function above), the gradient tells you how fast the function's output is changing with respect to *each* dimension of the input. If the gradient is positive in a particular dimension, that means the function's value will *increase* if we increase the model's weight for that dimension; we want the loss to *decrease*, so we should modify our model by *negating* the gradient - i.e. do the *opposite* of what the gradient says. Since we move the model in the opposite direction of the gradient, this is called *descending* the gradient.\n",
"\n",
"When we iteratively perform many steps of this descent process, we slowly get closer and closer to the model which minimizes the loss. This algorithm is called *gradient descent*. Let's see how this looks in Python; first, we'll define the gradient function."
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [],
"source": [
"# This is the gradient of the logistic loss\n",
"# The gradient is a vector that indicates the rate of change of the loss in each direction\n",
"def gradient(theta, xi, yi):\n",
" exponent = yi * (xi.dot(theta))\n",
" return - (yi*xi) / (1+np.exp(exponent))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### A Single Step of Gradient Descent\n",
"\n",
"Next, let's perform a single step of gradient descent. We can apply the `gradient` function to a single example from our training data, which should give us enough information to improve the model for that example. We \"descend\" the gradient by subtracting it from our current model `theta`."
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"tags": [
"hide-output"
]
},
"outputs": [
{
"data": {
"text/plain": [
"array([ 0. , 0. , 0. , 0. , -0.5 ,\n",
" 0. , 0. , 0. , -0.5 , 0. ,\n",
" 0. , 0. , 0. , 0. , 0. ,\n",
" 0. , 0. , 0. , 0. , 0. ,\n",
" 0. , 0. , 0. , 0. , 0. ,\n",
" -0.5 , 0. , 0. , 0. , 0. ,\n",
" 0. , 0. , -0.5 , 0. , 0. ,\n",
" 0. , 0. , 0. , 0. , 0. ,\n",
" 0. , 0. , 0. , 0. , -0.5 ,\n",
" 0. , 0. , 0. , 0. , 0. ,\n",
" 0. , 0. , 0. , 0. , -0.5 ,\n",
" 0. , -0.5 , 0. , 0. , 0. ,\n",
" 0. , 0. , 0. , 0. , 0. ,\n",
" 0. , 0. , 0. , 0. , 0. ,\n",
" 0. , 0. , 0. , 0. , 0. ,\n",
" 0. , 0. , 0. , 0. , 0. ,\n",
" 0. , 0. , 0. , 0. , 0. ,\n",
" 0. , 0. , 0. , 0. , 0. ,\n",
" 0. , 0. , 0. , 0. , 0. ,\n",
" -0.5 , 0. , 0. , -0.25 , -0.0606146 ,\n",
" -0.21875 , 0. , 0. , -0.17676768])"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# If we take a step in the *opposite* direction from the gradient (by negating it), we should \n",
"# move theta in a direction that makes the loss *lower*\n",
"# This is one step of gradient descent - in each step, we're trying to \"descend\" the gradient\n",
"# In this example, we're taking the gradient on just a single training example (the first one)\n",
"theta = theta - gradient(theta, X_train[0], y_train[0])\n",
"theta"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, if we call `predict` on the same example from the training data, its label is predicted correctly! That means our update did indeed improve the model, since it's now capable of classifying this example."
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(-1.0, -1.0)"
]
},
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"y_train[0], predict(theta, X_train[0])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We'll be measuring the accuracy of our model many times, so let's define a helper function for measuring accuracy. It works in the same way as the accuracy measurement for the sklearn model above. We can use it on the `theta` we've built by descending the gradient for one example, to see how good our model is on the test set."
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0.7585139318885449"
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def accuracy(theta):\n",
" return np.sum(predict(X_test, theta) == y_test)/X_test.shape[0]\n",
"\n",
"accuracy(theta)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Our improved model now predicts 75% of the labels for the test set correctly! That's good progress - we've improved the model considerably."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### A Gradient Descent Algorithm\n",
"\n",
"We need to make two changes to arrive at a basic gradient descent algorithm. First, our single step above used only a single example from the training data; we want to consider the *whole* training set when updating the model, so that we improve the model for *all* examples. Second, we need to perform multiple iterations, to get as close as possible to minimizing the loss.\n",
"\n",
"We can solve the first problem by calculating the *average gradient* over all of the training examples, and using it for the descent step in place of the single-example gradient we used before. Our `avg_grad` function calculates the average gradient over a whole array of training examples and the corresponding labels."
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {
"scrolled": true,
"tags": [
"hide-output"
]
},
"outputs": [
{
"data": {
"text/plain": [
"array([-8.03202480e-03, -1.09365062e-02, -5.86649848e-02, -1.70297784e-02,\n",
" -1.85949049e-02, -5.32762100e-03, 3.15432083e-05, 2.24692568e-03,\n",
" 1.80942171e-03, 1.10891317e-03, 7.17940863e-04, 1.22012681e-03,\n",
" 1.09385854e-03, 1.42352970e-03, -4.29266203e-03, -5.73114012e-03,\n",
" -4.96409990e-02, -7.90844879e-03, -1.08970068e-02, -2.50609905e-02,\n",
" 3.27410319e-04, -1.20102580e-02, -1.29608985e-02, 1.15182321e-02,\n",
" -2.26895536e-04, -1.83255483e-01, 1.34642262e-03, 4.47703452e-02,\n",
" 4.31895523e-03, 2.97414610e-03, 6.16295082e-03, -4.88903955e-05,\n",
" -2.13933205e-02, -4.86969833e-02, -8.62802483e-04, 3.11463168e-03,\n",
" 1.23013848e-03, 1.54486498e-02, 1.21336873e-03, -4.38864985e-02,\n",
" -4.34689131e-03, -1.64743409e-02, -4.53583200e-03, -5.47845717e-03,\n",
" -1.67472715e-01, 1.93015718e-02, 4.73608091e-03, 2.44149704e-02,\n",
" 1.61917788e-02, -1.57259641e-02, 6.59058497e-04, -1.58429762e-03,\n",
" 9.21938268e-03, 8.76978910e-04, -1.27725399e-01, 3.39811988e-02,\n",
" -1.52535476e-01, -1.11859092e-04, -7.43481028e-04, -2.46346175e-04,\n",
" 2.71911076e-04, -2.55366711e-04, 4.50825450e-04, 1.10378277e-04,\n",
" 3.56606530e-04, -6.45268003e-04, -2.29994332e-04, -3.86436617e-04,\n",
" -3.08625397e-04, 2.96102401e-04, 1.88227302e-04, 8.58078928e-06,\n",
" 7.20867325e-05, -4.19942412e-05, -8.78083803e-05, -8.39666492e-04,\n",
" -3.06575834e-04, -8.40712924e-05, -5.70563641e-04, 4.00302057e-04,\n",
" -2.64158094e-04, 6.99057157e-05, 2.42709304e-03, 1.82470777e-04,\n",
" 8.76079931e-05, 1.54645694e-04, -2.72063515e-04, -6.37207436e-05,\n",
" 1.24980547e-05, 4.45197135e-04, 4.61621071e-05, 1.15265174e-04,\n",
" -2.77439358e-04, 5.96595409e-05, 1.20539191e-04, -1.18965672e-01,\n",
" 3.44932395e-04, -7.41634269e-05, -6.91870325e-02, -1.45516103e-02,\n",
" -9.95735544e-02, -8.85669054e-03, -9.10018120e-03, -6.35462985e-02])"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def avg_grad(theta, X, y):\n",
" grads = [gradient(theta, xi, yi) for xi, yi in zip(X, y)]\n",
" return np.mean(grads, axis=0)\n",
"\n",
"avg_grad(theta, X_train, y_train)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To solve the second problem, we'll define an iterative algorithm that descends the gradient multiple times."
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [],
"source": [
"def gradient_descent(iterations):\n",
" # Start by \"guessing\" what the model should be (all zeros)\n",
" theta = np.zeros(X_train.shape[1])\n",
"\n",
" # Perform `iterations` steps of gradient descent using training data\n",
" for i in range(iterations):\n",
" theta = theta - avg_grad(theta, X_train, y_train)\n",
"\n",
" return theta"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0.7787483414418399"
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"theta = gradient_descent(10)\n",
"accuracy(theta)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"After 10 iterations, our model reaches nearly 78% accuracy - not bad! Our gradient descent algorithm looks simple (and it is!) but don't let its simplicity fool you - this basic approach is behind many of the recent successes in large-scale deep learning, and our algorithm is *very* close in its design to the ones implemented in popular frameworks for machine learning like TensorFlow.\n",
"\n",
"Notice that we didn't quite make it to the 84% accuracy of the sklearn model we trained earlier. Don't worry - our algorithm is definitely capable of this! We just need more iterations, to get closer to the minimum of the loss."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"With 100 iterations, we get closer - 82% accuracy. However, the algorithm takes a long time to run when we ask for so many iterations. Even worse, the closer we get to minimizing the loss, the more difficult it is to improve - so we might get to 82% accuracy after 100 iterations, but it might take 1000 iterations to get to 84%. This points to a fundamental tension in machine learning - generally speaking, more iterations of training can improve accuracy, but more iterations requires more computation time. Most of the \"tricks\" used to make large-scale deep learning practical are actually aimed at speeding up each iteration of gradient descent, so that more iterations can be performed in the same amount of time.\n",
"\n",
"One more thing that's interesting to note: the value of the loss function does indeed go down with each iteration of gradient descent we perform - so as we perform more iterations, we slowly get closer to minimizing the loss. Also note that the training and testing loss are very close to one another, suggesting that our model is not *overfitting* to the training data."
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"tags": [
"hide-cell"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Training loss: 0.549109439168421\n",
"Testing loss: 0.5415350837580458\n",
"\n",
"Training loss: 0.5224689105514977\n",
"Testing loss: 0.5162665121068426\n",
"\n",
"Training loss: 0.5028090736020403\n",
"Testing loss: 0.49753785424732383\n",
"\n",
"Training loss: 0.4878874803989895\n",
"Testing loss: 0.48335633696635527\n",
"\n",
"Training loss: 0.47628573924997925\n",
"Testing loss: 0.4723742456095848\n",
"\n"
]
}
],
"source": [
"def gradient_descent_log(iterations):\n",
" theta = np.zeros(X_train.shape[1])\n",
"\n",
" for i in range(iterations):\n",
" theta = theta - avg_grad(theta, X_train, y_train)\n",
" print(f'Training loss: {np.mean(loss(theta, X_train, y_train))}')\n",
" print(f'Testing loss: {np.mean(loss(theta, X_test, y_test))}\\n')\n",
"\n",
" return theta\n",
"\n",
"gradient_descent_log(5);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Gradient Descent with Differential Privacy\n",
"\n",
"How can we make the above algorithm differentially private? We'd like to design an algorithm that ensures differential privacy for the training data, so that the final model doesn't reveal anything about individual training examples.\n",
"\n",
"The only part of the algorithm which uses the training data is the gradient calculation. One way to make the algorithm differentially private is to add noise to the gradient itself at each iteration before updating the model. This approach is usually called *noisy gradient descent*, since we add noise directly to the gradient.\n",
"\n",
"Our gradient function is a vector-valued function, so we can use `gaussian_mech_vec` to add noise to its output:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"def noisy_gradient_descent(iterations, epsilon, delta):\n",
" theta = np.zeros(X_train.shape[1])\n",
" sensitivity = '???'\n",
"\n",
" for i in range(iterations):\n",
" grad = avg_grad(theta, X_train, y_train)\n",
" noisy_grad = gaussian_mech_vec(grad, sensitivity, epsilon, delta)\n",
" theta = theta - noisy_grad\n",
"\n",
" return theta"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"There's just one piece of the puzzle missing - **what is the sensitivity of the gradient function**? Answering this question is the central difficulty in making the algorithm work.\n",
"\n",
"There are two major challenges here. First, the gradient is the result of an *average query* - it's the mean of many per-example gradients. As we've seen previously, it's best to split queries like this up into a sum query and a count query. This isn't difficult to do - we can compute the noisy sum of the per-example gradients, rather than their average, and divide by a noisy count later. Second, we need to bound the sensitivity of each per-example gradient. There are two basic approaches for this: we can either analyze the gradient function itself (as we have done with previous queries) to determine its worst-case global sensitivity, or we can *enforce* a sensitivity by clipping the output of the gradient function (as we did in sample and aggregate).\n",
"\n",
"We'll start with the second approach - often called *gradient clipping* - because it's simpler conceptually and more general in its applications."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Gradient Clipping\n",
"\n",
"Recall that when we implemented sample and aggregate, we enforced a desired sensitivity on a function $f$ with unknown sensitivity by clipping its output. The sensitivity of $f$ was:\n",
"\n",
"\\begin{align}\n",
"\\lvert f(x) - f(x') \\rvert\n",
"\\end{align}\n",
"\n",
"After clipping with parameter $b$, this becomes:\n",
"\n",
"\\begin{align}\n",
"\\lvert \\mathsf{clip}(f(x), b) - \\mathsf{clip}(f(x'),b) \\rvert\n",
"\\end{align}\n",
"\n",
"In the worst case, $\\mathsf{clip}(f(x), b) = b$, and $\\mathsf{clip}(f(x'),b) = 0$, so the sensitivity of the clipped result is exactly $b$ (the value of the clipping parameter).\n",
"\n",
"We can use the same trick to bound the $L2$ sensitivity of our gradient function. We'll need to define a function which \"clips\" a vector so that it has $L2$ norm within a desired range. We can accomplish this by *scaling* the vector: if we divide the vector elementwise by its $L2$ norm, then the resulting vector will have an $L2$ norm of 1. If we want to target a particular clipping parameter $b$, we can multiply the scaled vector by $b$ to scale it back up to have $L2$ norm $b$. We want to avoid modifying vectors that already have $L2$ norm below $b$; in that case, we just return the original vector. We can use `np.linalg.norm` with the parameter `ord=2` to calculate the $L2$ norm of a vector."
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [],
"source": [
"def L2_clip(v, b):\n",
" norm = np.linalg.norm(v, ord=2)\n",
" \n",
" if norm > b:\n",
" return b * (v / norm)\n",
" else:\n",
" return v"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we're ready to analyze the sensitivity of the clipped gradient. We denote the gradient as $\\nabla(\\theta; X, y)$ (corresponding to `gradient` in our Python code):\n",
"\n",
"\\begin{align}\n",
"\\lVert \\mathsf{L2\\_clip}( \\nabla (\\theta; X, y), b) - \\mathsf{L2\\_clip}( \\nabla (\\theta; X', y), 0) \\rVert_2\n",
"\\end{align}\n",
"\n",
"In the worst case, $\\mathsf{L2\\_clip}( \\nabla (\\theta; X, y), b)$ has $L2$ norm of $b$, and $\\mathsf{L2\\_clip}( \\nabla (\\theta; X', y))$ is all zeros - so that the $L2$ norm of the difference is equal to $b$. Thus, the $L2$ sensitivity of the clipped gradient is bounded by the clipping parameter $b$!\n",
"\n",
"Now we can proceed to compute the sum of clipped gradients, and add noise based on the $L2$ sensitivity $b$ that we've enforced by clipping."
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {},
"outputs": [],
"source": [
"def gradient_sum(theta, X, y, b):\n",
" gradients = [L2_clip(gradient(theta, x_i, y_i), b) for x_i, y_i in zip(X,y)]\n",
" \n",
" # sum query\n",
" # L2 sensitivity is b (by clipping performed above)\n",
" return np.sum(gradients, axis=0)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we're ready to complete our noisy gradient descent algorithm. To compute the noisy average gradient, we need to:\n",
"\n",
"1. Add noise to the sum of the gradients based on its sensitivity $b$\n",
"2. Compute a noisy count of the number of training examples (sensitivity 1)\n",
"3. Divide the noisy sum from (1) by the noisy count from (2)"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {},
"outputs": [],
"source": [
"def noisy_gradient_descent(iterations, epsilon, delta):\n",
" theta = np.zeros(X_train.shape[1])\n",
" sensitivity = 5.0\n",
" \n",
" noisy_count = laplace_mech(X_train.shape[0], 1, epsilon)\n",
"\n",
" for i in range(iterations):\n",
" grad_sum = gradient_sum(theta, X_train, y_train, sensitivity)\n",
" noisy_grad_sum = gaussian_mech_vec(grad_sum, sensitivity, epsilon, delta)\n",
" noisy_avg_grad = noisy_grad_sum / noisy_count\n",
" theta = theta - noisy_avg_grad\n",
"\n",
" return theta"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0.7800751879699248"
]
},
"execution_count": 30,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"theta = noisy_gradient_descent(10, 0.1, 1e-5)\n",
"accuracy(theta)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Each iteration of this algorithm satisfies $(\\epsilon, \\delta)$-differential privacy, and we perform one additional query to determine the noisy count which satisfies $\\epsilon$-differential privacy. If we perform $k$ iterations, then by sequential composition, the algorithm satisfies $(k\\epsilon + \\epsilon, k\\delta)$-differential privacy. We can also use advanced composition to analyze the total privacy cost; even better, we could convert the algorithm to Rényi differential privacy or zero-concentrated differential privacy, and obtain tight bounds on composition."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Sensitivity of the Gradient\n",
"\n",
"Our previous approach is very general, since it makes no assumptions about the behavior of the gradient. Sometimes, however, we *do* know something about the behavior of the gradient. In particular, a large class of useful gradient functions (including the gradient of the logistic loss, which we're using here) are *Lipschitz continuous* - meaning they have bounded global sensitivity. Formally, it is possible to show that:\n",
"\n",
"\\begin{align}\n",
"\\text{If}\\; \\lVert x_i \\rVert_2 \\leq b\\; \\text{then}\\; \\lVert \\nabla(\\theta; x_i, y_i) \\rVert_2 \\leq b\n",
"\\end{align}\n",
"\n",
"This fact allows us to clip the values of the *training examples* (i.e. the *inputs* to the gradient function), instead of the *output* of the gradient function, and obtain a bound on the $L2$ sensitivity of the gradient.\n",
"\n",
"Clipping the training examples instead of the gradients has two advantages. First, it's often easier to estimate the scale of the training data (and thus to pick a good clipping parameter) than it is to estimate the scale of the gradients you'll compute during training. Second, it's computationally more efficient: we can clip the training examples *once*, and re-use the clipped training data every time we train a model; with gradient clipping, we need to clip each gradient during training. Furthermore, we're no longer forced to compute per-example gradients so that we can clip them; instead, we can compute all of the gradients at once, which can be done very efficiently (this is a commonly used trick in machine learning, but we won't discuss it here).\n",
"\n",
"Note, however, that many useful loss functions - in particular, those derived from neural networks in deep learning - do *not* have bounded global sensitivity. For these loss functions, we're forced to use gradient clipping.\n",
"\n",
"We can clip the training examples instead of the gradients with a couple of simple modifications to our algorithm. First, we clip the training examples using `L2_clip` before we start training. Second, we simply delete the code for clipping the gradients."
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {},
"outputs": [],
"source": [
"def gradient_sum(theta, X, y, b):\n",
" gradients = [gradient(theta, x_i, y_i) for x_i, y_i in zip(X,y)]\n",
" \n",
" # sum query\n",
" # L2 sensitivity is b (by sensitivity of the gradient)\n",
" return np.sum(gradients, axis=0)"
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {},
"outputs": [],
"source": [
"def noisy_gradient_descent(iterations, epsilon, delta):\n",
" theta = np.zeros(X_train.shape[1])\n",
" sensitivity = 5.0\n",
" \n",
" noisy_count = laplace_mech(X_train.shape[0], 1, epsilon)\n",
" clipped_X = [L2_clip(x_i, sensitivity) for x_i in X_train]\n",
"\n",
" for i in range(iterations):\n",
" grad_sum = gradient_sum(theta, clipped_X, y_train, sensitivity)\n",
" noisy_grad_sum = gaussian_mech_vec(grad_sum, sensitivity, epsilon, delta)\n",
" noisy_avg_grad = noisy_grad_sum / noisy_count\n",
" theta = theta - noisy_avg_grad\n",
"\n",
" return theta"
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0.7786377708978328"
]
},
"execution_count": 33,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"theta = noisy_gradient_descent(10, 0.1, 1e-5)\n",
"accuracy(theta)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Many improvements to this algorithm are possible, which can improve privacy cost and accuracy. Many are drawn from the machine learning literature. Some examples include:\n",
"\n",
"- Bounding the *total* privacy cost by $\\epsilon$ by calculating a per-iteration $\\epsilon_i$ as part of the algorithm.\n",
"- Better composition for large numbers of iterations via advanced composition, RDP, or zCDP.\n",
"- Minibatching: calculating the gradient for each iteration using a small chunk of the training data, rather than the whole training set (this reduces the computation needed to calculate the gradient).\n",
"- Parallel composition in conjunction with minibatching.\n",
"- Random sampling of batches in conjunction with minibatching.\n",
"- Other hyperparameters, like a learning rate $\\eta$."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Effect of Noise on Training\n",
"\n",
"So far, we've seen that the number of iterations has a big effect on the accuracy of the model we get, since more iterations can get you closer to the minimum of the loss. Since our differentially private algorithm adds noise to the gradient, this can also affect accuracy - the noise can cause our algorithm to move in *the wrong direction* during training, and actually make the model *worse*.\n",
"\n",
"It's reasonable to expect that smaller values of $\\epsilon$ will result in less accurate models (since this has been the trend in every differentially private algorithm we have seen so far). This is true, but there's also a slightly more subtle tradeoff which occurs because of the composition we need to consider when we perform many iterations of the algorithm: more iterations means a larger privacy cost. In the standard gradient descent algorithm, more iterations generally result in a better model. In our differentially private version, more iterations can make the model *worse*, since we have to use a smaller $\\epsilon$ for each iteration, and so the scale of the noise goes up. In differentially private machine learning, it's important (and sometimes, very challenging) to strike the right balance between the number of iterations used and the scale of the noise added.\n",
"\n",
"Let's do a small experiment to see how the setting of $\\epsilon$ effects the accuracy of our model. We'll train a model for several values of $\\epsilon$, using 20 iterations each time, and graph the accuracy of each model against the $\\epsilon$ value used in training it."
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {
"tags": [
"hide-cell"
]
},
"outputs": [],
"source": [
"delta = 1e-5\n",
"\n",
"epsilons = [0.001, 0.003, 0.005, 0.008, 0.01, 0.03, 0.05, 0.08, 0.1]\n",
"thetas = [noisy_gradient_descent(10, epsilon, delta) for epsilon in epsilons]\n",
"accs = [accuracy(theta) for theta in thetas]"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {
"tags": [
"hide-input"
]
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEDCAYAAADeP8iwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de1RV953//+fhpsAhqMA5Eat+UTQg1okknYyXNEZBV9Kfzb3yM7GX6CRpMpiMWo1Eh0RHTYwr06hZSVauU9v5DoFgG9u0+OtUncyUYIuUeEEjJkESFA6KwuF+ztm/P4AjeFAuckA4r8darrL32Z+93+9oP2/2Z5/9+ZgMwzAQERFpx2+gAxARkeuPioOIiHhQcRAREQ8qDiIi4kHFQUREPKg4iIiIh4CBDqCv5OfnD3QIIiKD0i233OKxb8gUB+g8wSspKioiPj7ei9Fcn3wxb1/MGXwzb1/MGa4t7yv9Yq1hJRER8aDiICIiHlQcRETEg4qDiIh4UHEQEREPKg4iIuJBxUHESwzDQDPiy2A1pN5zEN9iGAbNToMmp4tmh4smp4smh4tmp6t1n0GT00nx2Xps/rYOnzU5XO52bedov6/J2Xqsw/DY19h6nmb39YxLbdvF0OxsKQx+JvD3M+FnMhHgZ8LPz4S/X+vPJtOlz/xN+JtaPu/wWet2y2cQ4OfXcg4T+Pv54e/XyflNJmqqLxJx3NHhs/bnazu/+0/7a7duX4q39Vqmlp/b4u1wDlPn529/vvbXu/p/DzCZTAP6b8tlgMswcLoMDAOchoHLMHC5Wj5r2W+07qd1fyefuWjd33KutvO6XC2fG63Hu9znv3S8q5PPnK2/dDhbjwsbHsCEwL7/JUTFQbyqyeHir1+dp/DrizQ6nJc6aIeLptZOtX1H23RZ59r2c2OHTv9Sh959Z7p1VKC/iUB/P4IC/Fr+t/XnIH8/AgNM7n3mYQEt+9ofG+BHULv2Af5+mGj5P7DD1doZuFp/dncUBg5nWyfS8TP3n9aOx+Fy4XJBvdPZ4XxO16X2ztbzuQyDxqZmTGWNONuu5XHtXv2V9ptLBau1ILYWWXdB7KSYNTU1Evj7CndH7dk5067TbtdRtzuurcMeLIL8/Xj9+2P6/LwqDtLnyqsb2He8gn0nKvifk5XUNjndnwX4tXSegf4mggL8CfI3dehc2zrfkKCA1mPad7yXjmnfGXfWvv2+M1+XMmni/7nqMYH+JoL8/Qb0t9W+1tVbs4ZxqbC0FZm230zbCpHTMHA6Ox7jbFdgOi9m7QqRy7Nth2LW4RzgdLla/tcw3D93dn5nu4LqbFd8q6tdjAgPw9Tubq3lT8u2qd2dT9uftu0rfWZqdzdzaX/rdtvdjsmEqfXzlut0fn2/1oLmZ6K1Tdfn6uyzS+eC4EB/vv6yuM///ag4yDVzOF38rfQC+05UsO+4jWNnqgEYHT6ce6aP4c6bLNw2YRShQQH4+/V/51vkrCR+/Kh+v+71ztQ6NDSUOgFfnT7DG4bSvwvpR+fsjRz43Ma+Ezb++3MbF+ub8fczcev4kTx7Vxx33mRhstU8pH4TF/ElKg7SLS6XweFvLrbcHZyw8dnXFzAMiDQPI3mKlTtvsjB7UiThwYEDHaqI9AGvFofNmzdTWFiIyWQiLS2NadOmuT/71a9+xUcffYSfnx9Tp07lueeeu2KbM2fOsHr1apxOJ1FRUbz88ssEBQV5M3QBLtY180mxjX3HbRz4vIJKexMmE9w8dgT/nDSZO2+ykBB9A34DMFQkIt7lteJw8OBBSkpKyMjIoLi4mLVr15KZmQmA3W7nnXfeYe/evQQEBPDoo4/yt7/9jaampk7bbN++ncWLF3PXXXexdetWsrKyWLx4sbdC91mGYXD8bA37TlSw/7iN/NNVOF0G4cGB3DE5irlxFr47OYpRoSrMIkOd14pDbm4uSUlJAMTGxlJdXY3dbsdsNhMYGEhgYCB1dXWEhIRQX19PeHg4H330Uadt8vLyeOGFFwCYN28e77//vopDH6ltdPA/xZXsb32YfLa6AYCE6Bv46R0TuTMuipvHjhyQB8kiMnC8VhwqKytJSEhwb0dERGCz2TCbzQwbNoynnnqKpKQkhg8fzve+9z1iYmKu2Ka+vt49jBQVFYXNZuv0mkVFRd2Or6GhoUfHDxX19fX8f58W8pev6/jLN/UcLq/H4YLgQBOJ0SGkTI3k1jEhRIQEAC6oK+fzE+UDHfY18dW/a1/M2xdzBu/k7bXicPm0AYZhuL+5YrfbefPNN/nDH/6A2WzmRz/6EcePH79im/bfeLnadAQ9+QqbL33lraHZSe4X59h/vIKcwzbO2h0ATLaaWTp7AnNusnDL+JEEBQzN2VR86e+6PV/M2xdzBu+sBOe14mC1WqmsrHRvV1RUEBkZCcCpU6cYO3Yso0a1fPf81ltv5ciRI1dsExwcTENDA8OHD6e8vByLxeKtsIeM0vN1re8dVPDnU+dodLgYHujHNOtwnkqK486bovjWyJCBDlNErlNeKw6zZs1ix44dpKSkcOzYMSwWC2azGYAxY8Zw6tQpGhoaGDZsGEeOHOGOO+5gwoQJnbaZOXMmOTk53HPPPezdu5fbb7/dW2EPWm3TVOw7UcGfjldwylYLwPiIEP7fvx/HnXEWbosZxZfFnxMfP36AoxWR653XikNiYiIJCQmkpKRgMplIT08nOzubsLAwkpOTWbp0KT/84Q/x9/dn+vTp3HrrrQAebQBSU1NZs2YNGRkZREdHc++993or7EHl7MWGlgfJ7aapCPL347YJo3j4tvHcGWchJjJ0oMMUkUHIq+85rFq1qsN2XFyc++eUlBRSUlK6bANgsVh47733+j7AQcbhdFFQeqF13iIbRa3TVES3TlMx9yYLMyZGEDpM7zaKyLVRL3Kdq7Q38t+apkJE+pmKw3XmatNUzJ9i5c44C7NiNU2FiHiXisN14GJdM/990sa+ExUcOGHjXO2laSpWJE3mzjgLU0ZrmgoR6T8qDgOg/TQV+45XcOj0BZwugxEhLdNU3HmTpqkQkYGl4tDP3vrvL3jnf750T1MxdcwNPDlnInNusnDz2BGapkJErgsqDv2ottHBi384zrfHhLNi/mTmTI7CcsPwgQ5LRMSDikM/KixtGT56OmkSd96kt7xF5Po1NCfTuU7ll1QBkDh25ABHIiJydSoO/Sj/dBWTrWbCQ/Q1VBG5vqk49BOXy+BQSRW3jNddg4hc/1Qc+skpm53qBgeJ41QcROT6p+LQT/7a+rxBdw4iMhioOPST/JIqRoYEapZUERkUVBz6SdvzBk2QJyKDgYpDPzhf28QXlbXcMn7UQIciItItKg794JCeN4jIIKPi0A/+WlJFgJ+Jad8KH+hQRES6RcWhHxwqqSJhTDjDA/0HOhQRkW7x6txKmzdvprCwEJPJRFpaGtOmTQOgvLy8w3KgpaWlrFy5koaGBj766CP3/iNHjlBQUMDGjRspKCggNLTlmz5Lly5lzpw53gy9zzQ5XBR+fYGHbxs/0KGIiHSb14rDwYMHKSkpISMjg+LiYtauXUtmZiYAVquVXbt2AeBwOFiyZAlz584lNDSUhx56yN3+97//PQB1dXVs2rSJ+Ph4b4XrNcfOVNPocHHr/9HzBhEZPLw2rJSbm0tSUhIAsbGxVFdXY7fbPY7bvXs3CxYscN8VtHnttdd48sknAaitrfVWmF6Xr4fRIjIIee3OobKykoSEBPd2REQENpsNs9nc4bjMzEzefffdDvs+++wzRo8eTVRUFNBSHHbu3El1dTVWq5V169YxYsQIj2sWFRV1O76GhoYeHd9b+w+XYwkN4Pw3X3L+G69frkv9lff1xBdzBt/M2xdzBu/k7bXiYBiGx/blL4AVFBQwYcIEj4KRlZXFfffd595OSUkhNjaWmJgYXn/9dXbs2MH69es9rtmTYaeioiKvD1MZhsHnu7/hH2It182QWH/kfb3xxZzBN/P2xZzh2vLOz8/vdL/XhpWsViuVlZXu7YqKCiIjIzscs3//fmbMmOHRNi8vj+nTp7u3k5OTiYmJcf984sQJL0Xdt765UE95daOGlERk0PFacZg1axY5OTkAHDt2DIvF4nGHcPjwYeLi4jrsKy8vJzQ0lKCgIPe+J554grKyMqClcEyaNMlbYfcpPW8QkcHKa8NKiYmJJCQkkJKSgslkIj09nezsbMLCwkhOTgbAZrMRERHRoZ3NZmPUqI7TTDzyyCOkpqYSEhJCcHAwW7Zs8VbYfepQSRUhQf7E3Rg20KGIiPSIV99zaP8uA+Bxl7Bnzx6PNlOnTuXtt9/usG/27NnMnj277wP0svzTVdw8dgQB/nrXUEQGF/VaXlLb6KDoTI2GlERkUFJx8JLC0gs4XQaJKg4iMgipOHhJ28PoxLEqDiIy+Kg4eEn+6SomW82EhwQOdCgiIj2m4uAF9kaHe+U3EZHBSMWhjxVX2Lln5/9gb3Rw97dHD3Q4IiK94tWvsvqa3x8+w6rMQoYH+vPLZbcxc2Jk141ERK5DKg59wOF08fLeE7x54Av+buwI3ngkkdHhwQMdlohIr6k4XKNz9kZS/28Bfz51jodvG8e/LJzCsACt+CYig5uKwzUoLL3AT3+ZT2VtEy8/OI2Hbh070CGJiPQJFYde+r8HT5P+m6NEhQ0j+6czmTomfKBDEhHpMyoOPeRwunhu9xEy/lrK7ZMi2Z4ynZGhQV03FBEZRFQceuiPReVk/LWUJ+6YyM8W3IS/n6nrRiIig4zec+ih/JIqhgX4sSJ5sgqDiAxZKg49VHD6At8eE05QgP7TicjQpR6uB5ocLg5/c5Hp40YMdCgiIl6l4tADx89W0+hwMX2c5kwSkaHNqw+kN2/eTGFhISaTibS0NKZNmwa0rBPdfpW40tJSVq5cSUxMDE8++STjx48HYPLkyaxfv54zZ86wevVqnE4nUVFRvPzyyx3WmO4vBacvAOjOQUSGPK8Vh4MHD1JSUkJGRgbFxcWsXbuWzMxMAKxWK7t27QLA4XCwZMkS5s6dy9GjR1mwYAHPPfdch3Nt376dxYsXc9ddd7F161aysrJYvHixt0K/ooLTVdx4w3BNjSEiQ57XhpVyc3NJSkoCIDY2lurqaux2u8dxu3fvZsGCBYSGhlJbW9vpufLy8pg3bx4A8+bNIzc311thX1VB6QXdNYiIT/BacaisrGTkyEtj8xEREdhsNo/jMjMzefDBBwGoq6sjPz+fZcuW8fDDD/Ppp58CUF9f7x5GioqK6vQ83nbO3kjJuToVBxHxCV4bVjIMw2PbZOr4XkBBQQETJkzAbDYDEBcXx1NPPcW8efP48ssv+clPfsLevXs7tLv8vO0VFRV1O76GhoYeHZ9X2nJXM9JV3aN215ue5j0U+GLO4Jt5+2LO4J28vVYcrFYrlZWV7u2KigoiIzuub7B//35mzJjh3p44cSITJ04EICYmhsjISMrLywkODqahoYHhw4dTXl6OxWLp9Jrx8fHdjq+oqKhHx//u9AkC/Cr4f2ZOIzho8M662tO8hwJfzBl8M29fzBmuLe/8/PxO93ttWGnWrFnk5OQAcOzYMSwWi/sOoc3hw4eJi4tzb2dlZfGLX/wCAJvNxrlz57BarcycOdN9rr1793L77bd7K+wrKiitIn70DYO6MIiIdJfX7hwSExNJSEggJSUFk8lEeno62dnZhIWFkZycDLQUgIiICHeb5ORkVq1aRU5ODk1NTTz//PMEBQWRmprKmjVryMjIIDo6mnvvvddbYXfK6TIoLL3I/Ylj+vW6IiIDxavvObR/lwHocJcAsGfPng7b4eHhvPXWWx7nsVgsvPfee30fYDcVV9ixNzr0MFpEfIbekO6GgtNVAEwfqzejRcQ3qDh0Q8HpC4wMCWR8RMhAhyIi0i9UHLqhoLSK6eNGenwVV0RkqFJx6EJ1QzMnK+zcPFbPG0TEd6g4dOGz0osYhibbExHfouLQhYLTVZhM8He6cxARH6Li0IWC0gvERpm5YXjgQIciItJvVByuwjAMCk5XaUhJRHyOisNVlJyro6quWSu/iYjPUXG4ioLS1pffdOcgIj5GxeEqCk5fIDTIn0mWsIEORUSkX6k4XEXB6Qv83dgR+Pvp5TcR8S0qDldQ3+Sk6Ey1hpRExCd1WRxSU1PZu3cvTU1N/RHPdeNI2UUcLkOT7YmIT+qyOPz4xz+msLCQhx9+mLS0NHJzc6+6VOdQcfSbiwBM+1b4AEciItL/ulzP4ZZbbuGWW24BWlZu27BhA+Xl5fzgBz/g0UcfJSRkaM5UerHeAcCo0KABjkREpP91WRzq6+v505/+xMcff0xlZSV33303d999N//7v//LU089NaCL8HiTvbGZ4YF+BPjrsYyI+J4ui8M999xDUlISTz/9NJMnT3bvv//++ykoKPBqcAPJ3ujEPExTZoiIb+qyOGRnZ5OXl+cuDL/+9a+ZP38+ISEhbNy48aptN2/eTGFhISaTibS0NKZNmwZAeXl5hyVES0tLWblyJQsXLmTr1q3k5+fjcDh4/PHHmT9/Phs3bqSgoIDQ0FAAli5dypw5c3qbc7fYGx2Yh/l79RoiIterLovDqlWrmDFjhnu7sbGRlStX8vrrr1+13cGDBykpKSEjI4Pi4mLWrl1LZmYmAFarlV27dgHgcDhYsmQJc+fO5dNPP+XkyZNkZGRQVVXFfffdx/z586mrq2PTpk3Ex8dfS649UtvowDzcq0tsi4hct7ocUK+pqeFHP/qRe3vRokXY7fYuT5ybm0tSUhIAsbGxVFdXd9pu9+7dLFiwgNDQUL7zne/w6quvAhAeHk59fT1Op5Pa2tpuJ9RX7A0OQoNUHETEN3XZ+5nNZn75y1+SmJiIy+Xi008/JSys6+kkKisrSUhIcG9HRERgs9kwm80djsvMzOTdd98FwN/f3/3tp8zMTL773e/i7+9PbW0tO3fupLq6GqvVyrp16xgxwrsvp9kbHUSPGO7Va4iIXK+6LA7btm3jnXfe4ec//zl+fn5MmzaNrVu3dnniy9+FMAzDYw3mgoICJkyY4FEw/vjHP5KVleUuGikpKcTGxhITE8Prr7/Ojh07WL9+vcc1i4qKuoyrTUNDw1WPP19ThzXY1aNzDgZd5T0U+WLO4Jt5+2LO4J28uywOYWFh/OM//iMXL7a8FNbU1MTy5cvdHfeVWK1WKisr3dsVFRVERkZ2OGb//v0dnmcAfPLJJ7zxxhu8/fbb7juU5ORk9+fJyck8//zznV6zJ88kioqKrnp8k/E10VGj+vU5R3/oKu+hyBdzBt/M2xdzhmvLOz8/v9P9XT5z2LlzJ9///vdZuHAhjz/+OA888ABxcXFdXnDWrFnk5OQAcOzYMSwWi8cdwuHDhzucq6amhq1bt/Lmm292GDZ64oknKCsrAyAvL49JkyZ1ef1rZdcDaRHxYV32fp988gn/9V//xZIlS9i1axdHjx7lD3/4Q5cnTkxMJCEhgZSUFEwmE+np6WRnZxMWFua+E7DZbERERLjbfPzxx1RVVfHMM8+497300ks88sgjpKamEhISQnBwMFu2bOlNrt3W5HDR5HBh1gNpEfFRXfZ+JpMJwzBwOp00NDSQkJDApk2bunXy9u8yAB53HHv27OmwvWjRIhYtWuRxnujoaGbPnt2ta/aF2saWqTN05yAivqrL3m/BggX8+7//OwsXLuSee+4hIiKC4ODg/ohtwNjbisMwFQcR8U1d9n633XYbU6ZMAeCOO+6gqqpqyD/wUXEQEV/X5QPpF198EYejpbOMjo4mISEBP7+hPRmdXcNKIuLjuuz9goODmT9/PnFxcQQGXpqIru1N5qHI3tBSHEJ15yAiPqrL3m/p0qX9Ecd1pe3OIUzFQUR8VJe938GDBzvd//d///d9Hsz1oq046M5BRHxVl73fyJGX1lBubm7m0KFDWK1WrwY10PRVVhHxdV32fg8//HCH7R//+Mc88cQTXgvoelDT9sxBL8GJiI/qsvcrLi7usG2z2fjyyy+9FtD1oLbRQUiQP/5+pq4PFhEZgrosDi+88IL7Z5PJRFhYGGlpaV4NaqC1rAKnuwYR8V1d9oC7du2irKyM6OhoAE6dOsXEiRO9HthAqlFxEBEf1+XbbFu3bmX79u3u7Xfffbdb6zkMZloiVER8XZfF4W9/+xsvvviie3vTpk0UFhZ6NaiBpiVCRcTXdVkcXC4XJ0+edG9/9tlnHqu8DTVay0FEfF2XPeC//Mu/kJ6ezunTpwGIjY294kpsQ4W90aG3o0XEp3XZA06ZMoVt27b51ANpe6NDb0eLiE/TA+nLGIahB9Ii4vP0QPoyjQ4XzU5DX2UVEZ/WZQ/Y9kB60qRJQM8eSG/evJnCwkJMJhNpaWlMmzYNgPLy8g5LiJaWlrJy5UoWLlzYaZszZ86wevVqnE4nUVFRvPzyywQFBfUm3y7VaqEfEZGeP5CeOHEis2bN6vLEBw8epKSkhIyMDIqLi1m7di2ZmZkAWK1Wdu3aBYDD4WDJkiXMnTv3im22b9/O4sWLueuuu9i6dStZWVksXrz4WvK+Is3IKiLSjWGlKVOm8G//9m8sW7aM0aNHc+bMmW6tBJebm0tSUhLQ8g2n6upq7Ha7x3G7d+9mwYIFhIaGXrFNXl4e8+bNA2DevHnk5ub2KMme0BKhIiJXuXO4cOECOTk5/Pa3v6WkpIT58+dTU1PD3r17u3XiyspKEhIS3NsRERHYbDbMZnOH4zIzM3n33Xev2qa+vt49jBQVFYXNZut+hj3UtgpcmB5Ii4gPu2IPOHv2bMaNG8eaNWu4/fbb8fPz49577+32iS9/LmEYBiZTx1lOCwoKmDBhgrtgXKlN+3ZXe95RVFTU7fgaGho6Pb6otBYAW1kpRc3eK0ID5Up5D2W+mDP4Zt6+mDN4J+8rFoctW7bwu9/9jrS0NObOncvdd9/doxNbrVYqKyvd2xUVFURGRnY4Zv/+/cyYMaPLNsHBwTQ0NDB8+HDKy8uxWCydXjM+Pr7b8RUVFXV6/OeN3wDlTI2bRKzF7NlwkLtS3kOZL+YMvpm3L+YM15Z3fn5+p/uv+PBg4cKFvPHGG3z88cckJCTw2muv8cUXX/DSSy95rPHQmVmzZpGTkwPAsWPHsFgsHkNKhw8fJi4urss2M2fOdO/fu3cvt99+e5fX7y09cxAR6ca3lcLDw0lJSSElJYWzZ8/y29/+ltWrV5OdnX3VdomJiSQkJJCSkoLJZCI9PZ3s7GzCwsJITk4GWhYOioiIuGobgNTUVNasWUNGRgbR0dE9Gt7qKS0RKiLSjeLQ3o033siyZctYtmxZt45v/y4D0OEuAWDPnj1dtgGwWCy89957PYi099oeSIcE+vfL9URErkddfyfVx9gbnZiHBeCnJUJFxIepOFzG3tis5w0i4vNUHC7TMiOrhpRExLepOFzG3ujEPDxwoMMQERlQKg6XsTc0Y9adg4j4OBWHy9S2PpAWEfFlKg6XsTc6MA/TsJKI+DYVh8vUaFhJRETFoT3DMKhtcurtaBHxeSoO7TQ0u3C6DC30IyI+T8WhnbZJ98JUHETEx6k4tKMlQkVEWqg4tFOr6bpFRAAVhw5qGjRdt4gIqDh0oIV+RERaqDi0o2ElEZEWKg7t1Kg4iIgAKg4daIlQEZEWXu0FN2/eTGFhISaTibS0NKZNm+b+7MyZM6xYsYLm5mamTJnChg0byMzM5KOPPnIfc+TIEQoKCti4cSMFBQWEhoYCsHTpUubMmdPn8dobHPiZIFhLhIqIj/NacTh48CAlJSVkZGRQXFzM2rVryczMdH/+4osv8uijj5KcnMwLL7xAWVkZDz30EA899JC7/e9//3sA6urq2LRpE/Hx8d4KF2hb6CcAk0lLhIqIb/PasFJubi5JSUkAxMbGUl1djd1uB8DlcpGfn8/cuXMBSE9PJzo6ukP71157jSeffBKA2tpab4XZgb3RobejRUTw4p1DZWUlCQkJ7u2IiAhsNhtms5nz589jNpvZvn07+fn5TJ8+nRUrVrh/Y//ss88YPXo0UVFRQEtx2LlzJ9XV1VitVtatW8eIESP6PGZ7g0NvR4uI4MXiYBiGx3Zb528YBuXl5TzwwAMsX76cxx57jAMHDrifI2RlZXHfffe526akpBAbG0tMTAyvv/46O3bsYP369R7XLCoq6nZ8DQ0NHseXn7+Iv+Hq0XkGm87yHup8MWfwzbx9MWfwTt5eKw5Wq5XKykr3dkVFBZGRkQCMHDmS0aNHM27cOABmzJjByZMn3cUhLy+PdevWudsmJyd3+Pn555/v9Jo9eSZRVFTkcbzxp/NEmQO8/mxjIHWW91DnizmDb+btiznDteWdn5/f6X6vPXOYNWsWOTk5ABw7dgyLxYLZbAYgICCAsWPH8tVXXwFw9OhRYmJiACgvLyc0NJSgoCD3uZ544gnKysqAlsIxadIkr8Rc2+jQOw4iInjxziExMZGEhARSUlIwmUykp6eTnZ1NWFgYycnJpKWlkZ6eTmNjI5MmTXI/nLbZbIwaNarDuR555BFSU1MJCQkhODiYLVu2eCXmtm8riYj4Oq/2hKtWreqwHRcX5/55/PjxvP/++x5tpk6dyttvv91h3+zZs5k9e7ZXYmzP3qA7BxER0BvSboZhYG9yEKa3o0VEVBza1DU5MQwt9CMiAioObpqRVUTkEhWHVpqRVUTkEhWHVrpzEBG5RMWhlV1LhIqIuKk4tNKwkojIJSoOrTSsJCJyiYpDK3trcdBXWUVEVBzc2oqDXoITEVFxcLM3OPD3MzEsQP9JRETUE7Zqm5FVS4SKiKg4uNVoum4RETcVh1aakVVE5BIVh1a1TQ69ACci0krFoZW9QQv9iIi0UXFoZW90EKbiICICqDi4tSwR6j/QYYiIXBe8+qvy5s2bKSwsxGQykZaWxrRp09yfnTlzhhUrVtDc3MyUKVPYsGEDR44c4cknn2T8+PEATJ48mfXr13PmzBlWr16N0+kkKiqKl19+maCgoD6NtbbRiXlYYJ+eU0RksPLancPBgwcpKSkhIyODf/3Xf2Xjxo0dPn/xxRd59NFHycrKwt/fn7KyMurq6liwYAG7djYYAYwAAA0+SURBVO1i165drF+/HoDt27ezePFi/uM//oMxY8aQlZXVp7G6XAb2Rj2QFhFp47XikJubS1JSEgCxsbFUV1djt9sBcLlc5OfnM3fuXADS09OJjo6mtra203Pl5eUxb948AObNm0dubm6fxlrb1DbpnoaVRETAi8NKlZWVJCQkuLcjIiKw2WyYzWbOnz+P2Wxm+/bt5OfnM336dFasWEFdXR35+fksW7aM+vp6UlNT+Yd/+Afq6+vdw0hRUVHYbLZOr1lUVNTt+BoaGtzHV9a2FIeaqkqKipp6m/Kg0D5vX+GLOYNv5u2LOYN38vZacTAMw2O7bWoKwzAoLy/ngQceYPny5Tz22GMcOHCAuLg4nnrqKebNm8eXX37JT37yE/bu3dthSovLz9tefHx8t+MrKipyH19cUQOcJnb8t4iPH9ODLAef9nn7Cl/MGXwzb1/MGa4t7/z8/E73e21YyWq1UllZ6d6uqKggMjISgJEjRzJ69GjGjRuHv78/M2bM4OTJk0ycONE9fBQTE0NkZCTl5eUEBwfT0NAAQHl5ORaLpU9jtTc6Ac3IKiLSxmvFYdasWeTk5ABw7NgxLBYLZrMZgICAAMaOHctXX30FwNGjR4mJiSErK4tf/OIXANhsNs6dO4fVamXmzJnuc+3du5fbb7+9T2NtWyI0NEjFQUQEvDislJiYSEJCAikpKZhMJtLT08nOziYsLIzk5GTS0tJIT0+nsbGRSZMmMXfuXGpqali1ahU5OTk0NTXx/PPPExQURGpqKmvWrCEjI4Po6GjuvffePo3V3tgMaP1oEZE2Xu0NV61a1WE7Li7O/fP48eN5//33O3weHh7OW2+95XEei8XCe++955UYod2wkt5zEBEB9IY0APaGljsHvSEtItJCxQGobWq5c9CwkohICxUHoKbBQaC/iWEBunMQEQEVB+DSEqEiItJCxYG2GVlVHERE2qg40DKspDsHEZFLVBxoGVbS29EiIpeoOKBhJRGRy6k4oAfSIiKXU3EAalQcREQ6UHFAdw4iIpfz+eLgdBnUNTn1drSISDs+XxzsjW1LhKo4iIi08fniUKviICLiweeLQ9udg77KKiJyiYpD252DnjmIiLipODRoWElE5HIqDnrmICLiwas94ubNmyksLMRkMpGWlsa0adPcn505c4YVK1bQ3NzMlClT2LBhAwBbt24lPz8fh8PB448/zvz589m4cSMFBQWEhoYCsHTpUubMmdMnMao4iIh48lqPePDgQUpKSsjIyKC4uJi1a9eSmZnp/vzFF1/k0UcfJTk5mRdeeIGysjJOnz7NyZMnycjIoKqqivvuu4/58+dTV1fHpk2biI+P7/M4NawkIuLJaz1ibm4uSUlJAMTGxlJdXY3dbsdsNuNyucjPz+eVV14BID09HQCr1eq+uwgPD6e+vh6n00ltba23wnR/lVXfVhIRucRrPWJlZSUJCQnu7YiICGw2G2azmfPnz2M2m9m+fTv5+flMnz6dFStW4O/vT0hICACZmZl897vfxd/fn9raWnbu3El1dTVWq5V169YxYsQIj2sWFRV1O76GhgaKioooKTtHoJ+JUydPXHvSg0Bb3r7EF3MG38zbF3MG7+TtteJgGIbHtslkcv9cXl7OAw88wPLly3nsscc4cOCA+znCH//4R7Kysnj33XcBSElJITY2lpiYGF5//XV27NjB+vXrPa7Zk2GnoqIi4uPjGV92kphKp1eGrK5HbXn7El/MGXwzb1/MGa4t7/z8/E73e+3bSlarlcrKSvd2RUUFkZGRAIwcOZLRo0czbtw4/P39mTFjBidPngTgk08+4Y033uCtt94iLCwMgOTkZGJiYtw/nzjRd7/l/3TORH7z1Ow+O5+IyFDgteIwa9YscnJyADh27BgWiwWz2QxAQEAAY8eO5auvvgLg6NGjxMTEUFNTw9atW3nzzTc7DBs98cQTlJWVAZCXl8ekSZP6LM4Afz+Cg/z77HwiIkOB14aVEhMTSUhIICUlBZPJRHp6OtnZ2YSFhZGcnExaWhrp6ek0NjYyadIk5s6dS2ZmJlVVVTzzzDPu87z00ks88sgjpKamEhISQnBwMFu2bPFW2CIigpffc1i1alWH7bi4OPfP48eP5/333+/w+aJFi1i0aJHHeaKjo5k9W0M/IiL9xeffkBYREU8qDiIi4kHFQUREPKg4iIiIBxUHERHxYDIuf5V5kLrSW34iInJ1t9xyi8e+IVMcRESk72hYSUREPKg4iIiIhyFbHDZv3syiRYtISUnhs88+6/DZn//8Zx588EEWLVrEa6+91q02g0Fvct66dSuLFi3igQceYO/evf0dcp/oTd7QMs3xvHnzyM7O7s9w+0Rvcv7oo4/4/ve/z/3338+BAwf6O+Q+0dO8a2tr+ad/+ieWLFlCSkoKn3zyyUCEfU2ulnNjYyOrV6/m/vvv73abbjOGoLy8POOxxx4zDMMwTp48aTz44IMdPr/rrruMsrIyw+l0GosWLTJOnjzZZZvrXW9yzs3NNZYtW2YYhmGcP3/euOOOO/o77GvWm7zbvPLKK8b9999vfPjhh/0a87XqTc7nz5835s+fb9TU1Bjl5eXGunXrBiL0a9KbvHft2mVs27bNMAzDOHv2rLFgwYJ+j/tadJXzhg0bjPfee8+47777ut2mu4bkncOVVqEDKC0tJTw8nNGjR+Pn58cdd9xBbm7uVdsMBr3J+Tvf+Q6vvvoq0HHlvcGkN3kDnDp1iuLi4j5bi7w/9fbf94wZMzCbzVgsFjZu3DiQKfRKb/IeOXIkFy5cAKC6upqRI0cOWPy90VW/9M///M/uz7vbpruGZHGorKzs8I+gbRU6AJvNxqhRo9yfRUZGYrPZrtpmMOhNzldaeW8w6U3e0DLb77PPPtu/wfaR3uT89ddfYxgGzzzzDIsXL3YXycGkN3l/73vfo6ysjOTkZB555BHWrFnT73Ffi676pbZlEHrSpruG5MLJRher0F3OZDJdtc1g0Juc21y+8t5g0pu8f/3rX3PzzTczduzYfomxr/X277q8vJydO3dSVlbGD3/4Q/bt2zfk/43/5je/ITo6mnfeeYfjx4/z3HPP8eGHH/ZLvH2hN/1SX/VlQ7I4XG0Vuss/Ky8vJyoqioCAgCu2GQx6kzNcWnnv7bffdq+8N5j0Ju/9+/dTWlrK/v37OXv2LEFBQdx4443MnDmz3+Pvjd7kHBwczPTp0wkICGDcuHGEhoZy/vx5IiIi+j3+3upN3ocOHXJP9x8XF0d5eTkOh4OAgMHR9V0t575s05khOax0tVXovvWtb2G32/n6669xOBzs27ePWbNmXbXNYNCbnK+08t5g0pu8f/7zn/Phhx/ywQcf8NBDD/Hkk08OmsIAvct59uzZfPrpp7hcLs6fP09dXd2gG3/vTd7jx4+nsLAQgG+++YbQ0NBBUxjg6jn3ZZvODNk3pLdt28Zf//pX9yp0x44dc69C95e//IVt27YBMH/+fJYuXdppm/aLEw0GPc05IyODHTt2uNfnhpax+Ojo6IFKoVd683fdZseOHYwZM8bjq4DXu97k/J//+Z/87ne/o76+np/+9KfMmzdvIFPolZ7mXVtbS1paGufOncPhcPD0008zY8aMAc6iZ66W8/Llyzl79iwnT55k6tSp/OAHP2DhwoV90pcN2eIgIiK9NySHlURE5NqoOIiIiAcVBxER8aDiICIiHlQcRETEw+D5wq/IAPr6669ZuHAhU6dO7bB/x44d3XpHxGazsWPHDjZs2MDcuXPZs2cPoaGh3gpX5JqpOIh0U0xMDLt27epV26ioKDZs2NDHEYl4j4qDyDV49tlnCQkJ4YsvvqCqqootW7YwadIkfvazn2Gz2WhqaiI1NZUJEyawfPnyDmtHnD17lrS0NJqbmzGZTGzatAmTycSzzz7L2LFjOXHiBPHx8WzatGkAMxRfpWcOItfI4XDw/vvv8/TTT/Paa6/x+eefU1VVxa9+9SveeecdLl682Gm7V199lQcffJBdu3axePFidu7cCcDRo0dZsWIFWVlZHDhwgOrq6v5MRwTQnYNIt3355ZcsWbLEvd027UjbvEw333wz27ZtY8KECdTW1vKzn/2M5ORk97TRlzty5AgrV64E4NZbb3WvXjZu3Dj3xIgWi4WamhpuuOEGr+YmcjkVB5Fu6uyZw7PPPovL5XJvm0wmgoOD+eCDDzh06BC7d+9m3759PPXUUx7naz9VvMvlws+v5Ub+8jU1NMONDAQNK4lco0OHDgFQUFDAxIkTOXr0KHv27OHWW2/l+eef59SpU522+/a3v01eXh4Af/nLXzy+CSUykHTnINJNlw8rAQwfPpyAgAAef/xxzp49y9atW7nxxht55ZVXyMjIwN/f32Mm2DbLly/nueee44MPPiAwMJDNmzfT3NzcH6mIdEmzsopcg2effZYFCxZw5513DnQoIn1Kw0oiIuJBdw4iIuJBdw4iIuJBxUFERDyoOIiIiAcVBxER8aDiICIiHlQcRETEw/8PIKr6D3A4UsoAAAAASUVORK5CYII=\n",
"text/plain": [
"