{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "using PyPlot\n", "using LinearAlgebra\n", "using Statistics\n", "using Random\n", "import Base.MathConstants.e" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Part 1: Hyperparameter Optimization\n", "\n", "Let's look at using stochastic gradient descent with various methods to optimize logistic regression.\n", "First, we'll generate a training set at random from the generative model associated with logistic regression.\n", "This generative model is, for label $y \\in \\{-1,1\\}$, features $x \\in \\mathbb{R}^d$ and model $w \\in \\mathbb{R}^d$,\n", "\n", "$$\\mathbf{P}_w(y | x) = \\frac{1}{1 + \\exp(-y x^T w)}.$$\n", "\n", "This means that if we make a bunch of independent observations, the total probability is\n", "\n", "$$p(w) = \\prod_{i=1}^N \\frac{1}{1 + \\exp(-y_i x_i^T w)}$$\n", "\n", "and so maximizing this is equivalent to maximizing the log likelihood\n", "\n", "$$\\log p(w) = -\\sum_{i=1}^N \\log \\left( 1 + \\exp(-y_i x_i^T w) \\right).$$\n", "\n", "The gradient of this is\n", "\n", "$$\\nabla \\log p(w) = -\\sum_{i=1}^N \\frac{\\exp(-y_i x_i^T w) \\cdot (-y_i x_i)}{1 + \\exp(-y_i x_i^T w)}$$\n", "\n", "which reduces to\n", "\n", "$$\\nabla \\log p(w) = \\sum_{i=1}^N \\frac{y_i x_i}{1 + \\exp(y_i x_i^T w)}.$$\n", "\n", "Anyway, we can see that this corresponds to logistic regression." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "# generate the data\n", "Random.seed!(424242)\n", "d = 20;\n", "N = 10000;\n", "wtrue = randn(d);\n", "wtrue = d^2 * wtrue / norm(wtrue);\n", "X = randn(N, d);\n", "X ./= sqrt.(sum(X.^2; dims=2));\n", "Y = (1 ./ (1 .+ exp.(-X * wtrue)) .>= rand(N)) .* 2 .- 1;\n", "sigma = 0.001;" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's do logistic regression with regularization here. Our objective samples will be of the form\n", "\n", "$$f_i(w) = -\\log \\left( 1 + \\exp(-y_i x_i^T w) \\right) + \\frac{\\sigma}{2} \\| w \\|^2$$\n", "\n", "and the SGD updates will look like\n", "\n", "$$w_{t+1} = w_t + \\alpha_t \\left( \\frac{y_i x_i}{1 + \\exp(y_i x_i^T w_t)} - \\sigma w_t \\right).$$\n", "\n", "Let's look at the constants of strong convexity and Lipschitz continuity for this problem, to get a handle on the theory/optimal parameters. If we differentiate the objective twice, we get\n", "\n", "$$\\nabla^2 f_i(w) = x_i x_i^T \\frac{1}{(1 + \\exp(y_i x_i^T w_t)) (1 + \\exp(-y_i x_i^T w_t))} + \\sigma I.$$\n", "\n", "It's pretty easy to see that\n", "\n", "$$0 < \\frac{1}{(1 + \\exp(u)) (1 + \\exp(-u))} \\le \\frac{1}{4},$$\n", "\n", "and so since we initialized such that $\\| x_i \\|^2 = 1$, from the way we generated the examples, we can approximate \n", "\n", "$$\\sigma I \\preceq \\nabla^2 f_i(w) \\preceq \\left(\\sigma + \\frac{1}{4} \\right) I.$$\n", "\n", "So we can set $\\mu = \\sigma$ and $L = \\sigma + \\frac{1}{4}$.\n", "What about bounding the variance of the gradient samples? (Again here I'm using the nonstandard definition of variance for vectors: $\\mathbf{Var}(X) = \\mathbf{E}[\\| X \\|^2] - \\| \\mathbf{E}[ X ] \\|^2$.)\n", "Well,\n", "\n", "\\begin{align*}\n", " \\mathbf{Var}(\\nabla f_i(w))\n", " &=\n", " \\mathbf{Var}\\left( \\frac{y_i x_i}{1 + \\exp(y_i x_i^T w)} - \\sigma w \\right) \\\\\n", " &=\n", " \\mathbf{Var}\\left( \\frac{y_i x_i}{1 + \\exp(y_i x_i^T w)} \\right) \\\\\n", " &\\le\n", " \\mathbf{E}\\left[ \\left\\| \\frac{y_i x_i}{1 + \\exp(y_i x_i^T w)} \\right\\|^2 \\right] \\\\\n", " &\\le\n", " \\mathbf{E}\\left[ \\left\\| x_i \\right\\|^2 \\right] \\\\\n", " &\\le\n", " 1\n", "\\end{align*}\n", "\n", "where this last line happens because we sampled $x_i$ uniformly from the unit ball. So we can set $M = 1$." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "mu = sigma;\n", "L = sigma + 0.25;\n", "M = 1;" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "What is the optimal step size for SGD under these conditions?\n", "Well, from Lecture 2, we had\n", "$$\\alpha_t = \\frac{2 \\mu \\| w_0 - w^* \\|^2}{4 M + \\mu^2 \\| w_0 - w^* \\|^2 t}$$\n", "or\n", "$$\\alpha_t = \\frac{\\alpha_0}{1 + \\gamma t}$$\n", "where\n", "$$\\alpha_0 = \\frac{2 \\mu \\| w_0 - w^* \\|^2}{4 M}$$\n", "and\n", "$$\\gamma = \\frac{\\mu^2 \\| w_0 - w^* \\|^2}{4 M}.$$" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "w0 = randn(d);" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "sgd_logreg (generic function with 1 method)" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "function sgd_logreg(w0, alpha0, gamma, X, Y, sigma, niters, wopt)\n", " w = w0\n", " (N, d) = size(X)\n", " dist_to_optimum = zeros(niters)\n", " for k = 1:niters\n", " alpha = alpha0 / (1 + gamma * (k-1));\n", " i = rand(1:N)\n", " xi = X[i,:];\n", " yi = Y[i];\n", " w = (1 - alpha * sigma) * w + alpha * xi * yi / (1 .+ exp.(yi * dot(xi, w)));\n", " dist_to_optimum[k] = norm(w - wopt);\n", " end\n", " return (w, dist_to_optimum);\n", "end" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "newton_logreg (generic function with 1 method)" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# find the true minimum\n", "function newton_logreg(w0, X, Y, sigma, niters)\n", " N = size(X, 1);\n", " d = size(X, 2);\n", " w = w0;\n", " for k = 1:niters\n", " g = -X' * (Y ./ (1 .+ exp.(Y .* (X * w)))) + N * sigma * w;\n", " H = X' * ((1 ./ ((1 .+ exp.(Y .* (X * w))) .* (1 .+ exp.(-Y .* (X * w))))) .* X) + N * sigma * I;\n", " w = w - H \\ g;\n", " println(\"gradient norm: \$(norm(g))\")\n", " end\n", " return w\n", "end" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "gradient norm: 4000.0143011621494\n", "gradient norm: 897.3056519964077\n", "gradient norm: 233.1829505201235\n", "gradient norm: 59.11585397232122\n", "gradient norm: 5.912513207989782\n", "gradient norm: 0.06801933640344747\n", "gradient norm: 9.136508292096821e-6\n", "gradient norm: 1.7241302462688477e-13\n", "gradient norm: 5.164864426886177e-14\n", "gradient norm: 5.6426423075409946e-14\n" ] } ], "source": [ "wopt = newton_logreg(wtrue, X, Y, sigma, 10);" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "alpha0 = 2 * mu * norm(w0 - wopt)^2 / (4 * M);\n", "gamma = mu^2 * norm(w0 - wopt)^2 / (4 * M);" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "Random.seed!(123456);\n", "(w, dto) = sgd_logreg(w0, alpha0, gamma, X, Y, sigma, 50000, wopt);" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjYAAAG0CAYAAAAhJm17AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAPYQAAD2EBqD+naQAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzs3Xd4VHXe/vF7MkkmhWRIgDQJJNI7CiqIdKSIuK5rwbaKrqu7ICC6Ciorui5BXVlxUXn02Uf0Z8FV0HXXRjGACCqEYijSSyghlJBK2uT8/ggZHZNgJpnJmUzer+ua65o5ZeaTg8ncfs+3WAzDMAQAAOAHAswuAAAAwFMINgAAwG8QbAAAgN8g2AAAAL9BsAEAAH6DYAMAAPwGwQYAAPgNgg0AAPAbBBsAAOA3CDYAAMBvEGwAAIDfCDS7AG8rLy/X0aNHFRERIYvFYnY5AACgFgzDUF5enhISEhQQUPt2GL8PNkePHlViYqLZZQAAgDrIyMhQ69ata3283webiIgISRUXJjIy0uRqAABAbeTm5ioxMdH5PV5bfh9sKm8/RUZGEmwAAGhk3O1GQudhAADgNwg2AADAbxBsAACA3yDYAAAAv0GwAQAAfoNgAwAA/AbBBgAA+A2CDQAA8BsEGwAA4DcINgAAwG8QbAAAgN8g2AAAAL9BsKmH4jKH2SUAAICfINjU0f9bd0BdZn6uZduPm10KAAA4h2BTRzP/vU3lhvTWNwfNLgUAAJxDsKmjKcM7SJLi7SEmVwIAACoRbOqomS1QknS2lH42AAD4CoJNHTULqQg2BcVlJlcCAAAqEWzqKPxci00+wQYAAJ9BsKmjZjarJKmgmFtRAAD4CoJNHYUHcysKAABfQ7CpI25FAQDgewg2dVQ5KooWGwAAfAfBpo6co6JKHCovN0yuBgAASASbOqtssZGkghJabQAA8AUEmzqyBQbIGmCRxMgoAAB8BcGmjiwWi8KDK4Z804EYAADfQLCpBzoQAwDgWwg29RBOsAEAwKcQbOqhcmQUt6IAAPANBJt6cN6KYlQUAAA+gWBTD5XLKuQzKgoAAJ9AsKkH57IKRbTYAADgCwg29RDh7GNTanIlAABAItjUS2RokCQp4/RZkysBAACSycFm9erVGjdunBISEmSxWPTRRx8595WWluqRRx5Rjx49FB4eroSEBP32t7/V0aNHTazY1Ym8IknS59syTa4EAABIJgebgoIC9erVS/Pnz6+yr7CwUBs3btTMmTO1ceNGLVmyRLt27dI111xjQqXV6xATIUmyn2u5AQAA5gr85UO8Z8yYMRozZky1++x2u5YtW+ay7R//+IcuvfRSHTp0SG3atGmIEs8r3h4iSUpqEWZyJQAAQDI52LgrJydHFotFzZs3r/GY4uJiFRcXO1/n5uZ6rZ7Qc2tFFZYw3BsAAF/QaDoPFxUVafr06brlllsUGRlZ43EpKSmy2+3OR2JiotdqqhzuTbABAMA3NIpgU1paqvHjx6u8vFwvv/zyeY+dMWOGcnJynI+MjAyv1RUaVNliwzw2AAD4Ap+/FVVaWqobb7xR+/fv15dffnne1hpJstlsstlsDVLbj4tg0mIDAIAv8OlgUxlqdu/erdTUVLVo0cLsklxE/GQRTEe5IWuAxeSKAABo2kwNNvn5+dqzZ4/z9f79+7V582ZFR0crISFB119/vTZu3Kj//ve/cjgcysysmC8mOjpawcHBZpXtVBlspIplFexhDPsGAMBMpgabDRs2aOjQoc7X06ZNkyTdcccdmjVrlj7++GNJUu/evV3OS01N1ZAhQxqszprYAq0KCQpQUWm5cotKCTYAAJjM1GAzZMgQGYZR4/7z7fMVRaXlkqTTBSVKjGY+GwAAzNQoRkU1BjuP55ldAgAATR7Bpp4q+9lEhZnf5wcAgKaOYFNPvRMrZkHOLy41uRIAAECwqadm5+ayyS9ikj4AAMxGsKmnymCTV0ywAQDAbASbemoWQosNAAC+gmBTT85bUbTYAABgOoJNPdHHBgAA30GwqafKW1H0sQEAwHwEm3pq5lzhm2ADAIDZCDb19NMVvgEAgLkINvXUzFax8CV9bAAAMB/Bpp6YxwYAAN9BsKmnCOaxAQDAZxBs6qmyxeZsqUNljnKTqwEAoGkj2NRT+LlgI0kFxQ4TKwEAAASbegoODJAtsOIy5rHCNwAApiLYeABDvgEA8A0EGw9gWQUAAHwDwcYDWFYBAADfQLDxAFpsAADwDQQbD3DOPkyLDQAApiLYeACT9AEA4BsINh7AsgoAAPgGgo0HNKPFBgAAn0Cw8QBn52Em6AMAwFQEGw9ggj4AAHwDwcYDnH1suBUFAICpCDYe8OOtKIINAABmIth4AJ2HAQDwDQQbD4hggj4AAHwCwcYDmtF5GAAAn0Cw8YCf9rExDMPkagAAaLoINh5QOdzbMGi1AQDATAQbDwgJsio4sOJSnilkkj4AAMxCsPEQe2hFB+LcIoINAABmIdh4iDPYnOVWFAAAZiHYeEjkuX42OWdpsQEAwCwEGw+J5FYUAACmI9h4SOWtqBw6DwMAYBqCjYdEhQVLkk4XlphcCQAATRfBxkMqgw3DvQEAMI+pwWb16tUaN26cEhISZLFY9NFHH7nsNwxDs2bNUkJCgkJDQzVkyBBt27bNpGrPzx5a0Xk4l87DAACYxtRgU1BQoF69emn+/PnV7n/22Wc1d+5czZ8/X+vXr1dcXJyuvPJK5eXlNXClv6yy8zCjogAAME+gmR8+ZswYjRkzptp9hmHohRde0GOPPabrrrtOkvTGG28oNjZW77zzju69996GLPUXMUEfAADm89k+Nvv371dmZqZGjhzp3Gaz2TR48GCtXbu2xvOKi4uVm5vr8mgIdlpsAAAwnc8Gm8zMTElSbGysy/bY2FjnvuqkpKTIbrc7H4mJiV6ts1LzMIINAABm89lgU8lisbi8NgyjyrafmjFjhnJycpyPjIwMb5coybWPTXm50SCfCQAAXJnax+Z84uLiJFW03MTHxzu3Z2VlVWnF+SmbzSabzeb1+n6u8laUYUh5RWWyn2vBAQAADcdnW2ySk5MVFxenZcuWObeVlJRo1apVuvzyy02srHq2QKvCgq2SpDNnmaQPAAAzmNpik5+frz179jhf79+/X5s3b1Z0dLTatGmjqVOnavbs2erQoYM6dOig2bNnKywsTLfccouJVdeseWiQCkscOlNYqrYtzK4GAICmx9Rgs2HDBg0dOtT5etq0aZKkO+64QwsXLtTDDz+ss2fP6o9//KOys7N12WWXaenSpYqIiDCr5POyhwXraE6RztCBGAAAU5gabIYMGSLDqLmjrcVi0axZszRr1qyGK6oemp/rZ3OG9aIAADCFz/axaYwY8g0AgLkINh5UGWxYCBMAAHMQbDwoMpRgAwCAmQg2HlQ5l82h04UmVwIAQNNEsPGC5TuOm10CAABNEsHGgxLsoZKkznG+ORwdAAB/R7DxoKSW4ZIqllQAAAANj2DjQdFhwZKk0wXMYwMAgBkINh4UFV7RefhsqUNFpQ6TqwEAoOkh2HhQM1uggqwWSbTaAABgBoKNB1ksFkVxOwoAANMQbDwsOpxgAwCAWQg2HtaiWUWwOVVQbHIlAAA0PW6v7n3q1Cn9+c9/VmpqqrKyslReXu6y//Tp0x4rrjGKDrdJkk4XsKwCAAANze1gc9ttt2nv3r26++67FRsbK4vF4o26Gq3ocwthZnMrCgCABud2sFmzZo3WrFmjXr16eaOeRi+qso9NIcEGAICG5nYfm86dO+vs2bPeqMUvODsP5xNsAABoaG4Hm5dfflmPPfaYVq1apVOnTik3N9fl0dRF02IDAIBp3L4V1bx5c+Xk5GjYsGEu2w3DkMVikcPRtGfcrVxWgT42AAA0PLeDza233qrg4GC98847dB6uRmUfm91Z+SZXAgBA0+N2sNm6das2bdqkTp06eaOeRq9y5mFJKnOUK9DKVEEAADQUt791+/btq4yMDG/U4hdiImzO5wdOFZpYCQAATY/bLTb333+/pkyZoj/96U/q0aOHgoKCXPb37NnTY8U1RgEBP96aO3LmrNrHNDOxGgAAmha3g81NN90kSbrrrruc2ywWC52Hf6LfhdH6Zt9pnWZZBQAAGpTbwWb//v3eqMOvxNtDJUlZuQQbAAAaktvBpm3btt6ow6+wwjcAAOZwO9i8+eab593/29/+ts7F+IvKFb5PMvswAAANyu1gM2XKFJfXpaWlKiwsVHBwsMLCwgg2klo2qxgZdSKfW1EAADQkt4d7Z2dnuzzy8/O1c+dOXXHFFXr33Xe9UWOjExsZIknKyi0yuRIAAJoWj8we16FDB82ZM6dKa05TFRtZ0WJznGADAECD8ti0uFarVUePHvXU2zVqcedabLILS1VcxvB3AAAaitt9bD7++GOX14Zh6NixY5o/f74GDBjgscIaM3tokIIDA1RSVq6s3GIlRoeZXRIAAE2C28Hm2muvdXltsVjUqlUrDRs2TM8//7zHCmvMLBaLYiNtyjh9Vsdziwg2AAA0ELeDTXl5uTfq8DtxkSHngg0jowAAaCgsPe0lMef62WTSgRgAgAbjdouNYRj64IMPlJqaqqysrCotOEuWLPFYcY1Zq3Nz2ZxkLhsAABpMnSboe/XVVzV06FDFxsbKYrH88klNUKuIimDDelEAADQct4PNW2+9pSVLluiqq67yRj1+I6Yy2ORxKwoAgIbidh8bu92uCy+80Bu1+JUfZx+mxQYAgIbidrCZNWuWnnzySZ09e9Yb9fiNeHtFsDmaw3UCAKChuH0r6oYbbtC7776rmJgYJSUlKSgoyGX/xo0bPVZcYxZ3LtjkFZWpoLhM4Ta3LzUAAHCT29+2d955p9LS0nTbbbfRefg8IkKCFGELVF5xmY7lFKl9TDOzSwIAwO+5HWw++eQTffHFF7riiiu8UY+LsrIyzZo1S2+//bYyMzMVHx+vO++8U48//rgCAnx/Cp745iHKO56vo2fOEmwAAGgAbgebxMRERUZGeqOWKp555hktWLBAb7zxhrp166YNGzZowoQJstvtjWIl8djIEO06nq91+05pUMdWZpcDAIDfc7vZ4/nnn9fDDz+sAwcOeKEcV+vWrdOvfvUrjR07VklJSbr++us1cuRIbdiwweuf7QkHThVIkkICrSZXAgBA0+B2sLntttuUmpqqdu3aKSIiQtHR0S4PT7riiiu0YsUK7dq1S5K0ZcsWrVmz5rxz6BQXFys3N9flYZZxPRMkSacLGPINAEBDcPtW1AsvvOCNOqr1yCOPKCcnR507d5bVapXD4dBf//pX3XzzzTWek5KSoieffLLBajyfykn6TrCsAgAADcLtYHPHHXd4o45qvffee3rrrbf0zjvvqFu3btq8ebOmTp2qhISEGuuYMWOGpk2b5nydm5urxMTEhirZReUkfZk5zD4MAEBDqFWwyc3NdXYY/qVbO57sWPynP/1J06dP1/jx4yVJPXr00MGDB5WSklJjsLHZbLLZbB6roT7im4dKko6eIdgAANAQahVsoqKidOzYMcXExKh58+bVzl1jGIYsFoscDofHiissLKwyrNtqtVZZUdxXJZybpC8rr0iOckPWAOb8AQDAm2oVbL788ktnx+DU1FSvFvRT48aN01//+le1adNG3bp106ZNmzR37lzdddddDVZDfbRoZlOARSo3pFMFxYqJCDG7JAAA/Fqtgs3gwYOdz5OTk5WYmFil1cYwDGVkZHi0uH/84x+aOXOm/vjHPyorK0sJCQm699579ec//9mjn+Mt1gCLWjSz6UResbJyCTYAAHib252Hk5OTnbelfur06dNKTk726K2oiIgIvfDCCw06EsvTYiMrgk1mTpG6X2A3uxwAAPya2/PYVPal+bn8/HyFhNAi8XPx9ooOxMdY5RsAAK+rdYtN5RBqi8WimTNnKiwszLnP4XDo22+/Ve/evT1fYSN3QfPKYMPIKAAAvK3WwWbTpk2SKlps0tPTFRwc7NwXHBysXr166aGHHvJ8hY1cQvOKVqwjZ2ixAQDA22odbCpHQ02YMEHz5s1rsIUwG7sE51w2BBsAALzN7c7Dr7/+uvN5RkaGLBaLWrdu7dGi/EkCk/QBANBg3O48XFZWppkzZ8putyspKUlt27aV3W7X448/rtLSUm/U2KhV9rHJzC1SmaNxTCwIAEBj5XaLzaRJk/Thhx/q2WefVf/+/SVJ69at06xZs3Ty5EktWLDA40U2Zq2a2RRktajUYeh4XrEz6AAAAM9zO9i8++67WrRokcaMGePc1rNnT7Vp00bjx48n2PxMQIBFcfYQZZw+q2NnzhJsAADwIrdvRYWEhCgpKanK9qSkJJeRUvhR5Vw2jIwCAMC73A42EydO1F/+8hcVFxc7txUXF+uvf/2rJk2a5NHi/EX7mGaSpB3H8kyuBAAA/+b2rahNmzZpxYoVat26tXr16iVJ2rJli0pKSjR8+HBdd911zmOXLFniuUobsa7xFUPjd2bmmlwJAAD+ze1g07x5c/3mN79x2ZaYmOixgvzRhS3DJUkHThWaXAkAAP6tXvPYoHaSW1UEm0OnC1XqKFeQ1e07gAAAoBbcDjaVTpw4oZ07d8pisahjx45q1aqVJ+vyK3GRIQoLtqqwxKGM04W6sFUzs0sCAMAvud10UFBQoLvuukvx8fEaNGiQBg4cqISEBN19990qLORWS3UsFovatqi8HVVgcjUAAPgvt4PNtGnTtGrVKv3nP//RmTNndObMGf373//WqlWr9OCDD3qjRr+Q3LJiNfT9Jwl/AAB4i9u3ohYvXqwPPvhAQ4YMcW676qqrFBoaqhtvvFGvvPKKJ+vzG0mVLTYnabEBAMBb3G6xKSwsVGxsbJXtMTEx3Io6j6SW3IoCAMDb3A42/fv31xNPPKGioh9Xqz579qyefPJJ59pRqCr5XLDZT4sNAABe4/atqHnz5mn06NHOCfosFos2b96skJAQffHFF96o0S9U3oo6euasisscsgVaTa4IAAD/43aw6d69u3bv3q233npLP/zwgwzD0Pjx43XrrbcqNJQFHmvSslmwmtkClV9cpozThWofE2F2SQAA+J06zWMTGhqqe+65x9O1+DWLxaKklmHaeiRX+08SbAAA8AamwG1AiVEVQ74zTtPJGgAAbyDYNKA2LSqCzUFGRgEA4BUEmwbU/txSCntO5JtcCQAA/olg04DaxVQEm71ZtNgAAOANdV4EMy0tTTt27JDFYlGXLl108cUXe7Iuv9TuXItNZm6R8opKFRESZHJFAAD4F7eDTVZWlsaPH6+VK1eqefPmMgxDOTk5Gjp0qBYtWsQq3+dhDw1SqwibTuQVa9+JAvVKbG52SQAA+BW3b0Xdf//9ys3N1bZt23T69GllZ2dr69atys3N1eTJk71Ro19x9rPJop8NAACe5naLzeeff67ly5erS5cuzm1du3bVSy+9pJEjR3q0OH/ULiZc6/adogMxAABe4HaLTXl5uYKCqvYNCQoKUnl5uUeK8meV/Wz20mIDAIDHuR1shg0bpilTpujo0aPObUeOHNEDDzyg4cOHe7Q4f9Q+hiHfAAB4i9vBZv78+crLy1NSUpLatWun9u3bKzk5WXl5efrHP/7hjRr9SmWLzaFThSp10MIFAIAnud3HJjExURs3btSyZcuci2B27dpVI0aM8EZ9fifeHqKwYKsKSxw6eKqANaMAAPAgt1ts3nzzTRUXF+vKK6/U/fffr8mTJ2vEiBEqKSnRm2++6Y0a/YrFYnG22uxhoj4AADzK7WAzYcIE5eTkVNmel5enCRMmeKQof+fsZ5OVZ3IlAAD4F7eDjWEYslgsVbYfPnxYdrvdI0X5uw6xFcFm13E6EAMA4Em17mNz0UUXyWKxyGKxaPjw4QoM/PFUh8Oh/fv3a/To0V4p0t90iq3oV7PrOC02AAB4Uq2DzbXXXitJ2rx5s0aNGqVmzZo59wUHByspKUm/+c1vPF+hH+p4LtjsPZGvUke5gqysRQoAgCfUOtg88cQTkqSkpCTddNNNCgkJ8VpR/u6C5qEKD7aqoMSh/ScLnEEHAADUj9tNBXfccQehpp4CAiy6ICpUkvTNvlMmVwMAgP/w+XsgR44c0W233aYWLVooLCxMvXv3Vlpamtll1Vub6HBJ0rf7TptcCQAA/sOng012drYGDBigoKAgffbZZ9q+fbuef/55NW/e3OzS6q1n64oRZHQgBgDAc9yeebghPfPMM0pMTNTrr7/u3JaUlGReQR40sENLzV22S7uz6EAMAICn1PnbtKSkRDt37lRZWZkn63Hx8ccfq2/fvrrhhhsUExOjiy66SK+99prXPq8h9bjgxzl/th6pOuEhAABwn9vBprCwUHfffbfCwsLUrVs3HTp0SJI0efJkzZkzx6PF7du3T6+88oo6dOigL774Qvfdd58mT5583qUbiouLlZub6/LwRYHWAA3q2EqStO2ob9YIAEBj43awmTFjhrZs2aKVK1e6jI4aMWKE3nvvPY8WV15erosvvlizZ8/WRRddpHvvvVf33HOPXnnllRrPSUlJkd1udz4SExM9WpMndUuIlCRtP0awAQDAE9wONh999JHmz5+vK664wmVpha5du2rv3r0eLS4+Pl5du3Z12dalSxdnK1F1ZsyYoZycHOcjIyPDozV5UmWwocUGAADPcLvz8IkTJxQTE1Nle0FBQbVrSNXHgAEDtHPnTpdtu3btUtu2bWs8x2azyWazebQOb+kaXxFsfjiWqzJHuQLpQAwAQL24/U16ySWX6JNPPnG+rgwzr732mvr37++5yiQ98MAD+uabbzR79mzt2bNH77zzjl599VVNnDjRo59jlqQW4QoLtqq4rFz7TxaYXQ4AAI2e2y02KSkpGj16tLZv366ysjLNmzdP27Zt07p167Rq1SqPFnfJJZfoww8/1IwZM/TUU08pOTlZL7zwgm699VaPfo5ZAgIs6hIfqbSD2dp2NFcdWFoBAIB6cbvF5vLLL9fXX3+twsJCtWvXTkuXLlVsbKzWrVunPn36eLzAq6++Wunp6SoqKtKOHTt0zz33ePwzzEQHYgAAPKdOE/T16NFDb7zxhqdraZJ+7EDMXDYAANSX2y02n376qb744osq27/44gt99tlnHimqKekaXzFR37ajuTIMw+RqAABo3NwONtOnT5fD4aiy3TAMTZ8+3SNFNSWd4iIUGmTVmcJS7cnKN7scAAAaNbeDze7du6vMLSNJnTt31p49ezxSVFMSHBigTnEVnYZ/yGRBTAAA6sPtYGO327Vv374q2/fs2aPw8HCPFNXUdImvCDY7CTYAANSL28Hmmmuu0dSpU11mGd6zZ48efPBBXXPNNR4trqnomlDRz2bjoWyTKwEAoHFzO9g899xzCg8PV+fOnZWcnKzk5GR16dJFLVq00N/+9jdv1Oj3BrRrIUnacCBbZ0uq9l8CAAC14/Zwb7vdrrVr12rZsmXasmWLQkND1bNnTw0aNMgb9TUJyS3DFRtp0/HcYqUfydGlydFmlwQAQKNUp3lsLBaLRo4cqZEjR3q6nibJYrHo4jZR+mxrptIOZhNsAACoozoFmxUrVmjFihXKyspSeXm5y77/+7//80hhTU2ftj8GGwAAUDduB5snn3xSTz31lPr27av4+HiPr+jdVPVpGyVJ+m7/KZU6yhXESt8AALjN7WCzYMECLVy4ULfffrs36mmyerZuLknKLSrTzsw8db/AbnJFAAA0Pm43C5SUlOjyyy/3Ri1NmjXgx5avT9OPmVgJAACNl9vB5ne/+53eeecdb9TS5CW1CJMkfbjpiMmVAADQOLl9K6qoqEivvvqqli9frp49eyooKMhl/9y5cz1WXFPz2/5Jeuq/23Usp0iGYdB/CQAAN7kdbL7//nv17t1bkrR161aXfXwR18/Nl7bRU//dLqlitW/62QAA4B63g01qaqo36oCk0GCr8/k3+04RbAAAcBNjin3MI6M7S5LWHzhtciUAADQ+dZqgb/369Xr//fd16NAhlZSUuOxbsmSJRwprqi5NrpjPZv2BbJWXGwoI4PYeAAC15XaLzaJFizRgwABt375dH374oUpLS7V9+3Z9+eWXstu5dVJfPVs3V2iQVacLSrTzeJ7Z5QAA0Ki4HWxmz56tv//97/rvf/+r4OBgzZs3Tzt27NCNN96oNm3aeKPGJiXIGqB+F1asFbVq1wmTqwEAoHFxO9js3btXY8eOlSTZbDYVFBTIYrHogQce0KuvvurxApuiIZ1iJEkrd2aZXAkAAI2L28EmOjpaeXkVt0guuOAC55DvM2fOqLCw0LPVNVEDO7SUJG08eEZnSxwmVwMAQOPhdrAZOHCgli1bJkm68cYbNWXKFN1zzz26+eabNXz4cI8X2BQltwzXBc1DVeIo13eMjgIAoNbcHhU1f/58FRUVSZJmzJihoKAgrVmzRtddd51mzpzp8QKbIovFogHtW+hfGw7r6z0nNbhjK7NLAgCgUbAYhmGYXYQ35ebmym63KycnR5GRkWaXU2v/3nxEUxZtVsfYZlr6wGCzywEAoEHV9fvb7VtRVqtVWVlVO7WeOnVKVqu1mjNQF5e3q+hns+t4vo7lnDW5GgAAGge3g01NDTzFxcUKDg6ud0Go0CrC5ny+YgejowAAqI1a97F58cUXJVX0//jf//1fNWvWzLnP4XBo9erV6ty5s+crbMImDm2nl1L3auPBbN3Wr63Z5QAA4PNqHWz+/ve/S6posVmwYIHLbafg4GAlJSVpwYIFnq+wCRvQvqVeSt2rJZuO6NnreyrQytJeAACcT62Dzf79+yVJQ4cO1ZIlSxQVFeW1olDh0qRo5/O0g9m67MIWJlYDAIDvc7sJIDU11SXUOBwObd68WdnZ2R4tDFKgNUC9EptLkpZuP25yNQAA+D63g83UqVP1z3/+U1JFqBk0aJAuvvhiJSYmauXKlZ6ur8kbcm4Omw1M1AcAwC9yO9i8//776tWrlyTpP//5jw4cOKAffvhBU6dO1WOPPebxApu62/q1lcUibTmco+O5RWaXAwCAT3M72Jw6dUpxcXGSpE8//VQ33HCDOnZTLFsvAAAgAElEQVTsqLvvvlvp6ekeL7CpaxVhU+9zt6OW7+B2FAAA5+N2sImNjdX27dvlcDj0+eefa8SIEZKkwsJCJujzkhFdYiVJy+lnAwDAebkdbCZMmKAbb7xR3bt3l8Vi0ZVXXilJ+vbbb5nHxkuu7FoRbL7ee0oFxWUmVwMAgO9yexHMWbNmqXv37srIyNANN9wgm61ihlyr1arp06d7vEBIHWKaqU10mA6dLtRXu09qdPc4s0sCAMAnuR1sJOn666+vsu2OO+6odzGonsVi0Ygusfq/r/dr+Y7jBBsAAGpQq2Dz4osv6ve//71CQkKcSyvUZPLkyR4pDK6u7FoRbL78IUuOckPWAIvZJQEA4HMsRk2rWv5EcnKyNmzYoBYtWig5ObnmN7NYtG/fPo8WWF91Xfbc15Q6ytX36eXKOVuq9+/rr0t+MisxAAD+pq7f37VqsalcTuHnz9FwgqwBGtqplT7afFTLtx8n2AAAUI1GtapiSkqKLBaLpk6danYpphhxbnTUMuazAQCgWrVqsZk2bVqt33Du3Ll1LuZ81q9fr1dffVU9e/b0yvs3BoM7tlKQ1aJ9Jwq090S+2rVqZnZJAAD4lFoFm02bNrm8TktLk8PhUKdOnSRJu3btktVqVZ8+fTxfoaT8/Hzdeuuteu211/T000975TMag4iQIPVs3VxpB7P1h7fStPSBwWaXBACAT6lVsElNTXU+nzt3riIiIvTGG284V/nOzs7WhAkTNHDgQK8UOXHiRI0dO1YjRoz4xWBTXFys4uJi5+vc3Fyv1GSWq3rEK+1gtnYdz1eZo1yB1kZ1NxEAAK9y+1vx+eefV0pKijPUSFJUVJSefvppPf/88x4tTpIWLVqkjRs3KiUlpVbHp6SkyG63Ox+JiYker8lMN13y48+zfEeWiZUAAOB73A42ubm5On68aufVrKws5eXleaSoShkZGZoyZYreeusthYSE1OqcGTNmKCcnx/nIyMjwaE1ma2YL1N1XVAy5v++tNJOrAQDAt7gdbH79619rwoQJ+uCDD3T48GEdPnxYH3zwge6++25dd911Hi0uLS1NWVlZ6tOnjwIDAxUYGKhVq1bpxRdfVGBgoBwOR5VzbDabIiMjXR7+ZvxPWm02Z5wxsRIAAHyL20sqLFiwQA899JBuu+02lZaWVrxJYKDuvvtuPffccx4tbvjw4UpPT3fZNmHCBHXu3FmPPPJIk11NvENshPP5ih3H1TuxuYnVAADgO2o183B1CgoKtHfvXhmGofbt2ys8PNzTtVVryJAh6t27t1544YVaHe8vMw//3IebDuuB97ZIkvanXCWLhSUWAAD+w6szD1cnPDy8Sc8pY7ZhnWOdzzcczGYmYgAAVI9gY5aVK1eaXYJPsIcGOZ/f9D/rtC9lrInVAADgG5gEpRF7YlxXSVK5IZWUlZtcDQAA5iPYNGK392vrfL6c9aMAACDYNGaB1gBNGtpekvT2twdNrgYAAPMRbBq58Zcmyhpg0dd7Tin9cI7Z5QAAYCqCTSPXOipM1/RKkCS9lLrH5GoAADAXwcYP/HFIO0nS59sytfu4Z5e1AACgMSHY+IEOsREa1a1iXptXVu41uRoAAMxDsPETE891Iv73lqPKOF1ocjUAAJiDYOMnerZuroEdWspRbuh/VtNqAwBomgg2fqSy1eZfGw4rK7fI5GoAAGh4BBs/cllytBKjQ1VSVq5p/9pidjkAADQ4go0fsVgsuqN/kiRpzZ6TKiwpM7cgAAAaGMHGz0wYkKwLmodKkt7fcNjkagAAaFgEGz9jDbDo3sEXSpJm/WebyssNkysCAKDhEGz80I19EyVJhiG9sGK3ydUAANBwCDZ+KCTIqrjIEEnS/1t3wNRaAABoSAQbP/XhxMslSdmFpdqcccbkagAAaBgEGz8Vbw9V66iKTsTXvvQ1fW0AAE0CwcaPLf7D5c7nS7dnmlgJAAANg2Djx2IjQzSsc4wk6b63Nqq4zGFyRQAAeBfBxs89f0Mv5/ORf19tYiUAAHgfwcbPRYUH6/5hFWtIHTxVqPUHTptcEQAA3kOwaQKmXdnR+fyGBevoSAwA8FsEmybAYrHou8eGO1//+eOtJlYDAID3EGyaiJiIEOfzt745RKsNAMAvEWyakO9njVRESKAk6f5Fm0yuBgAAzyPYNCGRIUEa1S1OkvTJ98d0PLfI5IoAAPAsgk0T88xvejqfz/yIvjYAAP9CsGlirAEWvf27yyRJS7cf187MPJMrAgDAcwg2TdCA9i2dz//66Q4ZBh2JAQD+gWDTRH34x4p1pFbvOqHFG4+YXA0AAJ5BsGmiLmoTpbsGJEuSHnp/i0rKyk2uCACA+iPYNGFTRnRwPu/+xBcmVgIAgGcQbJowe2iQJp9bR6rEUa7UnVkmVwQAQP0QbJq4aSM7OZ9PeH29jpw5a2I1AADUD8EGSp810vl8wJwvVeqgvw0AoHEi2EARIUF6/77+ztcdHvuMIeAAgEaJYANJ0iVJ0bqtXxvn6/lf7jGxGgAA6oZgA6e//Kq72rYIkyTNXb5LO47lmlwRAADuIdjAyWKxaOVDQzSya6wMQxoz7yvCDQCgUSHYwIXFYtHTv+7ufD1m3lcqKnWYWBEAALVHsEEVMREhmn/LRc7XnWd+rqMMAwcANAI+HWxSUlJ0ySWXKCIiQjExMbr22mu1c+dOs8tqEq7umaCb+iY6X18+50sVlpSZWBEAAL/Mp4PNqlWrNHHiRH3zzTdatmyZysrKNHLkSBUUFJhdWpPwzPU9NbZnvPN1z1lLVcYcNwAAH2YxGtGEJSdOnFBMTIxWrVqlQYMG1eqc3Nxc2e125eTkKDIy0ssV+qe7F67Xih8qlltIahGmlX8aanJFAAB/V9fvb59usfm5nJwcSVJ0dHSNxxQXFys3N9flgfr5552X6Lf920qSDpwqVNL0T5RXVGpyVQAAVNVogo1hGJo2bZquuOIKde/evcbjUlJSZLfbnY/ExMQaj0XtPXlNNyVGhzpf95i1VGkHs02sCACAqhrNraiJEyfqk08+0Zo1a9S6desajysuLlZxcbHzdW5urhITE7kV5SEpn+7Q/6ze53z9xdRB6hQXYWJFAAB/5Ne3ou6//359/PHHSk1NPW+okSSbzabIyEiXBzxnxlVddNeAZOfrUS+sZp4bAIDP8OlgYxiGJk2apCVLlujLL79UcnLyL58Er/vzuK56fGwX5+trX/qaRTMBAD7Bp4PNxIkT9dZbb+mdd95RRESEMjMzlZmZqbNnmSzObL8beKFaNrNJkn7IzFPyjE9NrggAAB/vY2OxWKrd/vrrr+vOO++s1Xsw3Nu7ftrn5tKkaP3rvv4mVwQA8Ad+2cfGMIxqH7UNNfC+6WM6O59/d+C0Hnp/i4nVAACaOp8ONvB9FotFu54e43z9QdphJU3/RKXMUAwAMAHBBvUWHBjgEm4kqcNjnylp+icqKSPgAAAaDsEGHhEcGKADc8ZW2d7x8c90toTh4ACAhkGwgUcdmDNW797Tz2Vblz9/rn9vPmJSRQCApoRgA4/r366FDswZqxv7/jiZ4pRFm7k1BQDwOoINvObZ63vp5kvbuGzr+Phnyi8uM6kiAIC/8+l5bDyBeWzMV1zmUKfHP6+y/dnre+rGvixSCgCoyi/nsYF/sAVadWDOWD19reuq7A9/8L0GPvulSVUBAPwRwQYN5rZ+bTXz6q4u2zJOn1XS9E+0MzPPpKoAAP6EW1FocIUlZTpdUKIrnkmtsm/hhEs0pFOMCVUBAHwJt6LQaIQFB6p1VJj2p1xVZd+dr6/XB2mHTagKAOAPCDYwjcVi0YE5Y/WfSVe4bH/o/S36x4rdJlUFAGjMuBUFn7El44x+9dLXVba/9/t+uuzCFiZUBAAwC7ei0Oj1SmyubU+OqrL9ple/Ud+nl2v3cToYAwDOjxYb+Jw9WXkaMXf1eY/5dPJAdU3g3xMA/FVdv78JNvBp5eWGrn35a31/OKfGY9655zJd3q5lA1YFAPA2gk0NCDb+If1wjsbNX1OrY4d2aqXhXWJ1y6VtFBBg8XJlAABvINjUgGDjf3Ycy9WYeV/V+vitT45SM1ugFysCAHgawaYGBBv/lpVXpLe/OaR5vzA8/MCcsQ1UEQDAE+r6/c3/xqJRi4kI0QNXdtRdA5J14FSBIkODNPRvK6sc98W2TI3sGiuLhVtTAODPaLGBX3v0w3S98+2havfdMzBZj43tWu0+AIC5uBVVA4JN01ZQXKbrXl6rnbWYA4fbVQDgOwg2NSDYoNRRrv9sOapp/9pSq+ODrBY9fW13PbI4XZK07clRCqfzMQA0KIJNDQg2qM7x3CJdNntFnc9f9sAgdYiN8GBFAICfItjUgGCD80k7eFoxESEa+Gxqnc6fPKy9po3sVKdzHeWGth/NVW5RqRKjwtSmRVid3gcA/BHBpgYEG7ijuMyhnMJSpe7MUu/EKE19b7N2HMs97zkLJ1yiIZ1i9PGWo5r87iaXfTOv7qr/W7NfR86clSTteGq0QoOtkqSk6Z+4HLvz6dEyDGnSOxv10KhO6hzHf68Ami6CTQ0INvAER3nFr4k1wKLtR3N11Yu1nyCwtgIDLCor//HX8achCACaGoJNDQg28JaxL36lbUfP35rjKd/PGqkIW6DH5+E5U1ii3k8tkyT1Tmyuf93bX8GBAR79DACoC4JNDQg28KYPNx3WA+/9ONoqfdZIHTxVqBJHuSJsgUr57Af9aVQnXRAVqre+OahnP9/pcv7e2VfpL//droVrD7j1ub++6AJ9t/+08xZXpQ/u66++SdHO10WlDu07UaAOsc309Z6TuvP19XpiXFcVlZbrmc9/qNVn/XQY/OmCEp3KL6bjNACvI9jUgGADX5RfXOayftXBUwV6+9tDunfQhYoKC9aFj35qYnVVbXtylLo98YXLtvm3XKSreya4bDuZX6xP04/pz//eVuU99qdcxczPAGqNYFMDgg0ao2M5Z3X/O5s065pu2p2V59Iq5A0bHh+hvk8vr9O5t/drq4dGdVKvJ5fW6vi0x0eoRTNbnT4LQNNBsKkBwQb+5ofMXI1+4Std1SNOv7m4te5+Y4Pi7SE6llP0i+fe1DdR723I0IguMZp7U2/ZAgMUbA2otiUlv7hM3X/WSuMpXz08VInRYTqVX6w+5wLVd48OV3R4sKwBFlp2ABBsakKwQVOyJytf1gCLkluGe+T9Xli+Sy8s/3Hl9H2zr1JAgEXJMz5RTX85IkICteQPl7v0w5m++HstWp9R68+dN763ftX7gjrXXV8HTxUoyBqghOahtT7n863H1KKZTZckRavUUa4gK52wgfog2NSAYAN4z6ZD2fr1y2slSQtu66PR3ePOe/zP5+6pjY6xzbT0gcF1qu/nsgtKdNFfllW77/5h7fWr3gkaMXe1y/atT45SM1ugDMPQwVOFirOHaO+JfCW3DFdIoFWGpGc+/0Gvrt7nct7yaYPVPqZZvWsuLzdUVm4oODBAGacLtXT7cd18aaK+2n1S9/6/NH318FBtPZKjK7vG+n1rl2EYLj9fmaNcT3+yo0rn+39PHKBeic1r/b6FJWX6Ylum+l3YQvH2ijBbfm7qhYAA/72evo5gUwOCDeA7DMNQ8gzXjtHWAItznqDzqesipYZhKL+4TIOeTVV2YWmd3qM+vn10uGIjQ+p07oodx3X3GxvcOufzqQOrTO7oKDfU5c+fq6Ss3LmtV2u7brwkUUM7xbi0TFV+Jfw0QDjKDb2cukffHTitp37VXSFBAbIGWLRu7ynFRoao34UtVFJWXqupArYdzdHYF9do/i0XaVjnGOUXlel4brHGzV8jSRrbI16zr+uhT9OPacaSdLd+9p9bO31YlVa3J/69VW+sO+jW+9wzMFmPje1ar1rgPoJNDQg2gG8pc5Rrx7E8tYsJV1hw1cVFn/viB72UuvcX38caYNEfh7TTP77c47J9bM94Tbuyoy5sGa6tR3KdX5hmmzS0vR4c2bFWLSpljnK1f+yzen3eX37VTbde1lbvp2U4F3Q9n8V/6K+QIKvGvlj/67X7r2NktVj0zneH9PhHW+v9fr6iugVx3/72oLrER+riNlHObftO5GvY86vUOipU7/yun4ICLXr6vzv0Sfoxl3NXPDhYbaLDtO9EgTrFRai4zKFga4BmLEnXtqO5eut3l8keGtQgP5svItjUgGADNF77TxZo6N9WevQ9L0mK0hPjuqn7BXbntjW7T+q2f37rfL3lzyMVZrPq3e8OVTt0vTrX92mtv93Qq+Iz/rpcJ/KKazy2placuxau15c/ZJ33c0KDrDpb6pAktY4K1eHss+c9vin47tHhijl3Pfdk5WvE3FVe+6zJwzvoxRW7q913RfuWWrPnpNc+u9IjoztrwoAkhQS5PzP5vhP5mr44XX2TotSymU2to0J12YUt1MwWKOu5227vb8jQnz74XpKU1CJM+cUOncx3/e/5h7+MlqPc0Im8YiV5qE/fzxFsakCwARo3T31RTRiQpCnDO6h5WLBb5znKDbV79FNdmhStRb/v59LnorCkTMWl5YoKr/qepY5ydahnq4tU8QVS2y+wiW9vrNIq8FNbnqicwVr6YttxLVp/SCt3nqjVe4/pHqd+F7bQEx/XLuidz7v39NNzX/yghOahOnS6UKUOQwsnXKKosGB1fPwzJbcM1zW9EvSbi1urTYuwKn1rDMPQ4eyzsgUFKCai+tt8D72/RR+kHa523/anRiksOLDK+1bHU/+O3vToVZ312/5JyswpUqsIm0urUpmjXF/tPqm9J/L19Cc7vFbDS7dcrLE94z36ngSbGhBsAP+QW1Qqq8Uia4BF7353SFd2jdUf396oEV1iNXl4B6UdPK2Faw9q34l8l6Uunr2+p27sm2hi5XULZxaLtD+lbv2KfvXS19qSccb5+v/u7KthnWNrPH7e8t36+/JdkqSre8Zr/i0XS5Iyc4r03++P6tbL2tZq3bLycsNlcsmHR3fS3VckyxZo7ppnhmFo+7FcdUuw//LB1SgpK1fHx90LNx9PGiBrgEVjX1yjIZ1a6Ve9E3Rl1zi9unqfpg7voBJHueat2K1XVu5Vr9Z2bTmc4zz3uet76lhOkeYu21Wnes3w/A299Js+rT36ngSbGhBsAPiSXxoZtuWJkU26X4Wv+mm/p29mDFec3bWlqPLf9bMpA9Ul3vPfNXtP5Cs0yKp4e4jeTzush8/dKqqLpQ8M0saD2eocH6kVO45r/YHT+mbfaZdj9qdcJUmau2yX2sc0c5l+4af/DcfbKzqPP3d9TwV6eIoDgk0NCDYAfFluUanCgqx+P1Qb3pNxulADn01V9wsitfVI1YV5P5syUKFBVsVE2qrtsO+r6vr93Xh+QgDwQ5EhtM6gfhKjw+o8HYI/ahRTY7788stKTk5WSEiI+vTpo6+++srskgAAgA/y+WDz3nvvaerUqXrssce0adMmDRw4UGPGjNGhQ4fMLg0AAPgYn+9jc9lll+niiy/WK6+84tzWpUsXXXvttUpJSfnF8+ljAwBA41PX72+fbrEpKSlRWlqaRo4c6bJ95MiRWrt2bbXnFBcXKzc31+UBAACaBp8ONidPnpTD4VBsrOv8C7GxscrMzKz2nJSUFNntducjMdHc+SsAAEDD8elgU+nnQyDPN1vkjBkzlJOT43xkZGQ0RIkAAMAH+PRw75YtW8pqtVZpncnKyqrSilPJZrPJZrM1RHkAAMDH+HSLTXBwsPr06aNly5a5bF+2bJkuv/xyk6oCAAC+yqdbbCRp2rRpuv3229W3b1/1799fr776qg4dOqT77rvP7NIAAICP8flgc9NNN+nUqVN66qmndOzYMXXv3l2ffvqp2rZta3ZpAADAx/j8PDb1xTw2AAA0Pn45jw0AAIA7CDYAAMBvEGwAAIDf8PnOw/VV2YWIpRUAAGg8Kr+33e0K7PfBJi8vT5JYWgEAgEYoLy9Pdru91sf7/aio8vJyHT16VBERETUuw1AXubm5SkxMVEZGBqOtvIxr3TC4zg2D69wwuM4Nw5vX2TAM5eXlKSEhQQEBte854/ctNgEBAWrdurXX3j8yMpJfmgbCtW4YXOeGwXVuGFznhuGt6+xOS00lOg8DAAC/QbABAAB+wzpr1qxZZhfRWFmtVg0ZMkSBgX5/R890XOuGwXVuGFznhsF1bhi+dp39vvMwAABoOrgVBQAA/AbBBgAA+A2CDQAA8BsEGwAA4DcINnX08ssvKzk5WSEhIerTp4+++uors0vyGatXr9a4ceOUkJAgi8Wijz76yGW/YRiaNWuWEhISFBoaqiFDhmjbtm0ux2RnZ+v222+X3W6X3W7X7bffrjNnzrgck56ersGDBys0NFQXXHCBnnrqqSpriixevFhdu3aVzWZT165d9eGHH3rnhzZBSkqKLrnkEkVERCgmJkbXXnutdu7c6XJMcXGx7r//frVs2VLh4eG65pprdPjwYZdjDh06pHHjxik8PFwtW7bU5MmTVVJS4nLMqlWr1KdPH4WEhOjCCy/UggULqtTjr78Tr7zyinr27OmcgKx///767LPPnPu5xt6RkpIii8WiqVOnOrdxretv1qxZslgsLo+4uDjnfr/4+2zAbYsWLTKCgoKM1157zdi+fbsxZcoUIzw83Dh48KDZpfmETz/91HjssceMxYsXG5KMDz/80GX/nDlzjIiICGPx4sVGenq6cdNNNxnx8fFGbm6u85jRo0cb3bt3N9auXWusXbvW6N69u3H11Vc79+fk5BixsbHG+PHjjfT0dGPx4sVGRESE8be//c15zNq1aw2r1WrMnj3b2LFjhzF79mwjMDDQ+Oabb7x/ERrAqFGjjNdff93YunWrsXnzZmPs2LFGmzZtjPz8fOcx9913n3HBBRcYy5YtMzZu3GgMHTrU6NWrl1FWVmYYhmGUlZUZ3bt3N4YOHWps3LjRWLZsmZGQkGBMmjTJ+R779u0zwsLCjClTphjbt283XnvtNSMoKMj44IMPnMf48+/Exx9/bHzyySfGzp07jZ07dxqPPvqoERQUZGzdutUwDK6xN3z33XdGUlKS0bNnT2PKlCnO7Vzr+nviiSeMbt26GceOHXM+srKynPv94e8zwaYOLr30UuO+++5z2da5c2dj+vTpJlXku34ebMrLy424uDhjzpw5zm1FRUWG3W43FixYYBiGYWzfvt2Q5PIf+Lp16wxJxg8//GAYhmG8/PLLht1uN4qKipzHpKSkGAkJCUZ5eblhGIZx4403GqNHj3apZ9SoUcb48eM9/4P6gKysLEOSsWrVKsMwDOPMmTNGUFCQsWjRIucxR44cMQICAozPP//cMIyKEBoQEGAcOXLEecy7775r2Gw2IycnxzAMw3j44YeNzp07u3zWvffea/Tr18/5uqn9TkRFRRn/+7//yzX2gry8PKNDhw7GsmXLjMGDBzuDDdfaM5544gmjV69e1e7zl7/P3IpyU0lJidLS0jRy5EiX7SNHjtTatWtNqqrx2L9/vzIzM12un81m0+DBg53Xb926dbLb7brsssucx/Tr1092u93lmMGDB8tmszmPGTVqlI4ePaoDBw44j/n5v9OoUaP89t8pJydHkhQdHS1JSktLU2lpqcs1SEhIUPfu3V2uY/fu3ZWQkOA8ZtSoUSouLlZaWprzmOqu44YNG1RaWtqkficcDocWLVqkgoIC9e/fn2vsBRMnTtTYsWM1YsQIl+1ca8/ZvXu3EhISlJycrPHjx2vfvn2S/Ofvs29ME9iInDx5Ug6HQ7GxsS7bY2NjlZmZaVJVjUflNaru+h08eNB5TExMTJVzY2JinOdnZmYqKSmpyntU7ktOTlZmZmaT+XcyDEPTpk3TFVdcoe7du0uquA7BwcGKiopyOfan16C6axQVFaXg4ODzHhMbG6uysjKdPHlShmH4/e9Eenq6+vfvr6KiIjVr1kwffvihunbtqs2bN3ONPWjRokXauHGj1q9fX2Uf/z17xmWXXaY333xTHTt21PHjx/X000/r8ssv17Zt2/zm7zPBpo4sFovLa8MwqmxDzX7p+lV3LX/pGONcx7RfOsYf/50mTZqk77//XmvWrPnFYz19rau77tW9R2PWqVMnbd68WWfOnNHixYt1xx13aNWqVTUezzV2X0ZGhqZMmaKlS5cqJCSk1udxrd0zZswY5/MePXqof//+ateund544w3169dPUuP/+8ytKDe1bNlSVqu1SqrMysqqkj5RVWXv+/Ndv7i4OB0/frzKuSdOnHA5prr3kPSLx/jbv9P999+vjz/+WKmpqWrdurVze1xcnEpKSpSdne1y/M+v9c+vUXZ2tkpLS3/xOgYGBqpFixZN4nciODhY7du3V9++fZWSkqJevXpp3rx5XGMPSktLU1ZWlvr06aPAwEAFBgZq1apVevHFFxUYGKjY2FiutReEh4erR48e2r17t9/8fSbYuCk4OFh9+vTRsmXLXLYvW7ZMl19+uUlVNR7JycmKi4tzuX4lJSVatWqV8/r1799fOTk5+u6775zHfPvtt8rJyXE5ZvXq1S7DOJcuXaqEhARnE2j//v2r/DstXbrUb/6dDMPQpEmTtGTJEn355ZdKTk522d+nTx8FBQW5XINjx45p69atLtdx69atOnbsmPOYpUuXymazqU+fPs5jqruOffv2VVBQUJP8nTAMQ8XFxVxjDxo+fLjS09O1efNm56Nv37669dZbnc+51p5XXFysHTt2KD4+3n/+Pte7+3ETVDkU8J///Kexfft2Y+rUqUZ4eLhx4MABs0vzCXl5ecamTZuMTZs2GZKMuXPnGps2bXIOlZwzZ45ht9uNJUuWGOnp6cbNN99c7XDCnj17GuvWrTPWrVtn9OjRw2U44ZkzZ4zY2Fjj5ptvNtLT040lS5YYkZGRLsMJv/76a8NqtRpz5swxduzYYcyZM8evhnv/4ePbCOkAAAafSURBVA9/MOx2u7Fy5UqXoZuFhYXOY+677z6jdevWxvLly42NGzcaw4YNq3Z47PDhw42NGzcay5cvN1q3bl3t8NgHHnjA2L59u/HPf/6zxuGx/vg7MWPGDGP16tXG/v37je+//9549NFHjYCAAGPp0qWGYXCNvemno6IMg2vtCQ8++KCxcuVKY9++fcY333xjXH311UZERITzZ/OHv88Emzp66aWXjLZt2xrBwcHGxRdf7BxiC8NITU01JFV53HHHHYZhVAwpfOKJJ4y4uDjDZrMZgwYNMtLT013e49SpU8att95qREREGBEREcatt95qZGdnuxzz/fffGwMHDjRsNpsRFxdnzJo1yzmUsNL7779vdOrUyQgKCjI6d+5sLF682Ks/e0Oq7hpLMl5//XXnMWfPnjUmTZpkREdHG6GhocbVV19tHDp0yOV9Dh48aIwdO9YIDQ01oqOjjUmTJrkM0zQMw1i5cqVx0UUXGcHBwUZSUpLxyiuvVKnHX38n7rrrLufP1apVK2P48OHOUGMYXGNv+nmw4VrXX+W8NEFBQUZCQoJx3XXXGdu2bXPu94e/zxbD+NlUgAAAAI0UfWwAAIDfINgAAAC/QbABAAB+g2ADAAD8BsEGAAD4DYINAADwGwQbAADgNwg2ADxqyJAhmjp1qtlluLBYLProo4/MLgNAA2CCPgAedfr0aQUFBSkiIkJJSUmaOnVqgwWdWbNm6aOPPtLmzZtdtmdmZioqKko2m61B6gBgnkCzCwDgX6Kjoz3+niUlJQoODq7z+ZWrFgPwf9yKAuBRlbeihgwZooMHD+qBBx6QxWKRxWJxHrN27VoNGjRIoaGhSkxM1OTJk1VQUODcn5SUpKefflp33nmn7Ha77rnnHknSI488oo4dOyosLEwXXnihZs6cqdLSUknSwoUL9eSTT2rLli3Oz1u4cKGkqrei0tPTNWzYMIWGhqpFixb6/e9/r/z8fOf+O++8U9dee63+9re/KT4+Xi1atNDEiROdnwXAdxFsAHjFkiVL1Lp1az311FM6duyYjh07JqkiVIwaNUrXXXedvv/+e7333ntas2aNJk2a5HL+c889p+7duystLU0zZ86UJEVERGjhwoXavn275s2bp9dee01///vfJUk33XSTHnzwQXXr1s35eTfddFOVugoLCzV69GhFRUVp/fr1ev/997V8+fIqn5+amqq9e/cqNTVVb7zxhhYuXOgMSgB8F7eiAHhFdHS0rFarIiIiXG4FPffcc7rllluc/W46dOigF198UYMHD9Yrr7yikJAQSdKwYcP00EMPubzn448/7nyelJSkBx98UO+9954efvhhhYaGqlmzZgoMDDzvrae3335bZ8+e1Ztvvqnw8HBJ0vz58zVu3Dg988wzio2NlSRFRUVp/vz5slqt6ty5s8aOHasVK1Y4W48A+CaCDYAGlZaWpj179ujtt992bjMMQ+Xl5dq/f7+6dOny/9u5Y5fUwjgO41+SQhpscnDRIHEwKAubxCEacmuoQQhpdEgD52jIXQjpH3DKQRDcmkJBaxRKIiyyoE2aahDy2HC5h7pCF+Hmvffl+UzyHs/70+3Bc46SpHA4PHRuqVTS0dGRbm9v9fLyore3N7lcrpHmX19fa3Fx0Y4aSYpEIrIsSzc3N3bYzM/Py+Fw2O/xeDy6vLwcaRaA8SNsAIyVZVlKJpPa29sbOub1eu3XH8NDki4uLhSPx3V4eKj19XXNzMyoWCwql8uNNH8wGHy63+ejj+uTk5NDxyzLGmkWgPEjbAB8m6mpKfX7/U9ry8vLarVa8vv9I+1Vr9fl8/m0v79vrz08PPx23q+CwaAKhYJeX1/teKrX65qYmFAgEBjpMwH493DzMIBvMzs7q1qtpqenJ3W7XUk/nmw6Pz/X7u6ums2m2u22KpWK0un0l3v5/X49Pj6qWCzq7u5O+Xxe5XJ5aN79/b2azaa63a56vd7QPtvb23I6ndrZ2dHV1ZXOzs6UTqeVSCTsy1AA/l+EDYBvk81m1el0NDc3J7fbLUlaWFhQtVpVu91WNBrV0tKSDg4O5PF4vtxrY2NDmUxGqVRKoVBIjUbDflrqp83NTcViMa2ursrtduvk5GRon+npaZ2enur5+VkrKyva2trS2tqajo+P/9wXB/DX8M/DAADAGPxiAwAAjEHYAAAAYxA2AADAGIQNAAAwBmEDAACMQdgAAABjEDYAAMAYhA0AADAGYQMAAIxB2AAAAGMQNgAAwBiEDQAAMMY7iG9PT9qsEpcAAAAASUVORK5CYII=", "text/plain": [ "Figure(PyObject