{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![logo](https://www.tensorflow.org/_static/images/tensorflow/logo.png)\n",
    "## _An open-source software library for Machine Intelligence_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](https://www.tensorflow.org/skelimages/partner-logos/google.192.png) ![](https://www.tensorflow.org/skelimages/partner-logos/deepmind.192.png) ![](https://www.tensorflow.org/skelimages/partner-logos/nvidia.png) ![](https://www.tensorflow.org/skelimages/partner-logos/intel.192.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# How to start\n",
    "## Requirements (CUDA)\n",
    "* NVIDIA drivers\n",
    "* CUDA® Toolkit 8.0\n",
    "* NVIDIA CUDA® Deep Neural Network library (cuDNN) v6.0\n",
    "* GPU card with CUDA Compute Capability 3.0 or higher.\n",
    "* libcupti-dev (NVIDIA CUDA Profile Tools Interface)\n",
    "## Installation (virtualenv)\n",
    "```\n",
    "$ virtualenv --system-site-packages -p python3 tensorflow\n",
    "$ source tensorflow/bin/activate\n",
    "(tensorflow)$ easy_install -U pip\n",
    "(tensorflow)$ pip3 install --upgrade tensorflow-gpu\n",
    "(tensorflow)$ pip3 install ipython[notebook]\n",
    "(tensorflow)$ ipython notebook\n",
    "ssh name@server -L num:127.0.0.1:num\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# miracle memory saver\n",
    "import os\n",
    "os.environ[\"CUDA_VISIBLE_DEVICES\"]=\"3\"\n",
    "config = tf.ConfigProto()\n",
    "config.gpu_options.allow_growth=True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "sess = tf.Session(config=config)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b'Hello, TensorFlow!'\n"
     ]
    }
   ],
   "source": [
    "hello = tf.constant('Hello, TensorFlow!')\n",
    "sess = tf.Session()\n",
    "print(sess.run(hello))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import random as ran\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tensors\n",
    "```python\n",
    "3 # a rank 0 tensor; a scalar with shape []\n",
    "[1., 2., 3.] # a rank 1 tensor; a vector with shape [3]\n",
    "[[1., 2., 3.], [4., 5., 6.]] # a rank 2 tensor; a matrix with shape [2, 3]\n",
    "[[[1., 2., 3.]], [[7., 8., 9.]]] # a rank 3 tensor with shape [2, 1, 3]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Constants"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"Const_1:0\", shape=(), dtype=float32) Tensor(\"Const_2:0\", shape=(), dtype=float32)\n",
      "[3.0, 4.0]\n"
     ]
    }
   ],
   "source": [
    "node1 = tf.constant(3.0, dtype=tf.float32)\n",
    "node2 = tf.constant(4.0) # also tf.float32 implicitly\n",
    "print(node1, node2)\n",
    "print(sess.run([node1, node2]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Functions\n",
    "![](https://www.tensorflow.org/images/getting_started_add.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "node3: Tensor(\"Add:0\", shape=(), dtype=float32)\n",
      "sess.run(node3): 7.0\n"
     ]
    }
   ],
   "source": [
    "from __future__ import print_function\n",
    "node3 = tf.add(node1, node2)\n",
    "print(\"node3:\", node3)\n",
    "print(\"sess.run(node3):\", sess.run(node3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Parameters\n",
    "![](https://www.tensorflow.org/images/getting_started_adder.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7.5\n",
      "[ 3.  7.]\n"
     ]
    }
   ],
   "source": [
    "a = tf.placeholder(tf.float32)\n",
    "b = tf.placeholder(tf.float32)\n",
    "adder_node = a + b  # + provides a shortcut for tf.add(a, b)\n",
    "print(sess.run(adder_node, {a: 3, b: 4.5}))\n",
    "print(sess.run(adder_node, {a: [1, 3], b: [2, 4]}))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Flows\n",
    "![](https://www.tensorflow.org/images/getting_started_triple.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "22.5\n"
     ]
    }
   ],
   "source": [
    "add_and_triple = adder_node * 3.\n",
    "print(sess.run(add_and_triple, {a: 3, b: 4.5}))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "W = tf.Variable([.3], dtype=tf.float32)\n",
    "b = tf.Variable([-.3], dtype=tf.float32)\n",
    "x = tf.placeholder(tf.float32)\n",
    "linear_model = W*x + b\n",
    "\n",
    "init = tf.global_variables_initializer()\n",
    "sess.run(init)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.          0.30000001  0.60000002  0.90000004]\n"
     ]
    }
   ],
   "source": [
    "print(sess.run(linear_model, {x: [1, 2, 3, 4]}))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Simple linear regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAEACAYAAABVtcpZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAEvFJREFUeJzt3H+wpXVdwPH3B3ZxUtJQYmX4pY0lxWQIhYyYnjRj2dS1Rg1qRnAbK0utxhpR3Haxmi7MaLayTPGjSU1bGxoLUItl4TRjyQ8DBBFwDaXFAGf4oaIN3WE//XHO3Q7Xc+499zznnOfX+zVz554f33me7zPP7v3e93nOuZGZSJLa6aCyJyBJKo+LgCS1mIuAJLWYi4AktZiLgCS1mIuAJLXYVBaBiNgYEXdHxFci4t1Dnj8kInZFxN6I+HxEHDuN/UqSiim8CETEQcBFwOnACcBZEXH8smG/DjySmT8KfAi4sOh+JUnFTaMETgH2ZuZ9mbkI7AI2LxuzGfhI//YVwKumsF9JUkHTWASOAvYN3L+//9jQMZn5JPBYRDx7CvuWJBVQ1oXhKGm/kqQB66awjW8Agxd6j+4/Nuh+4BjgvyPiYOCZmfnIsI1FhH/MSJLWKDMn+uV6GiVwM/CCiDguIg4BzgSuXDbmKuDs/u03AtettMH3vz85/PDkssuS/fuTzGZ8bdu2rfQ5eHwen8fXvK8iCi8C2XuN/+3ANcCdwK7MvCsizo+I1/SHXQ4cHhF7gd8Dzl1pm1u3wp49sHMnnHEG7Nu30mhJ0qSm8XIQmfnPwAuXPbZt4PYTwJvWss0XvQhuvBEWFuCkk3rft2yB8GqCJE1NpT8xvH59s6qg0+mUPYWZ8vjqzeNrpyj6etK0RUQOm9PiYq8GduywCiRpUESQE14Yrs0isOT22+Gcc+CII+DSS+GYY+Y3N0mqoiKLQKVfDhpm6VrBaaf1rhVcfjlUbB2TpNqoXQkMsgokqWUlMMgqkKRial0Cg6wCSW3V2hIYZBVI0to1pgQGWQWS2sQSWMYqkKTxNLIEBlkFkprOEliBVSBJozW+BAZZBZKayBIYk1UgSU/VqhIYZBVIagpLYAJWgSS1uAQGWQWS6swSKMgqkNRWlsAyVoGkurEEpsgqkNQmlsAKrAJJdWAJzIhVIKnpLIExWQWSqsoSmAOrQFITWQITsAokVYklMGdWgaSmsAQKsgoklc0SKJFVIKnOCpVARBwGfBI4Dvg68KbM/NaQcU8CXwQCuC8zX7/CNmtVAoOsAkllKLMEzgWuzcwXAtcB7xkx7ruZeVJmvnilBaDurAJJdVO0BO4GXpGZD0XEc4FuZh4/ZNx3MvMHx9xmbUtgkFUgaV7KLIEjMvMhgMx8EDhixLinRcRNEfHvEbG54D5rwSqQVAerlkBE7AY2DD4EJPA+4G8y89kDYx/OzOcM2caRmflARDyf3stGr8zMr43YXyNKYJBVIGmWipTAutUGZOarV9jxQxGxYeDloG+O2MYD/e9fi4gu8GJg6CIAsH379gO3O50OnU5ntWlW2lIVLCz0qmBhAbZsgZjolElqu263S7fbncq2il4TuAB4JDMviIh3A4dl5rnLxvwQ8L3M/N+IOBz4N2BzZt49YpuNK4FBVoGkaSvzmsAFwKsj4h7gVcBCf0InR8Ql/TE/DnwhIm4F9gB/NmoBaAOvFUiqEj8xXCKrQNI0+InhmrIKJJXNEqgIq0DSpCyBBrAKJJXBEqggq0DSWlgCDWMVSJoXS6DirAJJq7EEGswqkDRLlkCNWAWShrEEWsIqkDRtlkBNWQWSllgCLWQVSJoGS6ABrAKp3SyBlrMKJE3KEmgYq0BqH0tAB1gFktbCEmgwq0BqB0tAQ1kFklZjCbSEVSA1lyWgVVkFkoaxBFrIKpCaxRLQmlgFkpZYAi1nFUj1ZwloYlaB1G6WgA6wCqR6sgQ0FVaB1D6WgIayCqT6sAQ0dVaB1A6WgFZlFUjVZglopqwCqbkKLQIR8YaI+FJEPBkRJ60wbmNE3B0RX4mIdxfZp8qxfj1s3Qp79sDOnXDGGbBvX9mzklRU0RK4A/gl4F9HDYiIg4CLgNOBE4CzIuL4gvtVSawCqVkKLQKZeU9m7gVWei3qFGBvZt6XmYvALmBzkf2qXFaB1BzzuCZwFDD4I+L+/mOqueVVcNllVoFUN+tWGxARu4ENgw8BCZyXmVfNYlLbt28/cLvT6dDpdGaxG03BUhVs3tx7B9EVV/gOImnWut0u3W53KtuayltEI+J64F2ZecuQ504Ftmfmxv79c4HMzAtGbMu3iNbU4iIsLMCOHb3vW7ZATPSmNUlrUZW3iI6awM3ACyLiuIg4BDgTuHKK+1VFeK1Aqp+ibxF9fUTsA04Fro6Iz/YfPzIirgbIzCeBtwPXAHcCuzLzrmLTVpX5DiKpPvzEsGbKTxtLs1eVl4Ok72MVSNVmCWhurAJpNiwB1YJVIFWPJaBSWAXS9FgCqh2rQKoGS0ClswqkYiwB1ZpVIJXHElClWAXS2lkCagyrQJovS0CVZRVI47EE1EhWgTR7loBqwSqQRrME1HhWgTQbloBqxyqQnsoSUKtYBdL0WAKqNatAsgTUYlaBVIwloMawCtRWloCEVSBNwhJQI1kFahNLQFrGKpDGYwmo8ZaqYMMGuOQSq0DNYwlIK1iqgpe+1CqQlrME1CpWgZrIEpDGZBVIT2UJqLWsAjWFJSBNwCqQLAEJsApUb6WVQES8ISK+FBFPRsRJK4z7ekR8MSJujYibiuxTmgWrQG1VqAQi4oXAfuCvgD/IzFtGjLsXODkzHx1jm5aASmUVqG5KK4HMvCcz9wKr7TyK7kuaF6tAbTKvH8wJ/EtE3BwRb53TPqWJrV8PW7fCnj2wcyds2gT79pU9K2n61q02ICJ2AxsGH6L3Q/28zLxqzP2clpkPRMQPA7sj4q7M/Nyowdu3bz9wu9Pp0Ol0xtyNNF1LVbCw0KuChQXYsgViovCWpqPb7dLtdqeyram8OygirgfeNeqawLKx24DvZOYHRzzvNQFVktcKVFVV+ZzA0AlExNMj4tD+7WcAvwB8aYr7lebCawVqoqLvDno98GHgcOAx4LbMPCMijgQuzczXRMTzgU/RewlpHfDxzFxYYZuWgCrPKlCVFCkBPywmTWhxsXeNYMcOrxWoXC4CUomsApWtKtcEpFbyWoHqzBKQpsgqUBksAakirALVjSUgzYhVoHmxBKQKsgpUB5aANAdWgWbJEpAqzipQVVkC0pxZBZo2S0CqEatAVWIJSCWyCjQNloBUU1aBymYJSBVhFWhSloDUAFaBymAJSBVkFWgtLAGpYawCzYslIFWcVaDVWAJSg1kFmiVLQKoRq0DDWAJSS1gFmjZLQKopq0BLLAGphawCTYMlIDWAVdBuloDUclaBJmUJSA1jFbSPJSDpAKtAa2EJSA1mFbSDJSBpKKtAqylUAhFxIfBa4AngP4G3ZOa3h4zbCHyI3qJzeWZesMI2LQFpBqyC5iqzBK4BTsjME4G9wHuGTO4g4CLgdOAE4KyIOL7gfiWtkVWgYQotApl5bWbu79+9ATh6yLBTgL2ZeV9mLgK7gM1F9itpMuvXw9atsGcP7NwJmzbBvn1lz0plmuY1gS3AZ4c8fhQw+M/s/v5jkkpiFWjJutUGRMRuYMPgQ0AC52XmVf0x5wGLmfmJaUxq+/btB253Oh06nc40NitpwFIVbN7cu1ZwxRVeK6iLbrdLt9udyrYKv0U0Is4B3gq8MjOfGPL8qcD2zNzYv38ukKMuDnthWJq/xUVYWIAdO3rft2yBmOgyo8pQ5MJw0XcHbQQ+ALw8Mx8eMeZg4B7gVcADwE3AWZl514jxLgJSSXwHUT2V+e6gDwOHArsj4paIuLg/oSMj4mqAzHwSeDu9dxLdCewatQBIKpfXCtrHTwxLGsoqqA8/MSxp6qyCdrAEJK3KKqg2S0DSTFkFzWUJSFoTq6B6LAFJc2MVNIslIGliVkE1WAKSSmEV1J8lIGkqrILyWAKSSmcV1JMlIGnqrIL5sgQkVYpVUB+WgKSZsgpmzxKQVFlWQbVZApLmxiqYDUtAUi1YBdVjCUgqhVUwPZaApNqxCqrBEpBUOqugGEtAUq1ZBeWxBCRVilWwdpaApMawCubLEpBUWVbBeCwBSY1kFcyeJSCpFqyC0SwBSY1nFcyGJSCpdqyCp7IEJLWKVTA9loCkWrMKSiyBiLgwIu6KiNsi4h8i4pkjxn09Ir4YEbdGxE1F9ilJg6yCYgqVQET8PHBdZu6PiAUgM/M9Q8bdC5ycmY+OsU1LQNJE2loFpZVAZl6bmfv7d28Ajh4xNIruS5JWYxWs3dSuCUTElcCuzPzEkOfuBR4BErgkMy9dYTuWgKTC2lQFMy2BiNgdEbcPfN3R//7agTHnAYvDFoC+0zLzp4FNwO9ExMsmmawkjcsqGE/hEoiIc4C3Aq/MzCfGGL8N+E5mfnDE87lt27YD9zudDp1Op9AcJbVb06qg2+3S7XYP3D///PMnLoGiF4Y3Ah8AXp6ZD48Y83TgoMx8PCKeAVwDnJ+Z14wY78tBkqZucREWFmDHjt73LVsgJvqxWT1FXg4qugjsBQ4BlhaAGzLztyPiSODSzHxNRDwf+BS96wHrgI9n5sIK23QRkDQzTasCKHERmAUXAUmz1rQqcBGQpAk0pQr820GSNAHfQWQJSBJQ7yqwBCSpoLZWgSUgScvUrQosAUmaojZVgSUgSSuoQxVYApI0I02vAktAksZU1SqwBCRpDppYBZaAJE2gSlVgCUjSnDWlCiwBSSqo7CqwBCSpRHWuAktAkqaojCqwBCSpIupWBZaAJM3IvKrAEpCkCqpDFVgCkjQHs6wCS0CSKq6qVWAJSNKcTbsKLAFJqpEqVYElIEklmkYVWAKSVFNlV4ElIEkVMWkVWAKS1ABlVIElIEkVtJYqsAQkqWHmVQWFSyAi3g9sBvYDDwHnZOaDQ8adDZwHJPCnmfnREduzBCRpwGpVUHYJXJiZP5WZLwY+DWxbPiAiDgP+CPgZ4CXAtoh41hT2XSvdbrfsKcyUx1dvHl91zbIKCi8Cmfn4wN1n0CuC5U4HrsnMb2XmY8A1wMai+66bOv8jHIfHV28eX7WtXw9bt8KePbBzJ2zaBPv2Fd/uVK4JRMSfRMR/Ab9K7zf+5Y4CBqf7jf5jkqQ1GFYFRYy1CETE7oi4feDrjv731wJk5vsy81jg48A7ik1JkrSSwSq4+OJi25rqW0Qj4hjgM5n5k8sePxPoZOZv9e//JXB9Zn5yyDa8KixJazTpheFpvDvoBZn51f7tdwA/m5lvWjbmMOALwEn06uMLwMn96wOSpJKsm8I2FiLix+hdEL4PWPpt/2TgNzPzNzLz0Yj4Y3o//BM43wVAkspXuU8MS5Lmp5RPDEfE5RHxUETcvsKYHRGxNyJui4gT5zm/IlY7toh4RUQ8FhG39L/eN+85FhERR0fEdRFxZ/8NAu8cMa6u52/V46vzOYyIp0XEjRFxa//4hn2u55CI2NU/f5+PiGPLmOtajXlsZ0fENwfO3ZYy5lpERBzUn/uVQ55b+7nLzLl/AS8DTgRuH/H8GcCn+7dfAtxQxjxndGyvAK4se54Fju+5wIn924cC9wDHN+j8jXN8dT+HT+9/Pxi4AThl2fNvAy7u3/4VYFfZc57isZ0N7Ch7ngWP8feBvx32b3CSc1dKCWTm54BHVxiyGfhof+yNwLMiYsM85lbUGMcGMNFV/CrIzAcz87b+7ceBu/j+z3zU+fyNc3xQ73P4vf7Np9G7Lrj8NeHNwEf6t68AXjWnqRU2xrFBjc9dRBwNbAIuGzFkzeeuqn9ArukfLju1n6yfjoifKHsyk4qI59GrnhuXPdWI87fC8UGNz2H/5YRbgQeB3Zl587IhB85fZj4JPBYRz57zNCcyxrEB/HL/Zcq/7/9QrZM/B/6Q4YsbTHDuqroINNl/AMdl728tXQT8Y8nzmUhEHErvN43fzaf+6ZBGWOX4an0OM3N/f+5HAy8ZYxGrzW/OYxzblcDzMvNE4Fr+/7fmyouIXwQe6pdqMN55WXVMVReBbwCDfyfv6P5jtZeZjy8la2Z+Flhfl9+ylkTEOno/ID+Wmf80ZEitz99qx9eEcwiQmd8Gruf7/47X/fTPX0QcDDwzMx+Z8/QKGXVsmfloZi72714GnDzvuRVwGvC6iLgX+Dvg5yJi+V9jXvO5K3MRWGkluxJ4M0BEnAo8lpkPzWtiUzDy2AZfG4+IU+i9TbdW/8GAvwa+nJl/MeL5up+/FY+vzucwIg5f+gu+EfEDwKuBu5cNu4reBVSANwLXzW+Gkxvn2CLiuQN3NwNfnt8Mi8nM92bmsZn5I8CZwHWZ+eZlw9Z87qbxYbE1i4hPAB3gOf0/PLcNOATIzLwkMz8TEZsi4qvAd4G3lDHPSax2bMAbIuJtwCLwP/Su4NdGRJwG/BpwR/+11wTeCxxHM87fqsdHvc/hkcBHIuIger8EfrJ/vs4Hbs7Mq4HLgY9FxF7gYXo/cOpgnGN7Z0S8jt65ewQ4p7TZTknRc+eHxSSpxap6TUCSNAcuApLUYi4CktRiLgKS1GIuApLUYi4CktRiLgKS1GIuApLUYv8HNieV6V4ehwMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f6698704908>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.show(plt.plot([1, 2, 3, 4], [0, -1, -2, -3]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "23.66\n"
     ]
    }
   ],
   "source": [
    "# standard loss model for linear regression \n",
    "# sums the squares of the deltas between the current model and the provided data\n",
    "y = tf.placeholder(tf.float32)\n",
    "squared_deltas = tf.square(linear_model - y)\n",
    "loss = tf.reduce_sum(squared_deltas)\n",
    "print(sess.run(loss, {x: [1, 2, 3, 4], y: [0, -1, -2, -3]}))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0\n"
     ]
    }
   ],
   "source": [
    "fixW = tf.assign(W, [-1.])\n",
    "fixb = tf.assign(b, [1.])\n",
    "sess.run([fixW, fixb])\n",
    "print(sess.run(loss, {x: [1, 2, 3, 4], y: [0, -1, -2, -3]}))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ...but can we do it automatically?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "optimizer = tf.train.GradientDescentOptimizer(0.01)\n",
    "train = optimizer.minimize(loss)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[array([-0.9999969], dtype=float32), array([ 0.99999082], dtype=float32)]\n"
     ]
    }
   ],
   "source": [
    "sess.run(init) # reset values to incorrect defaults.\n",
    "for i in range(1000):\n",
    "  sess.run(train, {x: [1, 2, 3, 4], y: [0, -1, -2, -3]})\n",
    "\n",
    "print(sess.run([W, b]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Complete program"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "W: [-0.9999969] b: [ 0.99999082] loss: 5.69997e-11\n"
     ]
    }
   ],
   "source": [
    "# Model parameters\n",
    "W = tf.Variable([.3], dtype=tf.float32)\n",
    "b = tf.Variable([-.3], dtype=tf.float32)\n",
    "# Model input and output\n",
    "x = tf.placeholder(tf.float32)\n",
    "linear_model = W*x + b\n",
    "y = tf.placeholder(tf.float32)\n",
    "\n",
    "# loss\n",
    "loss = tf.reduce_sum(tf.square(linear_model - y)) # sum of the squares\n",
    "# optimizer\n",
    "optimizer = tf.train.GradientDescentOptimizer(0.01)\n",
    "train = optimizer.minimize(loss)\n",
    "\n",
    "# training data\n",
    "x_train = [1, 2, 3, 4]\n",
    "y_train = [0, -1, -2, -3]\n",
    "# training loop\n",
    "init = tf.global_variables_initializer()\n",
    "sess.run(init) # reset values to wrong\n",
    "for i in range(1000):\n",
    "  sess.run(train, {x: x_train, y: y_train})\n",
    "\n",
    "# evaluate training accuracy\n",
    "curr_W, curr_b, curr_loss = sess.run([W, b, loss], {x: x_train, y: y_train})\n",
    "print(\"W: %s b: %s loss: %s\"%(curr_W, curr_b, curr_loss))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](https://www.tensorflow.org/images/getting_started_final.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# tf.estimator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Using default config.\n",
      "WARNING:tensorflow:Using temporary folder as model directory: /tmp/user/43350/tmp66utldiv\n",
      "INFO:tensorflow:Using config: {'_log_step_count_steps': 100, '_session_config': None, '_model_dir': '/tmp/user/43350/tmp66utldiv', '_num_ps_replicas': 0, '_save_checkpoints_secs': 600, '_master': '', '_is_chief': True, '_keep_checkpoint_max': 5, '_save_summary_steps': 100, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f669875e710>, '_task_id': 0, '_tf_random_seed': None, '_service': None, '_task_type': 'worker', '_save_checkpoints_steps': None, '_keep_checkpoint_every_n_hours': 10000, '_num_worker_replicas': 1}\n",
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Saving checkpoints for 1 into /tmp/user/43350/tmp66utldiv/model.ckpt.\n",
      "INFO:tensorflow:loss = 19.0, step = 1\n",
      "INFO:tensorflow:global_step/sec: 400.965\n",
      "INFO:tensorflow:loss = 0.481899, step = 101 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 397.11\n",
      "INFO:tensorflow:loss = 0.0777452, step = 201 (0.251 sec)\n",
      "INFO:tensorflow:global_step/sec: 397.359\n",
      "INFO:tensorflow:loss = 0.0153986, step = 301 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 405.704\n",
      "INFO:tensorflow:loss = 0.00453881, step = 401 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 411.165\n",
      "INFO:tensorflow:loss = 0.000645445, step = 501 (0.243 sec)\n",
      "INFO:tensorflow:global_step/sec: 404.756\n",
      "INFO:tensorflow:loss = 0.000235079, step = 601 (0.247 sec)\n",
      "INFO:tensorflow:global_step/sec: 402.054\n",
      "INFO:tensorflow:loss = 3.17636e-05, step = 701 (0.249 sec)\n",
      "INFO:tensorflow:global_step/sec: 396.249\n",
      "INFO:tensorflow:loss = 1.2088e-05, step = 801 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 397.626\n",
      "INFO:tensorflow:loss = 8.24171e-07, step = 901 (0.251 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 1000 into /tmp/user/43350/tmp66utldiv/model.ckpt.\n",
      "INFO:tensorflow:Loss for final step: 8.65676e-07.\n",
      "INFO:tensorflow:Starting evaluation at 2017-12-13-12:24:00\n",
      "INFO:tensorflow:Restoring parameters from /tmp/user/43350/tmp66utldiv/model.ckpt-1000\n",
      "INFO:tensorflow:Finished evaluation at 2017-12-13-12:24:01\n",
      "INFO:tensorflow:Saving dict for global step 1000: average_loss = 1.84357e-07, global_step = 1000, loss = 7.37429e-07\n",
      "INFO:tensorflow:Starting evaluation at 2017-12-13-12:24:02\n",
      "INFO:tensorflow:Restoring parameters from /tmp/user/43350/tmp66utldiv/model.ckpt-1000\n",
      "INFO:tensorflow:Finished evaluation at 2017-12-13-12:24:03\n",
      "INFO:tensorflow:Saving dict for global step 1000: average_loss = 0.00256159, global_step = 1000, loss = 0.0102464\n",
      "train metrics: {'global_step': 1000, 'loss': 7.374291e-07, 'average_loss': 1.8435728e-07}\n",
      "eval metrics: {'global_step': 1000, 'loss': 0.01024637, 'average_loss': 0.0025615925}\n"
     ]
    }
   ],
   "source": [
    "# Declare list of features.\n",
    "feature_columns = [tf.feature_column.numeric_column(\"x\", shape=[1])]\n",
    "\n",
    "# An estimator is the front end to invoke training (fitting) and evaluation\n",
    "# (inference).\n",
    "estimator = tf.estimator.LinearRegressor(feature_columns=feature_columns)\n",
    "\n",
    "# Two data sets: one for training and one for evaluation\n",
    "# Tell the function how many batches of data (num_epochs) we want and how big each batch should be.\n",
    "x_train = np.array([1., 2., 3., 4.])\n",
    "y_train = np.array([0., -1., -2., -3.])\n",
    "x_eval = np.array([2., 5., 8., 1.])\n",
    "y_eval = np.array([-1.01, -4.1, -7, 0.])\n",
    "input_fn = tf.estimator.inputs.numpy_input_fn(\n",
    "    {\"x\": x_train}, y_train, batch_size=4, num_epochs=None, shuffle=True)\n",
    "train_input_fn = tf.estimator.inputs.numpy_input_fn(\n",
    "    {\"x\": x_train}, y_train, batch_size=4, num_epochs=1000, shuffle=False)\n",
    "eval_input_fn = tf.estimator.inputs.numpy_input_fn(\n",
    "    {\"x\": x_eval}, y_eval, batch_size=4, num_epochs=1000, shuffle=False)\n",
    "predict_input_fn = tf.estimator.inputs.numpy_input_fn(\n",
    "    {\"x\": x_eval}, batch_size=4, num_epochs=1, shuffle=False)\n",
    "\n",
    "# Invoke 1000 training steps by invoking the  method and passing the\n",
    "# training data set.\n",
    "estimator.train(input_fn=input_fn, steps=1000)\n",
    "\n",
    "train_metrics = estimator.evaluate(input_fn=train_input_fn)\n",
    "eval_metrics = estimator.evaluate(input_fn=eval_input_fn)\n",
    "print(\"train metrics: %r\"% train_metrics)\n",
    "print(\"eval metrics: %r\"% eval_metrics)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from /tmp/user/43350/tmp66utldiv/model.ckpt-1000\n",
      "2.0: predicted: -1.0003325939178467 true: -1.01\n",
      "5.0: predicted: -3.9992563724517822 true: -4.1\n",
      "8.0: predicted: -6.998179912567139 true: -7.0\n",
      "1.0: predicted: -0.0006914138793945312 true: 0.0\n"
     ]
    }
   ],
   "source": [
    "prediction_results = list(estimator.predict(input_fn=predict_input_fn))\n",
    "for x, prediction, y in zip(x_eval, prediction_results, y_eval):\n",
    "    print('{}: predicted: {} true: {}'.format(x, prediction['predictions'][0], y))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "sess.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Something more impressive - computer vision"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### MNIST - handwritten digits database\n",
    "![](https://www.tensorflow.org/images/MNIST.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "images 28x28\n",
    "![](https://www.tensorflow.org/images/MNIST-Matrix.png)\n",
    "...We can flatten this array into a vector of 28x28 = 784 numbers!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting MNIST_data/train-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/train-labels-idx1-ubyte.gz\n",
      "Extracting MNIST_data/t10k-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/t10k-labels-idx1-ubyte.gz\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "mnist = input_data.read_data_sets(\"MNIST_data/\", one_hot=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAD8CAYAAABXXhlaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJztfVuIbNl53rfqfunLOdJEM6CJ5ZBAAgExkFgQZEgbGyOCQcYPiiITpDgIPyixifIgWS9zJuTB8sOAMPhFkYVkLBzb4Eh+sWQjmiCDo0kiJbKtiyGZsWVrjpQzp69VXdWXlYfuf/e3//rXrl3du6ure/8fLPal61St2qe+9f/rv4YYIxwOR73QuO0JOByO5cOJ73DUEE58h6OGcOI7HDWEE9/hqCGc+A5HDXEt4ocQ3hVC+FYI4TshhI9UNSmHw3GzCFf144cQGgC+A+DHAfwNgFcAvDfG+C31Og8UcDhuCTHGYN2/jsR/B4C/iDG+FmM8BvBbAN6d+PBsvPjii7nrVRs+v/s7v1We203MrwjXIf5bAfwVXX/34p7D4VhxuHHP4aghWtf4t38N4Ifo+vmLezN49OhRdv7gwYNrfOTNY2tr67anUAif39WxynMDrj+/7e1tbG9vl3rtdYx7TQDfxrlx73sAvgrgX8QYv6leF6/6GQ6H4+oIISAmjHtXlvgxxtMQwr8B8CWcbxk+pUnvcDhWE1eW+KU/wCW+w3ErKJL4btxzOGoIJ77DUUM48R2OGsKJ73DUEE58h6OGcOI7HDWEE9/hqCGc+A5HDeHEdzhqCCe+w1FDOPEdjhrCie9w1BBOfIejhnDiOxw1hBPf4aghnPgORw3hxHc4aggnvsNRQzjxHY4awonvcNQQTnyHo4Zw4jscNYQT3+GoIZz4DkcN4cR3OGoIJ77DUUM48R2OGsKJ73DUEE58h6OGcOI7HDWEE9/hqCFa1/nHIYRXAewCOANwHGN8RxWTcjgcN4trER/nhN+KMT6tYjIOh2M5uK6qHyp4D4fDsWRcl7QRwBdDCK+EED5YxYQcDsfN47qq/jtjjN8LIfwtAH8YQvhmjPEr+kWPHj3Kzre2trC1tXXNj727iDFe+bVyzcfUvdS4aYQQsuO8Yb2u7Ps7ZrG9vY3t7e1Srw1V/RhCCC8C2I8xvqzux2X84O4KyjwLi6xyfnZ2lh1T4/T0NDlSi8lVoYnYaDTQbDbRbDZz5zzkvvVafj+L5E788gghIMZoPrArS/wQwgBAI8Z4EEIYAvhJAC9d9f0c59AEt8guJD45OcmR+uTkBCcnJzg+Ps6OPOT18jn6cxdBiqDNZhPtdjsbrVbLvOajnIcQ0Gg0sveTOTnZq8d1VP1nAfxeCCFevM9vxhi/VM206g0mvCXNmcj6OJ1OMZlMMJlMcudyfXJykn0GH/V5EVhN18dms4lut5sbnU4ndy7Xch5jnCE9v6/83VEdrkz8GOP/BfBChXOpPVi919JdzoXc0+kUx8fH2bmMyWSC8XiM8XiMo6Oj7Fyuj4+Ps8/iz+R782Dt0eW83W6j1+uh3+9ng697vR56vV6mfTDpW60WGo3GzGc5qsd1jXuOisDkK9qrC9mPjo5y0nwymWREH41GODw8xOHh4cz5dDrNPucq5LeMcnzd6XQwHA4xGAwwHA7N8xTpeR4X+9PcuS8C1cGJvwKw9tuyAOj9u0j1o6OjjOh8PDw8xMHBAfb397G/vz9zPplMcp9hGRGLMM8y3+12sb6+jvX1daytrWXHyWSSszNo0p+cnGS2DP4sx83AiX/LKCK9DCG97OWF+KPRKJPwMg4ODrC7u4u9vb3cUc6Pjo7muv6KMM9F1+/3sbGxgc3NTWxsbGA8Hmekly2LJn2n00Gv18PZ2Zn5TORzHdXBiV8xioxmcmSLvR5svLOs85PJJEd0PTTxmfS7u7uYTCbX9vOnSN9oNHB0dJRbtHgR0+fyXuzWOzs7Q6PRSI4yc3PMhxP/BqCJzddsqLOGqPOyl9dH3suz8U7ORdU/ODjAaDTKDHqsSt9UQI98V/kOR0dHaLfbmX9evr/1nWX0+/2ci0+7+5zY1cCJXzEsVV3Izmr7PKmurfVstRfya8OeWPTFmCdqtrjxmPg83+t+X4YmfhHpmfAyBoNBzt0nR3EVOqqBE/8GYBnmtFVe+9nn+d/5vGiwO080ASH+VS35Rd+Tre9Anvia9LLIyfPQtovj42McHR3lXIHy3hIYJOq+S/7rwYlfMZj0/MOWa62qW5Z5LdH5npA4FcRjLRrasCbzvO73ZDcb2y+E+Jr0rH3oId9HFjDRUIBz0kugj/58x9XgxL8BWORniSaGOPGt8zUvCNaYTqczWoQ+16G6vMeX+fFcr/M9NfmF+Bbp2+02JpOJSXg9V3lPIX23283mrz/fsTic+BVD++BZEsu+V6zveuzv72cuOivqbjwe4/j4+MoJOlVKfP6+DCG+kJ7V/mazOUN8y95xdnaWkV4iAXmrwmR38l8NTvwbAJNQq+BM/L29vdyQIBvxzfNRzjnkNrVft4YOjqka2lV5enpquvy63e6MQZO9FvLvhPTdbje7zxqLk/16cOIviHnkYXWXw2u1u02i6YT04msX4jPZ+VqSbG4CVZBpnpuQ/fSchMOLhbjvut1uFucv2xwxGHLswCL5/I5zOPErBu9reT/PATYi3VnKy36fDXxi5NJq+m1Aq9dXhfb1t1qtXB6+7OlT4/T0NJnvr7P7HGk48StGjDEXUz8ej2f28Tp+XojPvncd264NW1VjEcJcZxHQGtFkMslcdDFGNBoNM61XxtnZ2UxgD0cBOsrBiV8xOGdeJP7h4eHMPl7IzucSacd7Xk5quSmJf5VKN0y2RealjX7yOfLcGo2GKellAYgx5u7LZ5cJ53VcwolfMeSHrSX+/v4+dnZ2sL+/n0l3PUajUU7Si1S8SYk/r9SVdV/77hchP0t8lvSyGEhqryXtJYJPZ/KJyn+bW6G7Bid+xdC+a5b4YsDT/nseLOVTrriqUET6lBaQCtxZBGLEA/IaktQK0ITn6xBCrnagZPixyu+YDyd+xZA9Pqv6YtDb2dnB7u5uodV+Op3OxPrrjLYqYJG+jMp/XfKzxGdJL5Z+rcrr0Wg0Mu1HJH273b514+ddgxO/YsxT9Xd3d5Mhu7o0luWnrxoW6ctI/6uq+xxTYPn6Y4zJ2nydTidL1GFJz0VEHeXgxFco++NNDSYzh+KyZd+Kwxf3XdkfcBE5iyrk6GvtE+f3so6sjViaCT8Lflb8bIsWsmazOfMMpU5ft9vN3H9SwKPT6WR7/jJS3y3/53DiLwiRVKmxv7+P3d3dnPWe1XlOnBHDnVXvnmGRPFUCi4tWiG87dW39rai6jqjZnBhkjVRKchli6nBnXYOg0+mg3W6j0+lgMplk4bz8ufqZOdln4cRfEGy8s0YqMEdcdaLOc5ZaihSpfbiOWmPCijSUkbrWTS7kHkfV8VHOz87OcqnC+liUelvWM6HLiMuWSW8Ber1ebgGV56ifm4f4zsKJvyD4R6mr3E6n08xyLwE6VkQex6qnLPZFpC8qTSWRbyIVpZEFn6cq3GjyW1rCycnJTAIRJxKxRqP99GXsAGz80xJ/NBplpJ8X3chkd/LPwom/INhqb9WwZ+LriDx5nZVOm9r3WntzTUo+Soy7bmSh/eJ6QZAjk98KjT0+Ps7FHvB3PDw8zBaSyWSSi8WX71vm+WoXnxBf5inx+7LAsIYhC0wZA2Wd4cRfENpPr+vYC/El4Yb3+CKleO/LKqpAE53PmfTWEGLI4IYWcs8qbSVHqZFnbQlarRam02ku16Df76Pb7WbVcXix4Oe1SGSdtceXuXQ6HVO7KIp1cNLPwom/ICziHx4eZqq9JfF5jy/EZ7eWVQsvZaW3yM/7dyFxv9/HYDCYGbIAyB5ZjnLe6XQKbQPT6RQ7OzvY2dnJkV7cbLxYseQum0CTkviymHQ6HQwGgxzpWdqLxGcPhcfyz8KJvyCKiM/RebzH16q+vI91ZBRZ2DXpdSqrdK9ZW1vLjmtraxgMBjMtrfhcXGYpG8DR0RGGw6FJevZO6Jj8RYnP+fqSyCPhvJbEZyOpBALxc3XS5+HELwkuNKEDdIT4XMOeVXxthCqCZcDja919lvfr7XYb/X5/huxlid/v9zOJnzICisZiVfzhPbwmvmwFmIApQ1/RosHEl8EGVum/p+0fbuDLw4k/B/rHqUNyLeKLpNddZMoEB7FLTu+1WaqnjHe9Xi/rUZdS9bV6L/t+JrjOcechwTPdbheDwSBXHFOr+mwIbbfb2cJXFOTDi4reJgjxdY0DrsrLWpAMwDP4GE78AjBRiyS+hOQK8XVufRnisxGPSa4HV6WZd7TupdpVi9rOxj2tdfD8pBaeNMHUgTPa7SmaRKvVMiP8rGfMMfzydyb+4eFhtpjJAJDzVHAyj8fyX8KJn4D1gwRmpZj8ADkWn33bZYjP6ie75XR6Ku/fZQ+vJbpI8dTQ7jzLx2+pyZr44j3QxTElyEfbQdhoyKG/cpTnLIOr9cpzlyIcOhyaPRYAsufEpPcknjzmEj+E8CkAPwXgcYzx7Rf3HgL4zwDeBuBVAO+JMe7e4DxvDZYRTiR+StXXgT1FxLes9yLxWSWXMRgMsv26tZfv9/umf1776lPDCuu1VH2R+Fz7vtVqIYSQK0QyHo9zWwkJ6pF/pwN72KrPcQByr9VqYTQaYTAYZMTnLYt8vk7blTbcjnOUkfifBvCrAD5L9z4K4I9ijL8SQvgIgF+6uHcvoX+UOoCH8+13d3fNUF6L+KnoPJb4oqKLRJfW0+vr69jY2Jg57/V6c4ltDXYPWiHBAo4OZNLLNqTRaMzEOAjxWeJrIx/fk8WVn7nEArRarex9WdKLRsOLiE7bdVxiLvFjjF8JIbxN3X43gH96cf4ZANu4h8S31H0x7uka+SzxUwk8RX56OVqqPrvm1tfXsbm5mRvSlnpzcxP9ft8Mt00l6FgeBB4yLz6XfTqA3FyPj48RQshpQkJO3uNrA58mPav+7JeX78ELit4KySKl03Zd1c/jqnv8t8QYHwNAjPH1EMJbKpzTrSO1v9fBJSl3XlHargYTSo6cdioSX4i/sbGBBw8e4MGDB3j48OHMUdRdTdoiIltzsObI85PceLHUy2g0Ghkx9/f3MRgMZqIC5VnqPb2ASWrN1SK+fIYscLLISNpuWa9KXVCVca/wiT569Cg739rawtbWVkUfuxykyG81y5hMJnPfj9VofW4F4Igqz5KdJb38bWNjIzNw3SQ4Kk562ss4Pj7OjI58lHPd608Kj+gMxVRgk9UuXJJ32CMhWogO6bVCoxl32c+/vb2N7e3tUq+9KvEfhxCejTE+DiE8B+D7RS9m4tcdHHVnjX6/P0Nqi+gSjCNuODGsLfN76C0KkN/vSzDRxsZGFrIsbbR0F2De1xdBB/ew10CIz8ZMmYcsNvycLG3nLkML1Zdeein52rLEDxdD8AUAHwDwcQDvB/D5BedYW1h+ej4fDAYzkl1LeR2FJyrusn68OgpOjmyf4C3K+vp6VoeA9+jj8Tiz8LPrbh64IYcQnyP2xGXZ6/VmEnlSrb34e9QBZdx5nwOwBeDNIYS/BPAigF8G8DshhJ8D8BqA99zkJO8TRDqKRNJ15URCCtEfPHiQO66vr+f89hwvv2yJL5DgmBhjTuKLF4JrEEiiDfvzRe0vM3+W+OIyZG+BuEE5UUk3J9HbLP196oAyVv33Jf70ExXPpRawjHc8hPhCfjHkyRgOh2a47bKJL99FQ6v6w+EwF9LL/n4dE1GW+KkWXBKuyzEP3KCEjXziJeCknjqR3yP3lgyRMtpqLxKc3XUs7cVqPxgMzOCcZav6gF3lRhOfYxjYkKndomXnr/f4/O9kYZHnmcrgY7C2UifyO/GXDC3x2U8/HA5nrPdMeiG+VSjjNox7fC6k0ao+dwESV5uW9OLjX1Tiy/aA7zWbzex56uKmqXiKOrr5nPi3ACsyT4xg84jf7/dnDFM6nn5ZsIKQ2LjHpbDE6BZCyKS1xEAsorHoOH4dW9FsNjEcDnPFTbWqz/OV51Y38jvxE9AZZHLNdfJ0Lfky0LXxOP6eQ3A5/p6TcZbhp78uUsFDHIGXsqyXQVEsf6fTMYuByj0dwsxbkDrBia+g88F10QlOvNHqY5lY/JSaz0Y97aoTV9WqwYpwtKIauT24ZDBKoRKRyNJSq8xn8mKsFwH5bAmn5hZmkrGn23IBl1uwusCJb4Ary7CEl9LSqZJPjFQo7Dw1X1x2VnmrVTI8WRF2sne3Epg4iUmKlQjxFylUIp8lWwgO+okxZpF9nC9wcHCQeUFijJnVn42OzWa9uu068RX4B8ytquVcJBQ3xNASvyhW3rLo6/29qPaczrpKxNdk19mLqTyGp0+fZsTXEn9R4jP5+T5LfPl8LiQKYMbTIIuxE7/mEAnOsfii2ltlndlyzbD2sUWZdxKZxxVluF/cqhAfSPcPTCUw7ezsZMTnCsSLqvry2fy85bMbjcZMAw5dPRhApiWwvaVuSTxOfAX+Aes2TqJC8h6fUz61m8iyuqcy71jic+lrkfhlq9QuCxbp52UuvvHGG9jb28vVy1tU1RcprdN3WdtgVV8n8PB7COkX3WrcBzjxDaSy7zgDj6vmaolfpObPS7nd2NgwK+isksTXRj32ehQRXyQ+P8erqvr82RLvH0LItTazSn7x/wMHGjnxaw62FnP/Npb2lnGvSOKzG4vj9DmslSW+1cxilYgPzFrXLYnP9QiZ+Lo60VVUfZb8AiG+/F/pGoLymqLowrrAia+g1VaW+qmOsKn2zHJtLQAs+bmQZrfbTZbDWtb3L7pmj4euq396eprroycWdR6j0SjXQZffa1HiWXO1eu5ZSTwS1st5BGXiMVZp8b0OnPgGtI+YF4AU6bXkSeGqQSvLRJHVnu0e2vB5fHyMvb09PHnyBE+fPs01FtEZclcNgCozdx3Pz402rE67dSzN5cQ3oIlvufV008t5qv51ib7sRaIocpH359r2sbu7iydPnuQCdSRyzupzZ8VAVDF31tRY3RficyivDsJa1QW5SjjxFbRrSlv3WeqnSF+EVZb0AmvvLkMkqFjldVebvb09vPHGG4USX5O+aomvt2nSew84r7mvk3e0xOd4jPsKJ76BIomfUlfnQRO+jDZwGz88yy+va+pxdWE99vb2MmlvEV+/n/68Kuavk3Y4pt9S9fUCzq+/r+R34hvQBr55e/yiH65F+BRWRRvg78/bGW2xF6ILyWVI52AJ1NGqPmsTVZJewAv2dDrNvtPZ2VmO+FYshrz2vpPfia9g/ejZCm1Zo1OGIcvCL8dV/zFpdVy+M3fIOTg4wO7ubhaVt7Ozg729vZxF//DwcEbVTxkPq4CW+PqelvipPf59JbzAiW9AS/siiZ/KzNNIqfKr+OOyFj+9Z2aJ//TpUzx58gRPnjzB/v7+TEos9xDkVtryWVXPXRYqAS8Eeo9vWfU1+e/jIuDEL0AZYhb93YpwK7IfSKyA5InHGLPKNGXTchchkuUHZ4u4rn9/cnKS883v7+9nar2o/Pv7+7m+gdp6vmyXmd5OWKOOcOIrcEgnB9pwVVxuPqnbSAOzJGdYNeElwu3g4CCL7JNxdnbeIbbKYhFFBOC5cdCSnEsUHu/hOe6eVWg25i2LZJwTwTX2udGnBEpZkX3yHny8j3DiG9Bx9RxiqzvO6lbS2jCkFwAmvviTObRVyk93u92MMFXmi1uuOj7KHp599Ty4mAZn2YkBT1R6bTGv2lefQur/jmsgcDlz6//Qes/7Bie+AevHw+S3esnrHwfvFzlvXIeUWhK/3+/PkF5SSquANtxZfnren4u/fjwemxJfDHhMfG0IXaZKnVq0JVRXL+DW/+F9JDvDia9gZdJJTP3x8fGMpOAutAL+kQvpRRvQqj5byKXTrUX6KkJKtWqvXXVc+ZY1ER4pVZ+NeGz4nOf5qBpF2hr3Lygi/n0nPeDEN8GJNLoXG9eyL1ITLcNZCGEmx58lfr/fz1R6nbt/XampXWjaYs8Riql6eWLM0356rerrcOZlS3yudKSJLxKfNbciVf++LgJOfIWUxJc0zqI9voDdQHIt721JfK4UI5FmTPrrJpGkrPfaXWkZHYXwHJwjhBfSa1VfB+gs27insx+Z9LrAic7V1+91X+HEN2CpiyK9WOqzqp+S+Ex6UfWtPb7V313aPFsdYMrCIj1LfK47kCpUKa66nZ0d7O/vz6j/ouqL6876/quwx2fyu6rvyMGS+Ez8lHFvnqtNfvgcTKKlvu6cy0apXq+H6XRq/jgtw2JqyD5eJx3JfEajUVYQk8NxeV/PiTlWt5rbhPxfWKQfDAYzUr+MVf8+wolvQBfLOD09zSq0snGvyKqfgg4p5TZS7L+XH65uACFto1JDyG0Z7iTJRnzy1hiNRrk9PJ/LXn5e5NuyYC1+QnoucqL7E6Ys+3WCE19BB4CIRV1Q5M4rQ36L+FwpJkV4mYu0jpLP5AAi8QjovAJ2r3H9QD6XwZZ83sPLtZBeNxVZlp9eoDUfK3BHS3vuRqRLl69aMdObhhPfAKv6Ejkn93Xk3lV+NExOXSHGIj4bGaXHPG8xWEMRXzwPXTosFZwjQ/vu+Vyy7Li3wFVLZ10Vmux8zc+Q25SlJL48Vye+QgjhUwB+CsDjGOPbL+69COCDAL5/8bKPxRj/4MZmuWQwkcRCLz8qTXpt4JuHlKov/14X2tQLgHSEtYZIfKsyDhOde8lZ59a/tYqM3rbEt0aRRV8alWgDn0t8G58G8KsAPqvuvxxjfLn6Kd0u9A9I39N7QytWvwgW8TlWnPeouspus9nE6elprkW2Pp6enprZcVYknq6gwxLdMvxxERIrSGfZvvoi0lvWfFb1+f9Rx+rXAXOJH2P8Sgjhbcaf7u3yyAkxWu2/rqrPGXCi6su/PTs7yxHfkuqnp6eFtoCTk5PMFSf7dd6363v6KDXuLXLrSDyrhNYyoI162ubB5J+n6vMC7hK/HD4UQviXAP47gH8fY9ytaE63Cov07Pu+rnEPyLv0OKHn9PQ0p+7zkYnP1n/tAmTiW6WxdOCNnMtxOp2WSmXVkYDLhpb0bOjU2ZQW8UXqX9VOc9dxVeL/GoD/EGOMIYT/COBlAP869eJHjx5l51tbW9ja2rrixy4PTHjg/AeujW/84xJ/e6fTSZJDx8iLhV4gn5HSJmKMmE6nM2TncXx8nCM4R9mlSM9DqtZUgVQkHC8Uiy4amvB66BgIseAL8Zn0OmTXmvNdwvb2Nra3t0u99krEjzH+gC4/CeD3i17PxL9r4NBbdhlxJ5zBYJC1wNJ+dN18Apit6ddoNDLCNZtNHB0d5X6MMcbMmNbr9WbUe1b5T05O5qr23KXWqjlXxTOzLO5M/NTCWPb9OZeCjZu8CGvicyPS63pmVhFaqL700kvJ15YlfgDt6UMIz8UYX7+4/BkAf7rwLFcYTHZ9z+q7xr3vJKed/ediINPx60J8JkSj0cB4PM7t+9ntJ/H8VzXusWVfV76tmvisgvMWyqoFIN+/7Psz8dkmYpGeyb9IrP59Rhl33ucAbAF4cwjhLwG8CODHQggvADgD8CqAn7/BOd4ahOxa6jPxe71ervedqOMcIANc7uu1tBdw3r7e93PGXKfTMaUd2wC0O85yy2nXXFWhtkVqOBNfvquQvqw7UL+/1np0SzIt7fv9/ox9xK36BmKM7zNuf/oG5rLy0AEiKVVfq+q6+COTn+/pa4mrtzq/6sAdObfCcvnaant1U6o+L1DsHpXtj97zW5qWBR1SzURmaa+lvkh8y116H1T9ReCRe3Ogpb6W+NztdjweI8aYuYeAfDsneQ+t3oqKL38XbUAMeePx2EwKsizaVsguh+7qcuE34Yfn+bA20mq1cgFRUmpbL3rz3ttS9Tlt2iI/S3xtF6gb6QEnfhJavecfrFb1ReJLLjqTXgisrfNcZYdVfC5jrYnDP9LU4IWj6Mg2hipr4llWd96Hy2vkOfAWZ5HPsIJ1tGHPIn2v18tpSzzPOpHfiV8Ai/wp454YynjPKkY5cRnpH7wORJFj0eDXWudFbsSiv8m9qmAZ39rt9sznS9ASL1xFKJL4KYu+3uvr51g3wx7gxF8I8gPRe/xU3Xjts5cEG/mbPqbcWzdBzNR342v9d34dbzO0hNcRhXwtxs9ms5kr2rGIcdGKx9fSXdR8rqgrdgCHE38utBSSH5384AaDQc4qntrbCgGOjo5y7jx2aVnX+vw6wS/6e6WORdqGlUSk76U8DuxuFLcib4vKBA/JVkuH5EoCztrampmBVzdVfh6c+CVQ5M6zSG+poaKKjsdjM/bdiom3/gbMtnG+TvSbvk7539mjwUY0fWSSWXtpqVTcbrez6EghfRli8jysICqdgVfXQhvz4MQvCTbwCZl7vV62p9fJNVrNFfKPx+PCQhlWc05tFBQs6gaT16aGTnSx/PCdTicX/ipDrlutVqEtgm0ewKUthHvYz5u/RXwJonKJXw5O/AXApOh0OjOk5zRPJj53cxmPxzOprtqvLtciIa3gHsFV1f0i6a49CXzd7XYzgllDDHgp4+F4PM4kPXffZePnvHmzJsVx+DKH4XA40ybLiZ+HE39BsKoP5EkvxR3k7zzEwDQajWYi+/hcDF/aC6Dr07NXQEcXlvkOmuw8rMxAOReCbW5uYmNjY+YoC6LlLjw7O8NoNAJw2UpsMplkz6wsMYtU/ZTEX+T96wAn/oIQVR/IG/p6vR5OTk6yfa4mvfwI+/2+WfZKym9xFJkO79US/7rS3iK9EN3K9W+1Wuj3+1mU4sOHD/Hw4UO86U1vys6l+QcHB3Hxjm63mwtDHo1GGTkXlfgcmptS9VniOy7hxF8Q8gOSKDmRcEJQCQnlzrrsW+ZKtRKGm6rmw6QX1Z/3/IxFpL0cNfG1dV7bJ5j4m5ubePjwIZ555pnc6Ha7ZottuZZWZBKGfHBwsJDE5z2+FTatVX3f49tw4ivM+3FwZB5LESFes5nvaqsDTjie3BqyGFjltlutVrbnZ21A76dT30Hmol1sWqXXZOd5COk3Nzfx4MGDmcHEt+r3n56eYjj2IAohAAAOt0lEQVQczrSyKktOHbE3r7RWXWPx58GJXxHY6q8lkuzN2T7AlWGsKrepopic3ps6poJvALumX2pY6v5wOMzU+83NzUy15mo2sp+XgB1GakHjAdgdgAAk3aS8nbJy7V3Vz8OJfw1ogxpLVFH3OQ5eGwJT9e05wIUXACZ+Ud27ogg8y/BopammjoPBIDPmCfFFejPxdbCRYN5n82JhBSvpVFydlWf1N3RpPwsn/jVhkZ8lkv7BimRKufOK6t0L8dlSrs/13PS5GCP1YJejduGxa6/X68248Pr9fk6t1m48+fwQQimJb4Uvy7m1JbHKn+lW5nWMxy+CE78C6Mg+IbmV0Sfdb1PBO9yf3mp8IRLfiuzTqbVWII1E3jFJ+FokZCrnXzwTHLSjVf0U6UMIuQXG2k5IYE8qFiCl6qckvqv6Npz4FUHIJSRhSS+kLypZLYPVfl1BhyV+auhwXj4XV6Rkq+kMNjG0lQnZLVo0UqQX4muJr8mfyioUwyq/1vKepOrpucS/hBO/YjBBxOXXarUK9+R8Leo+Dy6dxcU0rKMmvj622+1cBps+b7fbhXH8VpIOj5Tdg0N+U3t7Jr7lrTg7OyuU+Lp6rkRR+h5/Fk78iiHk4PTbVC68nPM9iWZLNbXUWwNdUcciPp+32+2ciq7PdSCNdZ7SCPRrrViBeYY9WSRl0WTSsyuyaI+vFxIn/yyc+AuibJDJVSFBLp1OB9PpFN1uN1sEuDhmGeJbUl8TXx+vm68uBNUqv96jWxGDvBdPLZJWgQ9t4dcGSVfzZ+HEXzHoIBuRgAIOYLFaXZVR9VO9426CHGXrB1h/07kIcs+KOEx5IVzS23DiryA40u/s7CwnhTXpdb6+Jgm/J5C36msD2DJg+ffnhRvr72FpCjoCUdcBcPLn4cRfQfCPWrsJrcaVfF1k1Qcwsy9ehgHsqpLeglUjQIcf67+zvcFxDif+ikHH9ss9+RHP8wro99LXYmBjN9oyJL5F8EUTi+Q8lXOg9/W+v0/Dib+CkB8tn0u9OsvVpa+tH7qOM7CkZJUEmUf0FOmtACS9fUlJfFnEePF0aW/Dib9isEgvxLaCW4DippMpqW+NZZGjDOkZEhugXYgpw54muxN/Fk78FQSTX+LT5VxQ1lqeen8rQKdqLKLe6/tMdn2/aJ8v2X38escsnPgrBssifxfB25CiKsLaWMn5DZbk1rUKOEhnmd6Juw4nvqNSaLLrqsFWw07dx49ddPq8qIquozyc+I7KYZFfSM4RiNYCIPH4nNykKwDpmnq8t3eUgxPfUTmE+CzxuRyXJr+W+pIvwPULRLp7Tb1qMFc/CiE8H0L4cgjhz0II3wgh/MLF/YchhC+FEL4dQvhiCGHz5qfrWHWwh4FDipn0Wtpria/LlElBzXlVdJ345VFmY3QC4MMxxn8I4J8A+FAI4R8A+CiAP4ox/n0AXwbwSzc3TcddAqv6WtrrikNW5yBgtkxZmbr5jvKYS/wY4+sxxq9fnB8A+CaA5wG8G8BnLl72GQA/fVOTdNwdsDVfS/wiVV+OujBpivjeKed6WMgUGkL4YQAvAPgTAM/GGB8D54sDgLdUPTnH3YR24S2q6gOXEl+r+sPhMFee2/f4V0Np414IYQ3A7wL4xRjjQQhBR2Iko0gePXqUnW9tbWFra2uxWTruDHh/z4SX6sFSMVhqDLCkt9qBp7LxPAFnFtvb29je3i712lLEDyG0cE7634gxfv7i9uMQwrMxxschhOcAfD/175n4jvsPJr2UDhuNRjg8PMTh4SFGo1FGfiG+rrbD9gFZOLgAqbVo1B1aqL700kvJ15ZV9X8dwJ/HGD9B974A4AMX5+8H8Hn9jxz1gxCXiS+SXogvUl/Iq4uI8DaBtwia+Ez6VK6Cw8ZciR9CeCeAnwXwjRDC13Cu0n8MwMcB/HYI4ecAvAbgPTc5UcfdABv2WFJriT8ej3MEFoldtFVot9s5TcEl/tUxl/gxxj8GkPKV/ES103HcdWjSMvEPDg5wcHBQqOoDyKn6Eu0n3YRFU2BV3yX+4vDIPUel0Kr+dDrFeDzOSXyt6lt7fEviN5vNmT0+J/k4ysOJ76gcQvwiVZ8lPhNXR/0J8cVlp/f47Pt3iV8eTnxHpbCMe9L48+DgIGnV18Y9fg9OwrGs+lbZMUcxnPiOSpHKzLMi9qyef1acv2gPIYRkgk9K5bcKeTic+I4VhBXyKwE789J6T05OktWFnPSXcOI7Vg6WkY+JbyX6MPmtQpteqCMPJ75jpZAivUhrTXpL4kv7Li7Mmao+XFc48R0rB90s5OTkJPtbqnQXX0sjEilUOq/0eB3hxHesHFjii6ouxJ2X3SeNQ7kZifx7xyWc+I6Vglb1ualGjDGZ3qsLeQCzrbodl3DiO1YOTH6R4Bz/b0l7Jr+ASe8SPw8nvmPlIETX92KMOZLzkYdAdyJy8l/Cie9YKbABTvcHBGAGB+myXgLuO+ikz8OJ71hJcCTf2dlZrlOwkN6q53d8fDzTcZgjAx3ncOI7Vg5cdovj90MIubZbbMlnia+barqqPwsnvmNlwVJfpLgO5WXSi+Tnlltcx89xCSe+Y+WgOwHznr/MHl8q9LZarVwRT8clnPiOlYSW0HLN0j5Vs1/aZTPxXdXPw4nvWFmwqg8gqeprdb/dbpslux2X8HAmx52Bdu/xIqAHx/s7ZuHEdzhqCCe+w1FDOPEdjhrCie9w1BBOfIejhnDiOxw1hBPf4aghPIDHUSk4M45j5iWSjgf3ueequAIr6Iar6vDnWEO/n+MSTnxH5RDitVottNttdDoddLtd9Pt9TCaTLNKu2+3mWmQ1m81c6SxglvycbqtHu93O3keGLwA2nPiOSsHE1MTv9Xro9/u5+HohKxNWJ+nwOUv0Im1CSM/S33GJuXv8EMLzIYQvhxD+LITwjRDCv724/2II4bshhP95Md5189N13AWEEHLE73a7GfFldLtddDodtNtttNvtnITWpOX6+Ez6FPktiS/zcpyjjMQ/AfDhGOPXQwhrAP5HCOEPL/72cozx5ZubnuOuISXxRdpLBl2328VkMkGn00lKfM6ok/OyEt9aOByXmEv8GOPrAF6/OD8IIXwTwFsv/uxP05HDPFV/MplkgyU+q+dAPiGHr7XhUP7dPInvxM9jIXdeCOGHAbwA4L9d3PpQCOHrIYT/FELYrHhujjsKrerrPb5W9TVh56n52rgn/16/l1v30yhN/As1/3cB/GKM8QDArwH4uzHGF3CuEbjK78iR1LLqC+mF+EJ+LaVTrrkU+XkBsN7LyZ9HKat+CKGFc9L/Rozx8wAQY/wBveSTAH4/9e8fPXqUnW9tbWFra+sKU3XcFXDpKyH9yclJ1iAj1dc+xohut5vl27O6L2NtbQ3r6+tYW1vDYDDILSa8beB9fl1Iv729je3t7VKvDWUqk4QQPgvg/8UYP0z3nrvY/yOE8O8A/EiM8X3Gv41e/aQ+mE6nGI1GGI1GGI/H2bmMg4MD7O3t5cb+/n52PplMTMLLGA6HeOaZZ5JjOBxmC4AsAtqOUBdcVCk2V7y5Ej+E8E4APwvgGyGErwGIAD4G4H0hhBcAnAF4FcDPVzZjx52F5cqTKjgccSfaQK/Xw2AwwNraGjY2NjCdTk2jnlz3ej08fPgwGw8ePMD6+joGg0G2fbBsBnWR+mVRxqr/xwCsZfIPqp+O465D9uFCfCG9dr9p0ot2IC2wLPIDQLfbxcbGBjY2NrC+vp4dhfhiL2ADn5N+Fh6556gUTHwha8qvf3R0hLW1NRwdHWEymeDo6ChreqkLbcqx3W5jOBzmxmAwyEl8HeDjfvxZOPEdlYODcFj173Q6OD4+Rq/XyyriciOM6XQ6UxFXE7/VauWi/3Q0YLvdNt2ALvXzcOI7KgVLfCF9u93OLPe6BZa+1kQXyDVrDinjHbv+2K7guIQT31EpmHAi+RcdKejIPf4sKzTXY/TTcOI7KoVL17sBr8DjcNQQTnyHo4Zw4jscNYQT3+GoIZz4DkcN4cR3OGoIJ77DUUMsnfhl84VvCz6/62GV57fKcwOWOz8nvoLP73pY5fmt8tyAe058h8Nx+3DiOxw1RKnSW9f6gBC87pbDcUtIld66ceI7HI7Vg6v6DkcN4cR3OGqIpRE/hPCuEMK3QgjfCSF8ZFmfWxYhhFdDCP8rhPC1EMJXV2A+nwohPA4h/G+69zCE8KUQwrdDCF+8ze5FifmtTCNVo9nrL1zcX4lneNvNaJeyxw8hNAB8B8CPA/gbAK8AeG+M8Vs3/uElEUL4PwD+UYzx6W3PBQBCCD8K4ADAZ2OMb7+493EAT2KMv3KxeD6MMX50heb3IoD9VWikGkJ4DsBz3OwVwLsB/CuswDMsmN8/xxKe4bIk/jsA/EWM8bUY4zGA38L5l1wlBKzQ1ifG+BUAehF6N4DPXJx/BsBPL3VShMT8gBVppBpjfD3G+PWL8wMA3wTwPFbkGSbmt7RmtMv6ob8VwF/R9Xdx+SVXBRHAF0MIr4QQPnjbk0ngLTHGx0DWxfgttzwfCyvXSJWavf4JgGdX7RneRjPalZFwK4B3xhj/MYB/hvMH/6O3PaESWDVf7Mo1UjWavepndqvP8Laa0S6L+H8N4Ifo+vmLeyuDGOP3Lo4/APB7ON+erBoehxCeBbI94vdveT45xBh/QI0SPwngR25zPlazV6zQM0w1o13GM1wW8V8B8PdCCG8LIXQAvBfAF5b02XMRQhhcrLwIIQwB/CSAP73dWQE43+vxfu8LAD5wcf5+AJ/X/2DJyM3vgkiCn8HtP8NfB/DnMcZP0L1VeoYz81vWM1xa5N6FW+ITOF9sPhVj/OWlfHAJhBD+Ds6lfMR5yfHfvO35hRA+B2ALwJsBPAbwIoD/AuB3APxtAK8BeE+McWeF5vdjON+rZo1UZT99C/N7J4D/CuAbOP9/lWavXwXw27jlZ1gwv/dhCc/QQ3YdjhrCjXsORw3hxHc4aggnvsNRQzjxHY4awonvcNQQTnyHo4Zw4jscNYQT3+GoIf4/7Ui6+8WWn0gAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f668c9b69b0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(mnist.test.images[99].reshape([28,28]), cmap=plt.get_cmap('gray_r'))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\\text{evidence}_i = \\sum_j W_{i,~ j} x_j + b_i$$\n",
    "$$y = \\text{softmax}(\\text{evidence})$$\n",
    "$$\\text{softmax}(evidence)_i = \\frac{\\exp(evidence_i)}{\\sum_j \\exp(evidence_j)}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](https://www.tensorflow.org/images/softmax-regression-scalargraph.png)\n",
    "![](https://www.tensorflow.org/images/softmax-regression-vectorequation.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$y = \\text{softmax}(Wx + b)$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the model\n",
    "x = tf.placeholder(tf.float32, [None, 784]) # image\n",
    "W = tf.Variable(tf.zeros([784, 10])) # weights\n",
    "b = tf.Variable(tf.zeros([10])) # biases\n",
    "y = tf.matmul(x, W) + b # model\n",
    "y_ = tf.placeholder(tf.float32, [None, 10]) # correct answers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define loss and optimizer\n",
    "The raw formulation of cross-entropy\n",
    "$$H_{y'}(y) = -\\sum_i y'_i \\log(y_i)$$\n",
    "...can be numerically unstable.\n",
    "\n",
    "So here we use tf.nn.softmax_cross_entropy_with_logits on the raw outputs of 'y', and then average across the batch,\n",
    "because this more numerically stable function internally computes the softmax activation.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "cross_entropy = tf.reduce_mean(\n",
    "    tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Gradient descent - find local minimum of a function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "sess = tf.InteractiveSession(config=config)\n",
    "tf.global_variables_initializer().run()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Train\n",
    "for _ in range(1000):\n",
    "    batch_xs, batch_ys = mnist.train.next_batch(100)\n",
    "    sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.9176\n"
     ]
    }
   ],
   "source": [
    "# Test trained model\n",
    "correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))\n",
    "accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))\n",
    "print(sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels}))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADQCAYAAACk0bIFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzsvUmMZEl65/cze7vv7rFH7lmZ2VXVrKpms9nkzHCFlhEogIfhQSNodJEAQQJHECBgoIPmIgrQQXPRgRRG0AJBQ110kEaUZjCEhhI4YDfFJtldXVtnZeVSmZFLLB7h4etzf4uZDvbM38uo6uqqyqiKojo+4ME9PJ4/t2fP7G/f8v8+E1przuVczuVczuXLEXnWDTiXczmXc/lpknPQPZdzOZdz+RLlHHTP5VzO5Vy+RDkH3XM5l3M5ly9RzkH3XM7lXM7lS5Rz0D2XczmXc/kS5Rx0z+VczuVcvkT5yoCuEKIrhPjfhBATIcQDIcS/edZtOmsRQvy2EOLPhRBzIcT/cNbt+SqIEMIXQvx3QogPhRBDIcT3hRD/2lm366xFCPGPhBBPiz65LYT4d8+6TV8VEULcFELEQoj/6azbAuCedQMq8l8Dc2AN+CbwT4QQb2qtf3S2zTpTeQL858DfBKIzbstXRVzgEfDLWusdIcS/DvwvQoif0Vo/OuO2naX8F8C/o7VOhRC3gD8WQnxfa/2Ds27YV0B+F/jeWTfCyldC0xVC1IC/Bfx9rXWstf4O8AfAv322LTtb0Vr/Y631HwBHZ92Wr4porWda69/RWu8Uf/8T4AHwc2fbsrMVrfWPtNZp8acANPDSGTbpKyFCiL8NDIA/Ouu2WPlKgC5wC0i11vcqn/0Q+PoZtedc/oqIEGIDuAm8e9ZtOWsRQvyeEGIK/Ah4CvzTM27SmYoQogX8Z8B/jFmIvhLyVQHdBjA68dkQaJ5BW87lr4gIIVzg94H/UWt956zbc9aitf5tzFz6JeB/BRZn26Izl98B/lut9dOzbkhVviqgOwFaJz5rAeMzaMu5/BUQIYTAAO4C+A/PuDlfGdFGvgtcAv6Ds27PWYkQ4hvAvwz8V2fdlpPyVQmk3QFcIcRLFRfDG5ybjOfy4+W/B1aB39Ba52fdmK+guPx0+3R/FbgCPCoW6AbgCCFe1Vp/6ywb9pXQdLXWM4w59DtCiJoQ4m8Avwn8o7Nt2dmKEMIRQoSAg1mUAiGEc9btOmsRQvxD4GXgN7XWyVm356xFCLEmhPg3hBB1IYQUQvxN4G8D//ys23aG8t9gFp1vYBS4fwj8n8C/epaNgq8I6Bby20AN2Af+Z+Df/ymniwH8fWAG/CfAv1W8/0/PtEVnLEKIy8C/h5lMe0KIsRBi9FPO69YYV8IOhunyXwL/UcHs+KkUrfVca71vD4wLc661PnMmkDgvYn4u53Iu5/LlyVdJ0z2XczmXc/n/vZyD7rmcy7mcy5co56B7LudyLufyJconUsaEED81Dl+t9afKWDnvk4+Xn5Z+Oe+Tj5fz+fNR+XF98hN5uvrv/b0vojXmUKp87zjmEMIc9vM8N+eB+VzK8pxTEvEP/sFnOv/3fu/LHTf2du3tK2W65YuU3/7tz96/+vd//9OfbG/m05xXPZY/Vowf2xl5DmlqPvd9cN1yTNnz7XFS7Dj8SU35O3/nU95cpZm/+7uf+TunIrZ/bT99kT/1d//uZzpff4bBqwGNQOWQq+IDQEhwiuGTK8z/i3OyzPztuuC45tV1+cg1pATpFK+CT5UoLIov/6RThfPjmZ1ffHLEx4GjEGYgWAQRopwgjlMOFgu69iH9uMlXfT35/ismn6aZJ0HWHrZ7LL5YzPlxWPJx1/3KyEnQrT5HrT/6jD/uMwu4WWY6JMvKBdzzylf7PdthH3fdPP9oR36lOuyE/KR2VkH3x33vkz77ionSYql/aV1Ah2PeJ0n5+NMUFgvz6vvPHxZK7CN3XTM87Pp8El4+2q0aEAg0Vof9PCPkdEH3JCCe1FI+aRC7lSXJ88rvVzUa2+NVpPmk/52UT4tOX7BUm1htjsUD2wWeZwaL45jPfB+CwAyo+dwMrsWiWNkrxgB8PH6cKYactGBOShVE87xciC2o2pliLR27IGeZmXXV79gOdl0Iw3JGVRdwKK2oPDfXzTJz2Paetfy4eWPvvboQVc0gi0qu+9EB8XED7wUtgC9DVKHBnnyEaQqzmZkPSWJeJxPzaudLFJlhYB+17dIwhHrdnFOFH6v32QPKLpICkGKp8X4eOT3Qrapl9u/q53bg289OHvYcuyzZO7f/tz1WBVn7mV3mqpPoJPhWAfkMxP7syaZX1w47mHzfvFplzQ6GIIBazQwu2yXzebnC27lm/2fnZFXpObkmfilSBcvqM6gCgAW8+dwcAFKiC9VFFxaRLsaYAESWISzwQtlJQixVGF2rle8zc64AECDsuLHHYvHlj5Ef5/aoLjK276BEn/kcnSRoIdCV85RSKKXA8xC+j5ASIQRC6+WBUqYPqte1IG2BurpIfZw1+SWJ1oWLoZgjafr8dJ/PYTwugXY2g8HA/G2hpFYzx8n1qNGATsf8LwjKw/fNkJFCl3NGg1YaLeE5r/TnmEMvBronbeCqe8BOMOtGsAhyctbb808+YClL8K0edvkSorQl7FJXfSJWA0rTUiOyckYa70lruNpceytKmRU4iswqXK+Xt16rmb+tojOdwnBYDjilyu60Yr9rNefqHP641RxOGYyrgHuyM+wsyjLzDGcz2N83RxxDkqAWC9RiQa41ClAFyDiAD7iOYzrM+nFrNdjchO1tWFkxs6peR4c10kSRpRrPUXhOZQZbk2EyMUeSlNq17Zwv0i9qVTT7MCywWqSxZg6Y84ZD2NlBHxww15pEa0TRv1OtmWqNcBwcKQmEIBQCD3C1xgUcIRCeZ/rKqoGtFly4AOvrJfhW7faqQvNliv7o2hzH5XCZTsvHNp3CaAQHB3B4WI7net0ALJjPwtDceqdjhkizaf7faJTvHQeEB1IrhNZopRFKIzQIhVF5peTzoO6Lg251hTypm+e5eXAWdK1/zQKyEOVkmc3MA65et6r52mWo0YB223xnsTBPYDQyPW4nT5KYI46XkxcobYsquH/BclLDrQJs1UUwn5umZlkJtkpp5HJd06hcFDggEKI0pYZDc/uFgoPrlt1nH4ttx0kDomqB2kdyaqB70vqpzh47gW0nHB/D8TH6wQP0vXvm/WRCNp+TJgmZ1mSAjYN4QuBIiRNF0Okg7ExptUwHhyFcvQq1GrrTRbe7ZHNYJICrkF4OSQppgpjH5nA90NpMo6padNJEeBH5OI12PjdjuFBCdKEw6CwzRwGQWmuYTNC7u+h33yV/8IBpljGzmiumWveAolgHpspLC7PtSCgEQkqklAZ1ej1zdLvorS3Y2ETXGwir7i2eH6DCBhK+qLnzMdfVCFRFR8rzcsofH5tXC7qjkdFynz2Dvb3St2sB1T7CVssA7uqq6fpu17zmuZk/YahBaaRSCJUXx4n7PmkVWPkU4+R03AtWY602xA5Uq91WfWcW+CwyBEGJRha47d+NhlmN89zYEXb1tdpu1bd1UtsOQ3M9u1rbCW7b/CWZknaFTpJyDFfXB9sMq5GurMDmhubSJc3FbYXIUkSW4oQ+Ti1gXqw1Upa3Y9ek9XWzJtm1zP6+1bDtb1pNoRpzqgLwC8tJDbeK7FazsytNHEO/D/0+6bNnpP0+yWxGkiQssoy51qRAihmwIRAVwJ2nKf5shmvV/CwzHeF5Zuy4LuL6dTMWcg+tXUYTQTJ3IAeUJHJdakGI35S4GoTVNk8qAacpdgXOMnPvu7tLiy1fLEgXCxZ5zkwpUsch8zxSrZknCYvJhKTfJ81ztNZFeMccCnNbXvF3BiRA4Hk4QYDruub+Oh24eNEcly6hL10huXyDbPsSbuDgBg46zdFZhtPfR/b3YTb94hWWE35nIRykcJZrt51L1pUwGhmt9vjYwMPxsQHe6dRcTghzXhyXel+t9rweZ9cY6+Ot+xnebIIYxZClz1sdVqyFYL980sT8BDkd0K1qu77/vMe6StnJsnJm24fnOOggQCQJ2gKm65Y2hNWOs8z0qpSmx+t1RBQ9H6G17ciy8jpRVJpH1SCL7cgvGHSrvqg4LrHGarfVdchayqurcPkKvPaK4pWv5TAz4JQFddLAZzAUHB6a785m5rtRZFbsr30NtrY0UQS+B1nh8h6PYDQ2WvFwKJa/b2NVNpjwcQHvzyUnXQpVCydJjGoyHi/RX+/twbNnZLu7zA4PmWUZU2CKqfKzwIBuBPSKSyqt0WmKE8c4acpS/QfzO75vqDv1Onp9A4EAXEZjweGhwOQGeXS7ApoGbB1Hgyps2Crb4TR9LpVBoePYoMajRwZ8Dw/JZjMWiwVDpRgUfZAU/TDEFJm2n0VAgAFYidFsG8XfTtFnCaA8D7fRwPV9cy+9Hly6BLduwcsvo67dYNG5yKK9ThiCCCq377jIeAppUvbvacrJoF4lKC4cgXQcpNQG07RVHsQSdIv1mtHIDKnRyAwr65u1UBJFBnDzXBcabQG6hTEdBhZ0U8RoiDg+LpW8k8yXZtOozFUV+lOaiZ8PdO3FT/pZq3atBcuqLWtDjbNZGXYfDkEI1JMn6GfPUK5LbrXTNIVaDdFokCpFslgghMDzfbxmE3d1FafVMr/tOGUU2wJup2PUvqpvTOvy9+29nIJUg8TVz6yHxboBhsNyUCwW5pxarQyy2yYKAUkumc3Bw8cLFMJzkY5ZxZ8+NfP0yZPSmta6cGGnCt9PqDsJap6QL1Ii4dKqu7S8iHY7xHXF0rK1bu9TdS/Yi1U7x3aQ1mWn7O/DwYHRbvt94tGImVLEmF1KM4w7waXUch1KUFFaM01TpkqRAjrPcbTGSxKiLCOYTEzgTUrc3gZRb50V3yVacUm0S6rc5RrvOj5BrQHzYjZXI5SnJVqXK+5gAIMByc4OyZMnzKdT5nHMvNDuFxjAtF7UALPgdOz9C0HoeYSFSSMA3/MIfB8xmyFms2XbRbMJV64YM6peN4DR6y2BI/dCxrHDYF7ijL3tC7LO9toGrhSlmnnaGm/V7VT1w0mJkA6OFvha0ghd1KpLEAjabTO9LUPBBtTGYzNH7Ji+eNFgpD3/wmbO5a2MdkdQazlEzAmSEfXRjGixQMjCL2Etsjg2F7UXnk5NX169auIHYVgyrj6FfH7QrZryVje3k6vqaqiq3rZHbOPtw5vN0Ldvo95/n1QIlgVSiwCBEIIYGGuNBOpCUOt0kJcv41y4ABsbplcXC3O9Vsv0brdrgLcqdjRVNd5TkKoiV/V2KFXGaI6OzIp8cGC6YLEw3WPHfhSZw14vzQTThUPd8/BCENJDSsF0asB2Z8e8ui6srZVjVaUKL59TzyewGEM8RwcBuhYybsPEDZdz/+io7DYb8zwV0P1xVoT9zILu3h7cv08yHDIZDpkrxVzrj4CugzGZg+K9LA6tNdMsY5FlxECeJISzGdFwiBwMCPb3i7hAgHczwav7hH5ItxEwUyFT5S7dNX7dQ9cbMJ+YmWwVgy8CdMdjs+A8e8bi8WMmT58yzDKOi3tfQKGXm/sUxb03MdptE2gIQeT7BGFYXr9eR9TrZIeHZPM5szxnBshWC3H1Kly+bEypKDIPvNmEep3cC5lMXHYLc30wqEzzlxpsXt8ElZp2V9lBp9kvVcaRDXwUk8mRDo7rIP2AIJK0O5LFwkxzS1axQbVqcC2O4aWX4MYNM0dWV2Gzm7PZmRNE0jA8RlN48gxx2EeOh+b+CtfUMjq3t2dcQIWSwM/+bOmrWFv7TLd6OoE06zS0/ttqEMz6PKo8KKvpTKdL7VfPZqgkMW42ykmVYybevDg0ZkAuhkPajx9Ti2Oc42NErUZeXMMJQ5xGw/R4FJlOWVsrw/fVyNIpDZyTFpJ119nV9+jIgq6m39dMpzlZtsB1BWnqs1h4rKyYZ9jvg+MIBgN48EDjSQffgeFEcjyGnZ2cDz/MC1+WoNEQxLFEa2kAM5eElwPCQEPggkzRjksuPUaLgOPCBLMxxpP0slORqupvO0Sp0vG2v28CQv0+6vgYEcf4eW4YChiAbQG63UatriJ931Zyx5MSTym8gjLm5TmB1kRCoBwHz/fxowiv2TSz7MYNxPYW9LoGlArnnkhcKPzbR0eAkjQbLtLxkI5rXBM2PvGi4+REJEgfH6P39tAPH5IcHTHLMuJCu3WANqV2rzFzQGA0/ZqUNIKARhDg1mo4vk88mxHPZkilkElCHsfkhcUQA85ohHrwgHw4JK/VoNvF3dhA5jncuYP7ZJ+Ou0Em10nTNUbOyhK4htsBaahxojoyjBBVdscpzB8NIIq0sDw3tLY0henUUAKtNuC6OGENvw6OG+A5knlDUq+D7wuGw9KQns0MNvb7mslE8fSpotGQNBqSjRXJ1mpAvSHwIgc/i3Ana7RkyHprlXZT4TUCXKFw8qc4h4dwcIC+fx99cIA6OEBubCAuX0ZsbZWmoh0rP8G3++I+XftDlkpifbaW4W91f7uyWvC10ZzitQq4VdBNMSA7L16T4v+L+Rxvfx9/NEIcHSF9n2w8JpvN8IXAsSr/+rqZeOvrZfDNahsnmdYvICfHn2WyjUbGpTAYGA3CHJo4TtB6guMIZrMWi4VHEBjFY3/ffK+8tgQt2XlstNvxOGc6TQpXk2BlxSVNxZKyLIWk2wtprAbISEGkyZUgzQSHQ0n/sOQy2m6oEktORapUKzs+kqTk9BSag+r3yUYjpFJYfU1TgAsgraO62TTBomLyiTxHxPEyRK3z3HB4fR9aLUSrZb67umpUnWvXSn5QoRRoJdGLcoJ6rmR9Q+ILD2EDTna8nAZlzPbDbGbYGbu75I8ekeT5EnBTjE+2h9Fs7RyYFf0SAKHjEEYRQbNplA3XZRLHHM5meHGMKwRaKVRxzRjwBwNUHKOkNBSzCxeQ3/62qXi1s4OHYGXrItHqZWbuz3BUW+HoqLDKFh6J7+KFdWQYQjwr4yOnMn+EyfASwrAEoFTKrK/DPvdmjut7OJ7Edz1qNUEUiiWZyQLveGxYDI8ewb17RnVzHBcpJRvrLtvbDp02NJqCelSnFoZcurDOG5uaKxc0tYYkyieI8Qgnz40mdP8+6uiI7OgId28P5+jITCIbqIdPFY3+7KBbvaDVGqsJERZwq+HBPDdo0++Xs72qZkmJjCKcZhPyHKkUTpbhZNlyhbeBAvvrSmvyLCNLEpw0RUqJ1Bqn0KKzJEE+fYp4/33E6qqZdJakZwH51KJGRqrKs2UKWOvEarxxnJKmC/J8BkzRWqCUw2QC/b6JrtfrZo0ybm2xdG/t7yfs76csFoos00gpcRzJfC45OhJEkcGU0ViwdyBwPNDaeY4Sa60je9jHVY0xvhDwVpHbvlqgmU7L8HK/D3t7iCwzfnl7vu8jwxA/ighqNeTqqrFS6vUySDqfl8/SdlbhGE/DBpNghbTepbFeo7ZaMOMt4FWCIt4CojkwC5lNQkZjyfEQZC5pSA9px/aLjJMfY1FppZhnGXGSMMYEx3IM97guJW3HIQhD4yqQkjDP0UU8ww9D/EYDGQQQx4jpFD9NaShFggFZS61zHYeO65oNwuIYEYa4rZaJlcSxGQSDAUIpPDSR67Ny8Rrbq+YRTacwOJbsHcDazKEtXdwqsJzGKi3MvNYaUPnz/lQLwgXWKGn88GnssMgFBwdi6bLb24OnTxWj0YLhMKXfFwWVcoxSYyx6JInDZOpQq9UJwxaNhke77TFZCJwajLMiccLL2XJ6rG1sGD9GGCJ8H6dIPFmyZKrMHMui+QT5bKB7kgJkfbcn8+aqoOv7BnWePDHRn/19sxxZUmrB4XXqdWSvh2s5TXGMyHNSrXFgebiUQZQcyITAdxxEEBjit+OQTyakiwXu/j7ue+8ZT/rNmyXYWj6V1WJOUaxSZClicVz6l4xinxYDYArEaC3Jc0kcw95enfHYpdMx4GkXect6mM0S5vMxSpnUAMdx8X0PrSXDoaBeL6k0VZ6iteyzzHT/3l4JujaeYtfGF6KNnaSJWfXfgu5gYEB3ODSL8P4+st02i2LxPdnp4Hc6iLU1xNpamZ5r/Wd2FYtj8xybTfN8Nzeh2yWrrdDPNhi7XS5ektRWlUGP4bAMVxc8bj9TuLnCmXWZxz7jseToCHxPEkkX13LpXpQuZvuhwlvWUjIXgmNghAHdEOOvbTgOTd8n6PVgcxPteagieUI0m8hGw2icSsGDB8jRiMZiQQD0KQFcAT3PYz2KCLPMgG6thnf1KmJlxZju+/ssqTBhiNPq0rsVwxW4d8908+FhETtwJA3h4p6a4/95EdWF0XIqT3D1My9krgJGc4/hUPD0KTwuoOXpU3j8OOfwcMZgMCFNJXkuMLv17EFBPJxOPRaLACk3kdKj0XDp9cQS4/t9A7qbKx7B5RXWLl0y46vXQ2YZIkkQrVYZiPk0RZUq8tlAt0rNsiqSZQ5UDxtRstyogwNjF+/tlUkQlp+bpoZ07fvQbhuaxmxGohSJ1gwxZG8bxRUYf59fvDpam3TQNEW4LkoI1HxOGsdkgwFCCPxHj/AeP0asr5vA2snMuFOUajwASg2zTO5RaG3ZkwWDAkGWSeLYR6mAMBREkdFc2+3y+1JKhPCYzx3i2CMIHBoNiVJyyUucz80ksWw7myditGXN4aHi6EgxGklGI8nmplhq1S9sQVcHXBV4bSfY8ZAk4DgmoaFeR9g8zDA0o73VQmmN6vdBKWSeI+p1cz0bCK1G0W36XreLqEeEykf5Hm7DBb/gd+/slNaVBXghkFIs8wQ8v1jgcFF+xRo6DakmihTWQIrRSjOMFRdifLmNKMJfWcG1mXWW92Tdc7Va6SYpEn/c8Rg5GuFNJsjxeFkZwNOaQGs8rdFakycJ2Xi8nEckCcnxMXmWIZRCJCl+q0fbi+DZFuPRFkdHgr096LUdVN1/wZX5k/qIj+fa1+soP0R5AdMs4mjoEqeStDCDrZfDUC4FYegShmYeaQ1xnBHH46K35+R5nTxvE4aKMDTx9u1tE4+v1UxT8hzS3CENGqS9dZyr15GvG3+LODgwkbmtLVSrg3YDtPAMX1pLnJ+Qpfb5QBdKWpgF3aqmWw2UxbGJ+j1+/HxI1KZuHh2VCNFqwXiMjmPiPGeEWZ/2ip8MgTqFX6t49fIcGcfmhEYDgoDccUi1JhuNSOdzmo8e4X74oQmgfEmgC8+7NZfMAgVaWwp7SkkIclCqRp4rtJZ4nmBjwwwEa7E3GgHNpltMAkEYSno9wXgMDx+WFtnuboltlp1k34/HOZNJQpb5ZJkkDI31fmoMIEsVOwlWNjvDdkKjYUa6nWA2Eb7QLtXTp2QPH5qFptk0Wm+alkBoVZL9/ZIN4/t49SargSZ35/hZ02xH+PQp3LljzpvPSyphwbBprbhcDgVxsTAmuYMOQsiCUzKfxUdAV1OOAEnJSOgC9WYTefGi6R+bllsleAtR5rbajJjDQ0S/j/voEcFkskybdvMcYfPLlSKbTJg+eIAIAhpFUHsymxFnGe5ggLe7S5RmBPvHZDu/wnC4xWBg9KZLvkPeOWFSn6JoJMIWKbJHswmdDrkbkuIxOnLZO3BQsLQIbdp8twt57hAENRqNAK0Fea7Y3/eI4xyj6MwxKBLRaERcuOBy/brk1i3TjdYYdhyoNSSiFpE0evg3voYMPINXh4cmTnD5Mqq7QuZFKOmhVUln/CR5MZ6uTe89ycetaiKW12bpNxas2+0y0cFmmtlJ6bomWu26uEmCN5+Ta71kMthgWwrG/WAnsjVBh0MA4jRlkqZ4e3s07t0rSHoXShT8AhIjqnTDavUvG+NxXYkQAWbJWGCmnIuUDr4vl3Op1TJW862bmlDMiZjT0COaashxs8PBlTVSEaCFwRRrRBweGm16NsuJ45w0VSSJIk01SaJZLHIWiwwpBY7jkmWnPHnsGDip6drDrgyWZmgze2xKajFmRJIgjo6M78xSouyEtCB0MkvRcXCmU6K1NTMDj2umHc+emfO63ecDHsWY9X1oCFBF3EbpoojMyXH9efvDygnfk1ZqGTSsA60ool6r4W9tIS5fNskL29uoMEKliskg4eBhzGComO3XSI5DPC4RCbh0dcDWy4fwJ38Cjx6x0NoE34rfC5TC0RqVpmRpipxOSXyfTAiOkoRxnhPOZtTmc7ydhwRRm7b6GTa3zFi0a52oWjKfs09OzjrjVTCBYK0cdCZw3DpOKyPz66S6QZp6pNphnklyDZ6nqdc0joTVFcFsJmg2YX1dsFh4zOcegwEcHWXEccLBwQCtc0DSbtfo9Va4dKnJtWse164Jrl0z91nN/q7VBMJzmYsIVrcg8MmPJ2THE6a1VWbxGgwbiNzDjxwTwpIa91TdCyc1mOqgrH5ukcayA6xma23YIDATLIqMJjwcGtTY3zdmZ6tFFEW4YYgzHBL1+4zSlDEGdC2tRlMS532bT95qIQ8PkZh17RhoHB6i79wxfpmXXy4z1E6J8vJxYgNplkni+0Y5SVOPOG5U6MEZEOK6NTodn15PsroqWFszsb+f+bqmmw7pZvt4D+7gPrxDevNVFt/8azya9PjgvkOWCZN1tszGUaTpnCybo1SKUhlK5Widk+eG5eq6Lr4fLd3bp+amq2p0VbqYfW8LRlif2Pa2OTodA4pF1ocIQ9xaDVGvG5/v+rpR+y1/Nk1LRrzVdPt9o9HevGmuCaVrw/NYRhptNoq1Syt+FSFAoEqqkv3wRaXaF4sFYj5HpikOZRZZ1OngX7iAc/26Ad2XXoLr11GtHgsR8HRH8qfHOW89jHlyd8DxJKFZX2dzrcNv/itztv76EHVwQPad7zAAdoGaUgy0po5hg9QwWrWjTUrxRAj2leIYo2m3taaeZdTSBRcv5Pz8NnRNeQYaUYZMF6dS/lJX9EGlTU2MxQLyTJJlgshpEnYDJnOX0bFHpiQKsaTh1yJNp6VIQ7h4URKEgiQxXRxF5vW99+CddzQHB2PgKYaE2GV7u8s3vrHJjRttrlxxWV839+c4RnHJ8+dTD+LEgbCDDmvMgpRZlPF4P2DncUBQ96i3JatrpVHykxDl04OZEeDEAAAgAElEQVRudWWruhKWvVj5qZP2tFXzLOHYqnPWJ+z7JhEiTQ0Yb27irazg9XqIgwP8x4/xDw6Q/b5hK1DmmOdArhQqy4wZNZ+TZxkpJlR1DKzaQiovv2wCdKWd/6lv/5OkGqCuarrVeVtaTQ5BEJLnunAl5EgZ4nkhtZpPuy3pdKDX1WyvJFxbm9N5fI/O07fJPrxNeuc27rrCb71EKnw+lA3SVBYGg2YwUEynKUrFaD3DgLpNMbCkI2+pVVt3arVOw+eWqpZr/fq2EzyvrORTq0G7jV5ZIb90jezmK4hWA9FsoP0GWgcI7UPURoU1spUNks4688YGifZIFwle1KN9uUbYO8BZxMj5FGcyRA6HZvG24w9KQLeJOieCwRKJI0qFOnQUIq/k3J+WVANFSYIvJY0wpNlu02y1ENdukF+7yaR3kWl7m3F+kUn/IvGkzsLRfLif852DjB/uZzx+nDAYzGi1HC6nHb6pU1jzUfU6eeEvtlQzCurYAjNfJJBpU53sGDNHJhiN2xbDkb0O65dDXrlVlkZspTmOXfBOg5+rxXNdYt1gaSrIwoAsChjEcDQoFZfAzWmHGQ0xozYYo9KMDR1Sr3uoGjhS06yZSdevN3CcACGmQB/fbxEEG1y7usYv/nyTl657bKznRDXDOMiVxPNK3dDy7JNEEkURvh/Rn0J/AI/3jMe01YKVuYkHdLugLW3kE+TTgW7VbQDPp/l+nDZg3QT24diUkSAovdW2spItcmN1+5dfhldegevX4fp1vP195Icf4vzlX1L7zneYHxwss3UCwFMKnWVksxnO3h56MGA+HjOE5RGPx6g0NT5lSzf6glgLVc9FlUliuy0IJI2GAHxmsxpKKVw3xPc9fN9Z1sxtNRQ9b8xaeoDzw++S/uH/wXhwxOj4mObmJt0PPwQdQRKQZV5BCFekaYpS8wJwZ5RkOw/zuI2+E4YenY6gYA8tsxhPrWtsR1gep430WlWkXodmi/mFl5hcfh0n8nEjlzxYJ1u5gXx1ihNPmGcuUxXRn4bsHkccDiSDI0U3mvPaz024sDIndDPC2RHBw/fx95+UM9iOO1vHz7olguC5ZHwnDwhSgRcYD1R9kePOkudLg57meCmCg40wxO/18F5/He+114gvfo34wi0eDdvc263xwVsR7z8OGcyG5OIRk9kBe3tjDg+HTKeHKKWYTuccH86ZP1rAnQm63yfTmhqwiVlurVtugVFENGZUJBimg62m0AB6rku0tYX4+iu0rqxy6SLUirWycZDj7C5OtU+qGb82YGzjEZNJyTCs14uSjG5CS4yJjp7iPbqLnsas1Hq0/HoRyFJ4+3PiGcwf3GJ39zKTyQIY0mo1WF9/mVdudfn5N+Di5pzIy5jnLoM4RAm5LFK3u2t+28b1osgMn3v3zGFjJWCG12dZn38y6FY1lyp7obq9gZ2pdpBWB6vlrdlqYdZE3N0tWQzNJngeem2N8bXXGV/6JvP1a8Sd6+jsCGZXaV7y6b08odZ5yHQ6Rc1meLMZXpIglCJbLFBpSi4EU6UYYlbvGTBfLEgWC+RwiDuZlFVeTlnbrV7KdtHJGKPnCcJQkKYui0WAUhohPMAhyzSLRUaWSbRS+HlMMz1i+vAOoz/9LoM8ZwCwv09rMEC5U7Ikr5Q71Sili0CdLh6fi5QejuMXRw3Hieh2jQuj3S5B99RjI3blKawb7fvoqE6+usk09ZlkEfvJVfYOL+J6AtfV5HmXNNVIP8WJUubznPFYszvL2dlL2d9XHB5qVlZqzK5vcbUdEEUBHTViO9ygt/oBztEBzvjYFElxpBl3wmQ7Vcu76Zrh+Dq5R+AIHJnjyxwvm+PMiyDwqdA6PirCcQjbbYJajfSV15l/+9d4Fl7niXeV2/vw3rM577w75623xhwd7QGPgX1ghpQzPG9Oq+XRbC7Y6o1pHD+FH+ygnz1Da21qDWOANaHgwWK033nxd0qZ/Rd4Hq1ej/bFS7g3Xya78QrB+hqdLjRrOY26wh0mxr1ga5y84Pyx3pZqXWlbrdXOpzgumAnkhDKjnh7TmD3Df/A+vPUWDIc0VlbKgrlF3CCdeEzu13n6dIvRyFRj29xs8MYbW7x+KeaWuM9KokAEDFSLkXLRnklQAgOoR0fFsyq0bCHgwQPjwbLsQ8uaWBIuPkWM/tODLnyUp2tDhlazjWPTUrudgZ1wUWTcBtaXVvXzVgoP5GGdu9mrvLX7Kg9uB3x4HJOOfZhs89eu/gq/+VvX2Zrfw7l9G/3BB8i7dxEHB4YKozUzrYm1Zqg1U8zKrorXCSDiGHl0hOP7ZRtPeUJZsA1Dc3kbJ6rWUTddajRQrdUy8KVUznyuDW3MEUwuZ+g05TjPeYpZQGZAzXVRUUSWB0xjSZ5bFpFkPndZLEJAIEQdz3MJApd63aFel0SRu+z2brd8LCct78998yeTZywVrNVCN1qkW1eY5QHvve/wo/cd7jxY4f2BRogMKTOUyoqAxwAp+2RZTJIkzGZTJpNjZrM583lGFDV49GiLVmsb193i0lrIr7/+Em9cW6N2PabGlDAZEcTDkpxsx2zF9NBegOO4BBKcJMFbTHHGA8TRIUwnHy1m/qJiZ3CjYZyAtTrDC69yEH2NP3u/y3fecdl5csDe3g79/j7T6RGlLroCXMX3a6yteVy8GPLGG12+cU3wxgdvwh/9EfLhQ1xtag+nGGuwg9F2bSzE1q+IMAG8JuC023R/7dfwf+mXSS6/zHTrFougZW4/SXHkHDGv7ItzCpqujatayr4trzwalSAWRUX9W29OT46p7T7A+eBduP0u3L5t8MYOXlgG93O6HD+7xOPdm8xmDrDOKy83+K2/JXjl4EfU/vE/hdUGvPwy7sYNomaduRc9t/2PrewJZdx2sSghrdk082d1tfB5N0zFsp80VD4d6J78235mB0+1LureXplJYjmF1QCGzY+1USZ7B5cvo7Yu8+DtK/w/b1/h7XePePfdPkkSAHWy3/oa3/7WG6wH9wlbXRwpYThEzWZkeU6W5yyyjHGeM8YQzi0HMsaYUV4cEx4elu2wT/qU5CSpw7JebHVJW9nSaMACKQ3o5vmCPE9YLBbM5xrX7dCuBUxHOTpJGOY5O5gFBCD1fXS9jpqHZMo8YWMpC1zXQcoAIbyiax3qdYdOp6TY2Or41ZjSMjr9AoH6j3yxSg3sdlHtFY6ddXbTHm/mC/5kd8G77ya8885eMX9tlYEMEwZ6jFku5xjP4wFm2cmALu+/H2NcJy1u3GjS2dyi+zOXzIJSTxDpAf7oCYxGiPv3zU2vrCxBVEkHJV1yZbL23GSBOxviDI/gsF+S820S0GmILDTvTsdYfZubjFZv8ERe4s2HLv/XH8HBwYTF4ilKPQEO8LyAWm2TwF3HZ5VWY4VLV9rc+lqNX/12wi9c3aN55ynqL/+SbLEgKZg+YIB1peixSfF8Qq3xPA8ZRYggIHd9uHiN/PVfpf/N32Dmd5i7bZOQlIFeJAg1RcazMoj5IlruiczN6gYVNs5q507NT+k1M5rzPs3BM5z776J/8Beo27cRDx+ij47Is4xcKXJAOw5Oo0ESbZPM98iyBKWaCHGNK5t1fvUbEzp/+B76j/8QtbaGcBzcoEPU3kL4OXmm0Sn4QhN5AqRAa8EiFaSpsFnmyzCBLevS6WhjMfqnQRmr+mxPprvZTrdcpceP4YMPTO/ZbCGtzfv9/XKHCFv0ZDIpye1Xr6IvX+HZnwe8+cNjdvcekmX3MGkQLQbH29y+c5nGSsglr01rYwOuXkV4Hk6W4c3nOAcHcHzMDJNQMcZMWQvCtfkcdXj4PMH+lNwLJzNFrTvBbnZRTbcVwtQDNQC5QCmbAnKMUorF4iVm003S2IDuJM/ZwxDne0AUhsheDz9p0eq5NPvlYm/cF5IgsGnBYgm01W1JrAZeBdwXFhs0rXaCTXjo9VhEK/zwhyF/+sOcd97Z4Z13HtLvH6B1H1gF1ihqh2Ge2AxjHGcYI7hTHLYcjn2vmM0yPvzQ4e23DXHhwqaDXG1RW0mQnqnXvOQQF4k5qXKZTwWjYteBzjxmfXGIY7cMskk7Ns34RcSuxpa5U6vBhQvo7QvEyQpHA8lwqIhjRZq6KNXCTN9N1tfrvPbaGjc3HLbzAevRiObWCr1Vj0vTJ7T+/B5y5y7zxYLDPOdR0TM9yl0jpONQd11TuSzPkauriFdfNXV0Ny5x2LjGW9NXuP8HXaJuQL1rCpJdvAhKZ5DFpwO4hSj9PLRU82fiuFRWGukxrf4+4YMfId9/m/yD98nu3kXs7+NOJqRZxjDLGCrFBEjznOZkAmrI1mXBL1/ocvfu17h3bw25N4Pvv0X2wQckgwFOFOGPx7jzMU1nRhTM0E5Oo5HTvKCJVwR4Psr1iBOX6cJld9folVaJKdZN2i1wHf0TARc+DehWQ/N2QlWjRnlOUTgA/eQJ+v59c06tBisrZmM8MBvp2SpTo5HRPop6n6rVJskcxs1tnoyGvPejI7LsEfAexhjqcjyUfHBvm7UsZCXs0trehtkM0Wzi5DnucIibJDjHxyQUAbTiMFUOTJEcfXxs2mcL75wC6J5MQ7ddZS1JO1+rBdnMrg0SITKEGKD1Y2APrTWLxQqTyTrpzNhecZZxiDEFG0AYBNDu4CQNag2HIDDMBK2NBh0EpvJYsynodku/rd1jzVRlKl3y1S18Xkg+zrEdhuhGA9XsMBUd3vlA8s/+WcbOzlOePHkT2AEeAbeAVxDCR0oXx0lwHIXjOLiujxAeWjcQBAjqICIQIUqFpKkAcnZ3NR98YFmJDr21JllL49bqyOoNFzVBUu0yi8usZHcWszrvl2X8lLJVr09H07VRmU7HdPalS3DpCvOdLqNDwWxmONV57gItpGzhODU2N9t8+9td/vqNES9nfba9Pqxqg+Nv/hDe/D7TJw+YJAkDjOe34fu0fJ+mUkRZhu84OL5vCsokCfR66J/9Jotv/wrTi6+xv7jCn/3v8N3vGhC5cMHc8tYWqLyo82e59C/oklsaNZRrkQ2omUPjCkXdz2kM9mkc3sZ563vo736X/MMPSfp9ZJoiHYcEGOQ5u0pxgJnvW1lGR4zZ3BL8wi+0EaLN0dF1wtH/i3jrL8nv32cxGuE1m3iHh3jDQ7zFAJQLKgMnY6OZQVuaspdRnWkWMk4Enbag2xVLZabbNUcYgPvJJReW8vlGkgVdy7EtCsVmh4ekwyF5mppK9ZMJ4XCIY3lJloNhSzpKCaurJFGHt+/VePOfuPzozgStdzCmZJHbSsZ8rjk81Aw2I9KLW7ApzOCdTk3BnP19auMx2cOH1DD6cYJNPSg2MbS0MhvAO8XIUTXWWMUdO6gssNk573kaIXKUmqL1LvAQ2CPPBXF8k9FxwqJv+MveZEJNa5oUul0uWaQe45nD8VBwcJDz9GnK3h5Mpy5CmBpt1a3r7AFlyQur9Fku8amArtUmYTk5MyUZDCQfziQPH+Xs7MSMRlOMNusBW8BV4Cb1eo1Wy2dtTbG+nrG1pbhwQeN52rhnlCYix/U8dNhgMGlw/36D4TCk3XaWRJhezzRnNPepX7mFG7ilI7teB89DKUle7O6zsgKN2Rj59DE8emisNinNigXlIn0aYgdEEEAtwgk9PB+iSBY1N2osFh1aLcnGRp2bN03mVG2twfH8OonYxAsiwsWI5ijB392lP5nwBINl14CtW7fovvEG4WCA/+QJskg4IY5RaWqoeJde4knjFn/yvQ7fvwcffli6Rp/DVq0LLhTlwHkBZUXwvBvOGsZBYB5P4MO6c8T6+JD67b9Avvld8rt3yHZ2yG1xea1ReW62I9KmBvEEM+d9YEXCuAnTLfjWt8zi8Zozwx/uI8Zjk1q+vw/f/35JmdjaKrd8mkzMPW5vI9bWCcIW+E2yWh1nu/bcPF+W5xCfbmP2zw66VSKq9XofHcH+PtnREfFwSDqfG8r/YIDb7xstw3XNdtrjMXo+N7VKm01YW2NR7/H2/Tp/8I7DBx9MUOoxpnTHDDMpcxYLxWAAo7RGurKF3mgZTSHPEUGA3Nkh+sEPAJagawtA21oNrlIl6Pr+qVYZO0nwqH5+shaQoSYrIEPrKUY3eQTsorVHHI+YyAWL/jHs7uJOJqYQCsbF4CnJPHEZTSVHAwrQnbO/DxAQhh5CCBxHfGTPUChBV8ovoB51dTQWr5mSHA0lO/uSnScJT5+agoNCzBAiRIgOQlxBiJt02g22tgJu3oCv3dK8/hq88YamVjMFSYJ8SkcPCEKJ7nR51K/xL/5E8N6PpM0Ept0ua9dPEh//yk30rYslYBQ3rBJJlkMQmOym+sMRzrPH5ZYcvl/WvbTR8c/bJycDjMuiUCFO4BZeDEmzKZnNItJU02y6XL1a4/p1j+1tCFc8jibX2M+g5kNrvoMzTHF2dzmaTHhIuXyt3LxJ5zd+A2dnxwDL7i4MBiilUNMpWVQn2b7KbuMGf/ym4F/8sV72m3Mi/0mcBF07cD6vFHPFQRfdIAqfrsZzodNUrB4fsTK4h7j9F/B//3Py/X0WsxlalZtw6sJ3bcu/Wm9/AKw4kklTMN80mvu3vgXX3ovx3jlEW9Dt99GDAbqgropLl8r9GO12wrduIRcLgpUVvF6KrEnCldoyJ8cqLkIWgKt/sovh85V2tMEFrQ2AHR7CvXtkBwcsir2tYiDIc7L5HL/QaqVSppQfBgAd18XZ2CC/foPDd3s8eACDwRytjykjtgIwBWDW12FlTeDXXYhqEISmIHS/j97ZIR+PSTEA28as+qr4W9q2V0tRfoFykrdrk/NKRl2OUnOMNm+rjqU4jmcKLXckjekh4oMPCA8PaWtNSwgaUjJPJMMj+PBRwltvzbh7d8J0Oir6qo1SDebziNnMX7oVqmULqrXmq66RU3ExnGS7OA7SkYSOoN0WrK15XLhQB15GiDrr625xbLO+XqPrJ3TcEau6z6reZ+PelM5hjOcLAukjR0fo3Q/JfIm8cpVW7zqvrt6i90sby4ykRsPoAisr0GwJXNdHOZgdfxdzUicicWvk0sfzBP5iTDAZEhw9QxxUal/aJJ4XcS9UFyFrBdhBMRgggieE6Taddn1ZR8B1XYSI0FoWhZBMCEQp42pOkmLz0kTR6U/oDgZszuc4QNN1aTkO0XSKePzYWKO93rKGrwgC5OYm7koPkU3Zcvb5lW+12N4I6EUx7VqCDiPcRsjNG7C+BvVhjhgWvlzXLQH3BVZrgUYgcCpBZ4lCDo9xxgNq995G3P0h6u5d8tGIeZIwK37LbroJNuID6xgwU65Lr9XC3drG7zSWGnQYQr3rI5sNZmHISEoyrRFKUTs+pn3nDsHBQbkY2s0P6nWzEkURwnHMnm2VeWJpbtaleDJn7OPk840kO3Ptr/b7cO8e+XjMIs+ZYGLNQZ6T5/ly4zxbmtEHfK1NScaNDfJrNzh8p8f9+5osm2M8srYClzGVl6C7KvFrLtRctBeAGCCOj9E7O2STyTLk0qYsKOIXvy+qdv4XULCjKnauWU3SUsY+HnQn2K0GHSei03HYWBXUZ4dw584SdJtFMCROHI4HgocPF7z11pAHDw5R6rDoYYVSkvncYzr1aTTM7/l+yZ6AjwfZF3YxnLxoMQKlKwk9QUvD2prLhQsN4GWkvMXXvw5f/7ooDkl9ekBwfID84DbivXeR9/cQx4PlBpPZs2ekP3yTzHHwX3+d1i/+Eq/+Sx1uvrxBnhtv06NHxuu1tgatNoBPrhyc6QwxnZKEEVOnBlLgSginY+rDJ8jDp9AvQHd/30y2Xu/FfbonfU6WbjUYgNJE9QbdzibNppnrnucipbsMLhV4yXRqXABxbIw8X+Rs96c4R0ds5jnrGP6vDEMTL9nZKbdat2pZECC7XeRqFzedsikP+OWf9/jWL7qsuRNa7owBkrEMWVnRrPRAJBkcFiuaBd3qQvI5RQiNdITZ+y4EXyj8wRH+6AHi/jvwZ39G/uABaQG6U8qCMhZ4LejK4hXHodvt4l24gNdpLtkG7bYBXdFqkIQhQymJgUQpusfHhIsFgd1Rpt02G3ZubZWFmAprXTjyuSFua/NDYVE6IMVp1l4wPVVSaLIMvViQTKckccyiUDVk0Qk5LPmythkSiByHlVoNr7tOv3aFHXmFQxWTpm+j1H3KumIupu7SJTxvjXrdJ3QSnNExZKbeLnv78M7biNu3kUdHS1C3tUlrlFzEUEpTaay6X/0pab0nU4Cr9XTtRpTP19Y1VLHSIFoHLhME61y9usE3rms27o7Rd/cQk4lxlVy+jLhxg4Pey/zw/ZC7d4dMJndR6rjo7Tpg6i0kSb5k8cXx8wRuO29s/aGT93AqwGtlPkeKY+qOYjM55m9saNZ/zoxUkaZsbrps1Ty2Rgnt9xf4oz7u0T5idxfGA3AErK1WmPMz8vkcIQT6+Bh5sIv/9AFu5KFyhZs5bDsd4tUGkeuzmLt4MsMjIxU+md8jcSNAIKRACpB5iohn5EeHpI8fow4OYD7HyTL8PDdb2nzevfSqJo/1P1mu62wG/UOaV+psd1Z4+XKDwc9HNJuSO3fK+sdgxky9brqg29FsrmteakFny9T/VLMZ2WyGa8mktqbnxYtmA8V22/T5aGQQbrGA730P5533qak6Di6RnBE4Ga2wix/1qK3XEWt1kzZv6ZXWNXKiZsVnGiJl5yApAoJSo5wMlwyZZ2Yr62YTp91Gdzp4kwnufI5SahmrcQDH94laLdwgoK4UhCG1K1cQ167hr3eo1zRNPaY1HhE+eh/x5g9IHj5kuliQYBDG8TxTZtRmCW1uwmuvoV97DX3lGrq3ipBmXCeZt8yU6/cr+V0NjecoHHEalLGPk+o26dMpi+mUcVHzwCadOhg4sduMqMrRcF1arRZ6ZYN+eJn7XOZIvUeWvQfcxYCupQR1gSu47jpR5BOKKXJ4BEeHZiTu7MBbb/1Y0E0wgNvCgK604fqTlIMXlOrCbzVbq6VUd/+1e3PGcV4B3QiTtHmZMLzEtWubfPMbis1nZvNCoZRZyK5eRfz6r7M/fJUf3A65e3ePOP4A455oFb2+QKmExUIhRAn2NoHQvlYLvtn2W1x44QSJKsulKOxSZ0SoBSvrOb/YSGE8gekEtxnh+hHOwQT34dgkJRwdlpHybtfwciYTePIEnSTkSiG0Rk+niP4BzqMHODpDJymeHxBdukq2foHYEcSxg+OkSCchkQGzoLlsn7SKeZ5BPCc9OiJ+/Jj84ACtFEGe4yiFrNZx+KxizR1rklt1fDpdAlmz26Pub/PqFYEIAjxfMpuZWJ4tOL+zYxTWtTVYXdFc2FZ87QI4F2voXo9MKZLZDF3sGyeSBJllhrn/jW+U2u7e3tK1we3byIM+/nwBSYpEIVyHWqdHrdtDbG/C1qZJ4lhfL+tm2NX7FDjustjjwhEKLVIERQSvyOCR4zH+ZEKiFF6SmDrblFquGwRE6+vU220TMwoC5NWrqOtX8dc7NOqaRjykNX2CePAj+PPvkQwGTIrnGQCu5yFbrSKa2jCL1De+Aa+/gVrdQHe6SG2e1WLoMx4vw1jLpoa+wqkp4w8Xn6zIfT5N106s+RyOj0njmKlSS/K+9aMKylKMtuQKYGZ6t0vW2+TZtMWdD10OjxcYrmqKgcwesE27vU2vt8pLFz2utIasZX2C48eIUZEc/eQJPHmC6vdR8/myoIdPkWkDNGs1oloNr9s1xbBtrp9FmlOSqqZrtdzx2Li87c4R87kijk3mWZ4HwBpC+LiuIopW6fV61GoNHCdGYNDbw8By1t5gcOUNnr63xd1HKbu7xywWfYwH3TvZy8vHVX1kUC4M1S449UCaTbudzRBxbCLGk4lhYkyn5NMpeRzjrK7irK2ZEo6+b/bf6veXKUDq+Bj97BnJ/8feu8VYlqX5Xb+11r6d+4l7ZGRmZVbWLWu6e7qnezzj8XgunsG2jEayhY0EQmM/Iox4AFk88YAAYfGCMAheeLCxeGGEH/BF2EJgxNgIt5vp6Ut191RXVWdV5TWu55w4t31dPKz9nb0jKqs6qzIyIpnef2krMiLPZe211/p/92/NZiyOj1nu77OIYwJj8JdLfFE7jHHa8WKBufYeZncPu3sbs3UNL7SoQOGrFpEKUQf7qP19VJGjbI734B7q4w9IfvxjxtMpcZ6TA70kIZhM8KV95BdBPSdKpPDhIfbwEKZT7HzuToIY9lnrFtzc67N/7PJBxZcrsV85VrzTUXx8X/FqK2TY2ab/6qsYz8O3FvPKK6hbtzjaeIvD9bfJ0rewDzYZTmdsr13DG42I790jefCA7PFjsvGYLEkosoxTYKE1ZrHATEYEkxPCx49Qd+64BufXyj6PSrn5eE6FRZXzYxVkVpNbH9VZQ+0WmMyiw9A1uPd9PK2JSkGVAMr30VGEeeUVzC/+IurVV7HKYIMAtTZEr63TjiyMfkL0oz+g+NG3OP3DbzOZTDhJUxTOCl4D+sbgRZFzI0i+XNkVUSVLisWS08QwWwYcjQxHpaun7kLMckWaK0zhhPln4fn66S6XcHJCulgwt3ZFuiKFFBXhCgylZFlbI1nb4dG4w7uHlqMjKdYV78wucIf19evcvbvO27cTXusfsZPdx3vyITx+6MTNw4fw+DF2PCaP4xXphlTtXfrdLuHODmZjo0pSFe3jghvfCIHJMTuSAyq1IEmSl+0WoSg6KNVG65sEgcfams/WlofvG5JkQZFX2RctIOtscbT9JR78sMO9B0v290ek6Qinz/epSFeX/XIro6RemCFjPH9d2ATUowzjsSNROS/+4ACOjlzlVJoSvvoq5s4dl4n/yituIBIaThLsZEI+HjNbLjnKMqZpyjJN6bXb9OKY9nxeFdl873tOPex2UTu7hH/ilwi+/BV0pwXtFmHUwvdD+OB78O1vo+IlxAnq0QO4f5/l40eMlsvSuw5FkrA2HjtT5XlIV5x8smIAACAASURBVLRCYc79ffjoI+zREcXJCbrTQXW7tH+uz/beHW7cqE7R+egj99YwdFPz+LH7/Yc/0lwLQ94Id+m//jqetZg8R/2pP4X6rd/i8eQVvnt0k8W8S/HDiNdNj153m8i8y+TePaYffMAiTV3MpSgwOFdgURR48zlBHNMfj/E8D6OUM7+l8fuF5BdW82OVIrWKpPDQ/W10Zw2/2yfY3Vr1zfBmM1oPHriUN8ALAvzh0LXB/NN/Gn7xT1C0OljPh/kUvZjRnseEB+9S/It/SvqP/gEHkwkfxvGqWVYfV47TKXOY6fcd4e7uuskenaA9j9waRtMej6cRo5FiNHb3LjF5ax3pJpnB5JbghQTSRHqXNK+sXfW3LXDE6lOda2Zwulgur9EaK+20Cg+byDR0V+/2/Wv4/i3uvLrJr/1qwFe3j9g8eZ9w9rELdozH1UkAgIoijFJ4vk8Qx+RpijEG43m0BwPM9ja61zvrb7wgTbeeISWWqAQ/RqOM0Sjn9DRnuSzIc+dk0VoRhiGdjsdwGDAYeO5cpl3nhrt+A3rrrrKirRQbSjGix6PTHgenMJ2fkiRJOW+i1/eBPsZ0aLU8er3q3EZp0CEuhvrRXy8sbSxJYDzGPn5M9uGH7hqNyMqub0VRuKog368iEvfvu/TD6ZQsjklPT0kmE06zjCm18GoZeWY+d03KjalOkWi1HNfNTuHwCcxdCocKAte68ON78OEHbg1NJhTHx+RHRyxnM8Z5zgy3VnueRyHpH8+bp1tKOxvHFKMRxZMnJJMJ6XRKuL9PdO8e0fXXGfpTbmz7zN/yGI102aB+RpKcMJ9POD1dUBQF3/7DFukI3pmuszv5k4SzVwj9Q+KTr5P86OfZz9Z5nKyx2Y25Hp7QH9+DJ98nf/dd169kPl/1JmnjVpHk0BRF4eY2TYmAwPcJpOlsq+VUb3nOzwFbEnduYbFUTKfKNWXSPl6wgdcLiF61hK0O2vddR8EHD1CjEbrbxdy8ib59G65fJ9/ZJQ175FYT5AVmfIL5yXuod99l9IPvc/LwIZOyVLiDc1z2u12i4RB/bw9u3SK/dYfs1bsU166jogDVamH9Loss4Gjk8WBfryrmxDADoRCFkgDbhQfS4BPJp0apVYaCOLl93IO05ZfkVMSbK4X1PHTo0dKGvtGEYQcXTJoBc8LwGr3eLe6+1ea3/4zmtdMjen/wQzh4WHXGkKY6rRZqMMDPMnS5qG2aEvk+YauFNxi44146nbMpBc/jqzsHIVyJucxmThEfjTImkwVxnJDncjiLj+eFtNshOzser7/uOn4VhbNwXn8d7r6u6V0PUb0eXWOIPI+TrM3DB5rj4wVZNilns48j3a3y2sDz+vR6/qqXkLji5Aw90X7rFXIX2ven7mMZjyn290kePWLx4AHzLCPOMjxrCawllLOyp1P3czSC42OyxYJ5HLPIMpZlRowkEUqA1IAj3Y8/rjTkMHRR51u3nIQ5OVnFHlbjevTIqZFl4/xiPidbLpnnOeOiYFk+pTyKsJubztf3vGXA5XfbLCM/PSU5PGSeZcytZTCZED54QDDex9hTrm+28KMOk/Jot+VywsHBu0yn98iyJ6Rpyre+tc2P3rlGlF2nZd9ivZ0waGWMv9tn9J0+6zs+m7uaa2+c8vO377P5+A/x/vn/RvHBuwSTCWFtaNLcfFbO8Rwn3FaZARsb+G+95c4Y7HTOpow9JyyKoihPHT6pKQRFG5UHDG+08V+/hRn2UVFE9M47+O+955SsO3ec/7Xfp1CGJFWkSYE5XTjr6jvfwf7+73Py6BH3yvxZcSlsAt3hEO/uXbdW9vbIb7/B4rUvk+1cx/ga5WlyXPnvwcTw8OHZxBxpsFhv3arNiwikSa6rJL+1WgSeR4cqncPDka7olBlVoUIGZEVBEcfo5YKon9LtaoJgCNwEEpRK2dm5zhtv9Hn7dsLt9iEb4wOYnpyNCpXdqzAG1W6jpq6IPpjNUMYQdjoEgwFqOHSvExut3jviAgNpdU1XshZms5gkmZDn7mhNYyJ8P2Qw8NjaMly7ZpDTZYqi6vw13DT4r15DfeUr+L6P7/vE/T0eHRhGI12eADEEbqN1C2N28LwtfL9PpxOyualYX2el7T6tdaNk0NWz6J5rOup+CvnALMMulyTzOfP5HDkesItbI8vlEntyQjibERwduUh5mlIkCXkcr5LhNZKxXV5C6nHsnmvdghHfqbXu/6XJhGQPlJHN4vSUYjTiNE0Z5zkHuKiCxqUcBmGIXl+vUoa+KGoVejbLWMQxszhe5bMzmcD9+0Q/+gHhzhbdWz9HsfsmN292eestJ0tOThRxnAOn5PmEw8OMQ3IgROsOg0Gbft5jOu0wnbb5k+0pd1495Mb8h/Tf/S786JtM7r1HdnhIUX6vKEhioS5x2u6q40UQYMIQdnYpbt2BVlT6we3Z+3pO2PJECImdKqUoCo8890hUyNwfEA3eInjTEra3CG/eRoc+xc3bTHde5WS+xek9dzaip3J2lopWufbUfE5QFHR8n8AYQmMYRBG9KEK98SVmX/5l8uu3yAdrLPp7TPJrJNP11dGPUpx2/5EzwiR1O4rO7xvr9pO2P3VavhjpSvJw2aoqDEOUUitiFU1XHqbk1YkLIssyitNT1OiYoLuk3TX4/gbgo5RBa8OdOx1++7c9vnRtn9bje1XujHRoUapqppKmbmYePkTNZgTG4Bnj3Ak7O2dKP8+wI1xooUSddOV4uCRZYu1JOROaIAjp9QJ2dwNu39ZsblbZBHI7QYA7BO/11+A3f3PVK2B29AZPjnzG44A87+MCjtcwJqLVGtDptOl0QgYDR7iDQdXkSzhKUhGlP6jkgl9Ia0eo+ifLBxuz6uMqfTDm5cg1MC+7qA0XC9a0dpu8bFSqlkt8ylNrYXUaQgYkWUYxn1cNJSQNajp1js/lspReLrGdVmuVaym7O8tz0izjoCj4GNfX7IiqUUwvDDFrayvB/oUh0izPydOUaVFwBKvTHLLxmMVyyfo3v8nG0RHmN2aE/8oe29td7t6F4+M2H3xwHUeF4vzIcKXyMUXxkOl0kzjeBq7jeS3ubhzyO69/wMaH/xfJP/s/GD+4x2Q8Xp2qLVeII9scl1t/XP5bKYXX7dIZDmFnj+z6LUwa450cXFCayyenSArdJN0yScpU5lzR1TsMt9ps3HiTjWiKH2pir8OTSZfvf9Dh8Y8i2l3DRr8gGrbYHAxgext14wYbxtBOEnQYYlotvI0N/O1tpnd/maOf/y1mw+vERcA0b3H8sEP6qPIoCbXcu+esjp2d1UE3ZxQWIdyfFkSD58lekKax3S5ep4Nut8kXC7IyMujBqlGfFEVIGtkyTZkfHxO2P2bQfY/Xolc4vjEg/9VNlPLR2udPfj3lT9ydcUuPCWY1DVc2T71LhrSJDAKU52GiCJPnjmy3t92mqffcPK+RXSAkXiIdLNNUYa3GGI3nBfR6IZubPru7HtevO01UOtALJ8zncDjy6K1dp/fVr5F6LVITsfyjPdIDj04n4MaNAbOZJcs8wjCk243odis/rvCQuBFEi6g/wjo3Xug5aXB2jZRt9+tuJxHCWVEwKwpauM1eAHgeaZkaRPk6eS+wOvlgdY5ZXXpo7XaJrAk5VEt8zFqvGi6liwXTPGdsLcc4oaBxJugQ6BqDJ8dJfdGJqb+v1HTn1nKCo9AYmCYJJkmw9+7RGo8xg02C67fYbN/kLROyvO4z+4UO6+s3ODlJGI+7TKcZy2VRfkWO51miqDpK7qu3R/xc6ycUk3c5efcdTk9OOMIRbEEVbA6pMoyWOKHWAnpa0x0OCW7dIt/dJRsOYXxSdaU5f29fEKrM0/V9Vw68WFRnl7ouA4rxGHy/S6fTpd+zDAYWtbQsY8vDhwXf/37B/n7OYKDZXjcsrrWZDrfoem/RvZPSWXvA+t791SGnxfYuxbXrLK59laOtu4zMpkvtLF2CaVodtCwtZk5OXAhAysvFvSAKi++5rIVnmZIv3vDG2tW5V3owQK2vY0YjzGSCKgpXkkfl0xV3Qwosk4Sj/X1MBtt+j81gwZ1v/Aq/9a//EtZqrIUbnSk328f0lhO82FYbWDrGiENS/HXSYt53SdW0Wi5HUXy5L/DY9Trqpb4u1aeNtVtEkabbDdjaCtnd9dnZcQ8wiiq/qjHuIT98CHlseG2wRvdWwSztMEo7JIM2vYHmzp2Q3V2XyzmbaTzP1ey3WqzMovMtEESxl0ZbElCT68JkTz03reyvodbXCaOIDs5PmOM0XbGMKP+WwCoPd57nzMtBS6GNpAJqIFIKLQOv972NXGk487nbJfX+mnIq9ZMn8OgRi+WSk7JZClSHNm7jtN1OmRFwJiPji6J8IIW1LK1lgiM5iXMUgF4saOc5g+9+m5aCze2btNtbDNdu8+ZfeZsfT67z7W/3+f73F3zwgeXhQzdrxuSsrw+4dm3AL/9ym1/+ZcWXRjPCw32WkwnkOSnVSSoStg45614At0/3gBtKsbG5CW+9hdrexPgaTV41vr3ArB+tLGGgaLer/uhSvTkauf0wnbrH6uo+FHFckMQJ01nCyUlCkkC73WPYD/nRZp8bmz5vbLZ5/c03uN3e53Z7vwxyKbLBJsnaLtN0i9Fpl8MT9/mTiVsyYqnKET0iZ6TUV4hWDq+ULNRn3UPP3tpRFl09x8gYVKnJqHYbM5thlFp19pIHKRkMCrfA5nnO8XSKl2Xstr7DVnvB9btteG0L67n+lf7kiGB8iMqWoG1le7u+fWc39ulpJXpEszLGEe76+tluL/V7uuB0sbrnQqR3txvSbit6PZcStrur2dtzGsnGRkV+csqEKGJprGnrHu2ex4g2I9sm9hRRG270Ne22t/IdG+PMIc+r3Agih+rFREVxtveCPNYLsxTrHyaTEQSodhvP81wknEpThepEAyEgW5aOp+XrVpWMxhD5Ptr38YKAqNXC6/WqvpXiZAtDqT6BNHUNTYzB5rnLF57PKQ4OyEcjpAhbfJttnDBY8316QUDUbrtimmcpqH8WWIu17lSHegG4mPqjNOU4TTEf/oRgOSfc2SHa2WH41hNuv5Jxc/AavcU6bTYIjY8xmvl8SVEk3LrV4stfCvmVr875zZ8/pPfOCf7HE+LlElNmF4nrDyrLob5P28Cm1uwOBlxfX8d/7TXX8GVzA6WtKyJJa+0dn3P/yBrQtTCRGLBSsHdy4uKejx/nHB7mHB/njMcZi0VKmiYUhXM6eZ5Hr9di1GtxMo54PIqY6DWmW7dhd8zaq2NCLycwOVmrT9peJzv0EWlU749Sv0QpEXKVpSZesHpP6mfdQ8+m6Z6vD62rTrLB8nyl4a5K9Ki0W6hSxjJcwCJLU9STJ+gso2d8OgdPsBubsLmJzhJIYui0q2iQaLd1D7b48qT9vLUV8W5vO3VSZvBpC+WCwvZ1spX2gru7EASaViug39f0+4qdHRdcF3+rMc4LIinDkkO/jBU/eRhycGLAeFjPSWOt3euvX68krExHHDvNYDx2i9WY6ryp862QRSP3/RfgoqtXYZUVUrqWVih1TBaneY3L3+vrxsMRIOXrvVaLYH0ds76OWl/H6/fxJJ1LqqTktEA51fDRIzg8JD05IVcKvyjw8pzlaLTqtZxx1tRuA51ul2B3F7O3hxoMKpXneVDL+JHAoAgcv7zXFOdTNtMppiicq+Hjj+G99ym++U06N9/ma2/8Glu/8BVuXtvgB2/2ee+9gKMjj298w/Abv1bwmnrE4P0PCI4ell+p8XAJhbtUATRpAy/uBYAdIIoihj/3cwRf+xr6tddQd+5g+33XFlKqNC74NO2nTVNZAsDBgXPRP36ccnAwZzabEscT8jzFWovYP57XYTCwSCp+FLn3jcdgshadvmF707I5LCi8gBxDu+32aKvl9o34cZOkSrWUOqokcXtlc7Ny38n/ny9y/Wl4dtKto15LLt9oDFoCWNa6D1YKo9xx0F6e41u7alSxAIo8pzca0ZtOidIU78kj18njlVeqLttbW5UfV8RJvZ1PGFZ2h/ToleMRRAuSwwjPVwFcYFWAfIy4na9do2y2YRgMDFLavbHhZIEcaieyLE0r82Y6hdlM8Sj3KQp/1XIvSapj5fb2YHvLMhwUZSm8YjyBR48VWquViWZtlVlXL1Gud0ATDfi5SbduQcgXl1+mjcGEIUWeo4sCW2p9YtpKcyJ5GiHQLQsLMs/DGwyIdnfxdnfPBkclCRmqXs1xjA1D7KNHFCcnpPM5yXKJ8n08zyNL0xX5iH/TpyykATqdDv61a5idHSfwP4/t+FlQCm0Mgda0cFouOO1fUuBiYBHHxHGMNx5jjUE9fEgRRbRnUzbevsHeGzcZ3uyxeQfW1jwePYRf/HrBL/1CSv/eCZ0PP0bNR2ALjFIExtD3PAKtWZSuG6x1GUdKEWuNCQI2Ox3Wt7fR3/gG+td/HdbWKdbXUb0epAkqPXfg7AWSrgh9iTNIQaMc3zOZ5IzHKUmyQKkpSiUYY1AqROuQXs+wtaW5dq2iieNjePxYsbcXcnMcEqxZhi235tJErUhU5HarZWm3IEmrZeXK5F1nsU7HKVP9Pqv9XFd6nunYCJ6VdM+bjeJw8byqC8/GBl6aospGJto6P6wNQ1pl3oVN01XdtBySF+AS3bV4qyXcPhi4u5NaSDlrRgJi9dBhu+3+z9pqk4RhZW+L+gnVYnkBi6ZshEW36+SGDEuUdKXK49X7bkGNx1We+XLpCrbu3XPTMJlUGuprr1V5tuvr7ooi8FSOlyV4CozSrjTWGNeuMKgOx0ySqo5EYkKShCIL/MJQry8uSVcp5XKld3bwFguKxQIbx9g0ZRvnR5VeGXKgojsxwkN3u3jDIXp7G72359aF+GQGAyfFBgP3fY8euXV5fIzd3ycdjUjmc5ZpSqIUfmkj6ukUbzo9Y3Kb8vuHQCcIMLK76tHI50G5BrXvM9Caa7A6obdDlSu7CuLhTgjxej3UtWvYO3fQd+9irl3Dbxk2uoo3N1xrhGRZ8MrOkqGZE/R82N5ZHRTgAZ1Wi9AYumHIvCxtzpIED/CMoVdWbLa/9jX0V7+K/tKX4K23yIMWud/CqAIje/4FVHHCWe9g/Si742Pngp9OA05Pu3iej+/30NrlRXmeod322dgIeOONgGvXqti6+F/l/ATynICMZapZzD38QK2sRQkF3bgOWItnLMZYlNZkhXud6HDiZviip2j/dNI9b3eK6ZimlYOjPKHNJAlGwo5Zhi1Ty8IkwVsssGm6ip4uyp8+LucyWyxYnriyOwOOvJVy7RiFIOtkKtEgYNW/UBrHypjrkda6Knfhta8V6TrPhuXOrZw7r+QriZnmijhxbQ6DyFXfiKXmiiksH39seecdy9FRwXhcoLXG9xXttmZvz/WjlZNHWy0IdIZJFniF8536+Gilz2QmiDvb2kpO1gMCF0669bktn5syBtPvO81xMnFNo7WmKAp8axlYu3I/yNFKXpk54PX7ztoRDbfddqWjnS52fRO7c41ifQMKi8kK9GzmysLnrhtZkiQk1pIAqe+TR5HrIEblxrCAVoqO1gyModXpoOUUTyHd552kcoEYz6PveSitKaw7xbrF2Y54XeXyTMNuF7O7i37zTfiFX4BXX4W1NXyVs9aO6faWdN7UtFvAdAnTOXR9YAsO9kEp/FYLf3191ZklnE4xccwyy7DW4ochw/V1Oq+9Br/+6y5FcWMDO1zD5oo8V+j5pDqO/ryWewF7qJ5NIydnS2D54MA9/unUZzbzybLOSruEin52d51yslPKGwnzrApfC1C2wCOD3CvTJy1R6JrtBKpAtXMiL8M3TlGzKHJlSDJD5Bt6XYMfyL5RZxJm3Fw823x8MUeVhOdFjdvacpUhkpYzHsNsRpFl5JMJRdlQQ+MkuKUKYChgai2zOOaRtQyKgsHpKdHBAeH9+yg543hz01WZSIVQ3ewTFlounfddLpHOQgL1BXPB0rquPbZb0I5P6Dw5JPAKlzAdtdBRlyJsk9EiSbyVS+HJE3j//YI/+qOEn/xkyWIxYbmco1Qbz2txdNTh8LDtuvqXC7PXgyhZoI8PXPvDwYAi6ZDGHsulOTMV4tc9H22t5xheeGKH7KS6+jIcui9fW4PjY/TxMSZJUHFMkWWkeb6KrPtRhFpbq3aM+BHL3Ot8+xrJ1nUW7Q2meQtVFKxdM/Q6LfB9tFL4UuZU+nnT+ZxxnlPE8Urgd3AEHxhDZzAgXF9H37yJ2tx03yWW1UWsF61RnkcQRXR7PdaWS4qyF4D4lX3AD0O8Vgt96xbqy192deESIMxz9HROFBwRdPr417dga62K+pS9ClbsI824y/C/d3BAd7kkUgqbZZi1Nfy7d+HrX3esJf2DDw/QGHyl0bMpzMvg5Hm/1AWh7lqQAJU0a79zpyJXaaUoa7bbrU7kFVeBkKDnVS2RwxCUb8h0AL4mCBW+ztBxghfP4GSCOjlBnxxAnro3drtoZfBNwKC3RbC9QZ4rsryWKiZZQNq6BlWWn7qZPj/pitko+rukZkly6qw81yzPsZMJ2XRKnqariLVUrolWU1CenZCmzNOUvdnMRVh9nyAIsGXkSY1GVUmn52F9H1T5/Wtrq/xMNZmwqisUjVzG/QJ8UXC2NFCIrZWMaR9+hClK0h8OCba2WCjFoghYLJy0nUycS+GHP7T8+McpH344wyW9HwNrGDPk6MhwcNBmb68i3U4HwmSJGp1gtabwQ9I8Il7aM31066Qbhm66XjjpyocZUxXSiE9Fa1fRVKZbeGWbw2UckyUJqiiIAL/bdcQn1oxs8F4P7twhv/4qy+4Ok7zHwQGoIiPa6dHb67vvyDL8x48x9+9TWItNEpI4ZhnHLmdcKYLyZ9f36YchZnPT+YVu3qza/NVzwp93Tkqfbthu4/d65Nai45iUKrjlAdr30b0e6vp1+PKX3fo+PV05/I3WrgS60wHvLXd2j0BiLNvbLi94Y8MpRPv78PAhflG4rmni3N/agrffrjTp4RAOD1HHx6veJWWQwS2mesT3ItPGdLV/oIpfbG6638uTvVbNzUSm9/suvrG2VnkTZVi9nnvdYFDqZ54h0wZlLGFgCUgx8QIzPsY8euC6C73/vuOxV15BlUe063Yb/02f7uYG0xlMZ2qVuep57sghZcteIHIW0Wfg85Fu3bWwXLqFIEEMaYuWJHIIGDoI8D0PZjPSOCbJ85X5OCl/ShBFosgZLsDg5zmtJMHTGtPrOXG1ve3C9q++SrGxTaYDrO/jRT6GHLK8CvHLU5EI0gvQbuvTUs8KyHKw3Ta2twVxyXqlU2iWtHn4xHBwXPmrRiNYLDRZFqB1D99XeN6QJInI84jZLOLJk5WiswqS+V6IHgyYxx7H4y77s5D9IyOGxqo+QOKd4pWpxyJfWOpylTNX+f6vX680R+nmXmadeKVarsqfZjh0pGutu5HBwO2uGzdclHK4RmEDbFYeQ+QrTGjAK0vDt7acsL5xg0BrVJoS+j6576ML1yO3aLWwrRZBp+NafopVJWN+EfMhZspwiIljfMrgGdV+iOKY9nhM//591n7wA8J+3+03USBELRwMqlD94aFTSvb23FVWShSb2xRWo4J3MfsHFWmKCvnmm/CVr7j3aO2IfbmsrFaJZElQ4AVkLsjt1OMNYuAUhfu7+FTrfdTlpG2ZBmkeKKEncVn0epVhrBQExRKTzvGzBdouxdR0JWcHZcVdq+UGJ379M+O0eF51bI/imWNowBclXWlzd3pa2QPDYeVPLaWtKtNVrLUkpfk4xxHuGEeu9QIKDxdEmQGtoiAtnF9Td7vOzbC1VSPdHZLEUKAhAp0tXO+FemaDsEu918ILQt2DkWdQtDpuA0/LhKgygjY/aPHwieLhQ7e+Dw6EdBV57iKxUdRfHas+n1umU71qoCXKep5D7oXo4ZD5iceDwx4Pj4JVEG4+rxLNxe0hmsT5KXohqNt/vV7lYJY4QH3iAHN6ipHmw0dHritYedoBk0nlxrpxA3Z3se018lOFXZYfbRQ68CCI3C7c3HREcnDg+u6enlbRSHGky/nZw2G1c9vtarwXmUcnwdxWqyLd8ZiAKsYhhzb5SUIrSbh2/z4d3yfsdKpMEIlRSOhdhMQPf+iI45d+yRFqSe621SNr9dHLGP297zhtzFr3vl/5FfjGN5wrYjCoqhHkOyRN7PTUPQMZwwvYR9IsRohSDIsgcNNVP+FZKEdSvFZnfIZni5NkydX7JSgFfrGknRyj4rL87ckTeO8911xhuazeKANbW6vWqakUGK3LKrQzGeU/HZ+PdOuLsGxQDVTt3orC3eHNm67ueTqF8Rg9mxHGMSpJ8JOETpKwnqYkyyX29BQ7m6G1xmhN2G4Ttlp0ej2CXg/96quo1193WvTmJnZ7h6Lj7AZf55AnePMUtZzCfPbJPML6zxdMvqKUTYDHUYjv9dFpAEmP+aMO032fjx8p3nvfBdqPj+H4uODoKOPwMOP0NKcocuLYkueWNM2wNmc6DXj8OORb3/KZTgPW1hS9HmwOfHYGXdDaNVmO1ZmSYgkkSFMo2fei8b5Q0pUvFEddFGHbbWx/SN7puZa5k6rcU2cxxsZ43Tl+MCM3AamJCNsZnY2YdkfR6vt43S7EMWbxhPbxHG9Z0NEhOvSJ5hrispExwOYm6u5dR9TzeeVPES1bBMIn+/SdlW7ycC9iLsoqTrW2hj8e0xqP6cexa/lI1ZviFIhnM2YPH9LxfbyioCgKFoVrT2iMwT8+phXHRO+8Q76/jz06Ijg6InjnHdq+T9vzUH6I9gL08TE8uu/20de+RnHrNsWbd+HaHroVumyjqna90nIl97neQu+i95AQl6U8zUOtHkc9SUlap8g6rpevC0TxEU9nnjtylkMEJhMY+CF9bw1slyTP8dYsnS9lBNeuVWfJSecpcRj3+6vxBIGq9k3JLVLtlu5J5gAAIABJREFU9iwb6vlJVx6KlER1Os4N0OutEu1MlmGsJUwSurMZdjbDTqfYkxNX47e/D56H8jy3UTY3UXt76OvXK3OpdGzbIKIwruOEr1N0unRke3p6Vr37NMK9QOf/0yC5+Y+9kFQFKHpgLYdHikdPNPcfOCumzNtnNCqYzRKWy5iiiLE2Ybm0KFVgbQyknJ52mM26jEYd3nnHx/Ncg/Jbt3x+7q7H1ja02wq/Zp5JXTicnYpVcw5zdrG+ENTtxm7XteDb2CbrbXCcwYNjGE3hZAS+Zwk8S9SxtDasO3ljqRj0YWvLstlZ4IdTPO1I1ZtO6TzZp51m0OuD7qJmZTPRGumeSaxMU3cdHLjJFye3kIk0T4ezCc0XNVH1uVhbwx+N8EYjmEzwy9NuD6nOhz6ezXiyWNBSishaUlxTmiVlIYfWbLz/PgNjSPOcvCjofu979DyPTa2Jypx5U1jUxgZq7xr86q/CX/yL2DffJjeh0xyTGXoxrRoGSV67kG7djLvgEuA6XB8GtbLIRF6LrIqis3x/viCh7v0UBX02q9K7xmP3c2c7Qu2E5AqmiaW93ibY6BEsytMG8pxVcny9QyGikVuKQpXjsLjj6dUzr5PP716o/5R/S7Kp6PiyuMvkN7U6Y9lg0xybpBSLmOJ0TnpwQn4ycQ5r36D7XfSgi1pbQ62vYYY9vLVulalc5OjlKcQxejFDzWdVxGgyOVMCujIjXzDRwvliPcXpFNJMrdbryQkcn7hhKlWZR3GsVpkMWmuUCjDGorUlz1vkuRxcGVIUHstllbSRZYrpXNGeg/FcXLFOqE/LkqunXNfHfeGQ/GghrziG2RzljzGFomc9dnoe3XbI+nZAHKtVCwU88APQLQg7oCIoIk0aaYxJMTZzTcx7TuslMEDp14Eq6bLTqVwc7TarDtTi5pBJkvUq6pNIrfoBcs8LcXWVJxQo+e7BgODkBE5O2J7NMLMZ08WC2WJBXBSkeb46W1C6rEmnsDTPSdOUCVV/k1Wf3E4H1WpV186Os0DLxs1qdIzOLGQ5ukghSyotN45ZdZ6Rc+2F0V4A4a6URlwfBnT1N9cwqgojyfFykpQkepTko8uxiZKyLTRUt/A6HU2vJ58PvaDNbneTod8izNdpRwVdr0UrahEuWgR+hI/BV5ULZLV/JPdEBnzhmu6nBaPS9KzpXvfnSWXY2hpFp0dmQjLlk6cFaVwwn+Qs5wXGK3PfQo0fakfAvkcYWExoUcp9t1ouUdNZWb41OiuZJWQvElqI9xIgRCbTMCuHKMqBnCpTpksyHMo0afI8oCh8iiJaEbLn2bK6WfI+dJkbqFbJAK1WVTZcV+pVTehepFvyc6EeWZT+tpk76VUtF6z7EYNhi7w7IO8Gq1PPRUbWW076PljPIw27eEGONrlr62St+1wRqnLzosF+Iq/Hq2zTbrcSyuK7lIjNeXIRafW8EBt5MHDfv74Ot2/jHR25Y9QfP2bzyRPSw0OSJOGkKDjAuaukKY8Uc2RUjR41rv/vgLJkGtBicW5vo3d2nN+2dNExmaCWS7w4YdWK0+hqDmU+6krLC46JCJQCo9yal2+TyrTJpOo9L8Qr12zm/k/W0fFx9dhl6OcrMSUgNxwEXNsesrXRZzgo2Nq0XFOabU/TQ9ND0S0UnZosXsXp3ag/1z0+X4hWNpb8u65WyQOraZxqOkcbH6M8VJmyEMWgU9DWYKzGoPAKjVpYtLIYJTVKbqaUdMIo/cWrNJb694mLoT62F5i9UL/teuxOCE+sSomwimO/34fhULGxoc4cdCfcIC0l5DPF1JLa77rlXO+HW9dyr4x04RM+UVVUz1CHoWubmM+x2YRsrlCZ096LotTcFBir0LkiyCFIQHsFSheQxpVFI7uqfvPSp0NqoKFyINf9k7JrZa3IJF50xkvdtKhVr6goclpvFLmjiwYD8p0d8skEk2V4StFXyjUFUoollaZbKEVROj2lqGJoDH1jiIZDZy2K4iMVnUq51EpnKlULrt72VHxkdZa6DMItadZaiy7nyTN21fZRmsuIXBSFQzJ6rK16HnU6lb9XFHbZi7I0JCAXRhprNJkGG4BqgWmB1wI/claX8UBrWwXPlDuJwj3Wyybd+sOoF/uL5jmZONu63Xa9bi0Y5zMnsBAVYK2CwEcFPsozzrdbFGALdJai8tqmkOyJqkv4edv+bIOb+t8vAXWfqlR/wdlKGzGHzsmkM1JYblOSwesVZuLFESWufu6ZjEF458pR96/LLpHWaOUNKc+jlytauWvraQGVgl4CnkF5xgVYlC39fvYsGZz3ldRTBesTej76fiYV5Nx6EbxIiVW3CKNolRan0xSVpvSspaU1uVIUWjuSpSx8Ugor0jwIXO6u1q6KLwgwnufceqK1yveJgKlL5bqlIJZJvajoMiHCWWmMtvi+IjxnQEsSiPQpqadDSmrZbFbdjhQJ1V0MkmcrhRhh6OTSxoYzBqT/vdR/yX4z2jUrfx5cTDLi+aCVaBfSNLrseau0rlJW6qj70IRh6s6a+oaob6B6Iup5j/oVLJj6EM7vVdFUz09VfajnZYbcpkjnesuJetPx+v6pf9+Vabjn8SlkpmoD98vrqTdS18Kehrp0qb//PKGeF751X+XThPKLnMD6Z9elqbWrRHtdn5On3aO4KyQVRSJO9YYAy6X7tyhE9S5L9c+Cpwugq4CS7ADrNNdAoZQtCVOtTulaW6ti+aKgiEyv97B3fafPNh0XhaXeykU+dzCwrK9VKbrSecAzpc+Z51saF0e68iDFroaz+X7y8OX1ddT1fvlZ12Q+i6GethFfChXvLGRti8Q+7zI8/7N+nQ98nb/q3/HSor4uBCIw4ayW+mkk+mmk+1k3fl4AX7bm9iyo37tAxvlZEU+xKqVRgdjS9dcKkdbvu67AnHcLXoV2W8e5RW00aN8VI+Slq34wOJu2XI+V16dEKELip0KyYhWed8OJ4eD7EPjOrWHkNbVCiOfFxZbdfNrClgf/WZvjfMb+ea3k86R7XVTg4wLxwrMF/v+Ap23m83+rS5qfBTyv1KyzjTGVD/1Z3ndJvtpnxlPuX84c01jMUyivrnudL0CV/xdXfv3QmadZgnUDQnExBPs0KPsZk67UTznA/Y8RrLXPNMfNnDwdPyvz0szJ09Hsn0/i0+bkM0m3QYMGDRpcLF4+52eDBg0a/DFGQ7oNGjRocIloSLdBgwYNLhEN6TZo0KDBJaIh3QYNGjS4RDSk26BBgwaXiIZ0GzRo0OAS0ZBugwYNGlwiGtJt0KBBg0tEQ7oNGjRocIloSLdBgwYNLhEN6TZo0KDBJaIh3QYNGjS4RDSk26BBgwaXiIZ0GzRo0OAS0ZBugwYNGlwiGtJt0KBBg0tEQ7oNGjRocIloSLdBgwYNLhEN6TZo0KDBJaIh3QYNGjS4RDSk26BBgwaXiIZ0GzRo0OAS0ZBugwYNGlwiGtJt0KBBg0tEQ7oNGjRocIloSLdBgwYNLhEN6TZo0KDBJaIh3QYNGjS4RDSk26BBgwaXiIZ0GzRo0OAS0ZBugwYNGlwiGtJt0KBBg0tEQ7oNGjRocIloSLdBgwYNLhEN6TZo0KDBJaIh3QYNGjS4RDSk26BBgwaXiIZ0GzRo0OAS0ZBugwYNGlwiGtJt0KBBg0tEQ7oNGjRocIloSLdBgwYNLhEN6TZo0KDBJaIh3QYNGjS4RDSk26BBgwaXiIZ0GzRo0OAS0ZBugwYNGlwiGtJt0KBBg0vES0O6Sqn/Uym1UEpNlFKnSqkfXvWYXgYopf4NpdQPlFJTpdSPlVK/etVjukqUa2NSWyeZUupvXfW4rhpKqVtKqX+klDpWSj1USv03SqmXZn9fBZRSd5VS/7tSaqSUelcp9ZeuekzwEpEuYIG/bq3tW2t71tq3r3pAVw2l1J8F/ibw16y1XeDXgQ+udlRXi3Jt9K21fWAXmAO/d8XDehnw3wFPgB3ga8BvAH/9Skd0hVBKGeB/Af4+sAb828D/qJR6/UoHxstFugDqqgfwkuE/Bv4Ta+2/BLDWPrLWPrraIb1U+CvAvrX2n1/1QF4C3AZ+z1qbWmv3gX8MfOlqh3SluAtcs9b+LevwT4F/DvzuFY/rpSPdv6mU2ldK/b5S6jeuejBXidI0/EVgu3QrfFSajOFVj+0lwl8F/u5VD+IlwX8F/JtKqZZS6jrwF4D/9YrH9LJBAV++6kG8TKT7HwJ3gOvAfw/8A6XUq1c7pCvFDuADfxn4VZzJ+AvAf3SVg3pZoJS6hXO3/A9XPZaXBL+P02wnwEfAv7TW/v2rHdKV4o+AfaXU31BKeUqpP4dzubSveFwvD+laa/+ltXZWmkd/F2cK/KtXPa4rxKL8+V9ba/ettcfAf8nP9pzU8bvAP7PWfnjVA7lqKKUUzp3wP+NIZRNYV0r9F1c6sCuEtTYD/hLwO8Aj4N8H/ifg/lWOC14i0n0KLD/DPl5r7YhPLhB7FWN5SfG7wN+56kG8JFgHbgL/bam0nAB/G+di+JmFtfb71trftNZuWWv/AvAa8M2rHtdLQbpKqYFS6s8ppUKllFFK/VvAr+Gk988y/jbw7ymltpRSazhp/Q+ueExXDqXUnwL2cJrdzzystUfAT4B/p9w/Q+CvAd+52pFdLZRSXyk5pa2U+hu4bJe/c8XDejlIF+e7/M+AfeAA+HeBv2itfe9KR3X1+E+BbwHvAu8A/y/wn1/piF4O/FXg71lrZ1c9kJcI/xpOsz3ArZcE+A+udERXj9/FuRYeA38G+LPW2vRqhwTK2sZibdCgQYPLwsui6TZo0KDBzwQa0m3QoEGDS0RDug0aNGhwifA+6z+VUj8zDl9r7TOlpzVz8nT8rMxLMydPR7N/PolPm5PPJF2Av/f3XtwcWQtF4X4KtAal3HVZ+Mt/+fN9mT06evYX57m7yTx3l9y0UmCM+z3LqtdlGcQxpCn4vrvC0F0yOZ+FonA/lXr6ZJ7/+6d8ntrYePZ7LGH/4T/83O95ZtTHKvd4yVC/8zuf+z3/5J/88eeYP//nP9/++c53XiynyCVLXSBcIz9lSb0IzvnqVz/9w34q6V4E6qQq/5aJKYqze0jrsxNV5waZFKU++Tk/Ded559JgrSPUNIUkqcgXHOlmGSwWFdHGMcxmsFxCFLmr1YJ2273emOpzZbLqN1RfbfJ6eV39b/XPgkuelM+B+kOXhVHfVXX8Mc3EOb9/zt/mp+2v8/g0OfvTfn/a57wseNq9C6doffb/5e9y1fWe89voRfLFpZAuVNJFLlHs5KovproEMgY8r/opk1Mn7Cw7S9xPU+7qEu1SIQNNEpjPHakK+RaF+9vREYzHjnzncxiNYDqtSLfddsQrGq/cdBC4//O8szfoeU5DDoLqp1xhWP0Mw4qUXxY8TcLWrzqrnB/3pzHSs0rmlxiyjGTZyO/1/6u/5vwtC8HI0lCqkv2yr+R19UvrT2qGLxvO3//5uTn/ujrpyr2fN/7k9xexNZ6bdD9tLX+ahJEbT9NPEq9MhCwQY6oFIr8LAYPjnSw7u6/Of+95KVZX7i4Mn7Wh6zccx45Ul0t3JQmcnMCjR3B46DTc01P3t9NTR7hh6Ai31apIOMvc54UhdDpYr3yMIqWCAKIIJa8PQ0fO7TZ0OmcvId+XBZ+1A+Dsjnra388vgqeZUy8pAX+aBivDz/NqzYvBdH4PJYlbGvX9BtXeEcNJ64p0w9Dts/PGUJ1k6wrRVeJpWn99fupzURdQcPZ3a8/yyXmDsG5xP4t18HlwIZpunTDhk+u/vgBEwi4WbnHI62VCfN9NRBC4xeF57qpzT7tdLZo0rSx34SL5KRY7uO8Nw0opvPB992nalOyU86pCljlt9vTUEbBIIaUcGYqWKpMhxBjH1c6bzeDJE+xyiZ3NsFmG0hoVBKhu101UGLqd1ulArwfDobvW1921tQWbm+57rmJnnSfW+g6Qn15tmdZ94ud3Q323nVd76qT7aergFeO88lC35Ooeqjh2S0bWd33dT6duWcjtyXslNCBLwPfd342BbtddItfPe6GUcp9z1YQrOE+0cskcnL/O/12WW6mbrIy/pxHw+d8vQtv/3KT7tDUqCwPcgOoLpS59wN2AkO5yeZaDsqwiVmPcT5HAFW9YBgM3QW5CFXFcLcAkcZ+9WJzVgmXsdbfgheLTNvB5zUxWcZ5XO0fI1vOw7bbbAVqDqj15zyslTIKSHXV66jTkw0OKoyPsYoGyFhUE6H7ffY7vu8nrdlH9viPZ7W3Y23PfHQSwtlY5wC7bhjyvWoj9W0pb63lgPKxSrt2PtdjczadVdX+Rm2dVFJBnqDxH2QJsUf1dnofsvPOq0BWhvj7rGprI67piIYQ7n8NyYUnigjS2pIkliS0nRzmjo5zCKgqrSQtNmhu8QBO2FIOBYnOzMph838le2a/gHoPsO/ipMdcXOifn8TStvz4/Mkfy7ySxKz5IU0ua2tUSiyJFuy1XpfGLy0G2XX1Jyt/Pu24+D76QpnteIou1fH6hLBZO6oq2KdJFiFes7iSpJPT6OmxsOL4YDCplTci4FUErLMhzSGLNdArHx+77h0Po993rRHGU67xJduFKznk7rK4qnM9AyDInQYxxqsf2tntfFEEYYY1HoT3yQpEXCmUUymiMzfDIYH/fuSQ+/hisRWUZOs+xWruJyDKK+dxNgjEoz0OdnDh3Qxy77+r3qwcWx2cdWC8yO+D87pVVLZdo9aVrJMcjLTR5rkptxpLmttxwirzMylFYtALPFIS+JWwX+L7FNwW6yDBFVi0EkdL1BXHe9rwkIhYZ8DTt7LyfVoYYBG7ZRF5GLzuhk47JZwvyyZxl+oDlwUNsu43t9Ci2d8l395iHa0xVnxR/9XmSIDMcuj1zegoHB9XYRAtcW3OvEVKScZ83IC5yTj7NSyTzkCSVziJC6PS04pH5vLomk4LJJCfPl+T5Eq0txlh8PyIIOnS7PoOB2xK9nuOedtvdv4RExC1TD4WI/gSfj3i/MOnWLTgJuIt0ESIdj12MKI7d+3y/MmGCwA10sXATdXDgXJnXr7vPvXnTTcJw6PhJ3ucbi6FgPoc8U0yniidP3MT3etDv29Vi9X0I/PLBxDCZuDEtlxcssutBnqLAikrt++AH7jWZmxRlrdNIxdwvn5aNWrCzjV1bJ/dCch2s5lKUwNDLMV4OP/4xvPN997mzGSqOMVmGLQqKPKdYLLDz+ZkhGqgc2p4H165VDzKOK1/wC/G9nJurugolQkkCfaX/yHY60O2SZ4Y4VsSxchutgLmFZbqSL6uPEjdUv11tHKICv4jReQJpuUDnC1jMK4YrJ1rVzTJR+y4YT5vaLKs8TKKpiWx8WqwwDK0zVoYZtzhiL//YbZ6jIzj4Ayj+AFpbsLUDX/4KfP3rPI7gw6TFw5HHkydweurmU0jXGCfL79+vSKvbdXvv9dcr5Ufcc3UPzUXOzflEnLo/NqvJTVHoZjPHH5OJu/2TExeHnkyq6+go5+goxfVHGgMZYFFqgDEh/b638rJtbTmlb23N8Y/TAdSKjLtdNybZKvXAPjwb+X5h0hUtNU2dhJlM3IOazSrpl2XuYYmk7PUqAvU897rx2E1SnVinU3jwwP17e9tNxGDg3u97CoV2ZD1TKOWI2vPca/udHHsygvEYkyWYLCYM27SDNmHQo7XV5XRumEwq9+iFQGY/zyEvyKxHlvlIR2CFRvseOkox3WV1s6XNMo0DPrjX5fF3nSlQqGoj7u7Czg74RmG0wc82CbbfJtJ9ou1rePc/hHv34OFD1P4+ej53prnvV5qu5PyKL3dtrRLnItJftP1obaXCiaA6H0ktV3OSapYTxXimGJ3A6bTSZuqbarGovDMi1J3wFa1FEWiP0ChC3yPyQrphRDfsErULWkGBWs6rzBKZdHhhrofzwR7RzkRzE/NY9pHnuftSyv1d5xlDe8r1fJ/uh9+Bj3/gNtJ4DB995Cap33djPziA736XjvmQvbRL0LpBe/gW852NlZztdqsYiIQZjo8dgYnHp9Nx39/tujEJAV709MiUC7mK4iHWtMyREG1dOK2vO+Kczyv3YpLAe+8ZfvxjGI8DxuOIongCPMJaj6Los1j0ODrqsly2ODkJabcDosin1fIJQ0MYKkodYLV95Op03DYqt9qLJ13xn06n1TOfTKoNINlOW1tw65YjxV6vip5mmXNJ7u9XUlQk14MH7vNPT89GZIMAikKzXMJ05j5nb89xSBSBb3PUwRHq+OPS3jh1T2Jjk/Zwl+6whT8xqzSzCzOPxNkDYCHDsMw9bMm6K/9QlKO7CUpT2ogRGMPpvuF739J857saOTh7sXD3/pWvwM//PGitAEU72qSztc5g+zp+8Rre/fdXUk11OqjT04pQxVSvk7wx1Yqp//1FuhXALRwxh2S+zoeZS39rnComc8X+ofOkHB+79XV8vHJjc3Dg1ptwpCRp9HpOSK+tuZ9R6BFFHoMBDAaW3R3Y7ViGPQiHoOenqNOJWy91Byu8EI1XYh6i2Z6ens0YrAd+8tw9ImvdI5rPoUXK0I64nn+Eev878C/+n2rjyIdsbro3lZPVKSztLKf76tdov7JFenvjjAktJLa/Dx9+6LRGIblOp9L8RPOWR3bREO1W5mc+d2OrX7OZG+vJSaXYbWw44217u3Itih7R7WrSVPPhhyGnpwlFMQXeB2KKImSxWGe53OX4eB2leijVRakOxiiiSBFFepX4s7vrrps34ZVX3PdCFdx/FnyhQJq4xSTDaTqtFgu4B1lf+NvbbkI2NiqeEau27vheKYq1dR7HbqOJCh9FaqVUSsaTkPhiAYsYWsuCVpbAwRMn+X/iTFf/tbfgzS/RMVt0oz5F4V/8wtEaPLC5Ji8USaJI08ovhAmg3XfJB9OI6aHPeKp58FDz0X2YnKpVAFH46PgYPvjAmTnOLWzQ2tD1OgwDRRRbvGFE661X6d45IVIxOgrQoU9hPArtYz2fwvNXGqbeXEdvbqAHPXTYRmuFKsrg0/NOSj3Xqf5AJQBQD7vX32PtKrMi0C06RcTW3COYarbosBj2GfW6HGx1OD4NGI0qz4go8vW0ZsfraqWtuRiloj+E3gCCCHKAIML0amOW3fMCNF7R4sQnuVxW5rz8FA1U9opseK3d/a5Fik5PYUIfBn1n6vX7lXZrLTx+7K5eD7a20GWwJB68yZN5F/skpbsb0w0LLBFBELC25pQjYBVsE5/u2prbv+22G7tMy0UYR+czNbKsUuJGo8pzcnTk+EasGzibBbm5CTev5+gkxiymmPsfoh58TOvA8gqW49tdTu70yNUpEIJVYDXoAvyYxM+Z+SGnWZfRKOTkxHB0pJhMCuI4ZjpNWSx8jo58lksXZ0hTtdK1RNn8aeT7hUhX1P3ZzE2MkG6euy9st50itbvryHZzs9I6ej03MElZlbRVqBzVUVRpM1nmSEf8PZ2Oe3+n476j23UbSsy0dApqCS0xrX70o9UXeb90jIl82jvQ67TI8FcL6MJQOmBtoSnyap4kI0MHAbQ9plPF/pHiwSPFRx/Cg4duuHFc+bxF0xmN3JyL2SwWRhQE9LoenbBD1L3B+l7GtZ2ctaElCBTaUyXnKbIyKCcwoYcXGPxQ40cayMtgU3Yx2l3dX1zPDRSmEbWlHpJfLleO/MAP8I1PV3vsaJ9ia5ti+xVOe3scmoBxFjCbuTXR61UFexKz1NppQ9Pp2XTEIKh8djJEZXxMx1R/gE9m11+QxitKizxD2QPy79nMre0ocuPc2qrWPLj39nxFp++BbrkXvPYafOlL8PbblW/8934Pvv1t96bbt1evmcXXuf/+EDPPeKV1StTOiY3GGEe6ReH265e/7Lh8b68ivH7fzXM93nhRHikhXHGvHB46mbG/766HD531K5Zvq1Vpt57neGBzE25cK2jFM6LRY9Qf/t/wzd/ndlzwK7klf/1N8rfuQmhAdaAotT/fQDvntG150u5wf9nngw80776r+MM/hKOjguVyQVHMGI06aK3Icw+tzeq7JbtBiPez8MykW88dFEk9mTiiWC7dopHgc7fr1sL1PcvNG5a1NUu7BVFY0CLFJBkmtahcEUch2SBcTXqv567p1P0u6Rr1DAYJmNTTOMT8IgAdlrurfHrZaEQ2GmGGQ7z1dby8RfD6Nr7fvphFU881LZFbRVoGSE5Pz7pGlNKMp3A8Ks3kQye8wN2fOOklq+N89sXpqdsEoAkCjdYeSoWrhTccVoEP0SDE5BIXrvjK2xpCDaFn0Uqj6lnzXxR1ZhHNVkwVsRdlJ5esY0s1L48i8lYL63lYrVHGYIzB39hA7+3hbV7HG9xg0N0maQ/RoU8rjgkXC/RyjonnmNT5ZdpTy3ypKMI2RdhCBwYv9OjNOviLzsoK0IGBwFTCQRZyPffxORdK3WtRJyzJ6BGFY23NPZe9PVgfFvS7OcYokkxjlXaaVG54OOrw4GQbb3YX4+9SzG+RP7rJ4djjaOzzRvJ17v7aGIZDlru3mIWvcHp0k4ke0F/30RQ8GYfMMos/9PBaBW2T0FrLyLqWIsnZmHzIxsOPiBjSYY3CbJC1NklT/6l1KF90TiSTIo4rDffePfjJT2B/v+D/o+1NfyRbtuu+X8SZT56cs+ahu7q7uu/47n2DHmWQtCzBggwasg0b0Dd/0N9nQBYgAQQNUQJF8T2+gZd3nnru6hqzKjMrxzOHP0RGZnZTEu/tLh4g0QNqyIwTZ8fea6+19uWl4uoqo9/PKAoLKT1836LT0eu0vQ1bGyXtKKOaX+M9/Qrnuy9Iv/gd6eOvGCnFqCzJrAF5fopwHIQQVJWiWRQEzSb2zg6NUFCtNmi0PdpRSKvlLR7t01PJxYWgKHIg5vra4/zcotXS29k0bn9IkfijMl2TCJhFMhiQaahF0ZJPu76ug+7+bkGtqnQCWGTI6QSRzLDyAquQFF6T0vaYzfTPc92g1wQaAAAgAElEQVRlKWXwpEpFZ8mGyWA26qoQC/TvLoXAyR2YzImIgwHZ1RWzXg/v6VOsIEBW1nEOPsBybrAR8FrQLUuxoM2NRvr/guDV7LXf1xvM0OrMQeLOCQ9xrL/O8I3NPTBdZtMsSBL9d1MJRNFyrUysM0HX9NA+/FD/HvPgSE9gexJ5E4TMVQzKlDKmBhuP9Yc6O9Ppy2ikX5MJajIhF4JESgohKAFLCFwhcKII0WzibmzR2NmnduuA8s49iCpYwwGif4U4P4fuBeJKn2JOXlLFQq2tozodhO8jwgBrdxPR28QKK8gwRBgu4irNz/OWXOobul4XCRkYAZZNwCDQJf7du1ANCmyVMYsFvZGLkvo5mIwtfvd9lcffhYR2G8/KyV/6xEceX34t+PJLwf/1z39B5/88BNumP/M4vfI4+s6j1rZ55z2JFJJnj6tMz2B902K9U7JRiWlGU1RZwizG/v2vsf7iP9A4OCQ6OKSn3ufSqxPnzqIn8rYy2dWkII71M3F2puG0b76BblfR6+UkyYw8nxAEHlFk0+lYi3Xa2YHtzYJmGBNOuojPfk/5H/+c+NkzRi9ecAQcAZOLCyZff40UAhs9yfNQKTr7+1Q+/hjXcmjXI6K2y8bWOhv73oI59PvfW1xcuEAJzJhOJZeXHoOBWPRgfygV9UcH3VWYbjrVi7TaeF41ywKFN6ddlkqAEghbIjMBqqAsCoQoF1XnYKCDR7cLjx7pk67RWCYalqV/p5Ez6kCviObYrhQKUNiuRFkWZZJQ9HoMxmN6SULj4gLbtrHuvYcvEzxX3Zy2WgjNkwWU0o0gE09M0DVNi7JcqoDNgVWWywx+tUk5nS5peVdXmg7X7yd0uzFxnJLnKUmSMpslKFXiODa+71KphARBgBAOYC8Cd7stabclQSCo1fTh2G6DQODYOgt/4yVZpbWYLNfACIapYHh7V1eoqyvUaEQ5GiGSRL9sG8t1EZZFKSUySRCzGeVoRDYYICcTrCzByaYw6aFsm7Lfp+j1yHo9yl4PcX2NGI+1KEJK7Ks2jsGiqlVQMTgSgkBzl6NoiYnWass60TT8bpjVYTBmKZdwmudBkF0TJgPW84z1QYmvPKhGODaobIASkqoMUa6PtCXYgihKaQU5eTEmTqZ8Pgk5Pgn56rjJXx03abX0z5/aMC2gVmTUrRm+iJmohImSNPwa1YpDaMUE6fzenJ/D15/C559ho7BrAXZtF5UXC3Xnm0L/q5C/KYimU70t+v0ltGgKobKUKCUpCotqVXLvnj6YWq3V6lcwSy26cUBc7JD77+Pv1XCbTYKXL6kcH2PNZniAVa3iNhrUWy28Tgfr4ADeeYdsc5+RbDKdeBSWhRA6i37/fcHZmcPz54okyUiSlLIUrzRDVxOjv+/60fDCauA1pTMs96jh0OkMTmOKpRJkhQQcHDdACqAoUGlOnunm12DAnD+oXy9fau5/q/Uq565WW8oZWy39XgJf4dgKRyqUKpGWfkP5bEZ6dcVVnvNcKXYGA2pxjH95Rmgl+L7Ckjf0QC0yXYFSMIsFg8GS3gTLhwtepQSZdTVJlqHEmMBseM6DgaLfVyTJhDS9oiiuUWpIWQ4oij5QkCQ+s1md4XALy1oDIiCcZ1eCycRhNJILIUmeG2aF7tJKSyJ+uGXsq9dqW96k38PhMl3Psr/jL1GMx2STCbZSOFLihCFWtYqaA9vq+hqVZRRZRlaWWNfX+JUKMsvg2TNUHFMMBiTjMZMsI80y7CzDKgqUUkghqMQxTq+nT/Bmc9FcWnStDMBr2vnmfpou3Q0GXcMnXlU8LSiBz7vYj7/DPR7jnqSwuwX37+MW0ByeoJTCrm3jiTbthke6U3A/uuRWcIESgiS3eP5oi19HAc+fC/7dv9MVzccf68M+iqDqJYRJn7boUXF6FHUbZ+Mu7kYT9zqF7hC+/x6+/BK++QZ1coLY2oLBAJHMkKJcVGQmYXjTrWIq5jheshHMy+x/y5I0GoLJxKMoBJ2OxU9/KtncXBpdlSXEmUV35HOZbnLV+ifEH77Pu8ETDu2H8Od/TnSi1w/AabdxHzzA//BDgp/+FHv/FrKzxrCo8fzM5+rSo+w7iPmaPXggePjQ4bvvLPp9SZIopJSLs/nHChvfShyxetIZrtyqoMiRuc5eYhBosbdIJzAZwuUlYpriVG0CO8CzbCzLZjTShIOTk4LT04LxuGQyUVxdWZye2jSbOmB0OvqZVgp8T+DZBW4R4yRjGPRQl5fkwyFxHDMAzoEgTdlIU6zZBE/k2FIhbuCB0mstUEpQKt2wms03khGNGLGG5+kblWWvuqcZpYvn6c22SrY2X5emOd1uRp5fAS/Qg1+vgMH8BVAjz9cBH6jM/9TvUgiF7yuybAmrGj5rEAiiSOAgeCtPoNXAazLdfn+pzTYAtZSLDoQIQ8QcvJf1OrLRWHQTy8tLSttGDYc66wXIMsrBgOL6mnQ0Ip5MmKUpE/QIXHvl5QJZnpPOZlhKYUmpT3fTrTIn4dracqFfl0TdkELt9R+r/YoUTW/KupxhTV/A+XeImRG2FBBFWGmK9ejhPEqN8aMttoqQSs1mvzFms1oyzRz6Uwdhx+RFl9PTMb3eiDStkedt6nUfy3KQdbCkInAKgnCuw7dGUNiQzUurszN4/FiXnGm6aAJkfo1pbKEs/V9mH7/pOqz6Saw25vt9/XXaJkTQagmmU5t+X/Duu4L33pM0m/osd62CVpjScAukJck9ibO7QdFawwkCXDug9vw53rffIpVCWhbl/Q8pPvxHiA9/QvbuB6jtNdxaQDmySa9hNhKUGcjCBFWBbVsEgWQ0KgAXy9L8XQNx/tdUdP+t6wcH3VX9tTmlDGnb/NtUaq2W7iq2ohQnHiKlwvEDrci67ML5GRwfY81iag8kcifiqhFyuWZzcWGSpIzJJGY6zRkMCl6+9PC8iLU1yd4e7O7OaWZo1ZlVlDSLIc7sAo6OUE+fkvX7zIAx0Afq6NDkKIWzOJpuBtQtldAlxwr0Mhotydu+v6TGmvUcDHQgNhw/0/wylgjmpfmlMBqlvHgxRk+UfoSeLt0Fpuhw46MnMNXm/y7m/3YADeX4vqBe17/XNOoGAx14m00IrLeIL6tl0Ot14+t2lfU6tNtY83cojJORoWjM9eLi/BwZRTjn51hXVwgpkY5DkabMJhMmkwmjPF8E3BytvnOAKuABWVkyy3P8LMNKEk38nc2WEqTt7aV+3ADghtK2aoTwFtfrVaI5hCsheJM+4voYHj/SL6NC6Hb1xri+1gBnnsPVFf7GDntWnSysUYmqFM023X7EszTgZHLJYPAYpR4j5SMmk0OePPkFd+6s8957VXY7NqUfQlh9VWmw6hBl1BFZpuGXvT346U+ZscfVyMGf9w5WyR4/9nrdQ2FVnDEY6C1y547GbA8PIcskg4FgY0NweCgWW8gjYyca0gwShGNT2C7r6xUy5dDKKzBr4xwcYH34Idg2wvMY3PtHXL3zx5RbO9g0iBKPRimxbUUYioVCPk11YnJ+rm+BRposwMVxrEXT2xRDNx50X7+E0HvU0EyKQu/hjQ3Y3CjZ6hQ03SnO8AoxzrAMm6DbXXgHyDjGq9cQnsO6WmPS6jDe9BgNXfJ82eieTBTX1yVlmTEcqnlQlkyngjiWpBkkI5g50FESN3awZUBZqWOtrWFPp3jTKVIpciBXSjcLylLz9N72EgLFPMstljCLcXEKQ6gGOTU3I3IBAZlnEQY2tZpcZAzGjsEcbo6jY5BJwBwnQYhrlpntEJiwDDUeOvD6QIAQPp7n4Dg2rivwvIL9fcHBwTzAzlVOq747pYKFjO5Nrv8armsC73CoP5BlQa2G6HS0Us62NbDcai0tn+Y7WIShdkxbX8cy9pdpSpYkpFnGJE3pzVciRbc5AiCcr0IO2gSoLLHSFDke64y530fMPYmFUktSLLzqpnITbfrXlkbjfxoSq1dL/O4IcXGOODnRmJpl6TRvHoXU1ZUOunPKkLPdpWEI8NE+aTUgly5J0MCtjWi3SwaDGaNRn/H4nJOT58QxOI7NzppDXFgLJYmSEqSFKpVep6LQ0nKjPPB9VGcNdfuAuNuhf2YTCf323nRJXj+AVoUgoIPb5qY+C985SHj3TkxZCsYzScXLWfNSLKmoBzZuNmZteExtcA22TekHVFubpNU2ydDiJO3gth/gfpgRFw6x8rne+IBB6z28KKLmKZD6czsC6pGFRKCKkjiGIpOMAkG7pd+PEBpCrNfFokha5ebeeCMNltCl4+iTZpXffusWHBzA/nbOViOmEQ9wrs5hNFhmPgbrMyv95AnWxSUb63cJOneovtdh52CNL79yaDYFL14oXr5UjMcFSZIymaS8fCkZjWwuLx26XcnVFVzu2eytNdhu+bRaLRrtXbzLAY3eBZvPnzN7/lzTQ9CZrvixQMx/b0GEBPEq8V3KJTfXcaAZxlSSIb4oQVqEyqcWVGDDZXNz6a9QFEuzH4MZPX2qu7lpGqPUAEjQuVwNXUALdMCtAE1gE9jEtts0GiHttqBet2i1JPfvSw4PlzClMe9Y+UDwtjj36kKsWj/F8ZIDuLGhI78Jsrdv69TmdQJrp7OUGc1mOih9/TWq36eYr8Q1Ot+fzH/9+nw1UiCer46tFFkco/IcS0psKbGqVWSno7/JcLZMyrIaBUwX9wau1R/tOyWdeo57FSPMwXR5uaS3GMpOv486PtbB0BiVdDo6MiUJMlNU2x7b2x3efbfN5aXHN99YfPNNSJbNgMdcXBR88kmFtVqVP/o4h/acY1mJKGsNlOMi+1cIunqdBwP9ucMQVYkoqw1mlyGDoSCfn1F/d+/8uHVYDboGnpyfrzx4oKnHu941O+oc4Uhy38dOxvhnlwhKCq+CHF3jP/0auif6kK5E2O99QH77HY4umzztdWjZH9H+YJ+zc8nxmYWdNAlGHtsNfejVwxy7SLEUdOou9UiispwiV2xtONy6bVOrCao1ePlS8vKlwrbFYiub12rG+9+7/t6guwprmct0X03SYgyibu2XHNxS7LRTmv6UYDqG2Xgpkl+11DPvsNvFOj6mnsyoWhMqm++y+U4TgUUcuxSFQDeiE8oyJ8tKej3FeFwyHOZcXyuGw5zLy5LTbcHWlmBtrcZarULLfY/2xgApO+xGdSrxhEqSYDXb5DgL1sBbXUK8khiazeR58/LHVYSBoskUf9TFjUvwfcKiQsNRBI5HGElsV+o4FSuiMmEqcqp1iR9ajHse36UuRZEA884lTXSTrAQcpAyRsoKUEbbdxHXXqNdr3L5tsb8HnVpOp55yZyflYDMlFgGxCMiURaEsbHs10L5h0DXH/CqPLUlQqy4ujoMwgLbxDXbdJSZlZGYG+wjDZelvaCuXl4iLC81ykJJcKWKlMEiotjPRf2aALwSWENp9Lc+Rc6aJWm07m+7Wat1rNr0hgt/QZZbJthWBXyLLFCbzuvrqiiJJKBxHN6/6/QUhXqQp0nURq9xC10VkBf44p9aZcSuDYQ3UekA8eY/L/hHX108ZjaaMRhlPnpQcn8DumoOvQmzfQlbq2snu9BjV7ZL3+xTDoaZX+j4qSVHDIVbq4TnRQsK/OnHiTdZgVYFmboGG4BSHuzPe25zRGh9T7z/DcucBZ3CpSbxK6YNnOIRvv9CNICkpGy1mlTUG/gEvzi2+vqyyt1eFvX0uJVyWUHNnRJMhfndMJEYEnu5qCyFwpKQE8jSlLEtavs+6CBDtGv5hlVbFo1ZzmU41VWzVCOhGvRdWLeZWOYartothCLd2Sw52MhrWFCceL8oTWi39g1at9SZz4wTjsPHVV4iHDwn/icC6c4tG5BNVXGo1w1KwgYDptJyX7zMmk0uK4prJ5JqTkxmVikMYOgRBSMX32VabbJX/ko9/NuDj966p9E9xT08p7v+Esd1ikthzVsVbXGWJTnOXVYBtL5lHgVsSegVhd4B98lJTlTodfKtgLRtSSAe7CJDKQUlJIVIa8Tn5oIdDiMgrRMkGUm4hRIHO3arobFYCCsvycJwKnucTBA71usfGRoX9fYuPPhI8uFtQjXtEcZfa9Jzao3OKW3fIb91hpCKGWUAUaYpMqd4C5TawwqojyZzEWM6DrwQNKaxKbYtiaRN1fq6z2ctLjSsaDMv4AIch7O1hj0ZUrq+pT6c0koS4KLDR7z2ARTOwBBzLIrRt7amrFNJxkI6j8UpDlDUZObzajr5hI9nVxqgSFnEO7izDMaqA62vS2YyplJTX11jdLjJNkdMpdr2Oe/s28vZtHYBcV3/P8Ql59jtNoezBfg+82/+U23/yz/nkG5ff/CZkNKoDIZfXLn/zjQRXsbNZYW1dUsUjmF3Dd99R/vrXTJ4/ZxLH+HmOP51if/8t9n/5z6x3fsIH99+jcMMF9fFNFJ2rWa5BoUCHik4HqhEciDPaR08JukeIiyPw51nM6Sl89ZVexIMD/QNOTvReabVI3Covkk2e9vd51ovo9TRyVRQaHtjbg6h7RvX0e6q//xb35GuUShH1ur450ynZbMZ1mpKWJXXfJ6g22L7zM/z9j/A3NrErmwuevbHFXDVBf2sZ8Cr+siDSy2Unfg7PUa3C1nrBZjvDT1K4ThZs/TKsUDoelAoZT5CT8dLXsSyX6Plshv3eT8DKcCwXy1LYtsJ1S1xX4TgWliXnyXJJHA+J41MGgzN0kSnnH6mKlDV2dg7Y2TnEaroc/CMHe/wS++IpSaXNSNQZxzb5Tag7Fw05NT+QxMIWL1AJoZogZ104O16U064b48bxPErXQHnLJsblc7i8gLJFWqzhJwGoTYSQ2LaDbTfw/W3Am98Pdx50HcJQx6bDQ3j/3ZJ//NOEDw7GuC+e4Rw9QR0/pXz2DFkZIW479INtXMvD8iyd0JXzZXyT63Wmu3HsyTJUllFmmcZPV3emaVj1esuxRWZm3GRCkSty4TETGRNlk2VNsA9wGyXRVoyfS+pX52TjIU5ZkqODroMOvLaUOJUKfqXyqoNMEOjT3OjSTTYu5atNtBvMcFexeikFSghmKYhU4aTZgjdYAEmWaXGIlJpb3mxSbuyQ7L+HuH0fGfpYeYJ78Rk8esjs+JjrkxNcYAfYemeL8n/8P8Cr8+xZBctygIASh6cnNtW2wK2VRHlG2B8hei9QX31F8cknJMfHTLMMkefYQmC9PEJ+8RnNX7Zw3jtkbC3Rn7e5VgV/ulRX7Kxn7KylrD0/ov70U+TZ6ZJt0m7rPfL8+asg6rzpR6VCUt/gWbzJ355sMJ2CQGGnU7zRjPW1kvVOid/7Hrf7G9Tf/obiN78hyzLE5qbuzfR6TMdjBmlKohSu6xKurdH8k5T6Xo2i4VOub3J6uux3moD7+pif/9b1ozDdVTzXdF9dd8kp9z1tGI1l6UWac6Hy0mZqRZAXhFYPN0uX42ZGI0yerlpthk6LQS/g5MLh9ExwchLz8uWU6+ty7qnqUBQuOjLohpHGNQugh0b2bJRyGQwuKYpz/vRPd3n4cJeP71T46Z07hE5IOXKZvua58uaX0sIMqQBBWa7ALueXcPwMXjxbbh5TKo/HS+ldWWrw9uRk2U2+fRvcCvksI06YY7SSrS2fe/cqWJY1tyywiGO5SBqNL0W7EhNdvcC9foz12SfwxWckV1fEV1f4WYYfx3jv/pzG/Tpl6Cw+yxsHmtdnnKxMLhaANFj6qr1jmup98PKlDrRhqPHe3V1wHKayRp8mX72s89u/rHF+kVOO19iv7vA//OQn3D78ivpv/wz/8RecpymTPCdA74oKENk2wd6eXkszrqRW01nT+rr+XY3GMm0zyYDh+q167L7FZQKuwfjNdKTJBGw7JOh0EPfugVK4gwG18Ri1vo64exdZqSDTlD5tnrjv02OPim1Rtwbs+F0awSPObZtvgQ00pu1Xc9zthPP3PM7OG/QHNlK6VKsWzaYksDNazoTO8BT/bz6Hrz8j+eT3pEdH2MMhTcB1HDzXxTJ2XSvsJdN8fZPnZ1U1b9bDssCzSzpcsjm6IHj5LeKrr2B4vfSLNR4Bh4d6rxgjbSn1fvnwQ+Ltn/Hwmw3+5lvtH/HLXxTs9b9n/6vPCcUUX8bYTx/Cd1+THR0RT6fkRYHqdsmEIJ3NmGUZ03mWGec5s+kUt9vFOj6ifrDN7o5+S2dnrz4qP5Q29vcGXVNZmT9N+SzEkv1jHMU8F8pcUdg2MtT0DNKUovSIqaPSFFfFuGbWjnkIskyfYpvbDJ0OR5c+x+eS42PFyUnM6emAOFaAgxABluUghI2UARChVIWy9Oa47xWQoZRiNJoxGg05OZH85V+2+Rf/ooq31mYrkvjjv+to9qaXMOsjdG1eFOA6Cs9RWP0u4ttvNE2u19MPsuGIGR9MA7f8/vdaimeY7K0WKIVSClVCNaqxt1Plg/fhD36pkLbgeijp9cUiOZxO9f1ot6EdJVR6L3Au/hb1V39J8ZvfMEsShkkCZYmXJLi1FuLDD0j9eZmnmKe7b7IQrz1J5mmaeygI29beDqtfZ+R2WaYbRvfuweYWxdoGZaNNL27wrF/jV98J/p/fpDx5EpPnPh99tIf7zxpElTscnL+geXVEMRphz2Y64ApBHagGAdb2tjaDMQ447fbSpKJeXxrKmqfFpHFZduPQgpH+mhllw5HAkRFBYwNxAEQt1FUPu9en2D+g+PkvKZtNrHTGda/C1y9ucTRtse7Ann1Gzf2UyLLoCcELoOF5ND2PWlVRi3oM933Gv4jojxzyUqLQHtRRUNDypzRnJ/D5b8l+9WuSiy7xNCF0PIKWjxWGWGGAarUpgwql5VIiFqbrqwKfH3utikMMtz9yC5rZJZ2r7+DoW/ju21ddtMzElSjSyclXX+nnZmOD8vYB5bsfMul8xItPOnz/neLnHxf89N2E1q++o/3VnyHm9784PiY7OiJJEqZo3L9IU1I0vdQ8Bi6abpgUBdZ0in19jacSarWlOGR1JNiNUsZMH8EkJ0sK09JOLwggx+ZqFlAJSqJQYdt67IFdSCIyKGY4ydzu3SDxpk1/9y7lwSE9tcXTp4LHj1MePYq5uLgmz/tAiJQ1KpUKrZZDFFkEQQMhHOK4xni8zdXVAcPhFTrbncEi54kAjzTV4gvj/m56ejdyKQWlQpVakSbSBCtPkOdnmns5my2R9jRdZoGrqq3BXOBgyof9fcSHH1BX6+zNBB+9m7BWjdla0yXYMA04sqoUpW4KrlbN+/vQDnOCqwHq7Iz4+ppZHDMoCs1VHg6pHR1RXvUpkoIymH8OAQtD3x97mRN5PqWYen0xgVRIuVSCra/rN2nmsxl5bhRR7t2i2D+gn1e57IV8+q3kV59M+fzLYy4vv6coeiiV0u9v88knvyDablDb+5iDYELn6VOibhfbtnEsSzfQPA8Zhq8aP5tMwWSy06k+EM09MY08A4XcUOA1GZ4hd4xG80QuaXMlHqDCXcr2lMyZkVVmXOVNTn+7jaz4rDVyxonD9y8DUgvefw/ut1OiTy+Inz7FHwzYsCxaH39M7Re/wG80EH/+52xW7vCL1iEPxRrfnNRxKi63bsGdPYtoPYRpB+7cQaUlqd1gZkcIF4QLXmhjhTZJdY1ZtM64uct47DHOlkv3putgtophBwYBhHaBc3YGz77SVd/ZmT4gzX4xGLyUuukahvqHrK2RrO/TjQ45mrbwGz737hTs+le0B2cEL7/XboPzU2I4HnNVFJpOiIaifDTjxWKFgGnbVBoNgt1d5E9/TvqH/xOPh7f48i90YXZ6ukzADW/5h1w/KNN9nfhr9uEqgd/zIM0cRlOHwi7wrAJpWVr7XhTYZIjZBJLx0lDA86DVQjWb8OAd1Lsf0v92jWffCp48SXnyZMJsNgSuEcJByoAoCtnehvV1Qb3uYVkNhkO4vMzI8xHj8Qil+sAApWIgRogq4JNlNqOReIV1cWN+uvNTSZVaJEGeItUI2T3XG8i2dSm7GnTLUgeDY901Xphom6C7twfvv09jJrh1DR/fSfjl/SF+qb0An183mBEwyx1DDGBvDzY29PDOZpzjJ8ugO0pTBmgQpjYcoo6PUb0+RVos2XPiLShj5kT2PB1I54wFURRL17eNDf0ynNx5Q0it6/8vOluknS26Lx0ePof/8smYf/tvB5ydPQT+A3AMKAaDn/D553eoJVv85Bcf8eABtFeIk2pejqn5k12OxzrTmY8pEbPZ0hzWWNqtzl5ZxYhuwKDj9X7ccmKEwLI6WFaH3FNkEmJXMasonjyBL77Q9/XwUEu1X5wIGnVFp1lwf2/GJDtn9OwZPrDmurQ++IDav/pXyN/+Fv7iL1h/r8vaH7mkpcOXTyt4VZe9Pbh1YBFUAjKnjdq9Q+K2mOz+jOHWA4oKFCGUoUKEivFQcD3Uo7HG4+WZBG+2NKswi3HUC0OoWCXuYG7H+uKFxvaNv6Uhls+Drmo24f59/f+dNWJ3jdPeGi8vGngNuHeQshP0aA2fwdkzxNOni0g/jmOO5w9+FS2aMgwXe2605AuBGwS46+tYd+6RffRzpj//Yx79GfzFX8Bspgdeep5gbe0fyHvh9UUz7Jo01UmCccrKMhheC8Yji2YdmjWJV46h31s2SITQ0eHOHW3rlxek2wfMgjUmVOYx2cV1I/K8JM8tlAopS0VZ5ihlURSC2Wzpq9loSDwvoNmUTKcus1mdOM5JkpwwbFGp2DSbete/3py+kassUZTEM8n1NTiuIPLk0uvSLJqBVYpi6e5zdERxfs50OiUTgtDz8IqSNIFkKmi3tX5+pxxinbyAUp/Y+cRhNi4Wqjcjpmg2oFkvqE9yfFs3g8QcVw3RpVMYRYjNTaxWHTfU0s48h7JQKPkWzSMTeM3cHGMcYQKc6bqbrHeuQJv5LaZei6telctTyXcPdfX4/fcTZrNTtNw5QecjVWy7Q6XiUWl7OLsb0Lq3rCaOjijPzxnPsblyXs/7sxnBbIMGPv8AACAASURBVIaTptiTCSIM9fsyY+mNN8NqK/of4DKdeyPFNrO+er2EXm9GkgzIsh69XsL5eYllOVxfhzQadaKoxYaXEn76BXz+W6zHj3HmqxIUBfajR/Bnf6bXYmODWXuPsbeLt9XiI9smQz+Cn30uKHKb6UWV889vc3U8YVLziKuDeTAUOI6D6ziEoUVYEQvasHn234bYsdofMkKriitx1hq6TIsiHSNM0O10lge1EKioStlaI600iGWFq7hCb+ozmcznvTYk9maH/ADk//InyFv7Wnxyeor39Cn1x48RcUwFqNRqeBsbWO02VqOBqlSwXRdRqRGv7zPuHPBw8IDv/w387nfw+ecKzxML0y2Ttf9Q2tgbB10DqJs+yOqIkTAUDK4FOzuSoAZuOURc9TTwPRzqhdvf1yXCdIpKMlKvzchqMUEwnemg63kuaeqQ57r21UE3AwRFoeekOY6+L7WapFr16XR8rq5q9HpqIYLqdASdjqDZ1O/fYLk3lMQsygBVlsQz/XujukD5YpnNmUzKSO2My8d8qm9+esooTZm5LrLVwi0UaaJnKLbasLkF7sNr7G9faMzVcciT6JWga+J7q6loVQsimSOcgtJQpdBB1wOCKEJsbGA1a8hAUtjzswC0Wu9NLrMxDJ92VS9uLNUMqGnGwc9f07TKVVLl6ZnkyVP4/HP49FN4+XLCbHYGXKLRNg/oYNttKhWPqO1j72zAtlg2vE5PKc7PGU4mXM1mFGjLz6ZJRSYTbHP4w1xGuanVPUb1Y8rYG8R04VXBngm681jAs2cxz59fkyTPgccoNUSpDKjw9GmH3d09fvlLj01vQvi3v0Kc/CnWkyc4aPzRL0vshw/1D753Dw4PmXb26Hq7uPU6H98WdLt6runxseDy0uLF84gvPr/N82cJiBlKXM9vpUC3Ii3u3hXcvSt5/30tWDD2qm9zJpls1zyDmnY6D7q3bumAu0qbarf1PZqrBlWtSb61R2zX6Q8E3Qn0Z2IRdIOmhb3ZJjto4dy6hfzn/0z3TP76r/HynPrRETKOCYGgVsO9dw/78BDPzBWLIrKoSdy4zbna4rf/0eI//Wd49Ejx+LGamxSJV4Lu6xOC/1vXW81IM7TbgZ4DuXiubFvgOFrtOxrBbiOkE21SCUKd+dgOdNYgiMD2yJKCbj/kRVfy7bfw2WdwdpYynWbkuSb/W5bEti3abYvbtyWtlo5bpg81nYqFn28U6VPZuAqacSOdzpI48GO00n/vegiBEhY5NqOp5PISGr5D4UcIL0DYDsLMYjEg+Fyhl+U5cZIwShL6RUFqWVTjGDEZ4fbPUWdPcKZD7OkQeXWhK4b5BnSSNtWew3SOrderBa2ooM4E77yLPH0OV1eIssRvtRAHB6iiQBUF/t4e4vZtaLcQtoVYZee+KaZrniSllt5/xtTXBHLTAFhfR0URZaVKGdToDTyeHUteHIkF7TKOoSg8lGoAW2ic3gNu47p3abWqtOs5XjIfMpZl5EoximOGkwmXacqgLBfmN1M05yWdTsnQgcoBZLWKkFLzh43rkHF0uTH8aXmtWhrOZorLy5KnT0suL0fE8QVFMUQffxUgoFptsr6+xu3ba2xvh0TbFsn6z+lv5biz/w/n9BS7LDU7ZDTSvdDDQ9jbw9ro4EYOp33Jy5fw9Kni4UPF+fmM8bhPvz+ge3VNkhs5OeiCu45hbPf7AcfHFhsbYs62WHpZv0ngXQ1Kq1VzXkrKdgfEfT1U1XF1glGq+YNd0YdBWZLIiP7IZYKed+iueBbpkT6CZ88FVz1FmXuozOZw2uCd5jpWrYZvWQvhvF2pIPf3Ke+/Q3HrDkljg2nucjkN+faLJl+/tPn004RHj1KurmyKwkYIC9e1XoklJib+gwZdI62/vNSjNAzWYzLJ83Md9JIHId6DHfz2fOy3ACxjKKrIbOi+EDx6pEvKTz+FJMnI8wlK2YCHZenhgu225O5dHXcMDKofTp2wGM5wo7GcAGOYJosx7iuDb2/qmSqFRa4cxlP9vjbWXXLfQXo+lmPrxTEON2bCX56TFwWjLKOXZfSBQinW0hQxGuH2TnHOHiGOjhAvXyzTi81N2NnBTTrUpEMyx8catZJmJaWmBojjZ/DooVZvKYXXauHWaoubJG7d0jSqVnv+5Gi621thuvAqvQWWuIfJgA2/MIq0BDWIyPyI3ljw9JluUJyc6BiqcbIAaKOzXGPmcx/X3aTdrtCqTvFmAx2lZzOyoqAfx1xMp/SVYowOXRV00C2AdDYji2MqQhAB7Oxob5DVQZ6ex8L85gYD76oSy1CDLy9Lnj3LSNMRRXGOVh2K+Wfdolbb4PBwjfv3q+zuWlS2QuL2H9Ibb9B+9pTo07/GBuSc815OpyAlcncXe6NDENlcPoG/+itdQTx9quj3J5TlEWX5nDw/RXt5WOjG8y1Y8Zobjy3Oz4MFjGjOpbcV6q0G3LKErJAUrTVUx4VKhAorehkUUBaIIl9gmPHY5XLgMC2Wz7V5/s/P9evxY72XZjNJPJP8y8OIOw862FGENw+6LmBHEWJvj+LwAeneISN/je6l4PGF5D/9TvLr3yjOzmZ0u2OKIgQCLEvgedZia5t7+kOcBd4o6K6W5KsVgGE+DYcJw2FMt1twdqZ49Mjhk0991tcd6nVBpSJeOS2TRDcMPv9c8fixznAdp6Ra1cFWCJtq1ZpzVMXC9tS4dhkDcGOVavTbhvljNsnqZW70TVSPSml3sSQXi0mlo7FgNIGodAhcX0tfz8+XhrlzPmh+dcUojumhndCQkiQMEY06wrYgjWE4n7Qwnw44FSGTOGKQ+MSpRWCndJozOv6Yan+Apfowm+pF2t1FzDO5xWTDJNFqnoMDskabLLNJEZopZom3y3TnnwFYjkMwCjTbXmKnjgOOjVAFdjLFly7VyKXRWE7IvbgAKW0gJAg2qFar1OsVGo017t4NuX/fYrOZEky60DuG8Rg5GOAnCTWlcNBiaRed0bi2jWPbuLaNa1l49TqyVkMcHuoDaF5WLjDdG6O2/F1akVJLi4PJJCdNE4pCAlWCoEK1Ktndibh3p8btTY/99oSNjZzGXkRjzaVWsal0XZQvicsSXynWpST64APEBx/AH/4x5d5t7GqVilPgyYwis4jjIZPJKdPpCfAS7VoxQisdFZpANQUSPK+G51UIAhfX1fHu+nopijJivre5zJbJc4gTSVIJSELJNPeY9n2mMy23DdycapBTKTLCeASJC3mAJSwqIqEiMxAlM0cycXwuhM/5OXz5pVhQW1O3itrYxNrYwG+1kK6LHUXIBw8Qd+4wqW3z/LLG6cTn6gqePSt49GTG0VHMZJKSZQp9bGeARCnnlSD7Q2PJGwfdVeWF6UQqpTPLq6uY09M+eZ7iOCWOE+C6TTqdkIMDyeamWEiIpdQP2RdfaN/k6+uEshzh+yHNZoTjSGxb0G6LhfOQwYGazaW4yZhPmHlTvr+cVGwuI5JaLQNuoldiEqJZvBwPbVSddmbj+YE23T4/199gGOXDIdlwyGg2WwRdaVkktZrGQszQJUMrq9ehUmEiIs6uAy5jl9FE0HRS7jQHrKkL3IsziMfLU6jVWsqgtCmvfrNzW8MsaDNJbTI1dxgTvL3hDSy7JEYFAEvDjlpt8WVWkSFnQ6pulbU1ByHF4t4dHYGUFkL4hGHA9rbHwYHL3bsOd+5Y3L4t2BIJ4ZMLOD+CJMHq96nG8cIGyCRKCIHrOLjzaRHC95G3biHnhw8HB0uPB7NWN9ppfTVBMbdVWw3ruVsa7OjMGTp1/vgPbf63P4F77RFu7wLHldh721jNGlaRwGTK1EqZlCUB0HQcrD/4A6x//a8p1zcp2+vYQFTkVFzwfR/H6SPlF8AzdHZrzJMkS4ugBCHSuaS8gedJPE9TIQeDpez/puA5WHb/48BjZjlc9iUXl7pq7Hah1bTY3ZFsihxn2kPgYbsNpG/TsEbU5QTygklp0RVtlPK5vNT49d6eflmNCLa2sDc3kevr2uVuext+8hO4e5eht8G3D32+f6af3dPTgqOjEf3+aG5D4KCF5Rll6ZDnSjOVWLYzfkhz8Y1cxkxZYDh229v62Z5NCuJJyRdfCSZjj+5lwWSSzN/YbOEIf3yspxToBCgjz3NevFBcXKg5hisIQ4tOR2e3ZqzM1taykWlsCU3PxraXMn4TSE1GbOAOk128/jlu4iqLpRHWaLSEXKw8orq2sZwtYoQAc/ZCGcfMimLhiOuWJWrVCtGo1kzjaW+PPGwzzVysMmMjiGmlZzTPnhLOzpenjJl3bzh9hlQ9v7KwRh7WiAlJMgly7ppkm7FHb3GtKmpWX3P6lnIccmwy5RAnktnIYjS1ALGYkWcO5GrVxnVDtrctHjzwuL+fc3d9wHZ9Qiue0ojP8OLrRUCXUYQ3nWJHEVaeI4pCm6A7DrbvYxsOcRjqjbuzo+vS9fVl6QXLTXMTTnQr16ofkLEuTVOJUha2XcF1A+7d9fnHv6zwh/d7vJcdsXl0pE8g34fgI1RWh8ePST7/nNHzp1wLwabnUa/WGNd3GFfvM5lVmDx1aUYpm7WEMpdMJ2plfHkFIRo4joPva1PuJMnm8OAGUq7TalW5dctejLEPgqUS7W3ZC+Yy368ptHqk+WAoOTvTOUqvZ+jrgrV1QRn54DRwbZua66BETpBPcWdDUIpcuUR+vujhGNHhxgZUrSmyd4moVLB+9jNKP6DY3mO0eUhvusXji5BnL23mhm5zOEWS53pv6oDL/O/6er2J9kPiyRv76cKybN/Z0VVZYOsypt20ueg2SFKLokhIkhyYMZtlHB8XXFwIpHSQUgBjlJoQx4o0VShVBxpUKjbr6/qE2tlZjjQx47Mta9lIM/L54XAx43AxytrYTxoVjTmdTcC9iaC7itOZpPTiYs4mCGtsb+/D7nO9AwaDZXpfFBRlSaoUc2KVDhS9nn7I5uPIcRwdIOaT+Ip0jezapmJP2QquqJ49JfjuExhc6V/aaCzFB8Y60WSd80ZWSsiUkCS3SHK5uJeurZA3ZOwOvMqEn2eQCkFihUxElcshdC8FSWYRZ0sBkiEONBoO9brF/fvw8ccW95rX7Ksj6tMTnJcXONNr7DLTn3l/HxGG2GtrWA8e6OZlkmBVKqgwRJqnw4D8Rt1kXsYzwuC45s8bzHiVWrpXmleW2YCP50mqVZsP3oP//X8teZC+pPn5f4aHX2nearu9MDsp//RPSX/1K3rdLudSUoki1jpr9MoGR2chFwOHi67gwW1F811FMlP0enB97ZJlLYSoYll7+H6Tdlvg+4J+v5xPpdYtxrU1n3feWUJxqzbJ5tbexGVEiaYHE8fL0eumT2SaZHYjQlT3cFxBEx+mE7yL6cJAy5KCaqVgU+ut6PeXgbdZ9pCPH4Lvwh/9EWVznXR9m4tRky+fRTw+cnh5rMdsaRtoOWdOKZYW+QA2QkikfNXe8YeSXX500DUBRsr58+srOq2STqugMj4nvD7jslWj+4tN1tsOz15WuOqpOXdWkqZybpRhIYTAshxs26NSKel0FJ7n4fse+/s2+/uC3d1l0F1fX3rUCjGH3xxFtVJiWYqykOS5XJhoGFdAk7RMJksY5KbZQCZMmfUxHtDrd6vMdnexN3ext7Y1tmoQd9vWYz+EIEAXeE5RMLu+5vLkhLBSITR+d+32YuyEP76icX5BZXZJszwlOHmsJZNZtsRfzLFrTgMpUdZcuFCtkc1cJjMXpcQcHtKyZUcqRPkPEHRdF+X7KD8gd0KuZz6XM4/zuae9ORgN5Gwot7WaZHdXN08fPIBbckr7+JTg+jn0r/TvWFuj2Nwm3bhFWalh+zXs3VsQTxFZCkGICHxKJSmRZE5AZocI10a6NsJ1kJ6Llcc4zlSPyzEl0eopfQOX8S0YjZZz/6S0aDZdNjcttrdt3j+MOdycsHlyBb0zihcvKJ48QZ2dISoVyjAk/pu/YfLwIQKoeD7F/gN67/6Cb7JDfvdfXHJl6fMilcQjh0+/tDg9FSgVsrm5hWUJHGebSqXB+rreFicnevqulArPUxwcSHZ2lmp9k+WuZnU3kekKsRyfZwq8szP9MrCDES2Oxw7dtk0UaWFCWEosbGzXo3Q84jLkoufy/EK/72pV2zLcvw+bpYtVVBlYVXr2OpOyzWzY4uWFx3fPdWad5TrG6LxI4vvuXEQ1Fz3NCXp6fM/SxPzHrMkbZboGE/U8nXXWo5yam+C8fIz45Lfcr96m/k8tPvhJi0+/a/LomcXxsZ4bPxjoEkdKOQfjQ8KwYH1dB9VOx2FtzSGK5NK4pb2Uyc8nliClPv2isMSzMoQqcS0Hz9Opq1F6VqvL58ZQhE02ZT7LTVzyNWmjmb14a6vKqObire8it3ew1NxVrSxhOsWZzahLSYzWfedFwWA04nmWsQ2Ekwn89KcacyxLuLigenKF8+wU+/QlbvcIel2dQbdaOhs20zth2SIvS90gExLl+6QTyWSyxOf0cM8SS5Wv0sduZHHkXBocUVRqpH6dq67NsyNdwhlLClNymynTpu9msPytLWiOYpzBPA2K44XmOd+9w8hqk9gh0XaDcD/FokCilWXKsuZjYQSjqYYzbEdguxLLEVi2JPRSrFqMNRq8OtLghuEFM/m639d7uVqV7O05vPuu4P334Z3bMUHcWygUC2CaJBRXV9jTKbll0e92maIVVR3XJ3nnD3j5P//f/PWvN/h//43k9oEONF9/b/OrX0tevBQ8fympVCocHu7S6QgcJyCKdBkupe72n57quWSNhlhMiu73dVA02ZxhAL1tP2QVecoyfd9NT6TXWx7G5qC6uJh7irS1CmxjA7abFreqEV7LJXMrXI8rfPnU5de/X3pDrK/rR6ih1rEzm9PnHp89iuiOPMapzfVI3wellsWldqUVnJ/rmJKmYo7r+oCL50ld3a+0AP5BM12zBx0HKhGEVoKXjpEDDWR27nh0Nq6I6hYVfLZqIcd72jns5BSGQ7FoelUqHlG0lOSb3o/5EKbzaPoa5llY7XNIqQ2+Vs1EQBEGEAaKcj6h1/PEAta8CTzKXGLesTHS6DBcYlFXI49u7OGGmzi37mgzZhNRZjPkZIIrJaYRnCjFIE0p8pzGxYXeOVtb+nvmBjne8THes2f6aO52X3WUNl09M6XBpPXzRUwKm2RmM00EabqUYUoBllDzSc1vuBCvB6dVcxvbRrkemfSZlj7Dqd7orwddMzTCzEM0fsrNmh53FKgpIo0X8IyybMpmm7ixRbcXMBx5C762gSkcSxOgpukyoA8Gc+lpZf6ywHYkoauWa/lDOyM/YnmMENHAYHEs2NwU7O/DO+/Az34Gt6MMP58s5eJzL2Axl8+XUZV0+zb5gzq+BVGlwcvNX/Jl9hGfnud89fUMP7C5fduhN7B48txasA52dlw++MBld3cpHDSiIdeFVhO2Ngo21kq8wMILLZTSeYKxolhVXt0M+2dZkBmYwbQ+jDvoZKKDbhgqKhVFq1WwsZGzvZbwoiVo1Bwy16I7Uvzmk5TPPsupVh0aDWdRVYyLiDLxeNi1+O7M4fxCq0fNfIVqVR9AS0hSUKlYBIHAthV5LrEsB8uyaLXEK22TH2PR8caZrgl4jgMyiSGZj+Qxkyp7PdokfOjD7cMm09YeF0mNR48Fvd5ys5ts1ECPqw0uKZdBdjpd3vQwXLo0zWIxnwGmsDKJUyzhQ88ucZ2SvJBkpVz8v3nvNxl4Qf8sMzBvMtG/5/p63gPxWtTvvYvbrulAadswGKB6PQohiFlOPquhx8vkpsR9+nTZEcwy3aU7O1tKbM2p1W7rHdrt6q+r1/VizafuqjBkFDtcnS5NtYpiPgjTBuVJ4C15dK/zZ1Y6DUpI4kQwWgmspmdoICGTndj2q3CrL1Ks4bUu/b05RSXPKRFkls+4DDk5tzg+WwYFM6l2TvpYBHjDLjFzKTc39blWIUeVy4D+ykZ5y8skK2a9p9Nl8tBsLuB6Dg+hkxc4vfmb930s3yewLJTvaxXhnQdUfvZPEA9+gh1CgsuXX9zh3/97+P77KWU5oiwjiqKG6wparaWv1L178NFHunKApYrRPFsb64pbGynbnYzS8SgcSRwLzs9ftaS4icusiak2Tc5gcgijHF+9HZeXJVmWYdsxvj8h9BMqnsJ1FMqaEWdwfm4xGDhsbNSRss6XX87tNVKLLBFMZ4JJLOj39aNkHD1N/DHcW0PdrlQEGsfVE4HnTExdeTVfTQp/CBL1xuKIVxRdaQbTyYLHpaIIpEWFGRV7iKjHcDukb0s2Wj79kUXkZlRCRdS0Car2HMsTC5qXCZCGjWButDmdTRKXl4KskEgLpCXw3RJXJbh5im1bWNKmKMSCcum6rx4aN8VekHP6mZmIbJz1+309XaS6V2dj9wCnGWE1msg817VctwvD4ZIDjmYBxkoxyTKGZYk6Ofn/2zu35ziSK73/MrOuXX1DdwMkSIKXkXZmpF17ZCkkr8PhBztCL/u6/+s+bWw49k1+kLyr0a4045XIGQ4JkgC60ehL3TLTD1nZVcBQ0lwASl7jRCDAwQDVVVmZJ09+5zvfwRhDUFUEZUlwcUGwXLoOuuOx+8Dh0M1S2IULNklhNML0h5TRkML0WeYRyyu43G4MrHV3cV3H6StcPhuGVJXcRTLd5EmXvwot6cJHrSElcn2B8BF8I/VnooRKJqyriNOFY4z4vckxA1q9FE8/8tTB7bbFC40B18r5itOFa410u0VFPjfi9YH2hjUHY83g5Sl8/jt3FIgi5OEh4oPvuc4b9w7ZPvorTt77H5w++DHVXff3889gPnfJ6CjSJIkhy+xuc/VQoE9I37nj7ikMLP1UI9GElWadVtwfr7kzLthEYzZRTJJcDrK6B4HrGJOriW3v8Pr9VpBws6mpqprNxvVJrOslTrppg1s5vmOexmGuWaOPMOQ3v7E8e2aoKklVKQYDwXjcCs9tt+49R5FPbIrdRhCGzulGkdpV9A8GLsF/eOiWn2dPeZ94I07XD05du9ZOaWFBm/ZuDu5gHz9G+PrgszMIAnrjLUfjQw4maZN1LoiSPUgHbDYua+gbB/gAbTRqKWI+jPc7ov8qSlcDHUeWLK5Qb14jz06R+1PE/pQyl6xzhRe78keZa4PqhCuw6yqueRGxs7MGK0pSHhzNiKYJvfEIWRbw/DnyzRvixYLBcrlrqFjhps9pU8Janp1R5DkjYxgZw0Br+sYQNOW0u2yhP2l4ucS9MebeA8qDB5xcJJydxWgZIjuynH6c48AgdQWm05DxG41FJ8LtCqb2etAfYFcRet129emyFTyN2Np28/LqfdJoKPJWv6H5H2Y0oSD27dh2UaSXtTg/b53uYuF+7gtzPDnB0x9DqRHdssprxHK75nMi4O6jKNw82ZzlMF/Dv/wK/u7v3HN+97uYH/yI6if/DZKEMIuZr/f4x396wMf/C370I4fdHh7C3/wN/OxnKZuN4u7dkCdPJL/6lduIPOy22Tho4+KiqQFBE+icHhsCVlRmRX+xgYuSaiTZjMc7rNUHPH5+Xyc8192ffZW8tW5c3OkgZ71eUpbbplfgHHiFE0LyXbF9P+g+1k7ZbFLOzh6gdY3WJUKECBFx/76DrDzEeXEBi4WhrmE6lTsf49lRvV7rbCcTN5c803A0ulz/cyOYrj9tKdUelbaVYqNjwv6EIBsgDg7g7l1XWBqGu3Y8URQxPZjAIILTHJHn2LhPFYKu23Zp8/mXq8yyjN2O62mu1oKuLVVuMEbToyaVW1ieYk+OoRdjphOq0rLZtNorXtLxutaUwOPJ1qklZWLXrM7jSbNZzONVTLKXEk6HBJslHD1EvXhB+uYNw8UCWdfExnCGqxE6AwpjKFYritWKOziGQxSGZN2yIP8yAPp97GSCHYww033KvUO2w0PO13CWN5q7YVvtmqYO+1bGIKq65aZ+o4Ho8HO9J91hPY6TJnJ1qbjGF7P4TcA3Auj13IT25du70u1GSs1GrlWR7s8oVbLzxV5bx8l9tvk2j911y8L9PPbHyFBrhK6uvfX624apQ1umrt395ssSu7xAPH/u6nXv3oUf/AD74fcwD7+DGY7RxjD/VPBv/xjyz//sa00s+/vwnfegKCLevIl48MAlm5RqW815nHTTkDOEABsaAl2Syi1pcAHxEvIcvdVsRcWZafvKOsbAZQzzOhyvfw9eFbQbWfsoUuuaPN9S11vaIo41rk3XCQ6Y2+Ki3nFTfLIBDJuNi5DDUBKGltFINP7D1QXkuWax0FSV4OVLh9l6UZ80dZv2aNQGgLMZjbxsy174OuPwjSLdS2C6hHMz5KJS7A00eyNDOIhR1jqHO2vk+5IEM9qjDhKsiJHDCTKrMXGPWgu0uXwM8liej368X/H4rocdQmrSekVcr4jOV1A2wjJRhA4itIwpjGtr45sRd2VTr8OEsATKkgSWfl/sBHYmE3ecnc8dzfJf/xUCociymHi0h3z4kPDEOdwgz4nmc8R6zRbHZFjTthX37zQGAi/Ostk47GJvr23aOBphD+5Q7x+SDw841yMuGjzOdzz3EztJ3OlAYhD2mgsBup5VCNAaUZeEMqbXc/dyeNgiIx7u8J1yfKQ7mTTNCkcJQToFGlihtpS1pBApdTQkqlzk4Tfj9dp9hp9TQrQi774orts4oteDML9ejP9t5h2J1wjxHHOtXdeV3WnFq549eYKcTQirDZsTOC36zFcJR48UWrT5g6MjePTQ8pOfiEuqmnV9+X37gKMN5EVbPeihKq0xNZy82uOTT9pcrYdCuu3Gv+1Y7JKdYbv5eZQMWh2VzSbm4mJEXfdw0axX1DjARbpOd9s55DHWzijLMcZUVFUF1GjtggkPK1xc1BwfV5yd5ZRljpSKFy+GQMr9+7tapC69ncnEffnTE7Rj+VULRr6xtGNbfSE4rzMuthn5AOhBT1ZE2xJlI9R45hZ0cMN3ygAAFnNJREFUGKLjjNzG6DoijF3jSaxFVwZhLKGw9HuSMJSkqSsVjkLXqqYqXdVXUTohZa0deC5VTcSGrDqHxRyzWkEUY5IetUqobEhZi2+ldP9HxwMIpAVl6WewtyeYzRxMsnSBA69ewW9+A72eYnqgkMGYaPqI4NGKaL5G1RbRO8bOT1lXFeu6ZmstW3DshobhkFhL2OshGs4uZYnRFp0OsaN9GB9gJnepDh6yzfY5b0SBvHPzEWWaWpcAwCKtdp1yr7n66tIMrGtEWRIi6IWC6RDMgWCTGdZ9i1QSoQSbXLLeCKLIvX/fVac3ipC9MXU4bsSFFJVR1KWAjSFBc+dAEIaigRfELiHjH8s/u3d4fnMcDCxpAmHdqTW6Ae/rnUyXzuillj1rY5M7HDI8eoR48gQePoK9PeT5kvpizfm2R1HC48OCWa92Mh4rGOiaw7hG3Ic70nK2ijldxSSBYjYL6PddQs1LS+xEWrTLiVRxjBz0EEpRG8m2lLz5PODZM5qCiTYS9aqX15EPeVuU63PxXa79YhGhlEQp23z2AClHCJEjRIEQa6w9w0W7Q2CEMSOM0ShlCAKL1nZXiOFO1IY3b0pWKxfm5HnIq1cpQsTMZoJez9Pm7O7QNhrBuIl6hyOxY9x04cob4+l29XO9OMnZGfz2t9BPJf1eyCjtM05DksggpaKqQlbbiNpHHYmlF5TEomSmSpJeyVYN2Kj+7kVkPUs/cw9trCRoTqoYQ2grImpiQkhH6DRDV4bSBFQ2oBZD6os2aXETfmVnWiNsSZYE7O/LncZwtw/j8TH84hdunO4OM2bqiGmcMv3hfQb/8b8Q6Aum6zOizz7j4OVLKq2pjUENh6jBgJExDOuayDvdxntsoz3mckLem2KHU4TdIywyRNDqhV+FVIQAJa3DSq+rWVzX/NbvByDPEcYS1ytEJZBCMRgqKpFT2hyRpoheQi57bGVGGIldhGoMbAtJEoeYCpYXEpRkOBQkYU2otmSxQfdT0jTa1TUMh27j83IT3ul5Z+c7wQz7hhCNtHULrXTpLdcwYbqwnHe6HjbxPRY/Pujx4GjG+9H7HP3IEB/tY/emFOGAbRBRpIZpYhjK1xRvPqdcvkYbgbSGw5+dIn5+xuAkR57m7D36kHtP/pLkwztMJzPifsThITvurRDuc19vJLqMyXVAfygJheTlMbw8Frw+kTv4XIjL0hTXlYD2Y9PtJBHHLnr3OtHOGQuSxBUkONhJ0OtJ0jQhijRKjTBmD6gQIsbahOUyZbl0cEEQRJydhZyeuiTZ+TmsVnYX/YLCGEFROPy33w84PAx49MidyKLQFRAlsXWnQyURCkTjYb9OUv4bO12gIZu3YszLpRusft/p3t67F3F01GeYQtDABAsne8pgALpvSZKSKNoQqTWjNGfdC1ilfUyzZvupZdjXGCvJK0soXAWVsi6bL3WNJUSLxGF7It5lprUGfdEmbLpO91odb8MFEsaSJRIV2d0iryq3G7565cbo9NSR0Pf3Mx4+zHj06D5PPviI+/cse0PNQC9If/5zxL98jKgqJ2d3eNe1iPY1mGlKnWVO2fzhEasy4/UXguWF4yOHwKCEXgdOdX7VPbTj5boOxtKa69cY6HKBvNPdbhFFQVSWRFVF33H6oFhCuISBCzvLvqXoxwSxIowE87ng5THkpaIwiu3WcnzSZJV7liQsUVwgAw1DQTZS1JVESR/dO+U3L3RkbVv9O5k0TjczBNRuA7JXqC3XuBl5HDdN6RDrnebI8+eGO3dCZg+GiO++x+DDEePDFDmcUNiU86CPiEum8Yo+pwj7K8TqE7AWW5bYZ88wz57RWy7Jzs8RP/0pPLLMjioevx8TjHqMJoq05/QtFgs4fS1YLsGaAC0CRCqIleSL1/CbT1tYz2uYDIeXM/XXad0GlX6z9WsojiFNJVkm6fVcEuvgIGBvL94hIr5RpKe0ae0q7I6P2wT3737XzgPvqxwkIBAiQAiBMTVSlozHkvv3DI+PDI+OLEliSaJmfVjLughY5+rSOHxVjPtrOd2ufmYD0zmOZ+QmsK8cKUv3Uvv9lgPnF3/aNECMIkhCiKXCErJVQ7bBACN7rnCqySqLptQr3wjOFoK8Ka5SQpIEEYEMsMZRx9aFYlvuxmWHE5ZlG5XfRFDnP1BgCGUNUjAdKwIlkbJtXvrihRsXv0mdncGvf20YDGr6mSFNJD2ZMFw9YbDt008NWWpIl33SuI/VBltrqmVAKSLESR/1IsKGAqNBqi9LLXhWgI9OoshBNpEyCH0DDvfKmOzIt56Yu167AfClZ540+/778MEHqE1BdLFC9mJkLybWKeNhjyBSOwZLoEBVOdlmiVycI84XzlnmOWFvxCyKUXcisiRkOAyZz91H+E3XV+G1kZtFdJ3tdRO4G/N+3K8FT4M05hSt3/D0aY61Of/nl33+52jMD38y4L/+NGAwaTQbasWClLIckEVD0jCEZ8/Qn33G2ekp87MzekVBlueEH39MVBTE6YyDcIJ6dETyvScEkwEqlAzrkMfrmEIEJJUk2sao9YSNGQFNlWLasIOKdp1fJ2uhOy4+oemjRU8Zg3Yuj0YuSn/vvVaobjBwkWgct/kTX2Thk+Y+lwst+cUxZAImkxRrQ6SsGQwEh4chH7wv+c8/NHx4b8l0+ZLkn04JIgmxajNqog9kWCu+tjzHV3a63cS0/+7XUxiy6+SwWLT8y9GoJah71pAH4gGwAllKdBGxCSLmQUSsIBatg9853dI5qsW5r/+WDAcRUdwKZsznrciyx4bStHW6PhF3Iz6mie6CQBMoQbAHwz3JYNiKWI3HLu/19Knbgb/4wuFKVVVQ1xqISZKE+/e+w73D73JwAPsHMFnCXiAaH2bZNm1JhGz1bfyxsTvGHndXqh2TQFnCANAGcaO7UGM+2vVCA/O5222Oj1tO13nzUsdj5GZD5LNowyFJtsfeMEbGyi16CeORRZzn8MIRb8VisaO1hNOaWZYx6mcMRhmjacirVy0NSes2qTQYNAk01fCT4frAyivm143fAD27xQ3RCXX9Cb/73ZynT5cIPgBxyN+uBnznh5L+XsPv1ZJSJ+TVgCAakkYRvHiB/sUvOCkKnpYlU2AGZB9/jP34Y2LhhNrlD34AF/8d7h1CEBAlKUOPcZSKajNgsQ7ZMtrRpDzE5wWkvPLbTSQbvTP3G5HnxHpJZl+a++ABfO97DTd57u7z/ffd+/z0U/e36/VlplJXYM/7KwdbKLJM7Z7z/n1XOPLRXxn+018WvDdYwv/+xF3Yc9kePnQJzlRB2tsd6G7E6cKXL9rFdb0+bZa1ojRp2pDQV5pxUhEnhqCWCIuLDBv+ilxvSBcFLEqCRBHEATZJIEnBpGxsj6JUqKDtoqK1YLUGu7pcQuhfoC+e8Ak0v/brK7DddZoAMAZLjZLuAwY9p3wkhGiOaK588P59L10nWa8jNhtLUSiMkQwGgjCWFJXbZGoNm9xPeKdsFCeXucye8eG1hL2TddG+ddGCsig0ourguNfYFeGS+ev6l3NxcdnZHh9jj4+x6zV2vUb88peIskR4OkHzpZKMKM4QgUQK4yJSYxBl0Rbrr9fuZU4miL09GI1QwzH93gwZT0hmKdNxQqUFVS07CmaWJHaLQGhaSOSGxuRqkV6aepmMAULcAwKstVhKsF/w+ecV//APE54+TZoMumtJNZEK3Ru4SfTjH6MODphqjdCarCzJypLo/JxwsSC4uEBeXLgmnMNByyHTuu3q+t576P27rOoep6fuVfnu0l16W1dH6bo5ut5pdWnmHnsWAmRVoKote/2ag7RGJRF7WUbUCxmmGq0tVivqWu242r660RfjzGbw13/drplJmjNNVqRB7dRTo5x72TmHx6fsnbxGlMft8bShwXqqQlVaNqaFLrvPcyOJNGjL9bq6IOCcbjcTudnA5kIjRzmx1iAUaNH2Kzk/RywW9F6fkJ6cIJLYeZSGc7U2M1Y6oazVjrohZUvy7mJ1fqf0R5WuOl/X6cKNBDPOmg+TgTvuB6kiywKyHkwmLht6/34LL8znkvk8ZrGwLBbikui67z/n5SL9jt00kNjRV7q0p67D9U43CFzSTAkLlcZWZctWuCnzE8TvfN7pvngBz57teqrYokAXBaooEJ991nLtGs1bGQRE0gtvdHZ4z18Swl3faw43Z1C5f0B2tKF3zzDZn2CGIWWtKNqOLwwHEMcWWVto8O6b5OfCZYjBc5GTZETbHifHlcc85/PPLX//9xmPHyfcv+/mzeEh9IcBJmuc7mSC+ugjZkIwAcRq5b6ePUM8feqOU1984SbIeOwmhMdbwtCN5/e/j96/y/KLlJOTtm8qtOvk64q6fJ3x8MlS73Q9nOADt14PenVBrzgnrDbIusD2MupZiOgpQluxXFhsLahrtaMd+oDLv+9791yk7AscHg22fCd7wyBwmVexXKCeP0M9+y3q01/Dyy/cjfl5Np3u1kxZwbqEshNw+vG6MafrPwDaAfNHp66qYFnCaitZrCNkaAhjJ14eCghCu5OgF0Jgewl1kFCpBDnIUMM+NnA6a8q6BICHNLpO1j9kl+u34z5egRJumocJbcQLdaPYZYmVRKQCNZHEsWQ8dp0wVivRBGvue6Npcinx152EQeD8kac9+epf52wtcYPZhoEhlE4bV2pc00Iax3VjFI63WJfA6DtHeNZ+HCOWS+RyiTGGej7H5jlmuSSIIoIoQktJKQSmuY4wBqG1ayTZcJ9qrRFCECWJ63c1nSL29xGvX8HLF6gHD7D37iGTPkHSR1tJLQVxXiOrysEsvkLBl7Z1le+v2XzSyA/F0VFEUfQpy7toHbHdGrZbRVUNePbMslzmPH8O+/uSw0PFwShkYqf0TUhd1JjKECfuJHUwzDmYFoySQ0b3HqDmTfb24UOnxaxC9GBCtSqoakk1mFBxj/VZwuJC7Zg+V4W5b8Lp+ut080QuorZkjRhRJCpiURJV58T5GXLlMmAmCJ0aXBAgipy4kByWU9Q4Y61z1rpgQ8lGVtRFjS5qJtM+0zsD+mJD73zJdP6aiXlFUjmKk10ssI16unj92i1GH9HcuYN99Aizfxfd30MXKTZ3D/B1tSi+sdPtlu75fInHD7vCFVrDaqM4WSXo0NKzgkwYslgShLKto9/bg1pT2oStSQgSRRRLrJGoShE2O6IPnnyizGsc+OqRJGmzlf7zvRPrHmVu3LocNa0JhESFkjAM6A8ElRaXEnx1JwLr3rfPxnYnuz/qdTeYpuUYYWBR0iCMRhh3EWFtS0C9SUjhqnUBL09QnU53QLTY34dXr1CvXlGcnVGenVFvNuj5nEQIlBBUQjjJy+Z6AhcTKkAKgcGVTEugLwSq+Qwxm7mKlLt3Heh3cUFwcAe1f4BVClSA2G5dabF3sD5E8sehG6pG8xj7aOTpbCFxHJDnMUUx4+wMTk4Eq5Xm5cuaFy+2jfZ6wMFBwmgYEYp9hJ2Qby11o220N4GP/oPho0PLe3ePGAxOULo5W2cZjMeYsEfxfsymVKxXsCoCNjph/SpkuRS7Q4Pf6D1T4Qbzi5fMO/rhECZ7FpkXyPUKsVkg5meuG/Zi4TbeMHLk/c2GNIh4+OAJh9MphjlGLDDhChOvYJtjtzlqckiwdx958gb5/BnBySuCedNhu8kt2OVy18NQeKL4bOaqJP7iL6iH+5SDGTUSW4hdCuDrxDHfKtLtnsK8A+6+FO+MN1vByZkgL5ujQgpZokgiAcaCNY7gX1u2dcSmDokanqY/HnTXhcecuths19F3Hb6HFLu+5l1Eu5cGwRjXAkcYhLQOY5WCRDlxZIv7boyTsPD/NlfEQLwJQAiLki10IKWHEHA0MNOJ1Lqz4V1FuNDeuFLs6io9WNeozggnkIp684bg5ARR1yhjXFmytUitiYxBNdcSVYXcbJBVhTQGo11LefzvaO2q9fxOFQTOCU8mLrJOk/a+fMbXj5Hf8W54jPxtZZnLTQghyDIoCkVVRbvS8fW6YrORlKVuCj0UWeZaXQnhGiMGCkwJdQhbC69X8G/Hbu84WSVEskYJg6oCgiqCKKYOEyoTkJdQlC0TUZvLJ6obUrh8q3V5zN3XJ5VABgKRxjAaOmWpKHIyl173sapcx+v8nFjV8Oa5g6+8mpJP4mYbmCkQOQxDkGPIQnfdhkMm8hxrrROTGo3g4UPMoyeUjz+kSA6oRJ+qCCjrlp/r36k/0P0x+1aYrg9k4HKmvJuj0W4joqpaEfEkEfRSRRwpwO7me1VBXjhamOfrecftoz4vFeuPQF06iHeufs10/c0NwnR/2LofLATCGJQ0yCb0tM1ss0qABYtofibxYyN2b7J9GGEdbCEECJr/1hZhmt97m8P9U5jnqXUdrz+y1HWjRH0PdXqKPDnB+kXUJIaiskT58CuKnDd59QpxcQFVhSlLwmZChEIgPUDoPYnru9JS1ZbLLztZaIsg3sF4eefiGQI+edRdN+4RFFUlKQrbUb+Su4ODtW2SyNtm43jgz4OUOIiII0fmT3uCNJNEiTtx+YbPXYiuW97fjXDfhXWdbhAILJZKC0IUMgxbfU7PX5vPXVLW/7G1bX2vpwj5n580HS59QmQ8Zico7AnBTeZNaI1QqsXADw8x0wPWdsSSAaZQ6EK8dW++0eKItw2Y5751S+67OizQxWMFWgvyxll257+vHvL/hsu5jS7Dqev4/fh2f/5nZztnyZVyU9H9iXO4O27eWy/keKnNP3cP3H3odwUh/CHr8gv9jPTnR+9ZhIAkQXoJpyuUGFE73HUXHq5WbjEsl87pFgWyLLF1jZIS4bOtntoxHrddTL0qth8nP5ne1dmZyx/hudOeFtVNCLv5LHcnPS9+5jPy3un6vcUHOOCvoUAqTAg6hEqCsmBrt+i7QVLX8XbX7ruyt52Oi0K4+zKKQMcIIrcWQg3UkCUwDhFqDFWFNQarAmxZIWcadI/aKrQRqMECNVtg7j9G9x9jsjG2P4AodqRvYxFViTDaFQyFCtlPkX0HyehswHYbUObqim7F25/hj9m3crpXj73dgfMcxKtJLP83PsnmrZuU9pPxqt/wk0N2dmm4DBu+U+jgm1r3wd52w1/lId62q/xZ7jRXrMuacGTJtjZ2NLpM23obULbZuDR0k54WdY1sfk90PYY/p/rr+tbVXsbMRwl/QvPz1d8qfJnv2R2Obm6voyN06VDjf7+rNdQdli5c0P2d7v74p1xD3RPydgvCBggkcrcRWay2IFLkaIzou5DTGIs2Aqstav/7iLJgm7vOHImqiGVFEfQoVEYtQrQIsEjQAoFFBhYlLUFgCSNBlCrCMIAywOoQY8Wl9/Ft7Fsl0rrfv679vqO/n4Rf5/P83/+pJ8w3sv9Xnec3tavP5h2kzy5560bCPqzz5WRRtMOshNbtYeDqruuv64nj0IaD3aPRn8CuzuevEllezUt6JOXqOuoO2+9bD13H++dkV2FLlyKVX5oGQkTIJLsUePkjv99cfOFj3QPtawa2l1ElOjmTnQppBHUAkQS0+7pOZUJh/8BVhBD/jlf/ZbPWfiV3fTsmb7f/X8bldkzebrfr58v2+8bkDzrdW7u1W7u1W7te+zM7XNzard3arf37tlune2u3dmu39g7t1une2q3d2q29Q7t1urd2a7d2a+/Qbp3urd3ard3aO7T/C2E59V1JRSVDAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f668c0c30f0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "for i in range(10):\n",
    "    plt.subplot(2, 5, i+1)\n",
    "    weight = sess.run(W)[:,i]\n",
    "    plt.title(i)\n",
    "    plt.imshow(weight.reshape([28,28]), cmap=plt.get_cmap('seismic'))\n",
    "    frame1 = plt.gca()\n",
    "    frame1.axes.get_xaxis().set_visible(False)\n",
    "    frame1.axes.get_yaxis().set_visible(False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "def display_compare(num):\n",
    "    x_test = mnist.test.images[num,:].reshape(1,784)\n",
    "    y_test = mnist.test.labels[num,:]\n",
    "    label = y_test.argmax()\n",
    "    \n",
    "    prediction = sess.run(y, feed_dict={x: x_test}).argmax() \n",
    "    \n",
    "    plt.title('Prediction: %d Label: %d' % (prediction, label))\n",
    "    plt.imshow(x_test.reshape([28,28]), cmap=plt.get_cmap('gray_r'))\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "ename": "RuntimeError",
     "evalue": "Attempted to use a closed Session.",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mRuntimeError\u001b[0m                              Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-51-336c892e9f37>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mdisplay_compare\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m<ipython-input-30-0c3a8f7a122c>\u001b[0m in \u001b[0;36mdisplay_compare\u001b[0;34m(num)\u001b[0m\n\u001b[1;32m      4\u001b[0m     \u001b[0mlabel\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0my_test\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 6\u001b[0;31m     \u001b[0mprediction\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msess\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0my\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfeed_dict\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m{\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mx_test\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      7\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      8\u001b[0m     \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtitle\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Prediction: %d Label: %d'\u001b[0m \u001b[0;34m%\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mprediction\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlabel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/tensor/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py\u001b[0m in \u001b[0;36mrun\u001b[0;34m(self, fetches, feed_dict, options, run_metadata)\u001b[0m\n\u001b[1;32m    887\u001b[0m     \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    888\u001b[0m       result = self._run(None, fetches, feed_dict, options_ptr,\n\u001b[0;32m--> 889\u001b[0;31m                          run_metadata_ptr)\n\u001b[0m\u001b[1;32m    890\u001b[0m       \u001b[0;32mif\u001b[0m \u001b[0mrun_metadata\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    891\u001b[0m         \u001b[0mproto_data\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtf_session\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mTF_GetBuffer\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrun_metadata_ptr\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/tensor/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py\u001b[0m in \u001b[0;36m_run\u001b[0;34m(self, handle, fetches, feed_dict, options, run_metadata)\u001b[0m\n\u001b[1;32m   1045\u001b[0m     \u001b[0;31m# Check session.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1046\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_closed\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1047\u001b[0;31m       \u001b[0;32mraise\u001b[0m \u001b[0mRuntimeError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Attempted to use a closed Session.'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1048\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgraph\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mversion\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1049\u001b[0m       raise RuntimeError('The Session graph is empty.  Add operations to the '\n",
      "\u001b[0;31mRuntimeError\u001b[0m: Attempted to use a closed Session."
     ]
    }
   ],
   "source": [
    "display_compare(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deep learning approach"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](https://www.tensorflow.org/images/mnist_deep.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ReLu\n",
    "$$A(x) = \\text{max}(0,x)$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "def deepnn(x):\n",
    "    \"\"\"deepnn builds the graph for a deep net for classifying digits.\n",
    "    Args:\n",
    "        x: an input tensor with the dimensions (N_examples, 784), where 784 is the\n",
    "        number of pixels in a standard MNIST image.\n",
    "    Returns:\n",
    "        A tuple (y, keep_prob). y is a tensor of shape (N_examples, 10), with values\n",
    "        equal to the logits of classifying the digit into one of 10 classes (the\n",
    "        digits 0-9). keep_prob is a scalar placeholder for the probability of\n",
    "        dropout.\n",
    "    \"\"\"\n",
    "    # Reshape to use within a convolutional neural net.\n",
    "    # Last dimension is for \"features\" - there is only one here, since images are\n",
    "    # grayscale -- it would be 3 for an RGB image, 4 for RGBA, etc.\n",
    "    with tf.name_scope('reshape'):\n",
    "        x_image = tf.reshape(x, [-1, 28, 28, 1])\n",
    "\n",
    "    # First convolutional layer - maps one grayscale image to 32 feature maps.\n",
    "    with tf.name_scope('conv1'):\n",
    "        W_conv1 = weight_variable([5, 5, 1, 32])\n",
    "        b_conv1 = bias_variable([32])\n",
    "        h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)\n",
    "\n",
    "    # Pooling layer - downsamples by 2X.\n",
    "    with tf.name_scope('pool1'):\n",
    "        h_pool1 = max_pool_2x2(h_conv1)\n",
    "\n",
    "    # Second convolutional layer -- maps 32 feature maps to 64.\n",
    "    with tf.name_scope('conv2'):\n",
    "        W_conv2 = weight_variable([5, 5, 32, 64])\n",
    "        b_conv2 = bias_variable([64])\n",
    "        h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)\n",
    "\n",
    "    # Second pooling layer.\n",
    "    with tf.name_scope('pool2'):\n",
    "        h_pool2 = max_pool_2x2(h_conv2)\n",
    "\n",
    "    # Fully connected layer 1 -- after 2 round of downsampling, our 28x28 image\n",
    "    # is down to 7x7x64 feature maps -- maps this to 1024 features.\n",
    "    with tf.name_scope('fc1'):\n",
    "        W_fc1 = weight_variable([7 * 7 * 64, 1024])\n",
    "        b_fc1 = bias_variable([1024])\n",
    "\n",
    "    h_pool2_flat = tf.reshape(h_pool2, [-1, 7*7*64])\n",
    "    h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, W_fc1) + b_fc1)\n",
    "\n",
    "    # Dropout - controls the complexity of the model, prevents co-adaptation of\n",
    "    # features.\n",
    "    with tf.name_scope('dropout'):\n",
    "        keep_prob = tf.placeholder(tf.float32)\n",
    "        h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)\n",
    "\n",
    "    # Map the 1024 features to 10 classes, one for each digit\n",
    "    with tf.name_scope('fc2'):\n",
    "        W_fc2 = weight_variable([1024, 10])\n",
    "        b_fc2 = bias_variable([10])\n",
    "\n",
    "    y_conv = tf.matmul(h_fc1_drop, W_fc2) + b_fc2\n",
    "    return y_conv, keep_prob"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "def conv2d(x, W):\n",
    "    \"\"\"conv2d returns a 2d convolution layer with full stride.\"\"\"\n",
    "    return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "def max_pool_2x2(x):\n",
    "    \"\"\"max_pool_2x2 downsamples a feature map by 2X.\"\"\"\n",
    "    return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],\n",
    "                        strides=[1, 2, 2, 1], padding='SAME')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "def weight_variable(shape):\n",
    "    \"\"\"\"weight_variable generates a weight variable of a given shape.\"\"\"\n",
    "    initial = tf.truncated_normal(shape, stddev=0.1)\n",
    "    return tf.Variable(initial)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "def bias_variable(shape):\n",
    "  \"\"\"bias_variable generates a bias variable of a given shape.\"\"\"\n",
    "  initial = tf.constant(0.1, shape=shape)\n",
    "  return tf.Variable(initial)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting MNIST_data/train-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/train-labels-idx1-ubyte.gz\n",
      "Extracting MNIST_data/t10k-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/t10k-labels-idx1-ubyte.gz\n"
     ]
    }
   ],
   "source": [
    "# Import data\n",
    "mnist = input_data.read_data_sets(\"MNIST_data/\", one_hot=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the model\n",
    "x = tf.placeholder(tf.float32, [None, 784])\n",
    "\n",
    "# Define loss and optimizer\n",
    "y_ = tf.placeholder(tf.float32, [None, 10])\n",
    "\n",
    "# Build the graph for the deep net\n",
    "y_conv, keep_prob = deepnn(x)\n",
    "\n",
    "with tf.name_scope('loss'):\n",
    "    cross_entropy = tf.nn.softmax_cross_entropy_with_logits(labels=y_,\n",
    "                                                            logits=y_conv)\n",
    "    cross_entropy = tf.reduce_mean(cross_entropy)\n",
    "\n",
    "with tf.name_scope('adam_optimizer'):\n",
    "    train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)\n",
    "\n",
    "with tf.name_scope('accuracy'):\n",
    "    correct_prediction = tf.equal(tf.argmax(y_conv, 1), tf.argmax(y_, 1))\n",
    "    correct_prediction = tf.cast(correct_prediction, tf.float32)\n",
    "accuracy = tf.reduce_mean(correct_prediction)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "sess = tf.InteractiveSession(config=config)\n",
    "sess.run(tf.global_variables_initializer())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "step 0, training accuracy 0.14\n",
      "step 100, training accuracy 0.84\n",
      "step 200, training accuracy 0.92\n",
      "step 300, training accuracy 0.96\n",
      "step 400, training accuracy 0.98\n",
      "step 500, training accuracy 0.86\n",
      "step 600, training accuracy 0.96\n",
      "step 700, training accuracy 0.96\n",
      "step 800, training accuracy 0.92\n",
      "step 900, training accuracy 0.94\n",
      "step 1000, training accuracy 0.94\n",
      "step 1100, training accuracy 0.96\n",
      "step 1200, training accuracy 0.98\n",
      "step 1300, training accuracy 1\n",
      "step 1400, training accuracy 1\n",
      "step 1500, training accuracy 0.94\n",
      "step 1600, training accuracy 1\n",
      "step 1700, training accuracy 0.98\n",
      "step 1800, training accuracy 0.96\n",
      "step 1900, training accuracy 0.98\n",
      "step 2000, training accuracy 0.98\n",
      "step 2100, training accuracy 1\n",
      "step 2200, training accuracy 0.96\n",
      "step 2300, training accuracy 1\n",
      "step 2400, training accuracy 1\n",
      "step 2500, training accuracy 1\n",
      "step 2600, training accuracy 0.94\n",
      "step 2700, training accuracy 1\n",
      "step 2800, training accuracy 0.96\n",
      "step 2900, training accuracy 0.98\n",
      "step 3000, training accuracy 0.98\n",
      "step 3100, training accuracy 1\n",
      "step 3200, training accuracy 1\n",
      "step 3300, training accuracy 1\n",
      "step 3400, training accuracy 0.96\n",
      "step 3500, training accuracy 1\n",
      "step 3600, training accuracy 1\n",
      "step 3700, training accuracy 0.96\n",
      "step 3800, training accuracy 0.98\n",
      "step 3900, training accuracy 0.98\n",
      "step 4000, training accuracy 0.98\n",
      "step 4100, training accuracy 1\n",
      "step 4200, training accuracy 1\n",
      "step 4300, training accuracy 1\n",
      "step 4400, training accuracy 0.98\n",
      "step 4500, training accuracy 1\n",
      "step 4600, training accuracy 1\n",
      "step 4700, training accuracy 0.98\n",
      "step 4800, training accuracy 1\n",
      "step 4900, training accuracy 0.98\n",
      "step 5000, training accuracy 1\n",
      "step 5100, training accuracy 1\n",
      "step 5200, training accuracy 0.98\n",
      "step 5300, training accuracy 1\n",
      "step 5400, training accuracy 0.98\n",
      "step 5500, training accuracy 1\n",
      "step 5600, training accuracy 0.98\n",
      "step 5700, training accuracy 1\n",
      "step 5800, training accuracy 0.98\n",
      "step 5900, training accuracy 1\n",
      "step 6000, training accuracy 1\n",
      "step 6100, training accuracy 1\n",
      "step 6200, training accuracy 0.98\n",
      "step 6300, training accuracy 0.98\n",
      "step 6400, training accuracy 0.98\n",
      "step 6500, training accuracy 1\n",
      "step 6600, training accuracy 1\n",
      "step 6700, training accuracy 1\n",
      "step 6800, training accuracy 1\n",
      "step 6900, training accuracy 1\n",
      "step 7000, training accuracy 0.98\n",
      "step 7100, training accuracy 1\n",
      "step 7200, training accuracy 1\n",
      "step 7300, training accuracy 1\n",
      "step 7400, training accuracy 0.98\n",
      "step 7500, training accuracy 0.98\n",
      "step 7600, training accuracy 1\n",
      "step 7700, training accuracy 1\n",
      "step 7800, training accuracy 0.98\n",
      "step 7900, training accuracy 0.96\n",
      "step 8000, training accuracy 0.98\n",
      "step 8100, training accuracy 0.98\n",
      "step 8200, training accuracy 1\n",
      "step 8300, training accuracy 0.98\n",
      "step 8400, training accuracy 1\n",
      "step 8500, training accuracy 1\n",
      "step 8600, training accuracy 0.98\n",
      "step 8700, training accuracy 1\n",
      "step 8800, training accuracy 1\n",
      "step 8900, training accuracy 1\n",
      "step 9000, training accuracy 1\n",
      "step 9100, training accuracy 0.98\n",
      "step 9200, training accuracy 1\n",
      "step 9300, training accuracy 1\n",
      "step 9400, training accuracy 1\n",
      "step 9500, training accuracy 1\n",
      "step 9600, training accuracy 0.98\n",
      "step 9700, training accuracy 1\n",
      "step 9800, training accuracy 1\n",
      "step 9900, training accuracy 1\n",
      "step 10000, training accuracy 1\n",
      "step 10100, training accuracy 1\n",
      "step 10200, training accuracy 1\n",
      "step 10300, training accuracy 0.98\n",
      "step 10400, training accuracy 1\n",
      "step 10500, training accuracy 1\n",
      "step 10600, training accuracy 1\n",
      "step 10700, training accuracy 1\n",
      "step 10800, training accuracy 0.98\n",
      "step 10900, training accuracy 1\n",
      "step 11000, training accuracy 1\n",
      "step 11100, training accuracy 0.98\n",
      "step 11200, training accuracy 1\n",
      "step 11300, training accuracy 1\n",
      "step 11400, training accuracy 1\n",
      "step 11500, training accuracy 1\n",
      "step 11600, training accuracy 0.98\n",
      "step 11700, training accuracy 1\n",
      "step 11800, training accuracy 1\n",
      "step 11900, training accuracy 1\n",
      "step 12000, training accuracy 1\n",
      "step 12100, training accuracy 1\n",
      "step 12200, training accuracy 0.98\n",
      "step 12300, training accuracy 1\n",
      "step 12400, training accuracy 1\n",
      "step 12500, training accuracy 1\n",
      "step 12600, training accuracy 1\n",
      "step 12700, training accuracy 1\n",
      "step 12800, training accuracy 1\n",
      "step 12900, training accuracy 1\n",
      "step 13000, training accuracy 0.98\n",
      "step 13100, training accuracy 1\n",
      "step 13200, training accuracy 1\n",
      "step 13300, training accuracy 1\n",
      "step 13400, training accuracy 1\n",
      "step 13500, training accuracy 1\n",
      "step 13600, training accuracy 1\n",
      "step 13700, training accuracy 1\n",
      "step 13800, training accuracy 1\n",
      "step 13900, training accuracy 1\n",
      "step 14000, training accuracy 1\n",
      "step 14100, training accuracy 1\n",
      "step 14200, training accuracy 0.98\n",
      "step 14300, training accuracy 1\n",
      "step 14400, training accuracy 1\n",
      "step 14500, training accuracy 0.98\n",
      "step 14600, training accuracy 0.98\n",
      "step 14700, training accuracy 1\n",
      "step 14800, training accuracy 1\n",
      "step 14900, training accuracy 1\n",
      "step 15000, training accuracy 0.98\n",
      "step 15100, training accuracy 0.98\n",
      "step 15200, training accuracy 1\n",
      "step 15300, training accuracy 1\n",
      "step 15400, training accuracy 1\n",
      "step 15500, training accuracy 1\n",
      "step 15600, training accuracy 1\n",
      "step 15700, training accuracy 1\n",
      "step 15800, training accuracy 1\n",
      "step 15900, training accuracy 1\n",
      "step 16000, training accuracy 1\n",
      "step 16100, training accuracy 1\n",
      "step 16200, training accuracy 1\n",
      "step 16300, training accuracy 1\n",
      "step 16400, training accuracy 1\n",
      "step 16500, training accuracy 1\n",
      "step 16600, training accuracy 1\n",
      "step 16700, training accuracy 1\n",
      "step 16800, training accuracy 1\n",
      "step 16900, training accuracy 1\n",
      "step 17000, training accuracy 1\n",
      "step 17100, training accuracy 1\n",
      "step 17200, training accuracy 1\n",
      "step 17300, training accuracy 1\n",
      "step 17400, training accuracy 1\n",
      "step 17500, training accuracy 1\n",
      "step 17600, training accuracy 1\n",
      "step 17700, training accuracy 1\n",
      "step 17800, training accuracy 1\n",
      "step 17900, training accuracy 1\n",
      "step 18000, training accuracy 1\n",
      "step 18100, training accuracy 1\n",
      "step 18200, training accuracy 1\n",
      "step 18300, training accuracy 1\n",
      "step 18400, training accuracy 1\n",
      "step 18500, training accuracy 1\n",
      "step 18600, training accuracy 1\n",
      "step 18700, training accuracy 1\n",
      "step 18800, training accuracy 0.98\n",
      "step 18900, training accuracy 1\n",
      "step 19000, training accuracy 1\n",
      "step 19100, training accuracy 1\n",
      "step 19200, training accuracy 1\n",
      "step 19300, training accuracy 1\n",
      "step 19400, training accuracy 1\n",
      "step 19500, training accuracy 1\n",
      "step 19600, training accuracy 1\n",
      "step 19700, training accuracy 1\n",
      "step 19800, training accuracy 1\n",
      "step 19900, training accuracy 1\n"
     ]
    }
   ],
   "source": [
    "for i in range(20000):\n",
    "    batch = mnist.train.next_batch(50)\n",
    "    if i % 100 == 0:\n",
    "        train_accuracy = accuracy.eval(feed_dict={\n",
    "            x: batch[0], y_: batch[1], keep_prob: 1.0})\n",
    "        print('step %d, training accuracy %g' % (i, train_accuracy))\n",
    "    train_step.run(feed_dict={x: batch[0], y_: batch[1], keep_prob: 0.5})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "test accuracy 0.9919\n"
     ]
    }
   ],
   "source": [
    "print('test accuracy %g' % accuracy.eval(feed_dict={x: mnist.test.images, y_: mnist.test.labels, keep_prob: 1.0}))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "sess.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](https://imgs.xkcd.com/comics/machine_learning.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The iris data set\n",
    "![](https://www.tensorflow.org/images/iris_three_species.jpg)\n",
    "The Iris data set contains 150 rows of data, comprising 50 samples from each of three related Iris species: Iris setosa, Iris virginica, and Iris versicolor.\n",
    "Four features from each sample: the length and the width of the sepals and petals, in centimetres."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "30,4,setosa,versicolor,virginica\n",
    "5.9,3.0,4.2,1.5,1\n",
    "6.9,3.1,5.4,2.1,2\n",
    "5.1,3.3,1.7,0.5,0\n",
    "6.0,3.4,4.5,1.6,1\n",
    "5.5,2.5,4.0,1.3,1\n",
    "6.2,2.9,4.3,1.3,1\n",
    "5.5,4.2,1.4,0.2,0\n",
    "6.3,2.8,5.1,1.5,2\n",
    "5.6,3.0,4.1,1.3,1\n",
    "6.7,2.5,5.8,1.8,2\n",
    "7.1,3.0,5.9,2.1,2\n",
    "4.3,3.0,1.1,0.1,0\n",
    "5.6,2.8,4.9,2.0,2\n",
    "5.5,2.3,4.0,1.3,1\n",
    "6.0,2.2,4.0,1.0,1\n",
    "5.1,3.5,1.4,0.2,0\n",
    "5.7,2.6,3.5,1.0,1\n",
    "4.8,3.4,1.9,0.2,0\n",
    "5.1,3.4,1.5,0.2,0\n",
    "5.7,2.5,5.0,2.0,2\n",
    "5.4,3.4,1.7,0.2,0\n",
    "5.6,3.0,4.5,1.5,1\n",
    "6.3,2.9,5.6,1.8,2\n",
    "6.3,2.5,4.9,1.5,1\n",
    "5.8,2.7,3.9,1.2,1\n",
    "6.1,3.0,4.6,1.4,1\n",
    "5.2,4.1,1.5,0.1,0\n",
    "6.7,3.1,4.7,1.5,1\n",
    "6.7,3.3,5.7,2.5,2\n",
    "6.4,2.9,4.3,1.3,1\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "from six.moves.urllib.request import urlopen"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "IRIS_TRAINING = 'iris_training.csv'\n",
    "IRIS_TRAINING_URL = 'http://download.tensorflow.org/data/iris_training.csv'\n",
    "\n",
    "IRIS_TEST = 'iris_test.csv'\n",
    "IRIS_TEST_URL = 'http://download.tensorflow.org/data/iris_test.csv'\n",
    "\n",
    "FEATURE_KEYS = ['sepal_length', 'sepal_width', 'petal_length', 'petal_width']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_iris_data(file_name, download_url):\n",
    "    \"\"\"Downloads the file and returns the number of data.\"\"\"\n",
    "    if not os.path.exists(file_name):\n",
    "        raw = urllib.urlopen(download_url).read()\n",
    "        with open(file_name, 'w') as f:\n",
    "            f.write(raw)\n",
    "\n",
    "    # The first line is a comma-separated string. The first one is the number of\n",
    "    # total data in the file.\n",
    "    with open(file_name, 'r') as f:\n",
    "        first_line = f.readline()\n",
    "    num_elements = first_line.split(',')[0]\n",
    "    return int(num_elements)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "def input_fn(file_name, num_data, batch_size, is_training):\n",
    "    \"\"\"Creates an input_fn required by Estimator train/evaluate.\"\"\"\n",
    "\n",
    "    def _parse_csv(rows_string_tensor):\n",
    "        \"\"\"Takes the string input tensor and returns tuple of (features, labels).\"\"\"\n",
    "        # Last dim is the label.\n",
    "        num_features = len(FEATURE_KEYS)\n",
    "        num_columns = num_features + 1\n",
    "        columns = tf.decode_csv(rows_string_tensor,\n",
    "                            record_defaults=[[]] * num_columns)\n",
    "        features = dict(zip(FEATURE_KEYS, columns[:num_features]))\n",
    "        labels = tf.cast(columns[num_features], tf.int32)\n",
    "        return features, labels\n",
    "\n",
    "    def _input_fn():\n",
    "        \"\"\"The input_fn.\"\"\"\n",
    "        dataset = tf.data.TextLineDataset([file_name])\n",
    "        # Skip the first line (which does not have data).\n",
    "        dataset = dataset.skip(1)\n",
    "        dataset = dataset.map(_parse_csv)\n",
    "\n",
    "        if is_training:\n",
    "            # For this small dataset, which can fit into memory, to achieve true\n",
    "            # randomness, the shuffle buffer size is set as the total number of\n",
    "            # elements in the dataset.\n",
    "            dataset = dataset.shuffle(num_data)\n",
    "            dataset = dataset.repeat()\n",
    "\n",
    "        dataset = dataset.batch(batch_size)\n",
    "        iterator = dataset.make_one_shot_iterator()\n",
    "        features, labels = iterator.get_next()\n",
    "        return features, labels\n",
    "    \n",
    "    return _input_fn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_training_data = get_iris_data(\n",
    "      IRIS_TRAINING, IRIS_TRAINING_URL)\n",
    "num_test_data = get_iris_data(IRIS_TEST, IRIS_TEST_URL)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Using default config.\n",
      "WARNING:tensorflow:Using temporary folder as model directory: /tmp/user/43350/tmpfk6ge4bt\n",
      "INFO:tensorflow:Using config: {'_log_step_count_steps': 100, '_session_config': None, '_model_dir': '/tmp/user/43350/tmpfk6ge4bt', '_num_ps_replicas': 0, '_save_checkpoints_secs': 600, '_master': '', '_is_chief': True, '_keep_checkpoint_max': 5, '_save_summary_steps': 100, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f668c186320>, '_task_id': 0, '_tf_random_seed': None, '_service': None, '_task_type': 'worker', '_save_checkpoints_steps': None, '_keep_checkpoint_every_n_hours': 10000, '_num_worker_replicas': 1}\n"
     ]
    }
   ],
   "source": [
    "# Build 3 layer DNN with 10, 20, 10 units respectively.\n",
    "feature_columns = [\n",
    "      tf.feature_column.numeric_column(key, shape=1) for key in FEATURE_KEYS]\n",
    "classifier = tf.estimator.DNNClassifier(\n",
    "      feature_columns=feature_columns, hidden_units=[10, 20, 10], n_classes=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Saving checkpoints for 1 into /tmp/user/43350/tmpfk6ge4bt/model.ckpt.\n",
      "INFO:tensorflow:loss = 43.3998, step = 1\n",
      "INFO:tensorflow:global_step/sec: 124.513\n",
      "INFO:tensorflow:loss = 8.67745, step = 101 (0.805 sec)\n",
      "INFO:tensorflow:global_step/sec: 126.554\n",
      "INFO:tensorflow:loss = 3.88408, step = 201 (0.790 sec)\n",
      "INFO:tensorflow:global_step/sec: 136.568\n",
      "INFO:tensorflow:loss = 2.51858, step = 301 (0.732 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 400 into /tmp/user/43350/tmpfk6ge4bt/model.ckpt.\n",
      "INFO:tensorflow:Loss for final step: 2.18418.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.estimator.canned.dnn.DNNClassifier at 0x7f666cef16d8>"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Train.\n",
    "train_input_fn = input_fn(IRIS_TRAINING, num_training_data, batch_size=32,\n",
    "                            is_training=True)\n",
    "classifier.train(input_fn=train_input_fn, steps=400)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Starting evaluation at 2017-12-13-12:26:36\n",
      "INFO:tensorflow:Restoring parameters from /tmp/user/43350/tmpfk6ge4bt/model.ckpt-400\n",
      "INFO:tensorflow:Finished evaluation at 2017-12-13-12:26:36\n",
      "INFO:tensorflow:Saving dict for global step 400: accuracy = 0.966667, average_loss = 0.0654595, global_step = 400, loss = 1.96379\n",
      "Accuracy (tensorflow): 0.966667\n"
     ]
    }
   ],
   "source": [
    "# Eval.\n",
    "test_input_fn = input_fn(IRIS_TEST, num_test_data, batch_size=32,\n",
    "                           is_training=False)\n",
    "scores = classifier.evaluate(input_fn=test_input_fn)\n",
    "print('Accuracy (tensorflow): {0:f}'.format(scores['accuracy']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# http://playground.tensorflow.org"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}