e253281_HW_4.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# IS709 Assignment 4\n",
    "\n",
    "### Due: January 22, Sunday-23.59\n",
    "\n",
    "### Submission and Grading Policy\n",
    "\n",
    "- Submit your assignments to the corresponding assignment link in https://odtuclass.metu.edu.tr.\n",
    "- Solutions should be submitted in a single **ipynb** file named: metu-username_HW_4.ipynb (e.g., “e123456_HW_4.ipynb”).\n",
    "- Late submissions will be accepted by January 25, 2021, 23:59 with a 5% per day penalty.\n",
    "- **This is an individual assignment. You have to adhere to the academic integrity principles.**\n",
    "- You are supposed to be able to interpret your findings. Hence, you shouldn't just find a number or result and just leave it. You need to comment on your findings by giving as much as **necessary** details.    \n",
    "\n",
    "# Questions\n",
    "## Q1.\n",
    "For the following use data in file q1_data.csv that consists of 40 feature columns (x1 .. x40) and target variable y.\n",
    "\n",
    "**a (20 pts).** \n",
    "\n",
    "    Randomly take samples of size 100, 150, 200, ...., 950, 1000 (i.e., incrementing sample sizes by 50) from the given dataset, apply train-test split (%70 train-%30 test), fit a multiple linear regression model, evaluate the mean square performance of the model on the train and test sets and plot results. Discuss the results.    \n",
    "\n",
    "    Are the performances on the train and test sets similar for different sample sizes? If not, why are the performances on the train set better/worse than the performance on the test set for small and large sample sizes? If you had more examples in the dataset would the performance improve? \n",
    "   \n",
    "    Does regularization improve the performance of the test set? Briefly explain your reasoning.       \n",
    "  \n",
    "\n",
    "**b (20 pts).**\n",
    "    \n",
    "    Repeat part (a) after applying polynomial expansion with degree=2 and set to the given dataset.(Do not include bias i.e., set parameter include_bias=False)\n",
    "\n",
    "**c (20 pts).**\n",
    "\n",
    "    Use Lasso regularization and try to fit the best possible model to the given dataset. When necessary you can use (%70 train - %30 test) or (%60 train - %20 validation - %20 test) splits. Report the model performance on the train/validation/test sets. \n",
    "    \n",
    "    What is the regression equation for the best model (consider only the significant coefficients - compared to the largest coefficients)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn import linear_model\n",
    "from sklearn import metrics \n",
    "from sklearn import datasets\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "import math\n",
    "\n",
    "q1_data=pd.read_csv(\"q1_data.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [],
   "source": [
    "def q1_compare_RMSE(q1_data,model=LinearRegression(), test_ratio=0.3):\n",
    "    student_id = 2532810\n",
    "    names = []\n",
    "    overall_train= 0\n",
    "    overall_test= 0\n",
    "    rmses_train = []\n",
    "    rmses_test = []\n",
    "    R2=[]\n",
    "    arraySize=100\n",
    "    while arraySize<1050:\n",
    "        globals()[f\"data_{arraySize}\"] = q1_data.sample(n=arraySize, random_state=student_id)\n",
    "        globals()[f\"data_{arraySize}\"].name=f\"data_{arraySize}\"\n",
    "        x=globals()[f\"data_{arraySize}\"].drop(\"y\", axis=1)\n",
    "        y=globals()[f\"data_{arraySize}\"][\"y\"]\n",
    "        x_train, x_test, y_train, y_test = train_test_split(x,y, test_size=test_ratio, random_state=student_id)\n",
    "        model.fit(x_train, y_train) ## regression model is fitted\n",
    "        #print('intercept=', model.intercept_, 'coefficients=', model.coef_) ##since there are three predictors, .coef_ returns three values in an array\n",
    "        predictions_train=model.predict(x_train)\n",
    "        err_train = mean_squared_error(y_train, predictions_train)\n",
    "        predictions_test=model.predict(x_test)\n",
    "        err_test = mean_squared_error(y_test, predictions_test)\n",
    "\n",
    "\n",
    "        rmse_train = math.sqrt(err_train)\n",
    "        overall_train = overall_train + rmse_train\n",
    "        rmse_test = math.sqrt(err_test)\n",
    "        overall_test = overall_test + rmse_test\n",
    "        R2_array=model.score(x_train,y_train)\n",
    "\n",
    "        #print(globals()[f\"data_{arraySize}\"].name, ' train set RMSE:',rmse_train)\n",
    "        #print(globals()[f\"data_{arraySize}\"].name, ' test set RMSE:',rmse_test)\n",
    "        #print(globals()[f\"data_{arraySize}\"].name, ' R2:',R2_array,'\\n')\n",
    "        names.append(globals()[f\"data_{arraySize}\"].name)\n",
    "        rmses_train.append(rmse_train)\n",
    "        rmses_test.append(rmse_test)\n",
    "        R2.append(R2_array)\n",
    "      \n",
    "        arraySize+=50\n",
    "    return (names,rmses_train,rmses_test,R2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [],
   "source": [
    "def q1_plot_RMSE(names,rmses_train,rmses_test,R2):\n",
    "    least_RMSE_train = min(rmses_train)\n",
    "    least_RMSE_train_name = names[rmses_train.index(least_RMSE_train)]\n",
    "\n",
    "    #print(\"The least train set RMSE value  :\", least_RMSE_train,\"\\n The least RMSE dataset:\", least_RMSE_train_name, \"\\n The index of dataset:\",rmses_train.index(least_RMSE_train))\n",
    "\n",
    "    most_RMSE_train = max(rmses_train)\n",
    "    most_RMSE_train_name = names[rmses_train.index(most_RMSE_train)]\n",
    "\n",
    "    #print(\"The most train set RMSE value  :\", most_RMSE_train,\"\\n The most RMSE dataset:\", most_RMSE_train_name)\n",
    "\n",
    "    least_RMSE_test = min(rmses_test)\n",
    "    least_RMSE_test_name = names[rmses_test.index(least_RMSE_test)]\n",
    "\n",
    "    #print(\"The least test set RMSE value  :\", least_RMSE_test,\"\\n The least RMSE dataset:\", least_RMSE_test_name, \"\\n The index of dataset:\",rmses_test.index(least_RMSE_test))\n",
    "\n",
    "    most_RMSE_test = max(rmses_test)\n",
    "    most_RMSE_test_name = names[rmses_test.index(most_RMSE_test)]\n",
    "\n",
    "    #print(\"The most test set RMSE value  :\", most_RMSE_test,\"\\n The most RMSE dataset:\", most_RMSE_test_name)\n",
    "\n",
    "    columns=['rmses_train','rmses_test','R2']\n",
    "    results=[]\n",
    "    \n",
    "    variables = range(len(names))\n",
    "    for i in variables:\n",
    "        results.append((rmses_train[i], rmses_test[i], R2[i]))\n",
    "    \n",
    "    res = pd.DataFrame(results, index=names, columns=columns)\n",
    "    display(res)\n",
    "    res.plot()\n",
    "    plt.ylim((min(least_RMSE_train,least_RMSE_test,min(R2))-1),(max(most_RMSE_train,most_RMSE_test,max(R2))+1))\n",
    "    plt.figsize=(20, 20)\n",
    "    #plt.xticks(names)\n",
    "    plt.grid()\n",
    "\n",
    "    \n",
    "    #f, axes = plt.subplots(2,1, figsize=(20, 20))\n",
    "    #axes[0].plot(names, rmses_train, 'o-', label='Train Set RMSEs') \n",
    "    #axes[1].plot(names, rmses_test, 'o-', label='Test Set RMSEs') \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### a (20 pts) \n",
    "Are the performances on the train and test sets similar for different sample sizes? If not, why are the performances on the train set better/worse than the performance on the test set for small and large sample sizes? If you had more examples in the dataset would the performance improve? \n",
    "   \n",
    "    Does regularization improve the performance of the test set? Briefly explain your reasoning.\n",
    "    \n",
    "### Answer\n",
    "\n",
    "We can see from the graph rmses_train (Train Set Error) is increasing by the increase in training set size, on the other hand rmses_test (Test Set Error) is decreasing by the increase in test set size. But at one point they start to perform nearly similar.\n",
    "\n",
    "R2 which tells us how much variance is explained by the model compared to how much variance there is to explain in the first place ( proportion of variance in the outcome variable that is shared by the predictor variabl) is very high and nearly constant against to set size.\n",
    "\n",
    "\n",
    "First of all we need to recall some terms from lecture notes:\n",
    "##### Bias-Variance\n",
    "**Bias**: the difference between the average prediction of our model and the correct value which we are trying to predict.\n",
    "High bias means the model does not fit to the training data well. That is, the model is an oversimplification. High bias causes high error on training and test data.\n",
    "**To fix high bias (underfit), you can**\n",
    "- get additional features\n",
    "- add polynomial features\n",
    "\n",
    "**Variance**: Variability of the model prediction for a given data point. High variance means the model overfits the training data and it does not generalize. High variance causes a small error on training data but a large error on test data.\n",
    "**To fix high variance (overfit), you can**\n",
    "- get more training example\n",
    "- try a smaller set of features\n",
    "\n",
    "**Adding more data** (considering only rows) will not solve bias problem but can solve variance problem. On the other hand, instead of adding more data we can consider adding more features to solve bias problem. Meantime by adding more features we are adding more data.\n",
    "\n",
    "**Regularization** is a form of regression, that constrains/ regularizes or shrinks the coefficient estimates towards zero. In other words, this technique discourages learning a more complex or flexible model, so as to avoid the risk of overfitting. Regularization, significantly reduces the variance of the model, without substantial increase in its bias. If there is noise in the training data, then the estimated coefficients won’t generalize well to the future data. So we can conclude that regularization improve the performance of the test set.\n",
    "But we must not forget that, for any fixed dataset, taking sufficiently strong regularization will make the linear model essentially constant, presumably not the best model. \n",
    "There's a bias-variance tradeoff: increasing regularization weakens the model's ability to learn the training data, which to a point will tend to prevent overfitting and improve test performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>rmses_train</th>\n",
       "      <th>rmses_test</th>\n",
       "      <th>R2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>data_100</th>\n",
       "      <td>1.426620</td>\n",
       "      <td>4.290559</td>\n",
       "      <td>0.995035</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_150</th>\n",
       "      <td>1.815064</td>\n",
       "      <td>3.011346</td>\n",
       "      <td>0.992661</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_200</th>\n",
       "      <td>2.181886</td>\n",
       "      <td>2.740987</td>\n",
       "      <td>0.989537</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_250</th>\n",
       "      <td>2.195522</td>\n",
       "      <td>3.008817</td>\n",
       "      <td>0.989722</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_300</th>\n",
       "      <td>2.225780</td>\n",
       "      <td>3.057531</td>\n",
       "      <td>0.989938</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_350</th>\n",
       "      <td>2.328125</td>\n",
       "      <td>2.659287</td>\n",
       "      <td>0.989132</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_400</th>\n",
       "      <td>2.342475</td>\n",
       "      <td>2.714704</td>\n",
       "      <td>0.988949</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_450</th>\n",
       "      <td>2.409858</td>\n",
       "      <td>2.785569</td>\n",
       "      <td>0.987832</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_500</th>\n",
       "      <td>2.514045</td>\n",
       "      <td>2.599215</td>\n",
       "      <td>0.986198</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_550</th>\n",
       "      <td>2.560283</td>\n",
       "      <td>2.568923</td>\n",
       "      <td>0.986576</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_600</th>\n",
       "      <td>2.553657</td>\n",
       "      <td>2.688438</td>\n",
       "      <td>0.986097</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_650</th>\n",
       "      <td>2.557527</td>\n",
       "      <td>2.721877</td>\n",
       "      <td>0.985696</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_700</th>\n",
       "      <td>2.592037</td>\n",
       "      <td>2.564911</td>\n",
       "      <td>0.985284</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_750</th>\n",
       "      <td>2.574133</td>\n",
       "      <td>2.635559</td>\n",
       "      <td>0.985454</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_800</th>\n",
       "      <td>2.611327</td>\n",
       "      <td>2.553529</td>\n",
       "      <td>0.985685</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_850</th>\n",
       "      <td>2.563395</td>\n",
       "      <td>2.700018</td>\n",
       "      <td>0.986087</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_900</th>\n",
       "      <td>2.557159</td>\n",
       "      <td>2.710920</td>\n",
       "      <td>0.986683</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_950</th>\n",
       "      <td>2.544864</td>\n",
       "      <td>2.761614</td>\n",
       "      <td>0.986594</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_1000</th>\n",
       "      <td>2.608883</td>\n",
       "      <td>2.582769</td>\n",
       "      <td>0.985552</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           rmses_train  rmses_test        R2\n",
       "data_100      1.426620    4.290559  0.995035\n",
       "data_150      1.815064    3.011346  0.992661\n",
       "data_200      2.181886    2.740987  0.989537\n",
       "data_250      2.195522    3.008817  0.989722\n",
       "data_300      2.225780    3.057531  0.989938\n",
       "data_350      2.328125    2.659287  0.989132\n",
       "data_400      2.342475    2.714704  0.988949\n",
       "data_450      2.409858    2.785569  0.987832\n",
       "data_500      2.514045    2.599215  0.986198\n",
       "data_550      2.560283    2.568923  0.986576\n",
       "data_600      2.553657    2.688438  0.986097\n",
       "data_650      2.557527    2.721877  0.985696\n",
       "data_700      2.592037    2.564911  0.985284\n",
       "data_750      2.574133    2.635559  0.985454\n",
       "data_800      2.611327    2.553529  0.985685\n",
       "data_850      2.563395    2.700018  0.986087\n",
       "data_900      2.557159    2.710920  0.986683\n",
       "data_950      2.544864    2.761614  0.986594\n",
       "data_1000     2.608883    2.582769  0.985552"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD5CAYAAAAOXX+6AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAuN0lEQVR4nO3dd3xUZbrA8d87kzJJJqSSQKQEQpQSIHQiuia6gkoRXXULWPaqbFX3Krbr2nZdd72W9bKyrt67C+5asKwFO6KJWCjSpAYTemhpJKQnM/PeP84kJCEhkzCTOZM834/zmTOnPnk5PvPOe97zHqW1RgghhHlZ/B2AEEKI05NELYQQJieJWgghTE4StRBCmJwkaiGEMLkgX+w0Pj5eJycnd2nbqqoqIiIivBuQD0ic3hcosUqc3hUocYJvY92wYUOx1rpvmwu11l5/TZgwQXdVdnZ2l7ftThKn9wVKrBKndwVKnFr7NlZgvW4np0rThxBCmJwkaiGEMDlJ1EIIYXI+uZgohAgMDQ0NFBQUUFtb67cYoqKi2Llzp9+O3xneiNVmszFgwACCg4M93kYStRC9WEFBAZGRkSQnJ6OU8ksMFRUVREZG+uXYnXWmsWqtKSkpoaCggCFDhni8nTR9CNGL1dbWEhcX57ck3dsopYiLi+v0LxhJ1EL0cpKku1dXytujpg+l1D6gAnACDq31xE4fSQghRJd0po06S2td7LNIhBBCtEmaPoQQwu3pp5+murq609s98MADrFy50gcRGZT24AkvSqm9wHFAA89prZ9vY50FwAKAxMTECcuWLetSQJWVldjt9i5t250kTu8LlFh7UpxRUVEMGzasmyJqm9PpxGq1AieHtLBY/FOHTEtL4/PPPycuLu6UZU6nE6Ap1jORn59PeXl5i3lZWVkb2mtW9rTpY5rW+rBSKgH4RCmVq7Ve1XwFd/J+HmDixIk6MzOz08ED5OTk0NVtu5PE6X2BEmtPinPnzp1N3c0efnc7Ow6f8GoMI5P68ODsUaddZ9u2bVx99dVkZWXxyiuvEB0dzUUXXcSaNWsYO3YsP/3pT3nwwQcpLCzkpZdeYvLkyXz++efcdtttgHFxbtWqVURGRvL444/z2muvUVdXxxVXXMHDDz9MVVUV11xzDQUFBTidTu6//35++MMfnhLHokWLOHLkCLNnzyY+Pp7s7Gzsdju33347H3/8MU8++SQffvghK1asoKamhnPPPZfnnnsOpRQ33HADs2bN4qqrriI5OZnrr7+ed999l4aGBl5//XWGDx/e4lg2m41x48Z5XI4efW1prQ+73wuBt4DJHh9BCCE6sGvXLq677jo2bdrEwYMHue2229iyZQu5ubm8/PLLfPnllzzxxBM8+uijADzxxBMsXryYzZs388UXXxAWFsaKFSvIy8tj3bp1bN68mQ0bNrBq1So++ugjkpKS+Pbbb9m2bRuXXHJJmzHceuutJCUlkZ2dTXZ2NmCMlpeWlsbatWs577zzWLBgAd988w3btm2jpqaG9957r819xcfHs3HjRn7xi1/wxBNPnHH5dFijVkpFABatdYV7ejrwuzM+shDCVDqq+frS4MGDmTp1Kvv27WPIkCGMHj0agFGjRnHRRRehlGL06NHs27cPgGnTpnH77bczb948rrzySgYMGMCKFStYsWJFU021srKSvLw8zj//fBYuXMjdd9/NrFmzOP/88z2Oy2q18oMf/KDp8xdffME111xDdXU1paWljBo1itmzZ5+y3ZVXXgnAhAkTePPNN7taLE08afpIBN5y9/0LAl7WWn90xkcWQgi35mM8h4aGNk1bLJamzxaLBYfDAcA999zDzJkz+eCDD5g6dSorV65Ea829997Lz372s1P2v2HDBj744APuvfdepk+fzgMPPOBRXDabralNura2lttvv50NGzYwcOBAHnrooXZvXGmM2Wq1NsV8JjpM1FrrPcDYMz6SEEJ4ye7duxk9ejSjR49m9erV5ObmMmPGDO6//37mzZuH3W7n0KFDBAcH43A4iI2NZf78+djtdpYuXdrufiMjI6moqCA+Pv6UZY1JOT4+nsrKSt544w2uuuoqX/2JLchYH0KIgPP000+TnZ2N1Wpl5MiRXHrppYSGhrJz504yMjIAsNvtvPjii+Tn53PnnXdisVgIDg7m2WefbXe/CxYs4NJLL6V///5N7dSNoqOjuf766xk9ejTJyclMmjTJp39jC+09UeBMXvKEF/MIlDi1DpxYe1KcO3bs8H0gHThx4oS/Q/CYt2Jtq9yRJ7wIIUTgkqYPIUSvc8UVV7B3794W8x577DFmzJjhp4hOTxK1EKLXeeutt/wdQqdI04cQQpicJGohhDA5SdRCCGFykqiFEMKtq8OcArz99tvs2LHDyxEZJFELIUxDa43L5fLb8SVRCyFEG/bv38+IESP45S9/SWxsLCkpKdx0002kpaUxb948Vq5cybRp00hNTWXdunUAfP7556Snp5Oens64ceOoqKgA4PHHH2fSpEmMGTOGBx98EDBGwJs5cyZjx44lLS2NV199tc04Fi1axOHDh8nKyiIrKwuAFStWkJGRwfjx47n66quprKwEjLFGRo4cyZgxY1i4cCFff/01y5cv58477yQ9PZ3du3d7tYyke54QwvDhPXB0q3f32W80XPqnDlfbtWsXS5Ys4a677mLYsGHcdtttPP/880yaNKlpmNPly5fz6KOP8vbbbzcNczpt2jQqKyux2WwthjnVWjNnzhxWrVpFUVERSUlJvP/++wCnDNjf6NZbb+Wpp54iOzub+Ph4iouLeeSRR1i5ciURERE89thjPPPMM9xxxx289dZb5ObmopSirKyM6Oho5syZ0zQmtbdJjVoI4XeNw5wCTcOcWiyWDoc5XbRoEWVlZQQFBbUY5nT8+PHk5uaSl5fH6NGjWblyJXfffTdffPEFUVFRHsW0Zs0aduzYwbRp00hPT+eFF17g4MGD9OnTB5vNxk033cSbb75JeHi4r4qlidSohRAGD2q+vmLGYU611lx88cW88sorTfMqKioICgpi3bp1fPrppyxbtoxnnnmGzz77rMt/uyekRi2ECDiNw5zefffdTJw4sWmY03/84x9N7ciHDh2isLCQw4cPEx4ezvz581m4cCEbN25sd7+Nw5wCTJ06la+++or8/HwAqqurycvLo7KykvLyci677DKefvppNm/efMq23iY1aiFEwOmuYU6XLl3Kj3/8Y+rq6gC477776N+/P5dffjm1tbVorfnzn/8MwI9+9CNuvvlmFi1axBtvvEFKSor3/uD2htU7k5cMc2oegRKn1oETa0+KU4Y57RwZ5lQIIUSbpOlDCNHryDCnQghhcjLMqRBCCK+SRC2EECZnnqYPlxP2fUlE5X5/RyKEEKZinhq1swGW/YSzDr3n70iEEMJUzJOog22QOp344rVG7VoI0StER0eTnp5OWloas2fPpqysDIDNmzeTkZHBqFGjGDNmTLuj3vUG5knUACNmE9JQDgdW+zsSIUQ3CQsLY/PmzWzbto3Y2FgWL14MQHh4OP/85z/Zvn07H330Eb/5zW+aknhvY65EnTodlwqGne/6OxIhhB9kZGRw6NAhAM4++2xSU1MBSEpKIiEhgaKiIn+G5zfmuZgIEGqnNHYc8TvfhRl/BIu5vkeE6MkeW/cYuaW5Xt3n8Njh3D35bo/WdTqdfPrpp9x4442nLFu3bh319fXeHT8jgJguExb1zYATh+Bw+yNcCSF6jpqaGtLT04mLi6O0tJSLL764xfIjR45w7bXXsmTJEiy9tPJmrho1UBI3GSxBsHM5DJjo73CE6DU8rfl6W2MbdXl5ObNmzWLx4sXceuutAJw4cYKZM2fyyCOPND1YoDcy3deTI9gOQ74HO5aD1v4ORwjRTaKioli0aBFPPPEEDQ0N1NfXc8UVV3Dddddx9dVX+zs8vzJdogZgxBw4vheObfd3JEKIbjRu3DjGjh3LsmXLeO2111i1ahVLly5tepBt4yD9vY3HTR9KKSuwHjiktZ7lu5CA4TPhvf80mj/6pfn0UEII/zpy5EiLz+++e7LX1/z587s7HFPqTI36NmCnrwJpwZ4Ag8+VbnpCCIGHiVopNQCYCfyfb8NpZsRsKNwBxfnddkghhDAjpT24YKeUegP4IxAJLGyr6UMptQBYAJCYmDhh2bJlXQqosrISu91OaG0RGWtuYs+Qazkw+Kou7cuXGuM0u0CJEwIn1p4UZ1RUFCkpKSiluimqUzmdTqxWq9+O3xneiFVrze7duykvL28xPysra4PWuu2ubu09o6vxBcwC/uqezgTe62gbrz0z8fksrZ+7oMv78qWe9Nw8swiUWHtSnHv27NFFRUXa5XL5PqB29KZnJrpcLl1UVKT37NlzyjJO88xETy4mTgPmKKUuA2xAH6XUi1pr37fyj5gNKx+CsgMQPcjnhxOitxkwYAAFBQV+vTW7trYWm83mt+N3hjditdlsDBgwoFPbdJiotdb3AvcCKKUyMZo+uudS7Ig5RqLe+R5k/LJbDilEbxIcHMyQIUP8GkNOTg7jxo3zawye8les5uxH3SguBRJGGd30hBCil+pUotZa52hf96FubeQcOLAGKo5162GFEMIszF2jBqOdGg258uQXIUTvZP5EnTASYlPk5hchRK9l/kStlNH8se8LqC71dzRCCNHtzJ+owWj+cDlg14f+jkQIIbpdYCTqpPHQZ4A0fwgheqXASNRKGbXq3Z9BXYW/oxFCiG4VGIkajHZqZx3krfB3JEII0a0CJ1EPnAIRfY0nvwghRC8SOInaYoXhsyDvE2io8Xc0QgjRbQInUYPR/NFQZbRVCyFELxFYiTr5fLBFS/OHEKJXCaxEbQ2Gcy6D7z4ER72/oxFCiG4RWIkajG56teWwb5W/IxFCiG4ReIk65UIIjpCbX4QQvUbgJepgG5w9HXLfB5fT39EIIYTPBV6iBuPJL1VFxjjVQgjRwwVmok6dDtZQefKLEKJXCMxEHWqHYRcZ7dQul7+jEUIInwrMRA1G88eJQ3B4k78jEUIInwrcRH3OJWAJgp3v+DsSIYTwqcBN1GExMOR7RvOH1v6ORgghfCZwEzUYN7+U7oFj2/0diRBC+ExgJ+rhswAlN78IIXq0wE7U9gQYfK500xNC9GiBnajBaP4o3AHF+f6ORAghfKJnJGqQWrUQoscK/EQdNcB4SrkkaiFEDxX4iRqMJ78c3gRlB/0diRBCeF3PSNQj5hjv0vtDCNED9YxEHZcCCaMkUQsheqSekajBaP44sBoqjvk7EiGE8Kqek6hHzAY07Hrf35EIIYRXBXW0glLKBqwCQt3rv6G1ftDXgXVawkiITTGeUD7xP/wdjee0huoSKD9oXAwtL3C/DoLLAdMfMZp2hBC9VoeJGqgDLtRaVyqlgoEvlVIfaq3N9XgVpYxa9epnoLoUwmP9HZHBUWcMx9o6CTd/d9S23CY43Oh2WHkMllwG178Lfc/2T/xCCL/rMFFrrTVQ6f4Y7H6Zc7i6kXPgq6ch+1GY8QcICvVPHLuz4fP/htLdRrJtzZ4IUQMhMQ3OvsSYjhoA0QON6bAY44uncCe8MAeWXgbXvQOJo7r/bxFC+J3SHgwRqpSyAhuAYcBirfXdbayzAFgAkJiYOGHZsmVdCqiyshK73d6lbdGas79bTNKRT6gKH0Tu8Fuo6OObmmhbcQY1VJKyewn9j66kxtaPsug0am19qbUlUBcaT62tL3Wh8WhLsMfHCas+RPrm+7G46vl27ENURg474zjNKlBilTi9K1DiBN/GmpWVtUFrPbHNhVprj19ANJANpJ1uvQkTJuiuys7O7vK2TXZ9rPWTI7R+KFrrj3+rdX31me+zlVPi3LFc68dTtX4oRutPHtK6vsZ7ByvZo/Wf07R+dKDWB9aeWZwmFiixSpzeFShxau3bWIH1up2c2qleH1rrMiAHuOQMvjh87+zp8MvVMP46+HoRPDsN9n/tm2NVFsJr18Or843R/G7+DL7/IATbvHeM2CHw0w8hIg7+ORf2fem9fQshTK/DRK2U6quUinZPhwHfB3J9HNeZs0XB7P8x2nZdDlhyKXxwJ9RVdrytJ7SGza/AM5Ng14dw0QNwczYkpXtn/61FDTCSdfRAePEqyP/UN8cRQpiOJzXq/kC2UmoL8A3widb6Pd+G5UVDM+EXX8OUn8O6/4VnM4yLfWei7ABjtjwMb/8c+g6Hn38J598BVs/bnrsksh/c8D7EDYNXfgS7PvLt8YQQptBhotZab9Faj9Naj9Fap2mtf9cdgXlVqB0ufQz+4yOwhsC/5sI7v4aass7tx+Uykv1fM4gq3wmXPm7Ucruz61xEPFy/3Ogx8uo82CEP9xWip+s5dyZ6YtBUo/Y77Tew+SX461TPa6XFeUY3uQ8WwsAprJv8F5iyACx+KMLwWLjubThrArz+U9jyWvfHIIToNr0rUQMEh8HFD8NNn0JYLLzyQ/j3zVBV0vb6zgb44knjgmThTpj7N5j/b+psCd0bd2u2KJj/pvEosjcXwMZ/+jceIYTPeHJnYs901nhYkANfPgWrHofdn8HMJ2DkXONmE4Aj38I7v4KjW2Hk5XDZE0bPDrMItcO8140eJ8tvMe6CnHyzv6MyNNRCSZ7x5Va4AyxBMOEG46KoEKJTem+iBggKgcx7jKeZv/MreP0G4zb06X+ADUvgq0VGm/A1/zLuejSj4DD40ctGE8gHC43b0c+9pfuO73TA8b1GMm5MyoU7oWQ3aKexjiUYtAu+eArSroSMX/uud4wQHdEaao5D2QH3GDsHTj50ZPy1prwDuHcn6kb90oymkNV/gew/nhzXetx8Y1CksBj/xteRoFC45gV482ZY8VujNnvBnd49hstlnNTNk3HhTijeBc5690oKYodCwgjjl0nCCGOwrLgUqDgKa/8GG16Ara9D8vlw7q0w7Pv+aef3J61P/moT3qc1VBW5x9c5cDIRN0/M9a266QZHGN141z5rnJtTf2EM72Cx+udvaEUSdSNrEJz3n0bt+sunYfQPIOVCf0flOWswXPl/YA2F7EfAUQMX3t+5fWht3MBzfC8c3wel7veSfCjKbXly9xlgJOKULCMZJ4yA+LMhJLztfUcPNMZfueAuI1mveRZevtro3pjxKxh9TVf/cv9z1EFVsTEKYnWxcb2jurjtedUlRm0uMQ3GXQujrzLPAGL+oDWhtcXGxXpHHTjrwFFvfPk760/Ocza4p+tPvjefrqswBjgrO+Ae6Kym5XFsURA1CGKGwJDvQfQgY1yd6IEQPdiojNUch40vGD27lv0EYpJh8gKjwmaL8kvxNJJE3Vp8Ksxd7O8ousYaBHOfNe6K/OJJo2YdenHLdRx1xsncmIRbJ+UWJ7iCPmcZd0amzztZQ04Y3vUT1xYF0241aizb3zLuHF1+C3z6ewYlXAzVY8yZuBpqYf+XkLeS0XnfQN7vjKRbVQL1FW1voywQHud+xRvlFx4PoZGwJxs+vBNW3AfnXGYk7ZQs09TgfK50r/HLauvrZBR/B10di9MSZFROQtwjTiaMgLNnGIm4eTL25HwNjzUqaxm3QO67RmXi4/8yBnlL/wlM/lkXgzxzkqh7GosFZj0NQTZYs5hR8Ruh/I2TSfnEYVoMfhgcbtQcYocYvyAap2OSjZPcm7fCN2cNhjHXwOirYe/n8PVfGJr/Evz5LeNLIeOXRjOKP5UdgLwVkPcJ7Pnc+BILCiPE1t9dTilGEo5wJ+KI+JNJOSIebNGnadZ52LhIvekl2PIq7HgbIpNg7I+MGlxPHIO8qhi2vQlbX4OCb4x5g6eRF/09UsdMNa4ZWUNPvltDTj/PGuKbZjNrEIy6wngd2ghrn4P1S2Dd84yOnQADnMb/K93YfOXR6HmdNXHiRL1+/foubZuTk0NmZqZ3A/IB08epNXz2e1xfPo0lom/LBByTbPwEjEk2erGYpL30m/deYJJjnZG4XA4YMctoxx44uXsCcDYYj3NrTM5F7pESYpIhdQakTofkaeR8tda7//aOOvjuIyNp539iXHgdlGEk7JFzjd49XdDmOepyQdl+4/pC6R6j1pmUbnwp++I8qKuEXR8Yff13f2ZcYE5MM5p80q6C6IHm/38JjEf8rf8H9V8/S0hDOcSfA1N+ZnyxhkR45RBKqXZHz5MadU+lFFz0AKss08jMusjf0Xikyj4YMq+Hi+6Hdc/DN383LuwOnGL0FBk+0/tNAxVHjaSct8IYWqC+wuilkjzNGNQrdbpxy74vv8yCQo3unyMvhxNHYMsy2PSi0RPpg7tg1FwjaQ/K8DwOrQmpK4H8lVCYe/IicFEuNFSfun5YrJGw+4+F/unGdPTgFserrndQXFFPUWUdxe5XeU0D8fZQzooOIyk6jP5RNmwWl1GWW1+D3PeN40UNNJq8Rl8DiSO9UGjdLDIRsu5ltZ7IBbHFxkXH92+HTx/GNe46DqVey3d10ZyobeCKcd7vgiqJuqdTAdjmGdnPGOTqvNuNO0hXL4bXrjWac8JijbbEsJiT703zYk+dZ4s2fso2cjmhYL271rwCjm5xHzPJ6Dp49gzjYlNopF/+dPr0h/P+k4apt1KR/zVq04tEbnuboM0vUR42kC3xM/kyYjr7G6KxWhRhIVbiLRUMdOwjqW4fiXV7ia/eTXTlbs5tOAGrjd02hPXFEXcOrrR5WBJHENx/FNa4FGoKd1OzfwOuw5sJLdqKfc8qLNoBQKUlkt1Bw9jmGsL6+kGsbxjMQZ0AtPVloRmv8phr/YrZQWuJ4QSVlkhyY6dzZNBsLIMzSIoJJyksjL4ujcVijl9xnXGitoHdJywUR53PnsHpoNYypfBVpn39DElfP8NW1yTesc5k7thbUV5ukpFELcwr1G78vJx0E+S+Z7RrVh+HmlLjcWuFuSenG/tst8EV0of6kGhqgvoQXnWQ0IZyXMrK8dh0itNup6T/BdTGDifIaiXIorAeqifIWorVYjE+WxTBVtX0uajaxb7iKlxa49K43zVOl0ZrcLp00zyXBpdL49Qnlzm15kRNA6VV9ZRW1VNcWU9pVR0llcbnkqp6ymsa3NHPIYzpXGZZx9Wuzzm/5m+cy/N8GzQarTXJzgPEUdb0t5brcHbpgXznmsQuPZA8PYDvXAMore0Dx4H8xjVLCLKU4nBpINn9mkso9ZyjDjLFdoBx1v0M13v4oWM58ywOCIX64D5UxYzCkTiGoIHjsMUnU7/jQ0Jz38RWeRCHJZQdkdN4IfgCVtSnsb/QQVWBE77e3BRjsFXRL8pG/6gwzooOo66sno31u4zytRrl3VjujS/j88l/jyCLwtJsvSCLhdBgC6FBFkKCLIQGWd3vJ+eFWC2oDn6RuFyaQ2U17C6qZHdRFXuKKpumiyrqjJXWfEuQRTEoLpnvBv2ebX0qubDiHaYf/DeXBO1BOX8OFu9e25FELczPYj3ZNNBMg9NFYUUdR45XU1hSTHnJUSpKi6g5UYSjogRXdSnB9ceJclQRU1NBDJUUM4ZsZzqrXKM5ccgOhwCqMB5g1AmrcrzzpymIjQghNiKEuIhQRiT1IT4ihNiIUOLsIcQ1LrPPIC7i97hqCrBueZnx2982av0Js9y9cYbjiBuOxZbA4AYXfeudOL9ew+VjxlFd76S63klNg4OqOic17s+1DidRYcHE20OJt4cQbw+lb2QosREhBFub1QgddUazyeHNhBzZTMjhzbBjCWx9DoAwZTFGqfz+fQSNmMWY0EjGAL/BeDDJiVoHR8prOFxWw6GyWo6UGdOHy2pZt7eUY+UNfLA3/5Sy8YWTydvaIomHBlmoc7jYW1xFncPVtH4fWxDDEuxccHZfUvraqTm2l8svnMqg2PCWZUQm1D9iNC354AK8JGphalprDpbWsGZPCblHKzh6wvgf/Eh5DUUVdbhaXQuPCLHTP7ov/WNsJCWH0S/KRni0jdgoo/00NTSIyS7NHS6N0+XC4dI4nEZt2OFyNZs++e5wupo+Nzhd7NqVy8gRI7BaFEopLAqsypi2WozPFovC0s4ypRRRYUHERoQSHRbcuWaAiKFw4W+NVytBQCQQGWZ83t/HysRkL3R1DAqFpHHGq5GjHorcFyQHnWu04bbB+FuDiQoLZni/Pm2u03gx0eUub5d2l7/T+Ddp/e/R/N+tcd0Gh4t6p4t6h4s6h4s6h7Npumleg5M6p4u6hubznU3TVovivGHxDO1rJ6VvBCkJduIiQlrUwnNyDpLSt52LuyERxkBpPiCJWphOwfFqVu8uYc2eUtbsKeFQmdG3OyzYSv9oG/2jbJyf2pekKBv9osLoH20jyf0eGRrU4c/bM5VTuZvM8b18zJKgEPeFx7Fe26XFoggJwLbr7iCJWvjd4TKjxvzW1jruX/cZB0uNxBwTHszUoXEs+N5QMlLiSE2w+zwJC2FGkqhFtzt2otZdYy5h9Z4S9pcY3cUigmFaahz/MW0IU4fGcU5iZED2DhDC2yRRC59yujQFx6v5tqCc1btLWLunhD3FVQBE2oKYMiSWa6cOJiMljmO7NnJhVpv9/YXo1SRRC69ocLrYX1JNfmEl+YUV5BVWknfM6NrUeBXdHhrE5CGx/HjyIKYOjWNkUh+szWrMRd9J7VmItkiiFp1S53Cyt7iKfHcizi+sJK+wgr3FVTQ4T3bBOCs6jGEJds5NiSM10c7wfn0YldSHIKsPxmYQooeTRN2LaK2pd7qobTC6KtU2uKh1OKl1T9c53PMa3PPcXZpKq+rdNeVK9pdW43T3iVMKBseGMyzBzoXDE0lNsJOaaCelr52IUDm1hPAW+b+ph3G6NDsOn2DNnhLW7i1h495qXKtWUOdOyl0ZgyvIokiOj+CcfpHMHNOfYQl2UhMiGdo3AltwAN6iLkSAkUQd4BxOF9ubEnMp3+wtpaLOGKthaHwE58RaGDYoCVuwBVuwFVuwcUdW47Qt2IItqNl0s+Wh7s/hwVZpshDCjyRRB5gGp4tth8pZu9e4GWT9vuNUuhNzSt8IZqcnMXVoHFOGxJLYx+a+6yvNz1ELIc6EJGqTa3C62FJQztq9xp16G/aVUlVvDEA0LMHO3HFJTBkSx5ShsSRE+miQfyGEX0mi9pPaBidl1Q0cr67neHU9ZdUNTZ/Lqus5Xt3AkfIaNh0oo9qdmM9OtPODCQOYMiSOyUNi6RsZ6ue/QgjRHSRRe6DBeXJQl8aeEAdOONl04Dh1DqOXRPP3OoeL2nonZTVGwi1zJ+LG6ePV9dQ2uNo9XliwlZjwYOLsoVwzcSBThsQyeUgscXZJzEL0RpKoW9lSUMZdb2yhsKKuKfE6Ww/R1ujrr0+7L6tFERNujBwWEx7CWdFhpCX1ISYipGleTHgw0eEhxEQEEx0WQnR4sPSkEEK0IIm6mexdhfzqpY3EhIdw2eh+2ILcPR+CTvaAaOwRkZe7kwnpY9yDlRs9Jlq/27thJDchRM8nidrttfUHuffNrQzvF8mSn07q8MJcTlkemcMTuik6IURv1usTtdaav3yWz1OffMf5qfE8O38CdrmrTghhIr06IzmcLu5/ZzuvrDvAlePP4k9XjiEkSG7sEEKYS4eJWik1EPgn0A9wAc9rrf/H14H5Wk29k1te2cjKnYX8KiuFhdPPkfZkIYQpeVKjdgB3aK03KqUigQ1KqU+01jt8HJvPlFTWceML69lSUMbv56Zx7dTB/g5JCCHa1WGi1lofAY64pyuUUjuBs4CATNQHSqq5fsk6DpfV8Oz8CcwY1c/fIQkhxGkp3Ynh1JRSycAqIE1rfaLVsgXAAoDExMQJy5Yt61JAlZWV2O3tPOX3DO0rd/LUhlqcGn4z3kZqTNf7K/syTm8KlDghcGKVOL0rUOIE38aalZW1QWvd9iOOtNYevQA7sAG4sqN1J0yYoLsqOzu7y9uedr+5x/SI+z/U5/7xU51fWHHm+/NRnN4WKHFqHTixSpzeFShxau3bWIH1up2c6lGvD6VUMPBv4CWt9Zve+f7oPq+vP8g9b27lnMRIlv50Egl9ZPAiIUTg8KTXhwL+DuzUWj/l+5C8R2vNM5/l86S7j/Rf540n0hbs77CEEKJTPKlRTwOuBbYqpTa75/2X1voDn0XlBS36SI87iz/9QPpICyECkye9Pr4EAqqDcfM+0r/MTOHOGdJHWggRuHrcnYmlVfXc+MI3bD5Yxu8vH8W1Gcn+DkkIIc5Ij0rUBcerue7v6zhUVsOz8yZwSZr0kRZCBL4ek6h3F1Vy7f+tpbLOwUs3TWFicqy/QxJCCK/oEYl6++Fyrvv7OpSCZQsyGJnUx98hCSGE1wR8ot6w/zg3LFlHZGgQL940haF9A+MOJyGE8FRAJ+ov84pZ8K/1JPax8eJNUzgrOszfIQkhhNcFbKJesf0ov355E0P7RvCvG6fIE7mFED1WQCbqtzYVsPD1LYwZEMXSGyYTFS53Gwoheq6AS9T/Wr2P+9/ZzrkpcfzvdROJkMdmCSF6uIDKcn/Nyee/P9rF90ck8sxPxmEL7vowpUIIESgCIlFrrXnso1387fPdXJ6exBNXjyXYKuN2CCF6B9MnapdL88Dybby45gDzpgzi95enYbHIuB1CiN7D1Ina4XRx5xtbeGvTIX52wVDuuWS4DK4khOh1TJuoaxuc3PLKJj7ZcYw7Z5zDr7KG+TskIYTwC1Mm6qo6Bwv+tZ6v8kv43eWjuE5GwBNC9GKmS9RVDZpr/76WbwvKeeqasVw5foC/QxJCCL8yVaIuqqjjT+tqOVZdy+KfjJdhSoUQAhMl6rLqen743GqOVbv4+w2TOT+1r79DEkIIUzBNoo4KC+aiEQkk1B+RJC2EEM2Y5q4RpRT3zRxJaozcbSiEEM2ZJlELIYRomyRqIYQwOUnUQghhcpKohRDC5CRRCyGEyUmiFkIIk5NELYQQJieJWgghTE4StRBCmJwkaiGEMDlJ1EIIYXKSqIUQwuQ6TNRKqX8opQqVUtu6IyAhhBAteVKjXgpc4uM4hBBCtKPD8ai11quUUsndEAsbj20kvzafPsf6oJRCoVBKYcFifHbPsyhL07Lmn1FgwWJ8Vsb8xm0tytK0XtN0q+2bz++IQztocDa0u1yjT7t9R8tbU6gW7yffWs2HpvgVCq07dxwhhPkoT/5Hdifq97TWaadZZwGwACAxMXHCsmXLOh3M7Qdup0G3n/yE/7T4Imj15dDel8Rp11ctPzd+qViU5ZT9tHUc8M6XYet9erKseZztbtvqy/50x/F0HU/20ZzL5cJiaT/OzlQWPF239Xqe5BeXdrVZOWrrmM3ndbay44nm51vzeU3HbOccbWS32Lmj/x1dOnZWVtYGrfXEtpZ57QkvWuvngecBJk6cqDMzMzu9j+eOPsfGTRsZM3YMWms0Gpd2odGefdYaF66Wy7Tx3jjfpV1N2zRNt9qX0+XssFa9d+9ehg4Zetp1PKmZd0bjSd94grb+3GLa/bZ3316Sk5Pb36cHJ3tj+bQXQ9M0GuO/tpc137ZxWfN1Dh06RP+k/qddt6ME2JkE2aLcWiWU9pZpNEePHqVfv/af53m6fbX12ZNtTv2oO0zcxwqPkZiQeNp12ttFW/tu73w+XWI73XaNjh49Sv9+/dtcv+lLuo15vtDWed58uvHfvsXyZttEhkSSOTXT63GZ5lFcAJP6TaIqrIqMpAx/h9KhnNIcMsdk+juMDuWU5ZCZnunvMDySk5NDZkamv8PoUE5ODpnnZfo7jA7l5OSQeUGmv8PoUKCUJ/gvVumeJ4QQJudJ97xXgNXAOUqpAqXUjb4PSwghRCNPen38uDsCEUII0TZp+hBCCJOTRC2EECYniVoIIUxOErUQQpicJGohhDA5SdRCCGFykqiFEMLkJFELIYTJSaIWQgiTk0QthBAmJ4laCCFMThK1EEKYnCRqIYQwOUnUQghhcpKohRDC5CRRCyGEyUmiFkIIk5NELYQQJieJWgghTE4StRBCmJwkaiGEMDlJ1EIIYXKSqIUQwuQkUQshhMlJohZCCJOTRC2EECYniVoIIUxOErUQQpicJGohhDA5SdRCCGFykqiFEMLkJFELIYTJeZSolVKXKKV2KaXylVL3+DooIYQQJ3WYqJVSVmAxcCkwEvixUmqkrwMTQghh8KRGPRnI11rv0VrXA8uAy30blhBCiEZKa336FZS6CrhEa32T+/O1wBSt9a9brbcAWACQmJg4YdmyZV0KqLKyErvd3qVtu5PE6X2BEqvE6V2BEif4NtasrKwNWuuJbS0L8mB71ca8U7K71vp54HkApVRRVlbW/k5FeVI8UNzFbbuTxOl9gRKrxOldgRIn+DbWwe0t8CRRFwADm30eABw+3QZa676exXUqpdT69r5VzETi9L5AiVXi9K5AiRP8F6snbdTfAKlKqSFKqRDgR8By34YlhBCiUYc1aq21Qyn1a+BjwAr8Q2u93eeRCSGEADxr+kBr/QHwgY9jafR8Nx3nTEmc3hcosUqc3hUocYKfYu2w14cQQgj/klvIhRDC5CRRCyGEyUmiFkIIk/NKolZKPaSUWnia5XO7Oj6IUup7SqmNSimH+y7J5suuV0rluV/XN5s/RCm11j3/VXe3woDi4zL9uVJqq1Jqs1Lqy+b7UUo53fM3K6WWN5sf0GXqy/J0b3+NUmqHUmq7UurlZvPlHO3avgcppbKVUpuUUluUUpc1W9Yjz9HT6a4a9VyMAZ264gBwA/By85lKqVjgQWAKxngkDyqlYtyLHwP+rLVOBY4DN3bx2GY2l66X6cta69Fa63Tgv4Gnmi2r0Vqnu19zms3v6WU6ly6Wp1IqFbgXmKa1HgX8xj1fztGun6O/BV7TWo/DuHfjr82W9bpztMuJWil1n3vo05XAOe55NyulvlFKfauU+rdSKlwpdS4wB3jc/Q2Y0tZ67R1Ha71Pa70FcLVaNAP4RGtdqrU+DnwCXKKUUsCFwBvu9V7AOGFMrxvL9ESzjxG0MSRAq7gCsky7qzyBm4HF7vMQrXWhe76co10vUw30cU9H0cHd0IFcph7RWnf6BUwAtgLhGIWZDywE4pqt8whwi3t6KXBVs2VtrtfBMVvvYyHw22af73fPi8cY7a9x/kBgW1f+zu58dXeZAr8CdgMHgdRm8x3AemANMNc9L+DKtDvLE3gb45fJV+5yu0TO0TMu0/7uYxVg1I4n9LRztDMvj254acP5wFta62qAZu1EaUqpR4BowI5xN2NbPF3vdNobLMqjQaRMqFvLVGu9GFislPoJxs/MxvbTQVrrw0qpocBnSqmtwIm2duHpH+Yn3VmeQUAqkIkxFs4XSqk05BxtrTNl+mNgqdb6SaVUBvAvpVSa1tpFzzlHPXYmbdRtFcJS4Nda69HAw4CtnW09Xe902hssqhiIVkoFtZofCPxRpsto9hNRa33Y/b4HyAHGEbhl2l3lWQC8o7Vu0FrvBXZhJG45R7u2Hhjty68BaK1Xu9eNd3/uSeeoR7qaqFcBVyilwpRSkcBs9/xI4IhSKhiY12z9CvcyOlivMz4GpiulYtwXaKYDH2vjd0820NhD5HrgnS4eozt1W5m6L341mgnkuefHKKVC3dPxwDRgR4CWaXeeo28DWdBUbmcDe5Bz9EzK9ABwEYBSagRGoi7qYeeox7p8C7lS6j7gOmA/Rs1hB1AF3OWetxWI1FrfoJSaBvwvUIdRkNPbWq+d40wC3gJigFrgqDaurKOU+g/gv9yr/kFrvcQ9fyhGTTEW2ATM11rXdekP7UbdWKb/A3wfaMBo//u11nq7+wLQcxgXbi3A01rrv7u3Cbgy7cbyVMCTwCWAE+NcXOZeJudo18p0pHtbO0Yt/i6t9Yqedo56Ssb6EEIIk5M7E4UQwuS62uvD69w/qa5uNft1rfUf/BFPTyBl6l1Snt4nZeoZafoQQgiTk6YPIYQwOUnUQghhcpKohRDC5CRRCyGEyf0/yCy3mJdk0eYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "q1_data=pd.read_csv(\"q1_data.csv\")\n",
    "\n",
    "model = LinearRegression()\n",
    "\n",
    "names,rmses_train,rmses_test,R2 = q1_compare_RMSE(q1_data,model)\n",
    "q1_plot_RMSE(names,rmses_train,rmses_test,R2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### b (20 pts) \n",
    "\n",
    "    Repeat part (a) after applying polynomial expansion with degree=2 and set to the given dataset.(Do not include bias i.e., set parameter include_bias=False)\n",
    "\n",
    "\n",
    "We can see from the graph rmses_train (Train Set Error) is nearly constant on different training set size, on the other hand rmses_test (Test Set Error) is decreasing by the increase in test set size. But they do ot perform similar for our sample sizes. \n",
    "R2 is very high and constant against to set size."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>rmses_train</th>\n",
       "      <th>rmses_test</th>\n",
       "      <th>R2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>data_100</th>\n",
       "      <td>4.867680e-14</td>\n",
       "      <td>19.516845</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_150</th>\n",
       "      <td>7.299504e-14</td>\n",
       "      <td>16.413674</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_200</th>\n",
       "      <td>6.397601e-14</td>\n",
       "      <td>15.330730</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_250</th>\n",
       "      <td>6.638561e-14</td>\n",
       "      <td>16.948453</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_300</th>\n",
       "      <td>6.570031e-14</td>\n",
       "      <td>13.164103</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_350</th>\n",
       "      <td>7.600682e-14</td>\n",
       "      <td>12.267090</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_400</th>\n",
       "      <td>7.312986e-14</td>\n",
       "      <td>11.235156</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_450</th>\n",
       "      <td>7.478335e-14</td>\n",
       "      <td>11.500781</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_500</th>\n",
       "      <td>6.966378e-14</td>\n",
       "      <td>11.126507</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_550</th>\n",
       "      <td>8.447860e-14</td>\n",
       "      <td>8.540843</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_600</th>\n",
       "      <td>8.153583e-14</td>\n",
       "      <td>8.211557</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_650</th>\n",
       "      <td>7.868059e-14</td>\n",
       "      <td>6.752342</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_700</th>\n",
       "      <td>8.108696e-14</td>\n",
       "      <td>6.587966</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_750</th>\n",
       "      <td>8.554272e-14</td>\n",
       "      <td>6.998786</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_800</th>\n",
       "      <td>8.122647e-14</td>\n",
       "      <td>5.787942</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_850</th>\n",
       "      <td>7.265919e-14</td>\n",
       "      <td>5.945031</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_900</th>\n",
       "      <td>9.314790e-14</td>\n",
       "      <td>8.811258</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_950</th>\n",
       "      <td>6.889907e-14</td>\n",
       "      <td>5.427103</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_1000</th>\n",
       "      <td>8.851408e-14</td>\n",
       "      <td>3.967694</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            rmses_train  rmses_test   R2\n",
       "data_100   4.867680e-14   19.516845  1.0\n",
       "data_150   7.299504e-14   16.413674  1.0\n",
       "data_200   6.397601e-14   15.330730  1.0\n",
       "data_250   6.638561e-14   16.948453  1.0\n",
       "data_300   6.570031e-14   13.164103  1.0\n",
       "data_350   7.600682e-14   12.267090  1.0\n",
       "data_400   7.312986e-14   11.235156  1.0\n",
       "data_450   7.478335e-14   11.500781  1.0\n",
       "data_500   6.966378e-14   11.126507  1.0\n",
       "data_550   8.447860e-14    8.540843  1.0\n",
       "data_600   8.153583e-14    8.211557  1.0\n",
       "data_650   7.868059e-14    6.752342  1.0\n",
       "data_700   8.108696e-14    6.587966  1.0\n",
       "data_750   8.554272e-14    6.998786  1.0\n",
       "data_800   8.122647e-14    5.787942  1.0\n",
       "data_850   7.265919e-14    5.945031  1.0\n",
       "data_900   9.314790e-14    8.811258  1.0\n",
       "data_950   6.889907e-14    5.427103  1.0\n",
       "data_1000  8.851408e-14    3.967694  1.0"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD5CAYAAAAp8/5SAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAw00lEQVR4nO3dd3hUZfr/8fedUAIECIiEEhREeoBAqGIhYgEEERUVGyrK2lZdG7D+FHVdV76iqwirsgq2VVQERURAMKFIB4PUSBEkgIAoJRBacv/+OAOGOCGTyZmZzOR+XddcM6d/chjuOXPmnOcRVcUYY0zkigp1AGOMMYFlhd4YYyKcFXpjjIlwVuiNMSbCWaE3xpgIVybUAbypUaOG1q9f369lDx48SKVKldwNFACW033hktVyuitcckJgsy5btuxXVT3T60RVLXGP5ORk9VdqaqrfywaT5XRfuGS1nO4Kl5yqgc0KLNUCamqhp25EpJ6IpIrIWhFZLSIPesZXF5FvRGS957laAct3F5EMEdkgIkOK84lljDGm6Hw5R38ceERVmwGdgPtEpDkwBJilqo2AWZ7hU4hINDAa6AE0B/p7ljXGGBMkhRZ6Vd2hqss9rw8Aa4G6QB/gXc9s7wJXeVm8A7BBVTep6lFgvGc5Y4wxQVKkH2NFpD7QBlgExKvqDnA+DESkppdF6gJb8wxnAh39i2qMKWmOHTtGZmYmhw8fDlmGqlWrsnbt2pBtvyjcyBoTE0NCQgJly5b1eRmfC72IxAKfAQ+p6n4R8WkxL+O8Nq4jIoOAQQDx8fGkpaX5Gu0UWVlZfi8bTJbTfeGSNZJyxsbGEh8fT926dfGxJrguJyeH6OjokGy7qIqbVVXZt28fK1asICsrq2gLFvYAygLTgYfzjMsAante1wYyvCzXGZieZ3goMLSw7dlVNyVHuORUDZ+skZRzzZo1mpubG/gwp7F///6Qbr8o3Miam5ura9as+dN4innVjQBvA2tV9eU8kyYDAzyvBwBfeFl8CdBIRBqISDngBs9yxpgIEaoj+dLKn/3ty1U3XYBbgItFJN3z6Am8AFwqIuuBSz3DiEgdEZkKoKrHgftxvg2sBT5R1dVFTumr3T9S7siegK3eGGPCUaHn6FV1Ht7PtQN08zL/dqBnnuGpwFR/A/osey+MuYgGZ5wHXBPwzRljTLiInLZuKsRB8u3U+uVb2BUev8AbYyLLK6+8wqFDh4q83FNPPcXMmTMDkMgROYUe4MJHyYmOgVnPhjqJMSYEVJXc3NyQbf90hT4nJ6fA5Z599lkuueSSQMUqmY2a+a1idX4+6xrOyXgftiyAszuHOpExpcYzX65mzfb9rq6zeZ0qDOvd4rTzbNmyhX79+pGSksJHH31EXFwc3bp1Y+HChbRu3Zrbb7+dYcOGsWvXLv73v//RoUMHZs+ezYMPPgg4P27OmTOHypUr8+KLL/LJJ59w5MgR+vbtyzPPPMPBgwe57rrryMzMJCcnhyeffJLrr7/+TzlGjhzJ9u3bSUlJoUaNGqSmphIbG8vDDz/M9OnTeemll/j666+ZMWMG2dnZnHfeebz55puICLfddhu9evXi2muvpX79+gwYMIAvv/ySY8eO8emnn9K0adNi7cfIOqIHMhN6Q2wtmDkMrD9cY0qFjIwMbr31Vr7//nu2bt3Kgw8+yA8//MC6dev48MMPmTdvHiNGjOD5558HYMSIEYwePZr09HTmzp1LhQoVmDFjBuvXr2fx4sWkp6ezbNky5syZw7Rp06hTpw4rVqxg1apVdO/e3WuGBx54gDp16pCamkpqairgtFaZmJjIokWLOP/88xk0aBBLlixh1apVZGdnM2XKFK/rqlGjBsuXL+eee+5hxIgRxd4/kXVED+RGl4eUofDlg5AxFZpeEepIxpQKhR15B9LZZ59Np06d2Lx5Mw0aNKBly5YAtGjRgm7duiEitGzZks2bNwPQpUsXHn74YW666SauvvpqEhISmDFjBjNmzKBNmzaAc8PY+vXrueCCC3j00UcZPHgwvXr14oILLvA5V3R0NNdc88fFIXPnzuW6667j0KFD/Pbbb7Ro0YLevXv/abmrr74agOTkZCZOnOjvbjkp4o7oAUi6Gc5oBDOfgZzjoU5jjAmwvG28ly9f/uTrqKiok8NRUVEcP+7UgyFDhvDWW2+RnZ1Np06dWLduHarK0KFDSU9PJz09nQ0bNjBw4EAaN27MsmXLaNmyJUOHDuXZZ33/DTAmJubknbCHDx/m4YcfZsKECaxcuZK77rqrwKYjTmSOjo4+mbk4IrPQR5eBS4bBrxmw4sNQpzHGlDAbN26kZcuWDB48mHbt2rFu3Touv/xyxo4de7JpgW3btrFr1y62b99OxYoVufnmm3n00UdZvnx5geutXLkyBw4c8DrtRFGvUaMGWVlZTJgwwf0/rAARd+rmpKa9IKE9pP4LEq+FchVDncgYU0K88sorpKamEh0dTfPmzenRowfly5dn7dq1dO7sXMQRGxvLBx98wIYNG3jssceIioqibNmyvP766wWud9CgQfTo0YPatWufPE9/QlxcHAMGDKBly5bUr1+f9u3bB/RvPEVBbSOE8uFaWzc/zVMdVkV17st+ry9QIqm9k5IiXLJGUk5vba4EW2lr60bV+36nOG3dhLX6XaBxd5j7bzj0W6jTGGNMSER2oQfoNgyO7Id5Lxc+rzHG+KBv374kJSWd8pg+fXqoYxUocs/RnxDfHJJuhEVjoMNfIK5eqBMZY8LcpEmTQh2hSCL/iB6g61DnOe1foc1hjDEhUDoKfVw96DgI0j+EnYFrJdkYY0qi0lHoAc5/GMpXsQbPjDGlTukp9BWrwwV/gx+nwebvQp3GGBOB/G2mGODzzz9nzZo1LidylJ5CD9Dxbqhcxxo8MyZCaQluprgwIS30IjJWRHaJyKo84z7O063gZhFJL2DZzSKy0jPfUhdz+6dsBafBs8wlsM57q3HGmPCyZcsWmjVrxr333kv16tVp2LAhd955J4mJidx0003MnDmTLl260KhRIxYvXgzA7NmzT14W2aZNm5PNFrz44ou0b9+eVq1aMWzYMMBpgfKKK66gdevWJCYm8vHHH3vNkbeZ4pSUFABmzJhB586dadu2Lf369TvZvMKQIUNo3rw5rVq14tFHH2X+/PlMnjyZxx57jKSkJDZu3OjqPvLl8sp3gFHAeydGqOrJxphF5CVg32mWT1HVX/0N6LrWN8L8UU6DZ417OO3iGGOK7+sh8MtKd9dZqyX0eKHQ2TIyMhg3bhyPP/445557Lg8++CBjxoyhffv2J5spnjx5Ms8//zyff/75yWaKu3TpQlZWFjExMac0U6yqXHnllcyZM4fdu3dTp04dvvrqKwD27fNe7h544AFefvllUlNTqVGjBr/++ivPPfccM2fOpFKlSgwfPpxRo0bxyCOPMGnSJNatW4eIsHfvXuLi4rjyyitPtknvtkKP6FV1DuD1tlJxuiO/DvjI5VyBc6LBsz3rIf2DUKcxxrjgRDPFwMlmiqOiogptpnjkyJHs3buXMmXKnNJMcdu2bVm3bh3r16+nZcuWzJw5k8GDBzN37lyqVq3qU6aFCxeyZs0aunTpQlJSEu+++y5bt26lSpUqxMTEcOeddzJx4kQqVgx8O1zFPZy9ANipqusLmK7ADBFR4E1VHVPQikRkEDAIID4+nrS0NL8CZWVlFb6sVqRNlabETH+aRXtrO23YB5lPOUuAcMkJ4ZM1knJWrVr1j9Yaz38iMEEKaA3yhNzcXCpUqMCBAwfIysqibNmyJzPl5OSQm5vLgQMHOHToEEePHuXAgQPcd999dO3alRkzZtCxY0cmT57MkSNH+Nvf/sYdd9zxp22kpaUxY8YMHn/8cS6++GKGDBniNYuqkpWVRfny5Tl06BBdu3Zl3LhxJ6fn5OSQnZ3NrFmzSEtL49NPP+XVV19lypQpHDt2jOzs7AJbv8zr8OHDRXsPFdQITt4HUB9Y5WX868Ajp1mujue5JrACuNCX7bnWqNnpbJ7vNHg2Z4Tf2yqOSGrYqqQIl6yRlLMkNGq2cuVKbdGihaqq/vTTTydfq6oOGDBAP/300z9N27Bhw8l5+vTpo5MmTdLp06drhw4d9MCBA6qqmpmZqTt37tRt27Zpdna2qqpOmjRJ+/TpU2CWxMRE3bRpk6qq7tq1S+vVq6fr169XVdWDBw/qsmXL9MCBA7pz505VVd2zZ49Wq1ZNVVXvv/9+HTt2rE9/c1EbNfP7iF5EygBXA8mn+RDZ7nneJSKTgA7AHH+36aqzO0OTnjDvFUi+3bn80hhTKgSrmeJ33nmH/v37c+TIEQCeeOIJateuTZ8+fTh8+DCqyr///W8AbrjhBu666y5GjhzJhAkTaNiwoXt/cEGfAKqnP6IHugOzT7NMJaByntfzge6+bC8oR/SqqjvXqD4dpzrt735vz1+RdFRXUoRL1kjKWRKO6K2ZYgfFaaZYRD4CFgBNRCRTRAZ6Jt1Avh9hRaSOiEz1DMYD80RkBbAY+EpVpxXjM8l9NZs5DZ4tHgN7fw51mj+owr7MUKcwxkSIQk/dqGr/Asbf5mXcdqCn5/UmoHUx8wVe17/DygmQ+jz0fSPUaZwi/82TMP81uO59aH5lqBMZY/Lp27cvP/300ynjhg8fzuWXXx6iRKdnF5FXrevcMfvdq9D5fqiVGLosubkwbbDzDSO6HMwfaYXemBLImikOR+c/BDFVYdYzocuQmwNTHnSK/Hl/hcuec+7g3bokdJmMMRHBCj1AhWpwwSOwfgb8NDf42885Dp/fA8vfgwsfh0v/AUk3QfmqsHB08PMYYyKKFfoTOgyCKgnBb/As5xh8NhB++BgufhIufgJEoHwsJA+ANZNL1g/FxpiwY4X+hLIxkPJ32LYM0l5wjrID7fgR+ORWWPM5XPZPuPDRU6d3/IvzvOjNwGcxxkQsK/R5tb4Bml8Fs1+At7q530BTHlE5R2D8jZAxFXqOgPPu//NMVROgeR/nlM6Rwm+LNqY0iouLIykpicTERHr37s3evXsBSE9Pp3PnzrRo0YJWrVoV2OpkaWCFPq+oaOj3DvR7F/ZvgzFdYdY/4Nhhd7dzJIuWK/8BG2bBlaOgw10Fz9v5PjiyH77/n7sZjIkQFSpUID09nVWrVlG9enVGj3Z+16pYsSLvvfceq1evZtq0aTz00EMnPwRKGyv0+YlAi6vgvsXQ6nqYOwLevAB+XujO+g/vgw+uIW7varh6DLS95fTzJ7SDhA6w6HXnyhxjTIE6d+7Mtm3bAGjcuDGNGjUCoE6dOtSsWZPdu3eHMl7I2HX0BalYHa76DyReA18+BGO7O0fe3Z6C8pX9W+eh3+CDa+CXH1jT/FFatLrOt+U63wuf3gYZX0OzXv5t25gAG754OOt+W+fqOptWb8rgDoN9mjcnJ4dZs2YxcODAP01bvHgxR48edbf9mDBiR/SFObcb3LvA+WF08X/hP51h/cyir+fgr/DelbBzFVz/AbtrdvF92aa9oepZsPA/Rd+uMREuOzubpKQkzjjjDH777TcuvfTSU6bv2LGDW265hXHjxhEVVTpLnh3R+6J8LPQY7hzdf3E//O8aaHUDdP+Xb61eHtjpFPnfN0P/j+DcS2BHmu/bjy7jfNDMeAK2fw912vj7lxgTML4eebvtxDn6ffv20atXL0aPHs0DDzwAwP79+7niiit47rnnTnZMUhqVzo83f9XrAHfPdW5qWjUBRrWHVZ+d/rr7fdvgnZ6wdyvcNMEp8v5oewuUi4UFdlRvjDdVq1Zl5MiRjBgxgmPHjnH06FH69u3LrbfeSr9+/UIdL6Ss0BdVmfLOTU2DZkNcPZhwh3OZ5P7tf5739y0wrgdk7YJbJkGDC/zfbkxVaHMLrJ7ofVvGGNq0aUPr1q0ZP348n3zyCXPmzOGdd9452RF4enp6qCOGhJ268VetRBg407ka5tt/wuiOcOmz0HYAREXBno3wbm84ehBu/QLqti3+Njv+BRa94fxWcMmw4q/PmAiwY8eOU4a//PLLk69vvvnmYMcpkeyIvjiiyzgNkN07H2q3hikPOefif5zuHMkfPwy3TXGnyANUbwBNr4Bl45wPEGOM8YEVejdUPwcGfAm9R8KOH+BDz2WTt02FWi3d3Vbn+yH7d1jxUeHzGmMMPhR6ERkrIrtEZFWecU+LyDYRSfc8ehawbHcRyRCRDSLivdv0SCHiNEJ23yI4/2G4/Wuo2dT97ZzVybnqZuHrTvv1xoSYBrMRQOPX/vbliP4dnP5h8/u3qiZ5HlPzTxSRaGA00ANoDvQXkeZFThhuqtR2zp+fEaAbM0Sg032wZwNs+CYw2zDGRzExMezZs8eKfZCoKnv27CEmJqZIy/nSleAcEanvR6YOwAZPl4KIyHigD7DGj3WZvFpcBd88BQtGQ+OS2XWZKR0SEhLIzMwMadMChw8fLnLhCxU3ssbExJCQkFCkZcSXT2JPoZ+iqome4aeB24D9wFLgEVX9Pd8y1wLdVfVOz/AtQEdV9dJMI4jIIGAQQHx8fPL48eOL9IeckJWVRWxsrF/LBlNxc561ZQLn/PQ+S9q9ysHY+u4Fyydc9ieET1bL6a5wyQmBzZqSkrJMVdt5naiqhT6A+sCqPMPxQDTOqZ9/AmO9LNMPeCvP8C3Aa75sLzk5Wf2Vmprq97LBVOycB/eoPldLddI9ruQpSLjsT9XwyWo53RUuOVUDmxVYqgXUVL+uulHVnaqao6q5wH9xTtPklwnUyzOcANidPm6pWB2SboSVnzpNLBhjTAH8KvQiUjvPYF9glZfZlgCNRKSBiJQDbgAm+7M9U4CO90DOUVj6dqiTGGNKMF8ur/wIWAA0EZFMERkI/J+IrBSRH4AU4G+eeeuIyFQAVT0O3A9MB9YCn6jq6gD9HaVTjXOhcXdY8rb7naMYYyKGL1fd9Pcy2ushpKpuB3rmGZ4K/OnSS+OiTvc6d+Ou/ATa3hrqNMaYEsjujA13DS6E+ESnVUu7ltkY44UV+nAn4vQru3stbPw21GmMMSWQFfpIkHgNVKppPVAZY7yyQh8JypR3+rPdMBN2Z4Q6jTGmhLFCHyna3QHR5e2o3hjzJ1boI0WlGtD6elgxHg7uCXUaY0wJYoU+knS61+nsZOnYUCcxxpQgVugjSc1m0LAbLPkvHD8S6jTGmBLCCn2k6XwvZO2EVRNDncQYU0JYoY80DbvBmU1h4Wi7gcoYA1ihjzwi0Oke+GUlbJ4X6jTGmBLACn0kanU9VDzD6YHKGFPqWaGPRGUrONfV/zgNZj4D2XtDncgYE0JW6CPVeQ84TSPMexlGJsH816wpY2NKKSv0kSqmClz7NvxlDtRpCzP+H7yWDN9/ALk5oU5njAkiK/SRrnZruGUiDPgSYmvCF/fB611g3VS7KseYUsKXHqbGisguEVmVZ9yLIrJORH4QkUkiElfAsps9PVGli8hSF3ObompwIdz1LfR7F3KPwfj+MLY7bFkQ6mTGmADz5Yj+HaB7vnHfAImq2gr4ERh6muVTVDVJVdv5F9G4RgRaXAX3LoRer8Dvm2Fcd/jweti5JsThjDGBUmihV9U5wG/5xs3w9AkLsBBICEA2EyjRZaHd7fDA99DtKeeo/vXzYNI9sPfnUKczxrhM1IfztCJSH5iiqolepn0JfKyqH3iZ9hPwO6DAm6o65jTbGAQMAoiPj08eP368r3/DKbKysoiNjfVr2WAqSTnLHNvPWT9/RkLmV4CyrW5Pfj6rH8fKVSlROQsTLlktp7vCJScENmtKSsqyAs+cqGqhD6A+sMrL+CeASXg+MLxMr+N5rgmsAC70ZXvJycnqr9TUVL+XDaYSmXPvVtVJ96o+Haf6fILq7P/T2TO/DnUqn5XIfeqF5XRXuORUDWxWYKkWUFP9vupGRAYAvYCbPBvx9iGy3fO8y/OB0MHf7ZkgqJoAV42Ge+ZD/Qvg2+fouOhuWDnBrtAxJoz5VehFpDswGLhSVQ8VME8lEal84jVwGbDK27ymhKnZDPp/CHfM4Ej5M+CzgfB+X9izMdTJjDF+8OXyyo+ABUATEckUkYHAKKAy8I3n0sk3PPPWEZGpnkXjgXkisgJYDHylqtMC8leYwDirI8vb/h/0HAHblsF/OkPaC3aHrTFhpkxhM6hqfy+j3y5g3u1AT8/rTUDrYqUzoSfRTsfjzXrD9L9D2r/gh0/gipegYUqo0xljfGB3xhrfVK4F146FmyeC5sL7V8Fnd8KBnaFOZowphBV6UzTndnNuuLpoCKz5Aka1hyVvWfs5xpRgVuhN0ZWNgZShcM8CqJMEXz0Cb18KO1aEOpkxxgsr9MZ/Nc6FW7+Aq99y7qgd0xW+HgKH97u3DVXn9NDerXaJpzF+KvTHWGNOSwRa9YNGl8Csf8CiN2DN59D9BWjex5nui9wcp+2dX390Hrt/hF8znNeH9znzxMZDQnuo1wHqdYTaSc63C2PMaVmhN+6oUA16vQxJN8KUh+DTAXDupdDzRaje4I/5jh6CPRs8xTzjj8K+ZwPkHP1jvko14cwmkHgt1GgMUdGQuRS2LoJ1U5x5ospC7VaQ0AHqtaf84dyg/snGhAsr9MZdCe3grjRYPAZS/wn/6eQU66ydzhH63q04TR8BEgVxZzsF/dxLnIJ+ZhOo0cj54Mivw13Oc9ZuyFwCmYth62JY9g4sep3OAKufgnrtPcW/o/NBUKZ8UP50Y0oqK/TGfdFloPO9zqmb6UOdUznVGjinXZJudgr5mU2gekP/Tr3EnglNezoPgJxjsHMV67/9kEYxvzkfAGu+8GQp55ziqdcBWl4Lddq49VcaEzas0JvAqVoXrnsv8NuJLgt12rAtYR+NunZ1xh3Y6TniXwRbl8Di/8Kyd+G+hU6bPsaUIlboTWSqHO/czdustzP8+2anCYcpD8ONH/v+I7ExEcAurzSlQ7X6cPGTsH660xqnMaWIFXpTenT8C9RtB9MGw8FfQ53GmKCxQm9Kj6houPI154auaUNCncaYoLFCb0qX+OZw4aOw8lP4cXqo0xgTFFboTelz/sNwZjOY8jd3m2swpoSyQm9KnzLloM8o2L8dZj4d6jTGBJwvPUyNFZFdIrIqz7jqIvKNiKz3PHu5jdHpclBEMkRkg4jYSVFTciS0g073wtK3YfN3oU5jTED5ckT/DtA937ghwCxVbQTM8gyfQkSigdFAD6A50F9EmhcrrTFuuvgJpwmGyX+FY9mhTmNMwBRa6FV1DvBbvtF9gHc9r98FrvKyaAdgg6puUtWjwHjPcsaUDOUqQe9X4beNMHt4qNMYEzD+nqOPV9UdAJ7nml7mqQtszTOc6RlnTMnRMAXa3AzfjYTt6aFOY0xAiPrQmYOI1AemqGqiZ3ivqsblmf67qlbLt0w/4HJVvdMzfAvQQVX/WsA2BgGDAOLj45PHjx/v1x+UlZVFbGysX8sGk+V0n79ZyxzLov2S+zlaLo7lbUegUYFtGSRc9qnldF8gs6akpCxT1XZeJ6pqoQ+gPrAqz3AGUNvzujaQ4WWZzsD0PMNDgaG+bC85OVn9lZqa6veywWQ53VesrKu/UB1WRXXOS67lKUi47FPL6b5AZgWWagE11d9TN5OBAZ7XA4AvvMyzBGgkIg1EpBxwg2c5Y0qe5ldCsysh7QX4dX2o0xjjKl8ur/wIWAA0EZFMERkIvABcKiLrgUs9w4hIHRGZCqCqx4H7genAWuATVV0dmD/DGBf0HOG0jz/5Aci13qpM5Cj0ZKSq9i9gUjcv824HeuYZngpM9TudMcFUOR4ufx6+uA+WjYX2d4Y6kTGusDtjjckr6SY4pyt8Mwz2ZYY6jTGusEJvTF4i0OsV0FynkxIfrkozpqSzQm9MftUbWCclJqJYoTfGmxOdlHz9uHVSYsKeFXpjvDnRScmRA9ZJiQl7VuiNKUjeTkoypoU6jQmkCP8txgq9MadjnZREvlWfwegOcGBnqJMEjBV6Y07nRCclB3bAzGGhTmPclr0Xpj4Ov/4I80eGOk3AWKE3pjAnOykZC2unQM6xUCcybpk9HA7tgXodYcnbkLU71IkCwgq9Mb64+AmoVh8+vgmerwtjusKXD8HScbD9ezh+JMQBTZHtzoDFY6DtrdBnNOQcgQWvhTpVQAS2PVZjIkW5SnBXKmz8Fnakw44VsHoiLBvnTI8qAzWbQe0kqN0a6rSB+BZQtkIoU5uCqDpXU5WtBN2egko1IPEaWPwWnPcgVDoj1AldZYXeGF9VrA4tr3Ue4BSL3zc7Rf9E8V/3FXz/vjNdouHMJn8U/9qtoVbLEIU3p8iY6nxod3/BKfIAFz7m3CC34DW45OmQxnObFXpj/CXi3EVbvQG0uMoZp+q0kZO3+G+YCSs+PLEQTeO7wvmdoUz50OQu7Y4dhul/hzObntpw3ZlNoEVfWPxfOO8B54M9QlihN8ZNIhBXz3k06/XH+P07nKK/KZVai96A9/vC9R9EVDEJGwtGOd/EbvkcosueOu2ix51TcgtGQ7cnQ5EuIOzHWGOCoUptaNIdegxnTbNHIHMJvH2ZU3BM8OzbBnNfgqa9nP6C86vZDJr3gUVvQvbvwc8XIFbojQmyXfEXOkeTB3fDW5fAtmWhjlR6zBwGuTlw+T8LnufCx+HoAVj4evByBZgVemNCoX4XGPiNc1XOuCtgnfXPE3A/L3Sas+jygHOpbEFqJTpH/AvfcG6oigB+F3oRaSIi6Xke+0XkoXzzdBWRfXnmearYiY2JFGc2hjtnOacLPr4JFo0JdaLIlZsDUx+DKnXh/L8VPv9Fg+HIPucUTgTwu9CraoaqJqlqEpAMHAImeZl17on5VPVZf7dnTESKrQm3TYHG3eHrx2D6E9ZfbSB8/z788gNc+qxzT0RhareCJlfAwtFweF/g8wWYW6duugEbVXWLS+szpvQoV8m5AqfDX5wrQibcBseyQ50qcmTvhVnPwlmdnZuifHXRY06RXxz+37REXWieU0TGAstVdVS+8V2Bz4BMYDvwqKquLmAdg4BBAPHx8cnjx4/3K0tWVhaxsbF+LRtMltN94ZK1wJyqJGROpuHGceyv0phViU9wrFzV4Af0CPv96XHu+reou20Ky5JfJqvyOUVad+LK56i6bx0LO40hp0zF4kYN6D5NSUlZpqrtvE5U1WI9gHLAr0C8l2lVgFjP657Ael/WmZycrP5KTU31e9lgspzuC5esheZc/bnqP2qqvpqk+uuGoGTyJiL25861qk9XU538oH8rz1yqOqyK6pwR/i2fTyD3KbBUC6ipbpy66YFzNP+nxpxVdb+qZnleTwXKikgNF7ZpTORq3gcGfOmccnjrEvh5UagThSdVmDYYysc6fQD7o24ynHspzB8FR7LczRdEbhT6/sBH3iaISC0REc/rDp7t7XFhm8ZEtnod4M6ZUCEO3u0Na74IdaLws+4r2JQGKU8Ur5GyiwZD9m+w5C3XogVbsQq9iFQELgUm5hl3t4jc7Rm8FlglIiuAkcANnq8YxpjCnNEQBs50GkP7ZIBzVBnq/z65ObA93Wki4PP7YMuC0OYpyMn2bJpBu4HFW1e99tDwYpj/Ghw96E6+ICtWWzeqegg4I9+4N/K8HgWMyr+cMcZHlc6AAZNh4iCY8QTs3eK0uBgVHZzt5xxz2ujZPA+2fOfcdHTE06Vi2YpOY20XPu60/BhdgprOWvCas69u/cKdXBcNhrGXO53PnPfX4q8vyErQv4wxxquyFaDfu/DNk87ll/sy4bLnoHJtKFf8K0FOcfwIbFsOW+bRasUU+G49HPMcxdZoDIlXw9nnO3f2lq/s3IQ0+wXnFMk1/4W4s9zN449922Duy9CsN5zT1Z11ntUJGlwE3410viG4vd8DzAq9MeEgKsppnyXubOcHxgxPkwnlq0DlWhAbn++5FlSO9zzXcoqy83PZqY5lQ+ZS52h98zynsbXjhwEoV+lsSLrRKepnd3Fu7sqv7xvQsJvTefob50PvkX802Rwq3zzlnGK67Dl319t1CIzrAcvegc73urvuALNCb0w46TgIzu4MO36ArF/gwM4/nrcuhqydJwv1KcpWPPXDoOIZsGuN06BazlFAnDZekm93CvtZ57F0yUq6du1aeKZW/Zx+dT8bCJ8OgI23OqeXfLkD1W1bFsCqCc7ppNO1Z+OPs8+D+hfAd69Au9vDqvcwK/TGhJtaLQvuqUrVuZszaycc+MV55P9A2LnK6QT7jIbQ8S/OqZizOjlX+PiregO4YzqkPg/z/u2cy7/mbacpgWDJzXGakahSF85/KDDbuGgwvNsLlr/n7LswYYXemEgi4hTsCnFOj0nBFF0WLhnmnBefOAje6gaXPAOd7vF+2shty9+DX1bCtWMD922i/vlw1nnOh1nbAVA2JjDbcZk1U2yMcdc5F8E9851z99OHwofXOd8gAin7d6c9m7O7QIurA7cdEeg6GA7s+KNv4DBghd4Y475KZ0D/j6DHi7BpNrzRxemMO1DSXoDDe53fBgL97aHBRVCvo3NUf/xIYLflEiv0xpjAEHF+PL7rW6hQzeknd8aTcPyoq5uplLXF6dA7+bbg/CYg4pyr378Nvv8g8NtzgRV6Y0xg1UqEu1Kh3R0wfySMvQz2bHRn3aqcu+Et5/LRlP/nzjp90fBiqNvOc1Tv7gdXINiPscaYwCtXEXr92ymQX9wPb1wAV7wErW/wfqrl+FE4uMu5eigr//Op46odO+ScIipOezZFJeJcV/+/a527g5NvC962/WCF3hgTPM16Q502zlU5n98NGV85N4HlL+DZv3tfvkJ15z6A2JqQ0AFia5KxR2nSvpjt2fjj3EugTluY+xIk3eRcdVRCWaE3xgRX1QSnGea5L8Ps4RBVxnMXbzzUaORcwniimJ98rgWVzoQy5f60uh1paTQJVts/eZ04V//R9bBiPLS9JfgZfGSF3hgTfFHRTld95//NeR2M6+wDofHlTuuic0c4p6FK6FG9/RhrjAmd6DLhW+Thj6P63zfDyk9DnaZAVuiNMaY4mvSE+JbOtfwH/tTRXolghd4YY4pDBHoMh4O74e1LYHdGqBP9SXF7mNosIitFJF1ElnqZLiIyUkQ2iMgPItK2ONszxpgSqX4XuO0rp9nnty91mnwuQdw4ok9R1SRVbedlWg+gkecxCHjdhe0ZY0zJU7et089vbLxzF/APJeecfaBP3fQB3lPHQiBORGoHeJvGGBMa1erDwBnONf4T73SusQ91P7+AFKevbhH5CfgdUOBNVR2Tb/oU4AVVnecZngUMVlVvp3kG4Rz1Ex8fnzx+/Hi/MmVlZREbG+vXssFkOd0XLlktp7tKYk7JPUbTdSOJ3zWH7bUvY32ju9Go6IBmTUlJWVbAmRVQVb8fQB3Pc01gBXBhvulfAefnGZ4FJBe23uTkZPVXamqq38sGk+V0X7hktZzuKrE5c3JUZz6jOqyK6vtXqx7eH9CswFItoKYW69SNqm73PO8CJgEd8s2SCdTLM5wAbC/ONo0xJixERUG3p6D3q7AxFcb1oNyRPaGJ4u+CIlJJRCqfeA1cBqzKN9tk4FbP1TedgH2qusPvtMYYE26Sb4MbP4bffqLt8sdh55qgRyjOEX08ME9EVgCLga9UdZqI3C0id3vmmQpsAjYA/wXCq+t0Y4xxQ6NL4favEc2BsZfDprSgbt7vtm5UdRPQ2sv4N/K8VuA+f7dhjDERo3Yrlrd9kc6bXoYProErR0FS/6Bs2u6MNcaYIDkScybcMc3p2/bzuyFteFAuv7RCb4wxwRRTFW6aAK1vhLTn4Yv7IOdYQDdpzRQbY0ywlSkHV/0Hqp0Naf9y+p+97j3nQyAA7IjeGGNC4UR3hH3+47SNM7YH7NsWkE1ZoTfGmFBqc5NzKmffVqdBtCNZrm/CTt0YY0yoNUxxfqTNXALl3W8iwQq9McaUBPEtnEcA2KkbY4yJcFbojTEmwlmhN8aYCGeF3hhjIpwVemOMiXBW6I0xJsJZoTfGmAhnhd4YYyKcFXpjjIlwxelKsJ6IpIrIWhFZLSIPepmnq4jsE5F0z+Op4sU1xhhTVMVpAuE48IiqLvf0HbtMRL5R1fwdIs5V1V7F2I4xxphi8PuIXlV3qOpyz+sDwFqgrlvBjDHGuMOVc/QiUh9oAyzyMrmziKwQka9FJDAt9hhjjCmQaDH7KxSRWGA28E9VnZhvWhUgV1WzRKQn8KqqNipgPYOAQQDx8fHJ48eP9ytPVlYWsbHuN/PpNsvpvnDJajndFS45IbBZU1JSlqlqO68TVdXvB1AWmA487OP8m4Eahc2XnJys/kpNTfV72WCynO4Ll6yW013hklM1sFmBpVpATS3OVTcCvA2sVdWXC5inlmc+RKQDzqmiPf5u0xhjTNEV56qbLsAtwEoRSfeM+ztwFoCqvgFcC9wjIseBbOAGzyePMcaYIPG70KvqPEAKmWcUMMrfbRhjjCk+uzPWGGMinBV6Y4yJcFbojTEmwlmhN8aYCGeF3hhjIpwVemOMiXBW6I0xJsJZoTfGmAhnhd4YYyJcsVuvDIR27drp0qVLi7zc8MXDWbhpIXFxce6HctnevXstp8vCJavldFe45ITCszat3pTBHQb7tW4RKbD1SjuiN8aYSFdQs5ahfFgzxSVHuORUDZ+sltNd4ZJTNQybKTbGGBMerNAbY0yEs0JvjDERzgq9McZEOCv0xhgT4YpV6EWku4hkiMgGERniZbqIyEjP9B9EpG1xtmeMMaboitM5eDQwGugBNAf6i0jzfLP1ABp5HoOA1/3dnjHGGP/4fWesiHQGnlbVyz3DQwFU9V955nkTSFPVjzzDGUBXVd1xunX7e2fsM1+uZv6an8PiLrlwuZsvXHJC+GS1nO4Kl5xQeNbmdaowrHcLv9Z9ujtj/e4cHKgLbM0znAl09GGeusCfCr2IDMI56ic+Pp60tLQiB8rMPEJOTg579+4t8rLBZjndFy5ZLae7wiUnFJ41M3c/aWm7Xd9ucQq9eBmX/+uBL/M4I1XHAGPAOaLv2rVrkQN17QppaWn4s2ywWU73hUtWy+mucMkJoctanB9jM4F6eYYTgO1+zGOMMSaAilPolwCNRKSBiJQDbgAm55tnMnCr5+qbTsC+ws7PG2OMcZffp25U9biI3A9MB6KBsaq6WkTu9kx/A5gK9AQ2AIeA24sf2RhjTFEU5xw9qjoVp5jnHfdGntcK3FecbRhjjCkeuzPWGGMinBV6Y4yJcFbojTEmwlmhN8aYCFciOwcXkd3AFj8XrwH86mKcQLGc7guXrJbTXeGSEwKb9WxVPdPbhBJZ6ItDRJYW1N5DSWI53RcuWS2nu8IlJ4Quq526McaYCGeF3hhjIlwkFvoxoQ7gI8vpvnDJajndFS45IURZI+4cvTHGmFNF4hG9McaYPKzQG2NMhLNCb4wxEa5EFHoReVpEHj3N9Ku8dDzu67ovFJHlInJcRK7NN22AiKz3PAbkGd9ARBZ5xn/saW8/rAR4n94tIitFJF1E5uVdj4jkeMani8jkPOPDep8Gcn96lr9ORNaIyGoR+TDPeHuP+rfus0QkVUS+F5EfRKRnnmkR+R49nRJR6H1wFeDvf6KfgduAD/OOFJHqwDCcfm47AMNEpJpn8nDg36raCPgdGOjntkuyq/B/n36oqi1VNQn4P+DlPNOyVTXJ87gyz/hI36dX4ef+FJFGwFCgi6q2AB7yjLf3qP/v0f8HfKKqbXA6RfpPnmml7j0askIvIk+ISIaIzASaeMbdJSJLRGSFiHwmIhVF5DzgSuBFzydwQ2/zFbQdVd2sqj8AufkmXQ58o6q/qervwDdAdxER4GJggme+d3HecCVeEPfp/jyDlSigH+A8ucJynwZrfwJ3AaM970NUdZdnvL1H/d+nClTxvK5KIV2YhvM+9UVICr2IJON8yrYBrgbaeyZNVNX2qtoaWAsMVNX5OF0SPub5BN7obT4/YtQFtuYZzvSMOwPYq6rH840v0YK9T0XkPhHZiHNE/0CeSTEislREForIVZ5xYbdPg7w/GwONReQ7z37r7hlv71H/9+nTwM0ikonTOdJf80yLiPdoURSrh6liuACYpKqHAPKcJ0sUkeeAOCAWp5tCb3yd73TEyzg9zfiSLqj7VFVHA6NF5Eacr8knzh+fparbReQc4FsRWQns97YKX/+wEAnm/iwDNAK6AgnAXBFJxN6j+RVln/YH3lHVl0SkM/C+iCSqai6R8x71WSjP0Xvbie8A96tqS+AZIKaAZX2d73QygXp5hhNwvt79CsSJSJl848NBKPbpePJ8xVXV7Z7nTUAaztFbuO7TYO3PTOALVT2mqj8BGTiF396j/s0HztH+JwCqusAzbw3PcCS9R30SqkI/B+grIhVEpDLQ2zO+MrBDRMoCN+WZ/4BnGoXMVxTTgctEpJrnB67LgOmefm5TgRNX6AwAvvBzG8EUtH0qzo+HJ1wBrPeMryYi5T2vawBdgDVhuk+D+R79HEiBk/utMbAJe48WZ5/+DHQDEJFmOIV+d4S9R30WsiYQROQJ4FacduczgTXAQeBxz7iVQGVVvU1EugD/BY7g/ENc5m2+ArbTHpgEVAMOA794rmxARO4A/u6Z9Z+qOs4z/hycI9XqwPfAzap6xM2/PxCCuE9fBS4BjuFcnXC/qq72/ID2Js4P31HAK6r6tmeZsNunQdyfArwEdAdycN6L4z3T7D3q3z5t7lk2FudbxOOqOiPS3qO+srZujDEmwoXLdfTGGGP8FKqrblzn+UrYL9/oT1X1n6HIEwlsn7rL9qf7bJ/6xk7dGGNMhLNTN8YYE+Gs0BtjTISzQm+MMRHOCr0xxkS4/w/RdR+h8Xp+qgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "poly = PolynomialFeatures(2, include_bias=False) ### Generate polynomial and interaction features\n",
    "poly_features = poly.fit_transform(q1_data.drop(\"y\", axis=1))\n",
    "q1_data_poly = pd.DataFrame(poly_features, columns=poly.get_feature_names())\n",
    "q1_data_poly[\"y\"] = q1_data[\"y\"]\n",
    "\n",
    "\n",
    "model = LinearRegression()\n",
    "\n",
    "names,rmses_train,rmses_test,R2 = q1_compare_RMSE(q1_data_poly,model)\n",
    "q1_plot_RMSE(names,rmses_train,rmses_test,R2)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [],
   "source": [
    "def q1_c_compare_RMSE(q1_data,model=LinearRegression(), test_ratio=0.3):\n",
    "    student_id = 2532810\n",
    "    names = []\n",
    "    overall_train= 0\n",
    "    overall_test= 0\n",
    "    rmses_train = []\n",
    "    rmses_test = []\n",
    "    intercepts = []\n",
    "    coefficients = []\n",
    "    R2=[]\n",
    "    arraySize=100\n",
    "    while arraySize<1050:\n",
    "        globals()[f\"data_{arraySize}\"] = q1_data.sample(n=arraySize, random_state=student_id)\n",
    "        globals()[f\"data_{arraySize}\"].name=f\"data_{arraySize}\"\n",
    "        x=globals()[f\"data_{arraySize}\"].drop(\"y\", axis=1)\n",
    "        y=globals()[f\"data_{arraySize}\"][\"y\"]\n",
    "        x_train, x_test, y_train, y_test = train_test_split(x,y, test_size=test_ratio, random_state=student_id)\n",
    "        model.fit(x_train, y_train) ## regression model is fitted\n",
    "        print('intercept=', model.intercept_, 'coefficients=', model.coef_) ##since there are three predictors, .coef_ returns three values in an array\n",
    "        intercepts.append(model.intercept_)\n",
    "        coefficients.append(model.coef_)\n",
    "        \n",
    "        predictions_train=model.predict(x_train)\n",
    "        err_train = mean_squared_error(y_train, predictions_train)\n",
    "        predictions_test=model.predict(x_test)\n",
    "        err_test = mean_squared_error(y_test, predictions_test)\n",
    "\n",
    "        rmse_train = math.sqrt(err_train)\n",
    "        overall_train = overall_train + rmse_train\n",
    "        rmse_test = math.sqrt(err_test)\n",
    "        overall_test = overall_test + rmse_test\n",
    "        R2_array=model.score(x_train,y_train)\n",
    "\n",
    "        #print(globals()[f\"data_{arraySize}\"].name, ' train set RMSE:',rmse_train)\n",
    "        #print(globals()[f\"data_{arraySize}\"].name, ' test set RMSE:',rmse_test)\n",
    "        #print(globals()[f\"data_{arraySize}\"].name, ' R2:',R2_array,'\\n')\n",
    "        names.append(globals()[f\"data_{arraySize}\"].name)\n",
    "        rmses_train.append(rmse_train)\n",
    "        rmses_test.append(rmse_test)\n",
    "        R2.append(R2_array)\n",
    "      \n",
    "        arraySize+=50\n",
    "    return (names,rmses_train,rmses_test,R2,intercepts,coefficients)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [],
   "source": [
    "def q1_c_plot_RMSE(names,rmses_train,rmses_test,R2):\n",
    "    least_RMSE_train = min(rmses_train)\n",
    "    least_RMSE_train_name = names[rmses_train.index(least_RMSE_train)]\n",
    "\n",
    "    #print(\"The least train set RMSE value  :\", least_RMSE_train,\"\\n The least RMSE dataset:\", least_RMSE_train_name, \"\\n The index of dataset:\",rmses_train.index(least_RMSE_train))\n",
    "\n",
    "    most_RMSE_train = max(rmses_train)\n",
    "    most_RMSE_train_name = names[rmses_train.index(most_RMSE_train)]\n",
    "\n",
    "    #print(\"The most train set RMSE value  :\", most_RMSE_train,\"\\n The most RMSE dataset:\", most_RMSE_train_name)\n",
    "\n",
    "    least_RMSE_test = min(rmses_test)\n",
    "    least_RMSE_test_name = names[rmses_test.index(least_RMSE_test)]\n",
    "\n",
    "    #print(\"The least test set RMSE value  :\", least_RMSE_test,\"\\n The least RMSE dataset:\", least_RMSE_test_name, \"\\n The index of dataset:\",rmses_test.index(least_RMSE_test))\n",
    "\n",
    "    most_RMSE_test = max(rmses_test)\n",
    "    most_RMSE_test_name = names[rmses_test.index(most_RMSE_test)]\n",
    "\n",
    "    #print(\"The most test set RMSE value  :\", most_RMSE_test,\"\\n The most RMSE dataset:\", most_RMSE_test_name)\n",
    "\n",
    "    columns=['rmses_train','rmses_test','R2']\n",
    "    results=[]\n",
    "    \n",
    "    variables = range(len(names))\n",
    "    for i in variables:\n",
    "        results.append((rmses_train[i], rmses_test[i], R2[i]))\n",
    "    \n",
    "    res = pd.DataFrame(results, index=names, columns=columns)\n",
    "    display(res)\n",
    "    res.plot()\n",
    "    plt.ylim((min(least_RMSE_train,least_RMSE_test,min(R2))-1),(max(most_RMSE_train,most_RMSE_test,max(R2))+1))\n",
    "    plt.figsize=(20, 20)\n",
    "    #plt.xticks(names)\n",
    "    plt.grid()\n",
    "    return rmses_test.index(least_RMSE_test)\n",
    "    \n",
    "    #f, axes = plt.subplots(2,1, figsize=(20, 20))\n",
    "    #axes[0].plot(names, rmses_train, 'o-', label='Train Set RMSEs') \n",
    "    #axes[1].plot(names, rmses_test, 'o-', label='Test Set RMSEs') \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### c (20 pts) \n",
    "\n",
    "    Use Lasso regularization and try to fit the best possible model to the given dataset. When necessary you can use (%70 train - %30 test) or (%60 train - %20 validation - %20 test) splits. Report the model performance on the train/validation/test sets. \n",
    "    \n",
    "    What is the regression equation for the best model (consider only the significant coefficients - compared to the largest coefficients)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "intercept= -0.025209034113742257 coefficients= [ -1.99852208  10.56708578   0.34280319 -19.46373829   0.20123021\n",
      "  -0.6487325   15.5928272    0.83057923   0.68691186   1.05647378\n",
      "  -0.35712247  -0.25092681 -25.0425635    0.68729005   0.07447374\n",
      "   0.49719831   0.20202832  -0.79593345  -0.12891115  -0.\n",
      "   0.47933336  -1.04805452   0.08181139   0.8356541    0.58895339\n",
      "  -0.93986501   0.64019976  -0.          -0.24379556   0.04244007\n",
      "  -0.          -0.14421589   1.20372316  -0.23921362  -0.35993343\n",
      "  -0.76480117  -0.30845251   0.85024186  -1.60844612   0.11173791]\n",
      "intercept= 0.3344887257487909 coefficients= [-1.21048842e+00  1.00529329e+01  9.32118448e-02 -2.00731285e+01\n",
      " -1.16513101e-01 -7.00355358e-01  1.55957745e+01  1.11922322e+00\n",
      "  3.80501167e-01  3.15188161e-01 -1.19001302e-02 -3.06690048e-01\n",
      " -2.50571773e+01 -2.04678259e-01  0.00000000e+00 -3.36268327e-01\n",
      " -6.32658588e-01  0.00000000e+00 -6.63935617e-02 -3.24843266e-01\n",
      "  3.03005914e-01 -5.91563991e-01  2.57451506e-01 -1.44229095e-01\n",
      " -3.11222445e-01 -5.03204153e-01  4.32975279e-03  9.19852212e-01\n",
      " -5.52955936e-02 -4.22697554e-01 -1.68147503e-01 -2.15427416e-01\n",
      "  2.05383474e-01  7.87650938e-01 -6.80831993e-01 -7.13556241e-01\n",
      "  7.17436605e-01  1.71898349e-01 -0.00000000e+00 -2.34147569e-01]\n",
      "intercept= 0.062153361534670104 coefficients= [-6.70770879e-01  1.00746769e+01  1.22029193e-01 -2.03021976e+01\n",
      " -5.99857639e-02 -7.25355663e-01  1.51676986e+01  5.41439260e-01\n",
      " -1.52891846e-01  8.09046993e-02  3.11104178e-01 -4.87428839e-01\n",
      " -2.51574900e+01  8.34575408e-02 -1.00521221e-01  5.05233950e-01\n",
      "  2.91297699e-01 -5.34182761e-02  0.00000000e+00 -1.00381477e+00\n",
      "  4.08508335e-01 -0.00000000e+00  4.71471359e-01  1.62906673e-01\n",
      " -0.00000000e+00 -2.60711139e-01 -0.00000000e+00  1.31486899e-01\n",
      "  3.50664733e-02 -3.80132441e-01  0.00000000e+00  2.89269565e-01\n",
      "  2.82375264e-03  4.87661921e-01 -2.65615743e-01 -1.08950186e+00\n",
      "  4.49770234e-02  4.79186024e-01  1.07019117e-01 -1.41014881e-01]\n",
      "intercept= -0.04474885483596447 coefficients= [-1.09067386e+00  1.02609992e+01  0.00000000e+00 -2.02463565e+01\n",
      "  8.20342727e-02 -7.34527408e-01  1.50439750e+01  2.99406729e-01\n",
      " -1.51975651e-01  0.00000000e+00  2.07301022e-01 -2.40857623e-01\n",
      " -2.50461174e+01 -0.00000000e+00  1.00549071e-02  8.77270389e-03\n",
      " -2.06384215e-01 -6.35494900e-02 -2.15414553e-01 -2.60414649e-01\n",
      "  1.79661496e-01 -2.01935038e-01  3.92531332e-02 -1.51663915e-01\n",
      "  0.00000000e+00 -2.16477937e-01 -7.65672663e-02 -4.10297890e-01\n",
      " -2.28727830e-01 -3.12019450e-01  2.79421429e-01  1.11127465e-01\n",
      "  2.49163672e-01  3.14052925e-01 -3.57918145e-01 -9.53877152e-01\n",
      " -2.80183007e-01  1.48231714e-01 -1.29691660e-02 -3.13591456e-01]\n",
      "intercept= -0.0703675221130571 coefficients= [-7.48955646e-01  1.03034327e+01 -1.72746900e-01 -1.95741849e+01\n",
      " -6.63218597e-02 -4.46023729e-01  1.49215736e+01  3.58983103e-01\n",
      "  4.58415252e-01  3.41555461e-02  2.68151009e-01 -1.10979577e-01\n",
      " -2.49562519e+01 -5.66030615e-02 -0.00000000e+00  2.33637746e-01\n",
      "  0.00000000e+00 -3.93323675e-02  2.82410083e-01  9.23710876e-02\n",
      "  6.53635692e-02 -6.32701014e-01  1.74720053e-01  2.12685000e-02\n",
      "  1.90774982e-01 -0.00000000e+00 -0.00000000e+00  5.84078818e-01\n",
      " -2.38333018e-01 -2.24349786e-01 -2.50961035e-01  1.99184726e-01\n",
      " -2.10592578e-01  4.72667397e-01 -4.19548367e-01 -6.71976940e-01\n",
      "  2.56944931e-01  1.99300728e-01 -1.79311592e-01 -6.99533332e-01]\n",
      "intercept= 0.11786701901399588 coefficients= [-8.45396907e-01  1.01281961e+01  0.00000000e+00 -1.98733247e+01\n",
      "  5.04872553e-01 -2.73377582e-01  1.48487927e+01  6.07134382e-01\n",
      "  0.00000000e+00 -1.41283098e-01  2.52889813e-01 -3.87516172e-01\n",
      " -2.52064033e+01  2.48606523e-01  1.90504429e-01  4.62277321e-01\n",
      " -2.91355805e-01 -2.75071271e-01 -1.71407015e-01  1.58046065e-01\n",
      "  4.99749115e-01 -3.85094688e-01  0.00000000e+00 -4.94638645e-02\n",
      "  1.73477255e-01  2.32786266e-01  1.99910947e-01  3.27118318e-01\n",
      "  9.62356655e-03 -7.86757993e-02  4.05961904e-02  9.83370828e-02\n",
      " -1.51572967e-02  2.91892012e-01 -3.51848837e-01 -4.62740245e-01\n",
      " -0.00000000e+00  2.07501306e-01 -6.82290077e-02 -7.01776893e-01]\n",
      "intercept= 0.014247551853827978 coefficients= [-8.67923047e-01  1.01143875e+01  3.41868204e-02 -2.03185621e+01\n",
      "  3.13983889e-01 -8.21007099e-02  1.49850143e+01  2.85535104e-01\n",
      " -1.31872205e-01 -0.00000000e+00  3.66275452e-01 -8.36801540e-02\n",
      " -2.51614628e+01  5.69059689e-02  2.81132392e-01  1.53407395e-01\n",
      " -0.00000000e+00  6.78772633e-02  2.95788091e-03  2.84212188e-01\n",
      "  1.39486053e-01 -6.19755797e-01 -9.18791278e-02  7.17947767e-02\n",
      "  1.58061227e-01 -0.00000000e+00  2.37733213e-02  4.52501319e-01\n",
      "  3.81369951e-01 -2.19751074e-01  2.72698597e-01  1.21330809e-02\n",
      " -1.55337284e-01  2.05951883e-01 -2.66557650e-01 -2.81636458e-01\n",
      "  3.90918178e-01  2.56818380e-01  4.54737410e-02 -2.75466055e-01]\n",
      "intercept= -0.050646253782038075 coefficients= [-3.86281409e-01  1.01639700e+01  1.85753854e-01 -1.98962721e+01\n",
      "  2.09163846e-01 -2.00966168e-01  1.50367443e+01  4.08284661e-01\n",
      " -2.55046078e-01 -2.09966571e-01  4.50460314e-01 -2.41941988e-01\n",
      " -2.51545010e+01 -4.53706631e-02  0.00000000e+00  1.11087592e-01\n",
      "  0.00000000e+00  2.17480016e-01  6.42511344e-02 -0.00000000e+00\n",
      "  4.09547576e-01 -4.50861035e-01  1.50407680e-01  1.52000811e-01\n",
      " -6.65274813e-03  1.01715683e-01  1.66868487e-01  2.64970605e-01\n",
      "  3.76214811e-01 -1.63815706e-01  1.28347149e-01  1.11457461e-02\n",
      " -1.65631840e-01  5.65243223e-02  0.00000000e+00 -4.97169938e-01\n",
      "  3.11989841e-01 -3.54545286e-03  1.06587558e-01 -4.01136595e-02]\n",
      "intercept= -0.038709288584889756 coefficients= [-2.73607688e-01  1.02566274e+01 -4.31617103e-02 -1.97233423e+01\n",
      "  2.27944899e-01 -1.71697046e-01  1.51415979e+01  1.19056828e-01\n",
      "  3.67402373e-02 -1.59892140e-01  1.36162264e-01 -1.69351491e-01\n",
      " -2.52569362e+01  8.71673221e-02  3.68796835e-01  2.27240809e-01\n",
      " -9.47112810e-02  1.43220527e-02  2.97252820e-02  1.81219679e-01\n",
      "  2.13472128e-01 -1.15123448e-01  3.65199706e-01  2.31272967e-01\n",
      "  2.85820457e-01  3.27273569e-02  1.96503324e-01  3.39999872e-01\n",
      "  4.39466108e-01 -2.24796253e-01  1.85182238e-01  2.19471852e-02\n",
      " -7.53787153e-02  2.50795754e-01  9.02999651e-02 -5.04706281e-01\n",
      "  2.02907450e-04 -1.02780068e-01  2.35783583e-01  0.00000000e+00]\n",
      "intercept= -0.02873143443462134 coefficients= [-3.38927737e-01  1.03978881e+01 -0.00000000e+00 -2.00183006e+01\n",
      "  1.50782824e-01 -2.40842196e-01  1.51426231e+01  3.58846413e-01\n",
      " -5.35106694e-03 -7.44016251e-02  5.18358928e-02  4.35670288e-02\n",
      " -2.49385307e+01  2.41029571e-01  9.10546971e-02  1.17723119e-01\n",
      " -1.10445513e-01 -1.98499793e-01 -7.11134513e-02 -9.97603104e-02\n",
      "  2.81612357e-01 -3.23326310e-01 -3.43708589e-02  1.26959230e-02\n",
      "  2.88727832e-01  3.41610620e-01 -1.08376591e-01 -7.23021116e-02\n",
      "  7.15517512e-02 -2.10258403e-01  2.50223568e-01  2.88664506e-01\n",
      "  3.66277175e-02  9.29006333e-02  2.91666862e-02 -4.12702015e-01\n",
      "  7.87751870e-02 -3.81393519e-02  1.99642512e-01 -3.33855702e-01]\n",
      "intercept= 0.0795087230903788 coefficients= [-2.91219895e-01  1.04340054e+01  2.46306388e-01 -1.99177681e+01\n",
      "  2.28608199e-01 -1.21836966e-01  1.50042434e+01  1.42187508e-01\n",
      " -3.56312573e-02 -2.48941490e-01  3.79291571e-01  1.10072389e-01\n",
      " -2.49586044e+01  3.81644314e-02  1.95943801e-02  2.63384362e-01\n",
      "  2.89266269e-01  1.77291574e-01 -0.00000000e+00  5.01956295e-02\n",
      "  8.08883104e-02 -3.75039994e-01  2.29918571e-01  1.81447090e-01\n",
      " -5.72746830e-02  2.11633728e-01 -0.00000000e+00  6.07237684e-02\n",
      " -4.62190219e-02 -1.69611817e-01  1.62489366e-02  1.07108728e-01\n",
      " -1.52679444e-01  9.88884181e-02  1.94599111e-02 -4.13881185e-01\n",
      "  3.00827953e-01  1.22167644e-02  1.01188489e-01 -1.77929433e-01]\n",
      "intercept= 0.16386673997436135 coefficients= [-1.04382294e-01  1.01105025e+01 -9.24599760e-02 -1.98504293e+01\n",
      "  3.57288970e-01 -1.35592419e-02  1.51232682e+01  1.39395690e-01\n",
      " -9.61549651e-02  8.55786768e-02  1.49512475e-01  5.54791133e-03\n",
      " -2.50515063e+01  2.17863609e-01  1.20842892e-01  3.22630918e-01\n",
      " -5.49051410e-02  2.09098139e-01 -2.70254099e-03  3.84760525e-03\n",
      "  1.27478025e-01 -1.01952949e-01  2.84514038e-02 -6.77216130e-03\n",
      "  2.50595203e-01  1.73744567e-01 -0.00000000e+00  1.67956713e-01\n",
      "  8.08906027e-03 -2.24650329e-01  3.07721561e-01  1.56548200e-01\n",
      " -0.00000000e+00  6.25664093e-02 -9.56329385e-02 -8.24600380e-02\n",
      "  2.31969632e-02  2.17617420e-01  3.25579997e-02 -2.59593148e-01]\n",
      "intercept= 0.17069700480477157 coefficients= [-2.66349101e-01  1.01483382e+01  0.00000000e+00 -1.97848556e+01\n",
      "  1.20329327e-01 -1.63959707e-01  1.50129106e+01  2.07589028e-01\n",
      " -2.56582839e-01 -1.70187868e-01  4.32551969e-01 -1.28025334e-01\n",
      " -2.51063527e+01  2.31526174e-01  0.00000000e+00  2.78624238e-01\n",
      " -3.77687465e-03  1.53248545e-02  1.41954413e-02  0.00000000e+00\n",
      " -0.00000000e+00 -1.81499203e-01  7.66625858e-02  6.50604262e-02\n",
      "  1.78155783e-01  9.01007305e-02 -0.00000000e+00  0.00000000e+00\n",
      " -6.49665913e-02 -1.59582555e-01  1.99135327e-01  5.06837983e-02\n",
      " -0.00000000e+00 -2.02561569e-02  0.00000000e+00 -1.04292840e-01\n",
      " -0.00000000e+00 -0.00000000e+00  4.16341741e-02 -2.86638103e-01]\n",
      "intercept= 0.14323955308449854 coefficients= [-3.55057654e-01  1.01541920e+01  0.00000000e+00 -1.96759238e+01\n",
      "  0.00000000e+00 -2.77008227e-02  1.49563919e+01  3.33421242e-02\n",
      " -2.77511665e-01 -2.51127991e-01  4.45092606e-01 -1.69209732e-02\n",
      " -2.50769752e+01  0.00000000e+00 -0.00000000e+00  4.44459029e-01\n",
      "  1.53623108e-01  1.92221046e-01 -6.89869333e-02  9.37621178e-02\n",
      " -0.00000000e+00 -1.76446323e-01  1.63080188e-01  1.79997842e-01\n",
      "  1.09611579e-01 -6.67781104e-02  0.00000000e+00  9.76011091e-02\n",
      " -1.57773534e-01 -4.15830215e-02  5.79371117e-02 -1.51682264e-02\n",
      " -3.69786073e-02 -0.00000000e+00 -9.69069936e-02 -8.85399202e-02\n",
      "  1.78324537e-02 -0.00000000e+00  2.13388081e-01 -2.31355218e-01]\n",
      "intercept= 0.200123347087644 coefficients= [-3.42655703e-01  1.01227316e+01 -1.57238162e-02 -1.99587691e+01\n",
      "  7.37350316e-02 -9.13682727e-02  1.50229131e+01  1.59327930e-01\n",
      " -2.52835454e-01 -2.12340146e-01  5.17615884e-01  0.00000000e+00\n",
      " -2.50324987e+01  4.63244967e-02 -0.00000000e+00  3.79858237e-01\n",
      "  1.13017224e-02  1.12433849e-01  0.00000000e+00  5.67153585e-02\n",
      "  4.04738068e-02 -0.00000000e+00  2.39995127e-01  1.62270096e-01\n",
      "  2.68628196e-02 -0.00000000e+00  0.00000000e+00 -6.49255539e-02\n",
      " -4.12499613e-02 -2.17240536e-03  2.60476421e-01  1.10031769e-01\n",
      " -0.00000000e+00  2.01197280e-01 -0.00000000e+00 -1.38565741e-01\n",
      "  6.85231459e-02  5.94503507e-02  3.68658095e-02 -4.06576205e-01]\n",
      "intercept= 0.1438388506435777 coefficients= [-3.69689190e-01  1.01098525e+01 -0.00000000e+00 -2.00710284e+01\n",
      "  1.32661681e-01 -6.62942947e-02  1.53423657e+01  0.00000000e+00\n",
      " -2.48447218e-01 -0.00000000e+00  4.26473930e-01 -5.81762148e-02\n",
      " -2.51122216e+01  0.00000000e+00 -1.40259851e-03  3.81077180e-01\n",
      "  3.47357547e-02  1.15195569e-01 -9.12841642e-02  4.60271799e-03\n",
      "  5.12562207e-02 -0.00000000e+00  1.94109025e-01  1.96995851e-01\n",
      "  1.10883968e-01 -1.15208678e-01  0.00000000e+00 -0.00000000e+00\n",
      "  3.46802602e-02  6.86727751e-02  3.41438527e-01  1.97729092e-01\n",
      " -5.65982152e-02  1.87897718e-01 -1.75578016e-02 -1.53929833e-01\n",
      " -1.65084295e-02  5.14351504e-02  1.69204222e-01 -1.79604741e-01]\n",
      "intercept= 0.22108680607360964 coefficients= [-3.20627049e-01  1.01787045e+01 -1.26770486e-01 -2.02272686e+01\n",
      "  0.00000000e+00  9.48158210e-03  1.51398230e+01  1.40170790e-01\n",
      " -1.90739208e-01 -8.13582632e-02  3.52917352e-01 -7.93606467e-02\n",
      " -2.49392145e+01  7.08883324e-02  1.16838954e-01  3.11762918e-01\n",
      "  1.37073933e-01  1.27167250e-01 -0.00000000e+00  0.00000000e+00\n",
      "  3.80816819e-02  0.00000000e+00  9.81098220e-02  0.00000000e+00\n",
      "  1.78043908e-01 -5.17343530e-02  0.00000000e+00 -2.40558415e-02\n",
      "  5.78376991e-02 -1.01676239e-01  3.55576176e-01  2.05802661e-01\n",
      " -0.00000000e+00  1.75463469e-01 -3.28318858e-02 -1.50807570e-02\n",
      " -3.29859984e-02  3.78649048e-02  7.61234036e-02 -1.60737332e-01]\n",
      "intercept= 0.2568031967208435 coefficients= [-3.65737714e-01  9.99236234e+00 -4.90986915e-02 -2.02404740e+01\n",
      " -0.00000000e+00 -0.00000000e+00  1.50893150e+01 -4.06546689e-03\n",
      " -1.02532462e-01  1.05838859e-01  2.95282950e-01 -1.43786741e-01\n",
      " -2.48383339e+01  6.79579466e-02 -0.00000000e+00  3.11394588e-01\n",
      "  5.48575720e-02  1.07283160e-01 -1.10205322e-01 -1.11787990e-02\n",
      "  5.81231283e-02 -3.13042733e-03  1.21656129e-01  2.11214920e-01\n",
      "  6.54853076e-02 -1.19200951e-02  6.38972548e-02 -1.31082747e-01\n",
      "  0.00000000e+00  6.55232421e-02  2.71403470e-01  1.44811167e-01\n",
      " -3.67432684e-02  4.96963384e-03 -1.02167360e-01 -5.99003683e-02\n",
      "  0.00000000e+00  5.49385920e-02  0.00000000e+00 -2.68713372e-01]\n",
      "intercept= 0.20482597621460674 coefficients= [-3.56047736e-01  1.01990988e+01 -0.00000000e+00 -2.01027197e+01\n",
      "  1.52534568e-01 -1.61359440e-02  1.53528517e+01 -0.00000000e+00\n",
      " -1.09780736e-01 -1.20117971e-02  4.15947721e-01 -1.95577351e-01\n",
      " -2.50333207e+01  6.91704906e-02 -4.11196083e-02  1.40922664e-01\n",
      "  0.00000000e+00  2.56815203e-01 -1.79145448e-02 -9.26503408e-03\n",
      "  4.57128463e-02 -9.59786545e-02  3.97310976e-02  6.78957711e-02\n",
      "  7.18999590e-03 -2.59711307e-02  0.00000000e+00 -3.42981589e-02\n",
      " -3.49268106e-02 -0.00000000e+00  2.58874053e-01  1.00300712e-01\n",
      "  7.27437845e-02  6.30865580e-02 -2.85453094e-01 -7.43369202e-02\n",
      "  8.20267501e-02  1.31461176e-01  0.00000000e+00 -1.49293333e-01]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>rmses_train</th>\n",
       "      <th>rmses_test</th>\n",
       "      <th>R2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>data_100</th>\n",
       "      <td>1.443899</td>\n",
       "      <td>3.833169</td>\n",
       "      <td>0.994914</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_150</th>\n",
       "      <td>1.821530</td>\n",
       "      <td>2.971520</td>\n",
       "      <td>0.992609</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_200</th>\n",
       "      <td>2.185832</td>\n",
       "      <td>2.686198</td>\n",
       "      <td>0.989499</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_250</th>\n",
       "      <td>2.200278</td>\n",
       "      <td>2.971428</td>\n",
       "      <td>0.989677</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_300</th>\n",
       "      <td>2.229557</td>\n",
       "      <td>2.993890</td>\n",
       "      <td>0.989904</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_350</th>\n",
       "      <td>2.331029</td>\n",
       "      <td>2.614199</td>\n",
       "      <td>0.989105</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_400</th>\n",
       "      <td>2.344868</td>\n",
       "      <td>2.690068</td>\n",
       "      <td>0.988926</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_450</th>\n",
       "      <td>2.412296</td>\n",
       "      <td>2.759733</td>\n",
       "      <td>0.987808</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_500</th>\n",
       "      <td>2.516665</td>\n",
       "      <td>2.566364</td>\n",
       "      <td>0.986169</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_550</th>\n",
       "      <td>2.563194</td>\n",
       "      <td>2.534760</td>\n",
       "      <td>0.986546</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_600</th>\n",
       "      <td>2.556348</td>\n",
       "      <td>2.658553</td>\n",
       "      <td>0.986068</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_650</th>\n",
       "      <td>2.560093</td>\n",
       "      <td>2.712406</td>\n",
       "      <td>0.985668</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_700</th>\n",
       "      <td>2.594179</td>\n",
       "      <td>2.562058</td>\n",
       "      <td>0.985260</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_750</th>\n",
       "      <td>2.576048</td>\n",
       "      <td>2.617946</td>\n",
       "      <td>0.985433</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_800</th>\n",
       "      <td>2.613374</td>\n",
       "      <td>2.545214</td>\n",
       "      <td>0.985663</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_850</th>\n",
       "      <td>2.565537</td>\n",
       "      <td>2.681672</td>\n",
       "      <td>0.986063</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_900</th>\n",
       "      <td>2.559286</td>\n",
       "      <td>2.697285</td>\n",
       "      <td>0.986660</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_950</th>\n",
       "      <td>2.547156</td>\n",
       "      <td>2.746880</td>\n",
       "      <td>0.986570</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_1000</th>\n",
       "      <td>2.611110</td>\n",
       "      <td>2.568379</td>\n",
       "      <td>0.985527</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           rmses_train  rmses_test        R2\n",
       "data_100      1.443899    3.833169  0.994914\n",
       "data_150      1.821530    2.971520  0.992609\n",
       "data_200      2.185832    2.686198  0.989499\n",
       "data_250      2.200278    2.971428  0.989677\n",
       "data_300      2.229557    2.993890  0.989904\n",
       "data_350      2.331029    2.614199  0.989105\n",
       "data_400      2.344868    2.690068  0.988926\n",
       "data_450      2.412296    2.759733  0.987808\n",
       "data_500      2.516665    2.566364  0.986169\n",
       "data_550      2.563194    2.534760  0.986546\n",
       "data_600      2.556348    2.658553  0.986068\n",
       "data_650      2.560093    2.712406  0.985668\n",
       "data_700      2.594179    2.562058  0.985260\n",
       "data_750      2.576048    2.617946  0.985433\n",
       "data_800      2.613374    2.545214  0.985663\n",
       "data_850      2.565537    2.681672  0.986063\n",
       "data_900      2.559286    2.697285  0.986660\n",
       "data_950      2.547156    2.746880  0.986570\n",
       "data_1000     2.611110    2.568379  0.985527"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-0.02873143443462134\n",
      "+  -0.33892773723638 * X0\n",
      "+  10.397888105692777 * X1\n",
      "+  -0.0 * X2\n",
      "+  -20.018300636265867 * X3\n",
      "+  0.15078282427754805 * X4\n",
      "+  -0.24084219639080937 * X5\n",
      "+  15.142623118157553 * X6\n",
      "+  0.358846413291785 * X7\n",
      "+  -0.0053510669401581334 * X8\n",
      "+  -0.0744016250663531 * X9\n",
      "+  0.05183589281730809 * X10\n",
      "+  0.043567028770406684 * X11\n",
      "+  -24.938530739437162 * X12\n",
      "+  0.2410295713030524 * X13\n",
      "+  0.09105469714735046 * X14\n",
      "+  0.11772311856770253 * X15\n",
      "+  -0.1104455130426506 * X16\n",
      "+  -0.1984997926007063 * X17\n",
      "+  -0.0711134513480476 * X18\n",
      "+  -0.09976031037534597 * X19\n",
      "+  0.28161235743360635 * X20\n",
      "+  -0.3233263101167149 * X21\n",
      "+  -0.03437085892405313 * X22\n",
      "+  0.012695922953665869 * X23\n",
      "+  0.28872783162458443 * X24\n",
      "+  0.3416106201610987 * X25\n",
      "+  -0.10837659076622282 * X26\n",
      "+  -0.0723021115885287 * X27\n",
      "+  0.07155175124979446 * X28\n",
      "+  -0.21025840325667036 * X29\n",
      "+  0.2502235684110799 * X30\n",
      "+  0.28866450604385663 * X31\n",
      "+  0.036627717470929785 * X32\n",
      "+  0.09290063331072011 * X33\n",
      "+  0.02916668622855717 * X34\n",
      "+  -0.41270201481051555 * X35\n",
      "+  0.07877518696322021 * X36\n",
      "+  -0.03813935186640797 * X37\n",
      "+  0.19964251211488768 * X38\n",
      "+  -0.33385570186895497 * X39\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD5CAYAAAAOXX+6AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAsJUlEQVR4nO3deXzU1b3/8deZyZ5MEkIgJOw7QoCwQ4FKXEEWV1xRpCq2vVZ6KXW53ltrr/qrlVrLlbbaqli3uNRdVIqCICr7vgnIFkgk+75MZs7vjzMJAbIzk/lO8nnqPDLz3eYzJ1/e+c6Z7/eM0lojhBDCumz+LkAIIUTDJKiFEMLiJKiFEMLiJKiFEMLiJKiFEMLignyx0fj4eN2rV68WrVtSUkJkZKR3C/IBqdP7AqVWqdO7AqVO8G2tmzdvztZad6pzptba67dRo0bpllq1alWL121NUqf3BUqtUqd3BUqdWvu2VmCTridTpetDCCEsToJaCCEsToJaCCEszicfJgohAoPT6SQ9PZ3y8nK/1RATE8PevXv99vzN4Y1aw8LC6NatG8HBwU1eR4JaiHYsPT0dh8NBr169UEr5pYaioiIcDodfnru5zrdWrTU5OTmkp6fTu3fvJq8nXR9CtGPl5eV07NjRbyHd3iil6NixY7PfwUhQC9HOSUi3rpa0twS1EEJYnAS1EEJYnAS1EEJ4PP3005SWljZ7vd/85jesXLnSBxUZEtRCCMvQWuN2u/32/A0Ftcvlqne93/3ud1xyySW+KktOzxNCGI98uJs9Jwu9us3BSdE8PHNIg8scPXqU2bNnk5qayuuvv05sbCwXX3wx3377LcOHD2fevHk8/PDDnDp1ildffZWxY8fy5ZdfsmDBAsB8OLdmzRocDgdPPvkkb775JhUVFVx99dU88sgjlJSUcP3115Oeno7L5eJ//ud/uOGGG86pY8mSJZw8eZLU1FTi4+NZtWoVUVFRLFy4kM8++4w//vGPfPLJJ6xYsYKysjJ+9KMf8eyzz6KU4vbbb2fGjBlcd9119OrVi7lz5/Lhhx/idDp56623GDRo0Hm1oxxRCyH8bv/+/dx2221s3bqV48ePs2DBAnbs2MG+fft47bXX+Oqrr1i8eDGPP/44AIsXL2bp0qVs27aNtWvXEh4ezooVKzhw4AAbNmxg27ZtbN68mTVr1vDpp5+SlJTE9u3b2bVrF1OnTq2zhnvvvZekpCRWrVrFqlWrADNaXnJyMuvXr2fSpEnMnz+fjRs3smvXLsrKyvjoo4/q3FZ8fDxbtmzhZz/7GYsXLz7v9pEjaiEEQKNHvr7Us2dPxo8fz5EjR+jduzdDhw4FYMiQIVx88cUopRg6dChHjhwBYOLEiSxcuJBbbrmFa665hm7durFixQpWrFjBiBEjACguLubAgQNMnjyZRYsWcf/99zNjxgwmT57c5LrsdjvXXnttzeO1a9dy/fXXU1paSm5uLkOGDGHmzJnnrHfNNdcAMGrUKN55552WNksNCWohhN/VHuM5NDS05r7NZqt5bLPZqKqqAuCBBx5g+vTpLF++nPHjx7Ny5Uq01jz44IPcfffd52x/8+bNLF++nAcffJDLLruM3/zmN02qKywsDLvdDpiLgxYuXMjmzZvp3r07v/3tb+u9cKW6ZrvdXlPz+ZCuDyFEwDl06BBDhw7l/vvvZ/To0ezbt4/LL7+cF154geLiYgBOnDjBqVOnOHnyJBEREcyZM4dFixaxZcuWerfrcDgoKiqqc151KMfHx1NcXMzbb7/t/RdWDzmiFkIEnKeffppVq1Zht9sZPHgw06ZNIzQ0lL179zJhwgQAoqKieOWVVzh48CC//vWvsdlsBAcH89e//rXe7c6fP59p06aRmJhY009dLTY2lrlz5zJ06FB69erFmDFjfPoaz1DfNwqcz02+4cU6AqVOrQOn1rZU5549e3xfSCMKCwv9XUKTeavWutod+YYXIYQIXNL1IYRod66++moOHz58xrQnnniCyy+/3E8VNUyCWgjR7rz77rv+LqFZpOtDCCEsToJaCCEsToJaCCEsToJaCCE8WjrMKcB7773Hnj17vFyRIUEthLAMbeFhThsjQS2EaLOOHj3KBRdcwM9//nPi4uLo27cvd955J8nJydxyyy2sXLmSiRMn0r9/fzZs2ADAl19+SUpKCikpKYwYMaLmsu8nn3ySMWPGMGzYMB5++GHAjIA3ffp0hg8fTnJyMm+88UadddQe5jQ1NRWAFStWMGHCBEaOHMns2bNrLk9/4IEHGDx4MMOGDWPRokV8/fXXfPDBB/z6178mJSWFQ4cOebWN5PQ8IYTxyQOQudO72+wyFKb9vtHF9u/fz4svvsh9991Hv379WLBgAc899xxjxoypGeb0gw8+4PHHH+e9996rGeZ04sSJFBcXExYWdsYwp1prZs2axZo1a8jKyiIpKYmPP/4YgIKCgjpruPfee3nqqadYtWoV8fHxZGdn8+ijj7Jy5UoiIyN54okneOaZZ/jVr37Fu+++y759+1BKkZ+fT2xsLLNmzaoZk9rb5IhaCOF31cOcAjXDnNpstkaHOV2yZAn5+fkEBQWdMczpyJEj2bdvHwcOHGDo0KGsXLmS+++/n7Vr1xITE9Okmr799lv27NnDxIkTSUlJ4aWXXuL48eNER0cTFhbGnXfeyTvvvENERISvmqWGHFELIYwmHPn6ihWHOdVac+mll/L666/XTCsqKiIoKIgNGzbw+eefk5aWxjPPPMMXX3zR4tfeFHJELYQIOK0xzOn48eNZt24dBw8eBKC0tJQDBw5QXFxMQUEBV1xxBU8//TTbtm07Z11vkyNqIUTAaa1hTpctW8ZNN91ERUUFAA899BCJiYlceeWVlJeXo7XmT3/6EwA33ngjd911F0uWLOHtt9+mb9++3nvB9Q2rdz43GebUOgKlTq0Dp9a2VKcMc9o8lh/mVCllV0ptVUrV/W2OQgghfKI5XR8LgL1AtI9qEUKIVtEmhzlVSnUDpgOPAQt9WpEQQvhYoA1zqkzXSCMLKfU28P8AB7BIaz2jjmXmA/MBEhISRqWlpbWooOLiYqKiolq0bmuSOr0vUGptS3XGxMTQr1+/Vqqobi6Xq+abvq3OW7UePHjwnAtvUlNTN2utR9e5Qn2d19U3YAbwF8/9KcBHja0jHyZaR6DUqXXg1NqW6pQPE5vHyh8mTgRmKaWOAGnARUqpV1r4h0QIIUQzNRrUWusHtdbdtNa9gBuBL7TWc3xemRBCCECuTBRC+FlsbCwpKSkkJyczc+ZM8vPzAdi2bRsTJkxgyJAhDBs2rN5R79qDZgW11nq1ruODRCGEaKnw8HC2bdvGrl27iIuLY+nSpQBERETwz3/+k927d/Ppp5/yy1/+sibE2xvrHFFXVcKWfxJdsN/flQgh/GTChAmcOHECgAEDBtC/f38AkpKS6Ny5M1lZWf4sz2+sM9aHdsPKR+gR3hs4d/QrIYRvPbHhCfbl7vPqNgfFDeL+sfc3aVmXy8Xnn3/OHXfccc68DRs2UFlZ6d3xMwKIdY6og8NgzB10zNkEOd79dgQhhHWVlZWRkpJCx44dyc3N5dJLLz1jfkZGBrfeeisvvvgiNpt1Iqs1WeeIGmD0Heg1f0Stfxau+IO/qxGiXWnqka+3VfdRFxQUMGPGDJYuXcq9994LQGFhIdOnT+fRRx+t+WKB9shaf54cCZzqPBm2vgJl+f6uRgjRimJiYliyZAmLFy/G6XRSWVnJ1VdfzW233cbs2bP9XZ5fWSuogfRuM8FZYsJaCNGujBgxguHDh5OWlsabb77JmjVrWLZsWc0X2VYP0t/eWKvrAyh29IWeE2H9szDup2C3XIlCCC/KyMg44/GHH35Yc3/OHLm2Dix4RA3A+J9BwTHYv9zflQghhN9ZM6gHXgGxPeDb+r8yRwgh2gtrBrXNbro9jn0NJ7f6uxoh2jTdhKGOhfe0pL2tGdQAI+ZASBR8+zd/VyJEmxUWFkZOTo6EdSvRWpOTk0NYWFiz1rPuJ3VhMSasNz4Plz4Cji7+rkiINqdbt26kp6f79dLs8vLyZgeXv3ij1rCwMLp169asdawb1ABj55uzPzY+Dxc95O9qhGhzgoOD6d27t19rWL16NSNGjPBrDU3lr1qt2/UB0LEvDJwGm54HZ7m/qxFCCL+wdlCDOVWvNAd2vuXvSoQQwi+sH9S9JkNCsjlVTz7wEEK0Q9YPaqXMUfWp3XB4jb+rEUKIVmf9oAZIvg4i4uUCGCFEuxQYQe0Zq5rvPpWxqoUQ7U5gBDXA6DvAFmRO1xNCiHYkcILakQBDr5OxqoUQ7U7gBDWY8T9krGohRDsTWEGdlHJ6rGpXlb+rEUKIVhFYQQ0yVrUQot0JvKCWsaqFEO1M4AW1jFUthGhnAi+oQcaqFkK0K4EZ1NVjVe/6FxRl+rsaIYTwqcAMajBjVburzFjVQgjRhgVuUMtY1UKIdiJwgxpkrGohRLsQ2EEtY1ULIdqBwA5qGataCNEOBHZQg4xVLYRo8wI/qGWsaiFEG9doUCulwpRSG5RS25VSu5VSj7RGYc1SM1a1XAAjhGh7mnJEXQFcpLUeDqQAU5VS431aVXPVjFX9qoxVLYRocxoNam0Uex4Ge27WO8WiZqzql/1diRBCeFVQUxZSStmBzUA/YKnWer1Pq2qJmrGqn4NxPwN7k15a69AaqsqhvADKC6Gi0HO/wHO/9rTC09PswTD9j+biHiFEu6V0M84/VkrFAu8Cv9Ba7zpr3nxgPkBCQsKotLS0FhVUXFxMVFRUi9aNz/qG5N2/52DfeaR3u9KcvucjDdUZk7+H3odfJqQyj6CqEoKqSrHphr/oQKOoCorAZY+kKiiCqqAIIkuO47aFsC3lMcoiEr1ep9UESq1Sp3cFSp3g21pTU1M3a61H1zWvWUENoJR6GCjRWi+ub5nRo0frTZs2Na9Kj9WrVzNlypQWrYvbBa/OhkOfw/CbzNFoSGTLttWIOut0u+HrJfD57yC6K3QfC2HREBptBpIKi4ZQz8+wGM90z/yQKLCd1RP1w254aSbYQ+H2j1p0ZH1e7dnKAqVWqdO7AqVO8G2tSql6g7rR/gGlVCfAqbXOV0qFA5cAT3i5Ru+w2eGWt2DNk7D692a86uv/CZ0G+v65S3PhvZ+Z0wQHXwmz/s+E8flIGAK3fWDC+qWZJqzj+ninXiFEwGjKWR+JwCql1A5gI/BvrfVHvi3rPNjsMOUBuPVdKMmG56bA9jd8+5zpm+DZH8PBz2HaH2D2S+cf0tW6JMPcD8BZCstmQu5h72xXCBEwmnLWxw6t9Qit9TCtdbLW+netUdh565sKP/0KkkbAu/Phg3vBWebd59DaXBH5wlRAwR2fwbi7vd833mWoObJ2lpgj67wj3t2+EMLSAv/KxIZEJ5qAm7QQtrwE/7jUa1cv2qtK4M1b4dMHoP+l8NM10HWUV7Zdp8RhcNv7UFFkjqzzjvruuYQQltK2gxrMaXqXPAw3vwWF6fDshbDrnfPb5sltjN60EPYth8sehRtfg/AO3qm3IYnDPWFdAC/NgPxjvn9OIYTftf2grjbgMrh7LXQeBG/Pg+W/hqqK5m1Da/ONMs9fitJOmPcJ/OgXPj0N8BxJKSasywtg2QzIP956zy2E8Iv2E9QAsd3h9uUw4R7Y8By8cHnT+3sriuBfd8LHC6H3j9k86mnoMc6X1dYvaYT5sLQs3xxZF6T7pw4hRKtoX0ENEBQClz8GN7wKOd+bszX2fdzwOj/sNmeP7H4HLv4N3PwWzpDoVim3Xl1HmbAuzTVH1gUn/FtPfaoqIGM7/LDH35UIEbAsdJ11K7tghjlP+a3bIe1mc5R9yW/NZdu1bX0FPl5kLky57QPoPdkf1datmyesX74alk2HecshOsl/9RRnwQ87IXMX/LALMndC9nfmS4gB+l8Oqf9lum+E8Be3G4ozzbvpvCPmg/m8I6YLc8xd5t+VxbTfoAaI6w13rIDPHoJvnoHjG2D2ixDTDSpLTEBvfw16XwjX/gOiOvu74nN1Gw1z3vGE9QxzUYyvw9rtgpyDJogzd3pCeZfZ+atFdzVfkzZwmvmZdxjWLYHnLoRBM0xgJwzxbZ1W5HJC4QlwJEJQqL+rabvKC0345h89N5Dzj4Gr9udTyuyvFUWw/XXo/WNzplifKa37+VMD2ndQg/nHMn0x9JwAHyyAv002Z4l8+zfI2gcXPgAX3mcupLGq7mPgVk9YvzQT5n5kTk30hooiE8KZO08fLZ/aYwaZArAFQ6dB5rz1LkNNKHcZChFx525rzJ3mvPNvlprupuRrYMqDEN/fO7VahctpwiDnEOR+D7menzmHzHTtgvA4SLkZRs6FTgP8XbH1uF3mYMlZZq4fcJZBZWmt+9XzSs398oJaoXwUynLP3F5oDMT1gs4XmIOHDr2gQ0/o0NscmAWFmn1904tm/3z5KkgaCZMXwsDp5w7v0MokqKslXwtdhsNbc+HDBebrvW59B/pe5O/Kmqb7WJjzL3jl2tOXmzu6NG8bFcWQuQNObjOX32dsg+wD1IxqGx5nrpQcc6cnkJMhfqDp92+KsBhz1ejY+fD1/8H6Z2H3uzDsRvPHMJDUFcbV96vDuFqIAzr2MR8CJ19rguHwl+b1f/OMGfVx1O1wwSzzjUXtRfZB2PEGI7e+Czv1mQHsauYZWbYgiO1hAjgxxRPEvU4HclNOnw11wMR7zf65/XVY92d4Y47Zxyf9EobObu4r9BoJ6tri+8GdK2Hba+avrj/7e1uix3i45e3TYT23gSv9K0vMUfLJraeDOfs7akLZkegJluvM+duJw8w0b7wVjIgz71rG/xzWPQ0b/wE732RAwkWQ0tecnWM15YVwYAXs/YBx36+HL7MaDuOOfc24LHF9ITL+3HYbPc/06W9/DTYvg3fugvD7zGBiI+ea00jbopJs2PUv2PEGnNgMyoY7ehB0GQjBkRAcDiEREOy51b5/9uMz5oV7r5siOMz8fkbcCnveg6/+ZMbxWfU4XTtNA+c483ytqNmj5zWF30bPa0WWrvPo1/DKdRDTlXUDHmLilMtMP/LJraeDOXs/aLdZPirBBEzSCHM0kpTS/KPx81GYAV89hXvjC9hsNnN0OflXrVtDXcryzSBbe94347i4KiCqC6fC+9J50ITGw7ip3G44stYE9t4Pwe2EHhNMOwy+ssWhYJl91FkG+5fDjjfh4Erz4XKXoTDsBki+jtVb9lujzvpobf5Ir30Kjn9r3m2P/5l5Zxke67WnOa/R80QA6vkjuOVNeHU249b/HL4pOx3KkZ1NIA+edTqYvdWf3VLRiXDFk6y3jWVC5VrY9AJs+af5hzDpP00ItpbSXBMqe96HQ6tMaEZ3NV+gPPhK6DaWPWvW0NmbwWKzQZ8Lza0k27yj27wM3r0bPrnfHGWPmmv6V1uiotjTNXPQdM/kHDS3/OPm3UuXYaffNXUeck73S0lFFaeKKvihsJxTRRWcqvUzv8xJp6hQusdF0K1DeM3PhKgQbMfWmQHR9rwPlUXgSIIJ/2G6uhIG13qG/S1uulahFAy4HAZcztb3ljKi+Av44n9h3Z+pGvUTDvS+lb1FYez/oYjyShePXJns9RIkqNuqXpNgzjvkfPwYCYPG1wrlJMt8kn22irBOMHWJCecv/wDf/sV8uDPubnMFaF0fUHpDSTbs+8gEyuE15ogvtgeM/ykMvsp8qNRaHyZFxlMy+udkDfgJFQfXEL3nZTpv+Af29X/lSMRQvoi8ghWMJ7NUEWS3ERliJyIkiOgQTQ91im76JElVJ0hwptM9/ztKv8kiouLUGU/hjErC1aEv9L4YnXuE4B1vE7T5RQDc2MkI6clBex92unqxobwrWyq7U0zEGdsICbKREB1KbHgIezMK+aHQ9Cn3V+lcY1/LVfZ1JKpcylQEu2MvJH3QLFTvSXSLc9A9IpxOWqMsuh/WpaLKxeHsEj4rHcjnnS6htOw6JmW+zJR1f6b3uqVsdE1hhZ5JdGJftA9emwR1W9ZzAnsHLyLBym8r6xLXG67+qyewf2/6CDf+wxzROrpAZCeI6Gh+Rsabn+FxdX79WkGZk/S8Uk7klZGeV0aZ04XdpnA4c+mTs4peP/ybhNxN2HBTHNmdzH7zyOoxldK4ZIKC7ASXK+xH8giy2wi2K+w2xbFCF7tOFODWGpdb49Yat+b0fTdmntZorXF5HrvdZprLrSksc5JVXElWUQXZxbVuRZWUOWv1fTOHOGZxjX0tc0pX8ZPSJ7hBRbE5+mKqVDBxxcdIcKbT2ZWJHXfNWjnawWGdyBb3IA7rVL7XiRzRXTiiEygvD4Xs2q2k6a5OkayOMDzoKCmuYwyv2syF7pXmkrgwKI7sQVnHZEgcRkSPkUT0HIGqPl21KBPn9jdxb0sjNHs3bmXn+5hxvBh2McurRnA43032hkrYsLPmGUODbHTrEE63DhFQWsHqwt3YbYogm2njILvt9H3Pz2C77axlFEE2s1yQ3UZokOcWbCcs2EZokL1mWliwuR9kb/gPrtPl5mhOCd/9UMz+zCIOnCpif2YRR3JKcblNN7Hddoje8Ulk9vlfjkXncnFOGrcee59b9ReopNlQNcrrfdgS1MK6Og2A614w/dWrfw/7PzGnXWn3OYtqFBVB0RTZY8kjmh/cDk46I8mscpCjHeTqaHKIpr9K5wr7BsaqfdiU5pA7kWfcs/jENY695T0gR8GOCsxXhDbg66/O++UpBXERIcRHhRLvCGFkjw50igol3hFqpkWZeZ0cocRF3kCwTcHRdURuXsaP97xvTo3s2Ac6joOO/U7f4voQG9aBrC9Wc+HYCYyqqKK00kVJRRUllVWUVLgorayirNJFbEQInR2hdI4Oo3N0KI7QoNNHg0WZkLEDMrYTlbmdqIwdcGw5VH9jqiMJYrrCic0Ea7d55zH1CWzJ19IvqhP9gHmeRUsrqziRV8bxvFLS88o4nuv5mVfKsSwX23JO4HJrnC43Lremyu2b78+229QZwW1uJtjLnW6+zy7G6dI1v5+ecREMSHAwLTmR/glRFB7fz/XTLiQ0qPbpupdCwe/MaX1ZeyHI+2fuSFALyyuPG8TGkU+xp0shJ3KLycs5RVl+JpWFWURV5RGniohXBcRVFZFgL6RLUAl9bemMDCokggIUZ/6jd8cPonLgIsr7zySqwwCu1XCly02VW1Plqh0WbpwufUaAOF2a3bt3MTQ5GbtNYVMKm01hU2Cvua+w20ApZaYphc2GZ7pZNjo8mLiIkEaP8M7Ra5K5XfkXcxVtPW+x7UB4kDIB3MJ2x9HF3AZcdnpaWZ45W8gT4OQdMReHDLuhwfPBI0KC6J/goH+C45x5dX3oqT3vUqrcbqpcJrirfycuz+/JTDO/N2eVptLlotzppqLKRYXTTbnnZ0WVmdbYvGC7YsqgTgxMcDAgwUHfTlGEh5x5/cTq/ANnhbRHTFeY+rj54NEHXToS1MJytNYcPFXMmgPZrPkui/WHcyh3mqPo6LAgunWIoGvnC+g7cCRdY83bZ/M2OpyY8OAz+wfdLvMBYWk2lGRBVBdsnQYQBrT0uCcsex9Thvj5jJSmnrvubeEdzJV7vX/s06cxf+TAbrMTGkgp5aN+90BqAtGGFZQ62ZBZxSdv72DtgSxOFpgrH/t0iuTGMT24cEAnRvbsQEx4cCNbOovNDlGdzI0WnjUhhJ9JUAu/qHK52Z5ewJrvslhzIIvtx/Nxa3CEZTCpXzy/uLgTk/vHmw+bhGjnJKhFqzmRX2aC+bss1h3MprC8CpuCYd1iueei/jiKjzNvVmrz+22FaOMkqIVP5JVUsjezkP2ZRezLKGLT0VwOZZUAkBgTxrTkRH48oBMT+3UkNsL0t65efVJCWog6SFCL81JZ5eZQVjH7MgvZl1HEvswi9mWevgACoGNkCMldY7hprOlr7tc5KqAudhDC3ySoRZNorfmhsIK9NYFsjpYPniquOec1xG6jf0IUE/vFc0GXaAYlOhjUJZpODhl3WYjzIUHdhmmtKa7UnMgvo9zpoqzSVXPOaFmli/Iql+enmwqnq2ZaudNNmdNFueeWXVzJ/swiCsqcNdvuGhvOoC4OLhrUmUGJ0VzQxUGv+EiCpetCCK+ToG5j8koqWXcom68OZLP2QDYn8svgiy+avL7dpgj3XIIbFmwnLNhOTHgw04clckEXBwO7RDOwi6P5p8kJIVpMgjrAlTtdbD6ax1cHTTjvOlmA1uAIDWJC345MSnAxMnlgTeiGBdvPCOLwYDuhwTbPNLscEQthQRLUAcbt1uzLLOKrg1msPZDNxiO5lDvdBNkUI3t04D8vGcCk/vEM6xpDkN1mLs8d08PfZQshzoMEdQDIKCjjqwPZfHUwm3UHs8kurgSgX+cobhzTg8n94xnXpyNRAXWtrRCiqeRftp9prSksryKvpJLc0krzs6SSvNJK0vPKWHcwu+b84/ioECb2i2dSv3gm9Y8nMaZ1vw5ICOEfEtReUOWqPkvCbc6ucLoorXSRd1bw5pY4yfPcr36cX1pZ75COYcE2xvSK44Yx3ZnUrxODujiw2eT8YyHaGwnqs5RVuvj72u/JLCyvOT2trNKEb5nTTbnnFLaC4jJcqz+j3OmqGb+2IXabokNEMB0iQugQGUKf+ChG9QwhLtIzLSKEuEgzLy4ihA6RwUTVHhtYCNFuSVDXUlJRxR0vbWT94Vw6RoYSHnL6bIiwYDux4cGER4cRHmInN6uSPj27Ee45cyI8xE5o9f1gO+EhNmIjqkM3BEdokBwNCyFaRILao7DcybwXN7LteD5P35DClSldG1zeDHY+pJWqE0K0ZxLUQH5pJbe9sIG9GYU8c9MIpg3187dyCyFELe0+qHOKK5jz/AYOnSrmb3NGcfEFCf4uSQghztCug/pUYTk3/2M96XmlPH/7aCb37+TvkoQQ4hyNXi+slOqulFqllNqrlNqtlFrQGoX52sn8Mq5/9hsy8stYNm+shLQQwrKackRdBfxKa71FKeUANiul/q213uPj2nzmeG4pN/39WwpKnfzzjnGM6tnB3yUJIUS9Gg1qrXUGkOG5X6SU2gt0BQIyqL/PKubmv6+nvMrFa3eNZ2i3GH+XJIQQDWpWH7VSqhcwAljvk2p87Lsfirj57+vRWvP6XeO5IDHa3yUJIUSjlNaNX1UHoJSKAr4EHtNav1PH/PnAfICEhIRRaWlpLSqouLiYqKioFq3bkKOFLhZvLMduU9w3JoykqPMbztNXdXpboNQJgVOr1OldgVIn+LbW1NTUzVrr0XXO1Fo3egOCgc+AhU1ZftSoUbqlVq1a1eJ167PtWJ4e+vCnesLjK/XhrGKvbNMXdfpCoNSpdeDUKnV6V6DUqbVvawU26XoytdGuD2UGm3ge2Ku1fsp7fz9ax6Yjudz+4kbiIkN49c5xdI+L8HdJQgjRLE15/z8RuBW4SCm1zXO7wsd1ecXXh7K57YUNdHaE8ubdEySkhRABqSlnfXwFBNxoQqv3n+LulzfTs2MEr9w5js6OMH+XJIQQLdImr0xcsTuTe17bSv+EKF6+YxxxkSH+LkkIIVqszX2T6cc7Mvj5q1sYnBTNa3eOl5AWQgS8NhXUH24/yS9e38KIHrG8fMdYYiKC/V2SEEKctzbT9fHh9pMsSNvK6F5xLJs3hoiQNvPShBDtXJs4opaQFkK0ZQEf1B9uP8kv39gmIS2EaLMCOqg/2mFCelTPDrx4u4S0EKJtCtig/mjHSRakbWNUDxPSkaES0kKItikgg/rjHRmnQ3qehLQQom0LuKD+eEcG96ZtlZAWQrQbARXU1SE9skeshLQQot0ImKBevvN0SC+bN1ZCWgjRbgREUC/fmcEvXq8+kpaQFkK0L5YP6k88IT2iuwnpKAlpIUQ7Y+mg/mRnBvd4QnrZTySkhRDtk2WDWkJaCCEMSwZ17e4OCWkhRHtnuQTclFnF31ZsZbiEtBBCABY7ov50VwZ/3V7B8O6xvCQhLYQQgIWCOq+kkkVv7aB3jI1l88ZISAshhIdl0rBDZAjL5o0h6+B2HGHyzSxCCFHNMkfUAKN7xREeFHBfeC6EED5lqaAWQghxLglqIYSwOAlqIYSwOAlqIYSwOAlqIYSwOAlqIYSwOAlqIYSwOAlqIYSwOAlqIYSwOAlqIYSwOAlqIYSwOAlqIYSwOAlqIYSwOAlqIYSwuEaDWin1glLqlFJqV2sUJIQQ4kxNOaJeBkz1cR1CCCHq0eg3vGit1yilerVCLRwvPE6WM4vjRcexKRsKhU2ZvyXVj5VSNdNrHiuFDdsZ82zKPLYre81yQggRiJTWuvGFTFB/pLVObmCZ+cB8gISEhFFpaWnNLmbhsYU4tbPZ6zWFOvu/6lCvHfDYmhzqbre7ReGvaby9G6I4/ZxnP3/1vNrLaK1r/tg1d/vnzFNnbv+MWs66X7u2s5ev/h2Y/09Pc7vdBNmC6n2us2s7uy292bZ1Pa7mdrmx2+11b6OBfaK+9mqsjqbOO1uVq6reOhvSnHasvWxD6zWUM/XVWb29+rZ7vr/vupyzH5/12OVyYbfbz9m/q5eLsEUwr9O8Fj13amrqZq316Lrmee07E7XWzwHPAYwePVpPmTKl2dt47PBj7Ny9k0EXDMKt3Wit0WhzH20ea42b0/O0PnN+7Xkut6vmsVu7a5ar3rZLu86YVnt6YyGccTKDpKSkBpep7x9VS4/uq3d2zys/Y1rNMmdNz8jIoEtilzq3c872m/APrfq5az+uXq3OebXWr/27NP+fXl6jycnJoUNch9O/2zqWq++PU32PG8w1Xftu/aFfu700moL8AqJjo8+Zd+7m699Gfcs1WNNZ22gstMuLyomJimlwmXq3UcfkevfnJhw81LmeZ9mC/AJiY2Pr3EZdBwVn3vXeO+W69mU4ve8C5Ofn44hxnF6m1jyNJjIkkpZkX2Ms8+W2ANN6TyP8aDhT+k7xdymNWr16NVMmTPF3GY1avXo1UyZO8XcZTbJ69Wqf7OTeJnV6V6DUCf6rVU7PE0IIi2vK6XmvA98AA5VS6UqpO3xflhBCiGpNOevjptYoRAghRN2k60MIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISxOgloIISyuSUGtlJqqlNqvlDqolHrA10UJIYQ4rdGgVkrZgaXANGAwcJNSarCvCxNCCGE05Yh6LHBQa/291roSSAOu9G1ZQgghqimtdcMLKHUdMFVrfafn8a3AOK31PWctNx+YD5CQkDAqLS2tRQUVFxcTFRXVonVbk9TpfYFSq9TpXYFSJ/i21tTU1M1a69F1zQtqwvqqjmnnpLvW+jngOQClVFZqaurRZlV5WjyQ3cJ1W5PU6X2BUqvU6V2BUif4ttae9c1oSlCnA91rPe4GnGxoBa11p6bVdS6l1Kb6/qpYidTpfYFSq9TpXYFSJ/iv1qb0UW8E+iuleiulQoAbgQ98W5YQQohqjR5Ra62rlFL3AJ8BduAFrfVun1cmhBACaFrXB1rr5cByH9dS7blWep7zJXV6X6DUKnV6V6DUCX6qtdGzPoQQQviXXEIuhBAWJ0EthBAWJ0EthBAW55WgVkr9Vim1qIH5V7V0fBCl1I+VUluUUlWeqyRrz5urlDrguc2tNb23Umq9Z/obntMKA4qP2/SnSqmdSqltSqmvam9HKeXyTN+mlPqg1vSAblNftqdn/euVUnuUUruVUq/Vmi77aMu23UMptUoptVUptUMpdUWteW1yH21Iax1RX4UZ0KkljgG3A6/VnqiUigMeBsZhxiN5WCnVwTP7CeBPWuv+QB5wRwuf28quouVt+prWeqjWOgX4A/BUrXllWusUz21WreltvU2vooXtqZTqDzwITNRaDwF+6Zku+2jL99H/Bt7UWo/AXLvxl1rz2t0+2uKgVko95Bn6dCUw0DPtLqXURqXUdqXUv5RSEUqpHwGzgCc9fwH71rVcfc+jtT6itd4BuM+adTnwb611rtY6D/g3MFUppYCLgLc9y72E2WEsrxXbtLDWw0jqGBLgrLoCsk1bqz2Bu4Clnv0QrfUpz3TZR1vephqI9tyPoZGroQO5TZtEa93sGzAK2AlEYBrzILAI6FhrmUeBX3juLwOuqzWvzuUaec6zt7EI+O9aj//HMy0eM9pf9fTuwK6WvM7WvLV2mwL/ARwCjgP9a02vAjYB3wJXeaYFXJu2ZnsC72HemazztNtU2UfPu00TPc+Vjjk6HtXW9tHm3Jp0wUsdJgPvaq1LAWr1EyUrpR4FYoEozNWMdWnqcg2pb7CoJg0iZUGt2qZa66XAUqXUzZi3mdX9pz201ieVUn2AL5RSO4HCujbR1BfmJ63ZnkFAf2AKZiyctUqpZGQfPVtz2vQmYJnW+o9KqQnAy0qpZK21m7azjzbZ+fRR19UIy4B7tNZDgUeAsHrWbepyDalvsKhsIFYpFXTW9EDgjzZNo9ZbRK31Sc/P74HVwAgCt01bqz3Tgfe11k6t9WFgPya4ZR9t2XJg+pffBNBaf+NZNt7zuC3to03S0qBeA1ytlApXSjmAmZ7pDiBDKRUM3FJr+SLPPBpZrjk+Ay5TSnXwfEBzGfCZNu97VgHVZ4jMBd5v4XO0plZrU8+HX9WmAwc80zsopUI99+OBicCeAG3T1txH3wNSoabdBgDfI/vo+bTpMeBiAKXUBZigzmpj+2iTtfgScqXUQ8BtwFHMkcMeoAS4zzNtJ+DQWt+ulJoI/B2owDTkZXUtV8/zjAHeBToA5UCmNp+so5T6CfBfnkUf01q/6JneB3OkGAdsBeZorSta9EJbUSu26Z+BSwAnpv/vHq31bs8HQM9iPri1AU9rrZ/3rBNwbdqK7amAPwJTARdmX0zzzJN9tGVtOtizbhTmKP4+rfWKtraPNpWM9SGEEBYnVyYKIYTFtfSsD6/zvKWafdbkt7TWj/mjnrZA2tS7pD29T9q0aaTrQwghLE66PoQQwuIkqIUQwuIkqIUQwuIkqIUQwuL+P4EWAFIGJV9/AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "q1_data=pd.read_csv(\"q1_data.csv\")\n",
    "model = linear_model.Lasso(alpha=0.01,max_iter=10000) \n",
    "\n",
    "names,rmses_train,rmses_test,R2,intercepts,coefficients = q1_c_compare_RMSE(q1_data,model)\n",
    "index= q1_c_plot_RMSE(names,rmses_train,rmses_test,R2)\n",
    "\n",
    "coefficient_length= range(len(coefficients[index]))\n",
    "print(intercepts[9],)\n",
    "for i in coefficient_length:\n",
    "        print('+ ',coefficients[index][i],'*',f\"X{i}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "intercept= 0.06348440770474106 coefficients= [ -1.73131283  10.12226528   0.50504321 -19.90874129   0.03659791\n",
      "  -0.29296944  15.681297     1.05275634   0.31139441   0.75268514\n",
      "   0.28903543   0.43497495 -25.68007332   0.56871152   0.48831357\n",
      "   0.26671225   0.04105158  -0.92169186  -0.41519406   0.0905594\n",
      "   0.5524444   -0.95535051  -0.22359251   0.50861802   0.0754481\n",
      "  -0.91537219   0.6978905    0.71726915  -0.71189647   0.12255698\n",
      "  -0.14414019  -0.           0.6325181    0.12118713  -0.27011147\n",
      "  -0.6680091   -0.20391148   0.13535541  -1.47428362  -0.51991863]\n",
      "intercept= 0.2170423256864495 coefficients= [ -1.31944205  10.27703575   0.22400922 -20.23785351  -0.08689862\n",
      "  -0.58538538  15.75858756   1.112332     0.09285898   0.55692129\n",
      "   0.43428569  -0.40917629 -25.50350738   0.08957089   0.15762821\n",
      "  -0.20339928   0.25253644  -0.          -0.20286963  -0.29379055\n",
      "   0.49825295  -0.52653372   0.04466793  -0.25299377  -0.26213581\n",
      "  -0.59644716   0.34859321   0.94028174  -0.26409071  -0.29784508\n",
      "   0.          -0.09995956   0.13279222   0.75754785  -0.34089812\n",
      "  -0.17485545   0.65263714   0.02890986  -0.30203777  -0.10239567]\n",
      "intercept= 0.18110358178048602 coefficients= [-7.25093623e-01  9.94990244e+00  2.25101022e-01 -2.02019715e+01\n",
      "  9.72955256e-02 -6.68893421e-01  1.51162858e+01  6.38111241e-01\n",
      " -1.14016689e-01  1.41299636e-01  3.56773117e-01 -4.99155753e-01\n",
      " -2.54148689e+01  3.08968296e-01  2.19048888e-01 -4.96833394e-02\n",
      "  2.40728454e-02 -3.13223531e-02  1.30270090e-01 -6.00322046e-01\n",
      "  3.74547504e-01 -3.23618720e-01  1.61128582e-01 -3.04279210e-02\n",
      " -0.00000000e+00 -2.09165510e-01  2.16807705e-04  5.72403918e-01\n",
      " -1.06568883e-01 -3.20596055e-02 -2.45365603e-02  1.07649203e-01\n",
      "  7.63780155e-02  2.58205614e-01 -3.50580394e-01 -6.59207375e-01\n",
      "  2.37605692e-01  1.73257683e-01 -2.47571830e-01 -2.15412786e-01]\n",
      "intercept= 0.050564502590846994 coefficients= [-1.08881770e+00  1.05795859e+01 -0.00000000e+00 -2.01375960e+01\n",
      "  2.94577087e-01 -5.24912085e-01  1.53288378e+01  6.79022754e-01\n",
      " -4.82540321e-02  2.50049635e-01  0.00000000e+00 -2.23361623e-01\n",
      " -2.52921916e+01  1.39935496e-01  2.28611164e-01  3.11850723e-02\n",
      " -1.06224097e-01 -1.01765301e-02  2.02244037e-01 -3.05670156e-01\n",
      "  2.73549607e-01 -1.86483623e-01  1.73312411e-01  0.00000000e+00\n",
      "  2.26331907e-01  0.00000000e+00  1.42153722e-01 -8.04566452e-02\n",
      " -2.57869113e-01 -1.73048523e-01  0.00000000e+00  3.53926975e-02\n",
      "  1.65181793e-01  4.15716669e-01 -5.50272348e-01 -6.50354138e-01\n",
      " -5.07481796e-02  2.01557928e-01 -2.06204388e-01 -2.18772779e-01]\n",
      "intercept= 0.11602509756683559 coefficients= [ -0.95237191  10.28424149   0.10896635 -19.65873063   0.20511703\n",
      "  -0.51955396  14.97062409   0.38510725   0.15014072  -0.04798214\n",
      "   0.09040214  -0.30767143 -25.23186939   0.           0.08986183\n",
      "   0.29144122  -0.16997333   0.           0.22003507   0.\n",
      "   0.19030385  -0.56700945   0.34858049  -0.03592237   0.4558487\n",
      "  -0.           0.15072393   0.40919218  -0.          -0.21819578\n",
      "   0.           0.02744569  -0.06880594   0.32683572  -0.40099297\n",
      "  -0.3354079    0.0649173    0.04861899  -0.09036477  -0.46477628]\n",
      "intercept= 0.10484978797581979 coefficients= [-6.86026544e-01  1.02669110e+01 -0.00000000e+00 -1.97384396e+01\n",
      "  4.01207943e-01 -2.84894405e-01  1.50722098e+01  5.19141020e-01\n",
      " -6.91806750e-02 -1.01958363e-01  0.00000000e+00 -3.96485028e-01\n",
      " -2.50010214e+01  0.00000000e+00  2.00778016e-01  8.53436981e-02\n",
      " -2.76613734e-01 -1.60663582e-01 -7.38519552e-02  1.05413609e-01\n",
      "  2.89197154e-01 -4.99880788e-01  1.45269542e-01 -1.55327650e-02\n",
      "  2.29766291e-01  2.21620405e-01  2.10432199e-01  4.19003093e-01\n",
      " -1.24443090e-01 -5.87619404e-02  5.17756810e-02  0.00000000e+00\n",
      " -0.00000000e+00  3.24481129e-01 -2.74076347e-01 -4.65343352e-01\n",
      "  7.95681918e-02  2.02031128e-02 -1.19722464e-02 -5.03816209e-01]\n",
      "intercept= 0.08984351355733465 coefficients= [-8.88512154e-01  1.03081287e+01 -0.00000000e+00 -1.99843751e+01\n",
      "  2.77412527e-01 -1.65427567e-01  1.49124088e+01  3.26638520e-01\n",
      " -1.65705559e-01  6.31853285e-03  2.61356648e-01 -2.25262137e-01\n",
      " -2.50684398e+01  6.88763154e-02  1.59051414e-01  1.59558834e-01\n",
      " -1.92080085e-01  3.62113312e-03 -1.14970209e-02  1.76471614e-01\n",
      "  1.04566537e-01 -6.70837274e-01  3.49921084e-02  5.50663346e-02\n",
      "  2.76759315e-01  9.15899699e-02  8.71478564e-02  4.68020555e-01\n",
      "  2.39079587e-01 -1.68207218e-01  2.67214826e-01 -0.00000000e+00\n",
      " -0.00000000e+00  2.40477840e-01 -1.60916533e-01 -2.76860535e-01\n",
      "  2.74706399e-01  1.74640603e-01  6.84433013e-02 -2.11531088e-01]\n",
      "intercept= 0.010237562284680313 coefficients= [-4.68991586e-01  1.02934621e+01  0.00000000e+00 -1.98635445e+01\n",
      "  0.00000000e+00 -1.69224333e-01  1.49545394e+01  2.21441208e-01\n",
      " -1.47946770e-01 -5.94035512e-02  2.91247623e-01 -1.01589099e-01\n",
      " -2.50536176e+01 -1.10661196e-01  0.00000000e+00  1.75849885e-01\n",
      " -0.00000000e+00  1.44070769e-01  1.00908855e-01 -1.43464499e-01\n",
      "  3.71772217e-01 -4.01060316e-01  3.26943941e-01 -1.05240787e-03\n",
      "  1.47234504e-01  2.45491664e-02  3.08619618e-01  2.14672645e-01\n",
      "  1.20226506e-01 -1.64162768e-01  2.30255627e-01  0.00000000e+00\n",
      " -1.21187235e-01  2.42880106e-01 -3.26097315e-02 -3.14396996e-01\n",
      "  1.62997862e-01  0.00000000e+00  1.57385385e-01 -1.66668919e-01]\n",
      "intercept= -0.019049534509498423 coefficients= [-2.50213104e-01  1.02225305e+01  7.33031029e-02 -1.98871419e+01\n",
      "  1.93974782e-01 -1.13912653e-01  1.50893198e+01  1.38088018e-01\n",
      " -4.11215115e-02 -3.59296379e-02  2.73837843e-01 -3.20161300e-02\n",
      " -2.51954186e+01  0.00000000e+00  2.54565792e-01  1.68173541e-01\n",
      " -8.84586472e-02  1.05557297e-01 -0.00000000e+00  3.41478627e-02\n",
      "  1.58344198e-01 -2.45340053e-01  2.55453101e-01  5.78973076e-02\n",
      "  2.27143295e-01  1.15914090e-01  2.86860246e-01  4.13781597e-01\n",
      "  2.05673181e-01 -2.58860343e-01  1.95996754e-01  0.00000000e+00\n",
      " -0.00000000e+00  2.29676494e-01 -0.00000000e+00 -3.89982723e-01\n",
      "  1.16668519e-01 -9.96599835e-02  2.63634094e-01 -1.18613109e-02]\n",
      "intercept= 0.06392731853367017 coefficients= [-4.66364410e-01  1.03226915e+01  1.10898796e-01 -1.98504587e+01\n",
      "  2.20940092e-01 -1.51692888e-01  1.49595624e+01  2.71225043e-01\n",
      "  0.00000000e+00 -0.00000000e+00  2.13435484e-01  1.62009060e-01\n",
      " -2.50822888e+01  1.18089926e-01  1.80330646e-01  1.37898224e-01\n",
      " -1.53537745e-01  0.00000000e+00 -6.69627977e-02  0.00000000e+00\n",
      "  1.86977450e-01 -3.71800068e-01  7.58031377e-02  1.23040647e-01\n",
      "  1.87656534e-01  5.59795689e-03  1.88798488e-02  1.19517747e-01\n",
      "  9.75275054e-02 -1.34726710e-01  1.94557439e-01  9.59465036e-02\n",
      " -3.40407104e-02  7.33273541e-02  1.05987731e-01 -3.58666779e-01\n",
      "  3.63213802e-02  0.00000000e+00  2.43691494e-01 -1.38271880e-01]\n",
      "intercept= 0.03501007175967403 coefficients= [-3.38471899e-01  1.02775627e+01  1.54376152e-01 -1.98328311e+01\n",
      "  9.16627826e-02 -6.15934035e-02  1.49957934e+01  1.53390699e-01\n",
      " -0.00000000e+00 -3.09916779e-02  2.83629026e-01  6.85448085e-02\n",
      " -2.50642025e+01  9.64943481e-02  7.93876955e-02  2.06627085e-01\n",
      "  3.05534413e-02  1.56916254e-01 -2.51542391e-02 -4.34594268e-02\n",
      "  1.90968288e-01 -3.16629814e-01  1.57411377e-01  1.00615573e-01\n",
      "  7.65538890e-02  6.69231724e-02  1.61780250e-01  8.89804090e-02\n",
      " -2.02452418e-02 -2.04493117e-01  1.59181985e-01  4.86613698e-02\n",
      " -9.50669991e-02  8.76978149e-02  7.84919456e-02 -2.33038223e-01\n",
      "  1.36007330e-01  0.00000000e+00  2.88475389e-01 -2.40441486e-01]\n",
      "intercept= 0.10456398172067277 coefficients= [-3.78322686e-01  1.02276707e+01  6.49754398e-03 -1.98782714e+01\n",
      "  1.44735965e-01 -2.93306347e-02  1.50335829e+01  2.50831091e-01\n",
      " -5.36045209e-02 -0.00000000e+00  3.00965218e-01  0.00000000e+00\n",
      " -2.51366801e+01  2.46611600e-01  9.42403333e-04  2.68393295e-01\n",
      " -4.67126446e-02  2.58882132e-01 -0.00000000e+00 -0.00000000e+00\n",
      "  6.47668923e-02 -1.71779214e-01  2.37091590e-02  0.00000000e+00\n",
      "  2.65165409e-01  2.54751604e-02  2.05507946e-02  1.00562806e-01\n",
      "  1.08657559e-01 -2.08313024e-01  2.97183291e-01  1.59252846e-01\n",
      " -5.73492613e-03  0.00000000e+00  0.00000000e+00 -2.97674537e-01\n",
      " -0.00000000e+00  7.15606557e-02  3.10043386e-01 -1.95347605e-01]\n",
      "intercept= 0.15155685633866967 coefficients= [-4.21416483e-01  1.02032370e+01  1.99301694e-02 -1.97817184e+01\n",
      "  7.41888710e-02 -0.00000000e+00  1.50894662e+01  2.62827812e-01\n",
      " -1.30943735e-01 -1.79503021e-01  4.41027458e-01  0.00000000e+00\n",
      " -2.51497938e+01  1.04578001e-01  0.00000000e+00  3.42748171e-01\n",
      "  0.00000000e+00  2.22535580e-01  1.07296339e-02 -0.00000000e+00\n",
      "  2.84469057e-02 -1.64285866e-01  2.03219020e-01  2.72003311e-01\n",
      "  1.32175359e-01 -1.30537052e-02 -0.00000000e+00  0.00000000e+00\n",
      "  1.68544031e-02 -1.29174545e-01  2.27968601e-01  1.39959872e-01\n",
      " -1.75115775e-02  0.00000000e+00 -0.00000000e+00 -2.35204462e-01\n",
      "  2.24079648e-02  7.25954303e-03  2.35347337e-01 -1.91360355e-01]\n",
      "intercept= 0.159939095824835 coefficients= [-5.01854650e-01  1.01711806e+01  7.95400713e-02 -1.98715651e+01\n",
      "  9.05743215e-02  0.00000000e+00  1.50290985e+01  7.64579564e-02\n",
      " -1.98022275e-01 -1.21826250e-01  3.79603384e-01 -0.00000000e+00\n",
      " -2.50164055e+01  2.54061608e-02  3.85122021e-02  2.72940141e-01\n",
      "  1.07513256e-01  9.57817921e-02 -4.52172261e-02  0.00000000e+00\n",
      "  2.94205456e-02 -1.01482444e-01  1.96626943e-01  2.45193675e-01\n",
      "  2.48721476e-02 -1.48853180e-01  2.23487666e-02 -0.00000000e+00\n",
      " -9.46932518e-02 -2.69262833e-02  1.24877922e-01  8.46796809e-02\n",
      " -1.36169934e-02  0.00000000e+00 -9.50523171e-02 -2.50185815e-01\n",
      "  6.11513077e-02  7.60344449e-02  2.20018910e-01 -1.00302624e-01]\n",
      "intercept= 0.2152652576967044 coefficients= [-4.56288600e-01  1.01445408e+01 -1.32764518e-02 -1.99305181e+01\n",
      "  1.06295534e-01  0.00000000e+00  1.50865109e+01  1.41473807e-01\n",
      " -1.68090707e-01 -1.54999514e-01  4.78144082e-01  1.62327642e-03\n",
      " -2.50660892e+01  5.84265262e-02  1.86267245e-02  2.09081319e-01\n",
      " -0.00000000e+00  1.36695428e-01  0.00000000e+00  2.11911955e-02\n",
      "  1.15257732e-01 -2.98530705e-02  2.69823084e-01  9.86075387e-02\n",
      "  1.33746603e-02 -3.24376174e-02  1.19371057e-01 -3.55904906e-03\n",
      " -1.79009247e-02 -1.08411817e-02  2.74681884e-01  9.65306653e-02\n",
      "  0.00000000e+00  1.01765465e-01  0.00000000e+00 -7.24649820e-02\n",
      "  9.25570511e-02  1.36050720e-01  1.82780743e-01 -2.58818237e-01]\n",
      "intercept= 0.13618685806979192 coefficients= [-5.05296228e-01  1.01244279e+01 -0.00000000e+00 -2.00179177e+01\n",
      "  6.34943988e-02 -5.55855243e-02  1.51358302e+01  1.15635729e-01\n",
      " -2.22671280e-01 -9.41255122e-03  3.54013253e-01 -6.62434498e-02\n",
      " -2.49344217e+01 -0.00000000e+00  2.89087712e-03  2.22217560e-01\n",
      " -7.11504186e-02  5.25588131e-02 -4.65252265e-02 -0.00000000e+00\n",
      "  6.68111251e-03 -6.99065958e-02  2.52466043e-01  1.01813371e-01\n",
      "  1.76032261e-01 -7.21729615e-03  0.00000000e+00  0.00000000e+00\n",
      "  6.56525107e-02  0.00000000e+00  2.62770694e-01  1.39655540e-01\n",
      " -3.64649140e-02  5.20797348e-02 -1.82306604e-02 -2.01416101e-01\n",
      "  4.54942918e-02  6.66668866e-02  9.28291104e-02 -1.07546010e-01]\n",
      "intercept= 0.17677721652310086 coefficients= [-3.61460884e-01  1.01059730e+01 -7.24938078e-02 -2.01290308e+01\n",
      "  7.76945374e-03  9.18789983e-03  1.51080973e+01  1.37716739e-01\n",
      " -1.29005498e-01 -0.00000000e+00  4.36003990e-01 -1.77836124e-03\n",
      " -2.49504618e+01  1.17675788e-01 -0.00000000e+00  2.10751660e-01\n",
      "  0.00000000e+00  8.00413778e-02 -3.75323413e-02  3.32178836e-02\n",
      "  5.30132523e-02 -3.16363449e-02  2.17446631e-01  2.10775011e-03\n",
      "  1.24171195e-01 -2.41191009e-02  1.14612576e-01 -2.00030015e-02\n",
      "  1.16798543e-01  0.00000000e+00  2.60663349e-01  8.76884075e-02\n",
      " -3.12570395e-02  4.69405255e-02 -1.42893256e-01 -1.01173517e-01\n",
      "  3.08118361e-04  0.00000000e+00  4.05347402e-02 -1.28568679e-01]\n",
      "intercept= 0.22978434544880133 coefficients= [-5.27290473e-01  9.98992296e+00 -0.00000000e+00 -2.01276392e+01\n",
      "  2.36855579e-02  0.00000000e+00  1.51421623e+01  3.38112435e-02\n",
      " -6.82120035e-02 -0.00000000e+00  4.17186873e-01 -1.67253898e-01\n",
      " -2.49239908e+01  4.66554765e-02 -0.00000000e+00  3.00041330e-01\n",
      " -2.97310424e-02  8.45914388e-02 -7.78160605e-02 -4.13969686e-03\n",
      "  7.09556948e-02 -5.99904638e-02  1.01984086e-01  1.69997935e-01\n",
      "  1.95996442e-02 -3.90125360e-02  2.21144771e-02 -1.42558869e-01\n",
      "  3.49539651e-02 -5.47834980e-02  1.52594847e-01  1.73257900e-01\n",
      "  0.00000000e+00  4.65189073e-02 -1.53474503e-01 -1.25787579e-01\n",
      "  0.00000000e+00  0.00000000e+00  2.84782050e-02 -8.68934129e-02]\n",
      "intercept= 0.1976706722759347 coefficients= [-4.56818578e-01  1.00428839e+01  1.80936998e-02 -2.01116091e+01\n",
      "  1.84585627e-01  7.48022103e-03  1.52246424e+01  3.46585629e-02\n",
      " -1.61967623e-01  4.72082526e-02  4.35686296e-01 -2.18224089e-01\n",
      " -2.49317127e+01  1.65005536e-02 -9.01410999e-02  1.47551654e-01\n",
      " -6.13086775e-02  1.94879273e-01 -0.00000000e+00  0.00000000e+00\n",
      "  1.78598697e-02 -8.95053155e-02  3.23974161e-02  8.03029851e-02\n",
      "  1.46151081e-02 -7.42412456e-02 -0.00000000e+00 -2.42310889e-02\n",
      " -1.29442412e-02 -9.06197890e-02  3.12181253e-01  1.71666170e-01\n",
      "  0.00000000e+00  8.31795696e-02 -1.45130670e-01 -9.05724115e-02\n",
      "  0.00000000e+00  6.54768589e-02  0.00000000e+00 -1.10198378e-01]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>rmses_train</th>\n",
       "      <th>rmses_test</th>\n",
       "      <th>R2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>data_100</th>\n",
       "      <td>1.646400</td>\n",
       "      <td>4.528095</td>\n",
       "      <td>0.994265</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_150</th>\n",
       "      <td>2.061995</td>\n",
       "      <td>2.286290</td>\n",
       "      <td>0.990312</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_200</th>\n",
       "      <td>2.188003</td>\n",
       "      <td>2.515446</td>\n",
       "      <td>0.989402</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_250</th>\n",
       "      <td>2.278590</td>\n",
       "      <td>3.241498</td>\n",
       "      <td>0.989068</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_300</th>\n",
       "      <td>2.317164</td>\n",
       "      <td>3.038299</td>\n",
       "      <td>0.989325</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_350</th>\n",
       "      <td>2.357420</td>\n",
       "      <td>2.627973</td>\n",
       "      <td>0.988573</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_400</th>\n",
       "      <td>2.391175</td>\n",
       "      <td>2.762583</td>\n",
       "      <td>0.988084</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_450</th>\n",
       "      <td>2.458150</td>\n",
       "      <td>2.852479</td>\n",
       "      <td>0.987888</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_500</th>\n",
       "      <td>2.502663</td>\n",
       "      <td>2.498159</td>\n",
       "      <td>0.987415</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_550</th>\n",
       "      <td>2.553449</td>\n",
       "      <td>2.230762</td>\n",
       "      <td>0.986671</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_600</th>\n",
       "      <td>2.568529</td>\n",
       "      <td>2.473738</td>\n",
       "      <td>0.986476</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_650</th>\n",
       "      <td>2.579249</td>\n",
       "      <td>2.562474</td>\n",
       "      <td>0.985804</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_700</th>\n",
       "      <td>2.582066</td>\n",
       "      <td>2.395939</td>\n",
       "      <td>0.985948</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_750</th>\n",
       "      <td>2.550690</td>\n",
       "      <td>2.746501</td>\n",
       "      <td>0.986022</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_800</th>\n",
       "      <td>2.574258</td>\n",
       "      <td>2.621212</td>\n",
       "      <td>0.986112</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_850</th>\n",
       "      <td>2.556844</td>\n",
       "      <td>2.869903</td>\n",
       "      <td>0.985798</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_900</th>\n",
       "      <td>2.583427</td>\n",
       "      <td>2.661321</td>\n",
       "      <td>0.985520</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_950</th>\n",
       "      <td>2.556969</td>\n",
       "      <td>2.960019</td>\n",
       "      <td>0.985964</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>data_1000</th>\n",
       "      <td>2.557206</td>\n",
       "      <td>2.893992</td>\n",
       "      <td>0.985934</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           rmses_train  rmses_test        R2\n",
       "data_100      1.646400    4.528095  0.994265\n",
       "data_150      2.061995    2.286290  0.990312\n",
       "data_200      2.188003    2.515446  0.989402\n",
       "data_250      2.278590    3.241498  0.989068\n",
       "data_300      2.317164    3.038299  0.989325\n",
       "data_350      2.357420    2.627973  0.988573\n",
       "data_400      2.391175    2.762583  0.988084\n",
       "data_450      2.458150    2.852479  0.987888\n",
       "data_500      2.502663    2.498159  0.987415\n",
       "data_550      2.553449    2.230762  0.986671\n",
       "data_600      2.568529    2.473738  0.986476\n",
       "data_650      2.579249    2.562474  0.985804\n",
       "data_700      2.582066    2.395939  0.985948\n",
       "data_750      2.550690    2.746501  0.986022\n",
       "data_800      2.574258    2.621212  0.986112\n",
       "data_850      2.556844    2.869903  0.985798\n",
       "data_900      2.583427    2.661321  0.985520\n",
       "data_950      2.556969    2.960019  0.985964\n",
       "data_1000     2.557206    2.893992  0.985934"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "f(x)=  0.06392731853367017\n",
      "+ -0.46636441049460886 * X0\n",
      "+ 10.322691526625897 * X1\n",
      "+ 0.1108987960213512 * X2\n",
      "+ -19.850458720932185 * X3\n",
      "+ 0.22094009230685951 * X4\n",
      "+ -0.15169288766219538 * X5\n",
      "+ 14.959562368587228 * X6\n",
      "+ 0.2712250430494371 * X7\n",
      "+ 0.0 * X8\n",
      "+ -0.0 * X9\n",
      "+ 0.21343548412339597 * X10\n",
      "+ 0.16200905992920286 * X11\n",
      "+ -25.08228875627463 * X12\n",
      "+ 0.1180899263885704 * X13\n",
      "+ 0.18033064647920652 * X14\n",
      "+ 0.13789822392086162 * X15\n",
      "+ -0.15353774460685007 * X16\n",
      "+ 0.0 * X17\n",
      "+ -0.06696279770099502 * X18\n",
      "+ 0.0 * X19\n",
      "+ 0.1869774499427069 * X20\n",
      "+ -0.37180006779864755 * X21\n",
      "+ 0.07580313770495156 * X22\n",
      "+ 0.12304064745973649 * X23\n",
      "+ 0.18765653392241688 * X24\n",
      "+ 0.0055979568937551015 * X25\n",
      "+ 0.018879848783960908 * X26\n",
      "+ 0.11951774748637625 * X27\n",
      "+ 0.09752750538490133 * X28\n",
      "+ -0.1347267103152522 * X29\n",
      "+ 0.19455743876099357 * X30\n",
      "+ 0.09594650358369856 * X31\n",
      "+ -0.034040710374350046 * X32\n",
      "+ 0.07332735413771002 * X33\n",
      "+ 0.1059877312404241 * X34\n",
      "+ -0.3586667787759165 * X35\n",
      "+ 0.036321380212320876 * X36\n",
      "+ 0.0 * X37\n",
      "+ 0.24369149402501653 * X38\n",
      "+ -0.13827187966462975 * X39\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD5CAYAAAAOXX+6AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAxNUlEQVR4nO3deXxU1dnA8d+ZyTLZSCA7i7IFEMjCpiAuBAVBwaot7mtV2lqrvlaLVqvSWq1Kq/WV9pVaRa0arYq7iGgQXBEQEDHIIkgAgQBZJutk5rx/nEkIkGUSZrmTPN/PZz4zc++de5+5uXnm3HPOPVdprRFCCGFdtlAHIIQQonWSqIUQwuIkUQshhMVJohZCCIuTRC2EEBYXEYiVpqSk6L59+3bos5WVlcTFxfk3oACQOP0vXGKVOP0rXOKEwMa6cuXKEq11arMztdZ+f4waNUp3VGFhYYc/G0wSp/+FS6wSp3+FS5xaBzZWYIVuIadK1YcQQlicJGohhLA4SdRCCGFxAWlMFEKEB5fLRXFxMTU1NSGLITExkW+//TZk228Pf8TqcDjo3bs3kZGRPn9GErUQXVhxcTEJCQn07dsXpVRIYqioqCAhISEk226vo41Va82+ffsoLi6mX79+Pn9Oqj6E6MJqampITk4OWZLuapRSJCcnt/sMRhK1EF2cJOng6sj+lkQthBAWJ4laCCEsThK1EEJ4PfLII1RVVbX7c3fddReLFy8OQESGJGohhGVorfF4PCHbfmuJ2u12t/i5P/7xj5x++umBCku65wkhjNlvfsP6neV+XefQnt24e/qwVpfZtm0bM2bMID8/nxdeeIGkpCROO+00Pv/8c3Jzc7nqqqu4++672bNnD8899xzHH388H330ETfeeCNgGueWLl1KQkICDz30EC+99BK1tbWce+65zJ49m8rKSs4//3yKi4txu9384Q9/4IILLjgijkcffZSdO3eSn59PSkoKhYWFxMfHc/PNN/Pee+/x17/+lXfffZdFixZRXV3NiSeeyOOPP45SiiuvvJJp06bxs5/9jL59+3LFFVfw5ptv4nK5+O9//8uQIUOOaj9KiVoIEXIbNmzg8ssv56uvvmL79u3ceOONrF27lqKiIp5//nk+/vhj5syZw3333QfAnDlzmDt3LqtXr2bZsmXExMSwaNEiNm7cyPLly1m9ejUrV65k6dKlLFy4kJ49e7JmzRrWrVvHlClTmo3hhhtuoGfPnhQWFlJYWAiY0fKGDx/OF198wUknncTMmTP58ssvWbduHdXV1bz11lvNrislJYVVq1bxq1/9ijlz5hz1/pEStRACoM2SbyAde+yxjB07lq1bt9KvXz+ys7MBGDZsGKeddhpKKbKzs9m6dSsA48eP5+abb+aSSy7hvPPOo3fv3ixatIhFixYxYsQIAJxOJxs3buTkk0/mlltuYdasWUybNo2TTz7Z57jsdjs//elPG98vW7aM888/n6qqKvbv38+wYcOYPn36EZ8777zzABg1ahSvvvpqR3dLI0nUQoiQazrGc3R0dONrm83W+N5ms1FfXw/AbbfdxllnncU777zD2LFjWbx4MVprbr/9dn7xi18csf6VK1fyzjvvcPvttzN58mTuuusun+JyOBzY7XbAXBx08803s3LlSvr06cM999zT4oUrDTHb7fbGmI+GVH0IIcLO5s2byc7OZtasWYwePZqioiLOOOMMnnzySZxOJwA7duxgz5497Ny5k9jYWC699FJuueUWVq1a1eJ6ExISqKioaHZeQ1JOSUnB6XTy8ssv+/+LtcCnErVSaitQAbiBeq316EAGJYQQrXnkkUcoLCzEbrczdOhQpk6dSnR0NN9++y3jxo0DID4+nv/85z9s2rSJW2+9FZvNRmRkJP/85z9bXO/MmTOZOnUqmZmZjfXUDZKSkrjiiivIzs6mb9++jBkzJqDf8RAt3VGg6QPYCqT4sqyWO7xYSrjEqXX4xNqZ4ly/fn3gA2lDeXl5qEPwmb9ibW6/I3d4EUKI8KVMIm9jIaW+Bw4AGnhcaz2vmWVmAjMB0tPTRxUUFHQoIKfTSXx8fIc+G0wSp/+FS6ydKc7ExEQGDhwYpIia53a7GxvsguXiiy9m27Zth0ybPXt2mxet+CvWTZs2UVZWdsi0/Pz8lbqlauWWitr60KqPnt7nNGANcEpry0vVh3WES5xah0+snSlOqfpoH0tXfWitd3qf9wALgOM7+EMihBCindpM1EqpOKVUQsNrYDKwLtCBCSGEMHzpnpcOLPAOdh0BPK+1XhjQqIQQQjRqs0Sttd6itc71PoZprf8cjMCEECLYOjrMKcBrr73G+vXr/RyRId3zhBCWoS08zGlbJFELITqtbdu2cdxxx3HdddfRo0cPBgwYwDXXXMPw4cO55JJLWLx4MePHjycrK4vly5cD8NFHH5GXl0deXh4jRoxovOz7oYceYsyYMeTk5HD33XcDZgS8s846i9zcXIYPH86LL77YbBxNhznNz88HYNGiRYwbN46RI0cyY8aMxsvTb7vtNoYOHUpOTg633HILn376KW+88Qa33noreXl5bN682a/7SAZlEkIY794GP37t33VmZMPUv7S52IYNG3jqqaf43e9+x8CBA7nxxhuZN28eY8aMaRzm9I033uC+++7jtddeaxzmdPz48TidThwOxyHDnGqtOfvss1m6dCl79+6lZ8+evP322wBH9F9ucMMNN/C3v/2NwsJCUlJSKCkp4d5772Xx4sXExcXxwAMP8Nhjj/Hb3/6WBQsWUFRUhFKK0tJSkpKSOPvssxvHpPY3KVELIUKuYZhToHGYU5vN1uYwp48++iilpaVEREQcMszpyJEjKSoqYuPGjWRnZ7N48WJmzZrFsmXLSExM9Cmmzz//nPXr1zN+/Hjy8vJ4+umn2b59O926dcPhcHDNNdfw6quvEhsbG6jd0khK1EIIw4eSb6BYcZhTrTWTJk3ihRdeaJxWUVFBREQEy5cv54MPPqCgoIDHHnuMDz/8sMPf3RdSohZChJ1gDHM6duxYPvnkEzZt2gRAVVUVGzduxOl0UlZWxplnnskjjzzC6tWrj/isv0mJWggRdoI1zOn8+fO56KKLqK2tBeCOO+4gMzOTn/zkJ9TU1KC15uGHHwbgwgsv5Nprr+XRRx/l5ZdfZsCAAf77wi1dW340DxnrwzrCJU6twyfWzhSnjPXRPpYe60MIIUToSNWHEKLLOffcc/n+++8PmfbAAw9wxhlnhCii1kmiFkJ0OQsWLAh1CO1iraqPuioiXM5QRyGEEJZinURdXwcP9KXP9vD6pRNCiECzTqKOiIKULOKd37e9rBBCdCHWSdQAGTkkVPh3MBMhhAh31krUmTlEuUqh4sdQRyKECJKkpCTy8vIYPnw406dPp7S0FIDVq1czbtw4hg0bRk5OTouj3nUF1krUGTnmedfa0MYhhAiamJgYVq9ezbp16+jRowdz584FIDY2lmeeeYZvvvmGhQsXctNNNzUm8a7GYok62zz/uCa0cQghQmLcuHHs2LEDgEGDBpGVlQVAz549SUtLY+/evaEML2Ss1Y/a0Y2qmExipUQtRNA9sPwBivYX+XWdQ3oMYdbxs3xa1u1288EHH3D11VcfMW/58uXU1dX5d/yMMGKtEjXgjO8Pu6RELURXUV1dTV5eHsnJyezfv59JkyYdMn/Xrl1cdtllPPXUU9hslktZQWGtEjUmUad9/wlUl0JMUqjDEaLL8LXk628NddRlZWVMmzaNuXPncsMNNwBQXl7OWWedxb333tt4Y4GuyHI/T874/uaFv28JJISwtMTERB599FHmzJmDy+Wirq6Oc889l8svv5wZM2aEOryQslyJuiKhIVGvhX4nhzYYIURQjRgxgtzcXAoKClBKsXTpUvbt28f8+fMBmD9/Pnl5eSGNMRQsl6hdUUmQkCn11EJ0Ebt27Trk/Ztvvtn4+tJLLw12OJZkuaoPwPSnlp4fQggBWDVRZ+ZAyXfgqg51JEIIEXLWTNQZOaDdsHt9qCMRotMzd4ESwdKR/W3NRJ2Za57lCkUhAsrhcLBv3z5J1kGitWbfvn04HI52fc5yjYkAJB0DjiRpUBQiwHr37k1xcXFIL82uqalpd+IKFX/E6nA46N27d7s+Y81ErZQZ90MaFIUIqMjISPr16xfSGJYsWcKIESNCGoOvQhWrz1UfSim7UuorpdRbgQyoUWYu7FkP7vqgbE4IIayqPXXUNwLfBiqQI2TkQH2N6f0hhBBdmE+JWinVGzgLeCKw4TTR0KAo9dRCiC5O+dLaq5R6GbgfSABu0VpPa2aZmcBMgPT09FEFBQUdCsjpdBIfHw/azcnLLmRnzzPYPPCaDq0rkBrjtLhwiRPCJ1aJ07/CJU4IbKz5+fkrtdajm52ptW71AUwD/uF9PQF4q63PjBo1SndUYWHhwTfzJmr95JkdXlcgHRKnhYVLnFqHT6wSp3+FS5xaBzZWYIVuIaf6UvUxHjhbKbUVKAAmKqX+c/S/Hz7IzDGj6EkfTyFEF9ZmotZa36617q217gtcCHyotQ7OSCmZuVBbBge2BmVzQghhRda8MrFB481upUFRCNF1tStRa62X6GYaEgMmbSgouxmbWgghuihrl6gjHZA6RK5QFEJ0adZO1OBtUJRELYTousIgUeeCczdU/BjqSIQQIiSsn6gbGxSlVC2E6JrCIFFnm2cZm1oI0UVZP1E7ukH3flKiFkJ0WdZP1GDqqaVBUQjRRYVJos4xVydWl4Y6EiGECLrwSNQZDfdQ/Dq0cQghRAiER6LO9Pb8kOoPIUQXFB6JOj4N4jOkQVEI0SWFR6IGaVAUQnRZYZSoc2DvBnBVhzoSIYQIqvBJ1Bk5oN2we32oIxFCiKAKn0Td2KAoVygKIbqW8EnUSceCI1EaFIUQXU74JGqlTPWH3O1FCNHFhE+iBtPzY896cNeHOhIhhAia8ErUGTlQXwMl34U6EiGECJrwStRyhaIQogsKr0SdnAURMdKgKIToUsIrUdsjIH2YNCgKIbqU8ErU4L3Z7degdagjEUKIoAi/RJ2RA7VlZnxqq6qtgML7oHhlqCMRQnQC4ZeoMxvGprZwPXXh/fDRA/DERHh6Omz+UM4AhBAdFn6JOm0oKLt166n3fAtf/B/kXgyT74W938Gz58K8CbD+dfC4Qx2hECLMhF+ijnRA6hBr9vzQGt651dyQ94w/w4m/gZvWwvS/Q205vHQ5zD0BVj0L9XWhjlYIESbCL1GDt0HRgon6mwWwdRlM/APE9jDTIqJh1JVw/Qr42VPmh+aN6+HvufDZXKh1hjRkIYT1hWeizsgB526o2B3qSA6qdcKiO01so648cr7NDsPPg18sg0tegR794b3fwyPDYclfoGp/0EMWQoSH8EzUVmxQXPZXKN8BZ84xSbklSkHW6XDV23D1+9BnLCy5Hx4eDgt/D+U7gxezECIstJmolVIOpdRypdQapdQ3SqnZwQisVRnZ5nnX6pCG0ahkE3z6v6YB8ZgTfP9cn+Ph4gL41Wdw3DTTCPlIDrx+vVmnEELgW4m6Fpiotc4F8oApSqmxAY2qLY5u0L2fNRoUtYaFsyAyBk6/p2PrSB8K582DG1aZapOv/wuPjYb/XilVIkKIthO1NhpavCK9j9B3CrZKg+KGd2HTYphwGySkH926uveFs+bATV/DSf8DRW/D/Gng3OOXUIUQ4cmnOmqllF0ptRrYA7yvtf4ioFH5IiPHXJ1YXRq6GFzVsPA2013w+Jn+W298Gpx+N1z8Ehz4Hp6aCmXF/lu/ECKsKN2OK+aUUknAAuA3Wut1h82bCcwESE9PH1VQUNChgJxOJ/Hx8W0u12PfKnK+ns3q3Hsp7Z7doW0dDafTybCSt+i39QVW5/6J0u45AdlOYul6sr/+E/UR8azO+yM1MZntjtOX/WkF4RKrxOlfVogzoXwjPfavpMaRRlVsL6pjelEfeWRMgYw1Pz9/pdZ6dHPz2pWoAZRSdwOVWus5LS0zevRovWLFivZF6bVkyRImTJjQ9oLOPTAnC864D8b9ukPbOhqfv/siY1feAIOnwoz5gd3Yzq/M1Y32aLj8dUgb4vNHfd6fFhAusUqc/hXSON0uWPoQLJ0D+rCrhuNSIWUQJA+ElCxIzuKLzQc44YwZZiRPP1NKtZio29yaUioVcGmtS5VSMcDpwAN+jrH94tMgPiNkDYoDNj8JymYuEw+0niPgynfg2XNg/plw2YKDXRSFEB2z9ztYMNMUhHIvgsl/hur9ULIR9m00d5Iq2QRFb0HVPgBOAFjxG3MdREqWN4kPOvi64UI3P/PlZyETeFopZcfUab+ktX4rING0V6gaFDctJrXkczjtLkjsHZxtpg+Fq96Fp8+G+dPh0pdN9z4hRPt4PPDlv+D9uyAyFs5/Bob+xMyLSzZJ93BVJoEXffImQ1LssG+TSeTfvQce18HlEo8xw0Yo5deQ20zUWuu1wAi/btVfMnNh0wemUS8yJjjbrK+Dd2dRFZNJ7Ljrg7PNBskD4OcL4Zmz4Zlz4KIXoP+pwY2hvepr4cd1EBULaceFOhphJZUl8OGfOHHtAqg8H8bfCEl9ArvN8p3w2nWwpRCyJsPZ/wsJGW1/LrYHHHMCP26pZkjTahp3PZRuM6Xwku/AVeX3JA2+laitKyPH1CvtXg+9RwVnm5//A/ZtYlP2XeRERAdnm00l9TEl62fOgedmwAXPwqAzgh9Hczxuc7DuWAU7VsLOVSZJe1ymmmjSH2Hc9QE5kEUYqa+D5fPgowfBVYkzMZseK+fDyqdMFcRJ/2MKJf729cvw9s2mXnrawzDqqqM/Fu0RJtbkATB4in/ibEZ4J+rGm92uCU6iLt9pDq7BZ7I/OUg/DM1JyICr3jENjAUXw0+fgGHnBjcGraH0B5OMd6yEHV+ZK0XrvF3uo7tBzzzT0NtrpPknWXSn+VGd9rAZnEp0Pd8tgvduN1UHA06DKfez9ptdTBgxED55FFY9Daufg+E/hZN/65+zsKr98M4tsO4V6D0Gzn08MD8EARTeiTrpWHAkBq9BcdEfwFNvepqs3RacbbYktgdc8QY8fwG8/HOoq4IRlwRue5Ulh5aUd6xsbGDBHmXObvIuhl6joOdI07Bia9JNf8h0WPqgGddk30a44Lmjv0BIhI+935lByDa9b46Ni18yVQ9KAbtMW8+ZD5rk/Nlj8OW/zRW6x02Hk28xP/odsekDeP3XULkXJt4J4/8nID02Ai38Im5KKZMggtGguPVjWPcynDoLevQDQpyowfxIXfoKFFwCr19n6seOv9Y/63bVwJYlpsX7+49M6RkAZS7wGTQVeo0wiTltGEREtb4+m81cvZl2HCz4JfwrHy58zvRoEaFRtd+c+QQycVUfMGehy+eZhrvJfzYXh7V0vCSkw+Q/meqPz/8JXzwO375pkvopt/regF5XBYvvNttNGQwXFXQ82VtAeCdqMA2KXz5hKvUDdcC5680NARKPgfE3BWYbHRUVZw7Cl68yp3d1TnOQd0T1AXNqWvSWt5G2EqISYEA+jLnWJOXMXIg+ig7/Q39ixmkpuBienArnzDWnuSJ4PB7Td3jJfeBIgoGnmUQ48HSIS/HTNtymGuPDe80PwsjLzTjt8am+fT62B0y8A068Hpb/y7QN/XsS9DvFJOy+J7dcv7xjJbz6C3PmNvY60zsrWJ0NAiT8E3VGDtTXmEas9KGB2caXT8Ce9XDBf0zvBauJdJguRgt+CYvvgbpKyL/Dt4aSsmIoesck560fm8bZ+AzIvQCGnGX+IfzdaJqZA9cWwkuXmWqb3etNvCLwap3w2q/g2zdMu0ZUHGx839TfosyPcdZkyJoEmXmHVl/56vulsPB22L0Ojh0PU+7veL9/RyKccguM/RWsnG/qsZ+eDr2PNwk7a9LB49ztMsMNf/Sgace5/HXoP6Fj27WY8E/UjQ2KawOTqJ17oPDPMGAiDJnm//X7iz3SjMAXFWtKS3WVpi79cFqbH52G5NwwVGzKIBh/g/mOPUd27B+0PeJT4fI3TCv8sjmw51vsqZcGdptd3YGt8MLFsPdbUwUx7tcmyXk85v9n4yLzWHK/KW3HpZlEmDXZnFU5Ette/6I7TVVF4jEw42lzBuWPXj5RcSbe0VfD6v/Ax4/A8zNMQe2UW0113Gu/NKXpnAtg6oMQk3T027WI8E/UyVkQ4TANirkX+n/9i2ebftpTH7R+tzKbHaY/CpFx5lSxzgkJ55jT0O1fmNH4it4y/1BgWsBPvwcGnwWpg4Ifb0SU6ceakQ0Lb2dE8ToYNdSMIij8a8tHZthc7YZLXjbVHQ1sNlN/2zMPTv2daTje9AFsfM8cM6ufA1uEucnFoMkmcacOOfj/UOuEj/8Gnz5mjsH8O02VRSCqGyIdMOYaGHkFrH3RlKBfuszMi+luhnMIdg+oIAj/RG2PgPThgWlQ3P6l+fUef2PzVytZkVLmVDM6HpY+xMiEz+DLA1BVYnpn9DvVfJ/BZ/rW0T8Y8Z7wC0gZRPQLl8C8fFON0+/kUEfWOWhtGtQW3m56W1z0Qttd0+JSTNVX7gWmfab4y4Ol7ffvMo/EY0xpO3mAqY5w/mhKsqfdDYm9Av+97JEw4lLT7/qbBbBrjamP7ta+QcvCRfgnajDVH1+/Yg5Kf5V6PW7TOJeQaU6twolSpitSdDeilj4KWRNMffPA081NF6xoQD6rRs7hhC0PmzFNpj4IY64OdVRhTXlc8MZv4KtnTS+d8+a1/+9vj4Bjx5nH6XdD2Q5v0n4f1hSYBueeI82FV6EY0sBmh+yfmUcn1jkSdUYOrHjSnNL36Oefda56xtTf/vTfEJ3gn3UG2/gb+NyVExYjqAFUx/aEaxbDK9eYuus962HKX0zpKQQ8Hk1NvZsal4cal9v78LCl1E3c1v246j3Uuj246j243Jo6txtXvabO7aGu3oPLbR519R7q3Lrxtcvtwe3ReDRoNFqDR7f87NEA5vnge4iPtpMYE0X32Ei6x0aR6H3uHhtJMgfIXnUHODeYgsaE33eo3UFrTW29h4qaepy19Thr4qnoPh1nzlSqBlZhL91CdVIWMfsjiXXuJibSjiPKTmyUnZhI78P7OsLu2/a11lTWuamocVFeXU95jYvyahflNS4qauq9r81zRY2ZX+NyEx8dQbeYSBIcEXRzRLby2jw7Ilu5t6nFdI5E3bRB0R+Jumo/fDDbtFhL17HgciSa7oYfzIZP/g57N5iqkHaOSqa15kCVi52l1RQfqGZnqXmUVrsaE25t/cHkW+NyH5KUa10e6tyeljfw+Wc+x5JCGSMjvifPvoVstZnBbGOb6s3bEaex1D6Wels0SoFNqYPPHHyvlMLmnW5TNJ417iytprSqjtIqF/UN2RvIUZt5POphHFRynesGln00hqQVS0iKiSLJm8yTYiNJjIk8LAm7cNbWH3xfW4+zpv6QdTdvXRvzjUi7OiRxx0RFEBNpo7KimofWLmtMuhU19bjb2KYj0kY3hzf5xkQSHWGjxFnHlpLKxmTeVtxRdltj0k5wRBATZcfs+ZYdKK3m/75r+W+fGBPJ45c1O1LpUekciTptGCi7aVBsGAXraHx4L9SUhUcDYmdks5txQdKGwhs3wLwJcPGLh1xOXFvv5seyGnaUVrOztIadpdXsOFDNzrJq77RqalyHJlpHpI0esVE4Iu1ER9pxRNpwRNhJiY/AEWn3PmxERxx87Yi044iwHbL8hm+/YdSIXKLsNiIjbETZbURF2Ii024iqLye2ZC2OPWuI3L0G+66vUOU7vBF4LxZKn0ha8XLGlD4M9kQYdj6MvKzDXdi01jhr6ymtcqHXFNB72Z+oiU7hye63M7h/PmlVdZRVuzhQVceBKhfb91dxoMqUUKMjbMRHm0QVH20efXrEkhAdQXzDNEdEk/eRxEdHNC4fG2Wntt78uFXVual2eR913ofr4HNVnflhrK5zU+V9rnG5cQIZ3RxkpZmk21DqTXBEHlICbigVJzgiiI5ovTSstaba5W5SAj9YCj+8NN7wuqquvo00jfesprXttvvP55POkagjHeYfYNeao1tP6XZY8hfTyn38TMgY7p/4RLtU17nZXV7DjwmTqBs/n1Gf/4aI/5vI0+mzeKd+DDtKq9lbUXvE51IToumZFMOQjAQmDk6jV/cYeibF0CvJPHePjUT54Yc3Zt8GTs5KNV0gd62FH1aZy+t3roL9Ww4u2L0fHDPW1OH2HHHoxUIeD2xdCqueNdVsX/7LVOGNvNzUt8Z09zkepRQJUTYSlt4Hn/4vHHsSsec/zfAv1zFhQsuN4Fprv+yPo2VuHDDGr+tUShEbFUFsVATp3fw3royJdZzf1uerzpGowVR/bP6wY5+tLIFlfzP/LCjTejzxTr+GJ8Dl9lDirOXHshp2l9eyp6Km8fXu8hq2/FiFc8l7lNfUH/K5dO7hX9EPM3PX3eTFnMSi/v9DQuogeiY5GpNwRqIj8HWOHg+sX8Dgoudg/e2wtwi0t9TerZdJxnmXmEGoMvNar66x2czFGP0nmKq2r1+Gr54xDdjv3QFDzza9Gvqe0nbdcvUBePlq2PyBuYJ0yv0+1etbIUkL33SeRJ2RA2tegIrdvg/2U1sBn/3DlEJclWZQoVNvC/yYuJ1YvdvDD/ur2Ly3ks17nWzZ62Tz3kp+2F9FibP2iFPDCJsiLSGatG4OMuNsDO/fi7RuDjK6OUjv5iC9WzTpiQ4SIs6Hzx7j+I8e5Pjvr4B+d8LImaaaJBi2fmwGFdq1huTIbnDsCWbAoIbS8tEMMBXbA06YaR47V5teGmv/awYlSjrWJOy8i5u/ScXeDfDCheZscPrfYdSVHY9DWFbnSdRNGxQTJrW+bH0trHjKXMFXVWL+4Sb+AVIHBz7OTqKs2tWYhDfvdbJ5j5MtJZVs21eJy30wG6cmRNM/JY6Jg9NIT2xIwNHeJOwgOS4Km82U7MxpZSvVTSf/FoadZ0qdC28zP8zTHjEl2EDZt9n0Gy56C7r1hvOe4NN9yUzInxiY7TVceDL5XnOF36pnzJWxhfeZi1RGXGb6wEdEwYZ34ZVrTdXfFW+aLnSiU+o8iTrDeyfyXWtMR/zmeNyw9iVz0Jf9YMaxOH128G46EEbq6j2UVtWxv6qOXWU1bDksITetI460K45NjmNAahyTh6bTPzWeAalx9E+NJzHGz13revQzV9atfw3evQ2eOM2c7k+80799xKsPmBuefvG4uVBo4p3mpgeRMbBkif+205LIGMg53zz2f2/aTb56Dv57BcQmQ9+TYP0bpoBy4fPBuyWcCInOk6gdiabxprkGRa1N6eODP5pxDjJz4ey/Q//8LtGrw+3RlFbVcaCqjv2VLvZXNryu40ClScbm2cUB77SK2voj1pMUG8mA1HjyB6cyIDWeAanx9E+No0+PWCJ97CPrF0qZy4QHTDQ9dJbPg/Wvw9S/wNBzju5v6naZs60l95tkPfIyc0l0KMfO7tHP/FBMuN20w6x6BjYshOwZprrDigOFCb/qPIkaTOni8ES99RMzolzxcugxwIwFcNxPAj/oUIC5PZr9lXWUOGvZW+F9NH3tfb/rQCVV773TYreh2Cg73WOj6BEXRfe4KPolx9I9Lopk7/sesVGkJEQzIDWeHnFtjDkdbI5EOPMhM8bLmzeZsSwGToKz5rR/vBCtzY1KF91phsfsd4oZ1KrhTM0KbHbvIEmTTMNmmB/DwnedK1Fn5JiSVU2ZGeh+8WxzR4mETFPyyLskZFe5+UJrTWmVi32VteytaJKEm0nA+5y1zfbnjIm0k9YtmtT4aLLS4jnGUUN2Vr/GRNwjNorucZHmvbdPcdjrNcoMm7p8nqnPnTvWDC504m98+3v/uA4W3WFulJCcBRe9aO5DaeWzLUnSXUrnStQNFww8fyH88KkZFH3SH02f6BANHN605FvirGWf05uAnbWUVByc3jCvuaupIu2KlPhoUhOiyUx0kNM7kdQE8z7VO71hflz0oX9S00AXgpHxgs0eAeOuMxc8LZxlrmxc+xJMf8T0ZW5OxW4ovBe++o8pnU99EEb/3NI/5qJr6nyJWtlg51dw0s1mlLggjUlbV+9hw48VrC4uZc32UtbvLGdPRQ37K+uaLflG2W2kxJtqhfRuDob17EZKvEm4yfFRpMZHk+JNxIkxkY09I0QbEnuZGzxsWGh6hzx5hrmI5PTZB/s1u6rhs7nw8cOmB9AJv4JTb23XRSZCBFPnStTxaXDVQuh+bECH8PR4ND9WeljwVTFrtpexensp63eVU1dvLn5IjotieK9EcvskkepNxk2TcEp8NN0cEXLBQSANnmKGSl3yF5OUi942g+XbI02bRdl2c5OESX8MuztSi66ncyVqgGNO8Psq95TXsHp7KWuLy1jjLTGbq+fWEBtlZ3ivRK48sS+5vZPI7ZNIr6QYScJWEBVnbpSacwG8dZO5AwiYtoxz/iljXouw0fkS9VGqq/ew6ocDfPWDSchrikvZVVYDmKvoBmckMC23J47KH7ng9LEMTIvHLtUS1pYxHH6+CL551bwfdm7wrmgUwg8kUWNKzIUb9vBh0R4+3lhCZZ0bgL7JsRzfr4e3pJzEsJ7dGntJLFmyj8EZYTpOdVdks3X6weVF59UlE7Xbo1lTXMqSoj18uGEP63aUA5CZ6OAnI3qRPziNMX27kxRrsX7DQoguqcsk6rIqF0s37qWwaA9LvtvL/so6bApGHdud300ZzMQhaQxOT5C6ZSGE5XTaRK215rvdTj4s2kNh0R5W/nAAt0fTPTaSUwelkj8kjVMHpUqpWQhheZ0uUX+6uYR3vt5FYdFedpRWAzA0sxu/OnUA+UNSyevTXRr/hBBhpc1ErZTqAzwDZAAeYJ7W+u+BDqwjHv9oM/e/W0RslJ3xA1O4fuJA8genkZHovzs8CCFEsPlSoq4Hfqu1XqWUSgBWKqXe11qvD3BsPtNa8+gHm3h48XeclZPJX2fkdo4xLIQQAh8StdZ6F7DL+7pCKfUt0AuwRKLWWvPgexv455LN/HRkbx78WY5UbQghOhWl23HbXKVUX2ApMFxrXX7YvJnATID09PRRBQUFHQrI6XQSHx/v07Jaa54vquP9bfVM6BPB5UOjsAWp10Z74gylcIkTwidWidO/wiVOCGys+fn5K7XWo5udqbX26QHEAyuB89padtSoUbqjCgsLfVrO7fbo219dq4+d9Za+54112uPxdHibHeFrnKEWLnFqHT6xSpz+FS5xah3YWIEVuoWc6lOvD6VUJPAK8JzW+lX//H50XL3bw+9eWcurq3Zw3YQB3HrGYOn/LITotHzp9aGAfwPfaq3/FviQWudye7jpxdW8vXYXN08axG8mDpQkLYTo1Hy5TcR44DJgolJqtfdxZoDjalZtvZvrnlvF22t38fszh3DDaVmSpIUQnZ4vvT4+BkKeDWtcbn7x7Eo++m4vs88exhUn9g11SEIIERRhcWViZW091zy9gs+/38cDP83mgjHHhDokIYQIGssn6vIaF1c99SWrt5fy8Pl5nDOiV6hDEkKIoLJ0oi6tquPyJ5ezfmc5j100gqnZmaEOSQghgs6yibrEWculT3zBlr2VPH7ZKE47Lj3UIQkhREhYMlHvLq/hkie+oPhAFf++cjQnZ6WGOiQhhAgZyyXqfdUezn/8M0oqapl/1fGM7Z8c6pCEECKkLJWot+2r5L4vaqjDzrPXnMDIY7qHOiQhhAg5yyTqA5V1nP/4Z9S6NQW/HMvwXomhDkkIISzBMom6e1wUV43vR3z5NknSQgjRhC+XkAfNL08dQO8ES4UkhBAhJ1lRCCEsThK1EEJYnCRqIYSwOEnUQghhcZKohRDC4iRRCyGExUmiFkIIi5NELYQQFieJWgghLE4StRBCWJwkaiGEsDhJ1EIIYXGSqIUQwuIkUQshhMVJohZCCIuTRC2EEBYniVoIISxOErUQQlicJGohhLA4SdRCCGFxbSZqpdSTSqk9Sql1wQhICCHEoXwpUc8HpgQ4DiGEEC2IaGsBrfVSpVTfIMRCQVEBG8o3sKtoF3ZlRymFDRs21cqjyXyllPkcyqyw8cm8UEod+h7VOK3B4csermH+5prNdNvdrdVlm6O1PnIaR05rabuHf4fG+U1iaPr9ttZuZV3JuiPmNZnQ7LzW1t9afL5qLt6ddTvZeGDjoXG08Pc5/G/b9HVL+6Lp51qNrY2F9tXvY4dzR/Prby3mZua1pLnjpL1K60vZXbm75W34cNwdjeb2Y3Pfv6y+jL1Ve5tfRzv+t1rSdF82/c4d2cdt7VObspEam9ru9bZF+RKsN1G/pbUe3soyM4GZAOnp6aMKCgraHczNP9yMS7va/TkhhLCCBFsC9/W5r0Ofzc/PX6m1Ht3cPL8l6qZGjx6tV6xY0a4gAcpqy/j4448Ze+JYNBqP9rT+4OBrrTVu7cajPcDBX86G79fS+2aXaaGk0XRfrVmzhtzc3JZLJZoWS2++ljRaiu/gU9vfZe3ateTk5BwRf0ufO3x6c++PDLD12Udsq4XtfvPNNwwdNrTl7R7+vZuJubUSky8lyLb+HzSaoqIihgwZ0urZUbPHWZN5vpQU23uWcrgNGzYwePDggG6jJc3t65b2/4YNGxg0aJBv623h76PRPp+5NHfG5au29mm0PZrpA6a3a51N4moxUbdZ9RFMidGJxNnjSI5JDnUobar9rpZxPceFOow2eTZ5OKX3KaEOwydRW6OY0HdCqMNo05LiJUwYOCHUYbRpyc4lTBg0IdRhtGnJziVMGDwh1GH4JFT7VLrnCSGExfnSPe8F4DNgsFKqWCl1deDDEkII0cCXXh8XBSMQIYQQzZOqDyGEsDhJ1EIIYXGSqIUQwuIkUQshhMVJohZCCIuTRC2EEBYniVoIISxOErUQQlicJGohhLA4SdRCCGFxkqiFEMLiJFELIYTFSaIWQgiLk0QthBAWJ4laCCEsThK1EEJYnCRqIYSwOEnUQghhcZKohRDC4iRRCyGExUmiFkIIi5NELYQQFieJWgghLE4StRBCWJwkaiGEsDhJ1EIIYXGSqIUQwuIkUQshhMVJohZCCIuTRC2EEBbnU6JWSk1RSm1QSm1SSt0W6KCEEEIc1GaiVkrZgbnAVGAocJFSamigAxNCCGH4UqI+Htiktd6ita4DCoCfBDYsIYQQDZTWuvUFlPoZMEVrfY33/WXACVrr6w9bbiYwEyA9PX1UQUFBhwJyOp3Ex8d36LPBJHH6X7jEKnH6V7jECYGNNT8/f6XWenRz8yJ8+LxqZtoR2V1rPQ+YB6CU2pufn7+tXVEelAKUdPCzwSRx+l+4xCpx+le4xAmBjfXYlmb4kqiLgT5N3vcGdrb2Aa11qm9xHUkptaKlXxUrkTj9L1xilTj9K1zihNDF6ksd9ZdAllKqn1IqCrgQeCOwYQkhhGjQZolaa12vlLoeeA+wA09qrb8JeGRCCCEA36o+0Fq/A7wT4FgazAvSdo6WxOl/4RKrxOlf4RInhCjWNnt9CCGECC25hFwIISxOErUQQlicJGohhLA4vyRqpdQ9SqlbWpl/TkfHB1FKnaKUWqWUqvdeJdl03hVKqY3exxVNpvdTSn3hnf6it1thWAnwPv2lUuprpdRqpdTHTdejlHJ7p69WSr3RZHpY79NA7k/v589XSq1XSn2jlHq+yXQ5Rju27mOUUoVKqa+UUmuVUmc2mdcpj9HWBKtEfQ5mQKeO+AG4Eni+6USlVA/gbuAEzHgkdyuluntnPwA8rLXOAg4AV3dw21Z2Dh3fp89rrbO11nnAg8Dfmsyr1lrneR9nN5ne2ffpOXRwfyqlsoDbgfFa62HATd7pcox2/Bi9E3hJaz0Cc+3GP5rM63LHaIcTtVLqDu/Qp4uBwd5p1yqlvlRKrVFKvaKUilVKnQicDTzk/QUc0NxyLW1Ha71Va70W8Bw26wzgfa31fq31AeB9YIpSSgETgZe9yz2NOWAsL4j7tLzJ2ziaGRLgsLjCcp8Ga38C1wJzvcchWus93ulyjHZ8n2qgm/d1Im1cDR3O+9QnWut2P4BRwNdALGZnbgJuAZKbLHMv8Bvv6/nAz5rMa3a5NrZ5+DpuAe5s8v4P3mkpmNH+Gqb3AdZ15HsG8xHsfQr8GtgMbAeymkyvB1YAnwPneKeF3T4N5v4EXsOcmXzi3W9T5Bg96n2a6d1WMaZ0PKqzHaPtefh0wUszTgYWaK2rAJrUEw1XSt0LJAHxmKsZm+Prcq1pabAonwaRsqCg7lOt9VxgrlLqYsxpZkP96TFa651Kqf7Ah0qpr4Hy5lbh6xcLkWDuzwggC5iAGQtnmVJqOHKMHq49+/QiYL7W+q9KqXHAs0qp4VprD53nGPXZ0dRRN7cT5gPXa62zgdmAo4XP+rpca1oaLKoESFJKRRw2PRyEYp8W0OQUUWu90/u8BVgCjCB892mw9mcx8LrW2qW1/h7YgEnccox2bDkw9csvAWitP/Mum+J935mOUZ90NFEvBc5VSsUopRKA6d7pCcAupVQkcEmT5Su882hjufZ4D5islOrubaCZDLynzXlPIdDQQ+QK4PUObiOYgrZPvY1fDc4CNnqnd1dKRXtfpwDjgfVhuk+DeYy+BuRD434bBGxBjtGj2ac/AKcBKKWOwyTqvZ3sGPVZhy8hV0rdAVwObMOUHNYDlcDvvNO+BhK01lcqpcYD/wJqMTtycnPLtbCdMcACoDtQA/yoTcs6SqmfA7/3LvpnrfVT3un9MSXFHsBXwKVa69oOfdEgCuI+/TtwOuDC1P9dr7X+xtsA9Dim4dYGPKK1/rf3M2G3T4O4PxXwV2AK4MYciwXeeXKMdmyfDvV+Nh5Tiv+d1npRZztGfSVjfQghhMXJlYlCCGFxHe314XfeU6oZh03+r9b6z6GIpzOQfepfsj/9T/apb6TqQwghLE6qPoQQwuIkUQshhMVJohZCCIuTRC2EEBb3//eXhcHfGWa8AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "q1_data=pd.read_csv(\"q1_data.csv\")\n",
    "model = linear_model.Lasso(alpha=0.01,max_iter=10000) \n",
    "test_ratio=0.2\n",
    "\n",
    "names,rmses_train,rmses_test,R2,intercepts,coefficients = q1_c_compare_RMSE(q1_data,model,test_ratio=0.1)\n",
    "index= q1_c_plot_RMSE(names,rmses_train,rmses_test,R2)\n",
    "\n",
    "coefficient_length= range(len(coefficients[index]))\n",
    "print('f(x)= ',intercepts[9],)\n",
    "for i in coefficient_length:\n",
    "        print('+',coefficients[index][i],'*',f\"X{i}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Q2\n",
    "\n",
    "In the given q2 _data.csv, x is the only feature and y is the target variable.\n",
    "\n",
    "**a (20 pts).** \n",
    "\n",
    "    Fit the best possible linear regression model to predict y in terms of x. You may apply polynomial expansion (try up to degree=15) but do not use regularization. If you find several models providing almost the same performance, choose the least complex model among them. When necessary you can use (%70 train - %30 test) or (%60 train - %20 validation - %20 test) splits. Report the model performance on the train/validation/test sets. Give the regression equation for the best model as y = f(x)=c0 + c1*x + c2*x^2 + ... ck*x^k.\n",
    "\n",
    "**b (20 pts).**\n",
    "    \n",
    "    Use the regression equation given in part (a) to find y value for x=1. Assume this is the correct value returned by the population function that is approximated by f(x) found in part (a). Then, apply the following procedure to fit several models with degree 1..0 by taking a sample of size 20 from the given dataset and predict y at x = 1:\n",
    "\n",
    "    for d in range(1,11):\n",
    "        add polynomial features up to degree d\n",
    "        pred = []\n",
    "        for i in range(1000):\n",
    "            take a sample of size 20 rows from the dataset with added polynomial features\n",
    "            fit a linear regression model (do not apply train/test split)\n",
    "            predict the value of y at x = 1 \n",
    "            append y to pred list\n",
    "        find and print d, and the mean and the standard deviation of pred list (i.e., 1000 predictions for degree=d)\n",
    "          \n",
    "    Compare the mean prediction found with the value returned by the function found in part (a) and standard deviations of the predictions made for different degrees. Comment on the obtained results.   \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [],
   "source": [
    "## This function generates polynomial and interaction features and add the data then fits polynomial regression\n",
    "def fit_poly(degree, data, model=LinearRegression(), test_size=0.3, plot=False):\n",
    "    poly = PolynomialFeatures(degree=degree, include_bias=False)  \n",
    "    pf = poly.fit_transform(data[['x']])  ## creates polynomial features with given degree\n",
    "    X = pd.DataFrame(pf, columns=poly.get_feature_names())\n",
    "\n",
    "    X_train, X_test, y_train, y_test = train_test_split(X,data['y'],test_size=test_size,random_state=2532810) ##split data as train and test\n",
    "    model.fit(X_train, y_train)\n",
    "    if plot:\n",
    "        x_m = np.linspace(data.x.min(), data.x.max(), 1000)\n",
    "        y_m = []\n",
    "        for x_ in x_m:\n",
    "            y_ = model.intercept_\n",
    "            for j,c in enumerate(model.coef_):\n",
    "                y_ += c * x_**(j+1)\n",
    "            y_m.append(y_)\n",
    "        plt.plot(x_m, y_m, color='black') ## shows model with black line\n",
    "        plt.scatter(X_train.x0,y_train, c='blue') ## shows train data with blue dots\n",
    "        plt.scatter(X_test.x0,y_test,c='red') ## shows test data with red dots\n",
    "\n",
    "    train_error = metrics.mean_squared_error(y_train,model.predict(X_train))\n",
    "    test_error = metrics.mean_squared_error(y_test,model.predict(X_test))\n",
    "    R2 = model.score(X_train,y_train)\n",
    "\n",
    "    return R2, train_error, test_error, model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### a (20 pts) \n",
    "    Fit the best possible linear regression model to predict y in terms of x. You may apply polynomial expansion (try up to degree=15) but do not use regularization. If you find several models providing almost the same performance, choose the least complex model among them. When necessary you can use (%70 train - %30 test) or (%60 train - %20 validation - %20 test) splits. Report the model performance on the train/validation/test sets. Give the regression equation for the best model as y = f(x)=c0 + c1*x + c2*x^2 + ... ck*x^k.\n",
    "    \n",
    "    \n",
    "**Answer**\n",
    "\n",
    "As given in results:\n",
    "We can choose degree 5 and get the error metrics:\n",
    "**R2=0.9985115260541232\n",
    "**Error(train)=0.0006348986285034617\n",
    "**Error(test)=0.0007280924136923384\n",
    "\n",
    "Model is\n",
    "**y=f(x)=13.944644570684666+ 16.150233800371456*X^1+ (-64.46420135366411)*X^2+ 88.61485475949736*X^3+ (-49.99899308237356)*X^4+ 9.997891666945343*X^5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>R2</th>\n",
       "      <th>Error(train)</th>\n",
       "      <th>Error(test)</th>\n",
       "      <th>model</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.221646</td>\n",
       "      <td>0.332002</td>\n",
       "      <td>0.323206</td>\n",
       "      <td>LinearRegression()</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.647869</td>\n",
       "      <td>0.150199</td>\n",
       "      <td>0.135917</td>\n",
       "      <td>LinearRegression()</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.649417</td>\n",
       "      <td>0.149539</td>\n",
       "      <td>0.133149</td>\n",
       "      <td>LinearRegression()</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.649420</td>\n",
       "      <td>0.149537</td>\n",
       "      <td>0.133110</td>\n",
       "      <td>LinearRegression()</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.998512</td>\n",
       "      <td>0.000635</td>\n",
       "      <td>0.000728</td>\n",
       "      <td>LinearRegression()</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.998513</td>\n",
       "      <td>0.000634</td>\n",
       "      <td>0.000727</td>\n",
       "      <td>LinearRegression()</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.998513</td>\n",
       "      <td>0.000634</td>\n",
       "      <td>0.000726</td>\n",
       "      <td>LinearRegression()</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.998515</td>\n",
       "      <td>0.000633</td>\n",
       "      <td>0.000727</td>\n",
       "      <td>LinearRegression()</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0.998515</td>\n",
       "      <td>0.000633</td>\n",
       "      <td>0.000727</td>\n",
       "      <td>LinearRegression()</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0.998520</td>\n",
       "      <td>0.000631</td>\n",
       "      <td>0.000730</td>\n",
       "      <td>LinearRegression()</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>0.998534</td>\n",
       "      <td>0.000625</td>\n",
       "      <td>0.000752</td>\n",
       "      <td>LinearRegression()</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>0.998541</td>\n",
       "      <td>0.000622</td>\n",
       "      <td>0.000758</td>\n",
       "      <td>LinearRegression()</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>0.998541</td>\n",
       "      <td>0.000622</td>\n",
       "      <td>0.000759</td>\n",
       "      <td>LinearRegression()</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>0.998550</td>\n",
       "      <td>0.000618</td>\n",
       "      <td>0.000767</td>\n",
       "      <td>LinearRegression()</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>0.998550</td>\n",
       "      <td>0.000618</td>\n",
       "      <td>0.000766</td>\n",
       "      <td>LinearRegression()</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          R2  Error(train)  Error(test)               model\n",
       "1   0.221646      0.332002     0.323206  LinearRegression()\n",
       "2   0.647869      0.150199     0.135917  LinearRegression()\n",
       "3   0.649417      0.149539     0.133149  LinearRegression()\n",
       "4   0.649420      0.149537     0.133110  LinearRegression()\n",
       "5   0.998512      0.000635     0.000728  LinearRegression()\n",
       "6   0.998513      0.000634     0.000727  LinearRegression()\n",
       "7   0.998513      0.000634     0.000726  LinearRegression()\n",
       "8   0.998515      0.000633     0.000727  LinearRegression()\n",
       "9   0.998515      0.000633     0.000727  LinearRegression()\n",
       "10  0.998520      0.000631     0.000730  LinearRegression()\n",
       "11  0.998534      0.000625     0.000752  LinearRegression()\n",
       "12  0.998541      0.000622     0.000758  LinearRegression()\n",
       "13  0.998541      0.000622     0.000759  LinearRegression()\n",
       "14  0.998550      0.000618     0.000767  LinearRegression()\n",
       "15  0.998550      0.000618     0.000766  LinearRegression()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAApkUlEQVR4nO3de3xU5bXw8d/KJCEhgQQIRCAREAEFyi1cvGIQW1GplwrHW/FClaNHarH1HHmP56319W5tT6vS8lqqvrUotbWHUopitQS0tZIoF7kFUggQQBJIIAQISWbW+8dM0hACCTN7GPfs9f185jOzZz+zZmWSWbPzzLOfR1QVY4wx7pcQ6wSMMcY4wwq6McbECSvoxhgTJ6ygG2NMnLCCbowxccIKujHGxIk2C7qIvCIi5SKy7iT7RUReEJESEVkrIqOcT9MYY0xb2nOE/how6RT7rwIGhC4zgJ9HnpYxxpjT1WZBV9UVQOUpmlwH/EqD/g5kikhPpxI0xhjTPokOxOgN7Gy2XRa6b0/LhiIyg+BRPKmpqXm5ublhPWEgECAhwfnuf6/HPdKglB+xM4eNibbOSdA1Nbz37ubNm/epavfW9jlR0KWV+1qtCqr6MvAywOjRo7WoqCisJywoKCA/Pz+sx1rck3vwN6t5b90uvnflYMdiNiopKeHcc8/1dNxo5jpgQJTiRiHfLVGIG42Y0YxbX76Ve26YGNZjRWT7yfY5UdDLgOaH2jnAbgfimjMoEFCWb65gWJaP6Zf0czx+QcN28j0eN6q5XhyFuPXuieumXAEKCk5akyPixP/ri4DbQ6NdLgAOquoJ3S3my23troNUHq5jWHcnPuONMbHQ5rtXRN4E8oEsESkDHgWSAFR1LrAEuBooAY4Ad0UrWRM9BcXliMDQLF+sUzHGhKnNgq6qt7SxX4H7HcvIxERBcQXDczLplFwf61TMl1x9fT1lZWXU1taG9fiMjAw2btzoaE7RiBnruCkpKeTk5JCUlNTuuPb/tWF/zTHWlB1g1sSBwK5Yp2O+5MrKyujUqRN9+/ZFpLUxEad26NAhOnXq5GhO0YgZy7iqyv79+ykrK6Nfv/b34dup/4YPt+xDFfIHtToSypjj1NbW0q1bt7CKuWkfEaFbt26n/V+QFXRDQXE53dKS+UrvjFinYlzCinn0hfMaW0H3OH9AWbFlH5cN7E5Cgr1JjXEzK+get7bsAJWH67jMuluMi/h8PkaMGMHQoUP5+te/zoEDBwBYvXo1F154IUOGDGHYsGH85je/iW2iZ5gVdI8rKK4gQWD8ACvoxj1SU1NZvXo169ato2vXrvziF78AoGPHjvzqV79i/fr1vPvuu8yaNaup2HuBFXSPK9hcwYjcTLqkJcc6FWPCcuGFF7J7d/Dk9IEDBzJgwAAAevXqRY8ePaioqIhlemeUDVv0sP01x1hbdoAHrxgY61SMSz32x/Vs2F19Wo/x+/34fCc/gW1wr848+vUh7Y71wQcfcMstJ54us3LlSurq6ujfv/9p5edmdoTuYSu2VNhwReNKR48eZcSIEXTr1o3Kykouv/zy4/bv2bOHadOm8eqrr0ZlptMvKztC97CC4gqy0pMZ2suGK5rwtPdIujknTtZp7EM/ePAgkydP5uWXX+Y//uM/AKiuruaaa67hiSee4IILLojoedzGOx9d5jj+0OyK4224onGxjIwMXnjhBV588UXq6+upq6vjhhtu4Pbbb2fq1KmxTu+MsyN0j1pTdoADR+rJH9Qj1qkYE5GRI0cydOhQFixYgIiwYsUK9u/fz2uvvQbAa6+9xogRI2Ka45liBd2j/jlcMSvWqRhz2mpqao7bfuutt5q6cb75zW/GIqUvBety8aiC4nJGnt2FzI42XNGYeGEF3YP21RxjbdlB8gfa6BZj4okVdA9asTl4osWE86z/3Jh4YgXdg5YVV5CV3oHBPTvHOhVjjIOsoHuMP6B8uKXCZlc0Jg5ZQfeY1TuDwxUnnGf958bEGyvoHlNQXE6CwKXnWkE3Jt5YQfeYguIKRp3dhYyO7V941pgvm8b50BsvP/7xjx1/jilTprBt2zYAnnrqqbBi3H333WzYsOGUbV566SVeffXVsOK3ZCcWeUjFoWN8vusgD33NZlc07tY4l0ujQ4cOndCm5ayObc3y2Lzdpk2b8Pv9TQs0P/XUU/znf/7nCW1VFVU96QRg8+bNa/P5pk+fzsUXX8xdd93VZtu2WEH3kOWh4Yp2ur9xzDuz4YvPT+shqf4G8J2i9Jz1FbjqmbDS6du3L9OnT+e9995j5syZzJ49+7htVeWpp55CVbnmmmt49tlnAUhPT+e73/0uS5cu5Uc/+hFLlizhuuuuA2D27NlNszsOGTKEJ598kquuuooJEybw8ccfs3DhQp555hkKCws5evQoU6ZM4bHHHgMgPz+f559/ntGjR5Oens53vvMdFi9eTHJyMosXLyY7O5uOHTvSt29fVq5cydixY8P6uRtZl4uHFBSX071TB4b0suGKxt0aC2zj5e23327al5KSwkcffcTNN9983Pb48eN5+OGH+ctf/sLq1aspLCxk4cKFABw+fJihQ4fyySefcMkll/DXv/6VvLw8AJ555pmm/wjmz58PQHFxMbfffjurVq2iT58+PPnkkxQVFbF27VqWL1/O2rVrT8j58OHDXHDBBaxZs4aLL764aZUlgNGjR/Phhx9G/LrYEbpHNPgDfLhlH18dnG0rthvnhHEkfdTB6XMbNe9yuemmm45r27hdWFhIfn4+3bsHBwTcdtttrFixguuvvx6fz8eNN97Y9Jg9e/Y0tWtNnz59jpua96233uLll1+moaGBPXv2sGHDBoYNG3bcY5KTk5k8eTIAI0aM4KOPPmra16NHDzZt2tTeH/+krKB7xOqdBzh4tJ4J1t1i4lxaWlqr26p60sekpKQc17+emppKbW3tCbFae45t27bx/PPPU1hYSJcuXbjzzjupra094TFJSUlNB1M+n4+GhoamfbW1taSmprbjpzs163LxiILiCnwJwiU2u6LxqHHjxrF8+XL27duH3+/nzTff5LLLLmu17fnnn09JSUnTdlJSEvX19a22ra6uJi0tjYyMDPbu3cs777xz2rlt3ryZoUOHnvbjWrKC7hEFm8sZdXYmGak2XNG4X8s+9EcffbTNx/Ts2ZOnn36aCRMmMHz4cEaNGtX0xWdL11xzDQUFBU3bM2bMYNiwYdx2220ntB0+fDgjR45kyJAhTSNWTtdf//pXrrjiitN+XEvW5eIB5YdqWbermn+/clCsUzHGEX6//7jtxj700tLS4+5vuX3rrbdy6623nhCv5fzqU6ZMYcKECTz00EMAPPvss00jYgDWrVt3XPvGxTRaav6h0Pw5rr/+eqZNmwbAqlWrGDJkCFlZkf/3bEfoHrC8uHG4op0dakx7pKam8thjj7F79+6oP9e+fft4/PHHHYllR+geULC5gh6dbHZFY07HlVde2eoJS0776le/6lgsO0KPcw3+AB9uriB/UHcbrmhMnLOCHudW7TxAdW2DnR1qjAdYQY9zBcXl+BKEi8+14YrGxDsr6HFu2aYK8vp0seGKxnhAuwq6iEwSkWIRKRGR2a3szxCRP4rIGhFZLyKRTxtmIlZeXcuGPdU2usXEHbdMnwvBIY3NR8vcfPPNx5205KQ2C7qI+IA5wFXAYOAWERncotn9wAZVHQ7kAz8SkWSHczWnqaBxdsWB1n9u4kvjXC6Nl+9+97sntGk5Vr3l9sn4/X7Wr19/wvS54WpZ0O+77z5++tOfhh3vVNozbHEsUKKqWwFEZAFwHdB81nYFOklwGEU6UAk0tAxkzqyC4nKyO3fg/J6RTYRkzMk8u/JZNlWe3qRSbc1Lfl7X83h47MNh5XMmps+dP38+v/71r3nhhReoq6tj3Lhx/OxnPwPgW9/6FkVFRYgI06dPJzc3l6KiIm677TZSU1P5+OOPufTSS7njjjtoaGggMdHZkePtidYb2NlsuwwY16LNS8AiYDfQCbhJVQMtA4nIDGAGQHZ29nFnUZ2OmpqasB/rlbj+gLJs4xHGnJXI8uXLHYkZCYvrrlxPFTcjI6NpfHZdXV27j3wbqeopH1NXV9fm+O+jR48eN5vhrFmzmDp1KqqKiDTNp/Lwww83be/Zs4eJEyeyYsUKMjMzuf7663nzzTeZPHkyhw8fpn///rz//vtAsIhfe+21+P1+HnnkEV566aWm6W2LioqYP38+7777LklJSTz44IPMmzeP888/nx07dvDxxx8DcODAATIzMxk5ciRPPPEEo0aNoqGhgYaGBvr168ff/vY3Ro4cecqfs7a29vR+t40rbpzsAkwF5jXbnga82KLNFOC/AQHOBbYBnU8VNy8vT8O1bNmysB/rlbifbN2vfR5erEvW7nYsZiQsrrtyPVXcDRs2RBS3uro6oserqqalpbUas0+fPlpaWtp0f/PthQsX6rRp05r2zZs3Tx988EFVVfX5fNrQ0NC0b8CAAbpnz56muM2f78UXX9SePXvq8OHDdfjw4Tpw4EB99NFHtbKyUs855xydOXOmvvPOO+r3+1VV9bLLLtPCwsLj8p06daouWrSozZ+ztdcaKNKT1NX2HKGXAbnNtnMIHok3dxfwTOjJSkRkG3AesLL9Hy3GSQXF5SQmCBfb7IrGY6I9fa6qcscdd/D000+fsG/NmjUsXbqUOXPm8NZbb/HKK6+0+nzHjh1zZLrcltozyqUQGCAi/UJfdN5MsHuluR3ARAARyQYGAVudTNScnmXFweGKnVNsuKIx4Nz0uRMnTuR3v/sd5eXlAFRWVrJ9+3b27dtHIBDgxhtv5PHHH+ezzz4DoFOnTid0IZWUlDBkyBDHf8Y2j9BVtUFEZgJLAR/wiqquF5F7Q/vnAo8Dr4nI5wS7XR5W1X2OZ2vaZW91LRv3VPPwpPNinYoxUdH4JWWjyy+/vM2hi82nz1VVrr766janzx03Lvh1YeP0uaNGjWL+/Pk88cQTfO1rXyMQCJCUlMScOXNITU3lrrvuIhAIfn3YeAR/5513cu+99zZ9KVpdXU1KSgo9e/Z04JU4Xru+YlXVJcCSFvfNbXZ7N/A1Z1Mz4WqcXXHCeTb+3MSnWE+fe9NNN52w1B3QdFTe3I033njc8nZz585l+vTprfxUkbMzRePQsuJyzuqcwqBsG65oTDiiOX1uZmZmqx8qTrCCHmfq/QE+2rLPZlc0UXWqLxjjxZVXXklubm7bDU/TXXfd1a7x5+G8xlbQ48xn26s4dMxmVzTRk5KSwv79+z1R1GNFVdm/fz8pKSmn9Thb4CLOLCuuCA5XPLdbrFMxcSonJ4eysjIqKirCenxtbe1pF6pYxIx13JSUFHJyck4rrhX0OFNQXM7ovl3oZMMVTZQkJSU1zXESjoKCgjbPkPwyxHRjXOtyiSNfHKxl0xeHmGDdLcZ4khX0OFJQHDzRwfrPjfEmK+hxpKC4gp4ZKQzMTo91KsaYGLCCHifq/QE+KtlH/qAeNlzRGI+ygh4nikqrqDnWYKsTGeNhVtDjRMHmcpJ8thi0MV5mBT1OLC+uYEzfrqR3sJGoxniVFfQ4sPvAUTZ9cci6W4zxOCvocWB542LQNlzRGE+zgh4HCorL6ZWRwoAeNlzRGC+zgu5ydQ2h2RXPs+GKxnidFXSXK9peyeE6P/kDrf/cGK+zgu5yy4srSPIJF9lwRWM8zwq6yy0rLmdsPxuuaIyxgu5q+48G2Ly3hvyBNrrFGGMF3dU+3xdcKNfGnxtjwAq6q62p8NM7M5VzbbiiMQYr6K5V1xBg436/LQZtjGliBd2likorqfXb2aHGmH+ygu5Sy4rLSRS4qL8tBm2MCbKxbmFSVfwBpSHQ8joQvPb/8/7dNQFKyg85+vx/2VTOoK4JpNlwRWNMiFWDkD+s3sVzHx2hQ1HBicU5oPj9/yza/lAxPy0frXA851vOS3Y8pjHGvaygh7y5cgeH6mDkOZ1JTBB8CQnBa5+EtuX4+xu3fSe5P0FI9AkJImzauJHBgwc7mm+ST0jYu8nRmMYYd7OCTnDEyOqdBxjfy8dLt45yPH7GgS3kD+/leNyCfcWOxzTGuJd9KQqs332Q2voAA7r4Yp2KMcaEzQo6wQWWAQZaQTfGuJh1uQCFpZX0y0ojo0OsMzHGmPB5/ghdVSnaXsXoPl1inYoxxkTE8wV9677DVB6uY0zfrrFOxRhjItKugi4ik0SkWERKRGT2Sdrki8hqEVkvIsudTTN6CrdVAjC6rx2hG2Pcrc0+dBHxAXOArwJlQKGILFLVDc3aZAI/Ayap6g4Rcc0EI4WlVXRLS6ZfVho7Yp2MMcZEoD1H6GOBElXdqqp1wALguhZtbgV+r6o7AFS13Nk0o6doeyWj+3axGQuNMa4nqqc+hV1EphA88r47tD0NGKeqM5u1+QmQBAwBOgE/VdVftRJrBjADIDs7O2/BggVhJV1TU0N6euRzgB+oDTCr4Cg3D0pmUr8kx+K25Ka4bsrVbXHdlKvb4rop10jjTpgw4VNVHd3qTlU95QWYCsxrtj0NeLFFm5eAvwNpQBawBRh4qrh5eXkarmXLloX92Ob+tHa39nl4sa7aUeVo3JbcFNdNubotrptydVtcN+UaaVygSE9SV9szDr0MyG22nQPsbqXNPlU9DBwWkRXAcGBzez5xYqWwtJKUpASG9Ooc61SMMSZi7elDLwQGiEg/EUkGbgYWtWjzB+BSEUkUkY7AOGCjs6k6r6i0ipG5XUjyeX70pjEmDrRZyVS1AZgJLCVYpN9S1fUicq+I3BtqsxF4F1gLrCTYRbMuemlHruZYA+t3H2SMDVc0xsSJdp36r6pLgCUt7pvbYvuHwA+dSy26Vu84QEBhtJ1QZIyJE57taygsrSRBYOTZmbFOxRhjHOHZgl60vZLze3amU0pSrFMxxhhHeLKg1/sDrNpxwOZvMcbEFU8W9I17qjlS57f5W4wxccWTBb0wtKDF6D52hG6MiR+eLOhFpZXkdk3lrIyUWKdijDGO8VxBV1UKS6sYY0fnxpg447mCvn3/EfbVHGNMPyvoxpj44rmCvrI0uKCFnSFqjIk3nivoRaWVdOmYRP/uzk+JaYwxseTBgl5FXp+utqCFMSbueKqg76s5xtZ9h627xRgTlzxV0Isax5/bGaLGmDjksYJeSYfEBIb2tgUtjDHxx1MFvXB7FcNzM+mQ6It1KsYY4zjPFPQjdQ2s32ULWhhj4pdnCvrqnQdoCKj1nxtj4pZnCnpRaRUiMOpsO0I3xsQnzxT0wtJKBmV3IiPVFrQwxsQnTxT0Bn+Az7ZX2YIWxpi45omCvumLQxy2BS2MMXHOEwW9qGlCLjtCN8bEL08U9MLtVfTOTKVXZmqsUzHGmKiJ+4KuqhSVVlp3izEm7sV9Qd9ZeZS91cds/LkxJu7FfUEvtAUtjDEeEfcFvWh7JZ1TEhnYo1OsUzHGmKiK+4JeWFrF6L5dSUiwBS2MMfEtrgt65eE6Sspr7AtRY4wnxHVB/3R7cEELG39ujPGCuC7oRaWVJPsS+ErvjFinYowxURfXBb2wtJJhORmkJNmCFsaY+Be3Bb223s/nuw7a+HNjjGfEbUFfs/MA9X618efGGM+I24JeFPpCNK+PFXRjjDe0q6CLyCQRKRaREhGZfYp2Y0TELyJTnEsxPIWllQzMTiezY3KsUzHGmDOizYIuIj5gDnAVMBi4RUQGn6Tds8BSp5M8Xf6A8un2Kus/N8Z4SnuO0McCJaq6VVXrgAXAda20+zbwNlDuYH5h2bz3EIdqG6z/3BjjKaKqp24Q7D6ZpKp3h7anAeNUdWazNr2BN4DLgV8Ci1X1d63EmgHMAMjOzs5bsGBBWEnX1NSQnp5+0v0f7Kjn9Q11/HB8Kt07tv9rgrbihstNcd2Uq9viuilXt8V1U66Rxp0wYcKnqjq61Z2qesoLMBWY12x7GvBiiza/BS4I3X4NmNJW3Ly8PA3XsmXLTrn/2298puOefF8DgYCjccPlprhuytVtcd2Uq9viuinXSOMCRXqSuprYjg+EMiC32XYOsLtFm9HAAhEByAKuFpEGVV3YjviOa1zQIpSPMcZ4QnsKeiEwQET6AbuAm4FbmzdQ1X6Nt0XkNYJdLgudS7P9yqqOsPtgLf9qX4gaYzymzYKuqg0iMpPg6BUf8IqqrheRe0P750Y5x9NSVBocf24zLBpjvKY9R+io6hJgSYv7Wi3kqnpn5GmFr7C0kvQOiZx3VudYpmGMMWdc3J0pWlRaxag+XfDZghbGGI+Jq4J+8Eg9xXsPMcZO9zfGeFBcFfRPdwQXhLYzRI0xXhRXBb2wtIoknzAiNzPWqRhjzBkXVwW9qLSSob0zSE22BS2MMd4TNwW9tt7Pmp0Hbf1QY4xnxU1BX7frIHX+AKPtC1FjjEe5r6Cr0ql6ywl3F5baghbGGG9zX0Ff9Tp5nz0EOwuPu7uotJL+3dPolt4hRokZY0xsua6g+8+/jmWdsuC9/4LQ1L+BgFK0vcr6z40xnua6gr5w5/s8kNWRTypWwabFAJRU1HDwaL2NPzfGeJrrCvrk/pPJ9GXy0x5noe8/Cv56CkuDJxTZCkXGGC9zXUHv4OvA1ZlX87lPef/obvj0NYpKq+jeqQNnd+0Y6/SMMSZmXFfQAcamjeWcjHN4oUdPGgqeYd3WMsbYghbGGI9zZUH3iY8HRj1AqTTwh4SjXHv4t4zuY/3nxhhvc2VBB7g893KGdR/Gi1nZfDNxCRd2r4t1SsYYE1OuLegiwqxRs9gvDbzdOZVBG1+IdUrGGBNTri3oAGPOGkNK/RBe7tKVmrVvwN71sU7JGGNixtUFvbq2nsqyiRxN8PNKtyz48/djnZIxxsSMqwv6Z9ur8Nf2YkzWROZ3SqN82zL4x7JYp2WMMTHh6oJeVFqFL0GYfcEsGkSY26M3/Pl/QyAQ69SMMeaMc3VBLyytZEivzgzs1pepA6fy+xShdN9G+PytWKdmjDFnnGsLel1DgNU7DzSNP58xbAbJiSm81KsPfPA41B+NcYbGGHNmubagr9t9kGMNAcb2C87fkpWaxe2Db2epr571tXvhk7kxztAYY84s1xb0otCEXHnNzhC9c8idZHbI5Cc558KHP4bD+2OVnjHGnHGuLeiFpVX0y0qje6d/LmiRnpzOPV+5h7/rYT5OqIcVP4xhhsYYc2a5sqCrKkWlla2uH3rTeTfRM60nP+19Dlo4Dyq3xiBDY4w581xZ0PccVqqO1Le6QlEHXwfuH3E/6/2HeC89Dd5/LAYZGmPMmefKgr6lyg/A6JMsaDH5nMmcm3kuL2b3pn7DwhPWHzXGmHjkyoK+uSpAt7Rk+mWltbrfl+DjgZEPsL3hEAu79Txu/VFjjIlXLi3ofka3saBFfm4+I7qP4OddMzla9gls+tMZzNAYY8481xX0vdW1VBzVVvvPmxMRZuXNoqLhMG/07Aeh9UeNMSZeua6gF5VWATC6jYIOkJedx/ic8fyyYxIHq/4Bn74W5eyMMSZ2XFfQ8/p04c4hyQzp1bld7R8Y+QA1/lp+efZgKHgGaqujnKExxsRGuwq6iEwSkWIRKRGR2a3sv01E1oYufxOR4c6nGnRWRgr5uUkk+dr3WTSo6yCuOeca3vDVsvdYFfz1p9FKzRhjYqrNqigiPmAOcBUwGLhFRAa3aLYNuExVhwGPAy87nWgk7h9xP36Un/cbDh/PgerdsU7JGGMc157D3LFAiapuVdU6YAFwXfMGqvo3Va0Kbf4dyHE2zcjkdMrhpkE3sTBQybYEhWVPxjolY4xxnGgb47NFZAowSVXvDm1PA8ap6syTtH8IOK+xfYt9M4AZANnZ2XkLFiwIK+mamhrS09NP6zGH/Id4bNdjjPGn8LPt6yga/RMOp/eNOG57uCmum3J1W1w35eq2uG7KNdK4EyZM+FRVR7e6U1VPeQGmAvOabU8DXjxJ2wnARqBbW3Hz8vI0XMuWLQvrcXNWzdGhrw3Vz5/vq/r6NxyL2xY3xXVTrm6L66Zc3RbXTblGGhco0pPU1fZ0uZQBuc22c4ATOqFFZBgwD7hOVb+U89beMeQOunTowk9yB6Il79v6o8aYuNKegl4IDBCRfiKSDNwMLGreQETOBn4PTFPVzc6n6Yy0pDRmDJvBJ0d383G3s+HP37f1R40xcaPNgq6qDcBMYCnB7pS3VHW9iNwrIveGmn0f6Ab8TERWi0hR1DKO0L8M+hd6p/fmJ9lnEfhira0/aoyJG+0azK2qS1R1oKr2V9UnQ/fNVdW5odt3q2oXVR0RurTeYf8lkOxL5v4R97Px6Be81/t8W3/UGBM3XHemqBOu7nd1cHrdzh2pry6z9UeNMXHBkwXdl+Bj1qhZ7Kit4H/6j7H1R40xccGTBR1gfM54RvUYxc+TjnG0/rCtP2qMcT3PFvTG6XX3HTvA/EGXQOE8Uo7uiXVaxhgTtsRYJxBLI3uMJD8nn1f2FjI1MZkh65+D+o9BEkAkdJ0ANLst8s99x93f2mOC27k7tsFHq5xNPjGFxPrcttsZYzzD0wUd4IFRD3Djohv55bAr+bfPl8PmpaABQIPXGgguX6fNtk/Y1+z+VvQH2Op87ud1GwdXXBP84DDGeJ7nC/qALgP4ev+v80bpUvqN+RE3XHFDZAG1ZbH3s2LFCsaPH+9Mwo0++b9kvf8ofP5bGPYvzsY2xriS5ws6BKfXfWfbO/x6/6/Z//l+khKS6ODrQLIvmaSEJJJ9ySQnJAevT3Jf8+2khCQkwdcUP+DrAEmpziZ90bc5uPINMpb8O/S9FDr3dDa+McZ1rKADvdJ7cd/w+3hh1Qts/syZmQuaF31/g5/kt5Lb9Tihfd0nyb5kbuw3mXvWz4HFs+CWBdb1YozHWUEPuWfYPfTf35+LLr2IukAddf466v31TbfrAqHt0O3jrkOX+kD9CffXB+op21VGz15tH0FrG1MZN/dZ+Wf8onoxky59kNxlT8OaN2HErZG8BMYYl7OC3kyCJJCSmEIKKY7GLSgoIP/CfEdjlh0q4xsLv8FD1at4/ewLSX5nNvS7DDJ6O/o8xhj38Ow4dLfL6ZTDbd1uY0PlBp7vPxwC9fDHB4JfxBpjPMkKuosN7zic2wffzpulS1g69ptQ8j6sej3WaRljYsQKusvNypvFsO7DeLTiQ7b3vQCWPgIHdsY6LWNMDFhBd7mkhCSeH/88iQmJPJSRQm3AD4u+bV0vxniQFfQ40DO9J09d8hSbqrfy7NDLYOsy+PS1WKdljDnDrKDHifE545k+dDq/O7COP/XLg/f+C6q2xzotY8wZZAU9jnx75LcZ1WMUj/kOsTUxARbNtDVTjfEQK+hxJDEhkefGP0dqUke+d/a5HC39EIp+Geu0jDFniBX0OJOdls3TlzzNP47t56m+g+HP34fKbbFOyxhzBlhBj0MX9b6IGcNmsJBDLEzrCH+437pejPEAK+hx6r7h9zH2rLE82bUzW3avhJUvxzolY0yUWUGPU74EH8+Of5a0Dhl8L6cPRz74Aez/R6zTMsZEkRX0OJaVmsVz459jOw38n24Z6ML7IOCPdVrGmCixgh7nxvYcy30j7uNPqcm8fWAdfDI31ikZY6LECroH3POVe7iw54U8nZXFphVPwb4tsU7JGBMFVtA9wJfg4+lLnyYzpSvfy8qkZuG/WteLMXHICrpHdEvtxnP5P2JXoo8fHCtF//ZirFMyxjjMCrqH5GXnMXPkAyxNT+M3hf8N5ZtinZIxxkFW0D1m+lemc+lZ43iuSyfW/+Ee8DfEOiVjjEOsoHtMgiTw1GXP0y05k+/JPqo/fC7WKRljHGIF3YMyUzL54RUvsTcpke9vfBX9Yn2sUzLGOMAKukeN6DGCWcPu44OOKfx68XTw18c6JWNMhKyge9jtI+5jQub5/DjxMGvefyTW6RhjImQF3cNEhMcn/YLshA78e9kfObjj41inZIyJQLsKuohMEpFiESkRkdmt7BcReSG0f62IjHI+VRMNGR0yeP7yF6jw+Xjk/X9D/cdinZIxJkyJbTUQER8wB/gqUAYUisgiVd3QrNlVwIDQZRzw89C1cYGhORfzUN9reWb7Hzl7x084vzzH8eeoObqdL8qd//I1unE3tN0wxjEBDkXpNXBT3ENHS9lTvs7RmNGMe6yu0vGY0I6CDowFSlR1K4CILACuA5r/ZV4H/EpVFfi7iGSKSE9V3eN4xiYqbr3sST5942+83lDG6+/cHJ0neSc6YV0V1025ui2ui3KdEjiLK/mG43HbU9B7AzubbZdx4tF3a216A8cVdBGZAcwIbdaISPFpZftPWcC+MB9rcc98TIsbvZgWN3oxoxZ3HeuyfjBdwo3b52Q72lPQpZX7NIw2qOrLQMRL54hIkaqOjjSOxT0zMS1u9GJa3OjFdGPc9nwpWgbkNtvOAXaH0cYYY0wUtaegFwIDRKSfiCQDNwOLWrRZBNweGu1yAXDQ+s+NMebMarPLRVUbRGQmsBTwAa+o6noRuTe0fy6wBLgaKAGOAHdFL2XAgW4bi3tGY1rc6MW0uNGL6bq4EhyYYowxxu3sTFFjjIkTVtCNMSZOuKqgi8grIlIuIo6euiUiuSKyTEQ2ish6EfmOAzFTRGSliKwJxXzMiVybxfeJyCoRWexgzFIR+VxEVotIkYNxM0XkdyKyKfQaXxhhvEGhHBsv1SIyy6FcHwz9vtaJyJsikuJQ3O+EYq6PJNfW3gMi0lVE/iwiW0LXXRyKOzWUb0BETnuI3Uli/jD0d7BWRP5HRDIdivt4KOZqEXlPRHo5EbfZvodEREUky6F8fyAiu5r9DV99unFbpaquuQDjgVHAOofj9gRGhW53AjYDgyOMKUB66HYS8AlwgYM5fxd4A1jsYMxSICsKv7f/B9wdup0MZDoY2wd8AfRxIFZvYBuQGtp+C7jTgbhDgXVAR4IDEd4HBoQZ64T3APAcMDt0ezbwrENxzwcGAQXAaIdifg1IDN1+1sFcOze7/QAw14m4oftzCQ4K2R7O++Mk+f4AeCjSv62WF1cdoavqCsDxSRBUdY+qfha6fQjYSPDNHUlMVdWa0GZS6OLIN9AikgNcA8xzIl40iUhngn/QvwRQ1TpVPeDgU0wE/qGq2x2KlwikikgiwQLsxPkU5wN/V9UjqtoALAduCCfQSd4D1xH80CR0fb0TcVV1o6qGezb3yWK+F3oNAP5O8JwVJ+JWN9tMI4z32inqy38D/xFOzDbiOs5VBf1MEJG+wEiCR9SRxvKJyGqgHPizqkYcM+QnBP/AAg7Fa6TAeyLyaWiaBiecA1QAr4a6iOaJSJpDsSF4XsSbTgRS1V3A88AOgtNWHFTV9xwIvQ4YLyLdRKQjwSG+uW085nRka+i8j9B1DwdjR9N0HJwpRUSeFJGdwG3A9x2KeS2wS1XXOBGvhZmhbqJXwukma40V9GZEJB14G5jV4hM/LKrqV9URBI9CxorI0EhjishkoFxVP400VisuVtVRBGfPvF9ExjsQM5Hgv5s/V9WRwGGC3QIRC53odi3wW4fidSF4tNsP6AWkicg3I42rqhsJdi/8GXgXWAN4enVuEXmE4Gsw36mYqvqIquaGYs6MNF7ow/cRHPpwaOHnQH9gBMGDhx85EdQKeoiIJBEs5vNV9fdOxg51MRQAkxwIdzFwrYiUAguAy0Xk1w7ERVV3h67Lgf8hONNmpMqAsmb/nfyOYIF3wlXAZ6q616F4VwDbVLVCVeuB3wMXORFYVX+pqqNUdTzBf7+3OBE3ZK+I9AQIXZc7GNtxInIHMBm4TUMdyg57A7jRgTj9CX64rwm933KAz0TkrEgDq+re0AFfAPgFzrzXrKBDcIEOgn28G1X1xw7F7N74Db6IpBIsFpsijauq/0tVc1S1L8Huhr+oasRHkSKSJiKdGm8T/PIq4tFEqvoFsFNEBoXumsjxUy9H4hYc6m4J2QFcICIdQ38TEwl+nxIxEekRuj4b+AbO5r0IuCN0+w7gDw7GdpSITAIeBq5V1SMOxh3QbPNanHmvfa6qPVS1b+j9VkZw8MQXkcZu/AAOuQEH3muA60a5vEnw35N6gi/utxyKewnB/uO1wOrQ5eoIYw4DVoVirgO+H4XXIx+HRrkQ7OteE7qsBx5xMM8RQFHotVgIdHEgZkdgP5Dh8Gv6GMFisA54HejgUNwPCX6QrQEmRhDnhPcA0A34gOBR/wdAV4fi3hC6fQzYCyx1IGYJwam2G99n4YxGaS3u26Hf2Vrgj0BvJ+K22F9KeKNcWsv3deDzUL6LgJ5O/J3Zqf/GGBMnrMvFGGPihBV0Y4yJE1bQjTEmTlhBN8aYOGEF3Rhj4oQVdGOMiRNW0I0xJk78f1aRXbb9mME6AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "q2_data=pd.read_csv(\"q2_data.csv\")\n",
    "test_size = 0.3\n",
    "columns = ['R2', 'Error(train)', 'Error(test)', 'model']\n",
    "results = []\n",
    "degrees = range(1,16)\n",
    "# try model = LinearRegression(), Lasso() and Ridge()\n",
    "model = LinearRegression() \n",
    "#model=linear_model.Ridge(alpha=0.1,max_iter=10000)\n",
    "#model = linear_model.Lasso(alpha=0.01,max_iter=10000) \n",
    "for d in degrees:\n",
    "    results.append(fit_poly(degree=d, data=q2_data, model=model))\n",
    "res = pd.DataFrame(results, index=degrees, columns=columns)\n",
    "display(res)\n",
    "res.plot()\n",
    "plt.ylim((0,1))\n",
    "plt.xticks(degrees)\n",
    "plt.grid()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "f(x)=  13.944644570684666\n",
      "+ 16.150233800371456 * X^1\n",
      "+ -64.46420135366411 * X^2\n",
      "+ 88.61485475949736 * X^3\n",
      "+ -49.99899308237356 * X^4\n",
      "+ 9.997891666945343 * X^5\n",
      "R2=0.9985115260541232\n",
      "Error(train)=0.0006348986285034617\n",
      "Error(test)=0.0007280924136923384\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA2d0lEQVR4nO2de3xcZZ3/399JmmIS5DIt15JJUUAB5WIAFUTYIgt1XX7uKmucAtJKbIKYXVdXICq6u1FX3dWKpBDYymUORdb7uhVYcAUqLBCgyEXQQpO0INs2aKEJbXP5/v54zmTOTGaSSTL3fN+v13llznOeM+c7k2c+5znf5/t8H1FVDMMwjMolVGwDDMMwjPxiQm8YhlHhmNAbhmFUOCb0hmEYFY4JvWEYRoVTXWwD0rFgwQJtbGwsthmGYRhlw6OPPrpdVRemO1aSQt/Y2EhPT0+xzTAMwygbRKQv0zFz3RiGYVQ4JvSGYRgVjgm9YRhGhWNCbxiGUeGY0BuGYVQ4JvSGYRhFxvOgsRFCIffX83L7/iUZXmkYhjFX8DxYvhz27HH7fX1uHyAazc01rEdvGIZRRNrb4yL/HPAY4Pbb23N3DRN6wzCMIjIwEH/1b8A5acpnjwm9YRhGSfACcHhe3tmE3jAMoyR4AVicl3c2oTcMwygi4TDACNBPsEfvynODCb1hGEYROf98gC04sXdCX1MDq1bl7hom9IZhGEXC8+CmmwA2+SWLEYEVK3IXWgkm9IZhGEWjowOGhsD55wEORxXWrcvtdUzoDcMwikR/f/zVJqAKWJRSnhtM6A3DMIpEQ0P81QtAhHiygkR5bjChNwzDKBKdnVBbC8HQytpaV55LTOgNwzCKRDQK3d0QCj0PvJlIxO3nciAWshB6EVkjIltF5KlA2fdFZIO/9YrIhgznniMiz4nIRhG5PId2G4ZhlD2eB1dc8SfGxraz775vprMz9yIP2fXobySYgAFQ1b9R1eNV9Xjgh8CPUk8SkSrgGuBc4GigWUSOnq3BhmEYlYDnwd0Xe1yz+a0AfO1PX+fui72cpyiGLIReVe8DXkl3TEQEOB9Ym+bwycBGVX1BVfcAtwHnzcJWwzCMiuGhdo/vDrcwxMsAvJttfHe4hYfac6/0s/XRvwf4P1X9fZpjhwKbA/tb/LK0iEiLiPSISM+2bdtmaZZhGEZp8+mBDuoYYqO/fzhQxxCfHujI+bVmK/TNpO/NA0iaMs30RqrarapNqtq0cOHCWZplGIZR2jTgguU3AocAdSnluWTGQi8i1cBfAd/PUGULcFhgfxHw0kyvZxiGUUm8GHLB8huBN6cpzyWz6dGfBTyrqlsyHH8EOEJEFotIDfAR4GezuJ5hGEbFcPlYJ4PUJgn9ILVcPpbjIHqyC69cCzwIHCUiW0RkhX/oI6S4bUTkEBFZB6CqI8AngTuB3wK3q+rTuTTeMAyjXLkzHOVjfIeXgTcBvUS4hG7uDOc+vlJUM7rNi0ZTU5P29PQU2wzDMIy8UV8Pg4NPAMcDtwMfBlwe+u3bp/9+IvKoqjalO2YzYw3DMAqM58HgIDAec5Pw0r+SNph9dpjQG4ZhFJiH2j020chX+RAAHyLhwch1QjMwoTcMwygsnsdXB1popI/ngQOA7/MJrqYNyH1CMzChNwzDKCwdbqIUJEIrQyhtXMsldV7Rct0YhmEYuSKwqkgwtDKE8lXJ/axY996GYRhGwdi5v3PCD+JmlR4ROLbfztzPigUTesMwjILy6V2djCH8zt9/S+BYP3kYicWE3jAMo6BcPxili5X81t8/yv87SC1XkoeRWOILFBqGYRgF4zK6OJaXEX48Piv2Sjq5Kw+zYsGE3jAMo6DU1bnJUk8xH1hMHS+MH4utys81zXVjGIZRFJ4j4biBmpr8LCMIJvSGYRgFxaU+GMMJfWIods+e/F3ThN4wDKOANOPxaxqAIf6Zm2kmD4vEpmBCbxiGUSDWt3ms4WIGeRGAU3mFNVxMMx7hcP6ua0JvGIZRIN56bTt7Mcxz/v5RwF4Ms4p2VuVpIBZM6A3DMArG/joAwLPAG4GD/PIFDORtIBZM6A3DMApOPN5GCnQ9E3rDMIwCkxxYmX9M6A3DMArEdsIMAptJFvrt5HEkluwWB18jIltF5KmU8stE5DkReVpEvp7h3F4ReVJENoiILQJrGMac5nbO5xn/9dH+393U8LfkcSSW7FIg3Ah8F7g5XiAiZwLnAW9X1d0icsAk55+pqjNY6tYwDKOC8Dw+Xn0Ta0fc7rHAGML1rGCf1jyOxJKF0KvqfSLSmFLcCnxNVXf7dbbmwTbDMIzKoaOD+SNDPA3MB96EW2ykuX4d4a78XnqmPvojgfeIyEMicq+InJShngJ3icijItIy2RuKSIuI9IhIz7Zt22ZolmEYRmmifW5RkaeAtwJVfnl4MD+LjQSZqdBXA/sB7wQ+C9wuIukihU5V1ROBc4FLReT0TG+oqt2q2qSqTQsXLpyhWYZhGKVHWxtsZ38AngaOCRx7rWb/vF9/pkK/BfiROh7GZehZkFpJVV/y/24FfgycPFNDDcMwypXubpjPbnbgIm6ODRzT3bvzfv2ZCv1PgD8DEJEjgRogacBVROpEZO/4a+Bs3FOLYRjGnGJ0FPZm53jETbBHvzc78379bMIr1wIPAkeJyBYRWQGsAQ73Qy5vAy5SVRWRQ0RknX/qgcB6EXkCeBj4L1W9Iz8fwzAMo/SJ93SPnbRW7skm6qY5w6Flaeq+BCz1X78AHDcr6wzDMCqE7YR5mgFqgUigfIDwRL93jrGZsYZhGHkmEoF2VvEkwjEkhHcX83i2Nb+TpcCE3jAMI+/Elnp8TTp4BuWtCGO4BcFvXfI9TuvK72QpMKE3DMPIL57HaTe1UKd9vAy8DWWX1LKltZPld+df5MGE3jAMI790dMDQUNJAbK0Ocdq6joKZYEJvGIaRT/rdzNcn/N3jU8oLgQm9YRhGPmloAGADLub8oJTyQmBCbxiGkUfWvLmTQWrZQCLefJBa1ry5s2A2mNAbhmHkkZZfRVlOF0/jhL6XCJfQTcuvCjMQC9nlozcMwzBmyOgo3M4JAHyDW/kG/hzU0cLZYD16wzCMPOLy+m7w944fL6+qmlg3X5jQG4Zh5AnPg2b1WMEneQOwkXNoxgOgZdIVOnKLuW4MwzDyxL2f8OimhQ8wxNuAN9HP9bQwrxq6CjAjNo716A3DMPLElYMd1DLEBhJOmzqG+PJI4SZLgQm9YRhGXvA8aKCfLcAfSU7l20DhJkuBCb1hGEZe6OiAfhp43N8PCv2LocJNlgITesMwjLzQ3w9X0skDVBMi4boZpJb/PrNwk6XAhN4wDCMvfLzW4yt0sIERjgbeQGKy1D9uLNxALFjUjWEYRu7xPL492MIbGKIHOA94nVqupJO1RJHCuuitR28YhpFzOly0TS8wAJyEi7b5Ci7apoD5zIDsFgdfIyJb/YXAg+WXichzIvK0iHw9w7nn+HU2isjluTLaMAyjpPFTED/i757k/22gHxHoLKyLPqse/Y3AOcECETkT9zTydlU9Bvhm6kkiUgVcA5wLHA00i8jRszXYMAyj1Bmoc132R4Aa4G1+eT8NrFwJ0cK66KcWelW9D3glpbgV+Jqq7vbrbE1z6snARlV9QVX3ALfhbg6GYRgVzWU7XWriR3DRNjW4aJsr6aSrq/D2zNRHfyTwHhF5SETuFZGT0tQ5FNgc2N/il6VFRFpEpEdEerZt2zZDswzDMIqL58FaoqzgWh5BOIlEtM1aCtyV95mp0FcD+wHvBD4L3C7icrQFSN0H0ExvqKrdqtqkqk0LFy6coVmGYRjFpcPPbvB9TmQI5RpuYjG9RRN5mLnQbwF+pI6HgTFgQZo6hwX2FwEvzfB6hmEYZUFfX/xV6lAs1NcX2hrHTIX+J8CfAYjIkTgX1PaUOo8AR4jIYhGpAT4C/GyG1zMMwygLRKAZj49yGfsAzwdSE197bXFsmnLClIisBc4AFojIFuAqYA2wxg+53ANcpKoqIocAN6jqUlUdEZFPAncCVcAaVX06Xx/EMAyjFPiIelxPC+9kiHcBh/upiQGihQ638RHVjG7zotHU1KQ9PT3FNsMwDGNaeB6cuqyRfehjf+CfgM/7x3qJ0Ki9ebu2iDyqqk3pjtnMWMMwjBzR3g4R+njQ3z81cKzQqYmDmNAbhmHkCG/gLAB+jfNXnxw4NhQucN6DACb0hmEYOeJs7kFwQn8CUOeXK1C/qsB5DwKY0BuGYeSQYeBhkt02QOHzHgQwoTcMw8gFnguhfBx4nTRCX0RM6A3DMHLAayvaEWC9vx8XegX+p2pJcYzyMaE3DMPIAfW7BwC4H1gMHBI49oeb7i6GSeOY0BuGYcwW320zCvwKP21AgCK65wETesMwjNnT0YEAG4A/kSz02wkXw6IkTOgNwzBmi7+i1C/93TP9vwq0s6oYFiVhQm8YhjFb/EVgfwm8FTjYL95OmAciRfbbYEJvGIYxe5YuZTdwHwm3jQI/Cp1f8PVh0zFl9krDMAwjM54H7+1eRy8wRELoBYjut4764nforUdvGIYxUzwPWlrgkNF+fokT9zMCx+tfKV4isyAm9IZhGDOkowOGhqCfBu7GLQS+f7BCQ/ESmQUxoTcMw5gh8WUDb2cJDwDnBg/W1lISDnpM6A3DMGZMVZVbNvAgYowCS/3yMQQuuqj4M6V8TOgNwzBmyOgofIUOfske9gNO8ctDKKxbV0zTkphS6EVkjYhs9deHjZd9SUReFJEN/rY0w7m9IvKkX8fWBjQMo6Koq4NF9PEL4BxSwhj7S2MgFrILr7wR+C5wc0r5t1T1m1mcf6aqbp+uYYZhGKWM58FfDno8QoitjDGht1siA7GQRY9eVe8DXimALYZhGGXD68vbiHEBdzmPPOcEjg1SOgOxMDsf/SdF5De+a2e/DHUUuEtEHhWRlsneTERaRKRHRHq2bds2C7MMwzDyy/o2j+V7riWE8l843/wC/9gIVVwR7i6ZgViYudCvBt6ECxv9A/CvGeqdqqon4qKOLhWR0zO9oap2q2qTqjYtXLhwhmYZhmHkn8buDkIom4FHgL8MHAsxyimrSkfkYYZCr6r/p6qjqjoGXE/yYufBei/5f7cCP85UzzAMo5w4ZNQF0P/I3//rwDGVqlLqzAMzFHoROTiw+0HgqTR16kRk7/hr4Ox09QzDMMoKz0P8lz8E3gYcGThcpaOFt2kKpoy6EZG1uPQNC0RkC3AVcIaIHI/zwfcCn/DrHgLcoKpLgQOBH4tI/Dq3quoduf8IhmEYBcLz4OKLEeBl3PqwV6XWiUQKbtZUTCn0qtqcpvjfM9R9CX9ymKq+ABw3K+sMwzBKifZ2GB4GnC9agQ8Fj4uUVLRNHJsZaxiGkS0DA+MvbweOAo729xWBlStLKtomjgm9YRhGlqj/tx+3CHgUl5pYgQvkFujqKpJlk2NCbxiGkQWeBwP+Qt+eX7bM/7udMJ6WXk8+jgm9YRhGFrS3w6dYxS7mcTNwGrAY2E0N7ayiqqrIBk6CCX0OaWuD6mo3HlNd7fYNw6gMBgZgLVHO4ws8i3Pb9BLhYtawligtk879Ly4m9NPE82DBAifm8W3BAjjrLFi92qUtBfd39WpXbhhG+dOMxyYaeTNfpAa4m+tYTC9ridLaWrLuecCEPmva2twiA8uWwdkD7h8+SohNNHL2gMc99yQawijCMNWMItxwTyPr27ypL2AYRuniedxACwvo4xbgfOAm/o5mPMLh0hZ5AFHVqWsVmKamJu3pKX76es9zfrmzBzy+QgcN9DPA/uzNq+zF8Hi9sfFxd3ELDqQwSC2Pt3ZzWlfpDtYYhpGZnQsaqR/ooxs3O3Q9cCrQR4T1sd6SiKgUkUdVtSntMRP69LS1OddLMx7X00IdQ+PHRoBHgd8BfwT2Bd4CnADMy/B+/RLh/ltKo0EYhpE9ngfNywQB3oH7/T9BIqxSSkRDJxN6c92kwfOcyINbJiwu8gPA5cBBwDuBC4F24CJcmtIw0AI8meY9F2k/7e35ttwwjFziebBumctt8zDwONAK47luxqSEQ20CmNCnoaPD9eR3UE8El6XuNlyv/Ru4xD/fB54DtuJ69rfjMtjFcHkfLvSPxemngYEB13AMwyh9PA8+9jHopAMBvgnsQyJ2HkBKMIFZOkzoU/E8HulbgMcy3sggCnwOaAaOwN3Rf4AbjDkSWOiXfxj4HvAi8Fmc8B8L/AQXZ1vHTkYJceqyRlrqPUIhaGw04TeMUqWjA0ZGoIF+fo/LVNkK7B2o00/pJTBLhwl9EM+DlhYWMjDuf2sDvo77B98LvH2Kt9gP+BfgMeAwXA7nq9hDmAFCKI30ce3gBYyo8Ku+Ru6+2DOxN4wS5NQ+F0UnKP+KG38Lel/HED4vpZfALB0m9AF2tnfAUGLQ9YvAdbge/TWkH2hVYCxN+dHAgzif/b/gMtzt9o+FUEJAI318d7iFh9pN6Y3SxPPck+ecewL1PNZwMY308TJwI24s7iD/8BhCFyt548oyia5Q1ZLb3vGOd2ihicVUR0HV337iNFxXgI4FyoPbKKK/XdKq97fGdLuE09YbA/2W/15/DjqUps4mIgX/vIYxGa2tqqHQxGZfW+uORSKqIu5vLFZsa3PPq/PD4x/6UtAq0N/7v+dNRLSZmC5ZUmwrkwF6NIOmFl3U023FEPpIRHXEBUrpJtB9QN8BuiuNcI+C9ocien9rSguPxbQ/FEkr+P8OKqB/Bvp6mhuGYRScWEw1HB5v1yNuaHFcyJqJ6SYiOoroDup11K83BjpIjW4lrKOIbiKil9TFKkbwYzEd/w2/ADoP9BOB3z+4r63UMKGfjFhMNRJJasRng9aDPh/4546B7qBuorinebs+iaR9ArjZ79n/FehIoHyoqq4wn9UwVPX+1pi+Sl3GJ1UFfZ15uouajMdTt53U6kf9Xm459/ZjMdWamoTQXwC6F+iWFKGXEuybmdBnIhbT4ZrapAbr+WL8nRTXCriGmw33t8Z0UJLfN75923//lSRcQmOgzcTK8odhlDZ+P2a8CTYT092k8cnkYHNPBSHdQd14T7+ZmLa2Juwo9RtA/LsaA33Yfwr/bMpnnI4WFBIT+gy8Fo4kNdTXQA8EPTnQ446LMEyzcY4/KYgOU5V0nX/wxf5rKQ1oK2H92LzKeQQ2ikNQ3EUSAr+JiW7FQdB7Qa8G/RToUtCTQI/wfwsHgx4GeiToe0A/DPoZ0DWgD/nnTyb+rzNP72CJDlOlY6DDVOnVtGptbWmKffx7Gsa5bg8G3RH4PFsJl6ztsxJ6YA1u7s9TgbIv4ULGN/jb0gznnoObV7QRuHyqa8W3Qgn9qO+Tj2//6Avwgyn/WHADUDMhFlONSkxfZ974e46C/o3fW/hFyg9jFzV6WbgEW5FR8gRc7uPivpXwuOsx3vYeBu0AbQKt9ts8oHWgx4OeC9oM2gJ6CejFoOeDvhf0KND5gXOqQU/xe72/YOKYljIxmGEM9Gpax3vGpSKa97fGdCfuSbzL/3xrU25al4VLtyM2W6E/HTgxjdB/ZorzqoDngcOBGlx6iKOnup4WUOg3EQkIOro36AcD/9hRRJuZ/T82FlO9LBzTHQG/6E7Qt4PuC7ox5YdgUTjGdInFXERMUOSDnYuNoFeCHuoLWJXfQ78S9D9BX0oR5FG/05Gulz4M+hzoj0AvBz0VtMZ/33rQvwa9BfTVSXr6w1SN75ZCDzkWU92Gu0u+4GvBEhLjc5tIE3xRYszadQM0zkDo3wXcGdi/Argim+sVSuivpnW8cV8OGgJ9JqWxz7Qnn45YzF0z/iTxPOh+oMf6wh+8weTyukZl09rq2lXQPRLvVNwD+j5fhEOg7we9CXT7JCIcb/vBqJsh9pp08HYIdB0uOuVg/3q1uMHMu/33S+3Vp75Nsdp8/Kl7zL+JnQr6RtDewHdRilE2qeRL6HuB3/iunf3SnPMh4IbA/gXAdye5RgvQA/Q0NDQU5IuJ9+hfxYVTfiil5W2pjuT8muGw623FfzR34Vw4y1N69KAlF6drlB5xkR9LEdH/xLlUAD0I55bcPIW4pz5VphY3E5sw3pTpJrEe5/rZx7ehAbQT9+QctzG+bSU8Pg5WV1fYAdt4lM1Wvzf/Jd/eWODzDFNV9CeObMiH0B/ou2ZCQCewJs05H04j9Fdnc71C++j/zf/nPpTyY8nHfzcWc5cI9nA6SPgDg4O/eTLBKGNSo1eCnQYFfRz0TL89NeJ8zanzNqbadlKb1AZTxT7ux04IoWTs7Q+B3oZzg4Bz8VwI+khKvTHQO1iSdHq+XTqtrYnPNIZzRYF7Ckm9cZYDORf6bI6Vuuvm/lbXO9mDiyo4PaXhvUp93q7d2po8PjAM+i7c4+LD7JvU2EsxjMsoDun88HHRHQD9OO7pcH9cFM2eKQQ9PvlvK+GkyU/pRL6uLvm6cZfOxMlVmWeSP4ObZVrvC+opuHDm3QF74oO08a2qyt3UwmG35aq3v2RJ4rMMU6WP4VxNp6S5MW6uiuTi35d38tGjPzjw+u+A29KcUw28gFsoPT4Ye0w218u70MdiutsfaPoPv9H9LPCP3UO1fpT8dqWDI/xKYjZuIwfo72gY/xHl2w6jfIiHTKaGSv4AFwpZBfp3oK9MIfDxbZTELM8lSxKhmPFNJNlvnvo00dqqWl8/8a2D41Dpth24eSpH+L+9g3Gupf8jEYuf6tZJvbnMOAw5FtPXwu59thLWXdToBtCw3+H7Q4qtu6gpm9/gbKNu1gJ/AIaBLcAK4Bbc+hq/AX4WF37gEGBd4NyluHTtzwMdU10rvuVd6AMxaH/u/4NHUhpWIXrS97cmN97lnK2AXpX0Y5TijVIZJUVqL347LsIF0BNwbpskd8MU2+aqSJJYznRSUzBuP54fJ3gzypwryg3gnkPCrXMR6KMp9UYgKYJISbiX5s9PXLOqKsNPxTdwLP57CrzPQ77IL2Ji9NsIoYJpQS6YdY++0Fu+hT7e8Hpxj7pxYQ1GAhTKNx73E4Jz5yzDxSYHG/sYmNjPVWIx3V6f6AzEBw3v9cVpHuhXce6/YHu5gyUTfOlJWx4d4OlcTFPdeH6Lc+vU+aJ/GujtKZ8rdRumKq2bqbVVAz33ieIe327BzQloTCPy8RtJKYR+ZosJfQrxRneVL/TxMKpiJSyKi/0ooq+AHgJ6DMmTT8aQ8mlxRm6IxXRXderAJ/plXKjkm0F7MghUXGDjT4w5d3JPbXrS00HqLPRM259wwRGLfcFf5N/IXspQP56Dagf1SZE8k91YtoF+1H//9/r76W4gpTSZKxtM6FNwfkDnsvnzlEYDxfnnxmKJAdp1fiP8XEoDfL2uDIJ5jZyQLjneH0m4OaJMnJCUrodbMj3S1G7+FKI8AvpTEtE6If/1v+MGnrO5aaRur4J+BTdYXY3r6KUOWMdvlOWICX2QmHuM/B8mTnEeA60rYiLJZmLjj5kr/Mb9YIp9pT47z5g9sZjzNwcF8Flcvplq0GsziOMoouDOhdJKL6CqSd387fWJ6J64OyrT9lvQL+CeYOKifzLo50F/jsssmakH/zLoD3Ehk3G30FLQ34x/Z0yIOCqHyVHpMKEPsL0+ograigunCs5I3Uq4qD+McDgx+WUHbpLJkSSHe20lrHV1JfYDNnJKanz8OlxE1kLQ+yYRxD6JlFW7iOt+M7GsUiLHM0pehQtHriKRc2dv/7fyTlyI5HF+zz1+fB9c+OnDE26OyTN0580r39+WCX2AUZyf8wBcoqZS6i3HJ1PFxf4uv5F2pNgZ37Xx2cokOAP1Or8HezyJsaR026DUFr39zpRYTHVZKLtZt8FtB+j9uDkDl/m/5/fh3LEf8Dtz3wR9gESsfuoWnAFcck9A08SEPk7MuUbu9gX0Byk95VJg/nxnUrw3dyHucf2JgNDHJ5WIjc9WBMGsk/HwyTES0/GXkvzkmdrLfS0cKfuGEIupXlI3cdbtZPH42TwBTDaBKz4Lvaam7L8+VTWhH+flvSKquBwcdSRyaY8iGpXS+E/He/Vxv+V23CN7phz55RLja6Qnnmslrj+biOgILjkYuLjyTDNcy7kXn5HxmHfRzVUR/S6tOjhZmGgGAd9B3fhvJDgLPbhtJayhUGWIvKoJvao6N8cooiO+cH4kqdeQ2yyVs2XJkmQfbXzVq1UBm+PRAaW4pJkxNUH/dHDS3BBuqUlwGVXT5XIfRSqiF581KbNZd6f483elrF+bGnmULj/PTmp1Wah0c8vPBBN6dZEIWwnrA0yMtinF/O+QcN+M4RaDqCPZT7uJiPXoy5B4pGEzMd0T8Eu/7v+fwS05mcmnXEniNCNSgvQvqUufgC24XVAV0z5J3FAraTHzOCb0Gs+/4RZaqCKRD2QXNSUZNxuJqO4gkUmq1xf6pYFenuWtL0/i6QKC/99B3ECi4AZg06nVTmq18+jSa6vFJhZz0TLBrysUKuj8sJLAhD6WiE8/DjcbLuinK8W42VhM9WPzkhdy/rbf27s1YHvJTIgxsiaePCy42tiZvsh/j0TOJRchVqWjfk/eRD4z5bL4eD4xofe7UP2+UH49yT8vJdso4ksQxtf9HMENyi7EDdLuoWrcNzlnW3c5keJrHsMtSP8eXAhlcLGL1IU/zEVnTIUJvd+FWu0L/TMpP6iywI+/ewIXbnlxusd7696XLrGYDtckDwgOgZ7huxJvS/lfxme52r/VyJbJhD7EHGDn/P0BWIdLjv8Wv1yBfwt3Fsmq6aEDrwDwduAzwPeAX6ZWGhqCjo7CGmZkR0cH1XuGxnf3AH8N3AvcDPxNSvV+GgCIRKC7G6LRAtlpVCTVxTagELy+C+YD/wMsA8Qvf406TllVHr+gF6saWDTaB8AXgf8APoFbEOANwYp9fQW3zZga7esfb3cjwEeBXwDX+6+DDFLLrUd3ok8X0kKjkpkTPfowr/AIsBNYEiivZ6hsekqfG+1kkFrACft1wEbcgr1BxkJVBbbMyITnwYIFIAJ9fg99DLgY+CHwbeDjgfoKDEiYx1u7ufLpMmmYRlkwJ4S+nwbuwfXkzwyUb/Z/fOXAryNRLqGbXiIo7oZ1EfAvwFOBejI26hTGKCqeB8uXw8CA27+STkaBNiCGu0G3p5wzPL+O8Nh2TusykTdyS8ULvefBF6STuwhxPBD2ywep5Z4l5eGfB+jshJ/WRllM73jZN4F9gUtwPUXw3VItLSb2RaajA/bsSeyvJcqVuCexK4Ar05xTU79XYYwz5hyVLfSex3svamS1LuNhxngXezGG0EuEy+Z3s/zu8uk5RaNuUC4cTpQtAL4F/C9wbbCyDcoWnYlDJVfzdaCVie62cV55JZ8mGXOYKYVeRNaIyFYReSrNsc+IiIrIggzn9orIkyKyQUR6cmFw1ngetLSwaLSPB3BRDu9nF12sZDG93LinfEQ+TjQK27enlAHvAy4HXgyUqw3KFgfPY+eCRkYJsYlGrqaN77IQ4VOcyzy+ybzxQdkJNJSPK9EoL7Lp0d8InJNaKCKH4TSmf4rzz1TV41W1afrmzYKODtezxYUhVgPvBdq4lma8sv5NDe6V6NYLrjc/AlyWUu9Xx7QV0CoDz2NkeQv1A32EUBrp41hW82m2827ghwwTAl6lHk09t7bW+ecMIw9MKfSqeh+Q7pnyW8A/wMQ2WxL0J+4/64EmoA4IoXxNOsr6N1V/wyr2SM34/uHAl4Af+xu4G8Dpz1zL+jbz1ReKne3JsfJPAf8PeBPwM1y01F4Ms5v5LK+JuSB5EQuWN/LOjHz0IvKXwIuq+sQUVRW4S0QeFZGWKd6zRUR6RKRn27ZtMzEriYE612XfDfQApwaOHab95f2bikapuWUNA/URxhAU+DvgOOCTwKt+tRDKcas/UTQz5xq1A4nOxWbcY3AdcAewf6DeAgZYsQLo7YWxMfe3rBukUepMW+hFpBbowM3bmYpTVfVE4FzgUhE5PVNFVe1W1SZVbVq4cOF0zZpA+2AnYwiP4sQ+KPQvVpWx3yZONEr4tV4+2TpGHxHm4Sbf/IHkiI56BuGYY4pj4xzis4d6qO99/yNO5F/DTYpKbW0CnLbOBsuNwjGTHv2bcJkEnhCRXmAR8JiIHJRaUVVf8v9uxXkVTp65qdPD0yhdrGS9v/9u/+8gtXxutIz9Nil0dcGXa9xN7STgU0AX8KB/XAB95hk466yi2VjprG/z+NJLLVQxxi7gPNxktp/gUlakpX+qoS3DyB3TFnpVfVJVD1DVRlVtBLYAJ6rqy8F6IlInInvHXwNnkzy3J+9cRhff4R1EqGahH1Z5Cd38OlJZj8k3DbubmgL/hLvztuAijcAX+3vusdj6HON50NgIi1Z3UMcQo7gUG/fj8tecOdnJ5RwNYJQd2YRXrsV1EI8SkS0ismKSuoeIyDp/90BgvYg8ATwM/Jeq3pELo7NHeZF++vgoVYyxmF7WEi3rgdh0NDS4mxrA3sA1uDvqNwN1BBhYae6CXNHWBsuWuXj5BvpR4G9xqQ2+xcQkZUlYhI1RaDKltSzmlos0xS4F/XMKKFw3nvK1FBcZmS3xBcVHA2luPww6H/R3ZZJ7v5xobU3ODr2JiH7VT4H992nSR++heu4td2QUHOZimuKlSwF+7e+5odjaWli1qlgW5Y9oFFpboZ/IeNkqYC9chst4/KugvOeCRnPhzALPg9Wrk8uaOYcrcFkov+6X+T0MthHmliU3upluFmFjFItMd4BibrPt0be2xtcaWaGwr8KoimjFr696SV3yavfX+XrzvZQe5q5qW8liJsRibpH55K/zDoVqPZBj9DkaxhefbsYtWF3pbc4oHZhLK0zFYm4h8GGq9G2gZ4NeTeucWI4tFlNtJhZYPNwtU7cf6EspYr+5KmJaPw1iMdWamlSR71GoUzhOYceEBb+g2FYbc4nJhL7iXDevL2/jUlazh1GexsVzXspqrqat4iPaolG4Kxylz3fhhHCx9a/jonCCU5gPGe23JJfToL09ORslPA8sxaWW+wXwxgnnRCITigyjKFSc0H9sTzcCbMCl7m3CRZyspHtORLStWgVfCCUWKTkK+Brwc9zyg3H6abAkl9Mgnlce4IN0EeEthNnK3QzTPHFRRwusMUqKihP6KkYBl/YAnNDHy+fCDy8ahXNuTixSMgZcCpyBC//rw/Xs69hJM17FP+XkBM9jE42MIvyJEJu5lK2M8HNgCS9xPS00k3g0stQ1RqlRcUI/iltKrwc4CDgkUD5XfnjRKOzb6hYpqUK5kBjfZF8UWI4T+oUM4LGM72gbjY3mwsnEs2e1MbZsGY30MQr8DcrjwO3AO/06dQzxFdyjUSxmgTVG6VFRQu95cC0tKE7o424bxZXPJbq6YIm/QO5aohzBMN/CpWy+xq8juPGLd/d5LF9uYp/K+jaPo+5ZTQjXhlYAd+JWifqLlLoN9LNkiQm8UaJkGqUt5jbTqJu6Ohfp8A1WqIBeBTpM1XjUzVwkFlMNhVTHQMdAzwV9A+izgdCQrYQrdjLZjInFdJhELOXn/FDVf0oXWgO6vT5SbIuNOQ5zJepmcND9/SwXosCX+TnzGOEyughV1CfNnmgUbr7ZvRbgBqAW+Aguqye4tLnNeEkDjnOZ9W0eg8taqPbHe1bhFmFvBdKOXdfWEr52DgwAGWVLxclfMx4d/D8AHmbl+CDZJ+ZwWvagO+EQXPTNBtyqMeBuAKkDinMVz4PD/CRlAN/H5fr/IHA1JC0DqOAW8bWRV6PEqSih/yge19PCC/yRRcBJbBkXsK6uYltXXIIC9QFcOuPvAP/pl9UxxC1cOGdXpPI8+NQCj1OXNdKAW2/3F8AFuAQaHvjD/E7gX6We9nDMpTYwkTdKnIoS+u/Uu57Yo8A7/LI6hri63oLFU2fvfB04HriYxKLiVYxx0urlfGqBN6cGZj0P7r7Y46sDLTTShwD3AX8FvA03B+ENOIEfJcQ1tLKvvMYpq0zgjfKgooR+/539DAG/x4lYsHzO09kJNYl1ZucDtwG7gChucXFXvodPD3TMqVmzD7V73DB84bi7pgcXVdOIWwZwH5zIX0Mr1YxyGV2sXGkdeaN8qCih3ywNPI37UQZX9qmIpQNnSzQKa9ZAODyeWfEo3GpU9wJXBKpG6OO8IY/29mIYWmDa2lg1sIwqxgB4BrcMYBi4G4gvainAX7COcNjFys91V6BRXlSM0Le1weXaycO4XutxfnmlLR04K6JR2L6dW2OJrDcXAm24RUpu98vig7NnD1SwC8fzYMECdPXq8fGLF4D3ATU4kT805ZRG6TeXvFGWiAu/LC2ampq0p6dn6ooBQiEX0Hwkf84W/psdKFuIcCWdPBCJ0tubH1vLlTEJEfLTnO3BLXv3BPC/wLF+nV4inBHprbzvzvOgpQWGhsaLXsB9BztxTzjHpjsvEqHyvgyjUhCRR1W1Kd2xiujRe54TeYDfsZshTmEeWrFLB+aCUOvK8WyWNcB/4JYh/CDwR788Qh+/6musLGe958FFF00Q+TNwIn836UV+tLrGspQZZUs2a8auEZGtIjJhYW8R+YyIqIgsyHDuOSLynIhsFJHLc2FwOhIZGBX4DckeenvUTktXF7JkybjYH4IT+z7gQ7hevgCN9LF72fLKCLuM9+RHR8eLnseJ/CBwD3CCX66BbUDCVN24xhqSUbZk06O/ETc+lYSIHIZzaaYNaRGRKlxalXOBo4FmETl6xpZOQl+fmyj1AIcBf+TL/MAm/2TD3Xfz69YY23ADtKfhZs7+kuQlCOezh6NWt5d/x76jI6knnyryx/vl8QibEEp9rXLHLeaYN8qcTLkRghsu0uyplLIf4MY8e4EFac55F3BnYP8K4IpsrjfdXDdRcUvo/ZffCbsfdCe12kys4leVygXgVuUaRVT9HEGk5HUZAxUp4xUIY4mVtxT0KdBDQPcH3ZCStyae+ycUKuPPa8w5yHWuGxH5S+BFVX1ikmqHApsD+1uYGMiQE/5Z3USpuDFvI5E61tyqUxOJuNDB+ODsVbgZoV8gebESVVi2zEU4lRPr2zyGlrWMR9c8CLwHdzf7FYkIrThhXgFcjiDryBuVwLSFXkRqcbmdvjhV1TRlGUN8RKRFRHpEpGfbtm3TsqnB9x79BojgJrjEy+2HOjWdnYnvEBLJz84GPo7L9/IadePHV692UU6lLvh+BCWLVndQ60+GuhM4C9gf+DWuU5BKPw2EwybyRuUwkx79m4DFwBMi0gssAh4TkYNS6m0BDgvsLwJeyvSmqtqtqk2q2rRw4cJM1dIyFHYTolKHYePlxuREoxO/qxrgx7g8L8uAn6Y0FVUn+KUq9vFx14GBxE0shsvzcwRO5BeDP00qwSC1fHleJ6tWFdRcw8gr0xZ6VX1SVQ9Q1UZVbcQJ+omq+nJK1UeAI0RksYjU4DLj/mzWFqehflUnr897A30kHsNHamqpX2V+m2ypX9XpFjoNUIvL83Ii8HFe43Q+ww7qGUMYQxihiqNXtxEKUVKrVK1vc8nJXhsKsYlGtrEfnyeRoOxe4MBAfbfkotBLhCvC3Zz1vaj15o3KIpPzPr4Ba4E/AMM4UV+RcrwXfzAWF6W3LnBsKfA7XIBDx1TXim8zWngkFtPhhgZ9DVQjERtFmwmxmGpVYrGN+DYA+naqtBp0bcqxMdA7WKKgWltb5K89FtNX54eTBl0HQf8KUUA/Dro7xf5NRBRUW1uLaLdh5AAmGYwtygpSU20zXWHKyAGxmFPsgBjupFb7qdPTQQX02jRi30xM4/fYYtk9XJNs90bQE3ybv0a1jqTYvZNavaQuZn0CoyKYTOgrYmaskUOiUejuZktVwp1xCd0sYpA7cI9oK3GLlsSnHQngsYxhqunuO8v5cQrlz/E8d51ly6jek4iR/xHO5dSLy7n/D4xwAbEkN83jrd107zQ3jTEHyHQHKOZmPfriE4up1tQkOsBxd8ge0FY/zv79oDvS9O6TXD/T9efEYu6xQGRSF9z9rTEdCIUnXG8I9FO+fSeBbgrYlWqaYVQSWI/emC7xrMapa+3Ow6U27sKFKp4EPBY4PiGmdmjIBd9n07uPh8r09Tkt7usjNTF+WxtExeOE1S3sPzaQdL0HcbNbvwNcBtyPm+kHsJ1w0qVS1mExjMom0x2gmJv16EuHeM9+K2FN7RL/CvRQ0Hmg3wAdTe0yZ+jdZ+y0RyLpz4tENBZTra93u5tIrvcq6N+DhkAbQP875fzXmTc+hlASg8aGkQewwVhjNsRiqhfPj+kuAr4c3x2yHfSDvqvkZNDHphD718KR1LFenTdPNRzWiW4ffxtFtJmYbiKio8h4vVHQ74Ee5F+/xXcljYEOU6WjflRNMzGtqprSG2QYZc1kQl8x+eiN/NNS73HlYAcN9DPA/uzHK1SjKHAr8PfANuAS4PO4GXKpKM6N8n3O5y9YRwN9jFFFiFGE9NOpx/xyCez/EPhn3CS5dwKrgJP949sIcwDbx8+vrYXubpvpalQ2FZ+P3igM770uyjG1vVQxxiD1VPsZLQS37uyzwKXAGtz06U8Cz6W8hwALGeBSVtNIHyGgmlFCZM6ZET/2KnAdLl/8+cBuwMPNcj05cE48Vw1AOGwibxgm9EbW+JGXRCLJuXHi7IsbCP09sIwQ1wFvwa3c9O+4WXdx0ol6OgZxoZIXAgfjQjtrcAubPw18lImNOJ6rJhbDlv4zDKC62AYY5UU06gtnY4OLiklBASXCCSzli9zArQxzPS45GriomHfgFvhoxKUi2BsXk78HlwxpM/AU8DCwwS/fD2gGWnCRPpluFMOhGhpv7mS7ibthjGM+emNmpFl3dZBaLqGbtUTZRCONuBuBAk/i8ub8EngcAs6V9NQBTcApuJVrTsWFdqYj3oIlHIZVq6wLb8xJJvPRW4/emBlxMe3oQPv6ebGqgX8Y7WQtrjw17fHb/e1KnDC/6G9bcb73apyQH4xLeXowUJWlKf1EWB/rNX03jAyY0Bszx/fjCC7C5v0e/NTv5PfTMN6jTyVeP11UznQZQ9jc2mkibxiTYIOxRs4IDtZ20MkgtVOfBEkLcU8LEUKtKzmty1TeMCbDhN7IKdEo9PaCp1Eeb+2mX1wSsW2E2U1NUl0FXqWOKDFCKFESC5Wnir7i4udHqHLHIhG45Rbo6irApzKM8saE3sgbp3VFuf+WXg6PjHGgbOez4TUM1CeyR0aJsQ87+WltlFgM3h+LclJkOyGUCyQ502SUGAeEle/HRtwS5r29NuhqGFliUTdGwfE86OiA/n5oaHBr1ppmG8bssKgbo6QYj8U3DKMgmOvGMAyjwjGhNwzDqHBM6A3DMCocE3rDMIwKx4TeMAyjwinJ8EoR2QYZ5s9PzQIIrDpROphd08Psmh5m1/SoRLsiqrow3YGSFPrZICI9mWJJi4nZNT3Mrulhdk2PuWaXuW4MwzAqHBN6wzCMCqcShb672AZkwOyaHmbX9DC7psecsqvifPSGYRhGMpXYozcMwzACmNAbhmFUOGUj9CJyjog8JyIbReTyNMdFRL7jH/+NiJyY7bl5tivq2/MbEXlARI4LHOsVkSdFZIOI5DQvcxZ2nSEiO/xrbxCRL2Z7bp7t+mzApqdEZFRE9veP5fP7WiMiW0XkqQzHi9W+prKrWO1rKruK1b6msqtY7eswEfkfEfmtiDwtIu1p6uSvjalqyW+4daKfBw4HaoAngKNT6iwFfoFbkvSdwEPZnptnu94N7Oe/Pjdul7/fCywo0vd1BvDzmZybT7tS6n8A+GW+vy//vU8HTgSeynC84O0rS7sK3r6ytKvg7Ssbu4rYvg4GTvRf7w38rpAaVi49+pOBjar6gqruAW4Dzkupcx5wszr+F9hXRA7O8ty82aWqD6jqH/3d/yU3a2LP2q48nZvr924G1ubo2pOiqvcBr0xSpRjta0q7itS+svm+MlHU7yuFQravP6jqY/7r14DfAoemVMtbGysXoT8U2BzY38LELylTnWzOzaddQVbg7thxFLhLRB4VkZYc2TQdu94lIk+IyC9E5JhpnptPuxCRWuAc4IeB4nx9X9lQjPY1XQrVvrKl0O0ra4rZvkSkETgBeCjlUN7aWLmsMCVpylLjQjPVyebcmZL1e4vImbgf4mmB4lNV9SUROQD4bxF51u+RFMKux3C5MXaKyFLgJ8ARWZ6bT7vifAD4taoGe2f5+r6yoRjtK2sK3L6yoRjtazoUpX2JSD3u5vK3qvpq6uE0p+SkjZVLj34LcFhgfxHwUpZ1sjk3n3YhIm8HbgDOU9WBeLmqvuT/3Qr8GPeIVhC7VPVVVd3pv14HzBORBdmcm0+7AnyElMfqPH5f2VCM9pUVRWhfU1Kk9jUdCt6+RGQeTuQ9Vf1Rmir5a2P5GHjI9YZ78ngBWExiMOKYlDrvJ3kg4+Fsz82zXQ3ARuDdKeV1wN6B1w8A5xTQroNITJg7Gej3v7uifl9+vX1wfta6QnxfgWs0knlwseDtK0u7Ct6+srSr4O0rG7uK1b78z34z8O1J6uStjZWF60ZVR0Tkk8CduBHoNar6tIis9I9fC6zDjVpvBIaAiyc7t4B2fREIA10iAjCiLjvdgcCP/bJq4FZVvaOAdn0IaBWREeB14CPqWlWxvy+ADwJ3qepg4PS8fV8AIrIWFymyQES2AFcB8wJ2Fbx9ZWlXwdtXlnYVvH1laRcUoX0BpwIXAE+KyAa/7ErcjTrvbcxSIBiGYVQ45eKjNwzDMGaICb1hGEaFY0JvGIZR4ZjQG4ZhVDgm9IZhGBWOCb1hGEaFY0JvGIZR4fx/58AO1w5mP0IAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "R2, Etrain, Etest, model = fit_poly(degree=5, data=q2_data, plot=True)\n",
    "\n",
    "coefficient_length= range(len(model.coef_))\n",
    "print('f(x)= ',model.intercept_,)\n",
    "for i in coefficient_length:\n",
    "        print('+',model.coef_[i],'*',f\"X^{i+1}\")\n",
    "\n",
    "print(f'R2={R2}\\nError(train)={Etrain}\\nError(test)={Etest}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### b (20 pts) \n",
    "    Use the regression equation given in part (a) to find y value for x=1. \n",
    "    Assume this is the correct value returned by the population function that is approximated by f(x) found in part (a). \n",
    "    Then, apply the following procedure to fit several models with degree 1..0 by taking a sample of size 20 from the given dataset and predict y at x = 1:\n",
    "\n",
    "for d in range(1,11):\n",
    "    add polynomial features up to degree d\n",
    "    pred = []\n",
    "    for i in range(1000):\n",
    "        take a sample of size 20 rows from the dataset with added polynomial features\n",
    "        fit a linear regression model (do not apply train/test split)\n",
    "        predict the value of y at x = 1 \n",
    "        append y to pred list\n",
    "    find and print d, and the mean and the standard deviation of pred list (i.e., 1000 predictions for degree=d)\n",
    "\n",
    "Compare the mean prediction found with the value returned by the function found in part (a) and standard deviations of the predictions made for different degrees. Comment on the obtained results.\n",
    "\n",
    "\n",
    "**Answer**\n",
    "When we calculate the correct y value regarding to regression equation founded in part (a) with \"x=1\" we find 14.244430361461163\n",
    "\n",
    "After that several models are created according to given degree range and prediction of \"y\" wrt \"x=1\" is calculated. We can see from the results table y takes nearly values of \"14.286522009933638\" with std nearly \"2.3092638912203257e-15\"\n",
    "\n",
    "We can see that our model in part(a) and new results are accurate as expected."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "14.244430361461163\n"
     ]
    }
   ],
   "source": [
    "X=1\n",
    "correct_y = 13.944644570684666+ 16.150233800371456*(pow(X,1))+ (-64.46420135366411)*(pow(X,2))+ 88.61485475949736*(pow(X,3))+ (-49.99899308237356)*(pow(X,4))+ 9.997891666945343*(pow(X,5))\n",
    "print(correct_y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {},
   "outputs": [],
   "source": [
    "## This function generates polynomial and interaction features and add the data then fits polynomial regression\n",
    "def fit_poly_b(degree, data, model=LinearRegression(),plot=False):\n",
    "    base_x_value=pd.DataFrame(data={'x': [1]})\n",
    "    predictions = []\n",
    "    poly = PolynomialFeatures(degree=degree, include_bias=False) \n",
    "    pf = poly.fit_transform(data[['x']])  ## creates polynomial features with given degree\n",
    "    pf_base_x=poly.fit_transform(base_x_value)  ## creates polynomial features with given degree for X=1\n",
    "    X = pd.DataFrame(pf, columns=poly.get_feature_names())\n",
    "    base_x_df = pd.DataFrame(pf_base_x, columns=poly.get_feature_names())\n",
    "    X['y'] = data['y']\n",
    "    for i in range(1000):\n",
    "        process_data = X.sample(n=20, random_state=2532810)\n",
    "        X_sampled = process_data.drop('y', axis=1)\n",
    "        Y_sampled = process_data['y']\n",
    "        model.fit(X_sampled,Y_sampled)\n",
    "        predictions.append(model.predict(base_x_df))\n",
    "    return np.mean(predictions),np.std(predictions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>means</th>\n",
       "      <th>stds</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>14.726572</td>\n",
       "      <td>5.329071e-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>14.291205</td>\n",
       "      <td>0.000000e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>14.294967</td>\n",
       "      <td>1.776357e-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>14.074431</td>\n",
       "      <td>5.329071e-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>14.259684</td>\n",
       "      <td>1.776357e-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>14.257820</td>\n",
       "      <td>1.776357e-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>14.252338</td>\n",
       "      <td>1.776357e-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>14.244928</td>\n",
       "      <td>3.552714e-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>14.240940</td>\n",
       "      <td>0.000000e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>14.222335</td>\n",
       "      <td>1.776357e-15</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        means          stds\n",
       "1   14.726572  5.329071e-15\n",
       "2   14.291205  0.000000e+00\n",
       "3   14.294967  1.776357e-15\n",
       "4   14.074431  5.329071e-15\n",
       "5   14.259684  1.776357e-15\n",
       "6   14.257820  1.776357e-15\n",
       "7   14.252338  1.776357e-15\n",
       "8   14.244928  3.552714e-15\n",
       "9   14.240940  0.000000e+00\n",
       "10  14.222335  1.776357e-15"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "q2_data=pd.read_csv(\"q2_data.csv\")\n",
    "columns = ['means', 'stds']\n",
    "results = []\n",
    "degrees = range(1,11)\n",
    "for d in degrees:\n",
    "    results.append(fit_poly_b(d, q2_data, model=LinearRegression(),plot=False))    \n",
    "res = pd.DataFrame(results, index=degrees, columns=columns)\n",
    "display(res)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "14.286522009933638\n",
      "2.3092638912203257e-15\n"
     ]
    }
   ],
   "source": [
    "print(np.mean(res['means']))\n",
    "print(np.mean(res['stds']))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}