02_transform.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Transform"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "import geopandas as gpd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.simplefilter(\"ignore\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Identify buildings in Houston's city limits\n",
    "\n",
    "For the purpose of this analysis, we are aiming to segregate buildings in the Houston city limits from those in the surrounding suburbs and unincorporated areas of Harris County.\n",
    "\n",
    "According to officials at the Harris County Appraisal District, this is accomplished by joining the \"tax district\" table to the building list and identifying those buildings taxed by the city government."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First read in the buildings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "res = gpd.pd.read_csv(\"input/building_res.csv\", dtype={\"ACCOUNT\": str})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then read in the tax districts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "jur = gpd.pd.read_csv(\"input/jur_value.csv\", dtype={\"ACCOUNT\": str, \"TAX_DISTRICT\": str})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Trim down the tax district table to the crucial column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "districts = jur[[\n",
    "    'ACCOUNT',\n",
    "    'TAX_DISTRICT'\n",
    "]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Filter the table down to properties listed in the Houston city tax district, which county officials said is encoded as \"061.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "in_houston = districts[districts.TAX_DISTRICT == '061']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Mark all buildings that are found in that filtered list "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "res['IN_HOUSTON'] = res.ACCOUNT.isin(in_houston.ACCOUNT)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Clean up the columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Trim down to the columns we want to keep"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "trimmed_buildings = res[[\n",
    "    'ACCOUNT',\n",
    "    'USE_CODE',\n",
    "    'CLASS_STRUCTURE',\n",
    "    'BUILDING_NUMBER',\n",
    "    'DATE_ERECTED',\n",
    "    'IN_HOUSTON'\n",
    "]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Add a decade column for later analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_decade(year):\n",
    "    s = str(year)\n",
    "    s = s[:-1]\n",
    "    s += \"0\"\n",
    "    return int(s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "trimmed_buildings['DECADE'] = trimmed_buildings.DATE_ERECTED.apply(get_decade)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Write out the buildings file with this extra data included"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "trimmed_buildings.to_csv(\"./output/buildings.csv\", index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Transform the parcels map to the same projection as the flood zones"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Read it in"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "parcels = gpd.read_file(\"./input/Parcels/Parcels.shp\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Trim to down to the columns we want to keep"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "trimmed_parcels = parcels[[\n",
    "    'HCAD_NUM',\n",
    "    'CONDO_FLAG',\n",
    "    'CurrOwner',\n",
    "    'LocAddr',\n",
    "    'city',\n",
    "    'zip',\n",
    "    'geometry'\n",
    "]].rename(columns={\n",
    "    \"CurrOwner\": \"OWNER\",\n",
    "    \"HCAD_NUM\": \"ACCOUNT\",\n",
    "    \"LocAddr\": \"ADDRESS\",\n",
    "    \"city\": \"CITY\",\n",
    "    \"zip\": \"ZIPCODE\"\n",
    "})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Reproject it and write it back out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "trimmed_parcels.to_crs({'init': 'epsg:4269'}).to_file(\"./output/parcels.shp\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Filter down to official FEMA flood zones\n",
    "\n",
    "According to FEMA officials, the 100-year and 500-year flood zones are the ones best describes as flood prone areas according to their standards. Here we will read in the complete set of flood zones, segregate out those two types and write them to separate files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "flood_zones = gpd.read_file(\"./input/48201C_20171002/S_FLD_HAZ_AR.shp\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Filter down as instructed by FEMA staff in a phone interview."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "one_hundred_year_zones = flood_zones[flood_zones.FLD_ZONE.isin(['AE', 'VE', 'A', 'AO'])]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "five_hundred_year_zones = flood_zones[\n",
    "    (flood_zones.FLD_ZONE == 'X') &\n",
    "    (flood_zones.ZONE_SUBTY == '0.2 PCT ANNUAL CHANCE FLOOD HAZARD')\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Write out the results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "one_hundred_year_zones.to_file(\"./output/one-hundred-year-flood-zones.shp\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "five_hundred_year_zones.to_file(\"./output/five-hundred-year-flood-zones.shp\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}