ArcGIS FeatureServer

Adding a runner for an ArcGIS FeatureServer

Set up your code

1. Make a new directory for your new state. Unless otherwise specified, the directory should be named runners/<state>/arcgis (e.g., runners/al/arcgis).

Find the data

1. Open developer console in your web browser, then open the Network panel

2. Open the ArcGIS dashboard for your state. Here is Alabama's dashboard, as an example.

3. In the Network developer panel, filter for urls with the string "query" in them.

4. For each tab in the dashboard, you should see urls that looks like:

   https://services7.arcgis.com/4RQmZZ0yaZkGR1zy/arcgis/rest/services/ApptOnly2_Public/FeatureServer/0/query?f=pbf&cacheHint=true&maxRecordCountFactor=4&resultOffset=0&resultRecordCount=4000&where=1%3D1&orderByFields=objectId%20ASC&outFields=*&outSR=102100&spatialRel=esriSpatialRelIntersects

5. Remove all query args from the urls found above, so they look like:

   https://services7.arcgis.com/4RQmZZ0yaZkGR1zy/arcgis/rest/services/ApptOnly2_Public/FeatureServer/0/query

6. Opening this url will open the query interface for this layer. In this example, the layer is named ApptOnly2, with ID: 0. We actually want to open the service url, so remove the layer number from the url. It should look like this:

   https://services7.arcgis.com/4RQmZZ0yaZkGR1zy/arcgis/rest/services/ApptOnly2_Public/FeatureServer

7. Open this url in a web browser. You will see a Service ItemId listed. Save this value, you'll need it in a moment. In this example, the Service ItemId is:

   d1a799c7f98e41fb8c6b4386ca6fe014

8. On the same page, you will see a bullet-point list of Layers. Save them, you'll need them in a moment. In this example, there is only one layer:

   ApptOnly2

Configure fetch.yml

We offer a shared fetcher - configurable via a .yml file - for fetching ArcGIS FeatureServers.

Example

See #76 for a sample PR.

Instructions

1. In the directory you created (runners/<state>/arcgis), create a file called fetch.yml

2. Using the following template, update your fetch.yml.

   1. Update the state key to your state's two-letter abbreviation.

   2. The arcgis key is a list of objects, each of which has an id key (set to the id from above) and a layer_names key (set to a list of layer names from above).

   This is a sample fetch.yml for Alabama. Your file may contain fewer (or more!) configurations.

   ---
   state: al
   arcgis:
   - id: d1a799c7f98e41fb8c6b4386ca6fe014
     layer_names:
     - ApptOnly2
   - id: d677f142234648a1a41b84d94df8e134 # A second feature server
     layer_names: # this feature server has several layers
     - Mass Vaccination Sites
     - Pharmacy Vaccination Sites
     - FederalPartners

3. Test your fetcher by running:

   poetry run vaccine-feed-ingest fetch <state>/arcgis

   In our case, this command will look like:

   poetry run vaccine-feed-ingest fetch al/arcgis

4. That's it! The arcgis python module (ingestors/arcgis.py) will download the locations from each layer. Behind the scenes, it will grab a url like:

   https://opendata.arcgis.com/datasets/<service_item_id>_<layer_id>.geojson

   In our case, it will look like:

   https://opendata.arcgis.com/datasets/51d4c310f1fe4d83a63e2b47acb77898_0.geojson

Configure parse.yml

We offer a shared parser - configurable via a .yml file - for parsing files fetched from ArcGIS FeatureServers.

Example

See #81 for a sample PR.

Instructions

1. In the directory you created (runners/<state>/arcgis), create a file called parse.yml

2. Using the following template, update your parse.yml.

   1. Update the state key to your state's two-letter abbreviation.

   2. Update the site key to the name of your directory (probably arcgis).

   3. Set the parser key to arcgis_features.

   This is a sample parse.yml for Alabama.

   ---
   state: al
   site: arcgis
   parser: arcgis_features

3. Test your parser by running:

   poetry run vaccine-feed-ingest parse <state>/arcgis

   In our case, this command will look like:

   poetry run vaccine-feed-ingest parse al/arcgis

4. That's it! We will parse your downloaded data using the shared parser.

Normalize

At this time there is no shared tooling for normalization. Take a look at runners/ak/arcgis/normalize.py as a sample!