ArcGIS: Determining Properties On Septic Systems

I've been given a task to determine the number of properties on septic systems (rather than a sewer scheme) in the local government area. It's not everyday I get a challenging spatial challenge like this! I've never done kind of analysis before but I have a basic understanding of the spatial tools I would need to use to narrow down the list of properties affected.

I've access to a list of spatial data layers in the corporate database and have been supplied a bunch of known variable which I need to identify and eliminate from the list of properties.

These are the steps I took through trial and error:

  1. Database join Land Parcels with Property tables.
  2. Add layer Sewer Scheme Areas and dissolve.
  3. Add layer National Parks and dissolve.
  4. Add layer Roads (Polygon) and dissolve.
  5. Add layer Crown Land and dissolve.
  6. Erase Sewer Scheme Areas polygon from Land Parcels Joined To Property layer.
  7. Erase National Parks polygon from Land Parcels Joined To Property layer.
  8. Erase Roads polygon from Land Parcels Joined To Property layer.
  9. Erase Crown Land polygon from Land Parcels Joined To Property layer.
  10. Create report of Land Parcels Joined To Property to show land ids, property ids, address and suburbs.
  11. Create map like product using Land Parcels Joined To Property layer.
  12. Data validation and checks.

Model Builder

Model builder is basically scripting without learning any programming knowledge or arcpy. You drag data from your workspace or catalog, and the required geoprocesses from the toolbox. Then link up the input data with the tools inside model builder. There is certainly a logical method to designing a clean and efficient model. I'm a newbie at this but it is a powerful tool when you learn how to use it.

I rarely use ArcGIS model builder in my daily tasks but this job requires an audit trail, and model builder was useful because it allows me to run multiple scenarios of the test data and ajdust the steps of the workflow after each simulation. It also gives me a visual diagram to point at (when shit isn't working properly) and I can go back through the steps to figure out what went wrong!

To use model builder create a toolbox and then create a model inside the toolbox. Note: toolboxes are strict with versioning so if you try to run a toolbox from a later version of ArcGIS it probably won't open.

In this job I used it mainly so that I could test different variations of data quickly. I also needed the audit trail so that my boss could check my workflow (if necessary). Doing the thing manually may be more efficient but repeating the same steps over multiple attempts is a chore. Maximize efficiency where possible!

Think About What You're Doing!

In the first attempt I tried to run a scenario by dissolving Land Parcels thinking that I could use this simple polygon as a selection mask to highlight all the relevant land parcels to the area. This attempt was a failure because I was grabbing too many parcels using an intersect functionality with a selection mask. After thinking about what I could change in order to reach some better results I decided to start from scratch and run another scenario using the same data but without dissolving it first.

Click to enlarge

In the second attempt I discovered that it was tricky to create a database join after the Land Parcel layer had been manipulated as I would have clipped out some parcels and left multipart features in the layer (which i didn't want to deal with). The solution was to join the tables first before running the simulation.

Click to enlarge

In the third attempt I joined the Land Parcel layer to the Property table first using Land Number as the unique key. I discarded any records with NULL vales for Land Numbers.

Then I went to work with erasing features from the Land Parcel Joined To Property layer as shown below.

Click to enlarge

The model only needs to be tweaked slightly for the whole simulation to be operated again. This time the geoprocess took a lot more time because the database join with the Property table appends a lot more fields to the Land Parcel layer, which already comes with 26,000 records! After about ten minutes the geoprocessing was completed successfully.

Click to enlarge

In this model the pink are land parcels that are probably on septic systems, whilst anything that isn't blotted out are variables which we have eliminated by design or gaps in the data where we have no data (i.e. the Land Parcels layer's geographic extent is only limited to areas contained within the Hawkesbury Local Government Area boundary). The light blue polygons are bodies of water.

Data Validation

The boss highlighted an issue in the definiton queries too. I need to do basic checks in the data as a form of validation. The geoprocessing tools are efficient but not the same as doing a manual check to ensure that the manual selections return the same results as the geoprocessing tools (i.e. topological errors)

I think one of the problems is my script, I used OR instead of AND which is the difference between having all of these conditions shown and none of them:

Problem script

"proprod_dbo_HCCnucPropertyLandGISView_PROPERTY_TYPE_DESC" <> 'CrownLand' OR 
"proprod_dbo_HCCnucPropertyLandGISView_PROPERTY_TYPE_DESC" <> 'InfRoad' OR 
"proprod_dbo_HCCnucPropertyLandGISView_PROPERTY_TYPE_DESC" <> 'InfSewMgmt' OR 
"proprod_dbo_HCCnucPropertyLandGISView_PROPERTY_TYPE_DESC" <> 'InfStaRail' OR 
"proprod_dbo_HCCnucPropertyLandGISView_PROPERTY_TYPE_DESC" <> 'ResUrbSHdr' OR 
"proprod_dbo_HCCnucPropertyLandGISView_PROPERTY_TYPE_DESC" <> 'ResRurVacL' OR 
"proprod_dbo_HCCnucPropertyLandGISView_PROPERTY_TYPE_DESC" <> 'ResUrbVacL' OR 
"proprod_dbo_HCCnucPropertyLandGISView_PROPERTY_TYPE_DESC" <> 'RsvNatPark' OR 
"proprod_dbo_HCCnucPropertyLandGISView_PROPERTY_TYPE_DESC" <> 'RsvNatStrp' OR 
"proprod_dbo_HCCnucPropertyLandGISView_PROPERTY_TYPE_DESC" <> 'RsvNature'

Suggested change

"proprod_dbo_HCCnucPropertyLandGISView_PROPERTY_TYPE_DESC" <> 'CrownLand' AND 
"proprod_dbo_HCCnucPropertyLandGISView_PROPERTY_TYPE_DESC" <> 'InfRoad' AND 
"proprod_dbo_HCCnucPropertyLandGISView_PROPERTY_TYPE_DESC" <> 'InfSewMgmt' AND 
"proprod_dbo_HCCnucPropertyLandGISView_PROPERTY_TYPE_DESC" <> 'InfStaRail' AND 
"proprod_dbo_HCCnucPropertyLandGISView_PROPERTY_TYPE_DESC" <> 'ResUrbSHdr' AND 
"proprod_dbo_HCCnucPropertyLandGISView_PROPERTY_TYPE_DESC" <> 'ResRurVacL' AND 
"proprod_dbo_HCCnucPropertyLandGISView_PROPERTY_TYPE_DESC" <> 'ResUrbVacL' AND 
"proprod_dbo_HCCnucPropertyLandGISView_PROPERTY_TYPE_DESC" <> 'RsvNatPark' AND 
"proprod_dbo_HCCnucPropertyLandGISView_PROPERTY_TYPE_DESC" <> 'RsvNatStrp' AND 
"proprod_dbo_HCCnucPropertyLandGISView_PROPERTY_TYPE_DESC" <> 'RsvNature'

The suggested change should remove any records in the script above under Property Type Descriptions or a better way of saying it is that the layer should only show records that are NOT the types above.

I will make one more check before submitting the table. The boss reckons the map is correct though so these records in the table are probably slivers due to the topology not matching with Land Parcels (housekeeping issues, rather than a flaw in the workflow).

geoprocessing, spatial analysis, arcgis, model builder, septic systems, databases