Lecture 02

class: inverse, center, title-slide, middle

# Lecture 02: Spatial Data

## Theory and Tools (a.k.a. GIS Tools Lab.)

### <img src="figs/unibo_logo_white.png" style="width: 15%" /> Bruno Conte

### 21/Sep/2023

---

# Spatial data in economics: schedule

1. ~~Introduction to (spatial) data and programming in `R`~~ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; [18.Sep.2023]

2. Spatial data basics: vector data + assignment &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; [21.Sep.2023]
 - Spatial data types (vector and raster) and data files
 - Basics of **vector data**: generating, wrangling, visualizing, exporting
 - Working with external files: loading, processing, exporting

3. Basic operations with vector data + assignment &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; [25.Sep.2023]

4. Geometry operations and miscelanea + follow-up &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; [28.Sep.2023]

5. Raster data and operations + assignment &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; [02.Oct.2023]

6. Take-home exam &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; [03.Nov.2023]

---

# Main references for this class

1. Lovelace, R., Nowosad, J. and Muenchow, J., 2019. **Geocomputation with R.** Chapman and Hall/CRC.

2. Pebesma, E., 2018. Simple Features for R: Standardized Support for Spatial Vector Data. The R Journal 10 (1), 439-446

3. Wickham, H. and Grolemund, G., 2016. R for data science: import, tidy, transform, visualize, and model data. " O'Reilly Media, Inc.".

---

# Spatial data types: vector and raster

- GIS systems represent spatial data in either vector or raster formats
 - **Vector data:** spatial geometries as a collection of points over a geography
 
 - Can represent different objects (points, lines, polygons, multiobjects)

.center[
<img src="https://r.geocompx.org/02-spatial-data_files/figure-html/sfcs-1.png" style="width: 80%" />
]

---
count: false

# Spatial data types: vector and raster

.center[
<img src="https://r.geocompx.org/02-spatial-data_files/figure-html/multis-1.png" style="width: 80%" />
]

---

# Spatial data types: vector and raster

- GIS systems represent spatial data in either vector or raster formats
 - **Raster data:** geography as continuos of pixels (gridcells) with associated values
 
 - Normally represents high resolution features of the geography (like an image)

.center[
<img src="https://r.geocompx.org/02-spatial-data_files/figure-html/raster-intro-plot-1.png" style="width: 70%" />
]

---

# Spatial data types: vector and raster

- Normally represents high resolution features of the geography (like an image)

.center[
<img src="https://r.geocompx.org/02-spatial-data_files/figure-html/raster-intro-plot2-1.png" style="width: 70%" />
]

---

# Spatial data files: vector and raster

- **Vector data:** file packages (usually multifiles)
 - Shapefiles (`*.shp`), contains also several auxiliar files (e.g. `*.dbf`, `*.shx`). Most used!
 
 - GeoJSON (`.json`) is written in Javascript (used mostly in web interfaces)
 
 - Geopackage (`*.gpk`), unique package/file
 
 - KMZ (`*.kmz`), from Google Earth format
 
- **Raster data:** imagery
 - `*.tiff` (most used)
 
 - Other image files (e.g. `jpeg`, `gif`, `png`)
 
 - NetCDF files (`*.nc`) standardized data for geoscience (CDF = common data format) 
 
---

# Spatial data: sources

There is almost **infinite** availability of spatial data in the internet. Here is a non-comprehensive list:

.pull-left[
- [Natural Earth:](https://www.naturalearthdata.com/downloads/) immense GIS database
- [SAGE:](https://sedac.ciesin.columbia.edu/data/sets/browse) also large GIS databse
- [DIVA:](https://www.diva-gis.org/gdata) nice GIS database by country
- [GADM:](https://gadm.org/data.html) country boundaries (ADM0-4)
- [USGS:](https://earthexplorer.usgs.gov/) satellite imagery
- [Modis:](https://www.earthdata.nasa.gov/learn/find-data/near-real-time/rapid-response/modis-subsets) satellite imagery
- [STRM:](https://dwtkns.com/srtm/) elevation
- [SAGE:](https://sage.nelson.wisc.edu/data-and-models/datasets/) land cover
- [GFC:](http://earthenginepartners.appspot.com/science-2013-global-forest/download_v1.1.html) forest change
- [gROADS:](https://sedac.ciesin.columbia.edu/data/collection/groads) road networks
- [Mineral Resources:](https://mrdata.usgs.gov/mrds/) location of minerals
]
.pull-right[
- [AQUASTAT:](https://www.fao.org/aquastat/en/databases/) water-related data
- [FAO-GAEZ:](https://gaez.fao.org/) farm/land-related data
- [Harvest Choice:](https://www.ifpri.org/project/harvestchoice) farm/land-related data
- [mapSPAM:](https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/PRFF8V) farm/land-related data
- [PS Lab:](https://psl.noaa.gov/data/gridded/index.html) temperature/precipitation
- [SPEI:](https://spei.csic.es/database.html) drought index
- [LSMS:](https://www.worldbank.org/en/programs/lsms) geocoded surveys
- [DHS surveys:](https://dhsprogram.com/data/) geocoded surveys
- [Geographic names:](https://geographic.org/geographic_names/index.html) to geocode localities
- [Long-lat:](https://www.latlong.net/convert-address-to-lat-long.html) API to geocode by names
- [NOAA VIIS:](https://www.ngdc.noaa.gov/eog/dmsp/downloadV4composites.html) Satellite Nighttime lights
]
 
---
class: inverse, center, middle
count: false

# Getting started: Vector data

# and the Simple Features in R

---

# Vector data and geographical projections

.pull-left[
- **Vector:** collection of points over a geography (longitude-latitude; i.e. X-Y)
 
- X-Y geographical axis: change depending on the geographical projection

- Same geometry can be represented by different combination of X-Y points

- Important takeaways:
 1. Know the data's projection system
 2. Standardize them in you applications
 
 
]

.pull-right[
<img src="https://pro.arcgis.com/en/pro-app/latest/help/mapping/properties/GUID-70E253E7-407E-469E-91DA-975B382EA6C9-web.png" style="width: 100%" />
]

---

# Vector data and geographical projections

- **Most usual is WGS 84:** longitude (-180,180), latitude (-90,90); CRS code EPSG:4326

- CRS = Coordinate Reference System (synonym to geographical projection)

.center[
<img src="https://i.stack.imgur.com/ILqJh.jpg" style="width: 70%" />
]

---

# Vector data in R: the simple features package

.pull-left[
- Spatial data in R: a Simple Feature (the `sf` library)

- State-of-art, standardized set of functions for GIS tasks

- Replace "old" libraries (e.g. `sp`, `rgdal`)

- Revolution on GIS in R (`#RSpatial`)
 - Interacts with `dplyr` "pipe" syntax
 
 - Computational- and memory-efficiency gains

]
.pull-right[
.center[
Downloads of `R` libraries:

<img src="https://r.geocompx.org/figures/01-cranlogs.png" style="width: 120%" />
]
]

---

# Vector data in R: the simple features package

- **Core elements of a Simple Feature:**
 1. Geometry (point, lines, polygons): a collection of points (`sfg`, simple feature geometry)
 
 2. Projection: a CRS parameter that places the points over the world's geography (`sfc`, simple feature column)
 
 3. Attributes: data associated with each feature/observation (1+2+3 = `sf`: simple feature)

.center[
<img src="figs/02-sfdiagram.png" style="width: 100%" />
]

---

# Vector data in R: the simple features package

- Representation of a Simple Feature in `R` console
.center[
<img src="figs/02-representation.png" style="width: 100%" />
]

---

# Vector data in R: creating simple features

1. Creating geometries:
 - Points: `st_point()` with a x-y **vector**
 
 - Lines: `st_linestring()` with a **matrix** of all x-y coordinates (columns) of each line vertex (rows)
 
 - Polygons: `st_polygon()` with a **list** containing a matrix of all x-y coordinates of each polygon vertex (first and last must be the same!)
 
2. Adding projection: `st_sfc(geometry,crs)`
 - Adds the `crs` projection to the `st_*()` geometry
 
 - WGS 84: use `crs = 'EPSG:4326'`

3. Creating a simple feature: `st_sf(data.attributes,geometry = sfc)`

---
class: inverse, center, middle
count: false

# Vector data with Simple Features:

# attribute data operations

---

# Vector data operations

- **Operations of spatial features** (i.e. manipulation): by attribute or geometry (spatial)
 - Attribute opperations: disciplined by the underlying attributes (feature's dataset)

- Spatial operations: manipulations across the space (i.e. rotating, moving, distances, etc.)
 
- Attribute data operations:
 - Nested on `dplyr` "pipe" operators/funtions (e.g. filter, slice, etc.)
 
 - Equivalent to data operations but also accounting for the geometry of the feature
 
- **Detailed exposition:** on class material `01_class02.R`

---

# Hands-in: your turn! (1/3)

.pull-left[
- Creating artificial spatial data with `sf`

- Generate the following features:
 - `MULTIPOINT ((3.2 4), (3 4.6), (3.8 4.4), (3.5 3.8), (3.4 3.6), (3.9 4.5))`
 
 - `LINESTRING (0 3, 0 4, 1 5, 2 5)`
 
 - `POLYGON ((0 0, 1 0, 3 2, 2 4, 1 4, 0 0))`
 
- Plot them together with `ggplot()`
]

.pull-right[
![](00_class02_files/figure-html/unnamed-chunk-1-1.png)

]

---

# Hands-in: your turn! (2/3)

- **Map of world airports:** download the shapefile of airports in the world from Natural Earth (large scale data). Differentiate airport types by color

.center[
<img src="figs/map_airports.png" style="width: 100%" />
]

---

# Hands-in: your turn! (3/3)

.pull-left[
- **Merge** the world's spatial data with World Bank's data `worldbank_df`
 - Watch out with missing values!
 
- Calculate (and plot) urban population share (%) in the Americas
 - Play with different projections!

- Calculate (and plot) average literacy in the Americas by subregion

- Use `filter()`, `mutate()`, `left_join()`, etc.
]

.pull-right[
![](00_class02_files/figure-html/unnamed-chunk-2-1.png)

]

---
count: false

# Hands-in: your turn! (3/3)

- Calculate (and plot) average literacy in the Americas by subregion

- Use `filter()`, `mutate()`, `left_join()`, etc.
]

.pull-right[
![](00_class02_files/figure-html/unnamed-chunk-3-1.png)

]

---
class: inverse, center, middle
count: false

# Your turn: Take-home

# Assignment

---

# Take-home assignment (1/2)

- **Main task:** replicate maps in academic publications/working papers in economics

- **Idea:** put in practice the `sf` tools to work with vector data

- **Delivery:** one document (.pdf,.html) featuring your code and the result of it
  - Hint: use R markdown to create a code notebook!
  
- **Deadline:** until next class (25 September 2023 8:00 am)

---

# Take-home assignment (2/2)

**Instructions:** search for, download, and reproduce the maps of the following papers:

1. Mettetal, E., 2019. *Irrigation dams, water and infant mortality: Evidence from South Africa* (**fig. 2:** hydro dams in South Africa)

2. Fried, S. and Lagakos, D., 2021. *Rural electrification, migration and structural transformation: Evidence from Ethiopia* (**fig. 4:** districts and electricity grid in Ethiopia)
 
3. Pellegrina, H.S. and Sotelo, S., 2021. *Migration, Specialization, and Trade: Evidence from Brazil's March to the West* (**fig. 2:** Population in Brazil's meso-regions (or districts) in different periods
 
4. Balboni, C.A., 2019. *In harm's way? infrastructure investments and the persistence of coastal cities*. Link [here](https://economics.mit.edu/sites/default/files/publications/Catastrophe_Risk_and_Settlement_Location.pdf) (**fig. 3:** Vietnam's road infrastructure by road type - if available)
 
5. Morten, M. & Oliveira, J., 2018. *The Effects of Roads on Trade and Migration: Evidence from a Planned Capital City* (**fig. 1:** Brazil's capital and main road infrastructure)