Contents

MODIS MOD021KM and FIRMS

Wildfires Sensors

Context

Purpose

Explore MODIS satellite imagery and wildfire data that is open and free for scientific use.

Sensor description

The MOD021KM product contains calibrated and geolocated at-aperture radiances for 36 discrete bands located in the 0.4 to 14.4 micron region of the electromagnetic spectrum.

Highlights

  • Use satpy to load, visualise, and regrid MODIS level 1B data.

  • Fetch a fire database containing some 497364 fires from 2020.

  • Visualisation of fire pixels from the database.

  • Visualisation of the fire pixels alongside bands from the MODIS satellite data.

Contributions

Notebook

Samuel Jackson (author), Science & Technology Facilities Council, @samueljackson92 Alejandro Coca-Castro (reviewer), The Alan Turing Institute, @acocac, 06/09/21 (latest revision)

Dataset originator/creator

MOD021KM

MODIS Characterization Support Team (MCST) MODIS Adaptive Processing System (MODAPS)

Firms

University of Maryland

Dataset authors

MOD021KM

MODIS Science Data Support Team (SDST)

Firms

NASA’s Applied Sciences Program

Dataset documentation

Note

The author acknowledges MODIS Science Team and the use of data and/or imagery from NASA’s Fire Information for Resource Management System (FIRMS) (https://earthdata.nasa.gov/firms), part of NASA’s Earth Observing System Data and Information System (EOSDIS).

Install and load libraries

! pip -q install pyspectral
! pip -q install 'satpy[modis_l1b]==0.26.0'

import pandas as pd
import numpy as np
import geopandas
import intake
import fsspec, aiohttp
import hvplot.xarray
import hvplot.pandas
import holoviews as hv
import panel as pn
import satpy
import xarray as xr
import tempfile
from pathlib import Path
from scipy.spatial import cKDTree
from satpy.writers import get_enhanced_image
from getpass import getpass
from pathlib import Path
from pyresample import geometry
import datetime
import urllib.request
import os.path

Fetch Data

Note

To download data from the NASA’s Earth Data site you must have a valid Earth Data account. Please register with Earth Data is you do not already have an account and then provide your username and password when prompted in the cell below.

Important

In order to download MODIS level1 data you must have authorized access to LAADS Web on your account. To do this navigate to your earthdata profile page

  • Click “Applications”

  • Click “Authorized Apps”

  • Click “Autorize More Applications”

  • Search for “LAADS Web”

  • Click “Authorize”

Now you should successfully be able to download MODIS data.

fname = 'MOD021KM.A2020245.0840.061.2020245193036.hdf'

if not os.path.isfile(fname) or os.path.getsize(fname) == 0:
    username = input('Username: ')
    password = getpass('Password: ')

    fsspec.config.conf['https'] = dict(client_kwargs={'auth': aiohttp.BasicAuth(username, password)})

    url = f'https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MOD021KM/2020/245/{fname}'
    filename = url.split('/')[-1]
    with fsspec.open(url) as f:
        with Path(filename).open('wb') as handle:
            data = f.read()
            try:
                data.decode('utf-8')
                raise RuntimeError('Could not download MODIS data! Have you authorized LAADS Web in your Eathdata account above?')
            except UnicodeDecodeError:
                handle.write(data)

Download the database of MODIS wildfires from a publically accessible location.

filepath = 'https://firms2.modaps.eosdis.nasa.gov/data/country/zips/'
filename = 'modis_2020_all_countries.zip'

if not os.path.isfile(filename) or os.path.getsize(filename) == 0:
    urllib.request.urlretrieve(filepath+filename, '../sensors/' + filename)
    !unzip modis_2020_all_countries.zip

Archive:  modis_2020_all_countries.zip
   creating: modis/2020/
  inflating: modis/2020/modis_2020_Afghanistan.csv  
  inflating: modis/2020/modis_2020_Albania.csv  
  inflating: modis/2020/modis_2020_Algeria.csv  
  inflating: modis/2020/modis_2020_Andorra.csv  
  inflating: modis/2020/modis_2020_Angola.csv  

  inflating: modis/2020/modis_2020_Antigua_and_Barbuda.csv  
  inflating: modis/2020/modis_2020_Argentina.csv  

  inflating: modis/2020/modis_2020_Armenia.csv  
  inflating: modis/2020/modis_2020_Australia.csv  

  inflating: modis/2020/modis_2020_Austria.csv  
  inflating: modis/2020/modis_2020_Azerbaijan.csv  
  inflating: modis/2020/modis_2020_Bahamas.csv  
  inflating: modis/2020/modis_2020_Bahrain.csv  
  inflating: modis/2020/modis_2020_Bangladesh.csv  
  inflating: modis/2020/modis_2020_Barbados.csv  
  inflating: modis/2020/modis_2020_Belarus.csv  
  inflating: modis/2020/modis_2020_Belgium.csv  
  inflating: modis/2020/modis_2020_Belize.csv  
  inflating: modis/2020/modis_2020_Benin.csv  
  inflating: modis/2020/modis_2020_Bermuda.csv  

  inflating: modis/2020/modis_2020_Bhutan.csv  
  inflating: modis/2020/modis_2020_Bolivia.csv  
  inflating: modis/2020/modis_2020_Bosnia_and_Herzegovina.csv  
  inflating: modis/2020/modis_2020_Botswana.csv  
  inflating: modis/2020/modis_2020_Brazil.csv  

  inflating: modis/2020/modis_2020_British_Virgin_Islands.csv  
  inflating: modis/2020/modis_2020_Brunei_Darussalam.csv  
  inflating: modis/2020/modis_2020_Bulgaria.csv  
  inflating: modis/2020/modis_2020_Burkina_Faso.csv  

  inflating: modis/2020/modis_2020_Burundi.csv  
  inflating: modis/2020/modis_2020_Cambodia.csv  
  inflating: modis/2020/modis_2020_Cameroon.csv  

  inflating: modis/2020/modis_2020_Canada.csv  
  inflating: modis/2020/modis_2020_Cape_Verde.csv  
  inflating: modis/2020/modis_2020_Cayman_Islands.csv  
  inflating: modis/2020/modis_2020_Central_African_Republic.csv  

  inflating: modis/2020/modis_2020_Chad.csv  
  inflating: modis/2020/modis_2020_Chile.csv  
  inflating: modis/2020/modis_2020_China.csv  

  inflating: modis/2020/modis_2020_Colombia.csv  
  inflating: modis/2020/modis_2020_Comoros.csv  
  inflating: modis/2020/modis_2020_Costa_Rica.csv  
  inflating: modis/2020/modis_2020_Cote_d_Ivoire.csv  
  inflating: modis/2020/modis_2020_Croatia.csv  
  inflating: modis/2020/modis_2020_Cuba.csv  

  inflating: modis/2020/modis_2020_Curacao.csv  
  inflating: modis/2020/modis_2020_Cyprus.csv  
  inflating: modis/2020/modis_2020_Czech_Republic.csv  
  inflating: modis/2020/modis_2020_Dem_Rep_Korea.csv  
  inflating: modis/2020/modis_2020_Democratic_Republic_of_the_Congo.csv  

  inflating: modis/2020/modis_2020_Denmark.csv  
  inflating: modis/2020/modis_2020_Djibouti.csv  
  inflating: modis/2020/modis_2020_Dominica.csv  
  inflating: modis/2020/modis_2020_Dominican_Republic.csv  
  inflating: modis/2020/modis_2020_Ecuador.csv  
  inflating: modis/2020/modis_2020_Egypt.csv  
  inflating: modis/2020/modis_2020_El_Salvador.csv  
  inflating: modis/2020/modis_2020_Equatorial_Guinea.csv  
  inflating: modis/2020/modis_2020_Eritrea.csv  
  inflating: modis/2020/modis_2020_Estonia.csv  
  inflating: modis/2020/modis_2020_Ethiopia.csv  

  inflating: modis/2020/modis_2020_Falkland_Islands.csv  
  inflating: modis/2020/modis_2020_Fiji.csv  
  inflating: modis/2020/modis_2020_Finland.csv  
  inflating: modis/2020/modis_2020_France.csv  
  inflating: modis/2020/modis_2020_French_Guiana.csv  
  inflating: modis/2020/modis_2020_French_Polynesia.csv  
  inflating: modis/2020/modis_2020_French_Southern_and_Antarctic_Lands.csv  
  inflating: modis/2020/modis_2020_Gabon.csv  
  inflating: modis/2020/modis_2020_Georgia.csv  
  inflating: modis/2020/modis_2020_Germany.csv  
  inflating: modis/2020/modis_2020_Ghana.csv  
  inflating: modis/2020/modis_2020_Greece.csv  
  inflating: modis/2020/modis_2020_Greenland.csv  
  inflating: modis/2020/modis_2020_Guadeloupe.csv  
  inflating: modis/2020/modis_2020_Guam.csv  
  inflating: modis/2020/modis_2020_Guatemala.csv  
  inflating: modis/2020/modis_2020_Guinea-Bissau.csv  
  inflating: modis/2020/modis_2020_Guinea.csv  

  inflating: modis/2020/modis_2020_Guyana.csv  
  inflating: modis/2020/modis_2020_Haiti.csv  
  inflating: modis/2020/modis_2020_Heard_I_and_McDonald_Islands.csv  
  inflating: modis/2020/modis_2020_Honduras.csv  
  inflating: modis/2020/modis_2020_Hong_Kong.csv  
  inflating: modis/2020/modis_2020_Hungary.csv  
  inflating: modis/2020/modis_2020_India.csv  

  inflating: modis/2020/modis_2020_Indonesia.csv  

  inflating: modis/2020/modis_2020_Iran.csv  
  inflating: modis/2020/modis_2020_Iraq.csv  
  inflating: modis/2020/modis_2020_Ireland.csv  
  inflating: modis/2020/modis_2020_Israel.csv  
  inflating: modis/2020/modis_2020_Italy.csv  
  inflating: modis/2020/modis_2020_Jamaica.csv  
  inflating: modis/2020/modis_2020_Japan.csv  
  inflating: modis/2020/modis_2020_Jordan.csv  
  inflating: modis/2020/modis_2020_Kazakhstan.csv  

  inflating: modis/2020/modis_2020_Kenya.csv  
  inflating: modis/2020/modis_2020_Kiribati.csv  
  inflating: modis/2020/modis_2020_Kosovo.csv  
  inflating: modis/2020/modis_2020_Kuwait.csv  
  inflating: modis/2020/modis_2020_Kyrgyzstan.csv  
  inflating: modis/2020/modis_2020_Lao_PDR.csv  
  inflating: modis/2020/modis_2020_Latvia.csv  
  inflating: modis/2020/modis_2020_Lebanon.csv  
  inflating: modis/2020/modis_2020_Lesotho.csv  
  inflating: modis/2020/modis_2020_Liberia.csv  

  inflating: modis/2020/modis_2020_Libya.csv  
  inflating: modis/2020/modis_2020_Lithuania.csv  
  inflating: modis/2020/modis_2020_Luxembourg.csv  
  inflating: modis/2020/modis_2020_Macedonia_Former_Yugoslav_Republic_of.csv  
  inflating: modis/2020/modis_2020_Madagascar.csv  
  inflating: modis/2020/modis_2020_Malawi.csv  
  inflating: modis/2020/modis_2020_Malaysia.csv  
  inflating: modis/2020/modis_2020_Maldives.csv  
  inflating: modis/2020/modis_2020_Mali.csv  

  inflating: modis/2020/modis_2020_Malta.csv  
  inflating: modis/2020/modis_2020_Martinique.csv  
  inflating: modis/2020/modis_2020_Mauritania.csv  
  inflating: modis/2020/modis_2020_Mauritius.csv  
  inflating: modis/2020/modis_2020_Mexico.csv  

  inflating: modis/2020/modis_2020_Moldova.csv  
  inflating: modis/2020/modis_2020_Mongolia.csv  
  inflating: modis/2020/modis_2020_Montenegro.csv  
  inflating: modis/2020/modis_2020_Montserrat.csv  
  inflating: modis/2020/modis_2020_Morocco.csv  
  inflating: modis/2020/modis_2020_Mozambique.csv  

  inflating: modis/2020/modis_2020_Myanmar.csv  
  inflating: modis/2020/modis_2020_Namibia.csv  

  inflating: modis/2020/modis_2020_Nepal.csv  
  inflating: modis/2020/modis_2020_Netherlands.csv  
  inflating: modis/2020/modis_2020_New_Caledonia.csv  
  inflating: modis/2020/modis_2020_New_Zealand.csv  
  inflating: modis/2020/modis_2020_Nicaragua.csv  
  inflating: modis/2020/modis_2020_Niger.csv  
  inflating: modis/2020/modis_2020_Nigeria.csv  
  inflating: modis/2020/modis_2020_Northern_Mariana_Islands.csv  
  inflating: modis/2020/modis_2020_Norway.csv  
  inflating: modis/2020/modis_2020_Oman.csv  
  inflating: modis/2020/modis_2020_Pakistan.csv  

  inflating: modis/2020/modis_2020_Palau.csv  
  inflating: modis/2020/modis_2020_Palestine.csv  
  inflating: modis/2020/modis_2020_Panama.csv  
  inflating: modis/2020/modis_2020_Papua_New_Guinea.csv  
  inflating: modis/2020/modis_2020_Paraguay.csv  
  inflating: modis/2020/modis_2020_Peru.csv  
  inflating: modis/2020/modis_2020_Philippines.csv  

  inflating: modis/2020/modis_2020_Poland.csv  
  inflating: modis/2020/modis_2020_Portugal.csv  
  inflating: modis/2020/modis_2020_Puerto_Rico.csv  
  inflating: modis/2020/modis_2020_Qatar.csv  
  inflating: modis/2020/modis_2020_Republic_of_Congo.csv  
  inflating: modis/2020/modis_2020_Republic_of_Korea.csv  
  inflating: modis/2020/modis_2020_Reunion.csv  
  inflating: modis/2020/modis_2020_Romania.csv  
  inflating: modis/2020/modis_2020_Russian_Federation.csv  

  inflating: modis/2020/modis_2020_Rwanda.csv  
  inflating: modis/2020/modis_2020_Saint_Helena.csv  
  inflating: modis/2020/modis_2020_Saint_Kitts_and_Nevis.csv  
  inflating: modis/2020/modis_2020_Saint_Lucia.csv  
  inflating: modis/2020/modis_2020_Saint_Vincent_and_the_Grenadines.csv  
  inflating: modis/2020/modis_2020_Samoa.csv  
  inflating: modis/2020/modis_2020_Sao_Tome_and_Principe.csv  
  inflating: modis/2020/modis_2020_Saudi_Arabia.csv  
  inflating: modis/2020/modis_2020_Senegal.csv  
  inflating: modis/2020/modis_2020_Serbia.csv  
  inflating: modis/2020/modis_2020_Seychelles.csv  

  inflating: modis/2020/modis_2020_Sierra_Leone.csv  
  inflating: modis/2020/modis_2020_Singapore.csv  
  inflating: modis/2020/modis_2020_Slovakia.csv  
  inflating: modis/2020/modis_2020_Slovenia.csv  
  inflating: modis/2020/modis_2020_Solomon_Islands.csv  
  inflating: modis/2020/modis_2020_Somalia.csv  
  inflating: modis/2020/modis_2020_South_Africa.csv  
  inflating: modis/2020/modis_2020_South_Sudan.csv  

  inflating: modis/2020/modis_2020_Spain.csv  
  inflating: modis/2020/modis_2020_Sri_Lanka.csv  
  inflating: modis/2020/modis_2020_Sudan.csv  

  inflating: modis/2020/modis_2020_Suriname.csv  
  inflating: modis/2020/modis_2020_Svalbard_and_Jan_Mayen.csv  
  inflating: modis/2020/modis_2020_Swaziland.csv  
  inflating: modis/2020/modis_2020_Sweden.csv  
  inflating: modis/2020/modis_2020_Switzerland.csv  
  inflating: modis/2020/modis_2020_Syria.csv  
  inflating: modis/2020/modis_2020_Taiwan.csv  
  inflating: modis/2020/modis_2020_Tajikistan.csv  
  inflating: modis/2020/modis_2020_Tanzania.csv  

  inflating: modis/2020/modis_2020_Thailand.csv  
  inflating: modis/2020/modis_2020_The_Gambia.csv  
  inflating: modis/2020/modis_2020_Timor-Leste.csv  
  inflating: modis/2020/modis_2020_Togo.csv  
  inflating: modis/2020/modis_2020_Tonga.csv  
  inflating: modis/2020/modis_2020_Trinidad_and_Tobago.csv  
  inflating: modis/2020/modis_2020_Tunisia.csv  
  inflating: modis/2020/modis_2020_Turkey.csv  
  inflating: modis/2020/modis_2020_Turkmenistan.csv  
  inflating: modis/2020/modis_2020_Turks_and_Caicos_Islands.csv  
  inflating: modis/2020/modis_2020_Uganda.csv  
  inflating: modis/2020/modis_2020_Ukraine.csv  

  inflating: modis/2020/modis_2020_United_Arab_Emirates.csv  
  inflating: modis/2020/modis_2020_United_Kingdom.csv  
  inflating: modis/2020/modis_2020_United_States.csv  

  inflating: modis/2020/modis_2020_United_States_Minor_Outlying_Islands.csv  
  inflating: modis/2020/modis_2020_United_States_Virgin_Islands.csv  
  inflating: modis/2020/modis_2020_Uruguay.csv  
  inflating: modis/2020/modis_2020_Uzbekistan.csv  
  inflating: modis/2020/modis_2020_Vanuatu.csv  
  inflating: modis/2020/modis_2020_Venezuela.csv  

  inflating: modis/2020/modis_2020_Vietnam.csv  
  inflating: modis/2020/modis_2020_Yemen.csv  
  inflating: modis/2020/modis_2020_Zambia.csv  

  inflating: modis/2020/modis_2020_Zimbabwe.csv

Load an intake catalog for the downloaded data

path = Path(tempfile.mkdtemp())
catalog_file = path / 'catalog.yaml'

with catalog_file.open('w') as f:
    f.write('''
sources:
    modis_l1b:
      args:
        urlpath: 'MOD021KM.A2020245.0840.061.2020245193036.hdf'
        reader: 'modis_l1b'
      description: "MODIS Level 1B Products"
      driver: SatpySource
      metadata: {}
    modis_fires:
      args:
        urlpath: 'modis/2020/modis_*.csv'
      description: "MODIS Level 2 Fires"
      driver: csv
      metadata: {}
''')

from intake.source.base import PatternMixin
from intake.source.base import DataSource, Schema

import glob

class SatpySource(DataSource, PatternMixin):
    """Intake driver for data supported by satpy.Scene"""
    
    container = 'python'
    name = 'satpy'
    version = '0.0.1'
    partition_access = False

    def __init__(self, urlpath, reader=None, metadata=None, path_as_pattern=True):
        """
        Parameters
        ----------
        path: str, location of model pkl file
        Either the absolute or relative path to the file
        opened. Some examples:
          - ``{{ CATALOG_DIR }}/file.nat``
          - ``{{ CATALOG_DIR }}/*.nc``
        reader: str, optional
        Name of the satpy reader to use when loading data (ie. ``modis_l1b``)
        metadata: dict, optional
        Additional metadata to pass to the intake catalog
        path_as_pattern: bool or str, optional
        Whether to treat the path as a pattern (ie. ``data_{field}.nc``)
        and create new coodinates in the output corresponding to pattern
        fields. If str, is treated as pattern to match on. Default is True.
        """

        self.path_as_pattern = path_as_pattern
        self.urlpath = urlpath
        self._reader = reader
        super(SatpySource, self).__init__(metadata=metadata)

    def _load(self):
        files = self.urlpath
        files = list(glob.glob(files))
        return satpy.Scene(files, reader=self._reader)
    
    def _get_schema(self):
        self._schema = Schema(
            npartitions=1,
            extra_metadata={}
        )
        return self._schema

    def read(self):
        self._load_metadata()
        return self._load()

intake.register_driver('SatpySource', SatpySource, overwrite=True)
cat = intake.open_catalog(catalog_file)

Load MODIS Satellite Data

Here we use satpy to load the MODIS level 1b data into memory. As well as loading the image data, satpy provides a easy way to regrid the data to different coordinate systems.

scn = cat['modis_l1b'].read()
scn

<satpy.scene.Scene at 0x7ff611fa13a0>

scn.load(['true_color', '20'], resolution=1000)
scn.show('true_color')

/Users/acoca/anaconda3/envs/envai-book/lib/python3.8/site-packages/dask/core.py:121: RuntimeWarning: invalid value encountered in log
  return func(*(_execute_task(a, cache) for a in args))
../../_images/wildfires-sensors-modis_13_1.png

Resample to a different projection

In the plot above you can see that the raw MODIS data has distortion towards edge of the image. By regridding the data we can avoid some of this distortion.

area_id = "Southern Africa"
description = "Southern Africa in Mercator-Projection"
proj_id = "Southern Africa"
proj_dict = {"proj": "eqc"}

width = 1000    # width of the result domain in pixels
height = 1000   # height of the result domain in pixels

llx =  23E5   # projection x coordinate of lower left corner of lower left pixel
lly =  -22.9E5   # projection y coordinate of lower left corner of lower left pixel
urx =  48E5   # projection x coordinate of upper right corner of upper right pixel
ury =  -1.9E5   # projection y coordinate of upper right corner of upper right pixel

area_extent = (llx,lly,urx,ury)
from pyresample import create_area_def
resolution=1000
center =(0,0)
area_def = create_area_def(area_id, proj_dict, center=center, resolution=resolution)

modis_scn = scn.resample(area_def, radius_of_influence=10000)

/Users/acoca/anaconda3/envs/envai-book/lib/python3.8/site-packages/pyproj/crs/crs.py:1216: UserWarning: You will likely lose important projection information when converting to a PROJ string from another format. See: https://proj.org/faq.html#what-is-the-best-format-for-describing-coordinate-reference-systems
  return self._crs.to_proj4(version=version)

modis_scn.show('true_color')

/Users/acoca/anaconda3/envs/envai-book/lib/python3.8/site-packages/dask/core.py:121: RuntimeWarning: invalid value encountered in log
  return func(*(_execute_task(a, cache) for a in args))
../../_images/wildfires-sensors-modis_16_1.png

Load MODIS Fire Database

Now we’re going to load the modis fire database from CSV files. This contains the longitude, latitude, and time of where fires have been detected to occur. It also includes an estimate of the Fire Radiative Power (FRP), a measure of the intensity of the fire, for each fire detected.

fires = cat['modis_fires'].read()

time = fires.acq_date.astype(str) + ' ' + fires.acq_time.astype(str).str.zfill(4)
fires['timestamp'] = pd.to_datetime(time, format='%Y-%m-%d %H%M')
fires = fires.loc[fires.satellite == 'Terra']
fires = geopandas.GeoDataFrame(
    fires, geometry=geopandas.points_from_xy(fires.longitude, fires.latitude))

# We're only interested in data from Southern Africa for now
llx =  0    # projection x coordinate of lower left corner of lower left pixel
lly =  -30  # projection y coordinate of lower left corner of lower left pixel
urx =  55   # projection x coordinate of upper right corner of upper right pixel
ury =  10   # projection y coordinate of upper right corner of upper right pixel

fires = fires.cx[llx:urx, lly:ury]
fires.set_index('timestamp', drop=False, inplace=True)
fires = fires.sort_index()
fires = fires.loc['2020-09-01 00:00:00':'2020-10-01 00:00:00']
fires
latitude longitude brightness scan track acq_date acq_time satellite instrument confidence version bright_t31 frp daynight type timestamp geometry
timestamp
2020-09-01 07:02:00 -15.8201 46.6282 320.8 1.2 1.1 2020-09-01 702 Terra MODIS 59 6.03 306.2 8.7 D 0 2020-09-01 07:02:00 POINT (46.62820 -15.82010)
2020-09-01 07:02:00 -15.8183 46.6171 323.7 1.2 1.1 2020-09-01 702 Terra MODIS 67 6.03 307.9 12.7 D 0 2020-09-01 07:02:00 POINT (46.61710 -15.81830)
2020-09-01 07:02:00 -15.7672 47.2667 324.8 1.1 1.1 2020-09-01 702 Terra MODIS 72 6.03 307.6 9.9 D 0 2020-09-01 07:02:00 POINT (47.26670 -15.76720)
2020-09-01 07:02:00 -15.7055 47.7069 328.2 1.1 1.0 2020-09-01 702 Terra MODIS 75 6.03 306.8 15.4 D 0 2020-09-01 07:02:00 POINT (47.70690 -15.70550)
2020-09-01 07:02:00 -15.0942 47.2146 320.4 1.2 1.1 2020-09-01 702 Terra MODIS 65 6.03 302.7 9.6 D 0 2020-09-01 07:02:00 POINT (47.21460 -15.09420)
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
2020-09-30 22:05:00 -4.5185 12.0523 306.7 1.8 1.3 2020-09-30 2205 Terra MODIS 69 6.03 288.2 17.6 N 2 2020-09-30 22:05:00 POINT (12.05230 -4.51850)
2020-09-30 22:06:00 -2.7462 10.7768 302.2 1.5 1.2 2020-09-30 2206 Terra MODIS 47 6.03 287.3 8.6 N 0 2020-09-30 22:06:00 POINT (10.77680 -2.74620)
2020-09-30 22:06:00 -0.8865 8.8320 302.2 1.2 1.1 2020-09-30 2206 Terra MODIS 48 6.03 289.0 4.9 N 0 2020-09-30 22:06:00 POINT (8.83200 -0.88650)
2020-09-30 22:06:00 -2.8421 10.8256 301.0 1.5 1.2 2020-09-30 2206 Terra MODIS 37 6.03 287.2 7.8 N 0 2020-09-30 22:06:00 POINT (10.82560 -2.84210)
2020-09-30 22:08:00 4.4222 7.1571 302.5 1.1 1.1 2020-09-30 2208 Terra MODIS 49 6.03 291.9 4.5 N 2 2020-09-30 22:08:00 POINT (7.15710 4.42220)

65459 rows × 17 columns

Daily Fire Radiative Power

Here we plot the Fire Radiative Power (FRP) for each day of the month of September. FRP is a measure of the intensity of a fire in units of MegaWatts.

from bokeh.models.formatters import DatetimeTickFormatter

def plot_timeseries(data, y_variable):
    """Timeseries plot showing the mean fire radiative power at each day as well as the 5% and 95%"""

    def percentile(n):
        def percentile_(x):
            return np.percentile(x, n)
        percentile_.__name__ = 'percentile_%s' % n
        return percentile_

    formatter = DatetimeTickFormatter(months='%b')

    bounds = data.groupby([data.index.day_of_year])[y_variable].agg([percentile(5), percentile(95)])
    avg = data.groupby([data.index.day_of_year])[y_variable].mean()

    bounds.index = data.groupby([data.index.day_of_year])['timestamp'].first()
    avg.index = bounds.index

    tseries = hv.Overlay([
        (bounds.hvplot.area('timestamp', 'percentile_5', 'percentile_95',
                      alpha=0.2, color='red', xformatter=formatter)),
            avg.hvplot.line(x='timestamp', label=f'Average Daily FRP', color='red', xformatter=formatter)])

    return tseries.options(width=800, height=400, xlabel='Day',ylabel=f'FRP (MW)',legend_position='top_left')

plot_timeseries(fires, 'frp')

Geographical distribution of Fires

Here we plot the geographical distribution of fires detected by MODIS over Southern Africa.

fires.hvplot.points('longitude', 'latitude', groupby='index.day_of_year', c='frp', geo=True, alpha=0.2,
                    tiles='ESRI', xlim=(llx, urx), ylim=(lly, ury), cmap='autumn', logz=True,
                    dynamic=False)

WARNING:param.GeoPointPlot11511: Log color mapper lower bound <= 0 and will not render correctly. Ensure you set a positive lower bound on the color dimension or using the `clim` option.

WARNING:param.GeoPointPlot11511: Log color mapper lower bound <= 0 and will not render correctly. Ensure you set a positive lower bound on the color dimension or using the `clim` option.

Visualising Fire Pixels with Satellite Imagery

Visualise a color image of the MODIS region using hvplot.

modis_dataset = modis_scn.to_xarray_dataset()

img = get_enhanced_image(modis_scn['true_color'].squeeze())
img = img.data
img = img.clip(0, 1)
img = (img * 255).astype(np.uint8)

modis_dataset['true_color'] = img

grid = modis_scn.min_area().get_lonlats()
lons, lats = grid[0][0], grid[1][:, 0]
modis_dataset = modis_dataset.assign_coords(dict(x=lons, y=lats))
modis_dataset = modis_dataset.rename(dict(x='longitude', y='latitude'))

rgb = modis_dataset['true_color'].hvplot.rgb(x='longitude', y='latitude', bands='bands', rasterize=True)
rgb

/Users/acoca/anaconda3/envs/envai-book/lib/python3.8/site-packages/pyproj/crs/crs.py:1216: UserWarning: You will likely lose important projection information when converting to a PROJ string from another format. See: https://proj.org/faq.html#what-is-the-best-format-for-describing-coordinate-reference-systems
  return self._crs.to_proj4(version=version)

/Users/acoca/anaconda3/envs/envai-book/lib/python3.8/site-packages/dask/core.py:121: RuntimeWarning: invalid value encountered in log
  return func(*(_execute_task(a, cache) for a in args))

/Users/acoca/anaconda3/envs/envai-book/lib/python3.8/site-packages/dask/core.py:121: RuntimeWarning: invalid value encountered in log
  return func(*(_execute_task(a, cache) for a in args))

/Users/acoca/anaconda3/envs/envai-book/lib/python3.8/site-packages/dask/core.py:121: RuntimeWarning: invalid value encountered in log
  return func(*(_execute_task(a, cache) for a in args))

OMP: Info #271: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.

Get the fire pixels that are visible in the the MODIS scene by filtering the time, longitude & latitude. We can also compute which row and column the fire pixel is in for this projection.

grid = modis_scn.min_area().get_lonlats()

# Mask any fire pixels not in this scene
time_mask = np.logical_and(fires.timestamp >= scn.attrs['start_time'],
                           fires.timestamp <= scn.attrs['end_time'])
fire_points = fires.loc[time_mask]
fire_points = fire_points.cx[grid[0].min():grid[0].max(), grid[1].min():grid[1].max()]

# extract the x and y coordinates as flat arrays
x = np.ravel(grid[0])
y = np.ravel(grid[1])
points = np.dstack([x, y]).squeeze()

# Find locations of fire pixels within satellite image
tree = cKDTree(points)
distance, idx = tree.query(fire_points[['longitude', 'latitude']], k=1)
index = np.unravel_index(idx, grid[0].shape)
index = np.vstack(index).T
index

# fire_points[['y', 'x']] = index
fire_points = fire_points[img.values[0, index[:, 0], index[:, 1]] != 0]
fire_points

/Users/acoca/anaconda3/envs/envai-book/lib/python3.8/site-packages/dask/core.py:121: RuntimeWarning: invalid value encountered in log
  return func(*(_execute_task(a, cache) for a in args))
latitude longitude brightness scan track acq_date acq_time satellite instrument confidence version bright_t31 frp daynight type timestamp geometry
timestamp
2020-09-01 08:40:00 -12.0362 30.9757 321.5 1.8 1.3 2020-09-01 840 Terra MODIS 67 6.03 305.2 20.0 D 0 2020-09-01 08:40:00 POINT (30.97570 -12.03620)
2020-09-01 08:40:00 -11.6813 29.0762 328.7 1.3 1.1 2020-09-01 840 Terra MODIS 79 6.03 307.0 20.8 D 0 2020-09-01 08:40:00 POINT (29.07620 -11.68130)
2020-09-01 08:40:00 -11.9208 31.1634 323.1 1.8 1.3 2020-09-01 840 Terra MODIS 65 6.03 306.3 24.3 D 0 2020-09-01 08:40:00 POINT (31.16340 -11.92080)
2020-09-01 08:40:00 -12.0026 30.6289 326.0 1.7 1.3 2020-09-01 840 Terra MODIS 75 6.03 304.3 31.9 D 0 2020-09-01 08:40:00 POINT (30.62890 -12.00260)
2020-09-01 08:40:00 -11.7261 29.1128 325.1 1.3 1.1 2020-09-01 840 Terra MODIS 71 6.03 304.9 16.8 D 0 2020-09-01 08:40:00 POINT (29.11280 -11.72610)
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
2020-09-01 08:42:00 -17.1728 20.0847 329.4 1.5 1.2 2020-09-01 842 Terra MODIS 79 6.03 304.9 30.4 D 0 2020-09-01 08:42:00 POINT (20.08470 -17.17280)
2020-09-01 08:42:00 -18.4965 19.8103 324.7 1.5 1.2 2020-09-01 842 Terra MODIS 72 6.03 306.0 20.2 D 0 2020-09-01 08:42:00 POINT (19.81030 -18.49650)
2020-09-01 08:44:00 -27.9381 24.6669 328.1 1.2 1.1 2020-09-01 844 Terra MODIS 83 6.03 297.1 27.9 D 0 2020-09-01 08:44:00 POINT (24.66690 -27.93810)
2020-09-01 08:44:00 -27.9366 24.6552 311.5 1.2 1.1 2020-09-01 844 Terra MODIS 62 6.03 296.1 7.7 D 0 2020-09-01 08:44:00 POINT (24.65520 -27.93660)
2020-09-01 08:44:00 -27.4436 24.7104 320.0 1.1 1.1 2020-09-01 844 Terra MODIS 70 6.03 299.1 15.0 D 0 2020-09-01 08:44:00 POINT (24.71040 -27.44360)

898 rows × 17 columns

Now overlay the fire pixels ontop of the MODIS image, along with the FRP for each fire pixel. Try zooming in, you should be able to see clear smoke trails at the locations of some of the fires!

rgb = modis_dataset['true_color'].hvplot.rgb(x='longitude', y='latitude', bands='bands', data_aspect=1, hover=False, title='True Colour', rasterize=True)
points = fire_points.hvplot.points('longitude', 'latitude', c='frp', cmap='autumn', logz=True, alpha=0.4)
rgb*points

/Users/acoca/anaconda3/envs/envai-book/lib/python3.8/site-packages/dask/core.py:121: RuntimeWarning: invalid value encountered in log
  return func(*(_execute_task(a, cache) for a in args))

/Users/acoca/anaconda3/envs/envai-book/lib/python3.8/site-packages/dask/core.py:121: RuntimeWarning: invalid value encountered in log
  return func(*(_execute_task(a, cache) for a in args))

/Users/acoca/anaconda3/envs/envai-book/lib/python3.8/site-packages/dask/core.py:121: RuntimeWarning: invalid value encountered in log
  return func(*(_execute_task(a, cache) for a in args))

We can also overlay the fire pixels ontop of the MODIS 3.7 micron wavelength band. The 3.7 band is a thermal band. Fires will appear as very bright pixels in image. Try zooming in, you should be able to clearly see bright spots at the location of each fire pixel.

rgb = modis_dataset['20'].hvplot.image(x='longitude', y='latitude', cmap='viridis', data_aspect=1, hover=False, title='Band 20: 3.7 micron', rasterize=True)
points = fire_points.hvplot.points('longitude', 'latitude', c='frp', cmap='autumn', logz=True, alpha=0.4)
rgb*points

/Users/acoca/anaconda3/envs/envai-book/lib/python3.8/site-packages/dask/core.py:121: RuntimeWarning: invalid value encountered in log
  return func(*(_execute_task(a, cache) for a in args))

/Users/acoca/anaconda3/envs/envai-book/lib/python3.8/site-packages/dask/core.py:121: RuntimeWarning: invalid value encountered in log
  return func(*(_execute_task(a, cache) for a in args))

/Users/acoca/anaconda3/envs/envai-book/lib/python3.8/site-packages/dask/core.py:121: RuntimeWarning: invalid value encountered in log
  return func(*(_execute_task(a, cache) for a in args))

/Users/acoca/anaconda3/envs/envai-book/lib/python3.8/site-packages/dask/core.py:121: RuntimeWarning: invalid value encountered in log
  return func(*(_execute_task(a, cache) for a in args))

/Users/acoca/anaconda3/envs/envai-book/lib/python3.8/site-packages/dask/core.py:121: RuntimeWarning: invalid value encountered in log
  return func(*(_execute_task(a, cache) for a in args))

/Users/acoca/anaconda3/envs/envai-book/lib/python3.8/site-packages/dask/core.py:121: RuntimeWarning: invalid value encountered in log
  return func(*(_execute_task(a, cache) for a in args))

Summary

This notebook has demonstrated the use of:

  • The satpy package to easily load, plot and regrid satellite data from the MODIS sensor.

  • hvplot to interatively visualise wildfire pixels detected from the MODIS sensor.

  • geopandas to load, filter, and manipulate historical wildfire pixel data.

Version

Wildfires: Sensors Wildfires: Modelling