finish draft glenn

ornldaac · Oct 7, 2024 · 8c7dca3 · 8c7dca3
1 parent 04e8fa1
commit 8c7dca3
Show file tree

Hide file tree

Showing 13 changed files with 3,159 additions and 20,286 deletions.
diff --git a/book/tutorials/EMIT/1_EMIT_intro.ipynb → book/tutorials/EMIT/EMIT_intro.ipynb b/book/tutorials/EMIT/1_EMIT_intro.ipynb → book/tutorials/EMIT/EMIT_intro.ipynb
@@ -5,13 +5,13 @@
    "id": "67b687d4-d0c7-4686-a970-cb32c095e4ea",
    "metadata": {},
    "source": [
-    "## Notebook Title: Exploring EMIT data and inspecting spectra for alien species\n",
+    "## Exploring EMIT data and inspecting spectra for alien species\n",
     "----\n",
     "\n",
     "### Overview \n",
     "----\n",
     "\n",
-    "In this notebook,  we will start analysing satellite imaging spectroscopy data. We will open data from [The Earth Surface Mineral Dust Source Investigation (EMIT)](https://earth.jpl.nasa.gov/emit/) and explore it. We will then extract some spectra for invasive plant species and visualize these.\n",
+    "In this notebook,  we will start analysing satellite imaging spectroscopy data. We will open data from [The Earth Surface Mineral Dust Source Investigation (EMIT)](https://earth.jpl.nasa.gov/emit/) stored remotely on AWS and explore it. We will then extract some spectra for invasive plant species and visualize these.\n",
     "\n",
     "- \n",
     "### Learning Objectives\n",
@@ -29,7 +29,7 @@
     "tags": []
    },
    "source": [
-    "## 1. Load libraries"
+    "#### Load libraries"
    ]
   },
   {
@@ -53,24 +53,8 @@
     "import panel as pn\n",
     "import xvec\n",
     "#our modules\n",
-    "import emit_tools #from https://github.com/nasa/EMIT-Data-Resources/blob/main/python/modules/emit_tools.py\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "37950de0",
-   "metadata": {},
-   "source": [
-    "#### custom functions"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "e0076660",
-   "metadata": {},
-   "outputs": [],
-   "source": [
+    "import emit_tools #from https://github.com/nasa/EMIT-Data-Resources/blob/main/python/modules/emit_tools.py\n",
+    "\n",
     "# Function to adjust gamma across all bands - adjust brightness\n",
     "def gamma_adjust(rgb_ds, bright=0.2, white_background=False):\n",
     "    array = rgb_ds.reflectance.data\n",
@@ -82,22 +66,12 @@
     "    return rgb_ds"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "id": "3687e606-e482-4ba7-bd61-ebfe4b57048f",
-   "metadata": {
-    "tags": []
-   },
-   "source": [
-    "## 2. Setup access"
-   ]
-  },
   {
    "cell_type": "markdown",
    "id": "92d11e01-4c1e-41d3-aea3-cfd55c2db496",
    "metadata": {},
    "source": [
-    "### Authenticate with NASA Earthdata portal\n",
+    "#### Authenticate with NASA Earthdata portal\n",
     "Earthaccess is a python library to search, download or stream NASA Earth science data. You will also need an account on NASA's Earthdata data portal\n",
     "https://search.earthdata.nasa.gov/"
    ]
@@ -115,20 +89,12 @@
     "auth = earthaccess.login(persist=True)"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "id": "150873cb-57e0-4eaf-a530-8be844676b7e",
-   "metadata": {},
-   "source": [
-    "## 3. Accessing and finding files"
-   ]
-  },
   {
    "cell_type": "markdown",
    "id": "f8ec817d-0213-47d0-8aaa-a114d0c47cb3",
    "metadata": {},
    "source": [
-    "### Accessing data on AWS S3\n",
+    "##### Accessing data on AWS S3\n",
     "\n",
     "The emit files are stored on Amazon S3 object storage. We will access these directly without needing to download them as we can stream s3 files. Also, because we are using xarray with dask and our data is stored in a chunked file format `netcdf`, we only need to stream the chunks we are currently working on and not the entire file at once.\n",
     "  \n",
@@ -140,7 +106,7 @@
    "id": "b0f6d101-f47d-4ec9-b708-675953cc9aac",
    "metadata": {},
    "source": [
-    "### Finding data"
+    "##### Finding data"
    ]
   },
   {
@@ -194,7 +160,7 @@
     "tags": []
    },
    "source": [
-    "## 4. Open the datset"
+    "#### Open the datset"
    ]
   },
   {
@@ -204,7 +170,7 @@
     "tags": []
    },
    "source": [
-    "### If you are in us-west-2: Authenticate with s3 and stream data"
+    "If you are in us-west-2: Authenticate with s3 and stream data"
    ]
   },
   {
@@ -249,7 +215,7 @@
     "tags": []
    },
    "source": [
-    "### If you are not in us-west-2: Download locally"
+    "If you are not in us-west-2: Download locally"
    ]
   },
   {
@@ -286,7 +252,7 @@
    "id": "6ac2589c-343c-4575-9a01-ad02c1f2d229",
    "metadata": {},
    "source": [
-    "### Load the mask\n",
+    "#### Load the mask\n",
     "The EMIT data is composed of 3 files, the actual data, a mask that tells us about various quality issues with the data, and the reflectance uncertainties (we will ignore the uncertainty for now). We can selecting which quality filter (mask) to use. Here we used the aggregate filter - the sum of all filters to be very strict about what data we include. The `quality_mask` function provided by the EMIT team reads and processes this file for us."
    ]
   },
@@ -310,7 +276,7 @@
     "tags": []
    },
    "source": [
-    "### Load the data\n",
+    "#### Load the data\n",
     "As you saw the raw data is the unprojected, unmasked focal plane array and does not include much metadata for us to tell what is going on.  The `emit_array` function provided by the EMIT team reads and processes this file for us. It can orthorectify the data and apply the mask too if we want.\n",
     "\n",
     "> Warning: in order to orthorectify the data, the entire array must be loaded into memory. This means you need a decent amount of RAM or the operation will fail. I found that 16GB is the minimum needed.\n"
@@ -373,7 +339,7 @@
    "id": "8426974c-4377-486a-bb71-b94f45cef95a",
    "metadata": {},
    "source": [
-    "### Quick plot"
+    "#### Quick plot"
    ]
   },
   {
@@ -467,7 +433,7 @@
     "tags": []
    },
    "source": [
-    "## 5. Extract points"
+    "#### Extract points"
    ]
   },
   {
@@ -488,7 +454,7 @@
    "outputs": [],
    "source": [
     "#load points\n",
-    "gdf = gpd.read_file(\"/home/gmoncrieff/emit_unmix.gpkg\")\n",
+    "gdf = gpd.read_file(\"shared/users/gmoncrieff/data/emit_unmix.gpkg\")\n",
     "#explore the data\n",
     "gdf[['Category','geometry']].explore('Category')"
    ]
@@ -531,8 +497,7 @@
     "             .to_dataframe() #turn into a dataframe\n",
     "             .reset_index() #convert the row names to a column\n",
     "             .merge( #merge with the original dataset\n",
-    "                 gdf\n",
-    "                 .reset_index(),on='index')\n",
+    "                 gdf.reset_index(),on='index')\n",
     "            )\n",
     "#select the columns we want\n",
     "extracted = extracted[['wavelengths','reflectance','geometry_x','Category','index']]\n",
@@ -564,7 +529,7 @@
    "id": "59c12525-e658-4c72-a4c6-eb8d3177b7fc",
    "metadata": {},
    "source": [
-    "## credits:\n",
+    "### credits\n",
     "\n",
     "This lesson has borrowed from:    \n",
     "\n",

diff --git a/book/tutorials/EMIT/data/emit_unmix.gpkg b/book/tutorials/EMIT/data/emit_unmix.gpkg