Include Electricity Maps open data in library

[fershad]

Is your feature request related to a problem? Please describe.
Currently, there is only one source of annual average grid intensity data available in this library. However, we know that Electricity Maps also publishes their data on an annual basis and that it is available under a very open, very permissive license.

Describe the solution you'd like
@mrchrisadams has demonstrated how this data can be collected, aggregated, and displayed in this Observable Notebook. There, he is using the hourly data provided by Electricity Maps, however that would likely be too much to ship with CO2.js (keeping package size in mind). Instead, we could look to use the annual (yearly) data they provide.

I am unsure of an easy way to get all the data from the Electricity Maps portal, besides going through each country and downloading it manually before combining it. Very open to hearing other ideas that could help automate this.

Additional context
This would be handy to have in CO2.js, and would be useful for the Grid-aware Websites library as well.

[fershad]

An important thing to note here is that the default functionality should not change. Someone doing import { averageIntensities } from "@tgwf/co2" should still receive the current average intensity data available.

What this change might open up is the chance for us to export data as a separately. Allowing users to do something like this:

import { electricityMapsAnnual } from "@tgwf/co2/data"
import { averageIntensityAnnual } from "@tgwf/co2/data" // This would be the current average intensity data that's available
import { marginalIntensityAnnual } from "@tgwf/co2/data" // This would be the current marginal intensity data that's available

[mrchrisadams]

oh hey @fershad -

I am unsure of an easy way to get all the data from the Electricity Maps portal, besides going through each country and downloading it manually before combining it. Very open to hearing other ideas that could help automate this.

We have a script for this from the first time I did this, and it's in the data-analysis repo for the platform. I've sent a link to you directly to the link for that code.

I've also attached the notebook in html form, which includes the code used, and commentary for creating a parquet file containing every hour of usage for every region in a given year range.

The size of the parquet file is about 14mb for 2023, and we could use a much smaller subset of these, with only one reading for each region, instead of 8760 of them (!) to make a nice rich dataset of annual figures to compare against if they don't already exist.

Electricity maps were pretty good about publishing open data last year, so I think it's plausible we could do the same thing for 2024 as soon as it's ready.

fetch-emaps-open-data.html.zip