Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix data freq y new #68

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

semvijverberg
Copy link
Member

@semvijverberg semvijverberg commented Jan 31, 2024

First of all, sorry for the confusing name of the PR. For us, this issue arises when we were dealing with yearly data (1 datapoint per year). Hence the initial name.

A better suited name would be: issues due to interval definition.

The following issue arise because the interval is not closed on both sides (but currently the intervals is closed on the left).

Example1 :
You have preprocessed target data with a single datapoint per year (say, the yield per year). You have assigned the timestamp of the first of October. The calender should check, [2015-10-01, 2015-10-02], but the last datapoint in 2015 is 2015-10-01, it cannot find the any closing date that is >= 2015-10-02 and thus it will discard the year 2015.

This exact problem also occurs when you have data up to 2015 and ask for an interval up to and including the 31st of December.

Example 2:
The target interval that include dates up to 31st of December, say the start point is the 1st of December.
Thus, intervals are defined as [2000-12-01 , 2001-01-01), [2001-12-01 , 2002-01-01), ...]. The last interval will be [2015-12-01 , 2016-01-01). When indexing this on our input array we will encounter missing data as we try to index 2016-01-01 but our data only runs until 2015-12-31. Hence, lilio discards the year 2015 as it cannot fit the data to the interval. The obvious solution, then, is to use open indices on both sides: thus our intervals become [2000-12-01, 2000-12-31].

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants