Skip to content
This repository has been archived by the owner on Oct 3, 2023. It is now read-only.

skip exporting points with no value due to prometheus staleness markers #303

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

kwiesmueller
Copy link
Contributor

This fixes cases where a staleness marker causes errors like:

rpc error: code = InvalidArgument desc = One or more TimeSeries could not be written: Field timeSeries[94].points[0].value had an invalid value: A point has an unrecognized value type.; Field timeSeries[95].points[0].value had an invalid value: A point has an unrecognized value type.\nerror details: name = Unknown  desc = total_point_count:155 success_point_count:153 errors:{status:{code:3} point_count:2}

By properly dropping any point that is a staleness marker instead of exporting them without a value.

The fixed issue was introduced with #300.
The bug should not cause any data loss, it just causes error logs and an increase in exporter error metrics.

@kwiesmueller
Copy link
Contributor Author

/assign @dashpole

@kwiesmueller
Copy link
Contributor Author

Actually, my test just failed as we're now sending timeseries with no points which is also not okay :-/
Needs more work.

@kwiesmueller kwiesmueller force-pushed the value-type-fix branch 2 times, most recently from bffcda0 to ef6be80 Compare August 9, 2022 18:29
@kwiesmueller
Copy link
Contributor Author

Alright, works now in our collector.
PTAL.

@dashpole
Copy link
Contributor

dashpole commented Aug 9, 2022

/gcbrun

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants