Error in plugin [inputs.exec]: metric parse error: expected field at offset 16736:

[accodepull]

I get this error (in title) when I run telegraf however if I run the command on its own its seems that I get all the data displayed on the screen without issues:
./vsanmetrics.py -s vcenter.xxx.xxx -u administrator@vsphere.local -p xxxxx -c xxxx --performance --capacity --health

Running vCenter 6.7.0.11000 and hosts are with 6.7.0, 8169922

Is this something to do with data format retrieved by telegraf and not compliant with influxdb?

[equelin]

Hello @contiandrea,

Could you check if the issue appears with --performance, --capacity or --health parameters? And then, could you provide the output of the command by running directly the script?

Thanks

[accodepull]

Hello, Am pretty sure the issue occurred when there was —performance only. However running the command on its own with —performance seems to work fine as can see lots of data on the output Will only have access to the same system late next week so can give it another try, in meantime let me know if you can suggest anything further. Thanks Get Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: Erwan Quélin <notifications@github.com> Sent: Tuesday, September 4, 2018 10:08:56 AM To: equelin/vsanmetrics Cc: contiandrea; Mention Subject: Re: [equelin/vsanmetrics] Error in plugin [inputs.exec]: metric parse error: expected field at offset 16736: (#5) Hello @contiandrea<https://github.com/contiandrea>, Could you check if the issue appears with --perfoamnce, --capacity or --health parameters ? And then, could you provide the output of the command by running directly the script ? Thanks — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#5 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AdItLgFj3kTOj42bpN5PRaJxbEZnmdzOks5uXkMogaJpZM4WWO-3>.

[knolan1]

If i may share some experience, I had also a parsing issue running the script within telegraf, and it was also running perfectly fine in shell. (with any performance/capacity or health).

I was using a password with some specific caracters (like "!" ) and after many try to escape these fields (-u and -p ) I add to change to the Adminstrator account and another password without "!".

Don't know for sure if this will help, but give it a try.
Regards,

[accodepull]

@knolan1 - thanks will give that a try

@equelin - please see below (when --performance is used)

Traceback (most recent call last): File "./vsanmetrics.py", line 494, in <module> main() File "./vsanmetrics.py", line 445, in main except vmodl.fault.NotFound as e: File "/usr/lib/python2.7/site-packages/pyVmomi/VmomiSupport.py", line 262, in __getattr__ raise AttributeError(attr) AttributeError: NotFound

[equelin]

Hello @knolan1, thanks for the feedback! I was not aware of that...

@accodepull , I'll need to find some time to look at this issue... In the meantime, could you check if you're using the latest release of the script ? Could you try to run the script with the same user who runs the telegraf service?

Regards,

[accodepull]

@equelin - am pretty sure I am not, should I just replace the vsanmetrics.py? (Are there any other dependencies I should replace), thanks

[equelin]

Hello,

Please test with the last release. Yes you just have to replace the file. Also be sure that the user who's running telegraf (he's named telegraf on my ubuntu server) can access to all the prerequisites (pyvmomi, vSAN sdk, etc...).

Feel free to ask if needed.

[accodepull]

Hi, No luck after replacing the file and running —performance (—capacity still works fine), please see below: Process Process-1: Traceback (most recent call last): File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run self._target(*self._args, **self._kwargs) File "./vsanmetrics.py", line 650, in getPerformance except vmodl.fault.NotFound as e: File "/usr/lib/python2.7/site-packages/pyVmomi/VmomiSupport.py", line 262, in __getattr__ raise AttributeError(attr) AttributeError: NotFound Get Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: Erwan Quélin <notifications@github.com> Sent: Monday, October 15, 2018 2:33:58 PM To: equelin/vsanmetrics Cc: codepull; Mention Subject: Re: [equelin/vsanmetrics] Error in plugin [inputs.exec]: metric parse error: expected field at offset 16736: (#5) Hello, Please test with the last release. Yes you just have to replace the file. Also be sure that the user who's running telegraf (he's named telegraf on my ubuntu server) can access to all the prerequisites (pyvmomi, vSAN sdk, etc...). Feel free to ask if needed. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#5 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AdItLi4x3hWRRzlvlzQ_qSzx-5BnXXthks5ulI7GgaJpZM4WWO-3>.

[equelin]

Ok then, I think we should start again from the beginning ;-) Could you provide more insights about your environment ?

• OS
• Python version
• Pyvmomi version
• vSAN SDK for Python version
• vCenter / VSAN version

Regards

[accodepull]

sure, the ultimate machine and target environment I want to run this on are: (and on which I got the last couple of errors I sent you)

OS --> CentOS 7
Python version --> 2.7.5
Pyvmomi version --> 6.7.0
vSAN SDK for Python version -- not sure how to obtain the version
vCenter / VSAN version --> VCSA 6.5U2/vSAN 6.2

[accodepull]

@equelin - did you manage to replicate the issue/understand why it is failing?

[myusufe]

I have the same issue, did anyone get resolve on this issue?

[accodepull]

am unaware of any fix as did not receive further feedback from @equelin

[myusufe]

I found the problem. To monitor VSAN performance, you have to turn on VSAN performance option on your vCenter.
Go to Monitor --> Performance --> Turn on VSAN Performance metrics.

Hope it will help you.

[equelin]

Hello @myusufe, @accodepull,

I'm currently starting a new gig and I don't have time to work on my side projects, I'm really sorry if I didn't answer sooner...

@accodepull, did you activated the performance service on your vSAN cluster ?

@myusufe, thanks for the feedback, and my apologize if the documentation is not clear about that. It reminds me that not everyne has the same knowledge and things that may be obvious for me may not be crystal clear for others... Thanks again !!

[accodepull]

yes performance service is enabled for all the clusters....can I specify just once cluster when running the script since I have multiple ones?

[equelin]

Hello @accodepull,

For now, you can specify only one cluster name per command.

[d-zalewski]

I'm seeing similar issue. It seems like some metrics output is not correct influx data format.

vSAN 6.5 update 2c
telegraf 1.9.3
influxdb 1.7.3
python-2.7.5-76.el7

[d-zalewski]

OK so it looks like the issue I'm having is that one of VMs has a space in the name. For example virtual-disk disk= value has a space in the name. Once I changed the name so it doesn't have space, metric format was correct

"In the line protocol, tag keys and values and field keys should not be quoted, spaces and commas must always be escaped by a backslash \

"For field values, strings should be single-quoted and spaces or commas need not be escaped.
For measurements always use a backslash character \ to escape: commas and spaces
Users do not need to escape all other special characters."

[equelin]

Hello @d-zalewski, thanks for your feedback.

I should add checks for that particuliar case, I just need to find some time to do it...

[accodepull]

@d-zalewski - did you get the same error as I posted on Oct 15, 2018?

Script was working fine and stopped after upgrade to VCSA 6.5U2 due to mentioned error.

[vblogio]

Hi all, Erwan, same problem since vSphere 6.7u1 upgrade from 6.5u2. It seems that this occurs, in my case, while retrieving the "vsan-cluster-capacity" entity type.

Hope this helps.

[equelin]

This is really strange, it works very well on my setup but I don't have time/possibility to test every configuration/version etc...

@vblogio , could you tell me if:

• You upgraded pyvmomi and vSAN SDK to the latest version ?
• You upgraded vsanmetrics to the latest release ?

Regards

[vblogio]

I just updated the vsanmetrics script to the latest version. Didn't already touched the pyvmoni. I'll try this.

[equelin]

Any update @vblogio ?

[bogski87]

Hi folks, i might be late to the party here but....

I've been having an issue with getting VSAN performance metrics into our monitoring system for about a week and stumbled upon this KB article

I noticed there was some people here who had updated to 6.7 who were having the same issue as me and it only began after we upgraded one of our clusters. We updated our version of the SDK but this didn't solve the issue.

The fix in the end was to disable the performance metrics in VSAN and then enable them again. You do lose the performance data that has been collected but performance metrics should start working again.

Not sure if everyone worked it out already but thought it might be worth dropping the link in this issue.