Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: [net][linux]: add netlink implementation for Linux connection stat #809

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

shirou
Copy link
Owner

@shirou shirou commented Dec 26, 2019

This PR introduces netlink support for Linux Connections.
To use netlink, users should add build tag -tags=netlink

Thanks for sample code of @Brian-Williams in #695.

I choose build tag to change implementation. Because current implementation is enough for most of users, I think. Some of users want to use netlink to work with their big system. They must be their environment and they can compile binary by their own. so adding build tag is not so big problem, I think.

This is what only I think. There must be other objection. Comments are welcome as always.

Limitation for using netlink

  • fd is always 0
    • to get fd becomes slow
  • support Linux 3.3 or later only (inet_diag_req_v2 is required)
    • until that, inet_diag has not Protocol field

TODO

  • benchmark

@shirou
Copy link
Owner Author

shirou commented Dec 31, 2019

Note: this version of netlink does NOT return FD and ProcessID from inode

func BenchmarkGetConnectionsInet(b *testing.B) {
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		Connections("inet")
	}
}
func BenchmarkGetConnectionsAll(b *testing.B) {
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		Connections("all")
	}
}
goos: linux
goarch: amd64
pkg: github.com/shirou/gopsutil/net
netlink
BenchmarkGetConnectionsInet-2               1162            882153 ns/op           84048 B/op       1792 allocs/op
BenchmarkGetConnectionsAll-2                 572           1958050 ns/op          458220 B/op      12660 allocs/op
original
BenchmarkGetConnectionsInet-2                 63          21234560 ns/op         3640324 B/op      40404 allocs/op
BenchmarkGetConnectionsAll-2                  57          23370421 ns/op         4523979 B/op      47422 allocs/op

@shirou
Copy link
Owner Author

shirou commented Dec 31, 2019

So getting Pid and FD from inodes IS a bottleneck.

goos: linux
goarch: amd64
pkg: github.com/shirou/gopsutil/net
netlink:
BenchmarkGetConnectionsInet-2                 66          20531232 ns/op         3297398 B/op      39004 allocs/op
BenchmarkGetConnectionsAll-2                  57          19901569 ns/op         3714031 B/op      50611 allocs/op
original:
BenchmarkGetConnectionsInet-2                 52          22912904 ns/op         3950026 B/op      44519 allocs/op
BenchmarkGetConnectionsAll-2                  43          24103518 ns/op         4856759 B/op      51875 allocs/op

@Lomanic
Copy link
Collaborator

Lomanic commented Jan 6, 2020

So getting Pid and FD from inodes IS a bottleneck.

Yes, #784 was opened regarding this.

Awesome work you've done here btw, using a build tag is an original solution but is it optimal?

@Hugozys
Copy link

Hugozys commented Jul 17, 2023

Hi! I followed this PR from #695. Thanks for working on adding this awesome feature! I just wanted to check and see if there is still plan to work on and merge this PR.

@shirou shirou added the v4 label Nov 5, 2023
florianl added a commit to florianl/gopsutil that referenced this pull request Jun 5, 2024
To retrieve network information use NETLINK_SOCK_DIAG instead of walking
/proc/<PID> and parsing files.
Consequently this reduces the number of syscalls, as walking /proc/<PID> and
opening, reading and closing files is no longer required. But it also reduces
the memory footprint as reading files into memory for processing is no longer
required.

Related issues:
- shirou#695
- shirou#784

Supersedes shirou#809

Signed-off-by: Florian Lehner <dev@der-flo.net>
florianl added a commit to florianl/gopsutil that referenced this pull request Jun 8, 2024
To retrieve network information use NETLINK_SOCK_DIAG instead of walking
/proc/<PID> and parsing files.
Consequently this reduces the number of syscalls, as walking /proc/<PID> and
opening, reading and closing files is no longer required. But it also reduces
the memory footprint as reading files into memory for processing is no longer
required.

Related issues:
- shirou#695
- shirou#784

Supersedes shirou#809

Signed-off-by: Florian Lehner <dev@der-flo.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants