Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stream upload facilities #1

Open
eloylp opened this issue Jul 9, 2022 · 0 comments
Open

Stream upload facilities #1

eloylp opened this issue Jul 9, 2022 · 0 comments
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@eloylp
Copy link
Owner

eloylp commented Jul 9, 2022

Problem

Goomerang its currently using protocol buffers as message serialization mechanism. That implies when sending/receiving messages, we need to load them entirely in memory. There are also limits in terms of sizes for proto message specification, being currently the maximum size per message around 2 GB.

In most of the use cases, proto messages are very lightweight and they only carry little application data. But there are cases like uploads that would benefit from an streaming protocol.

Goals

Research and provide to the users of the library a high level streaming protocol, built on the top of the current library facilities. It should be able to split large messages in chunks. We should keep in the equation the standard library. It should play well with the io.Reader and io.Writer Go interfaces.

This feature would cover a very specific use case, so it should probably not be part of the main client and server code.

@eloylp eloylp added enhancement New feature or request help wanted Extra attention is needed labels Jul 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant