Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add examples #1

Open
koalalorenzo opened this issue Mar 22, 2019 · 27 comments
Open

Add examples #1

koalalorenzo opened this issue Mar 22, 2019 · 27 comments

Comments

@koalalorenzo
Copy link

This project looks promising :)
It would be nice to add some examples to how to use it

@hsanjuan
Copy link
Owner

I'm not sure I wanted to commit this but since I did, it might be useful until I write a better example: https://github.com/hsanjuan/ipfs-lite/blob/d2191f217f81879baf5e5d09d51d5233628ac430/test/ipfs-lite.go

@bonedaddy
Copy link
Contributor

@hsanjuan The example is giving me the following output

02:12:02.368 ERROR   ipfslite: dial attempt failed: no good addresses ipfs.go:122
02:12:02.368 ERROR   ipfslite: dial attempt failed: no good addresses ipfs.go:122
02:12:02.369 ERROR   ipfslite: dial attempt failed: no good addresses ipfs.go:122
02:12:02.369 ERROR   ipfslite: dial attempt failed: no good addresses ipfs.go:122
Hello World

What's the error message being caused by?

@hsanjuan
Copy link
Owner

hey @postables that's probably because some of the default bootstrap peers are offline or don't work when it tries to connect to them. Maybe worth reducing this to a warning... however it does connect correctly to other bootstrap peers and as you see it was able to fetch the content from somewhere.

@bonedaddy
Copy link
Contributor

ah okay that makes sense, thanks!

@probonopd
Copy link

Getting

me@host:~$ go run /home/me/go/src/github.com/hsanjuan/ipfs-lite/examples/litepeer/litepeer.go 
2019-10-27T20:03:00.795+0100	WARN	ipfslite	go-log/log.go:187	failed to dial : all dials failed
  * [/ip6/2a03:b0c0:0:1010::23:1001/tcp/4001] failed to negotiate security protocol: rsa keys must be >= 2048 bits to be useful
  * [/ip4/178.62.158.247/tcp/4001] failed to negotiate security protocol: rsa keys must be >= 2048 bits to be useful
2019-10-27T20:03:00.943+0100	WARN	ipfslite	go-log/log.go:187	failed to dial : all dials failed
  * [/ip6/2604:a880:800:10::4a:5001/tcp/4001] dial tcp6 [2604:a880:800:10::4a:5001]:4001: connect: connection refused
  * [/ip4/104.236.76.40/tcp/4001] dial tcp4 104.236.76.40:4001: connect: connection refused
2019-10-27T20:03:01.074+0100	WARN	ipfslite	go-log/log.go:187	failed to dial : all dials failed
  * [/ip6/2604:a880:1:20::203:d001/tcp/4001] failed to negotiate security protocol: rsa keys must be >= 2048 bits to be useful
  * [/ip4/104.236.179.241/tcp/4001] failed to negotiate security protocol: rsa keys must be >= 2048 bits to be useful
2019-10-27T20:03:01.079+0100	WARN	ipfslite	go-log/log.go:187	failed to dial : all dials failed
  * [/ip4/104.131.131.82/tcp/4001] dial tcp4 104.131.131.82:4001: connect: connection refused
2019-10-27T20:03:01.121+0100	WARN	ipfslite	go-log/log.go:187	failed to dial : all dials failed
  * [/ip4/128.199.219.111/tcp/4001] failed to negotiate security protocol: rsa keys must be >= 2048 bits to be useful
  * [/ip6/2400:6180:0:d0::151:6001/tcp/4001] failed to negotiate security protocol: rsa keys must be >= 2048 bits to be useful
2019-10-27T20:03:05.798+0100	WARN	ipfslite	go-log/log.go:187	failed to dial : all dials failed
  * [/ip6/2604:1380:2000:7a00::1/tcp/4001] dial tcp6 [2604:1380:2000:7a00::1]:4001: connect: connection refused
  * [/ip4/147.75.83.83/tcp/4001] dial tcp4 147.75.83.83:4001: connect: connection refused
  * [/ip4/139.178.70.249/tcp/4001] dial tcp4 139.178.70.249:4001: connect: connection refused
  * [/ip6/2604:1380:1000:6000::7/tcp/4001] dial tcp6 [2604:1380:1000:6000::7]:4001: i/o timeout
2019-10-27T20:03:05.798+0100	WARN	ipfslite	go-log/log.go:187	failed to dial : all dials failed
  * [/ip6/2604:1380:1000:6000::1/tcp/4001] dial tcp6 [2604:1380:1000:6000::1]:4001: connect: connection refused
  * [/ip4/139.178.68.71/tcp/4001] dial tcp4 139.178.68.71:4001: connect: connection refused
  * [/ip6/2604:1380:1000:6000::3/tcp/4001] dial tcp6 [2604:1380:1000:6000::3]:4001: i/o timeout
2019-10-27T20:03:05.799+0100	WARN	ipfslite	go-log/log.go:187	failed to dial : all dials failed
  * [/ip6/2604:1380:0:c100::1/tcp/4001] dial tcp6 [2604:1380:0:c100::1]:4001: connect: connection refused
  * [/ip4/147.75.77.187/tcp/4001] dial tcp4 147.75.77.187:4001: connect: connection refused
  * [/ip4/139.178.70.181/tcp/4001] dial tcp4 139.178.70.181:4001: connect: connection refused
  * [/ip6/2604:1380:1000:6000::5/tcp/4001] dial tcp6 [2604:1380:1000:6000::5]:4001: i/o timeout
2019-10-27T20:03:05.800+0100	WARN	ipfslite	go-log/log.go:187	failed to dial : all dials failed
  * [/ip6/2604:1380:3000:1f00::1/tcp/4001] dial tcp6 [2604:1380:3000:1f00::1]:4001: connect: connection refused
  * [/ip4/147.75.94.115/tcp/4001] dial tcp4 147.75.94.115:4001: connect: connection refused
  * [/ip4/139.178.88.21/tcp/4001] dial tcp4 0.0.0.0:4005->139.178.88.21:4001: i/o timeout
  * [/ip6/2604:1380:1000:6000::9/tcp/4001] dial tcp6 [2604:1380:1000:6000::9]:4001: i/o timeout
2019-10-27T20:03:05.800+0100	WARN	ipfslite	go-log/log.go:192	only connected to 0 bootstrap peers out of 9
2019-10-27T20:03:05.800+0100	WARN	dht	go-log/log.go:192	self walk: error: failed to find any peer in table

What is going wrong here? Can the code be changed so that the default bootstrap peer list from https://docs.ipfs.io/guides/examples/bootstrap/ is used?

@hsanjuan
Copy link
Owner

Hi, the problem is libp2p was upgraded to only allow 2048 rsa keys and the default bootstrappers use 1024 ones.

You can either:

I'm going to update the example with the first fix though...

@probonopd
Copy link

Maybe it would be a good idea to extend the example to also write (add) something to IPFS.
But wait, why are we getting a different CID for the "Hello World"?

package main

// This example launches an IPFS-Lite peer, adds a hello-world to
// and fetches a hello-world hash from the IPFS network.

import (
	"context"
	"fmt"
	"io/ioutil"
	"strings"

	ipfslite "github.com/hsanjuan/ipfs-lite"
	"github.com/ipfs/go-cid"
	corecrypto "github.com/libp2p/go-libp2p-core/crypto"
	crypto "github.com/libp2p/go-libp2p-crypto"
	"github.com/multiformats/go-multiaddr"
)

func main() {
	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()

	// Bootstrappers are using 1024 keys. See:
	// https://github.com/ipfs/infra/issues/378
	corecrypto.MinRsaKeyBits = 1024

	ds, err := ipfslite.BadgerDatastore("test")
	if err != nil {
		panic(err)
	}
	priv, _, err := crypto.GenerateKeyPair(crypto.RSA, 2048)
	if err != nil {
		panic(err)
	}

	listen, _ := multiaddr.NewMultiaddr("/ip4/0.0.0.0/tcp/4005")

	h, dht, err := ipfslite.SetupLibp2p(
		ctx,
		priv,
		nil,
		[]multiaddr.Multiaddr{listen},
	)

	if err != nil {
		panic(err)
	}

	lite, err := ipfslite.New(ctx, ds, h, dht, nil)
	if err != nil {
		panic(err)
	}

	lite.Bootstrap(ipfslite.DefaultBootstrapPeers())

	fmt.Println("Adding file")
	n, err := lite.AddFile(ctx, strings.NewReader("Hello World"), nil)
	if err != nil {
		panic(err)
	}

	fmt.Println(n.Cid())
	// Why don't we get "QmWATWQ7fVPP2EFGu71UkfnqhYXDYH566qy47CnJDgvs8u"?

	fmt.Println("")
	cc := "QmWATWQ7fVPP2EFGu71UkfnqhYXDYH566qy47CnJDgvs8u"
	fmt.Println("Getting", cc)

	c, _ := cid.Decode(cc)
	rsc, err := lite.GetFile(ctx, c)
	if err != nil {
		panic(err)
	}
	defer rsc.Close()
	content, err := ioutil.ReadAll(rsc)
	if err != nil {
		panic(err)
	}

	fmt.Println(string(content))
}

@hsanjuan
Copy link
Owner

hsanjuan commented Oct 29, 2019 via email

@probonopd
Copy link

To answer my own question, we don't get a "Qm..." hash because there is apparently a new has format now that does not begin with "Qm...".

Follow-up question:
How can the example be made to actually serve the file? Right now the example just exits, and peers cannot download the added file.

@hsanjuan
Copy link
Owner

@probonopd ah, yes I guess it uses CIDv1 by default, instead of v0.

How can the example be made to actually serve the file? Right now the example just exits, and peers cannot download the added file.

You need to keep the peer (the example) running then.. I think adding select{ } at the end might make it.

@probonopd
Copy link

probonopd commented Oct 29, 2019

Can you please post a working example? This does not seem to work for me:

package main

// This example launches an IPFS-Lite peer, adds a hello-world to
// and fetches a hello-world hash from the IPFS network.

import (
	"context"
	"fmt"
	"io/ioutil"
	"os"

	ipfslite "github.com/hsanjuan/ipfs-lite"
	"github.com/ipfs/go-cid"
	corecrypto "github.com/libp2p/go-libp2p-core/crypto"
	crypto "github.com/libp2p/go-libp2p-crypto"
	"github.com/multiformats/go-multiaddr"
)

func main() {
	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()

	// Bootstrappers are using 1024 keys. See:
	// https://github.com/ipfs/infra/issues/378
	corecrypto.MinRsaKeyBits = 1024

	fmt.Println("Generating datastore")

	ds, err := ipfslite.BadgerDatastore("test")
	if err != nil {
		panic(err)
	}
	priv, _, err := crypto.GenerateKeyPair(crypto.RSA, 2048)
	if err != nil {
		panic(err)
	}

	fmt.Println("Listening")

	listen, _ := multiaddr.NewMultiaddr("/ip4/0.0.0.0/tcp/4005")

	h, dht, err := ipfslite.SetupLibp2p(
		ctx,
		priv,
		nil,
		[]multiaddr.Multiaddr{listen},
		ipfslite.Libp2pOptionsExtra...,
	)

	if err != nil {
		panic(err)
	}

	lite, err := ipfslite.New(ctx, ds, h, dht, nil)
	if err != nil {
		panic(err)
	}

        fmt.Println("Peer address", h.ID())
	fmt.Println("Bootstrapping")

	lite.Bootstrap(ipfslite.DefaultBootstrapPeers())

	fmt.Println("Adding file")

	f, err := os.Open("/etc/os-release")
	if err != nil {
		fmt.Println(err)
	} else {

		n, err := lite.AddFile(ctx, f, nil)
		if err != nil {
			fmt.Println(err)
		}

		f.Close()

		fmt.Println("Added, check it at https://explore.ipld.io/#/explore/" + n.String())

	}
	fmt.Println("")
	cc := "bafybeiduiecxoeiqs3gyc6r7v3lymmhserldnpw62qjnhmqsulqjxjmtzi"
	fmt.Println("Getting", cc)

	c, _ := cid.Decode(cc)
	rsc, err := lite.GetFile(ctx, c)
	if err != nil {
		fmt.Println(err)
	}
	defer rsc.Close()
	content, err := ioutil.ReadAll(rsc)
	if err != nil {
		fmt.Println(err)
	}

	fmt.Println(string(content))

	// Run forever. Is this sufficient to serve the files?

	ch := make(chan struct{})
	<-ch
}

@hsanjuan
Copy link
Owner

@probonopd that seems enough, but mind that your nodes needs to be online for a while to become well connected and should be reachable by others etc.

@probonopd
Copy link

Unfortunately it did not seem to work for me. I also cannot access the file from the browser on localhost.

@hsanjuan
Copy link
Owner

what if you manually connect your local ipfs daemon to /ip4/127.0.0.1/tcp/4005/ipfs/<the generated peer id-you will have to print h.ID() to get it> ?

The example does not run the mDNS discovery service so it can only be reached from dht and that may not work very well for an spontaneous peer (and the ports should be open, maybe even extra addresses should be added to the peerstore etc).

@probonopd
Copy link

Do I need to have a local ipfs daemon running in addition to my Go program? (I want to embed p2p file serving into my Go program, without the need for additional software to be installed on the user's machine.)

@hsanjuan
Copy link
Owner

When you said you cannot access the file from the browser on localhost I assumed you were using the local gateway provided by a running ipfs daemon. You mean you were using the ipfs.io gateways?

You don't need to have the local daemon, but it's a way to test, for example, what the Go program peer is reporting as addresses, whether it is reachable from an external IP etc.

@probonopd
Copy link

probonopd commented Oct 29, 2019

I was trying 127.0.0.1:4005/ipfs/<hash> under the assumption that would work. Nevermind if it doesn't. What I am more concerned with is that the file I tried to share never showed up on the ipfs.io gateway. Is there a way to somehow trigger the ipfs.io gateway to connect to my machine and fetch/cache the file as fast as possible?

And is there a way to see the "health" of my program's connection to the ipfs network?

@hsanjuan
Copy link
Owner

I was trying 127.0.0.1:4005/ipfs/ under the assumption that would work

Ah no, that port is not an http endpoint, it's a libp2p endpoint. What you were trying to access is a gateway, which ipfs-lite does not provide.

Is there a way to somehow trigger the ipfs.io gateway to connect to my machine and fetch/cache the file as fast as possible?

No.. you need to keep your node running, and when the gateway looks up for the hash in the DHT it will eventually find that your node is a provider and try to connect to it on the addresses that the node is reporting.

A good question here is what addresses is the example node reporting and I'm not sure. It may not be reporting any external address. Normally, if you want to use your host properly, you enable a bunch of things like NAT hole punching and autodiscovery.

You can peek through here for an example on how initialization of a host with autonat, circuit etc is done: https://github.com/ipfs/ipfs-cluster/blob/e713969a806f6a60171e00a405060b827ba8c84c/clusterhost.go

I will try to improve the example (or the helper function) to do all this, but I'll need a couple of days for that...

@probonopd
Copy link

Normally, if you want to use your host properly, you enable a bunch of things like NAT hole punching and autodiscovery.

Ah, I was hoping that this would be done automagically by IPFS, like ipfs-desktop seemingly does (because using it I was able to host files successfully in no time).

I will try to improve the example (or the helper function) to do all this, but I'll need a couple of days for that...

That will be much appreciated. Thanks for the great work you are doing 👍

@hsanjuan
Copy link
Owner

Yes, so the difference here is that ipfs-lite is the bare minimum and lets you provide your own preconfigured libp2p host, which you may use for a bunch of other things. I'd say that how the libp2p host is setup is a bit out of the scope of this library, but you're right that it should help a bit more (since it does provide helpers and tries to be useful).

Note that you could also run a full go-ipfs, with ALL the things the daemon running behind ipfs-desktop would provide. In fact, instructions were added recently: https://github.com/ipfs/go-ipfs/tree/master/docs/examples/go-ipfs-as-a-library

@hsanjuan
Copy link
Owner

hsanjuan commented Nov 4, 2019

@probonopd I added a variable Libp2pOptionsExtra. You can pass that to SetupLibp2p (I updated the example) and get a peer with nat hole punching etc. Might improve your issue.

@probonopd
Copy link

Thank you. However it is still not working for me; is there a way to increase verbosity? I can download but seemingly not upload/seed.

@hsanjuan
Copy link
Owner

hsanjuan commented Nov 4, 2019

Thank you. However it is still not working for me; is there a way to increase verbosity? I can download but seemingly not upload/seed.

Can you share your peer address, and the hash of the content you are seeding?

@probonopd
Copy link

How can I know my peer address?

@hsanjuan
Copy link
Owner

hsanjuan commented Nov 5, 2019

The address should be something like /ip4//tcp/4005/p2p/<whatever h.ID() prints>

@probonopd
Copy link

probonopd commented Nov 5, 2019

Updated #1 (comment), this is the output:

Generating datastore
Listening
Peer address QmcAtotNSxpbVuQpQ1sNun17Q8xVWTRQE5SrGVN2htPddT
Bootstrapping
Adding file

@hsanjuan
Copy link
Owner

hsanjuan commented Nov 6, 2019

What is the public ip?

hsanjuan pushed a commit that referenced this issue Aug 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants