Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: initial implementation #1

Merged
merged 61 commits into from
Oct 30, 2024
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
83323ca
feat: initial implementation
aschmahmann Aug 16, 2024
0ddd505
ci: add workflows
aschmahmann Aug 16, 2024
6566eb5
feat: add acme client hook for libp2p
aschmahmann Aug 22, 2024
d1a0b6e
test: add more complete e2e testing
aschmahmann Aug 28, 2024
c7e5892
fix: add HTTP 2 and 1.1 ALPNs
aschmahmann Aug 29, 2024
bc873f6
chore: update go-libp2p
aschmahmann Aug 30, 2024
4b96c4a
chore: switch from net.ParseIP to netip.ParseAddr
aschmahmann Sep 3, 2024
5f36c1a
fix: explicitly close test hosts
aschmahmann Sep 3, 2024
77c9330
fix: rename constructor to NewP2PForgeCertMgr
aschmahmann Sep 3, 2024
999ba3c
feat: add options
aschmahmann Sep 3, 2024
fab0c5b
feat: switch to http peerID auth
aschmahmann Sep 4, 2024
308b83a
chore: tidy up dependency imports
aschmahmann Sep 4, 2024
00b6c7c
docs: add more clarification that base64url does not include padding
aschmahmann Sep 5, 2024
f4b98ac
docs: clarify requirements for libp2p clients requesting certs
aschmahmann Sep 5, 2024
66744f1
feat: allow TLS termination and certificate management for registrati…
aschmahmann Sep 5, 2024
e03e370
Update go-libp2p http peer id auth
MarcoPolo Sep 6, 2024
26046df
feat: change syntax for Corefile, and add ability to declare external…
aschmahmann Sep 6, 2024
f2942d8
docs: add Corefile config docs
aschmahmann Sep 9, 2024
c34b6bb
Split Cert mgr apart from libp2p Host
MarcoPolo Sep 10, 2024
f494051
Don't require fx option
MarcoPolo Sep 10, 2024
c6feb2a
Fix check
MarcoPolo Sep 10, 2024
1e262a8
feat: expose tls.Config explicitly rather than as a WebSocket option
aschmahmann Sep 11, 2024
c55d539
feat: expose an AddrsFactory function explicitly rather than as a lib…
aschmahmann Sep 11, 2024
2d2f576
chore: update go-ds-dynamodb
aschmahmann Sep 11, 2024
e91c9c0
ws option
aschmahmann Sep 11, 2024
0d9c6f8
addrsfactory option
aschmahmann Sep 11, 2024
9779c7c
feat: only expose forge addresses once we have certificates available
aschmahmann Sep 11, 2024
4c9dc3e
fix: return an error if we can't write out the DNS response message
aschmahmann Sep 11, 2024
65b05b8
feat: add Forge-Authorization header as a guard on setting ACME
aschmahmann Sep 11, 2024
02e34f0
chore: remove unused code
aschmahmann Sep 11, 2024
db86cf2
feat: enable use of all default plugins
aschmahmann Sep 12, 2024
2020f9c
feat: client only sends the forge public addresses
aschmahmann Sep 12, 2024
6fff08e
fix: return server fail if failed to write response
aschmahmann Sep 13, 2024
6b008c3
fix: propagate WithAllowPrivateForgeAddrs correctly
aschmahmann Sep 13, 2024
39a5308
fix: apply p2pforge before file plugin
lidel Sep 13, 2024
b06ddb0
refactor: p2p-forge cli and docker support
lidel Sep 14, 2024
c9b535f
chore: reset version.json
lidel Sep 14, 2024
5011943
chore: switch SOA to ns1.p2p-forge.dwebops.net
lidel Sep 16, 2024
d25be74
chore: simplify zone and bump soa ttl
lidel Sep 16, 2024
ae0139e
fix: block ANY queries
lidel Sep 16, 2024
eef1770
tmp(acme-writer): force use of authentication env var
aschmahmann Sep 16, 2024
7fd1cfb
fix(acme-reader): handle subsequent plugins correctly
aschmahmann Sep 16, 2024
a0ee13f
chore: switch soa to ns1.libp2p.direct
lidel Sep 17, 2024
709fad7
chore: separate ip4 from ip6
lidel Sep 17, 2024
f9d8f07
respond with no answer but a successful query on peerID.forge and val…
aschmahmann Sep 18, 2024
d060dac
fix: normalize DNS names to lowercase
aschmahmann Sep 18, 2024
4c9cc8a
respond with no answer but a successful query on _acme-challenge.peer…
aschmahmann Sep 18, 2024
587c7fa
fix: escape arbitrary bytes before writing to log
lidel Sep 18, 2024
dbeb701
fix(client): handle when our certificate is loaded even if no custom …
aschmahmann Sep 18, 2024
2ecd19a
refactor: expose client defaults
lidel Sep 18, 2024
11dda24
feat(client): WithUserAgent + WithForgeAuth
lidel Sep 20, 2024
4db6b55
feat: prometheus metrics + docs
lidel Sep 24, 2024
65145f8
feat(metrics): forge_acme_registrations_total
lidel Oct 11, 2024
d6c1f74
feat: WithLogger(log *zap.SugaredLogger)
lidel Oct 18, 2024
97f25ad
chore: go-libp2p v0.37.0 and go 1.23
lidel Oct 28, 2024
f599f48
fix(client): use dedicated cert cache
lidel Oct 28, 2024
2595bf8
chore: remove requirement for FORGE_ACCESS_TOKEN
lidel Oct 28, 2024
1360e56
chore: typo
lidel Oct 28, 2024
78c3ae0
fix(test): go-libp2p v0.37
lidel Oct 28, 2024
688ddf7
chore(lint): redundant return statement (S1023)
lidel Oct 29, 2024
4468ad5
docs(readme): apply suggestions from code review
lidel Oct 30, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions .github/workflows/go-check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
name: Go Checks

on:
pull_request:
push:
branches: ["main"]
workflow_dispatch:

permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event_name == 'push' && github.sha || github.ref }}
cancel-in-progress: true

jobs:
go-check:
uses: ipdxco/unified-github-workflows/.github/workflows/go-check.yml@v1.0
3 changes: 3 additions & 0 deletions .github/workflows/go-test-config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"skip32bit": true
}
22 changes: 22 additions & 0 deletions .github/workflows/go-test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
name: Go Test

on:
pull_request:
push:
branches: ["main"]
workflow_dispatch:

permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event_name == 'push' && github.sha || github.ref }}
cancel-in-progress: true

jobs:
go-test:
uses: ipdxco/unified-github-workflows/.github/workflows/go-test.yml@v1.0
with:
go-versions: '["this"]'
secrets:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
19 changes: 19 additions & 0 deletions .github/workflows/release-check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
name: Release Checker

on:
pull_request_target:
paths: [ 'version.json' ]
types: [ opened, synchronize, reopened, labeled, unlabeled ]
workflow_dispatch:

permissions:
contents: write
pull-requests: write

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
release-check:
uses: ipdxco/unified-github-workflows/.github/workflows/release-check.yml@v1.0
17 changes: 17 additions & 0 deletions .github/workflows/releaser.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
name: Releaser

on:
push:
paths: [ 'version.json' ]
workflow_dispatch:

permissions:
contents: write

concurrency:
group: ${{ github.workflow }}-${{ github.sha }}
cancel-in-progress: true

jobs:
releaser:
uses: ipdxco/unified-github-workflows/.github/workflows/releaser.yml@v1.0
18 changes: 18 additions & 0 deletions .github/workflows/tagpush.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
name: Tag Push Checker

on:
push:
tags:
- v*

permissions:
contents: read
issues: write

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
releaser:
uses: ipdxco/unified-github-workflows/.github/workflows/tagpush.yml@v1.0
120 changes: 120 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,123 @@
# p2p-forge

> An Authoritative DNS server for distributing DNS subdomains to libp2p peers

## Build

`go build` will build the binary in your local directory

## Install

```console
$ go install github.com/ipshipyard/p2p-forge@latest
```

Will download using go mod, build and install the binary in your global Go binary directory (e.g. `~/go/bin`)

### From source
`go install` will build and install the binary in your global Go binary directory (e.g. `~/go/bin`)

## Usage

### Configuration

This binary is based on [CoreDNS](https://github.com/coredns/coredns) which is itself based on Caddy.
To run the binary create a file `Corefile` following the syntax listed in the CoreDNS documentation.

This binary introduces two additional plugins:
- `ipparser` which handles returning A and AAAA records for domains like `<encoded-ip-address>.<peerID>.libp2p.direct`
- `acme` which handles reading and writing DNS acme challenges for domains like `_acme-challenge.<peerID>.libp2p.direct`

#### ipparser Syntax

~~~
ipparser FORGE_DOMAIN
~~~

**FORGE_DOMAIN** the domain of the forge (e.g. libp2p.direct)

#### acme Syntax

~~~
acme FORGE_DOMAIN {
[registration-domain REGISTRATION_DOMAIN [listen-address=ADDRESS] [external-tls=true|false]
[database-type DB_TYPE [...DB_ARGS]]
}
~~~

- **FORGE_DOMAIN** the domain of the forge (e.g. libp2p.direct)
- **REGISTRATION_DOMAIN** the domain used by clients to send requests for setting ACME challenges (e.g. registration.libp2p.direct)
- **ADDRESS** is the address and port for the internal HTTP server to listen on (e.g. :1234), defaults to `:443`.
- external-tls should be set to true if the TLS termination (and validation of the registration domain name) will happen externally or should be handled locally, defaults to false
- **DB_TYPE** is the type of the backing database used for storing the ACME challenges. Options include:
- dynamo TABLE_NAME (where all credentials are set via AWS' standard environment variables)
- badger DB_PATH

### Example

Below is a basic example of starting a DNS server that handles the IP based domain names as well as ACME challenges.
It does the following:
- Handles IP-based names and ACME challenges for the libp2p.direct forge
- Sets up a standard HTTPS listener for registration.libp2p.direct to handle setting ACME challenges
- Uses dynamo as a backend for ACME challenges

``` corefile
. {
log
ipparser libp2p.direct
acme libp2p.direct {
registration-domain registration.libp2p.direct listen-address=:443 external-tls=false
database-type dynamo mytable
}
}
```

### Handled DNS records

There are 3 types of records handled for a given peer and forge (e.g. `<peerID>.libp2p.direct`):
- ACME Challenges for a given peerID `_acme-challenge.<peerID>.libp2p.direct`
- A records for an IPv4 prefixed subdomain like `1-2-3-4.<peerID>.libp2p.direct`
- AAAA records for an IPv6 prefixed subdomain like `2001-db8--.<peerID>.libp2p.direct`

#### IPv4 subdomain handling

IPv4 handling is fairly straightforward, for a given IPv4 address `1.2.3.4` convert the `.`s into `-`s and the result
will be valid.

#### IPv6 subdomain handling

Due to the length of IPv6 addresses there are a number of different formats for describing IPv6 addresses.

The addresses handled here are:
- For an address `A:B:C:D:1:2:3:4` convert the `:`s into `-`s and the result will be valid.
- Addresses of the form `A::C:D` can be converted either into their expanded form or into a condensed form by replacing
the `:`s with `-`s, like `A--C-D`
- When there is a `:` as the first or last character it must be converted to a 0 to comply with [rfc1123](https://datatracker.ietf.org/doc/html/rfc1123#section-2)
, so `::B:C:D` would become `0--B-C-D` and `1::` would become `1--0`

Other address formats (e.g. the dual IPv6/IPv4 format) are not supported

### Submitting Challenge Records

To claim a domain name like `<peerID>.libp2p.direct` requires:
1. The private key corresponding to the given peerID
2. A publicly reachable libp2p endpoint with
- one of the following libp2p transport configurations:
- QUIC-v1
- TCP or WS or WSS, Yamux, TLS or Noise
- WebTransport
- Other transports are under consideration (e.g. HTTP), if they are of interest please file an issue
- the [Identify protocol](https://github.com/libp2p/specs/tree/master/identify) (`/ipfs/id/1.0.0`)

To set an ACME challenge send an HTTP request to the server (for libp2p.direct this is registration.libp2p.direct)
```shell
curl -X POST "https://registration.libp2p.direct/v1/<peerID>/_acme-challenge" \
lidel marked this conversation as resolved.
Show resolved Hide resolved
-H "Authorization: libp2p-PeerID bearer=\"<base64-encoded-opaque-blob>\""
-H "Content-Type: application/json" \
-d '{
"value": "your_acme_challenge_token",
"addresses": "[your_multiaddrs, comma_separated]"
aschmahmann marked this conversation as resolved.
Show resolved Hide resolved
lidel marked this conversation as resolved.
Show resolved Hide resolved
}'
```

Where the bearer token is derived via the [libp2p HTTP PeerID Auth Specification](https://github.com/libp2p/specs/pull/564).
82 changes: 82 additions & 0 deletions acme/reader.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
package acme

import (
"context"
"strings"
"time"

"github.com/coredns/coredns/plugin"
"github.com/ipfs/go-datastore"
"github.com/libp2p/go-libp2p/core/peer"
"github.com/miekg/dns"
)

type acmeReader struct {
Next plugin.Handler
ForgeDomain string
Datastore datastore.Datastore
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should just define the interface we actually need in this package and not require a full Datastore implementation.

}

const ttl = 1 * time.Hour

// ServeDNS implements the plugin.Handler interface.
func (p acmeReader) ServeDNS(ctx context.Context, w dns.ResponseWriter, r *dns.Msg) (int, error) {
var answers []dns.RR
for _, q := range r.Question {
if q.Qtype != dns.TypeTXT && q.Qtype != dns.TypeANY {
continue
}

subdomain := strings.TrimSuffix(q.Name, "."+p.ForgeDomain+".")
if len(subdomain) == len(q.Name) || len(subdomain) == 0 {
continue
}

domainSegments := strings.Split(subdomain, ".")
if len(domainSegments) != 2 {
continue
}

peerIDStr := domainSegments[1]
peerID, err := peer.Decode(peerIDStr)
if err != nil {
continue
}

const acmeSubdomain = "_acme-challenge"
prefix := domainSegments[0]
if prefix != acmeSubdomain {
continue
}

val, err := p.Datastore.Get(ctx, datastore.NewKey(peerID.String()))
if err != nil {
continue
}

answers = append(answers, &dns.TXT{
Hdr: dns.RR_Header{
Name: dns.Fqdn(q.Name),
Rrtype: dns.TypeTXT,
Class: dns.ClassINET,
Ttl: uint32(ttl.Seconds()),
},
Txt: []string{string(val)},
})
}

if len(answers) > 0 {
var m dns.Msg
m.SetReply(r)
m.Authoritative = true
m.Answer = answers
w.WriteMsg(&m)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check err?

return dns.RcodeSuccess, nil
}

// Call next plugin (if any).
return plugin.NextOrFailure(p.Name(), p.Next, ctx, w, r)
}

// Name implements the Handler interface.
func (p acmeReader) Name() string { return pluginName }
Loading
Loading