Is what makes this a "P2P" VPN--as opposed to merely VPN software that you might choose to use between peers (such as Wireguard)--that it supports the group/"community" network behavior? When you do that, are you building a fully-connected overlay network--where everyone has a separate WebRTC link to each of the other participants--or are you building some kind of organized P2P network topology out of the participants?

(Note: I work on Orchid, a decentralized market for bandwidth built on Ethereum and WebRTC, which has a multi-hop-capable VPN client for various platforms available.)

> Is what makes this a "P2P" VPN [...] that it supports the group/"community" network behavior?

More or less! Compared to say OpenVPN there is no central server that all traffic is routed through (unless you choose to activate TURN of course, see the `--force-relay` flag), so hosting it is much cheaper as the only public part of the infrastructure is the part that exchanges the candidates ^^ Its pretty close to how n2n[1] does it, but has proper NAT hole punching support due to the WebRTC backend.

> When you do that, are you building a full-connected overlay network [...]

Yes! Its a full mesh; it seems to scale pretty well to a fairly large number of nodes, although I haven't tested it with more than ~100 peers in a single community yet. If the overhead of maintaining all of the connections is too much or peering between nodes in an issue, TURN can help to connect those peers with the rest of the mesh. Orchid looks interesting!

[1] https://github.com/ntop/n2n

Does this also work between two peers that are behind NAT's / routers?

From what I vaguely remember when I tried playing around with this stuff, some NAT's / routers have some sort of port randomization thing so that the same ports don't always reach the desired location behind it.

Does WebRTC make that port connection persistent due to the streaming nature of it; is that the idea? Does the stream ever break and does it require getting ip and port information from the STUN and ICE servers again? If so, how often does that happen?

From my experience it works basically everywhere, but if it doesn't - just specify a TURN server on `--ice` and you'll be able to connect to a relay ^^ The ports/IP can actually change - I was traveling through Germany by train recently for example and whenever my public IP changed it signaled the updated candidates using trickle ICE :)

The NAT hole punching they mentioned is a workaround for exactly that scenario.

While I'm not involved in the project whatsoever, I'd expect that to work in most scenarios considering they've explicitly mentioned it. It's not a guarantee however and can get circumvented if the network owner wants to

https://en.wikipedia.org/wiki/Hole_punching_(networking)#:~:....

Wait, is the signaling server "public infrastructure"? Are you then having users pre-set/share all their DTLS keys and ICE passwords (to prevent the obvious MITM)? I don't remember seeing that in your documentation (but I am not reading it extremely carefully or anything). (This isn't an issue for Orchid as the endpoints are expected to expose signaling ports for WHIP.)

Oh the signaling server is but a simple ICE candidate/offer/answer relay :) The messages being sent there are encrypted using a PSK so that the signaling server can't access the ICE candidates etc.

This looks pretty slick, and the WebRTC seems like it might offer some choices of traversal that would work where others don't. The inbuilt tools for "chat" (seems a useful test), throughput and latency also seem nice. I've been running into some weirdnesses at times with Nebula and Zero Tier where it's unclear where my latency or throughput limits are coming from.

I'm looking forward to doing a lot more with overlay networks in the short term, I've been playing with a variety of them over the last 18 months or so and just need to pick something and start deploying it.

Thank you! From my experience if you can get say Zoom or Meet to work in the network you're trying to deploy weron in, you'll get weron working as well. And if the traversal still doesn't work - just specify a TURN server on the `--ice` flag and it will definitely work through relay ^^

Why wouldn't you just tunnel WireGuard over WebRTC? There's already a very, very good implementation of WireGuard --- one of the two major production implementations --- in Go. We tunnel it over WebSockets, and it took like an hour to write that code.

You could totally do that! See https://github.com/pojntfx/weron#8-write-your-own-protocol-w..., but I agree with saurik - if DTLS is secure enough for your usecase, that would just add overhead ...

Yep, that's a good point. I jumped into the code (I just wanted to see what the encryption transform was) and should have zoomed out and looked more carefully at the whole design. Thanks!

That seems like it would offer no advantages, but would result in all of worse performance, smaller remaining available packet size, and "more code".

that is indeed a performance decrease. But the idea of this project is just great!

There is another way of combining WebRTC and Wireguard.

Netbird uses ICE protocol (used in WebRTC) for NAT traversal and then reuses that punched holes to establish a direct Wireguard tunnel.

https://github.com/netbirdio/netbird

P.S. I'm author of Netbird (formerly know as Wiretrustee)

So that's what, 4 layers of encapsulation now? Yikes.

Coders gonna code.

No one is raising the question of how to handle community name/key management for persistent communities. With n2n, which long predates Zerotier, Wireguard and Tailscale, one of the authors thought there needed to be automatic key rotation and also communication between "signal" servers. Original n2n allowed tunneling over HTTP as a fallback. It should be noted that n2n was never meant to be used for large communities, say, over 100 peers. Sure enough, the author of weron is issuing that same warning here.

Many years ago I had each peer run their own smtpd listening on the n2n TAP interface. It was peer-to-peer, encrypted email. This can be done for many other protocols, too. With Layer 2 overlay, there are no middlemen. Direct connectivity to friends and family with no need for Google/Facebook.

Layer 2 overlay networks with n2n can be significantly faster if automatic encryption is turned off. In that case, the user can encrypt and sign sensitive files with a separate program before sending them through the TAP interface. It works quite well.

Hi! n2n was a huge inspiration for the project, in fact I wrote Go bindings for n2n before starting it: https://github.com/pojntfx/gon2n

Weron also allows tunneling - just specify a TURN server on `--ice` and enable `--force-relay`, in which case it will probably scale to well over 100 nodes ^^ The signaling servers are fully horizontally scalable, so that you can benefit from a faster backbone on a global scale scenario - Redis coordinates messages, kicks etc. between them and a Postgres database maintains central state, such as credentials for persistent communities and client counts.

In terms of key rotation & encryption - weron heavily depends on DTLS as provided by Pion/WebRTC and thus inherits similar security properties. It is not possible to disable encryption of WebRTC, but tbf the performance benefits of using plain SCTP don't seem to be worth it as the RTT latency and resulting decrease in throughput[1] is the dominant performance bottleneck.

[1] https://tuhat.helsinki.fi/ws/portalfiles/portal/167373638/Es...

Hey, I have been using Wireguard and Nebula to connect to my home network for a while now, but theyre blocked by my uni. I bypass the blocking using shadowsocks but its still a hassle.

I'll test out Weron and hopefully I dont need to tunnel it through shadowsocks.

A side note, but a university network blocking VPNs, that's ridiculous! Uni IT should really be hit over the head with a clue bat. I mean what is the motivation for this?

That is basically the usecase for weron, good luck ^^ Networks blocking anything but HTTP has been one of the reasons why I started writing this (train WLAN in Spain does it for example)

This is so cool! I love the embedded aspect of it, that was a big goal of Pion. Really exciting for users to not have to setup/configure a service to get connectivity going.

re: concerns about STUN I would love to experiment with adding NAT-PMP/PCP support to pion/ice. STUN server usage seems to be a major sticking point for people. Talked about it a bit here [0]. I am all for doing it, just would like to know the code is actually wanted before writing it :)

[0] https://mobile.twitter.com/_pion/status/1522365018855165952

Thanks for your work on Pion! I really enjoyed working with it, even in other places (STFS[1], a file system for tape drives that I wrote in January, uses it to access remote servers behind NAT for example). NAT-PMP/PCP support would be awesome!

[1] https://github.com/pojntfx/stfs

Wow, some awesome work here. I would be interested to know your thoughts on OpenZiti (https://openziti.github.io/). Its an open source project that allows you to embed private connectivity into your app using one of the many SDKs together with strong identity allowing outbound only connections. It is a mesh overlay similar to TURN but uses a concept of smart routing to normally reduce latency through circumvention of BGP. Various superpowers can be seen here - https://www.youtube.com/playlist?list=PLMUj_5fklasKF1oisSSuL...

very cool! was recently musing about how the next frontier in vpns is steganography, and here we are.

are the webrtc streams running at a constant bitrate or can the streams be detected by doing traffic timing analysis?

(i once looked at adding constant bitrate support to wireguard but ultimately abandoned it as it was looking to be excessively complicated)

Thanks! Its using the Pion library[1] which doesn't seem to be using a constant bitrate; that would definitely be an interesting addition though. Sending e.g. zeroes when its not transmitting packets/frames is totally doable, but atm not implemented.

[1] https://github.com/pion/webrtc

the other big upshot would be that it would never leak anything in the flow characteristics of the stream.

i suspect that it might be possible to identify major websites being loaded by simply analyzing packet sizes and timings, but maybe not in http/2 world.

This is the kind of thing I would expect to see soon from Orchid, if that's interesting, which isn't "P2P" in this sense of group-networking but that doesn't seem to be your specific interest. (I am in charge of technology at Orchid.)

The constant bitrate idea is really interesting, I think we could do it! I wonder if we can publish a constant rate at the public API of if it would take some interaction with the Congestion Controller in SCTP.

i agree! it's been percolating for a very long time in my head. i was envisioning a paper that shows how traffic analysis can be used to identify activities in the encrypted stream and then a cbr patch to ameliorate it.

where it gets complicated is avoiding creation of new side channels. you need a very stable clock for pumping out the data and as you hint at, congestion control to avoid latency by actively adjusting the bitrate.

i was looking at doing this in the kernel implementation of wireguard, but those parts of the kernel didn't seem a good fit for this kind of time sensitive constant spinning algorithm.

if anyone is interested in this, would love to collaborate.

Here is some initial feedback: Nice selection of static binaries!

However the choice to hardcode Google as a default STUN server is off-putting. IMHO, 2008's n2n would never set a third party server as a default. https://luca.ntop.org/n2n.pdf

Thanks for your feedback! Is there a publicly hosted STUN server that guarantees better privacy? I'd love to use it if there is an alternative. It is of course possible to use any other STUN/TURN server by changing the `--ice` flag and host your own with i.e. `coturn` ^^

Two users don't need to use the same STUN server in order for udp hole punching to worj, so choosing randomly from a list might be a good way to go.

There is work for using libp2p instead of a fixed third-party server:

https://pkg.go.dev/github.com/libp2p/go-libp2p-webrtc-direct...

One of the beautiful things about n2n was that the supernode program is relatively small and simple. (n2n does not require the use of STUN or TURN.) There is no need for a third party to be the one running the supernode/STUN/TURN server software. Users can run it themselves.

Just because home users must involve commercial third parties, e.g., ISP, cloud hosting, etc., in order to listen on globally reachable IP addresses does not also mean that users must also let those third parties have control over the software users run from those addresses.

n2n is neat! It is super simple to self-host the signaler in STUN/TURN in the case of weron as well: https://github.com/pojntfx/weron#1-start-a-signaling-server-... and `coturn` is even vendored through the Debian repos nowadays.

There is a very neat implementation of a TURN from Pion

https://github.com/pion/turn

Can this be used as a library like libp2p, or does it require me to run binaries?

You can use every part of it as a library! See https://github.com/pojntfx/weron#8-write-your-own-protocol-w... and https://pkg.go.dev/github.com/pojntfx/weron ^^

That's very cool! Thanks for answering my question so fast!

This looks nice. Do you have Android/iOS mobile apps on the roadmap?

Thanks ^^ Android/iOS is not _yet_ on the roadmap (although weron works on the latter if your phone is rooted, since TUN/TAP is available natively) but totally possible! Both systems provide a L3 VPN API, and since Gomobile can generate bindings to the networking layer for Java/Swift that is definitely something I want to work on in the future ^^

Well done!

Disclaimer: I'm author of netbirdio - a similar project.

Using WebRTC makes it possible to have a tunnel from the browser directly. Combined with a virtual terminal it just makes it even cooler!

@pojntfx what do you think about that? Feasible with your implementation?

I've been playing with such functionality in Netbird but since it depends on Wireguard it is a bit tricky :)

Would love to have your feedback as well on our project

https://github.com/netbirdio/netbird

Having a docker image that sets up a VPN server with ideally zero config, plus short instructions for setting up the clients in popular OSes would go a long way to help adoption.

It is more or less zero config! Since its P2P there is no VPN server per se, and connecting to a overlay network is a platform-independent one-liner: https://github.com/pojntfx/weron#usage. The signaling server is also distributed as a Docker image if you want to host it yourself: https://github.com/pojntfx/weron#1-start-a-signaling-server-...

I somehow can't understand how to use it :) Sorry! But nice work.

I'm sorry to hear that! Do you have some recommendations on how to change the docs? I've added a small TL;DR here: https://github.com/pojntfx/weron#usage

TLDR is very good idea, yes! Thanks!

> Compared to for example Tailscale, WireGuard and ZeroTier, its much harder to block on a network level

Tailscale at least bundles in an HTTPS fallback for those situations. I’m not sure that claim holds up.

A HTTPS fallback is more or less what weron provides - if you want to block the "weron protocol", you either have to block access to the signaling service (which runs on 443, and its just WebSockets) or block the related protocols, but since Zoom etc. use the same protocols that would probably be pretty hard to implement without disrupting everyday operations.