-
Notifications
You must be signed in to change notification settings - Fork 8
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #152 from datproject/notes-53
Added notes for meeting 53
- Loading branch information
Showing
1 changed file
with
137 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,137 @@ | ||
| ## Dat comm-comm 2020-05-21 | ||
|
|
||
| ## Attendees | ||
|
|
||
| - Mauve | ||
| - Nina | ||
| - Kevin Faaborg @zootella, from Ara | ||
| - Don Blair from edgecollective.io | ||
| - Kevin Jahns | ||
| - Krzysztof Modras @chrmod (Cliqz) | ||
| - Micah (@micahscopes) | ||
| - YOUR NAME HERE | ||
|
|
||
| ## Intros | ||
| - Mauve | ||
| - working on dat-sdk, who knows what it'll be called in the future? | ||
| - Bruno | ||
| - working on dat-rs... wants to see dat everywhere! | ||
| - Don | ||
| - working with farmers on decentralized resilient ag tech, experimenting with dat / cabal / hyper for p2p farm communications | ||
| - more deets at [edgecollective](https://edgecollective.io/) | ||
| - sorry my baby is sleeping right here so can't use audio haha :) | ||
| - QUESTION I'd like to ask about: how to assess p2p connectivity with dat / hyper? e.g. is there some fancy hyperswarm equivalent of 'dat doctor' to figure out whether one is able to connect to network / other peers? | ||
| - Kevin Faaborg | ||
| - working on Ara https://ara.one, using hyper-stack for distributed payment platform for creators and fans | ||
|
|
||
| - Micah | ||
| - p2p enthusiast | ||
| - Kevin Jahns | ||
| - working on [Yjs](https://github.com/yjs/yjs) CRDT library | ||
| - Krzysztof | ||
| - working on cliqz on the iOS browser | ||
| - excited to hear more about how people are working with hypercore-protocol | ||
| - playing with DAT on React Native https://github.com/cliqz/dat-react-native | ||
| - Martin | ||
| - | ||
| - Mafintosh | ||
| - creates hypercore bugs! | ||
| - Nina | ||
| - interested in how p2p protocols can empower people without centralized structures! | ||
| - working on the @datdotorg p2p hosting platform | ||
| - organizing wizard amigos meetups (@wizardamigos) | ||
| - Santi | ||
| - computer scientist, farmer | ||
| - interested in all the p2p technologies | ||
| - would like to meet Guet! | ||
| - Tim Caswell | ||
| - interested in bringing decentralized tech to a wider market | ||
| - just here to hang out! | ||
|
|
||
| ## Agenda | ||
| ### Dat-rs, with Bruno | ||
| - dat-rs is a group of packages ported from node.js to rust | ||
| - started by Yoshua | ||
| - Bruno has been following the project for a while and decided at the beginning of the year to take a try working on dat-rs! | ||
| - recently working with @frando on hypercore-rs https://github.com/datrs/hypercore/ | ||
| - how can hypercore-rs be designed in a way that fits well with Rust's memory layout paradigm? | ||
| - and also have good ergonomics? | ||
| - Bruno's current approach: working on dat-rs packages alongside a personal project | ||
| - some success has been made with working directly with data from the network | ||
| - shifting to a slightly more structured workflow of aiming to support the formal hypercore-protocol and the newest beaker release | ||
| - Questions: | ||
| - Mathias: have you done any work on the DHT stuff yet? | ||
| - Bruno: currently working on mdns | ||
| - Mathias: working on the DHT could bring hypercore-rs functionality to wider variety of applications, interested in helping out with this! | ||
| - Bruno: hoping to get a kind of DHT backend working to start off, even if it doesnt quite mirror the js hypeswarm implementation at first. | ||
|
|
||
| ## Getting discovery working / debugging connectivity | ||
|
|
||
| - Don: Question I'd like to put out there if there's time at the end of the agenda: what are the best tools / techniques right now to test hyperswarm connectivity? E.g. when I'm setting up a Raspberry Pi on a farm, and want to connect p2p from there, I'd love to diagnose whether the problem is at the local router level, or the remote peer is offline, or etc ... one thought I had (maybe this is already implemented?) is the idea of having an always-online peer (unfortunate point of centralization) that can be a 'test connect' peer? Anyway would love to hear any thoughts if there's time for it ... cheers! | ||
| - Tim: didn't there used to be a dat-doctor tool? | ||
| - Mathias: yes, this was built arouthe old discovery stack. For the new hyperswarm stack, the daemon has some nice feedback for diagnosing issues (and also just works lots better) | ||
| - Kevin: what % of nodes out there tend to be hole-punchable? | ||
| - Mathias: About 90%, and it's definitely working a lot better than it used to | ||
| - Tim, Mafintosh: Beaker has some UI features to help check how connectivity is working, using hyperswarm tooling under the hood | ||
| - Kevin: is "hole punchable" the same thing as "as received unsolicited connection"? | ||
| - Mathias: would call it "opt-in" incoming connections, allows you to essentially turn your computer into a server | ||
| - Mauve: is UPnP relevant? | ||
| - Mathias: We could use legislation against three letter acronyms | ||
| - It wouldn't hurt to add it, but we'd probably only gain about 1/2% more, not the last 5%-10% | ||
| - Tim: in the cases when hole punching breaks, I'm not convinced it helps | ||
| - Mathias, Mauve: STUN, TURN, ICE, etc. | ||
| - Mathias: it'd be cool if people who have extra bandwidth could contribute some by offering to be tunnelers | ||
| - Mauve: would IPv6 help with these issues? | ||
| - Mathias: not sure? Technically, maybe so, but it's hard to say since it hasn't been rolled out yet. | ||
| - Tim: Yggdrasil, cjdns let you interface with ipv6 even if you don't have ipv6 through your ISP | ||
| - Micah: yggdrasil is TCP only but is really cool | ||
| - Mathias: networks are inherently hard to debug! | ||
| - IPFS folks made something called "test cloud" or something like this, not sure where it fits in, but it could be cool to look at | ||
| - Martin: peers with multiple interfaces could have multiple addresses, how to deduplicate? | ||
| - Mathias: this has to do with why getting that last 5% is tricky! | ||
| - Mathias: could be great to get some data on IPv6 connectivity! | ||
|
|
||
| ## Y-JS (with Kevin Jahns) | ||
| - Kevin: | ||
| - Introduction, [demo link](https://y-dat.hashbase.io/#72ed659d11295262bf50efec1ffd30b8c1be8c2a62dadf7e08c2265e8600dd5f)! | ||
| - A CRDT implementation designed to help developers add collaboration to their applications | ||
| - works with a lot of users! (~millions 0_0) | ||
| - vs. automerge? y-js can handle many more sessions than automerge because of optimized garbage collection | ||
| - lots of editors are compatible with and/or using y-js | ||
| - y-js is network agnostic | ||
| - can be used with pure webrtc, websockets | ||
| - would like to support hyper-protocol | ||
| - the way multiwriter works in hyper-protocol aligns well with how it works with y-js | ||
| - envisioning y-js supporting multiple networking stacks, including hyper-protocol | ||
| - demo uses dat-sdk by Mauve! it's great! | ||
| - Martin: | ||
| - Sam has been working on using y-js with hypercore-protocol | ||
| - Kevin: | ||
| - there are tons of integrations that nobody knows about :D | ||
| - yjs can handle a lot of users, but needs a lot of users in memory | ||
| - Krzysztof: Sam was using his y-js/hyper integration to sync bookmarks in cliqz | ||
| - | ||
| - Micah: I think people using kappa-db have had experience integrating CRDTs with hypercore. Just not as much in the browser itself itself. COuld be interesting to checkout what those folks are doing. Substack has experience. I've been investigating delta-crdt, made for IPFS use, peer-crdt. Been interested because they let you have more flexability about data and whether you squash it. Collegue at work side project porting to python. What kind of CRDTs does yjs use | ||
| - Kevin: Kind of like a delta CRDT. It's an operation based CRDT with some properties of delta CRDTs, if you read the protocol you can figure out what can be done. Something unique is optimizations for garbge collecting data and operations. Transforms operations in a way that don't take up space. Why it can support 2k sessions. Important for collaborative documents. Can't use peer-crdt because at some point the browser will slow down | ||
| - Micah: Interested in using bloom clock instead of vector clocks to keep updates. With them you can have a lot of replicas and wouldn't need to know about others to detect which pieces are before and after | ||
| - Kevin: Yeah, it's a big topic. SOunds like something you'd put into an ot based CRDT. There are a lot of implementations, not saying yjs is best for everything, but it's really good at doing shared editing and transforming text. Been working on it for 6 years. Have several companies that use it in production. Can be sure it supports large number of users | ||
| - Mauve: how are you using the hypercore protocol? | ||
| - Kevin: not really using hypercore at all, other than for communication, using y-js to sync the data sent around | ||
| - Mauve: in Beaker, this would probably be similar to using the new peer-sockets API, thinking of implementing it in the dat-sdk | ||
| - Kevin: in my implemetation, there's not a good way to store the data in a standards compliant way | ||
| - Micah: kappa-core is similar to that. It's a tool for storing hypercores and having a reference to that. One hypercore will reference other feeds and they can be merged together | ||
| - Kevin: I think it makes sense, but I'm worried about how you segregate the data, if I don't have _your_ hypercore I can't reconstruct the document. Would like to have it in a single core. | ||
| - Micah: With kappa-core and corestore there's one hypercore that's referencing all the others, you just need to swarm one of them to get all of them. | ||
| - Martin (Guet): props to Kevin for awesome work on y-js, been using it for 10 months and it works great, the examples were really useful for getting it working. Been using y-js + kappacore | ||
| - Kevin: what's the project? | ||
| - Martin: top secret project: wireline | ||
| - Kevin: it could be nice to have this implementation documented to help build community and awareness about y-js | ||
| - Martin: operations are persisted in corestore, using kappa-core to sync and aggregat | ||
|
|
||
| ## Finial thoughts | ||
|
|
||
| - Martin: Check out the announcements: https://github.com/datproject/comm-comm/issues/151 | ||
|
|
||
|
|
||
|
|
||
|
|