2018 Q4 OKR Planning #425
2018 Q4 OKR Planning #425
Conversation
ghost
assigned 
|
In today's call we agreed to conduct the discussion for go-libp2p similar to how js-libp2p is doing it. Each of us posts what we want to focus on for Q4, why it's important, and what the outcome would be. If something needs doing, but you're not claiming it for yourself, state so. Ethereum 2.0 + libp2pProposed KR: Ethereum 2.0 selects libp2p for their networking layer; measured by a formal decision and implementor activity – community participation, language bindings, etc. Ethereum is planning to be a major adopter of libp2p for sharding (Ethereum 2.0). Considerable research has gone into Gossipsub (with successful results), and current focus has shifted to is peer discovery simulations. The EF is developing a PoC on top of libp2p. The libp2p daemon is being relied on, so clients in other languages can create bindings to interact with libp2p (Java, Nim, etc). The daemon should be a priority for Q4 (@vyzo, @bigs). Moreover, the EF is funding a py-libp2p implementation through their Round 4 of developer grants. There's a lot going on, and there's a high confidence in choosing libp2p as their network layer; however, the decision is not formally made yet. Supporting the Ethereum Foundation, and in general the Ethereum ecosystem, as they continue evaluating libp2p should be a priority for Q4. Given my background in devp2p and Ethereum, I'm proposing to act as the point person, i.e. a bridge between both communities, facilitate discussions, review code, requirement gathering (esp. for the daemon), propose solutions, and support them where needed. Eth2.0 Implementers Call Connection tagging and connection managerProposed OKRsO:
KRs:
BackgroundSubsystems like DHT, pubsub, etc. tagging their important connections to prevent pruning was an objective for Q3. However, I only joined halfway through the quarter, so I'd like to take this up in Q4. For the connection manager to be truly useful, in my opinion it needs to be a bit more sophisticated. Some initial discussion here: protocol weights in scoring algorithm; batch vs. on-demand connection pruning. In a broader sense, the purpose of the connection manager is to ensure an intelligent use of limited network-related resources (bandwidth, file descriptors, hard quotas, etc.) Some of the features we've discussed are:
On a whole, the idea is to evolve the connection manager into a traffic shaping component, rather than a simple pruning mechanism. Part of this endeavour would be to write a spec. Exploring alternative data structures for DHTsProposed OKRsO:
KRs:
BackgroundThe analysis: k-bucket underutilisation and excessive thrashing proved that the Kademlia routing table is not an efficient data structure for its purpose. While the bucket thrashing can be minimised by using a different heuristic, we are discarding lots of peers along the way regardless. Theoretically, strategically/selectively storing those peers can give us faster/indexed access to subtrees of the network, allowing us to bypass iterations of the Kademlia lookup (what I call teleporting). I've had follow-up discussions with @bigs and @jhiesey, and I'd like to research, develop, and test alternative data structures. An initial idea is a segmented heap, as posted here: libp2p/go-libp2p-kad-dht#194 (comment). Ideally we'd finish Q4 with a way more efficient data structure that'll result in quicker DHT lookups. |
|
I recommend checking the Go Core Dev Team Retrospective, it is full of important ideas and what should be the next priorities and pain points that should be fixed near/mid term |
|
Here's how JS is presenting their OKRs: https://github.com/libp2p/js-libp2p/blob/c5385b312ab220bc403b80d558985e0bdfdd305c/OKR.md#2018-q4 I like the fact that it's in markdown, like the fact that it's short and clear. Downside is that it's redundant with the spreadsheet (which we must do to conform to PL standards) and I'm always a huge "single source of truth" zealot. @bigs I'll let you figure out if you want to emulate that style. If you do, though, let's keep the spreadsheet as the source of truth and maintain a link to it. |
|
@mgoelzer just fyi, once we've collectively decided on the OKRs, they get moved to the spreadsheet and what ultimately gets merged in git is a reference to the spreasheet, like https://github.com/libp2p/js-libp2p/blob/c5385b312ab220bc403b80d558985e0bdfdd305c/OKR.md#2018-q3. So there will be only 1 source of truth. I'll actually be doing this very thing tomorrow. |
|
@jacobheun Ah, ok, thank you. I had not understood that. |
|
@momack2 did a great summary of the Go Core Dev Team Retrospective here -- ipfs/kubo#5474 (comment) -- and highlighted some candidate KRs that fall withing go-libp2p scope (DHT, IPNS and PubSub). |
|
Thanks, @diasdavid. These are the two points from Molly's summary that jumped at me:
Re: DHT, do we have issues collecting Open Bazaar's pain points? A number of ideas have flourished from the research group, and I don't have the complete picture, but I assume we're implementing/experimenting with a subset of them in Q4 (@jhiesey, @bigs?). Personally, I am owning the exploration of alternative data structures. All in all, I'm sure that OpenBazaar's input is invaluable, and I'd like to ensure we're not only taking it into consideration along the way, but also targeting the pain points. Re: pubsub, do we have issues collecting Textile's feedback or proposal? I can reach out to them via our collab channel, but I want to make sure I have a clear picture before I do that. Cheers! |
|
Adding in @Stebalien who has the background of improvements inside libp2p that would be particularly valuable to improve these user's experiences - especially around making libp2p faster. |
|
Adding some thoughts here. I don't think they're properly constructed as KRs yet, but wanted to get thoughts down. Integration TestingI'd propose we collaborate with the infrastructure team to help lighten the load of this project. It has a few components: libp2p daemonOwner: libp2p OKRs
Ethereum collabrationOwner: raulk, ethereum OKRs
libp2p clientsOwner: libp2p OKRs
Cluster testing infrastructureOwner: infrastructure? OKRs
Stream optimizationI know @Stebalien is owner for this, but wanted to get it on the table as I also think it's quite important. Leaving these vague so we can flesh them out as a group Stream performance doesn't dictate protocol implementations
DHT ResearchI heartily second @raulk's data structure work! In addition: DHT Performance Analysis in the Wild
DHT Interop Testing
|
|
@raulk meta point that I think your "proposed KRs" are actually more on the level of "objectives" which would be supported by measurable factors (feature shipped, x% data saved, etc). Also - I'd encourage you to only make OKRs around things you can control (aka implement X things, accomplish milestone Y) instead of about what some other team might do (so and so adopts us). =] |
|
There are some Filecoin dependencies that we need to reflect in the go-libp2p OKRs. As background, last quarter I tried to address this with KRs like "x % of Filecoin's blockers are fixed." That didn't work well because we (I) weren't talking enough to FC to know what their libp2p blockers are, and I think that they themselves were not at a point yet where they knew. Now we have a clearer picture from @jbenet and @phritz. Filecoin / libp2p DependenciesProposed KR #1: libp2p has a DHT that satisfies 100% of Filecoin's scale tests Proposed KR #2: libp2p has a PubSub messaging system that satisfies 100% of Filecoin's tests Briefly, the reasoning on these two is that I want to let Filecoin's test framework drive our definition of success for these KRs. It's better than us doing a KR like "libp2p scales to N nodes," and then finding out that our definition of "scaling to N nodes" is not consistent with what FC's code is actually doing. Proposed KR #3: libp2p fixes 100% of problems identified by Filecoin's security audit partner I can own this one. I think that FC needs to figure out who they want to hire for audits and what they want the audit to achieve. Our job is just to open issues for everything the auditor requires us to fix, tag them as such, and get them closed by a certain date. (We may need to do all this on a branch, cut an audited release from that branch, and then rebase master. We'll figure it out.) |
|
@momack2 Ethereum's potential adoption of libp2p would grow our community a lot, so I do think it's a worthwhile goal, though no single downstream is more important than any other. I respectfully disagree about KRs being something you can control. Imagine a sales organization where the KRs are things like "x% of deals close within N days." The salespeople cannot per se control how fast a customer signs on the dotted line, but obviously a good salesperson can influence this tremendously and that's what the KR is capturing. |
|
@bigs I agree with the general workflow you're describing to get the daemon written. Some of these need to be rephrased as measurable results - e.g., "a daemon control spec is written and approved by 3 reviewers" etc.
On "Coordinate with ethereum foundation," I would keep this under Raul's KRs for Ethereum adoption. My reasoning: if Ethereum walked away tomorrow, writing a python client would be lower on our priorities list.
|
|
Each organisation does KRs differently :-) I've updated the connection manager and DHT KRs above to be more concrete – I hope this aligns better with expectations. Regarding the Ethereum ones, I see both your points, @momack2 and @mgoelzer. We cannot control what another party will do, but we can put the best effort on our side to support them towards the goal. I am of course committing to:
How would you best turn these into KRs? By referring to "% of questions answered before N days"? That'd be on the verge of ITIL – which I don't think is what we're pursuing. |
|
@mgoelzer i've made a significant update to my post! let me know what you think. |
|
@mgoelzer I'm considering adding Peer routing records to my OKRs for Q4 (both quick fix and long-term fix) – it was my stretch goal for Q3 but my focus shifted to Eth2.0 in the last weeks: O:
KRs:
WDYT? I hope I'm not overreaching. |
|
@raulk Ideally KR's should be higher level than Github issues. (That one is a particularly big issue, so maybe it violates the rule.) Also, @momack2 is right that it needs to be something we can objectively measure. Championing the Python client is important, but how will we measure it at EOQ? Too qualitative for OKRs. I would condense these into 2-3 measurable outcomes: Ethereum 2.0 officially commits to using libp2p through an announcement or contributor activity such as vendoring in libp2p; Ethereum developers get at least 3 PRs merged in libp2p (signals that they are active in our community); etc. Do those make any sense here? |
@raulk For OKR purposes, what if you just own these? (from bigs proposal) I agree with the bulleted steps you describe, but these are the key outcomes we want to achieve. |
|
@bigs Thanks for the KR rewording. This is definitely going in a good direction. We need fewer O's. The entire Go tab should be like 3-5 objectives max. The O's are supposed to be big picture goals, like "libp2p is testable at large scale." Then underneath that we have some measurable results that describe what the world would look like if that goal were to be achieved. So, they'd be things like "libp2p has a daemon spec with such and such properties (has an API, modular, DHT subsystem, etc)" and "The libp2p daemon spec is 80% implemented in Go," etc. |
|
@bigs I generally like with the way JS has done it, though I think O1 should be worded as a statement. I think what you have is a detailed execution plan, which is really important, but not what we want for OKRs. The OKR exercise is about what results we will achieve by the end of the quarter, not the steps we'll take to achieve those results. @momack2 may have a different perspective, but this is how I learned the system. |
|
Totally agree as to "result to achieve" >> "steps to achieve those results" and making KRs as quantitative as possible (to give you an easy rubric to grade them in 3 months). @mgoelzer - My point re taking an OKR like "so and so does X" is you could get a 0.0 on that even for doing all the work/influencing/coordination to make that a highly productive interaction, but it just happens to result in a slightly different outcome because the other party isn't shooting for that specific goal (maybe they do Y instead, which is also valuable but isn't X). When you take a KR like "land Z many deals" you have a lot more control over that goal because if one particular partner drops you can work extra hard to source more deals / improve your offer / close other deals to make up for it - there are more steps you can take when abstracting across a number of clients that aren't available when betting your entire outcome/goal on a single partner (they may say a hard no and then you're stuck). Maybe this is too picky of a point, but there seems to be an important difference between "Ethereum announces integration with libp2p" vs "Ethereum uses libp2p when prototyping sharding" vs "libp2p fulfills 100% of use cases / polish needed by Ethereum" - I think the second two are much more actionable goals to aim for, while the first is mostly outside our control. Maybe I should rephrase my point - we should take on goals where we have significant (but not necessarily direct) control over the outcome and are actionable to guide work even in the face of setbacks and surprises. Ideally, our KR would be somewhat resilient to Eth2.0 not launching, deciding not to make announcements, deferring sharding to a future release, etc - but still holds us accountable for being the BEST sharding solution and doing all the work to be an optimally supportive partner / "get picked". =] Realistically, getting a 0 on an OKR because a key partner did something extremely unexpected despite the team doing 1.0-level work is a valid thing to just call out and move on in OKR grading, so this isn't something to stress over too much unless it becomes a pattern (which would be it's own problem). =] |
|
@momack2 Everything you're saying makes sense to me. I particularly liked this phrasing:
because it focuses on building a better product (libp2p) that will help us with all downstream adopters. (Baked in assumption here is that Ethereum's needs are the same as other potential blockchain adopters. At least to a first approximation, I think this is probably true.) |
|
Thanks for the discussion. Closing this PR since it contains placeholder content. |
Ref: ipfs/team-mgmt#698
Urls:
It's time to do the OKR Planning for Q4 \o/. This is the first time that the go-ipfs team is going to do it this way, you can find a lot of information at ipfs/team-mgmt#698. Please make sure to read it to get the full context in how we are going to do this (Retrospective + Open OKR Planning).