Gossip Observer: New project to monitor the Lightning P2P network

[jonhbit]

@rustyrussell

@jonhbit

IMO we can adjust the size of these fields based on current constraints like min. tx size, etc. such that the amount of overlap is minimal, but we wouldn’t need this fallback path.

But now you need to detect clashes and decide who wins. Easier to up-front fail to decode, in practice.

I agree that we’ll need some logic to handle outpoints that we can’t encode. Given that the top block of a 24-bit blocknum will flip ~142 years from now, IMO it’s fine to XOR that with the bottom bit of txindex.

@rustyrussell

How about:

1. Send a sketch every 60 seconds. You may reduce the set before sending.
2. If you receive a sketch you can decode, simply reply with any messages the sender is missing.
3. If you can’t decode:
   1. If they are on a different blockheight, ignore.
   2. If you can send a larger set, do so next time.
   3. If you have other peers, wait until you get more failures.
   4. If they have much less gossip than you, start streaming gossip.
   5. Otherwise, something is wrong, query them for everything (or if you don’t have a channel with them, ignore?).

I don’t really think a query protocol is needed.

I got to talk to someone from the LDK team last week, and they presented some convincing arguments of why a query protocol doesn’t really add value. But the consequences of the ordering you’re describing seem a bit ‘flipped’ to me, specifically point 3.2.

Since sketch decode is most of the work, in order for Alice to increase her odds of a successful sketch decode for the next received sketch, she will send her peer Bob a larger sketch, and he’ll perform (much) more work, and possibly end up sending Alice more messages she’s missing? The extra work is done by Bob, for Alice’s benefit.

A simplified version of my diagram above is that Alice never explicitly requests initial sketches (so your step 1). But she MAY request an extension, and attempt decode again. The decode work is still done by Alice, not Bob. However, I think that would require new P2P messages so Alice could query Bob for missing set elements.

Another point that came out of discussions last week was that, instead of tracking the ‘freshness’ of sketches per peer (I make a new sketch for each peer right before sending a sketch to them), a node can have one ‘internal’ timer of when they will recompute their sketch, and separate peer-per timers of when they will send whichever sketch they have. So there would be no tracking of which elements were included in the last sketch sent to a particular peer, and the cost of serving a sketch extension is just bandwidth vs. more compute.

@rustyrussell

That API would be great, but meanwhile I think you just trim untrusted sources to use a small sketch, which bounds the time.

That would definitely work. FWIW, I think the odds of getting that feature added to libminisketch are high, if the use case can be clearly demonstrated.

@rustyrussell

If a failure to decode simply (eventually) results in you streaming all gossip, it’s no worse than the current case where a peer explicitly asks you for all gossip?

I’m not sure how often the ‘stream all gossip’ fallback is used now, so…maybe? If a node is offline for long enough, I definitely agree that reconciliation won’t work for catching up on gossip. But that feels separate from handling a decode failure during normal operation. It’s interesting that decode failure/success gives some signal on how much information a node is missing, compared to the situation now / with flooding.