{"post_stream":{"posts":[{"id":1046,"name":"Suhas Daftuar","username":"sdaftuar","avatar_template":"/letter_avatar_proxy/v4/letter/s/0ea827/{size}.png","created_at":"2024-01-09T17:45:37.198Z","cooked":"\u003cp\u003eLast spring, \u003ca class=\"mention\" href=\"/u/sipa\"\u003e@sipa\u003c/a\u003e and I first floated a concept for a new mempool design to a group of Bitcoin Core contributors, which I later wrote up as a github issue (\u003ca href=\"https://github.com/bitcoin/bitcoin/issues/27677\" class=\"inline-onebox\"\u003eProposal for a new mempool design · Issue #27677 · bitcoin/bitcoin · GitHub\u003c/a\u003e).  Over the course of the past year, the ideas have been refined, and I will use this post to provide an updated high level summary of the overall proposal – including the motivations and implications – for anyone looking to catch up on this topic.\u003c/p\u003e\n\u003chr\u003e\n\u003ch1\u003e\u003ca name=\"p-1046-proposal-overview-in-brief-1\" class=\"anchor\" href=\"#p-1046-proposal-overview-in-brief-1\" aria-label=\"Heading link\"\u003e\u003c/a\u003eProposal overview in brief\u003c/h1\u003e\n\u003cp\u003eHere is a summary of the mempool policy changes in the draft implementation (\u003ca href=\"https://github.com/bitcoin/bitcoin/pull/28676\"\u003ehttps://github.com/bitcoin/bitcoin/pull/28676\u003c/a\u003e) – note that all these terms and ideas are explained further below:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\n\u003cp\u003eEnforce two new limits governing the size of \u003cem\u003eclusters\u003c/em\u003e in the mempool, so that both the number of transactions in a cluster and the number of vbytes the transactions in the cluster take up are bounded.\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eDrop the existing ancestor and descendant limits.\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eUpdate the replace-by-fee rules, so that replacement transactions are accepted to the mempool if and only if they would result in a “strictly better feerate diagram” (explained below).\u003c/p\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eNote that adding a new restriction on the transaction graph immediately makes the existing “CPFP carveout” policy moot, so that is also removed.  See discussion below.\u003c/p\u003e\n\u003cp\u003eAlso, by bounding cluster sizes, we are able to achieve more optimal orderings of transactions in the mempool for mining (explained below), and so there are two additional behaviors that would be modified by this proposal:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\n\u003cp\u003eUpdate the mining code to take advantage of the more optimal sorting that can be achieved with bounded cluster sizes, and–\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eUpdate mempool eviction to always evict a set of transactions that would be the last ones to be selected by the mining algorithm.\u003c/p\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003chr\u003e\n\u003ch1\u003e\u003ca name=\"p-1046-background-and-motivation-2\" class=\"anchor\" href=\"#p-1046-background-and-motivation-2\" aria-label=\"Heading link\"\u003e\u003c/a\u003eBackground and motivation\u003c/h1\u003e\n\u003cp\u003eThe current mempool is designed around maintaining two different sort orders, where for each transaction we keep track of its position in an ancestor-feerate-sort and in a descendant-feerate-sort.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eNote\u003c/strong\u003e: By “ancestor feerate” of a transaction, we refer to the sum of fees divided by sum of vbytes of a transaction along with its unconfirmed ancestors, which represents the minimal set of transactions that would need to be included for the transaction to be valid in a block.  “Descendant feerate” is the analogous concept for a transaction with its in-mempool descendants, which represents the minimal set of transactions that would need to be evicted from the mempool in order for the target transaction to be evicted, in order to avoid orphan transactions from persisting in the mempool.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eWe use these two sort orders in order to implement eviction and transaction selection (for mining). Whenever the mempool exceeds its size limit, we evict the transaction that has the lowest descendant-feerate, along with all of the transaction’s descendants.  We repeat until the mempool is below its size limit.\u003c/p\u003e\n\u003cp\u003eWhen we select transactions during block template construction, we choose the transaction with the highest ancestor feerate, and select that transaction with all its ancestors to appear in the block. The ancestor scores of the remaining in-mempool transactions are updated to reflect these selections, and then we repeat by choosing the transaction with next highest-ancestor feerate, and so on until the block is full.\u003c/p\u003e\n\u003cp\u003eThose algorithms are deficient in several ways. In particular, the eviction algorithm \u003cstrong\u003edoes not guarantee that the first transaction evicted would be the last transaction mined\u003c/strong\u003e, and there exist situations where we could evict a transaction that would otherwise be mined in the next block (see below).\u003c/p\u003e\n\u003cp\u003eFurthermore, the mining algorithm is more complex than the ancestor feerate sort we maintain, and it is insufficient to merely do lookups into our ancestor feerate index to determine which of two transactions will be mined from the mempool first. This lack of a total ordering on mempool transactions makes RBF calculations difficult to align with miner incentives, and creates distortions where we sometimes can replace transactions via RBF that would actually be better for a miner than the replacement.\u003c/p\u003e\n\u003cp\u003eTo solve all these problems, we can instead maintain a total ordering on mempool transactions, which is used to implement a mining and eviction algorithm that are symmetrically opposite and provide better RBF rules for determining when to allow a replacement to occur.\u003c/p\u003e\n\u003ch2\u003e\u003ca name=\"p-1046-example-eviction-and-mining-are-not-opposite-3\" class=\"anchor\" href=\"#p-1046-example-eviction-and-mining-are-not-opposite-3\" aria-label=\"Heading link\"\u003e\u003c/a\u003eExample: Eviction and mining are not opposite\u003c/h2\u003e\n\u003cp\u003eConsider the following transaction graph:\u003c/p\u003e\n\u003cpre data-code-height=\"233\" data-code-wrap=\"mermaid\"\u003e\u003ccode class=\"lang-mermaid\"\u003egraph TD;\n    A[\"A: 3 sat/vB\"]--\u0026gt;B;\n    A--\u0026gt;C;\n    A--\u0026gt;E[\"E: 9 sat/vB\"];\n    B[\"B:1 sat/vB \"]--\u0026gt;D[\"D 11 sat/vB\"];\n    C[\"C: 1 sat/vB\"]--\u0026gt;D;\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eThe transaction with lowest descendant feerate is tx A, at 5 sat/vB.  However, the transaction with highest ancestor feerate is tx E, at 6 sat/vB.  This means that our mining algorithm would select transaction E (along with A) to appear in the next block, but our eviction algorithm would select tx A and evict this whole transaction graph if we were above our size limit.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eNote:\u003c/strong\u003e A more complex example that exaggerates the difference between these feerates can be found \u003ca href=\"https://github.com/bitcoin/bitcoin/issues/27677#eviction-mining-not-opposite\"\u003ehere\u003c/a\u003e.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003ch2\u003e\u003ca name=\"p-1046-example-our-current-rbf-rules-are-not-always-incentive-compatible-4\" class=\"anchor\" href=\"#p-1046-example-our-current-rbf-rules-are-not-always-incentive-compatible-4\" aria-label=\"Heading link\"\u003e\u003c/a\u003eExample: Our current RBF rules are not always incentive compatible\u003c/h2\u003e\n\u003cp\u003eWhen a transaction is received that conflicts with some existing in-mempool transaction(s), Bitcoin Core uses policy rules that are (roughly) described in \u003ca href=\"https://github.com/bitcoin/bips/blob/deae64bfd31f6938253c05392aa355bf6d7e7605/bip-0125.mediawiki\"\u003eBIP 125\u003c/a\u003e in order to determine if the new transaction should be accepted (which requires evicting the conflicts from the mempool). However, there are many forms of incentive incompatibility that are part of those rules:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eThe “no-new-parents” rule doesn’t consider the replacement transaction’s desirability to a miner – it might be better than all the to-be-replaced transactions, despite the new unconfirmed parent.\u003c/li\u003e\n\u003cli\u003eThe rule requiring that a new transaction pays a higher feerate than its direct conflicts is insufficient to ensure that the new transaction is more desirable than what is being evicted.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eTo illustrate some specific incentive compatibility issues, consider this example, where A+B are already in the mempool, and transaction A’ arrives that conflicts with A:\u003c/p\u003e\n\u003cpre data-code-height=\"146\" data-code-wrap=\"mermaid\"\u003e\u003ccode class=\"lang-mermaid\"\u003egraph TD;\n    A[\"A: 1 sat/vB, size 100\"]--\u0026gt;B;\n    B[\"B:99 sat/vB, size 100 \"]\n    A'[\"A': 3 sat/vB, size 10000\"]\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eHere, transaction B has an ancestor feerate of 50 sat/vB, while tx A’ is only paying 3 sat/vB.  Yet under BIP 125 rules we would evaluate A’ as a successful replacement of [A, B], because it pays a higher feerate than A (3 sat/vB vs 1 sat/vB) and pays a higher total fee than A+B (30,000 sats vs. 10,000 sats).\u003c/p\u003e\n\u003cp\u003eFailure to accept incentive compatible replacements can also occur; consider this example, where A and B are already in the mempool, and B’ is a potential replacement of B:\u003c/p\u003e\n\u003cpre data-code-height=\"146\" data-code-wrap=\"mermaid\"\u003e\u003ccode class=\"lang-mermaid\"\u003egraph TD;\n    A[\"A: 1 sat/vB, size 10000\"]--\u0026gt;B;\n    A-.-\u0026gt;B'\n    B[\"B:100 sat/vB, size 100 \"]\n\n    B'[\"B': 49 sat/vB, size 10000\"]\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eB’ cannot replace B in the mempool under the BIP 125 rules, because it pays a lower feerate than B; however B’ has an ancestor feerate of 25 sat/vB, while B has an ancestor feerate of just under 2 sat/vB(!).\u003c/p\u003e\n\u003cp\u003eMany other examples exist as well.\u003c/p\u003e\n\u003chr\u003e\n\u003ch1\u003e\u003ca name=\"p-1046-designing-a-solution-5\" class=\"anchor\" href=\"#p-1046-designing-a-solution-5\" aria-label=\"Heading link\"\u003e\u003c/a\u003eDesigning a solution\u003c/h1\u003e\n\u003ch2\u003e\u003ca name=\"p-1046-maintaining-a-total-ordering-on-the-mempool-6\" class=\"anchor\" href=\"#p-1046-maintaining-a-total-ordering-on-the-mempool-6\" aria-label=\"Heading link\"\u003e\u003c/a\u003eMaintaining a total ordering on the mempool\u003c/h2\u003e\n\u003cp\u003eTo fix the problem with eviction and mining, we start with the realization that those should be opposite operations on the mempool – when we need to evict, we should always evict the transactions that are least desirable to be mined.  The difficulty in achieving this with today’s mempool design is that we can’t easily calculate the last transaction that would be mined – the mining algorithm runs in time \u003cspan class=\"math\"\u003eO(dn \\log n)\u003c/span\u003e, where \u003cspan class=\"math\"\u003en\u003c/span\u003e is the number of transactions in the mempool, and \u003cspan class=\"math\"\u003ed\u003c/span\u003e is the descendant limit, so it is not currently feasible to run the algorithm on the entire mempool in order to calculate the transactions to evict first.\u003c/p\u003e\n\u003cp\u003eHowever, that is exactly what we’d like to do – so a natural question to ask is, what bound could we add to the mempool in order to make it feasible to have a total ordering?  One way to see the answer to this question is to consider how many mempool transactions’ mining scores might be changed when a single new transaction is added – if we always maintained a full sort on the mempool, then this gives an idea of how much work might need to be done to update the mempool as transactions are added.\u003c/p\u003e\n\u003cp\u003eThe answer to this is that every transaction that is connected to a new transaction, via either parent or child relationships, might have its mining score changed. Consider this example (credit to \u003ca class=\"mention\" href=\"/u/sipa\"\u003e@sipa\u003c/a\u003e):\u003c/p\u003e\n\u003cpre data-code-height=\"108\" data-code-wrap=\"mermaid\"\u003e\u003ccode class=\"lang-mermaid\"\u003egraph TD;\n    A[\"A: 72 sat/vB\"]--\u0026gt;B;\n    A--\u0026gt;D[\"D: 78 sat/vB\"]\n    B[\"B:74 sat/vB\"]\n    C[\"C: 66 sat/vB\"]--\u0026gt;D\n    C--\u0026gt;F[\"F: 86 sat/vB\"]\n    E[\"E: 56 sat/vB\"]--\u0026gt;F\n    E--\u0026gt;H[\"H: 98 sat/vB\"]\n    G[\"G: 42sat/vB\"]--\u0026gt;H\n    G--\u0026gt;J[\"J: 114 sat/vB\"]\n    I[\"I: 24 sat/vB\"]--\u0026gt;J\n    K[\"K: 2 sat/vB\"]--\u0026gt;L[\"L: 134 sat/vB\"]\n    I--\u0026gt;L\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eWith some calculation, we can see that tx B has the highest ancestor feerate at 73 sat/vB.  Once it and tx A are selected for inclusion in a block, tx D would have the next highest ancestor feerate at 72 sat/vB, followed by tx D, tx F, tx H, tx J, and tx L.\u003c/p\u003e\n\u003cp\u003eIf a new transaction M were to arrive as follows:\u003c/p\u003e\n\u003cpre data-code-height=\"84\" data-code-wrap=\"mermaid\"\u003e\u003ccode class=\"lang-mermaid\"\u003egraph TD;\n    A[\"A: 72 sat/vB\"]--\u0026gt;B;\n    A--\u0026gt;D[\"D: 78 sat/vB\"]\n    B[\"B:74 sat/vB\"]\n    C[\"C: 66 sat/vB\"]--\u0026gt;D\n    C--\u0026gt;F[\"F: 86 sat/vB\"]\n    E[\"E: 56 sat/vB\"]--\u0026gt;F\n    E--\u0026gt;H[\"H: 98 sat/vB\"]\n    G[\"G: 42sat/vB\"]--\u0026gt;H\n    G--\u0026gt;J[\"J: 114 sat/vB\"]\n    I[\"I: 24 sat/vB\"]--\u0026gt;J\n    K[\"K: 2 sat/vB\"]--\u0026gt;L[\"L: 134 sat/vB\"]\n    I--\u0026gt;L\n   K--\u0026gt;M[\"M: 158 sat/vB\"]\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eThen the ancestor feerates get reversed(!).  The highest ancestor feerate is now tx M (80 sat/vB), followed by tx L (at 79 sat/vB, once tx K has been selected), and then J, H, F, D, and B – exactly in the reverse order as before transaction L arrived.\u003c/p\u003e\n\u003cp\u003eThis illustrates that to bound the work required to update mining scores when a new transaction is added, we must bound the size of connected components in the graph of transactions.  We refer to these connected components as \u003cem\u003eclusters\u003c/em\u003e.\u003c/p\u003e\n\u003ch2\u003e\u003ca name=\"p-1046-implications-of-a-cluster-size-limit-7\" class=\"anchor\" href=\"#p-1046-implications-of-a-cluster-size-limit-7\" aria-label=\"Heading link\"\u003e\u003c/a\u003eImplications of a cluster-size limit\u003c/h2\u003e\n\u003ch3\u003e\u003ca name=\"p-1046-we-can-keep-the-mempool-fully-sorted-at-all-times-8\" class=\"anchor\" href=\"#p-1046-we-can-keep-the-mempool-fully-sorted-at-all-times-8\" aria-label=\"Heading link\"\u003e\u003c/a\u003eWe can keep the mempool fully sorted at all times.\u003c/h3\u003e\n\u003cp\u003eTo fully sort the mempool, it suffices to (a) sort each cluster using some sorting/transaction selection algorithm (such as our existing ancestor-feerate algorithm), and (b) devising a way to merge sorted clusters together.\u003c/p\u003e\n\u003cp\u003eFor part (a), we can now realize that we might be able to do better than just use our ancestor feerate algorithm when sorting a cluster.  For very small clusters, we can use exponential run-time algorithms that select the optimal sort order across all possible topologically valid orderings of transactions.  For larger clusters, we can still fall back to using our ancestor feerate algorithm, or consider more advanced strategies, as long as we bound CPU usage to some reasonable value.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eNote\u003c/strong\u003e: we use the term \u003cem\u003elinearization\u003c/em\u003e to refer to the process of producing a topologically valid ordering of transactions in a cluster.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eFor part (b), merging sorted clusters together: this is straightforward.  Suppose we have two clusters, \u003cspan class=\"math\"\u003eC_1 = [tx_{a_1}, ..., tx_{a_n}]\u003c/span\u003e, and \u003cspan class=\"math\"\u003eC_2 = [tx_{b_1}, ..., tx_{b_m}]\u003c/span\u003e. To determine the optimal merge of these clusters into a single sort order, we want to find the highest feerate prefix of \u003cspan class=\"math\"\u003eC_1\u003c/span\u003e and the highest feerate prefix of \u003cspan class=\"math\"\u003eC_2\u003c/span\u003e and compare them; this best prefix should be the first set of transactions we would include in a block.  Then we’d remove those transactions from whichever cluster it came from, and repeat until all transactions are selected.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eNote\u003c/strong\u003e: we refer to the calculation of successive, highest-feerate-prefixes of a cluster as the \u003cem\u003echunks\u003c/em\u003e of a cluster.  We can calculate the chunks of a linearized cluster in linear time.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eFor more information on these topics, and other related concepts, see \u003ca class=\"mention\" href=\"/u/sipa\"\u003e@sipa\u003c/a\u003e’s writeup \u003ca href=\"https://delvingbitcoin.org/t/cluster-mempool-definitions-theory/202\"\u003ehere\u003c/a\u003e.\u003c/p\u003e\n\u003ch3\u003e\u003ca name=\"p-1046-with-a-fully-sorted-mempool-mining-transaction-selection-is-very-fast-9\" class=\"anchor\" href=\"#p-1046-with-a-fully-sorted-mempool-mining-transaction-selection-is-very-fast-9\" aria-label=\"Heading link\"\u003e\u003c/a\u003eWith a fully sorted mempool, mining transaction selection is very fast.\u003c/h3\u003e\n\u003cp\u003eWe simply iterate the clusters in descending chunk feerate order, and select transactions from each chunk, until the block is full.\u003c/p\u003e\n\u003ch3\u003e\u003ca name=\"p-1046-with-a-fully-sorted-mempool-eviction-is-the-opposite-of-mining-10\" class=\"anchor\" href=\"#p-1046-with-a-fully-sorted-mempool-eviction-is-the-opposite-of-mining-10\" aria-label=\"Heading link\"\u003e\u003c/a\u003eWith a fully sorted mempool, eviction is the opposite of mining.\u003c/h3\u003e\n\u003cp\u003eTo run the eviction algorithm when the mempool is full, we simply iterate the chunks in ascending chunk feerate order, and select chunks for eviction until the mempool is below its size limit.\u003c/p\u003e\n\u003ch3\u003e\u003ca name=\"p-1046-rbf-can-now-be-made-incentive-compatible-for-miners-11\" class=\"anchor\" href=\"#p-1046-rbf-can-now-be-made-incentive-compatible-for-miners-11\" aria-label=\"Heading link\"\u003e\u003c/a\u003eRBF can now be made incentive compatible for miners.\u003c/h3\u003e\n\u003cp\u003eThe first question to ask is what “incentive compatible” should mean.  To consider what this should mean, consider the following example, where the mempool starts with these transactions (assume all transactions are 100 vB in size):\u003c/p\u003e\n\u003cpre data-code-height=\"146\" data-code-wrap=\"mermaid\"\u003e\u003ccode class=\"lang-mermaid\"\u003egraph TD;\n    A[Tx A: 1 sat/vB]--\u0026gt;B[Tx B: 3.5 sat/vB];\n    A--\u0026gt;C[Tx C: 5 sat/vB];\n    D[Tx D: 1 sat/vB];\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eAnd consider a potential replacement transaction C’, which conflicts with tx C, and would produce the following mempool:\u003c/p\u003e\n\u003cpre data-code-height=\"146\" data-code-wrap=\"mermaid\"\u003e\u003ccode class=\"lang-mermaid\"\u003egraph TD;\n    A[Tx A: 1 sat/vB]--\u0026gt;B[Tx B: 3.5 sat/vB];\n    D[Tx D: 1 sat/vB]--\u0026gt;C'[Tx C': 6 sat/vB];\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eHow can we tell whether transaction C’ should be accepted?  Using the BIP 125 rules, we might say that C’ should be rejected because it brings in a new unconfirmed parent; or we might say that the unconfirmed parent rule is silly since we’re just replacing a parent with another parent of the exact same fee/size, and transaction C’ pays a strictly higher fee and feerate than C, so we should obviously take it.\u003c/p\u003e\n\u003cp\u003eIt turns out that neither of these explanations is wholly satisfactory.  Consider the total mempool sort order for the “old” mempool that contains tx C, and the “new” mempool that contains C’ instead.\u003c/p\u003e\n\u003cp\u003eThe optimal sort order for the “old” mempool is [A, B, C], [D] (brackets indicate chunks).  A miner who mined just the first chunk of that old mempool would collect 950 sats and have included 300 vB of size; a miner who mined the whole mempool would collect 1050 sats and included 400 vB of size.\u003c/p\u003e\n\u003cp\u003eMeanwhile, the optimal sort of the new mempool would be: [D, C’], [A, B]. A miner that included the first chunk would collect 700 sats in 200 vB of size; a miner including both chunks would collect 1150 sats in 400 vB of size.\u003c/p\u003e\n\u003cp\u003eWe can graph this data in what we have termed the “feerate diagram” of each mempool option, where on the x-axis we plot the size of the transactions included by a miner, and on the y-axis we plot the maximum fees that the miner can achieve at that size.  If we look at the feerate diagrams for this “old” and “new” mempool, we get the following:\u003c/p\u003e\n\u003cp\u003e\u003cdiv class=\"lightbox-wrapper\"\u003e\u003ca class=\"lightbox\" href=\"https://delvingbitcoin.org/uploads/default/original/1X/1c2a247ebb45bc3a930e83d0e900d19c741e8ca6.png\" data-download-href=\"https://delvingbitcoin.org/uploads/default/1c2a247ebb45bc3a930e83d0e900d19c741e8ca6\" title=\"Figure_1\"\u003e\u003cimg src=\"https://delvingbitcoin.org/uploads/default/optimized/1X/1c2a247ebb45bc3a930e83d0e900d19c741e8ca6_2_690x428.png\" alt=\"Figure_1\" data-base62-sha1=\"419ERV2azSMD2X1vcJOmTZenlRQ\" width=\"690\" height=\"428\" srcset=\"https://delvingbitcoin.org/uploads/default/optimized/1X/1c2a247ebb45bc3a930e83d0e900d19c741e8ca6_2_690x428.png, https://delvingbitcoin.org/uploads/default/original/1X/1c2a247ebb45bc3a930e83d0e900d19c741e8ca6.png 1.5x, https://delvingbitcoin.org/uploads/default/original/1X/1c2a247ebb45bc3a930e83d0e900d19c741e8ca6.png 2x\" data-dominant-color=\"FCFCFC\"\u003e\u003cdiv class=\"meta\"\u003e\u003csvg class=\"fa d-icon d-icon-far-image svg-icon\" aria-hidden=\"true\"\u003e\u003cuse href=\"#far-image\"\u003e\u003c/use\u003e\u003c/svg\u003e\u003cspan class=\"filename\"\u003eFigure_1\u003c/span\u003e\u003cspan class=\"informations\"\u003e1014×630 26.3 KB\u003c/span\u003e\u003csvg class=\"fa d-icon d-icon-discourse-expand svg-icon\" aria-hidden=\"true\"\u003e\u003cuse href=\"#discourse-expand\"\u003e\u003c/use\u003e\u003c/svg\u003e\u003c/div\u003e\u003c/a\u003e\u003c/div\u003e\u003c/p\u003e\n\u003cp\u003eNote that the (300, 950) point from the first diagram lies outside the diagram for the new mempool!  What this tells is that there are sizes at which the old mempool is better than the new one; and similarly in this example there are sizes for which the new mempool is better than the old one.\u003c/p\u003e\n\u003cp\u003eWe would say that these feerate diagrams are “incomparable”, meaning that neither one is strictly better than the other, and they are also not equivalent.  If the red line were greater than or equal to the blue line everywhere, and strictly better at least at one point, then we’d say that the new mempool was “strictly better” than the old one. This would mean that no matter how much size was selected from the mempool, that – ignoring the tail effects from when a block is nearly full – a miner would always be better off by selecting transactions from the “new” mempool rather than the old one.\u003c/p\u003e\n\u003cp\u003eThis graph shows why various heuristics for whether a replacement should be taken are unsatisfactory.  Instead, to ensure that a replacement is always better for miners, we propose making the RBF validation rule be that the mempool must get “strictly better” using this feerate diagram check in order to be accepted.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eNote\u003c/strong\u003e: To guard against free relay (an anti-DoS concern), we still will require that the total fee of the new transaction exceed the total fee of the conflicting transactions by at least as much as the min-relay-feerate * size of the new transaction.  However this is a small effect, since the incentive compatibility rule described above already requires that the total fee cannot go down – this just bumps that value up slightly.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003ch3\u003e\u003ca name=\"p-1046-the-cpfp-carveout-rule-can-no-longer-be-supported-12\" class=\"anchor\" href=\"#p-1046-the-cpfp-carveout-rule-can-no-longer-be-supported-12\" aria-label=\"Heading link\"\u003e\u003c/a\u003eThe CPFP carveout rule can no longer be supported.\u003c/h3\u003e\n\u003cp\u003eThe CPFP carveout rule in Bitcoin Core currently allows the mempool to exceed the limit on the number of descendant transactions that a given transaction has, in a specific case that is relevant to the lightning protocol.\u003c/p\u003e\n\u003cp\u003eThe background here is that by default, our mempool policy is to reject an incoming transaction if it would cause some existing mempool transaction to have more than 25 descendant transactions.  In some lightning use case, as I understand it, two parties might both have spendable outputs from some parent transaction.  One of the parties might have an incentive to “pin” the transaction at a low feerate in the mempool by creating child spends that fill up the 25 permitted descendants, but don’t effectively bump the feerate to a significant value.  Meanwhile, the other party might wish to use CPFP to fee bump the parent, but be unable to do so because of this descendant limit having already been hit.\u003c/p\u003e\n\u003cp\u003eThe CPFP carveout rule specifically allows the mempool to exceed the descendant limit by exactly 1 transaction in the case where an incoming transaction has exactly 1 in-mempool ancestor (and where the incoming transaction has some small bounded size in vbytes).  The implication is that in a 2-party protocol where each party has a single spendable output from some shared transaction, that either party should always be able to add a child transaction that can bump the fee of the parent.  For more background and thoughts on this, see the mailing list \u003ca href=\"https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-November/016518.html\"\u003ethread\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003eHowever, this CPFP carveout only makes sense because there are no other topology restrictions other than the descendant limit which can prevent a new transaction from being accepted.  By proposing the addition of a new cluster size limit, the CPFP carveout ceases to be helpful – an adversary can utilize the cluster size limit to prevent the other party from being able to attach a child transaction to the shared parent.  Moreover, because cluster limits are not a property of a particular transaction (the way that the descendant limit is), there is no straightforward way to introduce a new carveout that would apply to allow bypassing this limit in a bounded way.\u003c/p\u003e\n\u003cp\u003eSo the conclusion here is that the carveout rule must be dropped, and projects like lightning that currently rely on it would need to have some other mechanism to bypass this issue.  The \u003ca href=\"https://delvingbitcoin.org/t/v3-transaction-policy-for-anti-pinning/340/1\"\u003ev3 transaction proposal\u003c/a\u003e is one attempt to provide such a mechanism.\u003c/p\u003e\n\u003chr\u003e\n\u003ch1\u003e\u003ca name=\"p-1046-considerations-13\" class=\"anchor\" href=\"#p-1046-considerations-13\" aria-label=\"Heading link\"\u003e\u003c/a\u003eConsiderations\u003c/h1\u003e\n\u003ch2\u003e\u003ca name=\"p-1046-is-it-reasonable-to-bound-cluster-sizes-14\" class=\"anchor\" href=\"#p-1046-is-it-reasonable-to-bound-cluster-sizes-14\" aria-label=\"Heading link\"\u003e\u003c/a\u003eIs it reasonable to bound cluster sizes?\u003c/h2\u003e\n\u003cp\u003eA fair question to raise is whether usage patterns on the network are inconsistent with the idea of a cluster size limit; specifically, if it is fair or if it’s a bad design that other participants on the network can attach children to a transaction in a way that interferes with someone’s ability to (say) attach their own child transaction to an unconfirmed parent.\u003c/p\u003e\n\u003cp\u003eFor this specific concern, I don’t believe that the cluster size limits are materially different in character than the existing descendant size limits.  In particular, any scenario where an adversary can attach transactions that would cause the cluster size to be reached (thus preventing an additional spend) is also a scenario where that same adversary could cause a descendant limit to be reached today.\u003c/p\u003e\n\u003cp\u003eSo my belief is that there’s nothing fundamentally worse about cluster size limits than descendant size limits (provided that we can set the cluster size limits to be at least as large as the current default descendant limit, which I expect should be the case).\u003c/p\u003e\n\u003ch2\u003e\u003ca name=\"p-1046-why-is-there-a-bound-on-cluster-size-in-vbytes-15\" class=\"anchor\" href=\"#p-1046-why-is-there-a-bound-on-cluster-size-in-vbytes-15\" aria-label=\"Heading link\"\u003e\u003c/a\u003eWhy is there a bound on cluster size in vbytes?\u003c/h2\u003e\n\u003cp\u003eThis is for exactly the same reason that we have a bound on the ancestor and descendant sizes – it provides a bound on how poorly transaction selection can work when we use a greedy algorithm, in situations where the knapsack problem is what we need to solve.  Specifically, by limiting ancestor sizes to approximately 10% of the size of a block, our greedy transaction selection algorithm will always be at least 90% of the maximum fee that we’d achieve if we had a knapsack solver instead.\u003c/p\u003e\n\u003cp\u003eSimilarly, by bounding descendant sizes we also bound the small amount of free relay that can occur when mempool eviction happens.\u003c/p\u003e\n\u003cp\u003eTo achieve the same effect in cluster mempool, we need to bound chunk sizes.  The simplest way to do that is to bound a cluster size (since a cluster might consist of a single chunk).  While it may be theoretically possible to bound chunk sizes at a smaller value than the cluster size limit, this would create more theoretical questions that we’d need to resolve, which may not be worth the effort and complexity (as no longer would it be the case that chunk feerates within a cluster are monotonically decreasing).\u003c/p\u003e\n\u003ch2\u003e\u003ca name=\"p-1046-why-are-ancestor-and-descendant-limits-being-dropped-16\" class=\"anchor\" href=\"#p-1046-why-are-ancestor-and-descendant-limits-being-dropped-16\" aria-label=\"Heading link\"\u003e\u003c/a\u003eWhy are ancestor and descendant limits being dropped?\u003c/h2\u003e\n\u003cp\u003eOne benefit of the cluster mempool approach is that we no longer need to maintain ancestor and descendant feerate state, which means that we no longer need to iterate over ancestors/descendants when updating the mempool as transactions are added and removed, or when blocks are found.  This means that we no longer need to enforce limits on ancestor and descendant count in order to bound CPU usage.\u003c/p\u003e\n\u003chr\u003e\n\u003ch1\u003e\u003ca name=\"p-1046-next-steps-17\" class=\"anchor\" href=\"#p-1046-next-steps-17\" aria-label=\"Heading link\"\u003e\u003c/a\u003eNext steps\u003c/h1\u003e\n\u003cp\u003eA draft implementation against a recent version of the Bitcoin Core master branch is available \u003ca href=\"https://github.com/bitcoin/bitcoin/pull/28676\"\u003ehere\u003c/a\u003e.  The implementation details are still being improved upon, and the particular choices of parameters for cluster size limits and RBF will be determined after further benchmarking of the code and analysis of the effects of this logic on historical transaction data.\u003c/p\u003e\n\u003cp\u003eLooking ahead, the framework introduced here for reasoning about the mempool (and RBF specifically) seems helpful for \u003ca href=\"https://delvingbitcoin.org/t/post-clustermempool-package-rbf-per-chunk-processing/190\"\u003ereasoning about package validation and package RBF\u003c/a\u003e, though that work is still in progress.\u003c/p\u003e","post_number":1,"post_type":1,"posts_count":24,"updated_at":"2024-04-25T20:25:17.720Z","reply_count":3,"reply_to_post_number":null,"quote_count":0,"incoming_link_count":2409,"reads":162,"readers_count":161,"score":12257.4,"yours":false,"topic_id":393,"topic_slug":"an-overview-of-the-cluster-mempool-proposal","display_username":"Suhas Daftuar","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":8,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"link_counts":[{"url":"https://github.com/bitcoin/bitcoin/issues/27677","internal":false,"reflection":false,"title":"Proposal for a new mempool design · Issue #27677 · bitcoin/bitcoin · GitHub","clicks":97},{"url":"https://github.com/bitcoin/bitcoin/pull/28676","internal":false,"reflection":false,"title":"[WIP] Cluster mempool implementation by sdaftuar · Pull Request #28676 · bitcoin/bitcoin · GitHub","clicks":58},{"url":"https://github.com/bitcoin/bitcoin/issues/27677#eviction-mining-not-opposite","internal":false,"reflection":false,"title":"Proposal for a new mempool design · Issue #27677 · bitcoin/bitcoin · GitHub","clicks":25},{"url":"https://delvingbitcoin.org/t/post-clustermempool-package-rbf-per-chunk-processing/190","internal":true,"reflection":false,"title":"Post-clustermempool package RBF: per-chunk processing","clicks":12},{"url":"https://delvingbitcoin.org/t/cluster-mempool-definitions-theory/202","internal":true,"reflection":false,"title":"Cluster mempool definitions \u0026 theory","clicks":10},{"url":"https://github.com/bitcoin/bips/blob/deae64bfd31f6938253c05392aa355bf6d7e7605/bip-0125.mediawiki","internal":false,"reflection":false,"clicks":10},{"url":"https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-November/016518.html","internal":false,"reflection":false,"title":"[bitcoin-dev] CPFP Carve-Out for Fee-Prediction Issues in Contracting Applications (eg Lightning)","clicks":7},{"url":"https://delvingbitcoin.org/t/v3-transaction-policy-for-anti-pinning/340","internal":true,"reflection":false,"title":"V3 transaction policy for anti-pinning","clicks":0},{"url":"https://delvingbitcoin.org/t/v3-transaction-policy-for-anti-pinning/340/32","internal":true,"reflection":true,"title":"V3 transaction policy for anti-pinning","clicks":5},{"url":"https://delvingbitcoin.org/t/mempool-incentive-compatibility/553","internal":true,"reflection":true,"title":"Mempool Incentive Compatibility","clicks":2},{"url":"https://delvingbitcoin.org/t/introduction-to-cluster-linearization/1032","internal":true,"reflection":true,"title":"Introduction to cluster linearization","clicks":2},{"url":"https://delvingbitcoin.org/t/research-into-the-effects-of-a-cluster-size-limited-mempool-in-2023/794","internal":true,"reflection":true,"title":"Research into the effects of a cluster size limited mempool in 2023","clicks":2},{"url":"https://delvingbitcoin.org/t/support-for-graphing-data/1180","internal":true,"reflection":true,"title":"Support for graphing data","clicks":1},{"url":"https://delvingbitcoin.org/t/replace-by-fee-rate-vs-v3/488/2","internal":true,"reflection":true,"title":"Replace-By-Fee-Rate vs V3","clicks":0},{"url":"https://delvingbitcoin.org/t/generalizing-rbf-under-cluster-mempool/2115","internal":true,"reflection":true,"title":"Generalizing RBF under Cluster Mempool","clicks":0},{"url":"https://delvingbitcoin.org/t/v3-and-some-possible-futures/523","internal":true,"reflection":true,"title":"V3 and some possible futures","clicks":0}],"read":true,"user_title":null,"bookmarked":false,"actions_summary":[{"id":2,"count":15}],"moderator":false,"admin":false,"staff":false,"user_id":95,"hidden":false,"trust_level":3,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/an-overview-of-the-cluster-mempool-proposal/393/1","reactions":[{"id":"rocket","type":"emoji","count":11},{"id":"+1","type":"emoji","count":4}],"current_user_reaction":null,"reaction_users_count":15,"current_user_used_main_reaction":false,"can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null,"can_vote":false},{"id":1087,"name":"David A. Harding","username":"harding","avatar_template":"/user_avatar/delvingbitcoin.org/harding/{size}/925_2.png","created_at":"2024-01-12T01:26:13.557Z","cooked":"\u003caside class=\"quote no-group\" data-username=\"sdaftuar\" data-post=\"1\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/letter_avatar_proxy/v4/letter/s/0ea827/48.png\" class=\"avatar\"\u003e sdaftuar:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003ewe propose making the RBF validation rule be that the mempool must get “strictly better” using this feerate diagram check in order to be accepted.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eHow is that comparison made when two feerate diagrams terminate at different points?  For example, OLD terminates at 123,456 vbytes and NEW terminates at 78,901 vbytes?  If the accumulated fees of all the transactions in NEW is above the accumulated fees at the 78,901th byte in OLD, do we choose NEW even if OLD accumulates more fees by the end of its final transaction?\u003c/p\u003e\n\u003cp\u003eWhat I’m really asking is what is the chance that RBF pinning will still be an issue because replacing a ~100,000 vbyte transaction at 1 sat/vbyte with a ~100 vbyte transaction will still cost more than 1,000x what the user wanted to pay because the terminal end of the new mempool with the replacement is ~100,000 sats lower than the old mempool.\u003c/p\u003e\n\u003caside class=\"quote no-group quote-modified\" data-username=\"sdaftuar\" data-post=\"1\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/letter_avatar_proxy/v4/letter/s/0ea827/48.png\" class=\"avatar\"\u003e sdaftuar:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003ch3\u003eThe CPFP carveout rule can no longer be supported.\u003c/h3\u003e\n\u003cp\u003e[…] because cluster limits are not a property of a particular transaction (the way that the descendant limit is), there is no straightforward way to introduce a new carveout that would apply to allow bypassing this limit in a bounded way.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eThere’s really no easy way around this?  For example, when a transaction comes in that matches the carve-out rules, it gets a simple flag set on it that prevents it from being counted against its max cluster size, or that prevents it from getting evicted from that cluster if the cluster size exceeds the maximum?\u003c/p\u003e\n\u003cp\u003e(I’m all for removing special case rules like carve-out; it’s just surprising to me that there’s no easy workaround here.)\u003c/p\u003e","post_number":2,"post_type":1,"posts_count":24,"updated_at":"2024-01-12T01:26:13.557Z","reply_count":1,"reply_to_post_number":null,"quote_count":1,"incoming_link_count":6,"reads":97,"readers_count":96,"score":54.4,"yours":false,"topic_id":393,"topic_slug":"an-overview-of-the-cluster-mempool-proposal","display_username":"David A. Harding","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":1,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"read":true,"user_title":null,"bookmarked":false,"actions_summary":[],"moderator":false,"admin":false,"staff":false,"user_id":6,"hidden":false,"trust_level":3,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/an-overview-of-the-cluster-mempool-proposal/393/2","reactions":[],"current_user_reaction":null,"reaction_users_count":0,"current_user_used_main_reaction":false,"can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":1089,"name":"Suhas Daftuar","username":"sdaftuar","avatar_template":"/letter_avatar_proxy/v4/letter/s/0ea827/{size}.png","created_at":"2024-01-12T15:04:58.503Z","cooked":"\u003cp\u003e\u003ca class=\"mention\" href=\"/u/harding\"\u003e@harding\u003c/a\u003e Thanks for giving this a thorough read!\u003c/p\u003e\n\u003caside class=\"quote no-group\" data-username=\"harding\" data-post=\"2\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/user_avatar/delvingbitcoin.org/harding/48/925_2.png\" class=\"avatar\"\u003e harding:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eHow is that comparison made when two feerate diagrams terminate at different points? For example, OLD terminates at 123,456 vbytes and NEW terminates at 78,901 vbytes? If the accumulated fees of all the transactions in NEW is above the accumulated fees at the 78,901th byte in OLD, do we choose NEW even if OLD accumulates more fees by the end of its final transaction?\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eGreat question, as this is something that I have discussed at length with others.\u003c/p\u003e\n\u003cp\u003eI take the view that the feerate diagram should represent the amount of fees that a miner would receive by mining a given number of vbytes from the mempool, and if the mempool runs out of vbytes when you’re trying to look up the corresponding total fees for that size, then the natural thing to do is to assume you have 0-fee paying transactions that take up the additional space.\u003c/p\u003e\n\u003cp\u003eSo in your example, I would pad the diagram for NEW to be of the same size as OLD, and the amount of fees in NEW’s diagram at 123,456 vbytes would be the same as the amount of fees in NEW’s diagram at 78,901 vbytes.  And so if NEW is greater than OLD at 78,901 vbytes, but less than old at 123,456 vbytes, I would say that the two mempools are incomparable, and not accept a transaction that would move us from one of those states to the other.\u003c/p\u003e\n\u003cp\u003eAnother point of view one could have is that there’s some “tail feerate”, possibly greater than 0, at which there is essentially an infinite supply of transactions that could be included by miners in blocks.  If this is your view, then you could imagine picking some tail feerate value (presumably, this is a value that has an upper bound given by the lowest chunk feerate in the mempool), and then extrapolating the feerate diagram of NEW at that feerate so that it has the same size as OLD, and then doing a comparison on the resulting diagrams which will then be the same size.\u003c/p\u003e\n\u003cp\u003eHowever, if you take this view that the tail feerate is greater than 0, then this should also change the way you view a NEW mempool that has \u003cem\u003emore\u003c/em\u003e vbytes than the OLD mempool, such that you ought to extrapolate the feerate diagram of OLD in that situation so that you are imputing more fees in the OLD mempool than are actually there.  This in turn would then prevent some replacements that would be successful if we used a tail feerate of 0 instead, and frankly it seems surprising to me that we’d assume the existence of fees that a miner running our software can’t actually realize.\u003c/p\u003e\n\u003cp\u003eSo my overall view is that a tail feerate of 0 is the most justifiable idea right now.  Moreover, even if we had an interpretation of mempool incentive compatibility that permitted some non-zero tail feerate, we’d still be limited by the anti-DoS rule that requires total fees to go up (to prevent free relay), making the point moot – so for now at least, I think this is more of a theoretical issue than a practical one.\u003c/p\u003e\n\u003cp\u003eI could imagine in the future that research could be done regarding incentive-compatibility to take into account the idea that feerates near the top of the mempool are more significant for miners than feerates near the bottom of the mempool, and therefore an optimal diagram comparison would apply some kind of discounting to things at the bottom versus things at the top. And should someone develop a robust model for how to do this, I think we could update our software in the future to implement a different test.\u003c/p\u003e\n\u003caside class=\"quote no-group\" data-username=\"harding\" data-post=\"2\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/user_avatar/delvingbitcoin.org/harding/48/925_2.png\" class=\"avatar\"\u003e harding:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThere’s really no easy way around this? For example, when a transaction comes in that matches the carve-out rules, it gets a simple flag set on it that prevents it from being counted against its max cluster size, or that prevents it from getting evicted from that cluster if the cluster size exceeds the maximum?\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eI don’t have a proof, but my intuition is that there is no solution here.\u003c/p\u003e\n\u003cp\u003eThe problem is that literally any non-child transaction in the mempool today might be a candidate for having a child that satisfies the criteria of the carveout rule.  For the sake of my illustrative capabilities, assume that we’d like to enforce a cluster size limit of 10, and we had a mempool cluster that looked like this:\u003c/p\u003e\n\u003cpre data-code-height=\"146\" data-code-wrap=\"mermaid\"\u003e\u003ccode class=\"lang-mermaid\"\u003egraph TD;\n    P1--\u0026gt;C1\n    P2--\u0026gt;C2\n    P2--\u0026gt;C1\n    P3--\u0026gt;C3;\n    P3--\u0026gt;C2;\n    P4--\u0026gt;C3;\n    P4--\u0026gt;C4;\n    P5--\u0026gt;C4;\n    P5--\u0026gt;C5;\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eThis cluster is full, but any any one of those transactions P1, P2, …, P5 might have another spendable output allowing creation of children that would match the requirements of the current carveout rule (which just checks that the incoming transaction have exactly 1 ancestor and be below some size limit).  So to implement carveout in its current form, we’d have to be either willing to (a) evict an existing transaction from this cluster to make room, or (b) allow all 5 new transactions to be added, blowing through our cluster size limit of 10 and creating a cluster of size 15.\u003c/p\u003e\n\u003cp\u003eNeither of these options is good.  For (a), there are two issues with evicting an existing transaction.  The first is that any of the in-mempool parent transactions might be a shared transaction in some multiparty contracting protocol, and the other participant might be relying on \u003cem\u003etheir\u003c/em\u003e spend of their output to be the one that gets the parent confirmed.  Now, we could mitigate that by trying to use our RBF rules to find another transaction in the cluster to evict, to make room for the new transaction; but this is problematic for two reasons.  One is that you could easily be pinned by transactions that are larger but pay a lower feerate, and be forced into a choice of supporting carveout at the expense of opening up free relay attacks; the other is that this introduces a great deal of complexity into our validation code if we have to find a transaction to evict, given that there may be many choices for which one to evict, creating a lot of processing overhead to determine which is the most incentive compatible outcome (and I imagine there could easily be incomparable options to sort amongst).\u003c/p\u003e\n\u003cp\u003eFor (b), my philosophy on this work is that we should strive for cluster size limits that we are allowed to meet and never exceed – for instance, if we thought that it was safe to have clusters with 100 transactions, maybe you could design a policy regime where the default limit is much lower (say 50), and then we permit carveout exceptions and try to reason that this could never take us above 100.  I think this would be bad design, though, as it means that we’re significantly restricting the permitted cluster size for the vast majority of network users in order to try to reason about what I view as a very edge case situation.  My engineering opinion is also that this is just bad software design, of having a limit in mind that you can’t actually enforce without some analysis of emergent properties at a higher layer.\u003c/p\u003e\n\u003cp\u003eAnyway, it’s possible that I’m wrong about there being no simple workarounds here, so if you have other ideas of things to consider please suggest away…\u003c/p\u003e\n\u003cp\u003eHowever it strikes me that there’s a more obvious solution available to us: if we have an idea for what \u003cem\u003etopology\u003c/em\u003e the carveout rule is intended to match, then we should instead just offer a policy regime that allows transactions to opt into that topology, and then we can enforce cluster size limits as we’d like to.  The v3 transaction proposal is designed with this in mind, so if it proves to be workable for the use cases that currently rely on carveout, I think this would be vastly superior.\u003c/p\u003e","post_number":3,"post_type":1,"posts_count":24,"updated_at":"2024-01-12T15:04:58.503Z","reply_count":2,"reply_to_post_number":2,"quote_count":1,"incoming_link_count":839,"reads":93,"readers_count":92,"score":4228.6,"yours":false,"topic_id":393,"topic_slug":"an-overview-of-the-cluster-mempool-proposal","display_username":"Suhas Daftuar","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":1,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"read":true,"user_title":null,"bookmarked":false,"actions_summary":[{"id":2,"count":2}],"moderator":false,"admin":false,"staff":false,"user_id":95,"hidden":false,"trust_level":3,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/an-overview-of-the-cluster-mempool-proposal/393/3","reactions":[{"id":"+1","type":"emoji","count":2},{"id":"heart","type":"emoji","count":1}],"current_user_reaction":null,"reaction_users_count":3,"current_user_used_main_reaction":false,"can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":1090,"name":"Gregory Sanders","username":"instagibbs","avatar_template":"/user_avatar/delvingbitcoin.org/instagibbs/{size}/28_2.png","created_at":"2024-01-12T15:21:25.770Z","cooked":"\u003caside class=\"quote no-group\" data-username=\"sdaftuar\" data-post=\"3\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/letter_avatar_proxy/v4/letter/s/0ea827/48.png\" class=\"avatar\"\u003e sdaftuar:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eI could imagine in the future that research could be done regarding incentive-compatibility to take into account the idea that feerates near the top of the mempool are more significant for miners than feerates near the bottom of the mempool, and therefore an optimal diagram comparison would apply some kind of discounting to things at the bottom versus things at the top. And should someone develop a robust model for how to do this, I think we could update our software in the future to implement a different test.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eYes, it’s completely natural to apply some sort of future discounting, as something far away from “top blocks” are likely to:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003ebe trimmed\u003c/li\u003e\n\u003cli\u003ebe RBF’d\u003c/li\u003e\n\u003cli\u003etimeout\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eso reflecting that uncertainty directly is something someone can explore further. I did some initial number analysis to see if it could help anti-pin(it can’t \u003cimg src=\"https://delvingbitcoin.org/images/emoji/twitter/slight_smile.png?v=12\" title=\":slight_smile:\" class=\"emoji\" alt=\":slight_smile:\" loading=\"lazy\" width=\"20\" height=\"20\"\u003e )\u003c/p\u003e\n\u003caside class=\"quote no-group\" data-username=\"sdaftuar\" data-post=\"3\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/letter_avatar_proxy/v4/letter/s/0ea827/48.png\" class=\"avatar\"\u003e sdaftuar:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003e(a) evict an existing transaction from this cluster to make room\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eA minor note, but this also necessarily opens up to “mempoolfullrbf” style RBFs, good or bad.\u003c/p\u003e\n\u003cp\u003eOne idea I’d had, which Suhas has essentially responded to above, is that we could:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003ereceive new chunk somehow that exceeds cluster limits\u003c/li\u003e\n\u003cli\u003eallow temporary exceeding of limits\u003c/li\u003e\n\u003cli\u003ere-linearize cluster with exceeded limits\u003c/li\u003e\n\u003cli\u003e“chop off” the excess at the end\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eAs Suhas notes, this can get pretty weird with pinning/anti-DoS. I don’t think it’s a dead-end, but more of a whiteboarding project at this point.\u003c/p\u003e","post_number":4,"post_type":1,"posts_count":24,"updated_at":"2024-01-12T15:31:36.404Z","reply_count":0,"reply_to_post_number":3,"quote_count":1,"incoming_link_count":19,"reads":74,"readers_count":73,"score":109.8,"yours":false,"topic_id":393,"topic_slug":"an-overview-of-the-cluster-mempool-proposal","display_username":"Gregory Sanders","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":2,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"read":true,"user_title":null,"bookmarked":false,"actions_summary":[],"moderator":false,"admin":false,"staff":false,"user_id":31,"hidden":false,"trust_level":3,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/an-overview-of-the-cluster-mempool-proposal/393/4","reactions":[],"current_user_reaction":null,"reaction_users_count":0,"current_user_used_main_reaction":false,"can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":1095,"name":"David A. Harding","username":"harding","avatar_template":"/user_avatar/delvingbitcoin.org/harding/{size}/925_2.png","created_at":"2024-01-12T18:04:39.566Z","cooked":"\u003cp\u003eThe idea described below in this post doesn’t work.  I tried to find a way to make carveout compatible with cluster mempool and failed.  Normally I wouldn’t advertise my failures, but I think it could be useful to future discussions to show several people tried working this problem and couldn’t find a solution.\u003c/p\u003e\n\u003caside class=\"quote no-group\" data-username=\"sdaftuar\" data-post=\"3\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/letter_avatar_proxy/v4/letter/s/0ea827/48.png\" class=\"avatar\"\u003e sdaftuar:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003ethe current carveout rule (which just checks that the incoming transaction have exactly 1 ancestor and be below some size limit).\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eYour description and diagram match how carveout works (AIUI) but I think it’s worth noting that LN anchor channels only have exactly 2 outputs that can be spent in the same block as the parent transaction (see BOLT3).  This is necessary to prevent a malicious counterparty who controls \u0026gt;1 output from using the carveout themselves and preventing its use by the honest counterparty, so I think we can safely assume that any other thoughtful users of carveouts are doing something similar.  An implication of this is that your diagram’s P2 to P5 could not have a carveout attached because both immediately-spendable outputs have already been spent.\u003c/p\u003e\n\u003cp\u003eI think we can reformulate carveout’s rules as used by LN:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003eA carveout is 1,000 vbytes or less\u003c/li\u003e\n\u003cli\u003eA carveout has exactly one unconfirmed ancestor (it’s parent)\u003c/li\u003e\n\u003cli\u003eA carveout’s parent has no unconfirmed ancestors\u003c/li\u003e\n\u003cli\u003eA carveout is either the first or second spent output of a transaction that has fewer than three of its outputs spent so far\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eChecking whether a newly received transaction obeys rule \u003cspan class=\"hashtag-raw\"\u003e#1\u003c/span\u003e is obviously easy. Checking that it obeys rule \u003cspan class=\"hashtag-raw\"\u003e#2\u003c/span\u003e is also easy because we have to look up each of a transaction’s inputs anyway.  When we find the carveout’s exactly one unconfirmed ancestor, we can check that it obeys rule \u003cspan class=\"hashtag-raw\"\u003e#3\u003c/span\u003e using the same mechanism (no special traits or caching required).  Now to check rule \u003cspan class=\"hashtag-raw\"\u003e#4\u003c/span\u003e we need only determine how many spends of the parent transaction’s outputs are currently in the mempool; if it’s currently 0 or 1, and all the previous rules were satisfied, we tag this transaction as a carveout and allow it to exceed the cluster size limit.\u003c/p\u003e\n\u003cp\u003eUsing free CPU cycles, we can periodically check every transaction tagged as a carveout to see if its parent now has three or more spends, in which case we can clear the carveout flag.\u003c/p\u003e\n\u003cp\u003eWhen writing the above, I thought my proposed scheme meant a cluster could never have more than two carveouts.  However, here’s a case where there can be many more:\u003c/p\u003e\n\u003cpre data-code-height=\"200\" data-code-wrap=\"mermaid\"\u003e\u003ccode class=\"lang-mermaid\"\u003egraph TD;\n  P1 --\u0026gt; CO1;\n  P2 --\u0026gt; CO2;\n  P3 --\u0026gt; CO3;\n  P1 \u0026amp; P2 \u0026amp; P3 --\u0026gt; C1[Regular child];\n\n\u003c/code\u003e\u003c/pre\u003e","post_number":5,"post_type":1,"posts_count":24,"updated_at":"2024-01-12T18:04:39.566Z","reply_count":2,"reply_to_post_number":3,"quote_count":1,"incoming_link_count":7,"reads":76,"readers_count":75,"score":75.2,"yours":false,"topic_id":393,"topic_slug":"an-overview-of-the-cluster-mempool-proposal","display_username":"David A. Harding","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":1,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"read":true,"user_title":null,"bookmarked":false,"actions_summary":[{"id":2,"count":1}],"moderator":false,"admin":false,"staff":false,"user_id":6,"hidden":false,"trust_level":3,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/an-overview-of-the-cluster-mempool-proposal/393/5","reactions":[{"id":"+1","type":"emoji","count":1}],"current_user_reaction":null,"reaction_users_count":1,"current_user_used_main_reaction":false,"can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":1096,"name":"Gregory Sanders","username":"instagibbs","avatar_template":"/user_avatar/delvingbitcoin.org/instagibbs/{size}/28_2.png","created_at":"2024-01-12T18:21:48.769Z","cooked":"\u003caside class=\"quote no-group\" data-username=\"harding\" data-post=\"5\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/user_avatar/delvingbitcoin.org/harding/48/925_2.png\" class=\"avatar\"\u003e harding:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eA carveout is 1,000 vbytes or less\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003edoesn’t actually matter for the example, but I believe it’s 10kvB\u003c/p\u003e","post_number":6,"post_type":1,"posts_count":24,"updated_at":"2024-01-12T18:21:48.769Z","reply_count":0,"reply_to_post_number":5,"quote_count":1,"incoming_link_count":1,"reads":69,"readers_count":68,"score":18.8,"yours":false,"topic_id":393,"topic_slug":"an-overview-of-the-cluster-mempool-proposal","display_username":"Gregory Sanders","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":1,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"read":true,"user_title":null,"bookmarked":false,"actions_summary":[],"moderator":false,"admin":false,"staff":false,"user_id":31,"hidden":false,"trust_level":3,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/an-overview-of-the-cluster-mempool-proposal/393/6","reactions":[],"current_user_reaction":null,"reaction_users_count":0,"current_user_used_main_reaction":false,"can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":1097,"name":"Suhas Daftuar","username":"sdaftuar","avatar_template":"/letter_avatar_proxy/v4/letter/s/0ea827/{size}.png","created_at":"2024-01-12T19:27:17.249Z","cooked":"\u003caside class=\"quote no-group\" data-username=\"harding\" data-post=\"5\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/user_avatar/delvingbitcoin.org/harding/48/925_2.png\" class=\"avatar\"\u003e harding:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eI think we can reformulate carveout’s rules as used by LN:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003eA carveout is 1,000 vbytes or less\u003c/li\u003e\n\u003cli\u003eA carveout has exactly one unconfirmed ancestor (it’s parent)\u003c/li\u003e\n\u003cli\u003eA carveout’s parent has no unconfirmed ancestors\u003c/li\u003e\n\u003cli\u003eA carveout is either the first or second spent output of a transaction that has fewer than three of its outputs spent so far\u003c/li\u003e\n\u003c/ol\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eJust thought it is worth noting here that in trying to think about what the LN carveout rule should match for, rules 2, 3 and 4 are all about the transaction graph topology – and yet currently, the LN anchors have no enforced topology restrictions (beyond their construction of having 2 spendable outputs – but those outputs can be spent arbitrarily).  If we had a way to enforce that spends of an anchor output were not permitted to have additional children or pull in additional parents, then this problem again becomes easy – which is exactly what v3 contemplates.\u003c/p\u003e","post_number":7,"post_type":1,"posts_count":24,"updated_at":"2024-01-12T19:27:17.249Z","reply_count":0,"reply_to_post_number":5,"quote_count":1,"incoming_link_count":128,"reads":70,"readers_count":69,"score":654.0,"yours":false,"topic_id":393,"topic_slug":"an-overview-of-the-cluster-mempool-proposal","display_username":"Suhas Daftuar","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":1,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"read":true,"user_title":null,"bookmarked":false,"actions_summary":[],"moderator":false,"admin":false,"staff":false,"user_id":95,"hidden":false,"trust_level":3,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/an-overview-of-the-cluster-mempool-proposal/393/7","reactions":[{"id":"heart","type":"emoji","count":1}],"current_user_reaction":null,"reaction_users_count":1,"current_user_used_main_reaction":false,"can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":1429,"name":"Suhas Daftuar","username":"sdaftuar","avatar_template":"/letter_avatar_proxy/v4/letter/s/0ea827/{size}.png","created_at":"2024-02-02T14:30:38.096Z","cooked":"\u003cp\u003eI was wondering if a form of sibling eviction might work as a topological (if not economic) substitute for the carveout rule going away.  That is, is there a sibling eviction policy that we could adopt which would give current users of the CPFP carveout rule an economic way to get a transaction into the mempool when the cluster it would be joining is full? If so, then this might be an alternative we could consider to the suggestion of using v3 policy rules as a carveout replacement.\u003c/p\u003e\n\u003cp\u003eSpecifically, I was wondering if we could implement the following sibling eviction policy, as part of the cluster mempool approach:\u003c/p\u003e\n\u003cp\u003eIf a new transaction is rejected from the mempool due to a cluster size limit, and it has a single unconfirmed parent, and that parent has exactly 1 other unconfirmed child, then consider that child as a candidate for eviction in order to accept the new transaction.  Specifically, we’d use our RBF rules (feerate diagram check) to determine if the mempool + new transaction - replacement candidate is strictly better than the existing mempool, and process the replacement if so.\u003c/p\u003e\n\u003cp\u003eI should point out immediately that with no other constraints on the sibling described in that rule, that RBF pinning with a high fee transaction would be possible.  Still, I wondered if having \u003cem\u003esome\u003c/em\u003e way to evict a transaction to make room for another might still be valuable, in a general setting, as that gives users more options than rules which have no economic mechanism to bypass.\u003c/p\u003e\n\u003cp\u003eHowever, I don’t think we can easily achieve this behavior while maintaining an additional intuitive property:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eAny transactions we evict under this policy should not succeed in re-entering the mempool if immediately resubmitted.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThe proposed rule fails this test.  Consider this picture:\u003c/p\u003e\n\u003cpre data-code-height=\"233\" data-code-wrap=\"mermaid\"\u003e\u003ccode class=\"lang-mermaid\"\u003e    flowchart \n         A --\u0026gt; B\n         A-.-\u0026gt; C\n         B--\u0026gt; D\n        E[\"E: some tx connected to a big cluster\"] --\u0026gt; B\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eImagine tx C arrives, and we consider eviction of tx B to make room for it. Even if “accept tx C and evict tx B (along with B’s descendants)” is a valid replacement under the feerate diagram check, it’s possible that immediately re-accepting tx B would be successful, because without tx D it might be under the cluster size limits.\u003c/p\u003e\n\u003cp\u003eTo resolve this, we could (say) require for our rule that tx B only be evaluated for eviction if it has no unconfirmed children.  However, then we immediately lose the benefits for something like this as a carveout alternative, because it’s trivially pinnable by a counterparty where if the other sibling has multiple children, no sibling eviction would be considered.\u003c/p\u003e\n\u003cp\u003eAlternatively, we could find some child transaction to consider evicting; however if there is more than one candidate, then we might be in a difficult situation of trying to pick which one to evaluate, and there’s no obvious way to make that choice.\u003c/p\u003e\n\u003cp\u003eBy contrast, the v3 rules resolve this problem, by requiring that v3 children have no children of their own.\u003c/p\u003e\n\u003cp\u003e(Just sharing this in the spirit of \u003ca class=\"mention\" href=\"/u/harding\"\u003e@harding\u003c/a\u003e’s comment about the potential value in looking at failed ideas!)\u003c/p\u003e","post_number":8,"post_type":1,"posts_count":24,"updated_at":"2024-02-02T14:30:38.096Z","reply_count":2,"reply_to_post_number":null,"quote_count":0,"incoming_link_count":6,"reads":72,"readers_count":71,"score":54.4,"yours":false,"topic_id":393,"topic_slug":"an-overview-of-the-cluster-mempool-proposal","display_username":"Suhas Daftuar","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":1,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"link_counts":[{"url":"https://delvingbitcoin.org/t/v3-and-some-possible-futures/523","internal":true,"reflection":true,"title":"V3 and some possible futures","clicks":1}],"read":true,"user_title":null,"bookmarked":false,"actions_summary":[],"moderator":false,"admin":false,"staff":false,"user_id":95,"hidden":false,"trust_level":3,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/an-overview-of-the-cluster-mempool-proposal/393/8","reactions":[],"current_user_reaction":null,"reaction_users_count":0,"current_user_used_main_reaction":false,"can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":1441,"name":"Gregory Sanders","username":"instagibbs","avatar_template":"/user_avatar/delvingbitcoin.org/instagibbs/{size}/28_2.png","created_at":"2024-02-02T14:41:04.334Z","cooked":"\u003caside class=\"quote no-group\" data-username=\"sdaftuar\" data-post=\"8\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/letter_avatar_proxy/v4/letter/s/0ea827/48.png\" class=\"avatar\"\u003e sdaftuar:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eHowever, I don’t think we can easily achieve this behavior while maintaining an additional intuitive property:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eAny transactions we evict under this policy should not succeed in re-entering the mempool if immediately resubmitted.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eCan you motivate this as a principled objection? Mempool quality increases, just not in the most optimal way possible. The alternative is to not improve the mempool at all?\u003c/p\u003e\n\u003caside class=\"quote no-group\" data-username=\"sdaftuar\" data-post=\"8\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/letter_avatar_proxy/v4/letter/s/0ea827/48.png\" class=\"avatar\"\u003e sdaftuar:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eAlternatively, we could find some child transaction to consider evicting; however if there is more than one candidate, then we might be in a difficult situation of trying to pick which one to evaluate, and there’s no obvious way to make that choice.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eAgain, if our constraint is “we can only pick the optimal evictions”, sure this is intractable, but I am not convinced of that. For now I’d rather focus on the premise of the objection on the simple non-v3 case.\u003c/p\u003e\n\u003chr\u003e\n\u003cp\u003eAlso while we’re here, the other strategy previously discussed to allow cluster limits to be breached, temporarily and in some limited size/count way, linearize, then “prune” until you get back to cluster limits. Accept/reject said linearization based on normal diagram checks. I know you’ve already dismissed this, but good to have alternatives all in one place.\u003c/p\u003e\n\u003cp\u003eWith other objections you’ve had aside, I think this strategy is more limited because you may not be able to practically make the sibling eviction candidates small.\u003c/p\u003e","post_number":9,"post_type":1,"posts_count":24,"updated_at":"2024-02-02T14:46:32.086Z","reply_count":1,"reply_to_post_number":8,"quote_count":1,"incoming_link_count":3,"reads":59,"readers_count":58,"score":31.8,"yours":false,"topic_id":393,"topic_slug":"an-overview-of-the-cluster-mempool-proposal","display_username":"Gregory Sanders","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":2,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"read":true,"user_title":null,"bookmarked":false,"actions_summary":[],"moderator":false,"admin":false,"staff":false,"user_id":31,"hidden":false,"trust_level":3,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/an-overview-of-the-cluster-mempool-proposal/393/9","reactions":[],"current_user_reaction":null,"reaction_users_count":0,"current_user_used_main_reaction":false,"can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":1442,"name":"Suhas Daftuar","username":"sdaftuar","avatar_template":"/letter_avatar_proxy/v4/letter/s/0ea827/{size}.png","created_at":"2024-02-02T14:51:20.319Z","cooked":"\u003caside class=\"quote no-group\" data-username=\"instagibbs\" data-post=\"9\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/user_avatar/delvingbitcoin.org/instagibbs/48/28_2.png\" class=\"avatar\"\u003e instagibbs:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eCan you motivate this as a principled objection? Mempool quality increases, just not in the most optimal way possible. The alternative is to not improve the mempool at all?\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eHmm, I’m not sure that I can – it just seems like this is a messy behavior, and so I think we’re missing something in the design if we can’t come up with a cleaner way of achieving the goal.  It’s wasteful to have to rebroadcast/revalidate transactions that never should have been evicted in the first place.\u003c/p\u003e\n\u003cp\u003eAlso I suppose there are other solutions we could consider; one is that we somehow require a peer sending us this transaction to do the work of calculating which transaction we should evict to make room for a new transaction, which we then just verify will satisfy our requirements? Seems like that could be a “clean” approach from an engineering aesthetics point of view, though it begs the question of how that determination might be made by a wallet in the first place.\u003c/p\u003e\n\u003cp\u003eEdit: Maybe the broader observation is, if this narrow idea for sibling eviction is still this messy, and it has obvious pinning vectors anyway due to how RBF works, and it only applies in a very limited topology to begin with, is it worth adding the complexity to the implementation?\u003c/p\u003e","post_number":10,"post_type":1,"posts_count":24,"updated_at":"2024-02-02T15:00:33.098Z","reply_count":1,"reply_to_post_number":9,"quote_count":1,"incoming_link_count":2,"reads":60,"readers_count":59,"score":42.0,"yours":false,"topic_id":393,"topic_slug":"an-overview-of-the-cluster-mempool-proposal","display_username":"Suhas Daftuar","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":2,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"read":true,"user_title":null,"bookmarked":false,"actions_summary":[{"id":2,"count":1}],"moderator":false,"admin":false,"staff":false,"user_id":95,"hidden":false,"trust_level":3,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/an-overview-of-the-cluster-mempool-proposal/393/10","reactions":[{"id":"+1","type":"emoji","count":1}],"current_user_reaction":null,"reaction_users_count":1,"current_user_used_main_reaction":false,"can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":1447,"name":"Gregory Sanders","username":"instagibbs","avatar_template":"/user_avatar/delvingbitcoin.org/instagibbs/{size}/28_2.png","created_at":"2024-02-02T15:02:39.613Z","cooked":"\u003caside class=\"quote no-group\" data-username=\"sdaftuar\" data-post=\"10\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/letter_avatar_proxy/v4/letter/s/0ea827/48.png\" class=\"avatar\"\u003e sdaftuar:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eone is that we somehow require a peer sending us this transaction to do the work of calculating which transaction we should evict to make room for a new transaction\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eDoesn’t work when the peer cannot “see” the other transactions. You can’t really give hints that will work consistently.\u003c/p\u003e\n\u003caside class=\"quote no-group\" data-username=\"sdaftuar\" data-post=\"10\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/letter_avatar_proxy/v4/letter/s/0ea827/48.png\" class=\"avatar\"\u003e sdaftuar:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eand it only applies in a very limited topology to begin with\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eI don’t buy that premise though either, I was deferring for sake of argument.\u003c/p\u003e\n\u003caside class=\"quote no-group\" data-username=\"sdaftuar\" data-post=\"10\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/letter_avatar_proxy/v4/letter/s/0ea827/48.png\" class=\"avatar\"\u003e sdaftuar:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003ethough it begs the question of how that determination might be made by a wallet in the first place.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eIt would certainly have to be well motivated. In a future package relay world there’s a few, less so in the near future where 1p1c makes most sense.\u003c/p\u003e\n\u003chr\u003e\n\u003cp\u003eedit: last thoughts for now\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003esomething I had thought of and forgot later, but \u003ca href=\"https://delvingbitcoin.org/t/0conf-ln-channels-and-v3-anchors/494/7#future-solution-2\"\u003esibling eviction is less important in a “top block” world\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eWe could decide that later, with “top block”-like solutions that allow relaxed topologies, sibling eviction could be an add-on measure where we do something simple to avoid DoS but perhaps not remove all pins\u003c/li\u003e\n\u003c/ol\u003e","post_number":11,"post_type":1,"posts_count":24,"updated_at":"2024-02-05T14:59:31.171Z","reply_count":0,"reply_to_post_number":10,"quote_count":1,"incoming_link_count":6,"reads":59,"readers_count":58,"score":36.8,"yours":false,"topic_id":393,"topic_slug":"an-overview-of-the-cluster-mempool-proposal","display_username":"Gregory Sanders","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":4,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"link_counts":[{"url":"https://delvingbitcoin.org/t/0conf-ln-channels-and-v3-anchors/494/7","internal":true,"reflection":false,"title":"0conf LN channels and v3 anchors","clicks":0}],"read":true,"user_title":null,"bookmarked":false,"actions_summary":[],"moderator":false,"admin":false,"staff":false,"user_id":31,"hidden":false,"trust_level":3,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/an-overview-of-the-cluster-mempool-proposal/393/11","reactions":[],"current_user_reaction":null,"reaction_users_count":0,"current_user_used_main_reaction":false,"can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":1450,"name":"Anthony Towns","username":"ajtowns","avatar_template":"/user_avatar/delvingbitcoin.org/ajtowns/{size}/417_2.png","created_at":"2024-02-03T11:29:09.208Z","cooked":"\u003caside class=\"quote no-group\" data-username=\"sdaftuar\" data-post=\"8\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/letter_avatar_proxy/v4/letter/s/0ea827/48.png\" class=\"avatar\"\u003e sdaftuar:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eImagine tx C arrives, and we consider eviction of tx B to make room for it. Even if “accept tx C and evict tx B (along with B’s descendants)” is a valid replacement under the feerate diagram check, it’s possible that immediately re-accepting tx B would be successful, because without tx D it might be under the cluster size limits.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eI’m not sure I see this as bad? The mempool would be going from:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e[E] [A]  (separate clusters)\u003c/li\u003e\n\u003cli\u003e[E, B] [A]\u003c/li\u003e\n\u003cli\u003e[E, B, D] [A]\u003c/li\u003e\n\u003cli\u003e[E, A, C]\u003c/li\u003e\n\u003cli\u003e[E, A, C, B]\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThose are all fairly different states, and each one is strictly improving the feerate diagram. That seems like a good thing!\u003c/p\u003e\n\u003cp\u003eThe only drawback I can see is that “B” would be potentially be being relayed and validated twice (which is wasteful, and you might be able to convert that into a DoS attack of some sort if you can cheaply construct lots of C’s and D’s?).\u003c/p\u003e\n\u003cp\u003eIf there was some reasonable way of jumping directly from “[EBD] [A]” to “[EACB]” without the intervening step where B is evicted from the mempool, that would seem perfect. If we already have a linearisation of the stuff we’re evicting (ie “[…BD]”) could we just try adding those txs back in in order immediately?\u003c/p\u003e","post_number":12,"post_type":1,"posts_count":24,"updated_at":"2024-02-03T11:29:09.208Z","reply_count":1,"reply_to_post_number":8,"quote_count":1,"incoming_link_count":2,"reads":61,"readers_count":60,"score":42.2,"yours":false,"topic_id":393,"topic_slug":"an-overview-of-the-cluster-mempool-proposal","display_username":"Anthony Towns","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":1,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"read":true,"user_title":null,"bookmarked":false,"actions_summary":[{"id":2,"count":1}],"moderator":true,"admin":true,"staff":true,"user_id":3,"hidden":false,"trust_level":4,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/an-overview-of-the-cluster-mempool-proposal/393/12","reactions":[{"id":"+1","type":"emoji","count":1}],"current_user_reaction":null,"reaction_users_count":1,"current_user_used_main_reaction":false,"can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":1461,"name":"Gregory Sanders","username":"instagibbs","avatar_template":"/user_avatar/delvingbitcoin.org/instagibbs/{size}/28_2.png","created_at":"2024-02-05T15:08:32.829Z","cooked":"\u003caside class=\"quote no-group\" data-username=\"ajtowns\" data-post=\"12\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/user_avatar/delvingbitcoin.org/ajtowns/48/3_2.png\" class=\"avatar\"\u003e ajtowns:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003ecould we just try adding those txs back in in order immediately?\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eAs a post-processing kind of step, the replacing chunk would have had to pay total/incremental for evicting the things being re-added, so this seems relatively conservative.\u003c/p\u003e\n\u003cp\u003eEven better is if we could cheaply compute a small subset of items(ordered by increasing CFR?) that we could remove to make the new chunk fit. This would allow the evict-er to pay less in fees, but also might stray into computationally dubious territory.\u003c/p\u003e","post_number":13,"post_type":1,"posts_count":24,"updated_at":"2024-02-05T15:08:32.829Z","reply_count":1,"reply_to_post_number":12,"quote_count":1,"incoming_link_count":4,"reads":60,"readers_count":59,"score":37.0,"yours":false,"topic_id":393,"topic_slug":"an-overview-of-the-cluster-mempool-proposal","display_username":"Gregory Sanders","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":1,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"read":true,"user_title":null,"bookmarked":false,"actions_summary":[],"moderator":false,"admin":false,"staff":false,"user_id":31,"hidden":false,"trust_level":3,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/an-overview-of-the-cluster-mempool-proposal/393/13","reactions":[],"current_user_reaction":null,"reaction_users_count":0,"current_user_used_main_reaction":false,"can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":1485,"name":"Gregory Sanders","username":"instagibbs","avatar_template":"/user_avatar/delvingbitcoin.org/instagibbs/{size}/28_2.png","created_at":"2024-02-08T14:14:45.277Z","cooked":"\u003cp\u003eSomething like this might be general enough to be useful(Uncle eviction?):\u003c/p\u003e\n\u003cp\u003eIf proposed chunk is too large for cluster by (remaining_size, remaining_count):\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003eadd all existing chunks to minheap by CFR\u003c/li\u003e\n\u003cli\u003eWhile heap not empty, pop chunk:\nFor each transaction in chunk(in reverse topo order):\u003cbr\u003e\nif in proposed chunks’ ancestor set: skip\u003cbr\u003e\nelse mark for eviction and reduce remaining_size by tx size and remaining_count by 1\nIf remaining_size \u0026lt;= 0 and remaining_count \u0026lt;= 0: break\u003c/li\u003e\n\u003cli\u003eIf anything was marked for eviction: Do diagram check (don’t check remaining_*, maybe we split clusters accidentally?)\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eYou could also restrict the space of transactions to evict to something more “local”\nlike the descendants of the proposed chunk’s ancestors’(not including the ancestor set itself).\nThis should allow eviction in every case still if the new chunk’s fees are high enough, while acting a bit more “locally”.\u003c/p\u003e\n\u003cp\u003eThese should allow improvements to the mempool with O(nlogn) processing, and no resubmission of transactions? Doesn’t solve incremental relay pins of course.\u003c/p\u003e","post_number":14,"post_type":1,"posts_count":24,"updated_at":"2024-02-08T14:15:36.144Z","reply_count":1,"reply_to_post_number":13,"quote_count":0,"incoming_link_count":5,"reads":56,"readers_count":55,"score":41.2,"yours":false,"topic_id":393,"topic_slug":"an-overview-of-the-cluster-mempool-proposal","display_username":"Gregory Sanders","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":1,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"link_counts":[{"url":"https://delvingbitcoin.org/t/v3-and-some-possible-futures/523/3","internal":true,"reflection":true,"title":"V3 and some possible futures","clicks":0}],"read":true,"user_title":null,"reply_to_user":{"id":31,"username":"instagibbs","name":"Gregory Sanders","avatar_template":"/user_avatar/delvingbitcoin.org/instagibbs/{size}/28_2.png"},"bookmarked":false,"actions_summary":[],"moderator":false,"admin":false,"staff":false,"user_id":31,"hidden":false,"trust_level":3,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/an-overview-of-the-cluster-mempool-proposal/393/14","reactions":[],"current_user_reaction":null,"reaction_users_count":0,"current_user_used_main_reaction":false,"can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":1489,"name":"Suhas Daftuar","username":"sdaftuar","avatar_template":"/letter_avatar_proxy/v4/letter/s/0ea827/{size}.png","created_at":"2024-02-08T14:48:06.857Z","cooked":"\u003cp\u003eNothing strikes me as particularly problematic with the algorithm you describe – I think there are a bunch of bounded search algorithms we could consider which look for a subset that is strictly better than what we started with – but I think if we accept that (a) we don’t currently have a framework for deciding what an optimal subset of transactions from a too-big-cluster is, and (b) that therefore any approach that tries to find something better than what we have is going to be guesswork for now, then I think it would make sense to motivate the logic with a use case that would need it.\u003c/p\u003e\n\u003cp\u003eI’d hope that with more research, at some point in the future we are able to come up with a rigorous notion of what an optimal subset of transactions from a too-big-cluster is, and at that point it would be a shame if we were held up in deploying it due to reliance that has developed on existing behavior.  (Of course, if there are use cases that would concretely benefit today from our guesswork solutions, then that would be a reason in favor of trying something like this.)\u003c/p\u003e\n\u003cp\u003eEDIT: regarding this:\u003c/p\u003e\n\u003caside class=\"quote no-group\" data-username=\"instagibbs\" data-post=\"14\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/user_avatar/delvingbitcoin.org/instagibbs/48/28_2.png\" class=\"avatar\"\u003e instagibbs:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThese should allow improvements to the mempool with O(nlogn) processing, and no resubmission of transactions? Doesn’t solve incremental relay pins of course.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eYou could have a resubmission of transactions that works, if I understand the algorithm right. Consider this picture:\u003c/p\u003e\n\u003cpre data-code-height=\"146\" data-code-wrap=\"mermaid\"\u003e\u003ccode class=\"lang-mermaid\"\u003e    flowchart\n         A[\"A: 10kvb, 1 sat/vb\"] --\u0026gt; B:[\"B: 90 kvb, 50 sat/vb\"]\n         A--\u0026gt;C:[\"C: 1kvb, 5 sat/vb\"]\n         A-.-\u0026gt;D:[\"D: 5kvb, 1M sat/vb\"]\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eSo A, B, C are in the mempool and D arrives.  ABC is at the cluster size limit, and the cluster should chunk as [AB, C]. Running your algorithm, we mark C for eviction, and then mark B for eviction – at which point we’ve freed enough space.  We compare [AD] to [AB, C] and assuming I made D’s feerate high enough, it gets in.  But now there is room for C to be re-added as well.\u003c/p\u003e","post_number":15,"post_type":1,"posts_count":24,"updated_at":"2024-02-08T15:00:36.660Z","reply_count":1,"reply_to_post_number":14,"quote_count":1,"incoming_link_count":1,"reads":58,"readers_count":57,"score":36.6,"yours":false,"topic_id":393,"topic_slug":"an-overview-of-the-cluster-mempool-proposal","display_username":"Suhas Daftuar","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":2,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"read":true,"user_title":null,"bookmarked":false,"actions_summary":[{"id":2,"count":1}],"moderator":false,"admin":false,"staff":false,"user_id":95,"hidden":false,"trust_level":3,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/an-overview-of-the-cluster-mempool-proposal/393/15","reactions":[{"id":"+1","type":"emoji","count":1}],"current_user_reaction":null,"reaction_users_count":1,"current_user_used_main_reaction":false,"can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":1490,"name":"Gregory Sanders","username":"instagibbs","avatar_template":"/user_avatar/delvingbitcoin.org/instagibbs/{size}/28_2.png","created_at":"2024-02-08T15:23:54.319Z","cooked":"\u003caside class=\"quote no-group\" data-username=\"sdaftuar\" data-post=\"15\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/letter_avatar_proxy/v4/letter/s/0ea827/48.png\" class=\"avatar\"\u003e sdaftuar:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eSo A, B, C are in the mempool and D arrives. ABC is at the cluster size limit, and the cluster should chunk as [AB, C]. Running your algorithm, we mark C for eviction, and then mark B for eviction – at which point we’ve freed enough space. We compare [AD] to [AB, C] and assuming I made D’s feerate high enough, it gets in. But now there is room for C to be re-added as well.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eYou’re of course right, since we’re not optimally pruning these effects can still happen. So you might want to have the evicting party pay for it, but then let it back in via re-submission. Or just be more optimal.\u003c/p\u003e\n\u003caside class=\"quote no-group\" data-username=\"sdaftuar\" data-post=\"15\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/letter_avatar_proxy/v4/letter/s/0ea827/48.png\" class=\"avatar\"\u003e sdaftuar:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eit would be a shame if we were held up in deploying it due to reliance that has developed on existing behavior.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eShower-thought level quality aside, the goal is to give wallets, who maybe don’t even see the cluster limit being hit, a way to improve the mempool and get their transactions confirmed. If we find an even better way later, we should take it.\u003c/p\u003e\n\u003caside class=\"quote no-group\" data-username=\"sdaftuar\" data-post=\"15\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/letter_avatar_proxy/v4/letter/s/0ea827/48.png\" class=\"avatar\"\u003e sdaftuar:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003ethen I think it would make sense to motivate the logic with a use case that would need it.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eA couple use-cases for motivation in the future, all relying on cluster sizes of \u003ca href=\"https://delvingbitcoin.org/t/v3-and-some-possible-futures/523\"\u003eabove 2\u003c/a\u003e:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e0-conf funding transactions in LN. Funding transaction may have descendants added on, so the anchor spend(or slightly larger replacement) is unable to enter the cluster.\u003c/li\u003e\n\u003cli\u003eArk/Timeout trees. You may want log(n) nodes in the tree published with a single anchor at the end of the chain, but once enough branches are put into the mempool, you’re unable to make attempts at inclusion.\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eAdding anchors at each step “fixes” these, but significant marginal bytes to protect against scenarios no one has seen in the wild is a hard sell.\u003c/p\u003e","post_number":16,"post_type":1,"posts_count":24,"updated_at":"2024-02-08T15:23:54.319Z","reply_count":0,"reply_to_post_number":15,"quote_count":1,"incoming_link_count":5,"reads":55,"readers_count":54,"score":51.0,"yours":false,"topic_id":393,"topic_slug":"an-overview-of-the-cluster-mempool-proposal","display_username":"Gregory Sanders","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":1,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"link_counts":[{"url":"https://delvingbitcoin.org/t/v3-and-some-possible-futures/523","internal":true,"reflection":false,"title":"V3 and some possible futures","clicks":1}],"read":true,"user_title":null,"bookmarked":false,"actions_summary":[{"id":2,"count":1}],"moderator":false,"admin":false,"staff":false,"user_id":31,"hidden":false,"trust_level":3,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/an-overview-of-the-cluster-mempool-proposal/393/16","reactions":[{"id":"+1","type":"emoji","count":1}],"current_user_reaction":null,"reaction_users_count":1,"current_user_used_main_reaction":false,"can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":1598,"name":"Hennadii Stepanov","username":"hebasto","avatar_template":"/user_avatar/delvingbitcoin.org/hebasto/{size}/366_2.png","created_at":"2024-02-18T15:28:06.571Z","cooked":"\u003cp\u003eDo I understand correctly that the cluster size limit might affect the experience of the CoinJoin transaction participants? Considering a transaction with no ancestors, which has the number of outputs that exceeds the cluster size limit, only first \u003ccode\u003emax_cluster_size - 1\u003c/code\u003e participants will be able to spend their outputs, no?\u003c/p\u003e","post_number":17,"post_type":1,"posts_count":24,"updated_at":"2024-02-18T15:28:06.571Z","reply_count":1,"reply_to_post_number":null,"quote_count":0,"incoming_link_count":1,"reads":49,"readers_count":48,"score":19.8,"yours":false,"topic_id":393,"topic_slug":"an-overview-of-the-cluster-mempool-proposal","display_username":"Hennadii Stepanov","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":1,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"read":true,"user_title":null,"bookmarked":false,"actions_summary":[],"moderator":false,"admin":false,"staff":false,"user_id":305,"hidden":false,"trust_level":3,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/an-overview-of-the-cluster-mempool-proposal/393/17","reactions":[],"current_user_reaction":null,"reaction_users_count":0,"current_user_used_main_reaction":false,"can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":1599,"name":"Suhas Daftuar","username":"sdaftuar","avatar_template":"/letter_avatar_proxy/v4/letter/s/0ea827/{size}.png","created_at":"2024-02-18T15:40:04.290Z","cooked":"\u003caside class=\"quote no-group\" data-username=\"hebasto\" data-post=\"17\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/user_avatar/delvingbitcoin.org/hebasto/48/366_2.png\" class=\"avatar\"\u003e hebasto:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eConsidering a transaction with no ancestors, which has the number of outputs that exceeds the cluster size limit, only first \u003ccode\u003emax_cluster_size - 1\u003c/code\u003e participants will be able to spend their outputs, no?\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eJust to be clear, only the first \u003ccode\u003emax_cluster_size - 1\u003c/code\u003e participants will be able to spend their outputs in separate transactions \u003cem\u003ebefore the parent transaction is confirmed\u003c/em\u003e, but yes that is correct.\u003c/p\u003e\n\u003cp\u003eNote that today, the descendant count limit (25 transactions by default) already imposes a restriction on unconfirmed spends of outputs from a single transaction.  While we haven’t finalized a proposed cluster size limit yet, my hope is that we’ll be able to support a number that is bigger than 25 (in my draft PR, this number is set to 100 for now).\u003c/p\u003e","post_number":18,"post_type":1,"posts_count":24,"updated_at":"2024-02-18T15:43:27.168Z","reply_count":0,"reply_to_post_number":17,"quote_count":1,"incoming_link_count":2,"reads":55,"readers_count":54,"score":66.0,"yours":false,"topic_id":393,"topic_slug":"an-overview-of-the-cluster-mempool-proposal","display_username":"Suhas Daftuar","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":3,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"read":true,"user_title":null,"bookmarked":false,"actions_summary":[{"id":2,"count":3}],"moderator":false,"admin":false,"staff":false,"user_id":95,"hidden":false,"trust_level":3,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/an-overview-of-the-cluster-mempool-proposal/393/18","reactions":[{"id":"+1","type":"emoji","count":3}],"current_user_reaction":null,"reaction_users_count":3,"current_user_used_main_reaction":false,"can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":1652,"name":"Hennadii Stepanov","username":"hebasto","avatar_template":"/user_avatar/delvingbitcoin.org/hebasto/{size}/366_2.png","created_at":"2024-02-21T12:31:28.329Z","cooked":"\u003caside class=\"quote no-group\" data-username=\"sdaftuar\" data-post=\"1\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/letter_avatar_proxy/v4/letter/s/0ea827/48.png\" class=\"avatar\"\u003e sdaftuar:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThis graph shows why various heuristics for whether a replacement should be taken are unsatisfactory. Instead, to ensure that a replacement is always better for miners, we propose making the RBF validation rule be that the mempool must get “strictly better” using this feerate diagram check in order to be accepted.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003col\u003e\n\u003cli\u003eWithout knowing a mempool ordering, is it possible for a wallet to construct an RBF transaction candidate that will be guaranteed to be accepted?\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eif not,\u003c/p\u003e\n\u003col start=\"2\"\u003e\n\u003cli\u003eWhat kind of a mempool ordering data would be sufficient for a wallet to achieve such a goal?\u003c/li\u003e\n\u003c/ol\u003e","post_number":19,"post_type":1,"posts_count":24,"updated_at":"2024-02-21T12:31:28.329Z","reply_count":1,"reply_to_post_number":null,"quote_count":1,"incoming_link_count":4,"reads":48,"readers_count":47,"score":34.6,"yours":false,"topic_id":393,"topic_slug":"an-overview-of-the-cluster-mempool-proposal","display_username":"Hennadii Stepanov","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":1,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"read":true,"user_title":null,"bookmarked":false,"actions_summary":[],"moderator":false,"admin":false,"staff":false,"user_id":305,"hidden":false,"trust_level":3,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/an-overview-of-the-cluster-mempool-proposal/393/19","reactions":[],"current_user_reaction":null,"reaction_users_count":0,"current_user_used_main_reaction":false,"can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":1654,"name":"Suhas Daftuar","username":"sdaftuar","avatar_template":"/letter_avatar_proxy/v4/letter/s/0ea827/{size}.png","created_at":"2024-02-21T13:59:55.418Z","cooked":"\u003caside class=\"quote no-group\" data-username=\"hebasto\" data-post=\"19\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/user_avatar/delvingbitcoin.org/hebasto/48/366_2.png\" class=\"avatar\"\u003e hebasto:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003col\u003e\n\u003cli\u003eWithout knowing a mempool ordering, is it possible for a wallet to construct an RBF transaction candidate that will be guaranteed to be accepted?\u003c/li\u003e\n\u003c/ol\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eI think the short answer is “no” – philosophically at least, what I’m proposing is that our RBF rules become even more of a black box than they are today.\u003c/p\u003e\n\u003cp\u003eWhen we first implemented RBF in Bitcoin Core (back in 2015), I had suggested that we write up our replacement rules in a BIP (BIP 125) so that wallets would be able to know what the rules are and be able to comply with them.  However, because those rules depend on the feerates and total fees of all conflicting transactions (including descendants of direct conflicts), wallets already need access to a full node’s mempool in order to know how to replace a given transaction.\u003c/p\u003e\n\u003cp\u003eBecause of this existing complexity, my understanding is that most (all?) wallets today, in practice, just try to keep bumping the fee on a transaction until it is accepted by a full node’s mempool (if access to one is available) or until it confirms.  I’d be curious to know if there are wallets in use that attempt to calculate the BIP 125 requirements for themselves though, and if so, how they achieve it.\u003c/p\u003e\n\u003caside class=\"quote no-group\" data-username=\"hebasto\" data-post=\"19\" data-topic=\"393\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://delvingbitcoin.org/user_avatar/delvingbitcoin.org/hebasto/48/366_2.png\" class=\"avatar\"\u003e hebasto:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003col start=\"2\"\u003e\n\u003cli\u003eWhat kind of a mempool ordering data would be sufficient for a wallet to achieve such a goal?\u003c/li\u003e\n\u003c/ol\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eLet’s say we’re constructing a new transaction \u003cspan class=\"math\"\u003eT\u003c/span\u003e, which would directly conflict with some existing transactions \u003cspan class=\"math\"\u003e{E_1, ..., E_N}\u003c/span\u003e. Then in theory, in order to calculate exactly whether T would be a successful replacement, we’d need to know:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003ethe existing orderings of the clusters that each \u003cspan class=\"math\"\u003eE_i\u003c/span\u003e is currently in.\u003c/li\u003e\n\u003cli\u003ethe orderings of the clusters that would be affected by adding \u003cspan class=\"math\"\u003eT\u003c/span\u003e and removing each \u003cspan class=\"math\"\u003eE_i\u003c/span\u003e from the mempool\u003c/li\u003e\n\u003cli\u003ethe total fees of \u003cspan class=\"math\"\u003eT\u003c/span\u003e, and the total fees of the transactions being removed from the mempool.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eIn practice, though, the rules line up largely with what I think users expect.  You can take a look at my draft PR (\u003ca href=\"https://github.com/bitcoin/bitcoin/pull/28676\"\u003ehttps://github.com/bitcoin/bitcoin/pull/28676\u003c/a\u003e) and look at the changes that needed to be made to the functional tests – basically no substantive changes were made to the tests in \u003ccode\u003efeature_rbf.py\u003c/code\u003e.  So I think that is indicative that the rules aren’t really changing in ways that are unexpected for real-world use cases.\u003c/p\u003e","post_number":20,"post_type":1,"posts_count":24,"updated_at":"2024-02-21T13:59:55.418Z","reply_count":0,"reply_to_post_number":19,"quote_count":1,"incoming_link_count":2,"reads":58,"readers_count":57,"score":51.6,"yours":false,"topic_id":393,"topic_slug":"an-overview-of-the-cluster-mempool-proposal","display_username":"Suhas Daftuar","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":1,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"link_counts":[{"url":"https://github.com/bitcoin/bitcoin/pull/28676","internal":false,"reflection":false,"title":"[WIP] Cluster mempool implementation by sdaftuar · Pull Request #28676 · bitcoin/bitcoin · GitHub","clicks":6}],"read":true,"user_title":null,"bookmarked":false,"actions_summary":[{"id":2,"count":2}],"moderator":false,"admin":false,"staff":false,"user_id":95,"hidden":false,"trust_level":3,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/an-overview-of-the-cluster-mempool-proposal/393/20","reactions":[{"id":"+1","type":"emoji","count":2}],"current_user_reaction":null,"reaction_users_count":2,"current_user_used_main_reaction":false,"can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null}],"stream":[1046,1087,1089,1090,1095,1096,1097,1429,1441,1442,1447,1450,1461,1485,1489,1490,1598,1599,1652,1654,1664,1668,4985,4986]},"timeline_lookup":[[1,1047],[2,1045],[3,1044],[8,1023],[12,1022],[13,1020],[14,1017],[17,1007],[19,1004],[21,1003],[23,558]],"tags":[{"id":11,"name":"mempool","slug":"mempool"}],"tags_descriptions":{},"fancy_title":"An overview of the cluster mempool proposal","id":393,"title":"An overview of the cluster mempool proposal","posts_count":24,"created_at":"2024-01-09T17:45:37.111Z","views":5779,"reply_count":17,"like_count":30,"last_posted_at":"2025-05-12T14:31:40.551Z","visible":true,"closed":false,"archived":false,"has_summary":false,"archetype":"regular","slug":"an-overview-of-the-cluster-mempool-proposal","category_id":8,"word_count":10023,"deleted_at":null,"user_id":95,"featured_link":null,"pinned_globally":false,"pinned_at":null,"pinned_until":null,"image_url":"https://delvingbitcoin.org/uploads/default/original/1X/1c2a247ebb45bc3a930e83d0e900d19c741e8ca6.png","slow_mode_seconds":0,"draft":null,"draft_key":"topic_393","draft_sequence":null,"unpinned":null,"pinned":false,"current_post_number":1,"highest_post_number":24,"deleted_by":null,"actions_summary":[{"id":4,"count":0,"hidden":false,"can_act":false},{"id":8,"count":0,"hidden":false,"can_act":false},{"id":10,"count":0,"hidden":false,"can_act":false},{"id":7,"count":0,"hidden":false,"can_act":false}],"chunk_size":20,"bookmarked":false,"topic_timer":null,"message_bus_last_id":0,"participant_count":7,"show_read_indicator":false,"thumbnails":[{"max_width":null,"max_height":null,"width":1014,"height":630,"url":"https://delvingbitcoin.org/uploads/default/original/1X/1c2a247ebb45bc3a930e83d0e900d19c741e8ca6.png"}],"slow_mode_enabled_until":null,"board_memberships":[],"valid_reactions":["+1","heart","laughing","open_mouth","eyes","thinking","clap","tada","rocket","hugs","yawning_face","-1","sob"],"has_accepted_answer":false,"can_create_shared_issue":false,"shared_issue_visible":false,"can_vote":false,"vote_count":0,"user_voted":false,"discourse_zendesk_plugin_zendesk_id":null,"discourse_zendesk_plugin_zendesk_url":"https://your-url.zendesk.com/agent/tickets/","details":{"can_edit":false,"notification_level":1,"participants":[{"id":95,"username":"sdaftuar","name":"Suhas Daftuar","avatar_template":"/letter_avatar_proxy/v4/letter/s/0ea827/{size}.png","post_count":9,"primary_group_name":null,"flair_name":null,"flair_url":null,"flair_color":null,"flair_bg_color":null,"flair_group_id":null,"trust_level":3},{"id":31,"username":"instagibbs","name":"Gregory Sanders","avatar_template":"/user_avatar/delvingbitcoin.org/instagibbs/{size}/28_2.png","post_count":7,"primary_group_name":null,"flair_name":null,"flair_url":null,"flair_color":null,"flair_bg_color":null,"flair_group_id":null,"trust_level":3},{"id":305,"username":"hebasto","name":"Hennadii Stepanov","avatar_template":"/user_avatar/delvingbitcoin.org/hebasto/{size}/366_2.png","post_count":3,"primary_group_name":null,"flair_name":null,"flair_url":null,"flair_color":null,"flair_bg_color":null,"flair_group_id":null,"trust_level":3},{"id":6,"username":"harding","name":"David A. Harding","avatar_template":"/user_avatar/delvingbitcoin.org/harding/{size}/925_2.png","post_count":2,"primary_group_name":null,"flair_name":null,"flair_url":null,"flair_color":null,"flair_bg_color":null,"flair_group_id":null,"trust_level":3},{"id":3,"username":"ajtowns","name":"Anthony Towns","avatar_template":"/user_avatar/delvingbitcoin.org/ajtowns/{size}/417_2.png","post_count":1,"primary_group_name":null,"flair_name":null,"flair_url":null,"flair_color":null,"flair_bg_color":null,"flair_group_id":null,"admin":true,"moderator":true,"trust_level":4},{"id":96,"username":"sipa","name":"Pieter Wuille","avatar_template":"/user_avatar/delvingbitcoin.org/sipa/{size}/1100_2.png","post_count":1,"primary_group_name":null,"flair_name":null,"flair_url":null,"flair_color":null,"flair_bg_color":null,"flair_group_id":null,"trust_level":3},{"id":780,"username":"blockchainhao","name":"HowHsu","avatar_template":"/user_avatar/delvingbitcoin.org/blockchainhao/{size}/1246_2.png","post_count":1,"primary_group_name":null,"flair_name":null,"flair_url":null,"flair_color":null,"flair_bg_color":null,"flair_group_id":null,"trust_level":2}],"created_by":{"id":95,"username":"sdaftuar","name":"Suhas Daftuar","avatar_template":"/letter_avatar_proxy/v4/letter/s/0ea827/{size}.png"},"last_poster":{"id":96,"username":"sipa","name":"Pieter Wuille","avatar_template":"/user_avatar/delvingbitcoin.org/sipa/{size}/1100_2.png"},"links":[{"url":"https://github.com/bitcoin/bitcoin/issues/27677","title":"Proposal for a new mempool design · Issue #27677 · bitcoin/bitcoin · GitHub","internal":false,"attachment":false,"reflection":false,"clicks":97,"user_id":95,"domain":"github.com","root_domain":"github.com"},{"url":"https://github.com/bitcoin/bitcoin/pull/28676","title":"[WIP] Cluster mempool implementation by sdaftuar · Pull Request #28676 · bitcoin/bitcoin · GitHub","internal":false,"attachment":false,"reflection":false,"clicks":64,"user_id":95,"domain":"github.com","root_domain":"github.com"},{"url":"https://github.com/bitcoin/bitcoin/issues/27677#eviction-mining-not-opposite","title":"Proposal for a new mempool design · Issue #27677 · bitcoin/bitcoin · GitHub","internal":false,"attachment":false,"reflection":false,"clicks":25,"user_id":95,"domain":"github.com","root_domain":"github.com"},{"url":"https://delvingbitcoin.org/t/post-clustermempool-package-rbf-per-chunk-processing/190","title":"Post-clustermempool package RBF: per-chunk processing","internal":true,"attachment":false,"reflection":false,"clicks":12,"user_id":95,"domain":"delvingbitcoin.org","root_domain":"delvingbitcoin.org"},{"url":"https://github.com/bitcoin/bips/blob/deae64bfd31f6938253c05392aa355bf6d7e7605/bip-0125.mediawiki","title":null,"internal":false,"attachment":false,"reflection":false,"clicks":10,"user_id":95,"domain":"github.com","root_domain":"github.com"},{"url":"https://delvingbitcoin.org/t/cluster-mempool-definitions-theory/202","title":"Cluster mempool definitions \u0026 theory","internal":true,"attachment":false,"reflection":false,"clicks":10,"user_id":95,"domain":"delvingbitcoin.org","root_domain":"delvingbitcoin.org"},{"url":"https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-November/016518.html","title":"[bitcoin-dev] CPFP Carve-Out for Fee-Prediction Issues in Contracting Applications (eg Lightning)","internal":false,"attachment":false,"reflection":false,"clicks":7,"user_id":95,"domain":"lists.linuxfoundation.org","root_domain":"linuxfoundation.org"},{"url":"https://delvingbitcoin.org/t/v3-transaction-policy-for-anti-pinning/340/32","title":"V3 transaction policy for anti-pinning","internal":true,"attachment":false,"reflection":true,"clicks":5,"user_id":31,"domain":"delvingbitcoin.org","root_domain":"delvingbitcoin.org"},{"url":"https://delvingbitcoin.org/t/research-into-the-effects-of-a-cluster-size-limited-mempool-in-2023/794","title":"Research into the effects of a cluster size limited mempool in 2023","internal":true,"attachment":false,"reflection":true,"clicks":2,"user_id":95,"domain":"delvingbitcoin.org","root_domain":"delvingbitcoin.org"},{"url":"https://delvingbitcoin.org/t/mempool-incentive-compatibility/553","title":"Mempool Incentive Compatibility","internal":true,"attachment":false,"reflection":true,"clicks":2,"user_id":95,"domain":"delvingbitcoin.org","root_domain":"delvingbitcoin.org"},{"url":"https://delvingbitcoin.org/t/introduction-to-cluster-linearization/1032","title":"Introduction to cluster linearization","internal":true,"attachment":false,"reflection":true,"clicks":2,"user_id":96,"domain":"delvingbitcoin.org","root_domain":"delvingbitcoin.org"},{"url":"https://delvingbitcoin.org/t/support-for-graphing-data/1180","title":"Support for graphing data","internal":true,"attachment":false,"reflection":true,"clicks":1,"user_id":3,"domain":"delvingbitcoin.org","root_domain":"delvingbitcoin.org"},{"url":"https://delvingbitcoin.org/t/v3-and-some-possible-futures/523","title":"V3 and some possible futures","internal":true,"attachment":false,"reflection":false,"clicks":1,"user_id":31,"domain":"delvingbitcoin.org","root_domain":"delvingbitcoin.org"},{"url":"https://delvingbitcoin.org/t/v3-and-some-possible-futures/523","title":"V3 and some possible futures","internal":true,"attachment":false,"reflection":true,"clicks":1,"user_id":31,"domain":"delvingbitcoin.org","root_domain":"delvingbitcoin.org"}]},"bookmarks":[]}