According to my quick computations, the average coinbase size for all blocks up to the most recent halving is 256 bytes and starting from segwit activation is 260 bytes. The average Merkle tree depth is 8 and 11 respectively. This implies an average segwit-era download cost of 11*32 + 260 = 612 bytes to validate the Merkle proofs for all txs of a given block (.34 seconds on a 14,400 baud modem).
Given the speeds involved and these averages, basing such a decision on worst case seems unreasonable to me. The largest coinbase in the above BTC history is 31,353 bytes, while the largest in the segwit era is just 6,825 bytes.
Its block hash is a sufficient coinbase identifier.
Another way to fix SPV wallets is to require, along with a Merkle-proof of the inclusion of a transaction E, a Merkle proof for the coinbase transaction. Because building a dual transaction-node for the coinbase transaction requires brute-forcing 225 bits, showing a valid coinbase and its corresponding Merkle inclusion proof is enough to discover the tree height. Both the E branch and the coinbase branch should have equal tree depths. - Leaf-Node weakness in Bitcoin Merkle Tree Design