What are interesting parts of the Bitcoin Core codebase?

Mempool design is an interesting theoretical problem. For instance, Mempool Incentive Compatibility

Moreover, privacy and DoS resistance in general are areas with open research questions. For instance, P2P attacks | Bitcoin Core Onboarding