Something to watch out for is the ZPrice competition:
They’re currently running a competition seeking for implementation improvements of ZK proofs and related tech, also using GPUs. See for example GitHub - td-kwj-zp2023/webgpu-msm-bls12-377: WebGPU MSM Implementation (BLS12-377 Curve) for ZPrize 2023 for a submission to the competition that uses WebGPU to speed up multi-scalar multiplication (also known as multi-exponentiation), which is the most important building block in batch verification of ZK proofs, but also in batch verification of signatures. Though they work on a different elliptic curve, their work shows what’s possible: WebGPU Performance - Google Sheets
If I read this correctly, that’s a multi-scalar multiplication of size 2^20 in 1277 milliseconds…
The winners of the competition should be announced soon.
edit: Okay, maybe this example is not all that impressive. That’s 1.2 usec / scalar, and libsecp256k1 can do 4.3 usec / scalar on my laptop CPU (on size 2^15, I haven’t tried 2^20). Anyway, it will be interesting to see the results of the competition…