If money is no object, an enterprise-grade PCIe SSD via Thunderbolt for >3.5GB/s sequential read or write, and over 1 million IOPS* (IO ops / sec).
Most likely a recent Intel Optane SSD using 3D XPoint memory, not flash, although there are fast enterprise SSDs from other vendors. Numbers in the title are a rough guesstimate of what you might get from an Optane SSD DC P5800X PCIe4.0 drive bottlenecked by PCIe3.0. Or some flash SSD from another vendor that can push up to the limits of PCIe3.0 x4. Or even a bunch of DRAM in a PCIe card, if they still make those.
You said "fastest", not "fastest that might be worth buying for consumer/personal use", so that's the point of this answer: ways in which SSDs can be fast, and how to spend thousands of dollars on a terabyte or more of SSD space that could keep up with many cores on a big Xeon for many workloads.
That machine has a Thunderbolt 3 port, so that gives you a PCIe3.0 x4 connection to work with, much more bandwidth and lower latency / less overhead than USB3.1 or even 3.2, even with UASP (USB-attached-SCSI that bypasses some of the USB protocol overhead and limitations). With the right adapter, that should let you connect an M.2 SSD (like modern laptops and desktops use for high-performance SSDs). Or with something designed as an external GPU enclosure, should give you a PCIe x16 card slot (electrically only x4 from Thunderbolt) with cooling fan, letting you use enterprise SSDs with serious heat-sinks designed to go in motherboard PCIe slots. e.g. like this one that came up in a search result.
(Sustained SSD performance typically involves some serious heat in the controller, e.g. 21W for a high-end Intel Optane DC like P5800X, so a heat sink is required to allow continuous operation without having to throttle to avoid overheating. M.2 consumer drives often will need to throttle if you run them hard, but usually that only happens during artificial benchmarks; real use is typically bursty like copying a few tens of GB for a few seconds and then going back to idle.)
The NVMe driver interface standard for computers to talk to SSDs means that you wouldn't (AFAIK) need a special driver to use enterprise-grade SSDs with a Mac. (The very fastest non-volatile storage memory is probably Optane DC PM that comes in DIMMs that you plug into memory slots (like this), for use with Cascade Lake and later server CPU, letting user-space processes truly map the storage into their own address space, bypassing the kernel and letting drivers control access via virtual memory page permissions. It's pretty neat, but you couldn't use it with an iMac).
SSDs that use 3D XPoint instead of Flash can be very fast, especially for small writes and for mixed read+write workloads, so that's what you'd want to go for in this hypothetical scenario. SSDs built around 3D XPoint instead of flash are definitely optimized for speed over capacity and price.
PCIe is backwards compatible: a drive capable of PCIe4.0 plugged into a slot / cable / adapter only capable of PCIe3.0 will negotiate the fastest speed both sides are capable of, so a super high-end drive like an Intel Optane SSD P5800X (2.5" NVMe PCIE4.0 x4) (review) can still run at PCIe3.0 speed.
Thunderbolt 3's PCIe3.0 x4 interface (32.4Gbit/s i.e. ~4GByte/s raw interface bandwidth) will be a bottleneck for sequential transfers, instead of the drive's native capability of up to 7.4GB/s read or write. You might still be able to get close to the 2 million IOPS mixed read+write operations this drive claims to be capable of, for small mixed read+write, though. (up to 1.55 million random 4k read IOPS, up to 1.6 million random 4k write IOPS). Also, the low latency guarantees should fully apply, like <6 us for 99% of small requests, and <66 us for 99.999% of small requests.
(I'm not sure if Thunderbolt adds any meaningful amount of extra latency to PCIe transactions beyond the few nanoseconds of light-speed delay from cable length and a few gate delays from muxing. wikichip shows some details of Thunderbolt 3 in Ice Lake, but doesn't go into detail about if/how Thunderbolt encapsulates PCIe packets.)
That drive comes in U.2 and E1.S form factors. U.2 is like a 2.5" laptop / server drive, but with PCIe connectors instead of SATA, intended to slide into a bay in a storage server. Lets just say that in theory you could connect it to Thunderbolt. See also https://nvmexpress.org/new-pcie-form-factor-enables-greater-pcie-ssd-adoption/ re: form factors.
Something you definitely could use with a Thunderbolt -> PCIe slot enclosure is a DapuStor H3100 which comes in HHHL card slot form factor (Half Height Half Length), PCIe3.0 x4. So the numbers you see on this review are using the same interface speed you could get via Thunderbolt. 3528 MB/s read, 2603 MB/s write, 803 kIOPS random read, 250 kIOPS random write.
This is an eTLC NAND flash device, so obviously it's much slower at random write than the Optane using 3D XPoint. And the latency is worse. MacOS isn't on the supported OS list, but it is an NVMe SSD.
Note that sequential bandwidth isn't the only criterion for storage speed. NVMe has some inherent protocol / driver advantages for low latency and high number of small IO operations per second (IOPS), important if you want to use this storage for a database server for example.
Some storage hardware-review sites do benchmarks that test drives in both low and high "queue depth" situations, i.e. number of outstanding requests in parallel. In a single-user desktop situation, it's common that a program won't do other reads until after the data comes back from the current read. (e.g. loading an executable, it can't run and make open
system calls until it itself is loaded). So desktop responsiveness is often more correlated with low queue depth IOPS and latency, like QD 1 to 4.