I have an eraly 2008 Mac Pro 3,1 and I'm interested in upgrading its graphic capabilities. It currently has an ATI Radeon HD 2600 XT. I have looked online for compatible graphics cards but I'm confused regarding compatibility. For example, the GTX 770 is said to be compatible for the Mac I have if flashed, but I don't understand how that works. Is any GTX 770 available on the market compatible off the bat? Would I need to flash it? Is it possible to keep the graphics card I curently have and add the other one?
Which version of macOS runs on your MacPro? I believe 10.11 is the last version that the cMP3,1 supports... Anyways, if you plan on using the Mojave Patcher I would recommend getting a GTX680. If you really can't live without a bootscreen I would recommend looking at flashing tutorials for NVIDIA GPUs. There are a few on YouTube (esp. GTX680) and macrumors is also a good source. You could also buy a flashed card from MacVidCards, but they are a bit pricey..
I don't miss the EFI Screen on my MacPro 5.1 with an unflashed RX580.
I’d definitely look at what MacVidCards has to say.
As you’re theoretically stuck on El Capitan & the 3,1 isn’t particularly fast by today’s standards I wouldn’t look at going too high end.
I’d also definitely consider whether you need the boot screen before even thinking about unflashed cards. Without you have no recourse to recovery or FileVault. You can keep the old card in & swap to it in emergency, though personally I wouldn’t leave it in to save sweating the ageing PSU, just keep it in case.
Apple’s own highest offering is the ATI HD 5770, which I have in a couple of old Pros here. I’m also happily running an HD7950 in a 4,1 though I’ve never tried it in the 3.
NVidia cards suffer if you ever need to run Mojave (using unsupported hack) as there are no drivers other than Apple’s,so no CUDA etc.