On the first EFI conversion I did with my original 302, I used one of those things that you plug into the harness to tell the computer that the EGR was closed. That configuration always ran a little lean. I was also running long tube headers and a non-EFI cam, which may have confused the issue.
On my 351W, I'm using the EGR and it no longer runs too lean, but I went to a stock cam designed for EFI and went back to stock exhaust manifolds. The stock heads and GT40 intake I have are designed for EGR. I originally ran my 351W without EGR, but found it ran better and did not effect performance when I had it hooked up.
I say if you have the correct plumbing for it use it, otherwise get some custom programming that turns it off in the ECU. Despite getting a closed signal from the EGR sensor, I believe stock ECU programs still leans the mixture when the EGR would normally open.