It's not about the jumpers per say, but improving everything in the chain for a better signal flow. Brass is not the optimal for signal flow. Will it work ? Sure, can you do better ? Yep. Some have indeed changed internal wiring for better quality wire but to me thats a tad over the top. Improvements with changing out jumpers, cables, etc, are small subtle changes on their own, but when you add up all the small changes, you can hear real world differences.
With that said, most who change out those brass jumpers probably won't hear a difference. It's that minimal. Over time and oxidation, the signal will degrade some. So the standard answer applies, if you don't try, you'll never know. Simple as that really.