I have a perplexing issue with a client SQL server. The backstory is their SQL jobs started to incrementally take longer and longer to complete. They say nothing has been done on the SQL server side. This doesn't seem to be host related, as the issue persists from host to host. As part of our troubleshooting we upgraded their hosts with 10GB NIC's to the SAN. The problem persisted. As we continued to troubleshoot, we moved their SQL VM to local storage on the host. They said that after the move the VM began performing flawlessly.
I am not really sure why the VM would be performing better on local disks. They moved from a LUN made up of 24x15K disks in a RAID10 to 2x7.2K disks in a RAID1. I completed a series of benchmark tests using FIO, HDParm, and QPerf. Every test I have ran has the SAN outperforming the local disks (in some cases by a great deal). The only thing I can really think of is latency. Their particular SAN utilizes copper instead of fiber. I can't imagine it would make this significant of an impact thought. Any ideas or thoughts are appreciated. Thank you.