1. Its just a suggestion, try to clear as many Unknown's/Down's/and elements that are not collecting data in Solarwinds.
2. Could be that you poller is eating up more CPU, check if the issue is happening due to JobEngine process.
3. Why dont you have a weekly maintenance on the server ? Why can't you reboot the VM once in a week ?