This turned out to be so ridiculously easy I probably shouldn’t even mention it, but only because I remembered that Update Manager exists. I thought I was going to have to burn an ISO, go to the datacenter, pop the disc in, etc. But with Update Manager this is all handled remotely. Just download the files from vmware.com and upload them to your vCenter server through the client. This YouTube video by VMware walks you through the procedure:
ESX to ESXi will have to wait for another time.
Evan Hoffman's Blog 
This video says upgrade esx4.0 to esx4.1 but it is actualy an upgrade from esx3.5 to esx4.1 When trying to upgrade from esx4.0, it says incompatible on the compliance screen. where do you find the upgrade zip for 4.0 to 4.1?
Pretty sure I used the ISO rather than the Zip file, but can’t remember. I think what happens basically is vCenter boots the 3.5/4.0 host off the 4.1 ISO and does the upgrade that way.
The URL for downloading (assuming you’re logged in to vmware.com) is http://www.vmware.com/downloads/download.do?downloadGroup=ESX41
Zip and ISO are downloadble on that page.
Nice video!