Slow HTTP downloads through Cisco ASA 5500

Recently we noticed weird behavior downloading files from certain sites. The transfer would start out fast (around 10 MB/s), then after a couple of seconds it would plummet to around 9 KB/s. It didn’t happen for every file or every site: downloads from S3 buckets were still particularly fast. But some files that I remember being particularly fast were now showing this weird fast/slow/fast/slow behavior, for example the Sun JDK and ISOs from rit.edu that used to saturate our pipe were now getting all cRAzY.

After some poking around I decided to test HTTP versus FTP to see if it could be an application/protocol-level issue. The easiest way to do this was to find a file available via both FTP and HTTP and download it via both protocols. This is where mirrors.rit.edu came in handy. I used cURL to download it and noticed that via HTTP it was much slower than over FTP:

[evan@boba 16:07:03 ~]$ curl -O ftp://mirrors.rit.edu/pub/centos/6/isos/x86_64/CentOS-6.2-x86_64-netinstall.iso
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  227M  100  227M    0     0   9.8M      0  0:00:22  0:00:22 --:--:-- 7816k
[evan@boba 16:07:33 ~]$ rm CentOS-6.2-x86_64-netinstall.iso 
[evan@boba 16:07:39 ~]$ curl -O http://mirrors.rit.edu/centos/6/isos/x86_64/CentOS-6.2-x86_64-netinstall.iso
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  227M  100  227M    0     0  5686k      0  0:00:40  0:00:40 --:--:-- 6269k

22 seconds via FTP at 9.8MB/s average, 40 seconds over HTTP at 5.6 MB/s average (which was one of the better HTTP runs).

This was affecting all machines on our network, and had nothing to do with the per-machine iptables rules (verified by flushing all rules). The only thing I could think of that might affect all machines, but only HTTP and not FTP would be something like packet inspection. Well, turns out that http packet inspection is on by default on the ASA. So I disabled it as described here:

Zeus(config)# conf t
Zeus(config)# policy-map global_policy
Zeus(config-pmap)# class inspection_default
Zeus(config-pmap-c)# no inspect http
Zeus(config-pmap-c)# write mem
Building configuration...

Since then HTTP transfers have been consistently fast.

Exchange 2010 – Out-of-office response (OOF) won't turn off?

Two users reported the same problem this week: they turned on their out-of-office reply while they were out, then came back and turned it off. Except even after they turned it off, the autoreply was still being sent out. I had them log in to OWA and make sure it was off (maybe some weird bug with Outlook not registering the change in the server), which it was in both cases. I Googled hard and fast and couldn’t find anyone with this same problem.

I went in with Powershell and checked their autoreply status via Get-MailboxAutoReplyConfiguration and it appears that it is, in fact, disabled:

[PS] C:\Windows\system32>Get-mailbox -identity username | Get-MailboxAutoReplyConfiguration
RunspaceId : 7ad7e9af-cd57-4572-a4fd-c1e999e4b9a5
AutoReplyState : Disabled
EndTime : 8/12/2010 12:00:00 PM
ExternalAudience : All
ExternalMessage :
InternalMessage :
StartTime : 8/11/2010 12:00:00 PM
MailboxOwnerId : [removed]
Identity : [removed]
IsValid : True

I used Set-MailboxAutoReplyConfiguration to set the messages to “” (empty string) and it’s still sending the user’s autoresponse, from before I blanked it out. My working theory right now is that the out-of-office message was set on both the Exchange 2010 server and the Exchange 2003 server (where the mailboxes were before I migrated them to 2010).

What a fun problem! It’s hard to test whether I’ve fixed it, since each sender only receives the message once, so I have to keep creating new test email addresses to send test messages.

Also, as an aside, why is “out-of-office” abbreviated “OOF” in Microsoft’s docs?

Edit 1: I had one user verify the message was off in OWA and then start Outlook via Start -> Run… “outlook /cleanrules” and this seemed to resolve the issue. Hopefully this isn’t required every time…

Exchange 2010 – Out-of-office response (OOF) won’t turn off?

Two users reported the same problem this week: they turned on their out-of-office reply while they were out, then came back and turned it off. Except even after they turned it off, the autoreply was still being sent out. I had them log in to OWA and make sure it was off (maybe some weird bug with Outlook not registering the change in the server), which it was in both cases. I Googled hard and fast and couldn’t find anyone with this same problem.

I went in with Powershell and checked their autoreply status via Get-MailboxAutoReplyConfiguration and it appears that it is, in fact, disabled:

[PS] C:Windowssystem32>Get-mailbox -identity username | Get-MailboxAutoReplyConfiguration
RunspaceId       : 7ad7e9af-cd57-4572-a4fd-c1e999e4b9a5
AutoReplyState   : Disabled
EndTime          : 8/12/2010 12:00:00 PM
ExternalAudience : All
ExternalMessage  :
InternalMessage  :
StartTime        : 8/11/2010 12:00:00 PM
MailboxOwnerId   :  [removed]
Identity         :  [removed]
IsValid          : True

I used Set-MailboxAutoReplyConfiguration to set the messages to “” (empty string) and it’s still sending the user’s autoresponse, from before I blanked it out. My working theory right now is that the out-of-office message was set on both the Exchange 2010 server and the Exchange 2003 server (where the mailboxes were before I migrated them to 2010).

What a fun problem! It’s hard to test whether I’ve fixed it, since each sender only receives the message once, so I have to keep creating new test email addresses to send test messages.

Also, as an aside, why is “out-of-office” abbreviated “OOF” in Microsoft’s docs?

Edit 1: I had one user verify the message was off in OWA and then start Outlook via Start -> Run… “outlook /cleanrules” and this seemed to resolve the issue. Hopefully this isn’t required every time…