Slow HTTP downloads through Cisco ASA 5500

Recently we noticed weird behavior downloading files from certain sites. The transfer would start out fast (around 10 MB/s), then after a couple of seconds it would plummet to around 9 KB/s. It didn’t happen for every file or every site: downloads from S3 buckets were still particularly fast. But some files that I remember being particularly fast were now showing this weird fast/slow/fast/slow behavior, for example the Sun JDK and ISOs from rit.edu that used to saturate our pipe were now getting all cRAzY.

After some poking around I decided to test HTTP versus FTP to see if it could be an application/protocol-level issue. The easiest way to do this was to find a file available via both FTP and HTTP and download it via both protocols. This is where mirrors.rit.edu came in handy. I used cURL to download it and noticed that via HTTP it was much slower than over FTP:

[evan@boba 16:07:03 ~]$ curl -O ftp://mirrors.rit.edu/pub/centos/6/isos/x86_64/CentOS-6.2-x86_64-netinstall.iso
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  227M  100  227M    0     0   9.8M      0  0:00:22  0:00:22 --:--:-- 7816k
[evan@boba 16:07:33 ~]$ rm CentOS-6.2-x86_64-netinstall.iso 
[evan@boba 16:07:39 ~]$ curl -O http://mirrors.rit.edu/centos/6/isos/x86_64/CentOS-6.2-x86_64-netinstall.iso
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  227M  100  227M    0     0  5686k      0  0:00:40  0:00:40 --:--:-- 6269k

22 seconds via FTP at 9.8MB/s average, 40 seconds over HTTP at 5.6 MB/s average (which was one of the better HTTP runs).

This was affecting all machines on our network, and had nothing to do with the per-machine iptables rules (verified by flushing all rules). The only thing I could think of that might affect all machines, but only HTTP and not FTP would be something like packet inspection. Well, turns out that http packet inspection is on by default on the ASA. So I disabled it as described here:

Zeus(config)# conf t
Zeus(config)# policy-map global_policy
Zeus(config-pmap)# class inspection_default
Zeus(config-pmap-c)# no inspect http
Zeus(config-pmap-c)# write mem
Building configuration...

Since then HTTP transfers have been consistently fast.

Macbook Pro locks up with SSD installed.

A few weeks ago I switched from my trusty old HP nc8430 to a Macbook Pro (MC118LL/A) that was left spare when another employee left. I mostly enjoyed using Linux but I was tired of dealing with weird quirks like having X lock up, essentially forcing me to do a hard reboot.

To transition, I copied my documents from Linux to Mac, then turned off the Linux laptop. Surprisingly I found I didn’t need to turn Linux back on at all.
Continue reading “Macbook Pro locks up with SSD installed.”

Amazon EC2 – ext3 mkfs takes 30+ minutes?

I’ve been playing around with Amazon EC2 for a new project I’m working on and so far I’m really impressed. One thing I’ve noticed, however, is that it takes forever to create an ext3 filesystem on a new volume. For example, the below command took over 30 minutes to create the filesystem on a 300 GB volume:

# mke2fs -j -m0 /dev/sdf1
mke2fs 1.40.4 (31-Dec-2007)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
39321600 inodes, 78642183 blocks
0 blocks (0.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
2400 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 38 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.

It took about 30 seconds do do everything up to the writing of the superblocks. Not sure why this takes so long, but it’s happened for every EBS volume I’ve formatted ext3. Annoying. Initially I thought it was hanging, and ended up terminating an instance that wouldn’t shutdown or let me cancel the operation. The terminated instance is still being displayed in the UI with a status of “terminated” and I can’t find any way to remove it from the list.