How (the hell) do you set up Splunk Cloud on Linux?

This took me way longer than I would’ve thought, mostly due to horrible documentation. Here’s my TL;DR version:

  1. Sign up for Splunk Cloud
  2. Download and install the forwarder binary from here.
  3. Log in here and note the URL of your Splunk instance:

    splunk_cloud
    In the above picture, assume the URL is https://prd-p-jxxxxxxxx.splunk6.splunktrial.com.

  4. Make sure your instances can connect to port tcp/9997 on your input host. Your input host is the hostname from above with “input-” prepended to it. So in our example, the input host is input-prd-p-jxxxxxxxx.splunk6.splunktrial.com. To ensure you can connect, try telnet input-prd-p-jxxxxxxxx.splunk6.splunktrial.com 9997. If it can’t connect you may need to adjust your firewall rules / Security groups to allow outbound tcp/9997

Below are the actual commands I used to get data into our Splunk Cloud trial instance:

$ curl -O http://download.splunk.com/products/splunk/releases/6.2.0/universalforwarder/linux/splunkforwarder-6.2.0-237341-linux-2.6-amd64.deb
$ sudo dpkg -i splunkforwarder-6.2.0-237341-linux-2.6-amd64.deb
$ sudo /opt/splunkforwarder/bin/splunk add forward-server input-prd-p-jxxxxxxxx.splunk6.splunktrial.com:9997
This appears to be your first time running this version of Splunk.
Added forwarding to: input-prd-p-jxxxxxxxx.splunk6.splunktrial.com:9997.
$ sudo /opt/splunkforwarder/bin/splunk add monitor '/var/log/postgresql/*.log'
Added monitor of '/var/log/postgresql/*.log'.
$ sudo /opt/splunkforwarder/bin/splunk list forward-server
Splunk username: admin
Password:
Active forwards:
	input-prd-p-jxxxxxxxx.splunk6.splunktrial.com:9997
Configured but inactive forwards:
	None
$ sudo /opt/splunkforwarder/bin/splunk list monitor
Monitored Directories:
		[No directories monitored.]
Monitored Files:
	/var/log/postgresql/*.log
$ sudo /opt/splunkforwarder/bin/splunk restart
Advertisements

Installing a new SSL certificate in your ELB via CLI

For future me:

  1. Create the key and CSR:
    $ openssl req -out wildcard.site.com.csr -new -newkey rsa:2048 -nodes -keyout wildcard.site.com.key
    
  2. Upload the CSR to your SSL vendor (in this case, DigiCert) and obtain the signed SSL certificate.
  3. Create a PEM-encoded version of the signing key. This is required for AWS/IAM certs. To check if your key is already PEM-encoded, just “head -1 site.key”. If the first line says “—–BEGIN PRIVATE KEY—–” then it’s NOT PEM-encoded. The first line should be “—–BEGIN RSA PRIVATE KEY—–“.
    $ openssl rsa -in wildcard.site.com.key -outform PEM -out wildcard.site.com.pem.key
    writing RSA key
    
  4. Upload the certificate to the IAM keystore:
    $ aws iam upload-server-certificate --server-certificate-name star_site_20141014 --certificate-body file:///Users/evan/certs_20141014/site/certs/star_site_com.crt --private-key file:///Users/evan/certs_20141014/wildcard.site.com.pem.key --certificate-chain file:///Users/evan/certs_20141014/site/certs/DigiCertCA.crt
    {
        "ServerCertificateMetadata": {
            "ServerCertificateId": "XXXXXXXXXXXXXXX",
            "ServerCertificateName": "star_site_20141014",
            "Expiration": "2017-12-18T12:00:00Z",
            "Path": "/",
            "Arn": "arn:aws:iam::9999999999:server-certificate/star_site_20141014",
            "UploadDate": "2014-10-14T15:29:28.164Z"
        }
    }
    

Once the above steps are complete, you can go into the web console (EC2 -> Load Balancers), select the ELB whose cert you want to change, click the “Listeners” tab, click the SSL port (443) and select the new cert from the dropdown.

Can I create an EC2 MySQL slave to an RDS master?

No.

Here’s what happens if you try:

mysql> grant replication slave on *.* to 'ec2-slave'@'%';
ERROR 1045 (28000): Access denied for user 'rds_root'@'%' (using password: YES)
mysql> update mysql.user set Repl_slave_priv='Y' WHERE user='rds_root' AND host='%';
ERROR 1054 (42S22): Unknown column 'ERROR (RDS): REPLICA SLAVE PRIVILEGE CANNOT BE GRANTED OR MAINTAINED' in 'field list'
mysql>

Note: this is for MySQL 5.5, which is unfortunately what I’m currently stuck with.