Sunday, October 14, 2012

SSH+Bash


Using ssh and bash together gives you a great opportunity to roll out changes and grab information from remote machines quickly and easily. Here are some syntax uses of bash+ssh that could come in handy:

1) "who" command output in remote host "micky"

[code]
$ ssh root@micky "who"
root pts/0 Jan 7 06:10 (192.168.23.2)
[/code]


2) Diff two files in two different boxes

Diff-ing the /etc/hosts files

$ diff <(ssh -n root@micky cat /etc/hosts) <(ssh -n root@gilbert cat /etc/hosts)

3) Execute a script in remote server, without copying it.

"getipline.sh" is a simple script to extract the IP add of a box. This is how we can execute this script in a reomte box without copying the same.


$ cat getipline.sh | ssh root@micky /bin/sh
addr:192.168.32.8


**Problem: Can't pass command line argument to the script. If you got a script with command line args to be excuted in remote box, I feel you have to copy the same to the remote box.

4) Suppose you want to access(query) the mysql db of remote host "micky". This is how we can achieve this.


$ ssh root@micky << ! > /usr/bin/mysql -u root
> show databases;
> !


Output:

Pseudo-terminal will not be allocated because stdin is not a terminal.
Database
mysql
test
NMS


5) An example BASH script to extract some informations from a remote host using "ssh"

#!/bin/sh
SSH="/usr/bin/ssh"
HOST=$1
USR=${2:-root}
NOARG=65

f_Usage() {
echo "Usage: `basename $0` "
}

[ -z $HOST ] && f_Usage && exit $NOARG

CMD="ssh $USR@$HOST"
rhostname="$($CMD hostname)"
echo "HostName = $rhostname"
ruptime="$($CMD uptime)"
echo "UpTime =$ruptime"
rips="$($CMD /sbin/ifconfig | sed -e '2!d' -e 's/inet//')"
echo $rips | awk '{for (i=1;i<=NF;i++) {print i")",$i}}'


Executing the script:


$ ./fetch.sh 172.22.22.121
HostName = gentle.crp.xyz.com
UpTime = 10:26:05 up 33 days, 18:38, 2 users, load average: 0.00, 0.00, 0.00
1) addr:172.22.22.121
2) Bcast:172.22.0.255
3) Mask:255.255.255.0

Sunday, March 18, 2012

Ubuntu 11.04 + Puppet (NGinX+Unicorn) + MCollective + RabbitMQ

I was reading http://www.uncompiled.com/centos-6-puppet-27-mcollective-foreman-rabbit  and managed to modify the script to work with Ubuntu. As the hosting company i use basically just provides hardware but doesn't allow dhcp servers i don't have a need for foreman.

# Configuration Parameters                                                                                                                                                                       
MYSQL_PASSWORD="puppetized"
RABBIT_USER="mcollective"
RABBIT_PASSWORD=`pwgen -1 32`
MCOLLECTIVE_PSK=`pwgen -1 32`
DOMAIN="domain.local"


# Configure hostname
echo -e "127.0.0.1 puppet.${DOMAIN} puppet localhost" > /etc/hosts
echo -e "puppet.${DOMAIN}" > /etc/hostname
hostname puppet.${DOMAIN}

apt-get update
apt-get -y install gpg


# Puppet Labs repository
echo 'deb http://apt.puppetlabs.com/ubuntu unstable main' > /etc/apt/sources.list.d/puppetlabs-unstable.list
/usr/bin/gpg --keyserver pgp.mit.edu --recv-key '4BD6EC30' && gpg --export --armor '4BD6EC30' | /usr/bin/apt-key add -

echo 'deb http://repo.percona.com/apt oneiric main' > /etc/apt/sources.list.d/perconda.list
/usr/bin/gpg --keyserver hkp://keys.gnupg.net --recv-keys 1C4CBDCDCD2EFD2A && gpg --export --armor 'CD2EFD2A' | /usr/bin/apt-key add -

apt-get update

apt-get -y install nginx-full rubygems ruby-dev
apt-get -y --force-yes install libmysqlclient16 libmysqlclient18 percona-server-client-5.5 percona-server-common-5.5 percona-server-server-5.5 libmysqlclient-dev
apt-get -y install libcurl4-openssl-dev libssl-dev openssl tcl tk unixODBC unixODBC-dev libaugeas-ruby


# Installation of stack gems
gem install --no-rdoc --no-ri rest-client json mime-types stomp unicorn
gem install --no-rdoc --no-ri puppet passenger rack mysql net-ping
gem install --no-rdoc --no-ri -v 3.0.10 rails activerecord

# Deploy required Puppet user, files, and directories
adduser puppet

mkdir -p /etc/puppet/{manifests,modules}
mkdir -p /etc/puppet/{public,tmp}
mkdir -p /var/lib/puppet/{bucket,yaml,rrd,server_data,reports}
chown puppet:puppet /var/lib/puppet/{bucket,yaml,rrd,server_data,reports}

cp /var/lib/gems/1.8/gems/puppet-2.7.12/ext/rack/files/config.ru /etc/puppet/config.ru
chown puppet:puppet /etc/puppet/config.ru

# run puppet from cron randomly
update-rc.d puppet disable

# mCollective & Plugins
apt-get -y --force-yes install mcollective mcollective-common mcollective-client

cd /usr/share/mcollective/plugins/mcollective/application
for i in filemgr nettest package puppetd service; do
wget https://raw.github.com/puppetlabs/mcollective-plugins/master/agent/$i/application/$i.rb
done
wget https://raw.github.com/phobos182/mcollective-plugins/master/agent/etcfacts/application/etcfacts.rb
wget https://raw.github.com/phobos182/mcollective-plugins/master/agent/shellcmd/application/shellcmd.rb
wget https://raw.github.com/phobos182/mcollective-plugins/master/agent/yum/application/yum.rb

cd /usr/share/mcollective/plugins/mcollective/agent
for i in nettest filemgr puppetd puppetral puppetca; do
wget https://raw.github.com/puppetlabs/mcollective-plugins/master/agent/$i/agent/$i.rb
wget https://raw.github.com/puppetlabs/mcollective-plugins/master/agent/$i/agent/$i.ddl
done

wget -O package.rb https://raw.github.com/puppetlabs/mcollective-plugins/master/agent/package/agent/puppet-package.rb
wget https://raw.github.com/puppetlabs/mcollective-plugins/master/agent/package/agent/package.ddl
wget -O service.rb https://raw.github.com/puppetlabs/mcollective-plugins/master/agent/service/agent/puppet-service.rb
wget https://raw.github.com/puppetlabs/mcollective-plugins/master/agent/service/agent/service.ddl
wget https://raw.github.com/phobos182/mcollective-plugins/master/agent/etcfacts/etc_facts.rb
wget https://raw.github.com/phobos182/mcollective-plugins/master/agent/etcfacts/etc_facts.ddl
wget https://raw.github.com/phobos182/mcollective-plugins/master/agent/shellcmd/shellcmd.rb
wget https://raw.github.com/phobos182/mcollective-plugins/master/agent/shellcmd/shellcmd.ddl
wget https://raw.github.com/mstanislav/mCollective-Agents/master/apt/apt.rb
wget https://raw.github.com/mstanislav/mCollective-Agents/master/apt/apt.ddl

cd /usr/share/mcollective/plugins/mcollective/facts/
wget https://raw.github.com/puppetlabs/mcollective-plugins/master/facts/facter/facter_facts.rb

# Install Erlang & RabbitMQ & Plugins
apt-get -y install erlang rabbitmq-stomp rabbitmq-erlang-client

update-rc.d rabbitmq-server defaults

# Configure RabbitMQ user/privileges
rabbitmqctl add_user ${RABBIT_USER} ${RABBIT_PASSWORD}
rabbitmqctl set_permissions ${RABBIT_USER} ".*" ".*" ".*"
rabbitmqctl delete_user guest

cat > /etc/cron.daily/puppetd << "EOF"
#!/bin/bash

test -x /usr/sbin/puppetd || exit 0

# puppet can sometimes die and leave pid file behind clean it up
if [ -f /var/run/puppet/puppetd.pid ]; then
  oldpid=`/bin/cat /var/run/puppet/puppetd.pid`
  if [ ! -e /proc/${oldpid} -a /proc/${oldpid}/exe ]; then
    /bin/rm -f /var/run/puppet/puppetd.pid
  fi
fi

pause=$RANDOM
let 'pause %= 3600'
sleep $pause
/usr/sbin/puppetd -o
EOF

cat > /etc/cron.d/puppetd << "EOF"
#!/bin/sh

MAILTO=root

0 0,6,12,18 * * *     root if [ -x /usr/sbin/puppetd ]; then /etc/cron.daily/puppetd; fi
EOF

# Configuration files for mCollective
cat > /etc/mcollective/server.cfg << "EOF"
topicprefix = /topic/
main_collective = mcollective
collectives = mcollective
libdir = /usr/share/mcollective
logfile = /var/log/mcollective.log
loglevel = info
daemonize = 1

securityprovider = psk
plugin.psk = MCOLLECTIVE_PSK_PH

connector = stomp
plugin.stomp.host = localhost
plugin.stomp.port = 61613
plugin.stomp.user = RABBIT_USER_PH
plugin.stomp.password = RABBIT_PASSWORD_PH

factsource = facter
EOF

cat > /etc/mcollective/client.cfg << "EOF"
topicprefix = /topic/
main_collective = mcollective
collectives = mcollective
libdir = /usr/share/mcollective
logfile = /dev/null
loglevel = info

securityprovider = psk
plugin.psk = MCOLLECTIVE_PSK_PH

connector = stomp
plugin.stomp.host = localhost
plugin.stomp.port = 61613
plugin.stomp.user = RABBIT_USER_PH
plugin.stomp.password = RABBIT_PASSWORD_PH

factsource = facter
EOF

# Configure MySQL
update-rc.d mysqld defaults

mysql -u root -e "CREATE DATABASE puppet;"
mysql -u root -e "GRANT ALL PRIVILEGES ON puppet.* TO puppet@localhost IDENTIFIED BY '${MYSQL_PASSWORD}';"

# Puppet configuration
cat > /etc/puppet/puppet.conf << "EOF"
[main]
logdir = /var/log/puppet
rundir = /var/run/puppet
ssldir = $vardir/ssl
factpath = $vardir/lib/facter
templatedir = $confdir/templates
pluginsync = true
classfile = $vardir/classes.txt
localconfig = $vardir/localconfig
reportdir = /var/lib/puppet/reports

[agent]
report = true
ignorecache = true

[master]
reports = http,store,log
ssl_client_header = HTTP_X_CLIENT_DN
ssl_client_verify_header = HTTP_X_CLIENT_VERIFY
storeconfigs = true
dbadapter = mysql
dbuser = puppet
dbpassword = MYSQL_PASSWORD_PH
dbname = puppet
dbserver = localhost
dbsocket = /var/run/mysqld/mysqld.sock
dns_alt_names = puppet
EOF

cat > /etc/puppet/unicorn.conf << "EOF"
worker_processes 8
working_directory "/etc/puppet"
listen '/var/run/puppet/puppetmaster_unicorn.sock', :backlog => 512
timeout 120
pid "/var/run/puppet/puppetmaster_unicorn.pid"

preload_app true
if GC.respond_to?(:copy_on_write_friendly=)
  GC.copy_on_write_friendly = true
end

before_fork do |server, worker|
  old_pid = "#{server.config[:pid]}.oldbin"
  if File.exists?(old_pid) && server.pid != old_pid
    begin
      Process.kill("QUIT", File.read(old_pid).to_i)
    rescue Errno::ENOENT, Errno::ESRCH
      # someone else did our job for us
    end
  end
end
EOF

cat > /etc/init.d/unicorn-puppet << "EOF"
#!/bin/bash
# unicorn-puppet         This init script enables the puppetmaster rackup application
#                        via unicorn.
#
# Authors:               Richard Crowley 
#                        Naresh V. 
#
#    Modified for Debian usage by Matt Carroll
#    
#

lockfile=/var/lock/puppetmaster-unicorn
pidfile=/var/run/puppet/puppetmaster_unicorn.pid

RETVAL=0
DAEMON=/var/lib/gems/1.8/bin/unicorn
DAEMON_OPTS="-D -c /etc/puppet/unicorn.conf"


start() {
    sudo -u $USER $DAEMON $DAEMON_OPTS
    RETVAL=$?
    [ $RETVAL -eq 0 ] && touch "$lockfile"
    echo
    return $RETVAL
}

stop() {
    sudo -u $USER kill `cat $pidfile`
    RETVAL=$?
    echo
    [ $RETVAL -eq 0 ] && rm -f "$lockfile"
    return $RETVAL
}

restart() {
    stop
    sleep 1
    start
    RETVAL=$?
    echo
    [ $RETVAL -ne 0 ] && rm -f "$lockfile"
    return $RETVAL
}

condrestart() {
    status
    RETVAL=$?
    [ $RETVAL -eq 0 ] && restart
}

status() {
    ps ax | egrep -q "unicorn (worker|master)"
    RETVAL=$?
    return $RETVAL
}


usage() {
    echo "Usage: $0 {start|stop|restart|status|condrestart}" >&2
    return 3
}

case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    restart)
        restart
        ;;
    condrestart)
        condrestart
        ;;
    status)
        status
        ;;
    *)
        usage
        ;;
esac

exit $RETVAL
EOF

chmod 755 /etc/init.d/unicorn-puppet
update-rc.d unicorn-puppet defaults

cat > /etc/nginx/sites-available/puppetmaster << "EOF"
upstream puppetmaster_unicorn {
   server unix:/var/run/puppet/puppetmaster_unicorn.sock fail_timeout=0;
}

server {
  listen 8140;

  ssl on;
  ssl_session_timeout 5m;

  ssl_certificate               /var/lib/puppet/ssl/certs/puppet.DOMAIN_PH.pem;
  ssl_certificate_key           /var/lib/puppet/ssl/private_keys/puppet.DOMAIN_PH.pem;
  ssl_client_certificate        /var/lib/puppet/ssl/ca/ca_crt.pem;

  ssl_ciphers SSLv2:-LOW:-EXPORT:RC4+RSA;
  ssl_verify_client optional;

  root /usr/share/empty;

  proxy_set_header Host $host;
  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header X-Client-Verify $ssl_client_verify;
  proxy_set_header X-Client-DN $ssl_client_s_dn;
  proxy_set_header X-SSL-Issuer $ssl_client_i_dn;
  proxy_read_timeout 120;

  location / {
    proxy_pass http://puppetmaster_unicorn;
    proxy_redirect off;
  }
}
EOF

ln -s  /etc/nginx/sites-available/puppetmaster /etc/nginx/sites-enabled/


# Replace placeholder values for configuration
sed -i "s/MYSQL_PASSWORD_PH/${MYSQL_PASSWORD}/g" /etc/puppet/puppet.conf
sed -i "s/MCOLLECTIVE_PSK_PH/${MCOLLECTIVE_PSK}/g" /etc/mcollective/server.cfg /etc/mcollective/client.cfg
sed -i "s/RABBIT_USER_PH/${RABBIT_USER}/g" /etc/mcollective/server.cfg /etc/mcollective/client.cfg
sed -i "s/RABBIT_PASSWORD_PH/${RABBIT_PASSWORD}/g" /etc/mcollective/server.cfg /etc/mcollective/client.cfg
sed -i "s/DOMAIN_PH/${DOMAIN}/g" /etc/nginx/sites-available/puppetmaster

# Enable mCollective
update-rc.d mcollective defaults
service mcollective restart

# Generate Puppet master CA
puppet cert --generate puppet.${DOMAIN}

# Enable Apache
update-rc.d nginx defaults
service nginx restart

# Execute Puppet agent
puppet agent -t

# Finished
exit


Thursday, March 1, 2012

innobackupex unable to backup to nfs

When trying to run:

TheSlave$ innobackupex --user=yourDBuser --password=MaGiCiGaM / --slave-info /path/to/backupdir 

The script tries to run 'cp -p' on Myisam datafiles which isn't allowed to nfs.

--stream=tar usage is required if you wish to backup to nfs mount points.

Silly security problems in Percona XtraDB Cluster


I'm just playing around setting up a cluster and noticed that rsync is spawned with no type of password authentication or ip limits to allow the sync:

# ps -fe | grep rsync
mysql 20370 1 0 19:29 ? 00:00:00 rsync --daemon --port 4444 --config /mysql/data//rsync_sst.conf
mysql 20393 20370 0 19:29 ? 00:00:00 rsync --daemon --port 4444 --config /mysql/data//rsync_sst.conf
mysql 20394 20393 21 19:29 ? 00:02:26 rsync --daemon --port 4444 --config /mysql/data//rsync_sst.conf
# cat /mysql/data//rsync_sst.conf
pid file = /mysql/data//rsync_sst.pid
use chroot = no
[rsync_sst]
        path = /mysql/data/
        read only = no
        timeout = 300
        uid = 110
        gid = 113
#

On a totally unrelated node:

# rsync -L rsync://somenode:4444/rsync_sst
drwxr-xr-x 4096 2012/02/29 19:29:24 .
-rw-rw---- 6490 2012/02/29 19:30:37 somenode.err
-rw------- 134219048 2012/02/29 19:41:04 galera.cache
-rw-rw---- 115 2012/02/29 19:29:23 grastate.dat
-rw-rw---- 140 2012/02/29 19:29:23 rsync_sst.conf
-rw-rw---- 6 2012/02/29 19:29:23 rsync_sst.pid
-rw-rw---- 59 2012/02/29 19:29:23 sst.err
drwxr-xr-x 4096 2012/02/29 19:29:59 db0005
drwxr-xr-x 20480 2012/02/29 19:41:07 db0004
drwxr-xr-x 4096 2012/02/29 19:29:24 db0003
drwxr-xr-x 4096 2012/02/29 19:29:24 db0002
drwxr-xr-x 4096 2012/02/29 19:29:24 db0001
drwxr-xr-x 4096 2012/02/29 19:29:24 mysql
#

In the rsync conf file shouldn't the server be chrooted and also have it's access restricted to only allow connections from the donor?

Wednesday, February 29, 2012

Welcome

Hello and welcome!

I intend to use this blog as a scratching post for any tasks that I'm working on. This place will be used as a dumping ground for useful posts/doco/pictures that help me complete tasks or notes for gotcha's.