Another MySQL daemon already running with the same unix socket.

After upgrading MySQL binaries mysqld will not start at all any more and shows below error :

CT-1977-bash-4.1# service mysqld start
Another MySQL daemon already running with the same unix socket.
Starting mysqld: [FAILED]
CT-1977-bash-4.1#

MySQL service does not shut down gracefully during the OS reboot, leaving the old /var/lib/mysql/mysql.sock such that mysqld will not start up. Some people were able to reproduce this error on a CentOS 6.5 KVM guest virtual system by rebooting the host CentOS 6.5 system. CentOS is supposed to gracefully shut down the guest systems, but this seems to be failing in the case of mysqld.

Confirmed Red Hat Linux 6.5 bug – https://bugzilla.redhat.com/show_bug.cgi?id=1037650

Issue discussion on MySQL bug tracker – http://bugs.mysql.com/bug.php?id=71086

Simple steps to reproduce this issue:

service mysqld start
killall -9 mysqld_safe mysqld
service mysqld start

A quick way to restart MySQL is as below :
Remove socket file and restart mysql:
CT-1977-bash-4.1# ls -la /var/lib/mysql/mysql.sock
srwxrwxrwx 1 mysql mysql 0 Jan 8 20:13 /var/lib/mysql/mysql.sock
CT-1977-bash-4.1# rm /var/lib/mysql/mysql.sock
CT-1977-bash-4.1# /etc/init.d/mysqld restart
Stopping mysqld: [ OK ]
Starting mysqld: [ OK ]
CT-1977-bash-4.1#

A work around for the issue is to modify ‘/etc/init.d/mysqld’ script:

Make a backup copy of the startup script.
cp -p /etc/init.d/mysqld /etc/init.d/mysqld.orig

Edit the file /etc/init.d/mysqld to look as below :
=========================
# if fuser "$socketfile" &>/dev/null ; then
# echo "Socket file $socketfile exists. Is another MySQL daemon already running with the same unix socket?"
# action $"Starting $prog: " /bin/false
# return 1

# We check if there is already a process using the socket file,
# since otherwise this init script could report false positive
# result and mysqld_safe would remove the socket file, which
# actually uses a different daemon.
if fuser "$socketfile" &>/dev/null ; then
echo "Socket file $socketfile exists. Is another MySQL daemon already running with the same unix socket?"
action $"Starting $prog: " /bin/false
return 1
fi
=========================

Now restart MySQL :
CT-1977-bash-4.1# service mysqld start
Starting mysqld: [ OK ]
CT-1977-bash-4.1#

How to verify matching private key with a certificate

The private key contains a series of numbers. Two of those numbers form the “public key”, the others are part of your “private key”. The “public key” bits are also embedded in your Certificate (we get them from your CSR). To check that the public key in your cert matches the public portion of your private key, you need to view the cert and the key and compare the numbers. To view the Certificate and the key run the commands:

$ openssl x509 -noout -text -in server.crt
$ openssl rsa -noout -text -in server.key

The `modulus’ and the `public exponent’ portions in the key and the Certificate must match. But since the public exponent is usually 65537 and it’s bothering comparing long modulus you can use the following approach:

$ openssl x509 -noout -modulus -in server.crt | openssl md5
$ openssl rsa -noout -modulus -in server.key | openssl md5

And then compare these really shorter numbers. With overwhelming probability they will differ if the keys are different. As a “one-liner”:

$ openssl x509 -noout -modulus -in server.pem | openssl md5 ;\
openssl rsa -noout -modulus -in server.key | openssl md5

And with auto-magic comparison (If more than one hash is displayed, they don’t match):

$ (openssl x509 -noout -modulus -in server.pem | openssl md5 ;\
openssl rsa -noout -modulus -in server.key | openssl md5) | uniq

BTW, if I want to check to which key or certificate a particular CSR belongs you can compute

$ openssl req -noout -modulus -in server.csr | openssl md5

How to Disable Telnet access on server

Telnet should be disabled on all web servers, and you should use SSH which is more secure.

Telnet sends password in plain text passwords and usernames through logins, and

‘crackers/hackers’ can obtain these passwords easily as compared to SSH.

TELNET server listens for incoming messages on port 23, and sends outgoing messages to port 23.

1. Login to your server through SSH and su to root.

2. Type pico /etc/xinetd.d/telnet

3. Look for the line: disable = no and replace with disable = yes

4. Now restart the inetd service: /etc/rc.d/init.d/xinetd restart

5. Turn off it through chkconfig as well because it can still start through that.
/sbin/chkconfig telnet off

6. Scan your server to ensure port 23 is closed.
nmap -sT -O localhost
Also run ps -aux | grep telnet and if you find anything other than “grep telnet” as result kill the process.

🙂

OpenSSL Tricks

Create a strong CSR and private key
openssl req -new -nodes -newkey rsa:2048 -out server.crt -keyout server.key

 

Parsing out the data within a certificate
openssl asn1parse -in server.crt

Checking a certificate/key modulus to see if they correspond
openssl rsa -in server.key -modulus -noout | openssl md5
openssl x509 -in server.crt -modulus -noout | openssl md5

Convert a key from PEM -> DER
openssl rsa -inform PEM -in key.pem -outform DER -out keyout.der

Convert a key from DER -> PEM
openssl rsa -inform DER -in key.der -outform PEM -out keyout.pem

Remove the password from an encrypted private key
openssl rsa -in server.key -out server-nopass.key

Reviewing a detailed SSL connection
openssl s_client -connect 192.168.1.1:443

Rebuilding the initial ram disk (initrd)

Installing new hardware may mean that new kernel need to be loaded when your server boots up. There’s a two step process to making a new initrd file:

 

First, add the appropriate line to your /etc/modules.conf or /etc/modprobe.conf which corresponds to your new kernel module.

Next, rebuild the initial ram disk after making a backup of the current one:

# cp /boot/initrd-`uname -r`.img /boot/initrd-`uname -r`.img.bak
# mkinitrd -f initrd-`uname -r`.img `uname -r`

Reboot the server now and make sure the new driver is loaded properly.

Register to Red Hat Network

rhn_register that will register your server over at RHN.

 

Well if you want to upgrade your kernel… you’ll have to edit the config to NOT skip the kernel

up2date –configure
type 24
and then type C and hit enter
and then hit enter again

Type:
up2date -p
that will update the packages your server to reflect the packages available to update over at RH.

Then type
up2date -u
and it will install any updates that are available

But remember you’ll have to setup an account over at Red Hat Network — if you haven’t already, type rhn_register, and that will register your server over at RHN.

And that’s it!

Adding IP aliases in FreeBSD

One question I hear quite often is “how do I add IP aliases in FreeBSD?” It’s not terribly intuitive, but you can follow these steps:

Example:
Server’s primary IP: 192.168.1.11

Additional IP’s to add: 192.168.1.12, 192.168.1.13, and 192.168.1.14

 

Boot-time configuration:
Add it to /etc/rc.conf first (so you don’t forget). In this example, we have a Realtek card called rl0:
ifconfig_rl0="inet 192.168.1.11 netmask 255.255.255.0"
ifconfig_rl0_alias0="inet 192.168.1.12 netmask 255.255.255.0"
ifconfig_rl0_alias1="inet 192.168.1.13 netmask 255.255.255.0"
ifconfig_rl0_alias2="inet 192.168.1.14 netmask 255.255.255.0"

IMPORTANT NOTE: Start with the number 0 (zero) any time that you make IP alias configurations in /etc/rc.conf.
This is BAD form:

ifconfig_rl0="inet 192.168.1.11 netmask 255.255.255.0"
ifconfig_rl0_alias1="inet 192.168.1.12 netmask 255.255.255.0"
ifconfig_rl0_alias2="inet 192.168.1.13 netmask 255.255.255.0"
ifconfig_rl0_alias3="inet 192.168.1.14 netmask 255.255.255.0"

If you do it the wrong way (which means starting alias with anything but alias0), only the primary comes up. Keep that in mind.

Bringing up the new IP’s:
You can do things the extraordinarily dangerous way:
# /etc/rc.network restart

Or, you can follow the recommended steps:
# ifconfig rl0 alias 192.168.1.12 netmask 255.255.255.0
# ifconfig rl0 alias 192.168.1.13 netmask 255.255.255.0
# ifconfig rl0 alias 192.168.1.14 netmask 255.255.255.0

Test your work:
Any good system administrator knows to test things once their configured. Make sure to ping your new IP’s from a source on your network and outside your network (if possible/applicable).

Crontab basics

cron is a unix, solaris utility that allows tasks to be automatically run in the background at regular intervals by the cron daemon. These tasks are often termed as cron jobs in unix , solaris.
Crontab (CRON TABle) is a file which contains the schedule of cron entries to be run and at specified times.


Crontab syntax


A crontab file has five fields for specifying day , date and time  followed by the command to be run at that interval.

*     *   *   *    *  command to be executed
–     –    –    –    –
|     |     |     |     |
|     |     |     |     +—– day of week (0 – 6) (Sunday=0)
|     |     |     +——- month (1 – 12)
|     |     +——— day of month (1 – 31)
|     +———– hour (0 – 23)
+————- min (0 – 59)

* in the value field above means all legal values as in braces for that column.
The value column can have a * or a list of elements separated by commas. An element is either a number in the ranges shown above or two numbers in the range separated by a hyphen (meaning an inclusive range).

Crontab Example

min      hour      day/month      month      day/week       Execution time
30     0     1     1,6,12     *     — 00:30 Hrs  on 1st of Jan, June & Dec.

0     20     *     10     1-5     –8.00 PM every weekday (Mon-Fri) only in Oct.

0     0     1,10,15     *     *     — midnight on 1st ,10th & 15th of month

5,10     0     10     *     1     — At 12.05,12.10 every Monday & on 10th of every month


By default cron jobs sends a email to the user account executing the cronjob. If this is not needed put the following command At the end of the cron job line .

>/dev/null 2>&1


To collect the cron execution execution log in a file :

30 18  *    *   *    rm /home/someuser/tmp/* > /home/someuser/cronlogs/clean_tmp_dir.log

Uninstall APF

Sometimes we may require to remove APF from the server. Here is a guide which shows how to remove APF completely from the server.

Stop the firewall first
service apf stop
/bin/rm -rfv /etc/apf
Remove the cron for APF
/bin/rm -fv /etc/cron.daily/fw
/bin/rm -fv /etc/init.d/apf
lastly disable at startup
chkconfig apf off

This should remove APF completely from the server as we removed the APF daemon, cron and files.