Taking Backup of MySQL database

MySQL is a very popular and widely used open source database. Most of the software engineers might have used mysql database atleast once. Keeping a backup of a database or migrating the tables/databases from one database server to another is a very important task. In mysql this task can be performed in a very simple way.

MySQL is providing a commandline utility to perform this operation. The command is mysqldump.

For getting more details about mysqldump, type the following command

>mysqldump --help

For taking the backup of a complete database, type the following command.

>mysqldump --databases [DB1 Name] ....[DBn Name] -u <username> -h <hostname> -p

This will print the entire database dump to the console. For storing the output of this in a file, do the following operation.

 >mysqldump --databases [DB1 Name] ....[DBn Name] -u <username> -h <hostname> -p >> dump.sql

Here the contents will be written to a file locally.

If you want to take the back up of all the contents in a mysql server, we can back up all databases.

The command to perform this operation is given below.

>mysqldump --all-databases -u <username> -h <hostname> -p >> dump.sql

In the above commands, <username> is the mysql username and <hostname> is the hostname or ip address of the machine where the mysql server is running.

Now your database is backed up. The next step is how to load this dump file to another mysql instance.

This is a very simple task.

• Copy the dump file to the machine where you are planning to perform the following operations
• Login to mysql instance with valid credentials
• In the mysql prompt, type the following command.

mysql> source <path to the dump file>

This will load the contents inside the dump file to the new mysql instance.

If you want to know more about the mysql dump file, open the file in a text file editor and read the contents. It contents the DDLs and DMLs that recreates your databases and tables.

Changing the Hostname of Redhat and CentOS machines

Sometimes, we need to change the hostname of machines. Hostname is just like our name which is a name assigned to machines. For changing the hostname of a machine, we need root access.
We can change the hostname through several ways. One of the method to change the hostname is listed below.

In linux OS, almost every configuration are present inside some files.
The hostname is configured in the following file
/proc/sys/kernel/hostname

If we change the value in this file with some other name, the hostname will be changed.

So for changing hostname of a machine, edit the /proc/sys/kernel/hostname file and add the desired hostname.

Linux commands to get IP address and Hostname of a machine

Knowing the IP address and Host name of a machine are some very common requirements.

The command to get the IP address of a linux machine is
>ifconfig
eth0      Link encap:Ethernet  HWaddr 0A:90:74:4A:62:42
          inet addr:172.31.9.245  Bcast:172.31.15.255  Mask:255.255.240.0
          inet6 addr: fe80::890:74ff:fe4a:6242/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9001  Metric:1
          RX packets:13139146 errors:0 dropped:0 overruns:0 frame:0
          TX packets:12665241 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:10433927585 (9.7 GiB)  TX bytes:16516355937 (15.3 GiB)
          Interrupt:77

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:40162017 errors:0 dropped:0 overruns:0 frame:0
          TX packets:40162017 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0

          RX bytes:6184024837 (5.7 GiB)  TX bytes:6184024837 (5.7 GiB)

The command to get the hostname of a linux machine is
>hostname
masternodemachine

Python code to find the md5 checksum of a file

Checksum calculation is an unavoidable and very important step in places where we transfer files/data. The simplest way to ensure whether a file reached the destination properly or not is by comparing the checksum of source and target files. Checksum can be calculated in several ways. One is by calculating the checksum by keeping the entire file as a single block. Another way is multipart checksum calculation, where we calculate the checksum of multiple small chunks in the file and finally calculating the aggregated checksum.
Here I am explaining about the calculation of checksum of a file using the simplest way. I am using the hashlib library in python for calculating the checksum.
Suppose I have a zip file located in the location /home/coder/data.zip. The checksum of the file can be calculated as follows.

import hashlib
file_name = ‘/home/coder/data.zip’
checksum = hashlib.md5(open(file_name).read()).hexdigest()
print checksum

One common mistake I have seen among people is passing the file name directly without opening the file

Eg: hashlib.md5(file_name).hexdigest()

This will also return a checksum. But it will be calculating the checksum of the file name, not the checksum calculated based on the contents of the file.  So always use the checksum calculation as follows

hashlib.md5(open(file_name).read()).hexdigest()

This will return the exact checksum. 

In linux, you can calculate the md5sum using a commandline utility also.

> md5sum file_name

Python Code Snippet to get the hostname and IP address of a machine

The following python code snippet will return the IP address of a machine

import socket

ip_address = socket.gethostbyname(socket.gethostname())

print ip_address

host_name = socket.getfqdn()

print host_name

The IP address may not be proper in all the cases. If the /etc/hosts file of the unix machine contains entries with hostname mapped to 127.0.0.1, will return 127.0.0.1 as IP address. So for ensuring proper working, it will be better to use the fully qualified hostname method. 

Python program to check whether a number is Odd or Even

This is a very basic program in python for checking whether a given number is odd or even.

__author__ = 'coder'

def OddEven(number):
    try:
        remainder = number % 2
        if number == 0:
            print "Number is Zero"
        elif remainder == 0:
            print "Number is Even"
        else:
            print "Number is Odd"
    except:
        print "Error while processing"

if __name__ == '__main__':
    OddEven(11)

Hadoop Interview Questions

1) What is the name of Hadoop's file system .?
Ans: HDFS

2) What is the full form of HDFS.?
Ans: Hadoop Distributed File System

3) What is the Processing Layer of Hadoop. ?
Ans: Mapreduce

4) Hadoop framework is written in which language .?
Ans: Java

5) What is the licencing cost for hadoop.?
Ans: Hadoop is an opensource technology. So it is free.

6) Who is known as father of Hadoop.?
Ans: Doug Cutting

7) How Hadoop differs from other data processing technologies..?
Ans: Hadoop is a framework which is having distributed storage as well as a distributed processing layer. The basic idea behind hadoop is to bring down the processing layer down to storage. Hadoop is a horizontally scaling framework So high end server grade hardware is not required. Only commodity hardware is required.

8) Is hadoop good for real time processing.?
Ans: Directly No. Hadoop is a batch processing framework. So it can't be used for real time processing. But it can work along with other technologies to produce real time outputs.

9) Is hadoop a replacement for RDBMS..?
Ans: Hadoop is not suitable for processing small or medium amount of data. Since hadoop is a batch processing framework, hadoop will not provide faster output. What hadoop guarantees is that, it will never fail with large data. In case of large data, which the other data processing technologies can't process, hadoop will perform well

10) If hadoop is open source and free, who is maintaining it and enhancing it.?
Ans: Hadoop is an Apache project, people all over the world are contributing and adding more enhancements to it. Lot of companies are also using hadoop, they are also contributing to hadoop.

11) Why hadoop became very popular.?
Ans: Analyzing hidden insight from data became a very important part of almost every organisation now. The correctness of the insights will be more as the size of the data is more. Now a days the usage of internet and social media is very high. So if we collect that data alone, we can analyse people upto some extent. Similar to this, we can analyse anything and everything using the history data. This is one reason. Similarly real time  monitoring and decision making also became very important now. This is another factor. If we go for a tool / product with licence, the licensing cost itself will be very high. Hadoop is opensource and free. Hadoop runs on commodity hardware, so the cost of the Infrastructure is also less. This made hadoop a hot cake in the market.

12) What do you mean by a pseudo distributed hadoop cluster.?
Ans If all the daemons of the hadoop are running in a single node, it is called pseudo distributed mode. This is not used for production. This is just for development and learning purpose.