Automation in System Administraton – are you a middle level unix admin? it is for you.

What is Automation?

Automation is nothing but performing system administration tasks without manual interventions, and complete the operations using specific tools/programs/scripts.

Example: Writing a script to backup the data from one server and run it using cron scheduler on every night.

Why the Automation is important for System Administrators?

My experience says ” if you ask a system admin to delete a single file from a set of 100 servers by login into each server one by one, he will be having 90% chances to make at least one human mistake during his task. And the mistake is not because of the person’s technical incompetence but it is because of natural human behavior called fatigue”.  The Sysadmins are more prone to this fatigue when they are involved in such manual repeated work during their routine job.

If you look at the career cycle of a system administrator, for the first 3 to 5 years most of the SAs will spend more amount of time in performing system admin tasks manually. During initial stage of career it will be fun to make every change manually and to have hands on experience for the complex tasks, but as experience grows the amount of work needs to performed also grows. And as a senior system administrator in an enterprise Unix environment our life never going to be easy unless we know how to make our work easy by avoiding repetitive manual tasks with the help of scripts/programs/tools.

In terms of competition in system administration careers, for first 5 years of career you can easily compete with other sysadmins with the help of your technical competence and problem resolution skills. but after 5 years of system admin experience it is difficult to compete with other champs of same experience with just regular system admin skills. The only skill that takes you out of the crowd is “your ability to simplify the job, and the time you are spending to complete a complex job on large set of servers” .. and the simplification of job only possible with your ability to automate the job.

 

What tasks can be automated?

1. Software Installation tasks: System admin tasks such as installation of software which are having some predefined  per-install and post-install tasks.

2. Server Health check tasks :  System Admin  tasks such as ‘verifying disk space usage, device health status and checking for other error logs’ are common tasks that needs to performed on daily basis.

3. Configuration Changes for large set of servers: Tasks such as modifying configuration files for making simple configuration changes.

4. Remote administration tasks: Performing System admin tasks for remote servers without entering into the server manually to each server.

5. Quality/Security Auditing tasks for Server environment: Tasks such as verifying current configuration for large of number of server and generating a simple reports for quality auditing purpose.

How to Automate the System Admin Tasks?

We can implement automation in system administration tasks using below components:

1. Scripts – Scripts are programs created using languages that can directly interact with operating system using operating system native commands.

We have several different scripting languages in use now and most popular scripting languages are   .. Shell Scripting, Perl Scripting , Python Scripting .. etc.

            — Shell scripting is useful for the sysadmin tasks which are less interactive and more operating system oriented. To get proficiency in shell scripting sysadmin must have a good understanding of the Operating System commands and their arguments. And also should have good hands on knowledge of the task that we want to automate.

         — Perl / Python scripting is more useful for the system admin tasks which are more interactive, dynamic in nature and requires more level of data processing. Proficiency in Perl/Python can make you real awesome administrator and highly valuable member of the team.

2. Job Scheduler: In automation, the first challenge is creating appropriate scripts to perform specific tasks and the second challenge is executing them in a specific time without has manual intervention.  And UNIX provides “CRON” scheduler utility for the purpose of automated job execution.

3. E-Mail setup :  Server e-mail system is useful to let the executing scripting to send you the script outputs in the form of email.

4. Password less remote authentication Setup:  It is common practice in larger environment to configure some servers as special privileged servers (and highly secured and monitored) which are having access to every other server in the environment without using the password. This kind of setup will allow sysadmins to perform complex tasks on large number of systems, within short amount of time.

 

 I will be discussing more practical automation techniques and scripts in my upcoming posts, until then drop  your comments on this post.

 

 

 

 

Ramdev

Ramdev

I have started unixadminschool.com ( aka gurkulindia.com) in 2009 as my own personal reference blog, and later sometime i have realized that my leanings might be helpful for other unixadmins if I manage my knowledge-base in more user friendly format. And the result is today's' unixadminschool.com. You can connect me at - https://www.linkedin.com/in/unixadminschool/

24 Responses

  1. Michael Michael says:

    Hello Ram ,Good one,Adding to this “cfengine” has been found to be of quite better tool in the industry  if possible try to include that in your  practical automation techniques and scripts  topic as well  ;-) 

  2. santhosh says:

    This info is very much needed for guys like me who have started their careers as SYS admins,will look forward for practical stuff.

  3. MTanvir says:

    really very helpful!

  4. Deepak says:

    A Good artical

  5. Ramesh says:

    Hello Ram .. Possible can you give some URL to learn perl script .. Thanks in ADV

  6. Yogesh Raheja says:

    @Ramesh, we are working on script part and soon will be start posting on script part for Unix Champs..

  7. Ramdev Ramdev says:

    @ramesh – please look at perl official learning page , http://www.perl.org/books/beginning-perl/ for perl fundamentals.

  8. Ramesh says:

    Thanks Ram & Yogesh

  9. Sai Pamidigantam says:

    I would classify the OS automation tasks into three categories
    1. That executes on individual servers. Example : mirroring and splitting of disks, error monitoring, vitals ( CPU, Disk, Network usage), user privilege checks. Dynamic resource allocations also come in this category.
    2. Centralized scripting: Operated from central servers. NIS/NIS+ pushes, cleanups, password changes to multiple servers ( non-NIS), Datacollection and compile, software and config ( like snmp and smtp ) pushes and adhoc reports ( do I have weblogic licenses on what servers !!!). These scripts have to well written for cross platform and cross version compatibility
    3. Interdependent servers: Servers in cluster ( OS, network or application level clustering ). Automation here requires knowledge of what is happening on one server to another. Jobs scheduled should here know not just at “what time”, but also “after success of X” or “on failure of Y”

  10. Hemant says:

    Great article Mr. Ram, and Thanks in advance for future articles…

  11. Manjuanth Loddnoor says:

    Good artical Yogesh.. Keep it up..

    Lets wait for the more articals on scripting… :)

  12. Yogesh Raheja says:

    @Manjuanth, thanks. Sure we are working on this part and will come with more articles on scripting.

  13. Esun says:

    Hello,
    its really good. you have explained the current stage in the industry . i am in 5+exp in linux. now i am thinking how to automate the manual effors. Jut i saw your website to check the nfs mounts. its really helpfull. All the best

    Regards Esun.

  14. Yogesh Raheja says:

    @Esun, thank you very much for your wonderful comments for our gurkulindia.

  15. goutham says:

    hello ram
                     good information

  16. uma says:

    Really Good..

  17. Ravi Ravi says:

    Hi bro… this is great idea…many of us are looking forward to learn scripting to make life easier and simple in this adminstration world. I am having hard time lately to learn r to find some training materials for perl scripting….it will wonderful if you can offer some paid classes too…

  18. ravi says:

    Awaiting from a long time……please start the practical scripting stuffs…..start with the basics and with pratical scenarios like…
    1) moving the application logs from local disk to a log server.(before moving the script should zip it and then move)
    2) script for userlogs(whenever a user logs in, a log should get created on some path which will contains all the activities done by user), the log name should consists of username_date/time_hostname.
    3) script for CPU and RAM usage for every 1 min and which will redirect the output to a file.
    4) script for sending mail to a particular user at a particular time………………etc

    • Ramdev Ramdev says:

      Sure Ravi. I know I lost my focus on this topic. I will have this covered by me or someone else, very soon. Thanks for bringing this to the light.

  19. ravi says:

    kindly also put hte examples of sed and awk

  1. September 15, 2015

    […] Automation in System Administraton – are you a middle level unix admin? it is for you […]

What is in your mind, about this post ? Leave a Reply

Close
  Our next learning article is ready, subscribe it in your email

What is your Learning Goal for Next Six Months ? Talk to us