Virtual Lab : Get Your hands dirty with grep & RegEx

Get Your Hands Dirty with Grep and RegEx

Regular expressions are extremely useful in extracting information from text such as code, log files, spreadsheets, or even documents. As an unix administrator we do regularly use the regex to perform various problem diagnosis , automation or remediation tasks. And it is almost impossible to imagine the life of unixadmin without RegEx usage. This quick virtual lab session will give you the basic usage knowledge of regular expressions with grep, which is a  essential skill required for unix system administrators.

 

Below is the Learning Map, that makes you comfortable with Grep  and RegEx ( i.e. Regular Expressions) within a day:

 

 Launch Your Linux Terminal for Exercises  

( get your lab setup script here )

 

Step 1 : Understand how do we search lines that matches particular words / numbers.       

Try Now

The first thing to recognize when using regular expressions is that everything is essentially a character, and we are writing patterns to match a specific sequence of characters (also known as a string). Characters include normal letters, but digits as well. In fact, numbers 0-9 are also just characters and if you look at an ascii table, they are listed sequentially.      

Step 2 : Understand how to work with  “.” ( dot) while extracting data.                                     

Try Now

In regular expressions, we are often matching pieces of text that we don’t know the exact contents of, other than the fact that they have some pattern (eg. are phone numbers for example). In  such cases we do use the dot as wildcard that can represent any character in a specific position of the word

Step 3 :  Understand how to match the words that have specific characters.                            

 Try Now

When you use the dot (.) as wild card, that will match to any character in the given position, it is useful but sometimes it is too powerful. Just incase if we want to match limited set of characters in a specific position we should go different kind of RegEx pattern.

Step 4 :  Understand how to exclude words with specific characters.                                           

Try Now

When we have big list of matching characters that we are looking in a word our RegEx will become lengthier and  complex. In such cases, we can use RegEx to specify the characters that we dont want to see instead of mentioning big list of characters that we want to see.

Step 5 : Understand how to use Character Ranges, instead of using  individual character list , while matching the words.     

Try Now 

If we want to match or exclude a large number of sequential characters? eg. Matching a…m but not n…z. Do we have to write [abcdefghijklm]? Not really required we can simply use the character ranges.

Step 6 : Understand how to match words with arbitrary number of characters.     

 Try Now

It is tricky to match the words, when they have arbitrary number characters ( that means some of the characters repetitive for one or more number of times.

Step 7 :  Understand how to retrieve data conditionally.                                                                 

Try Now

So far whatever we understand is finding the words either matching or excluding specific characters. Sometime we might require to match the lines using group of words, in such cases RegEx goes well with the logical operators.

Step 8 :  Understand how to match the words with optional characters                                    

Try Now

In Step 6 we learnt about matching words with the characters which are repeated one or more number of times, but it is often required to match the words which might be having optional characters ( i.e. zero or more number of times)

Step 9 :  Understand how to deal with the white spaces.                                                             

Try Now

When dealing with real-world input, such as log files and even user input, it’s difficult not to encounter white spaces  We use it to format pieces of information to make it to read and scan visually, and a single space can change the entire output that we are looking for.

Step 10 : Understand how to match the lines with specific characters either at the starting or the ending.    

Try Now

Sometimes it is not desirable to match the words/patterns from any where of the text, instead we want to search the patterns only either beginning or ending of the each line. 

Ramdev

Ramdev

I have started unixadminschool.com ( aka gurkulindia.com) in 2009 as my own personal reference blog, and later sometime i have realized that my leanings might be helpful for other unixadmins if I manage my knowledge-base in more user friendly format. And the result is today's' unixadminschool.com. You can connect me at - https://www.linkedin.com/in/unixadminschool/

8 Responses

  1. Elumalai M says:

    Hi Ramdev,

    Appreciate your effort on research… 

    I have executed the script sh setup.sh. All the task files are 0 size, any additional steps in this script.

    var/root/unixadminschool # ls -l                                               
    total 0                                                                         
    -rw-r–r–    1 root     root             0 May 23 09:07 task1                  
    -rw-r–r–    1 root     root             0 May 23 09:07 task10                 
    -rw-r–r–    1 root     root             0 May 23 09:07 task2                  
    -rw-r–r–    1 root     root             0 May 23 09:07 task3                  
    -rw-r–r–    1 root     root             0 May 23 09:07 task4                  
    -rw-r–r–    1 root     root             0 May 23 09:07 task5                  
    -rw-r–r–    1 root     root             0 May 23 09:07 task6                  
    -rw-r–r–    1 root     root             0 May 23 09:07 task7                  
    -rw-r–r–    1 root     root             0 May 23 09:07 task8                  
    -rw-r–r–    1 root     root             0 May 23 09:07 task9                  
      

    • Ramdev Ramdev says:

      Hi elumalai, yeah I have noticed it. Some browsers misinterpreting the single codes and doubles codes when you copy and paste them. Can you run the script with sh -x option so that you will know if any errors.

  2. Elumalai M says:

    Please see the below output…

    Welcome to Unixadminschool.com                                                  
    + echo Creating Required Directories and Files for your excercise               
    Creating Required Directories and Files for your excercise                      
    + mkdir -p /var/root/unixadminschool                                            
    + seq 1 10                                                                      
    + echo Create /var/root/unixadminschool/task1                                   
    Create /var/root/unixadminschool/task1                                          
    + grep ^#test1# setup.sh                                                        
    + cut -d# -f3                                                                   
    + echo Create /var/root/unixadminschool/task2                                   
    Create /var/root/unixadminschool/task2                                          
    + grep ^#test2# setup.sh                                                        
    + cut -d# -f3                                                                   
    + echo Create /var/root/unixadminschool/task3                                   
    Create /var/root/unixadminschool/task3                                          
    + grep ^#test3# setup.sh                                                        
    + cut -d# -f3                                                                   
    + echo Create /var/root/unixadminschool/task4                                   
    Create /var/root/unixadminschool/task4                                          
    + grep ^#test4# setup.sh                                                        
    + cut -d# -f3                                                                   
    + echo Create /var/root/unixadminschool/task5                                   
    Create /var/root/unixadminschool/task5                                          
    + grep ^#test5# setup.sh                                                        
    + cut -d# -f3                                                                   
    + echo Create /var/root/unixadminschool/task6                                   
    Create /var/root/unixadminschool/task6                                          
    + grep ^#test6# setup.sh                                                        
    + cut -d# -f3                                                                   
    + echo Create /var/root/unixadminschool/task7                                   
    Create /var/root/unixadminschool/task7                                          
    + grep ^#test7# setup.sh                                                        
    + cut -d# -f3                                                                   
    + echo Create /var/root/unixadminschool/task8                                   
    Create /var/root/unixadminschool/task8                                          
    + grep ^#test8# setup.sh                                                        
    + cut -d# -f3                                                                   
    + echo Create /var/root/unixadminschool/task9                                   
    Create /var/root/unixadminschool/task9                                          
    + grep ^#test9# setup.sh                                                        
    + cut -d# -f3                                                                   
    + echo Create /var/root/unixadminschool/task10                                  
    Create /var/root/unixadminschool/task10                                         
    + grep ^#test10# setup.sh                                                       
    + cut -d# -f3                   
    + echo Your Lab setup Ready                                                     
    Your Lab setup Ready                                                            
    /var/root #
    + echo Your Lab setup Ready                                                     
    Your Lab setup Ready                                                            
    /var/root #

  3. Ramdev Ramdev says:

    Hi Ellumalai,  Thanks you. I got that fixed now, can you check. As i mentioned earlier the problem is with the double quotes which the clipboard fails to recognize. But now It should be fine.

  4. snehal says:

    How can I replace one specific word in from multiple presence in vi editor?

  5. Ramdev Ramdev says:

    :1,$ s///g .

     We are having active facebook group to help unixadmin in quick manner…. please join  http://www.facebook.com/groups/Just.unix.no.noise/

  1. September 15, 2015

    […] Virtual Lab : Get Your hands dirty with grep & RegEx […]

  2. September 17, 2015

    […] Virtual Lab : Get Your hands dirty with grep & RegEx […]

What is in your mind, about this post ? Leave a Reply

Close
  Our next learning article is ready, subscribe it in your email

What is your Learning Goal for Next Six Months ? Talk to us