Nowadays, it’s a staple of Linux administration, and a powerful tool that you can use to improve your efficiency.
What is Grep?
Grep stands for "globally search for a regular expression and print matching lines". And as the name suggests, it is a tool well suited to sifting through text.
When Should You use Grep
There are two main use cases for grep
- Search text within files
- Filter output of another program, so you only get what you care about
We’ll go through some examples of grep in action. The text file we will work with will be a pretty trivial one, just so I can show you the absolute basics of what grep is capable of, and some of the options you can run it with. Perhaps in the future we could explore some more realistic applications.
Working With Grep: A Basic Example
Consider the following file, saved as
grep-example.txt. It's 17 lines long, 15 contain a word, with one rogue line containing a random IP address, and another line containing something that had ambitions of being an IP address once upon a time…
Let’s see what we can do with this.
vessel cemetery scintillating Scintillating shape 220.127.116.11 2049235%208a222 opine declare zinc disagreeable lizards mark Hello hello queen queue
The most basic way to use grep is to try to get it to produce an exact match. We don’t have to wrap our search term in single quotes, but it does help guard against some unpredictable outcomes if you’re new to the tool, so I’d recommend doing so to start.
If I type in
grep 'cemetery' grep-example.txt the command will return the following:
$ grep 'cemetery' grep-example.txt cemetery
Nice! What if I want all the entries that contain the letter "c". I can run
grep 'c' grep-example.txt
$ grep 'c' grep-example.txt cemetery scintillating Scintillating declare zinc
OK, but this by itself might be pretty useless: How am I going to actually look these up. One way to do this would be to use the
--line-number option , which can be abbreviated to
$ grep 'c' -n grep-example.txt 2:cemetery 3:scintillating 4:Scintillating 9:declare 10:zinc
So far so good. But what if we want to grab that IP address?:
$ grep '18.104.22.168' grep-example.txt 22.214.171.124 2049235%208a222
Wait! What? We didn’t get the results we were expecting.This didn’t work because Linux interpreted the
. character here as standing in for any character whatsoever. As we touched upon earlier, the RE in grep stands for "regular expressions"- the programme checks for these and uses them in your search.
Because of this the following things also would have been returned in the search as well:
204A235B208C222 204-235-208-222 204323532083222
We wanted to find the IP address though. What can we do to get the results we want?
There are two ways of dealing with this situation
You could escape characters, so their special meaning in Regex is taken away:
$ grep '204\.235\.208\.222' grep-example.txt 126.96.36.199
Which gives us what we want.
However, for a large string, this could be quite tedious, and if you were sending this line to a colleague who isn't used to working with regex, they might not find the command you wrote to be intuitive.
A cleaner alternative would be to pass the
-F flag (which is an abbreviation for
--fixed-strings)to grep, or use
fgrep. Both of these options will have the same result: they will let you search for a string that is an exact match to one you provide.
$ grep --fixed-strings 188.8.131.52 grep-example.txt 184.108.40.206 $ grep -F 220.127.116.11 grep-example.txt 18.104.22.168 $ fgrep 22.214.171.124 grep-example.txt 126.96.36.199 #All these commands return the very same result
When you do want to use regular expressions in your search term, it can be a good idea to use the
--extended-regexp option (
-E ) for short. you can also call egrep instead of grep to achieve the same thing:
$ grep -E '[Hh]ello' grep-example.txt Hello hello
A lot of Regex features are available without passing this flag, but in case you can't remember which are included and which are not, passing this flag can help you stay on the safe side.
Filtering Out Noise
Imagine we are getting a huge amount of input from a programme we are using which is giving us information we don’t care about right now. Often when we use grep, we use it to filter out things we don’t care about.
Imagine that, in the example from earlier, instead of searching for all lines that contained the letter c, what about lines that omit it?
To do this, we have the option to use the inversion flag
-v , short for
--invert-match, which works like this:
$ grep -v 'c' grep-example.txt vessel shape 188.8.131.52 2049235%208a222 opine disagreeable lizards mark Hello hello queen queue $ grep --invert-match 'c' grep-example.txt vessel shape 184.108.40.206 2049235%208a222 opine disagreeable lizards mark Hello hello queen queue #Again, both these commands do the very same thing
This command is especially useful when you are using grep to control the flow of data from one application to another (As you might do with a CI pipeline).
The next thing we should take a look at is when you are sifting through a large amount of text, and you want to check for all instances of a string, without filtering for caps/lower case. At the moment, we know how to get grep to return only one of the two
hello strings in our file:
$ grep 'Hello' grep-example.txt Hello $ grep 'hello' grep-example.txt hello
To do this. Grep has a special flag
--ignore-case which (as the name suggests) ignores cases when you do your searches. The short form of this command is
$ grep --ignore-case 'H' grep-example.txt shape Hello hello
The abbreviated version for this command
grep --i 'H' grep-example.txt would have accomplished the very same thing.
These commands so far should get you started, and give you some idea of how grep is used within a file. but what if you have a large number of files distributed over a large number of directories. This is probably how you are going to use grep a fair bit in reality, and fortunately, as we mentioned at the beginning of this article, It is a problem that grep was born to solve.
To show how grep works recursively, I've taken our example file from early and placed copies of it in several places within the directory structure of a project. I used the
tree command (which is a pretty handy tool to get your hands on, in and of itself, ) to generate the diagram below.
$ tree nested_texts/ . ├── grep-example.txt └── nest ├── grep-example.txt └── nest └── grep-example.txt
Right, so to simply do the job, and run a search recursively, through every file in a tree, we can pass grep the
-r flag as below.(The long form of this command being
grep -r hello nested_texts/ nested_texts/grep-example.txt:hello nested_texts/nest/grep-example.txt:hello nested_texts/nest/nest/grep-example.txt:hello
Pretty handy right? You can probably already start to see how this could be used to trawl through some of the larger repositories you work with.
A Side Note About Flags
That pretty much concludes everything I wanted to show you, however there is one last thing I wanted to show, that I couldn't find a good way to fit in. When you use flags with grep (and a number of other Linux tools, for that matter), you can combine your flags. A couple quick examples of this are below:
A command combining recursion with case insensitivity:
grep -ri 'H' nested_texts/ nested_texts/grep-example.txt:shape nested_texts/grep-example.txt:Hello nested_texts/grep-example.txt:hello nested_texts/nest/grep-example.txt:shape nested_texts/nest/grep-example.txt:Hello nested_texts/nest/grep-example.txt:hello nested_texts/nest/nest/grep-example.txt:shape nested_texts/nest/nest/grep-example.txt:Hello nested_texts/nest/nest/grep-example.txt:hello #Returns everything containing the letter 'h', Caps or not, in every file within the tree.
A command combining recursion with case insensitivity and prints line numbers, I've seen people use this a lot, and I'm sure you will too:
grep -rni 'H' nested_texts/ nested_texts/grep-example.txt:5:shape nested_texts/grep-example.txt:14:Hello nested_texts/grep-example.txt:15:hello nested_texts/nest/grep-example.txt:5:shape nested_texts/nest/grep-example.txt:14:Hello nested_texts/nest/grep-example.txt:15:hello nested_texts/nest/nest/grep-example.txt:5:shape nested_texts/nest/nest/grep-example.txt:14:Hello nested_texts/nest/nest/grep-example.txt:15:hello #Same as before, but with line numbers
So, If you're getting used to using grep for the first time, I hope this has served as a good introduction. Note that if you're working with really large repositories the are faster tools available to you, but grep is still worth getting to know.
Thanks for reading.