The AWK command is designed specifically for UNIX and Linux systems. Therefore, they cannot be used in Mac OS or Windows. The other operating system will have its own alternative to AWK, but this article is not about those commands. By using examples in this article, we will help you understand how Awk works and why it is needed in the first place.
What is Awk Command? [Definition]
We’d like to start by clarifying a few things, including the definition of the Awk command. Awk is a text processing command used in Linux. In addition to the Awk command, one could also use the sed command to process the text, but the latter has some limitations, so most Linux community forums prefer to use the Awk command instead. In addition, Awk gives users better control over the text than any other command.
Awk is, without a doubt, an excellent scripting language that is used for text scripting. With this command, one can search, replace and even sort the text according to their requirement. Programmers usually use this command as a means of defining the text patterns and designs that are present in a scaled-down effective program. The other name for the Awk command is gawk (GNU awk), which is also used by Linux-only operating systems.
What’s With The Name Awk?
You might be thinking, what in the world means, Awk? Well, in truth, the name of the command doesn’t have any meaning to it, which is quite clear at this point. The name of this command came from the first letters of three surnames. These three surnames were associated with the people who actually wrote the code for this command back in 1977. The names of those three programmers are Alfred Aho, Peter Weinberger, and Brian Kernighan. All three of these people were employees of AT & T Bell laboratories.
How Does Awk Command Work?
There are a number of different implementations of the Awk command, but in this article, we are sticking to the GNU implementation of Awk which is also known as gawk. When we process textual data files and streams to the Awk, the input which is taken by the command gets divided into records and fields. Until the end of the input is reached, the Awk will keep on working at the same file, and there can be no two files being simultaneously processed by Awk. The newline character is set as a default record separator. As a result, each line of the data will be a new record to Awk. If you want to change the new record separator, you can use the RS variable to add in the changes.
Now speaking of records, each record is made of fields that are separated by the field separator. By default, the fields are separated by the whitespace present in between the words. During the processing of data in Awk, if the pattern actually matches up with the record, then it will perform the specified action on that given record. But in case the awk command is not able to find a pattern in the record, it will show the user that all lines are matched.
Moreover, the awk action is present inside the braces {}, and it will always have statements. Each statement will provide awk command the required specification to perform the operation. Keep in mind that a single action can have more than one statement, which is separated by newline or semicolons. On the other hand, if you do not enter any rule with action, it will print out the whole record.
Also, when you are using the Awk command, anything that you write after the hash mark and at the end of the line will be considered as a comment.
With the use of a backslash (\), a user can break a single long line into multiple smaller ones using the continuation character.
How To Run AWK Command?
When using Awk you can run it in multiple ways. If the code you wrote can be passed directly to the Awk interpreter on the command line, you can use the following command to do so:
awk ‘program’ input-file…
Moreover, when you are running the program straight on the command line, it has to be enclosed in single quotes like the above example. This needs to be done because otherwise, the shell will not interpret the command as a program name.
On the other hand, if the code of the program is pretty big, it is best to put it in a file and use the -f option of the Awk command to get the file as the input.
awk -f programexample1-file input-file…
What Are Awk Patterns?
Patterns in Awk are basically the execution of rules. A rule gets executed when the pattern matches up with the input record of the command. Given below are the different types of Awk patterns that you can see during your implementation of the Awk.
- Regular expression
In a regular expression, the command matches up with the text of the input record fits with the regular expression
- Expression
A single expression will match up with the input data when the value is nonzero, or the value is not null.
- begpat, endpat
This pattern operation is used by separating the comma, and in this operation, a user needs to specify the range of records. The range has to include both the initial record, which matches up with the begpat, and the final record has to match with endpat
- BEGIN, END
These two are special pattern operations that are used to supply the startup and perform the cleanup actions for your awk program.
- BEGIN FILE, ENDFILE
These are pretty similar to the above-mentioned pattern operation; in these two operations, you will be supplying startup or cleanup actions which will be done on a per-file basis.
- Empty
This operation of the Awk command will match up each and every input record.
Awk Actions
A rule which is a part of the awk program contains patterns and actions; from these two, one will be omitted. In this action, the main objective is to tell the awk command what needs to be done for a pattern found in the input record.
To write the action in the Awk, you need to use braces no matter the number of actions present inside the braces. Even if there’s a single action that needs to be performed by the Awk, you need to use the braces (‘{ … }’). On the other hand, if you omit the action entirely from the command, you need to remove the braces as well.
Given below is the list of statements that can be used in the Awk command:
1. Expressions
The expressions are used to call out the functions or even assign values to the variables. When you are using this type of statement, it will only compute the values which are present in the expression.
2. Control Statements
This will provide an Awk program with the specification of flow; most of the awk constructs are like C programming along with some particular constructs designed explicitly for Awk scripting.
3. Compound Statements
This type of statement is used by enclosing more than one statement in braces.
4. Input Statement
The input statements are the getline and nextfile commands.
5. Output Statement
These are printf and print
6. Deletion Statement
This last statement is used to delete the elements which are present in the array.
AWK Command Usage With Example
One last thing we would like to tell you is Awk does come with the support of different types of statements which include expressions, conditionals, input, output statements, and more. Some of these are written below:
- Exit – It will stop the execution of the whole program and will exit out.
- Next – This will stop the processing of the current record and will go to the next record, which is present in the input provided by you.
- Print – this expression will do the printing of the records along with the printing of fields, variables, and even any custom text that you want.
- Print – This output statement will give you more control over the format of the output; its working is quite similar to that of C and bash’s printf statement.
Awk Commands
1. Printing The Table Line By Line
With this command, you get to print each line that is present in the input file.
awk” samplefile.txt
2. Printing Out Or Reading Awk Scripts
In order to define the awk script, you need to use braces which have to be surrounded by a single quotation. Given below is an example of how you should be writing this command:
awk ‘{print “Hi, have a great day to you.”}’
This command will print out, “Hi, have a great day to you.” In case you want to terminate the program, you can press ctrl+D. This might seem a little tricky to you but wait for what’s about to come.
3. Use Of Variables With Awk
With the use of variables, a user can process text files with the Awk command. The Awk command will put the variable for each data field that it finds on the input file.
- $0 is for the whole line.
- $1 is assigned to the first field of the line.
- $2 is given to the second field.
- $n is designated for the nth field.
Now let’s take an example of an input file by the name myexamplefile.txt where it is written: “Hi, have a great day to you.” We input the file which and now we want to find out print the first field from the above line.
awk ‘{print $1}’ myexamplefile
The above command will print the first word of each line that is present in the input file. So in this example, the output will be “Hello.”
4. Using Multiple Commands
When it comes to using multiple commands, we need to separate them with the use of semicolons. We will be using the echo command along with Awk in this example:
echo “Hi” | awk ‘{$1= “Good Morning”; print $0}’
Here we have used the echo command to select the word “Hi” from the file myexamplefile.txt. After that, the $1 command will make the field assigned to the variable $1 change to Good Morning. Lastly, the print command will print out the whole line with the change. So the end result of the above command will be “Good Morning, have a great day to you.”
5. Awk Postprocessing
These types of commands will be used when you have processed the data; in these commands, you need to use the END keyword.
$ awk ‘BEGIN {print “The File Contents Present In myexamplefile.txt:”}
{print $0}
END {print “This Is The End Of The File”}’ myfile
The above command will print this out in the terminal:
The file Contents Present In myexamplefile.txt:
Good Morning, have a great day you.
This Is The End Of The File
6. User-Defined Variables
In the Awk program, you can even define your own variables, but there’s only one exception, and that is the variable’s name can’t begin with a number. With the use of shell scripting, you can assign new variables.
$ awk’
BEGIN{
test1=” Good Evening, have a pleasant night.”
print test
}’
In the above command, we have defined the variable test1 and given it the value “Good Evening, have a pleasant night.” The output of the test, which is our custom variable, will be “Good Evening, have a pleasant night.” On the command line.
7. Structured Commands
Let’s create a new file and give it the name test2.txt. In that file, we have added numbers on each line, such as:
50
10
22
24
92
30
12
Now let’s use one of the structured commands to find out the list of numbers that are greater than 25. To get the result, you type in the following command:
$ awk ‘{if ($1 > 25) print $1}’ test2
The result of the above command will be the list of numbers greater than 25 that are present in the input file text2.txt. So on the terminal, you will see these numbers: 50, 92, 30. On the other hand, in case you are looking to use multiple statements, then remember to use braces’ {}.’
8. Formatted Printing
With the use of the printf command, you get to print the output with the format specifier. Some of the format specifiers are:
- %[modifier] control-letter
- c:- prints out the numeric output as a string
- d:- It will print the value of the integer.
- e:- prints out the scientific number.
- f:- Will give the output of float values.
- o:- Prints an octal value present in the input file.
- s:- Prints out a string of text.
$ awk ‘BEGIN{
x = 25 * 25.5
printf “The result is: %e\n”, x
}’
The above command will show you the output like this, “the result is “637.5.”
9. Built-In Functions
If you are someone who loves to use maths in their scripts, then Awk command and scripting language will seem like a blessing to you due to its inbuilt mathematical functions. Some of these functions are sin(x) | cos(x) | sqrt(x) | exp(x) |log(x) | rand(). To use them, you have to type in the command line, and the rest of it will be handled by Awk.
$ awk ‘BEGIN{x=log(5); print x}’
The output will be 0.69897000433
10. String Functions
This is the last example we are providing, there are tons of string functions, and you can use them all with Awk. We are going to show you one of the string functions in the command given below:
$ awk ‘BEGIN{x = “happy morning”; print toupper(x)}’
The output will be HAPPY MORNING.
In the above example, we have used the toupper string function, which will convert every single character of the string in the upper case.
Wrapping Up
There are so many Awk commands, and this article won’t be able to cover them all. These were just some of the most important and frequently used ones. Consequently, we ask you to explore them all and to share the ones you find most helpful below, so others can learn from your experience as well. Awk is both a command and a text scripting language, so if you learn all the different options of Awk, you will actually learn one more text scripting language, which is a bonus in the end. Take the plunge and find out what Linux commands have to offer!