Learn about awk command in Linux

awk is a powerful and versatile text processing tool available in Unix-like operating systems, including Linux. It is primarily used for extracting and manipulating data from text files, particularly when the data is organized in a structured format. awk is often used for tasks such as data extraction, report generation, and data transformation.

The basic idea behind awk is that it processes input files line by line, and for each line, it applies a set of user-defined patterns and corresponding actions. These patterns and actions are specified as a series of rules. The general syntax of an awk command is as follows:

awk 'pattern { action }' input_file

Here’s a detailed breakdown of how awk works:

Input Processing:

• awk reads the input file(s) line by line. By default, it treats each line as a record and each field (data separated by a delimiter, usually whitespace) within a record as a separate entity.

Patterns and Actions:

• A pattern is a condition that specifies when a particular action should be executed. If the pattern is true for a given line (record), the associated action is performed.
• An action is a set of commands that are executed when the corresponding pattern is true.
• Both the pattern and the action are enclosed in single quotes (' ').

Built-in Variables:

• awk provides a set of built-in variables that you can use in your patterns and actions. Some commonly used ones include:
  • $0: Represents the entire line (record).
  • $1, $2, etc.: Represent the first, second, etc., fields within a record.
  • NR: Represents the record (line) number being processed.
  • NF: Represents the number of fields in the current record.
  • FS: Specifies the input field separator (default is whitespace).

Using Patterns and Actions:

• Patterns can be simple or complex conditions involving comparisons, logical operators, regular expressions, etc. If a pattern is omitted, the associated action is performed for every record.
• Actions can consist of a sequence of commands that are enclosed in curly braces {}. These commands can be anything you would use in the shell, including print statements and arithmetic operations.

Example:

   awk '$3 > 50 { print $1, $2 }' data.txt

In this example:

• $3 > 50 is the pattern, which checks if the value of the third field is greater than 50.
• { print $1, $2 } is the action, which prints the first and second fields if the pattern is true.

Output:

• By default, awk prints the entire line (record) if an action is executed. You can use the print statement to control what is printed and how it is formatted.

Running awk:

• You can run awk directly in the terminal, or you can use it in scripts.
• To process input from a file, use the syntax: awk 'pattern { action }' input_file.
• To process input from a pipeline (e.g., cat file.txt | awk ...), you can omit the input file and use standard input.

Advanced Features:

• awk also supports loops, arrays, and functions, which can be used for more complex data manipulation tasks.

In summary, awk is a versatile text processing tool that allows you to specify patterns and actions to selectively process and manipulate data from text files. It is particularly useful for tasks involving structured data and can be a powerful tool in the hands of a skilled user.