How to Read a File Line By Line in Bash
4 min read
When writing Bash scripts, you will sometimes find yourself in situations where you need to read a file line by line. For example, you may have a text file containing data that should be processed by the script.
In this tutorial, we will discuss how to read a file line by line in Bash.
Reading a File Line By Line Syntax
The most general syntax for reading a file line-by-line is as follows:
while IFS= read -r line; do printf '%s\n' "$line" done < input_file
or the equivalent single-line version:
while IFS= read -r line; do printf '%s\n' "$line"; done < input_file
How does it work?
The input file (
input_file) is the name of the file redirected to the while loop. The
read command processes the file line by line, assigning each line to the
line variable. Once all lines are processed, the while loop terminates.
By default, the
read command interprets the backslash as an escape character and removes all leading and trailing white spaces, which sometimes may cause unexpected behavior. To disable backslash escaping, we’re invoking the command with the
-r option, and to disable the trimming, the internal field separator (
IFS) is cleared.
We’re using [
printf] instead of
echo to make the code more portable and to avoid unwanted behaviors. For example, if the line contains values such as “-e”, it will be treated as an echo option.
Reading a File Line By Line Examples
Let’s take a look at the following example. Suppose we have a file named
distros.txt containing a list of some of the most popular Linux distributions, and their package managers separated with comma (
Ubuntu,apt Debian,apt CentOS,yum Arch Linux,pacman Fedora,dnf
To read the file line by line, you would run the following code in your terminal:
while IFS= read -r line; do printf '%s\n' "$line" done < distros.txt
The code reads the file by line, assigns each line to a variable, and prints it. Basically, you would see the same output as if you would display the file content using the
while IFS= read -r line; do if [[ "$line" == *"apt"* ]]; then printf '%s\n' "$line" fi done < distros.txt
When reading a file line by line, you can also pass more than one variable to the
read command, which will split the line into fields based on
IFS. The first field is assigned to the first variable, the second to the second variable, and so on. If there are more fields than variables, the leftover fields are assigned to the last variable.
In the following example, we set
IFS to a comma (
,) and pass two variables
pm to the
read command. Everything from the beginning of the line until the first comma will be assigned to the first variable (
distro), and the rest of the line will be assigned to the second variable (
while IFS=, read -r distro pm; do printf '%s is the package manager for %s\n' "$pm" "$distro" done < distros.txt
apt is the package manager for Ubuntu apt is the package manager for Debian yum is the package manager for CentOS pacman is the package manager for Arch Linux dnf is the package manager for Fedora
Alternative File Reading Methods
Using a Process Substitution
Process substitution is a feature that allows you to use the output from command as a file:
while IFS= read -r line; do printf '%s\n' "$line" done < <(cat input_file )
Using a Here String
Here String is a variant of Here document . The string
(cat input_file ) keeps the newlines:
while IFS= read -r line; do printf '%s\n' "$line" done <<< $(cat input_file )
Using File descriptor
You can also provide the input to the loop using a file descriptor:
while IFS= read -r -u9 line; do printf '%s\n' "$line" done 9< input_file
When working with file descriptors , use a number between 4 and 9 to avoid conflict with shell internal file descriptors.
In Bash, we can read a file line-by-line using a while loop and the
If you have any questions or feedback, feel free to leave a comment.