This illustrates using pipes to eliminate temporary files. We start
with a bash
script that takes some demographic data (see MOCK_DATA.csv)
specified as a file name as a command line argument.
The script then outputs a count of how many people come from different
states. The output on the included data file is:
49 MN
21 IA
20 WI
7 ND
3 SD
The first version generates a lot of temporary text files (one for
each step); the second does the same thing but uses pipes (|
) to
turn the output of each command into the input of the next command. This
avoids creating a ton of extra temporary files, each of which we
have to name, and naming is hard. We should also delete each of the
temporary files when we're done so we don't clutter up the
world. Thus not having them is Very Nice.
Note that we need to escape the newline at the end of each line
(including comments) by putting a backslash (\
) at the end of the line.
(The backslash needs to be the very last character.) This causes bash
to "ignore" the newline and see this as one (very long) line.
We could actually make it one lone long, but breaking it up like this
is a lot more readable.
The mock data was generated using Mockaroo.
See this StackOverflow answer from Jonathan Leffler for more on the advantages of pipes vs temporary files.