Beginner · 30 minutes · Guided lesson

Turn text into useful information

Use redirection and a pipeline to answer a question about a small log.

By the end of this lesson

  • Distinguish standard output and standard error
  • Explain overwrite and append redirection
  • Filter and count a controlled set of records

Your environment: Bash and the named tools in a Linux lab. Minimal distributions may require extra packages. Setup help

Commands communicate through streams

Most command-line tools read input and write output. Standard output is the normal result; standard error carries diagnostics. The pipe character connects one command’s standard output to the next command’s input. It does not normally carry standard error, so an error may still appear on the screen while output goes into a file.

The shell handles redirection before the command runs. > creates or truncates an output file; >> appends. Never redirect a filter back onto the file it is reading: the shell can empty the source before the filter opens it. Write to a different filename and inspect the result.

Make the result explainable

A pipeline should answer a specific question. In this lab the question is how many records begin with ERROR. Anchoring the pattern at the beginning avoids matching the word inside an unrelated message. Test each stage on its own, then connect them.

A successful process exit status does not prove your interpretation of the output is correct. grep normally returns 1 for no matches, which is different from its error status. In Bash, a pipeline normally returns the status of its last command; advanced scripts can use pipefail when they need earlier failures to affect that result.

See it in context

Worked example

Create a tiny log, then keep only its error records.

mkdir -p "$HOME/training4linux/text-demo"
cd "$HOME/training4linux/text-demo"
printf "%s\n" "INFO ready" "ERROR timeout" "ERROR refused" > events.log
grep "^ERROR" events.log | sort > errors.txt
cat errors.txt

There are two sorted error lines. events.log is unchanged because the pipeline writes to errors.txt.

Your turn

Practical lab

Produce a short error report without modifying the source log.

Run commands in your own lab. This website displays instructions; the practice terminal below helps you rehearse the syntax.

1. Prepare your workspace

mkdir -p "$HOME/training4linux/streams"
cd "$HOME/training4linux/streams"
printf "%s\n" "INFO start" "ERROR disk" "INFO retry" "ERROR network" "WARN slow" > events.log

2. Complete the task

  1. Write lines beginning with ERROR to errors.txt.
  2. Count the lines in the source and in your report.
  3. Append ERROR quota to the source, rebuild the report, then recount it.

3. Check your result

  • The initial source has 5 lines and the initial report has 2.
  • After appending and rebuilding, the source has 6 lines and the report has 3.
  • The original INFO and WARN lines remain in events.log.
Need a hint?

Use >> only for adding the new event. Rebuild the derived report with >.

Reveal a worked solution

Compare this with your approach after trying the task. More than one valid solution may exist.

grep "^ERROR" events.log > errors.txt
wc -l events.log errors.txt
printf "%s\n" "ERROR quota" >> events.log
grep "^ERROR" events.log > errors.txt
wc -l events.log errors.txt

After the lab: No live logs were changed. Keep the small fixtures for revision.

Common mistakes to avoid

  • Do not run grep pattern events.log > events.log: it can truncate the source.
  • Run the setup again before repeating this lab if you want the original counts.

Check your understanding

Knowledge check

Answer all 3 questions correctly to pass. Retry as often as you need; your best score is kept.

1. What does > do to an existing output file?
2. What normally travels through |?
3. Which grep pattern matches ERROR only at the start of a line?

Your lesson progress

Not started · Best score: 0% · Attempts: 0

Practical completion is self-reported. Progress is linked to a private cookie in this browser and stored on the site. Clearing that cookie loses access to it. Manage or erase progress.

Keep the reference close

Further reading