Bash scripting is more than just a way to automate tasks; it’s a powerful tool for manipulating data and controlling your system’s behavior.
Let’s see some common commands in action. Imagine you have a log file, app.log, and you want to find all lines containing "ERROR" that occurred in the last hour.
# Get the current hour in HH format
current_hour=$(date +"%H")
# Filter the log file for lines containing "ERROR" and the current hour
grep "ERROR" app.log | awk -v hour="$current_hour" '$0 ~ hour'
This snippet demonstrates how grep and awk can be chained together to perform sophisticated filtering. grep finds lines matching a pattern, and awk processes those lines further, in this case, by checking if the line contains the current hour.
Here’s a breakdown of commands you’ll likely encounter and use daily:
File and Directory Operations:
ls: List directory contents.ls -l: Long listing format, showing permissions, owner, size, and modification date.ls -a: Show all files, including hidden ones (starting with.).
cd: Change directory.cd ..: Move up one directory.cd ~: Go to your home directory.cd -: Go to the previous directory you were in.
pwd: Print working directory. Shows your current location in the filesystem.mkdir: Make directories.mkdir -p projects/web/frontend: Creates nested directories if they don’t exist.
rm: Remove files or directories.rm myfile.txt: Removesmyfile.txt.rm -r mydir: Removesmydirand its contents recursively. Use with extreme caution!rm -f important.config: Force removal without prompting. Use with extreme caution!
cp: Copy files and directories.cp source.txt destination.txt: Copiessource.txttodestination.txt.cp -r sourcedir destdir: Copiessourcedirand its contents todestdir.
mv: Move or rename files and directories.mv oldname.txt newname.txt: Renamesoldname.txttonewname.txt.mv file.txt /path/to/new/location/: Movesfile.txtto a different directory.
touch: Create empty files or update timestamps.touch newfile.log: Createsnewfile.logif it doesn’t exist.
Text Processing and Searching:
cat: Concatenate and display file content.cat file1.txt file2.txt: Displays content of both files.cat file.txt: Displays content offile.txt.
grep: Search for patterns in text.grep "pattern" filename: Finds lines containing "pattern" infilename.grep -i "error" logfile.txt: Case-insensitive search for "error".grep -v "debug" logfile.txt: Invert match, showing lines without "debug".grep -r "config" /etc/: Recursively search for "config" in/etc/.
sed: Stream editor for filtering and transforming text.sed 's/old/new/g' filename: Replaces all occurrences of "old" with "new" infilename.sed -i 's/localhost/192.168.1.100/g' myconfig.conf: Editsmyconfig.confin-place, replacing "localhost" with "192.168.1.100".
awk: Pattern scanning and processing language. Powerful for structured text.awk '{print $1, $3}' data.txt: Prints the first and third fields of each line indata.txt.awk -F',' '$2 > 10 {print $1}' csvdata.csv: If the second field (comma-separated) is greater than 10, print the first field.
sort: Sort lines of text files.sort names.txt: Sortsnames.txtalphabetically.sort -n numbers.txt: Sortsnumbers.txtnumerically.sort -r data.txt: Sortsdata.txtin reverse order.
uniq: Report or omit repeated lines. Often used aftersort.sort data.txt | uniq: Shows unique lines fromdata.txt.sort data.txt | uniq -c: Counts occurrences of each unique line.
Process Management:
ps: Report a snapshot of the current processes.ps aux: Shows all processes for all users in BSD format.ps -ef: Shows all processes in System V format.
top: Display dynamic real-time view of running processes.kill: Send a signal to a process (default is TERM, to terminate).kill 1234: Sends SIGTERM to process ID 1234.kill -9 1234: Sends SIGKILL to process ID 1234 (force kill).
nohup: Run a command immune to hangups, with output to anohup.outfile.nohup ./my_long_script.sh &: Runs the script in the background, immune to terminal closure.
System Information and Networking:
df: Report file system disk space usage.df -h: Human-readable output (e.g., KB, MB, GB).
du: Estimate file space usage.du -sh /var/log: Summarizes disk usage for/var/login human-readable format.
ping: Send ICMP ECHO_REQUEST to network hosts.ping google.com: Checks network connectivity to google.com.
ssh: OpenSSH remote login client.ssh user@remote_host: Connects toremote_hostasuser.
scp: Secure copy (remote file copy program).scp localfile.txt user@remote_host:/remote/path/: Copieslocalfile.txtto the remote server.
curl: Transfer data from or to a server.curl -O https://example.com/data.zip: Downloadsdata.zipfrom the URL.
Permissions and Ownership:
chmod: Change file mode bits (permissions).chmod +x script.sh: Adds execute permission toscript.sh.chmod 755 script.sh: Sets permissions to rwxr-xr-x.
chown: Change file owner and group.chown user:group filename: Changes owner touserand group togroupforfilename.
The power of these commands often lies in their ability to be combined. For instance, you can create a script to automatically back up your web server’s configuration files:
#!/bin/bash
BACKUP_DIR="/opt/backups/configs/$(date +%Y%m%d_%H%M%S)"
mkdir -p "$BACKUP_DIR"
cp -r /etc/nginx "$BACKUP_DIR"
cp -r /etc/apache2 "$BACKUP_DIR"
tar -czvf "$BACKUP_DIR.tar.gz" "$BACKUP_DIR"
rm -rf "$BACKUP_DIR"
echo "Backup created at $BACKUP_DIR.tar.gz"
This script creates a timestamped directory, copies configuration directories, compresses them into a tarball, and then cleans up the intermediate directory.
One subtle but incredibly useful aspect of shell scripting is understanding shell expansion, particularly brace expansion. While it might seem like a minor feature, it can drastically simplify repetitive tasks. For example, echo file_{a,b,c}.txt expands to file_a.txt file_b.txt file_c.txt without you having to type each one. This applies to ranges too: seq 1..5 generates numbers from 1 to 5. This capability extends to creating multiple directories or files with slightly varied names, saving significant typing.
As you become more comfortable, you’ll explore more advanced features like shell variables, conditional statements (if, else), loops (for, while), and functions. Understanding how to pipe output from one command to another is fundamental to efficient shell usage.
Next, you’ll likely want to learn about managing background processes and job control.