<sub>2025-04-04 Friday</sub> <sub>#unix #r-programming #rstudio #git #github </sub>
<sub>[[maps-of-content]] </sub>
# Unix Command Line Essentials
> [!success]- Concept Sketch: [[]]
> ![[]]
> [!abstract]- Quick Review
>
> **Core Essence**: Unix treats programs as executable files and uses permissions, environment variables, and specialized syntax to enable powerful command-line operations.
>
> **Key Concepts**:
>
> - Executables are files found via the PATH environment variable
> - File permissions control who can read, write, and execute files
> - Commands accept arguments to modify their behavior
> - Wildcards allow pattern-matching for multiple files
>
> **Must Remember**:
>
> - Always specify path for local executables (./my-program)
> - Use rm with wildcards extremely cautiously
> - Environment variables start with $ (like $PATH)
> - Pipes (|) connect commands by feeding output of one into another
>
> **Critical Relationships**:
>
> - Shell → interprets → Commands → modify → Files/System
> - PATH → determines where → Executables → are found
> - Arguments → modify → Command behavior
> - Pipes → connect → Command output → to next command input
> [!NOTE]- Code Reference
>
> | Command/Syntax | Purpose | Example |
> | ------------------------------- | ------------------------------------------------------ | ------------------------------------------------- |
> | **File and Program Location** | | |
> | `which` | Locate executable files in PATH | `which git` |
> | `./filename` | Run executable in current directory | `./my-program` |
> | **Directory Listing** | | |
> | `ls` | List directory contents | `ls` |
> | `ls -l` | List in long format with details | `ls -l` |
> | `ls -a` | Show all files (including hidden) | `ls -a` |
> | `ls -t` | Sort by modification time | `ls -t` |
> | `ls -r` | Reverse order of listing | `ls -r` |
> | `ls -lart` | Combined: long format, all files, reverse, time-sorted | `ls -lart` |
> | **File Removal** | | |
> | `rm` | Remove files | `rm file.txt` |
> | `rm -r` | Remove directories recursively | `rm -r directory/` |
> | `rm -f` | Force removal without confirmation | `rm -f file.txt` |
> | `rm -rf` | Force recursive removal (DANGEROUS) | `rm -rf temp/` |
> | **Environment Variables** | | |
> | `echo $VARIABLE` | Display value of environment variable | `echo $HOME` |
> | `env` | Display all environment variables | `env` |
> | `export VARIABLE=value` | Set environment variable for session | `export PATH=$PATH:/new/directory` |
> | **Wildcards** | | |
> | `*` | Wildcard: match any characters | `ls *.txt` |
> | `?` | Wildcard: match single character | `ls file?.txt` |
> | `[abc]` | Wildcard: match any character in brackets | `ls file[123].txt` |
> | `[a-z]` | Wildcard: match any character in range | `ls file[a-z].txt` |
> | **File Operations** | | |
> | `open` | (macOS) Open file with default app | `open document.pdf` |
> | `start` | (Windows) Open file with default app | `start document.pdf` |
> | `nano` | Simple terminal text editor | `nano file.txt` |
> | `ln -s` | Create symbolic link | `ln -s target link_name` |
> | **Archiving and Remote Access** | | |
> | `tar -czf` | Create compressed archive | `tar -czf archive.tar.gz files/` |
> | `tar -xzf` | Extract compressed archive | `tar -xzf archive.tar.gz` |
> | `ssh` | Secure shell connection to remote system | `ssh user@hostname` |
> | **Text Processing and Search** | | |
> | `grep` | Search for patterns in files | `grep "pattern" file.txt` |
> | **Getting Help** | | |
> | `man` | Display manual pages | `man ls` |
> | `--help` | Display command help | `ls --help` |
> | **Command Chaining (Pipes)** | | |
> | \| (pipe) | Connect output of one command to input of another | ls -la \| sort -nrk 5 \| head -n 5
>
## Introduction
This guide covers essential concepts for effectively using the Unix command line interface. You'll learn how Unix treats programs as files, manages permissions, handles environment variables, utilizes wildcards, and employs powerful commands to streamline your workflow. These fundamentals will help you navigate and manipulate Unix-based systems with confidence and precision.
## 1. Executables: Programs as Files
**In Unix, all programs are simply files.** These special files, called executables, can be run to perform operations on your system.
### How Unix Finds Programs
Unix locates executables through the PATH environment variable, which contains a colon-separated list of directories to search through.
> [!visual] Visual Note Guide
>
> **Core Concept**: PATH Variable Search Process **Full Description**: When you type a command, Unix searches through each directory listed in the PATH variable in order until it finds a matching executable file. If not found, you'll get "command not found." **Memorable Description**: "PATH is Unix's GPS for finding programs" **Visual Representation**: Draw a path with multiple folders, with Unix checking each folder sequentially with checkmarks/X marks. Show command typed at beginning and either found program or error at end.
### Finding Program Locations
- **which command**: Reveals the directory where an executable resides
bash
```bash
which git
# Output example: /usr/bin/git
```
- **Common executable locations**:
- `/usr/bin` (system programs)
- `/usr/local/bin` (user-installed programs)
- `/bin` (essential system binaries)
### Running Local Executables
When running a program in your current directory that's not in your PATH:
bash
```bash
# WRONG - Unix won't find it
my-program
# CORRECT - Specify the path explicitly
./my-program
```
> [!tip] Remember The dot (.) represents the current directory, and ./ tells Unix to look for the program right here.
## 2. File Types and Permissions
Unix uses a detailed permission system to control access to files.
### Understanding File Information
When you run `ls -l`, you see output like:
```
drwxr-xr-x 2 user group 4096 Oct 15 14:30 Documents
-rw-r--r-- 1 user group 217 Oct 15 14:32 notes.txt
```
The first character indicates file type:
- `-`: Regular file
- `d`: Directory
- `l`: Symbolic link
### Permission Structure
The permission string (e.g., `rwxr-xr-x`) breaks down into three sections:
- First 3 characters: Owner permissions
- Middle 3 characters: Group permissions
- Last 3 characters: Everyone else's permissions
Where:
- `r`: Read permission
- `w`: Write permission
- `x`: Execute permission (can run the file or access directory)
- `-`: Permission denied
> [!warning] Executable Permission Required Without execute permission (`x`), a program file cannot be run, even if it contains valid code!
## 3. Command Arguments
**Arguments modify command behavior** and are typically specified with dashes.
### Argument Syntax
bash
```bash
command -a # Single-letter argument with one dash
command --all # Word argument with two dashes
command -abc # Multiple single-letter arguments combined
```
### Common Arguments for Essential Commands
#### For ls (list files)
bash
```bash
ls -l # Long format with details
ls -a # Show all files (including hidden ones)
ls -t # Sort by time modified
ls -r # Reverse order
ls -lart # Combining multiple arguments
```
#### For rm (remove files)
bash
```bash
rm file.txt # Remove a file
rm -r directory # Remove directory and contents recursively
rm -f file.txt # Force removal without confirmation
rm -rf directory # Force recursive removal (USE WITH EXTREME CAUTION)
```
> [!warning] rm Safety `rm -rf` is extremely powerful and dangerous. It can delete your entire system without confirmation. Always double-check what you're removing.
> [!tip] Most commands accept `--help` to display available arguments.
## 4. Environment Variables and Shells
**Environment variables store settings** that affect your command-line experience.
### Key Environment Variables
- **$PATH**: Directories where Unix looks for executables
- **$HOME**: Your home directory location
- **$SHELL**: Your current shell program
### Viewing Variables
bash
```bash
echo $HOME # View a specific variable
env # View all environment variables
```
> [!visual]- Visual Note Guide
>
> **Core Concept**: Environment Variables **Full Description**: Environment variables are system-wide settings that affect program behavior, referenced with a $ prefix. **Memorable Description**: "$ variables = System settings" **Visual Representation**: Draw a container labeled "Environment" with key-value pairs inside. Show how programs access these with the $ prefix. Highlight PATH with multiple directories connected by colons.
### Shells
A shell is the program that interprets your commands. Common shells include:
- **bash**: Bourne Again SHell (most common)
- **zsh**: Z Shell
- **fish**: Friendly Interactive SHell
### Setting Environment Variables
In bash or similar shells:
bash
```bash
# Temporary (for current session only)
export PATH=$PATH:/new/directory
# Permanent (add to ~/.bashrc, ~/.bash_profile, or similar)
export PATH=$PATH:/new/directory
```
> [!tip] When modifying PATH, always include $PATH to append rather than replace the existing path.
## 5. Wildcards
**Wildcards let you match multiple files using patterns.**
### Common Wildcards
- *****: Matches any number of any characters
- **?**: Matches exactly one character
- **[abc]**: Matches any one character in the brackets
- **[a-z]**: Matches any one character in the range
### Practical Examples
```bash
ls *.txt # List all files ending with .txt
rm temp_*.log # Remove all files starting with temp_ and ending with .log
mv file_?.txt docs/# Move files like file_1.txt, file_2.txt to docs directory
```
> [!visual]- Visual Note Guide
>
> **Core Concept**: Wildcards **Full Description**: Wildcards are pattern-matching symbols that represent multiple characters or files in commands. **Memorable Description**: "* means 'anything goes', ? means 'just one'" **Visual Representation**: Draw a set of files with patterns showing what matches. Use * as a cloud covering multiple characters, ? as a single empty box, and draw arrows connecting patterns to matching filenames.
> [!warning] Wildcard Safety Always test wildcards with `ls` first to see which files will be affected before using with destructive commands like `rm`.
> [!case]- Case Application: Cleaning Up Project Files
>
> You're working on a web project with many temporary files:
>
>```
> index.html
> style.css
> temp_1.html
> temp_2.html
> temp_index.bak
> backup_2023.zip
> notes.txt
>
>
>```
>
> To clean up all temporary HTML files while preserving your main files:
>
>
>```bash
> # First, verify which files will be affected
> ls temp_*.html
>
> # If output shows just temp_1.html and temp_2.html as expected:
> rm temp_*.html
>
>```
>
> This pattern safely removes only the temporary HTML files.
## 6. Essential Commands to Learn
Beyond basics, these commands will enhance your Unix proficiency:
### File Operations
- **open** (macOS) or **start** (Windows): Open a file with its default application
- **nano**: Simple text editor for the terminal
- **ln**: Create links between files
- **tar**: Create or extract archive files
### Finding and Processing
- **grep**: Search for patterns within files
- **awk** and **sed**: Powerful text processing utilities
- **find**: Locate files by name, type, or other attributes
### Connectivity
- **ssh**: Connect securely to remote computers
- **scp**: Copy files securely between computers
> [!tip] Running `command --help` or `man command` will provide detailed information about any command.
## 7. Getting Help and Pipes
### Finding Help
Unix offers built-in documentation:
```bash
man ls # Display manual page for ls command
ls --help # Show help for ls command
```
> [!note] `man` may not be available in minimal Unix implementations like Git Bash, but `--help` usually works everywhere.
### Pipes: Connecting Commands
**Pipes (|) redirect output from one command to another**, enabling powerful command combinations.
```bash
ls -l | grep "Oct" # List files and filter for those containing "Oct"
ls -la | less # List all files and view the output page by page
```
> [!visual]- Visual Note Guide
>
> **Core Concept**: Pipes **Full Description**: Pipes connect commands by sending the output of one command as input to another command, allowing for complex data processing chains. **Memorable Description**: "Pipes are command assembly lines" **Visual Representation**: Draw commands as boxes connected by pipe symbols (|) with data flowing through from left to right. Show transformation of data at each step.
```mermaid
flowchart LR
A[Command 1] -->|Output| B(("|"))
B -->|Input| C[Command 2]
C -->|Output| D(("|"))
D -->|Input| E[Command 3]
```
> [!case]- Case Application: Finding Large Files
>
> You need to find the 5 largest files in a directory with many files:
>
>```bash
> # List all files with details, sort by size (largest first), show top 5
> ls -la | sort -nrk 5 | head -n 5
>
>```
>
> This creates a pipeline where:
>
> 1. `ls -la` lists all files with details
> 2. `sort -nrk 5` sorts numerically (-n), in reverse (-r), by the 5th column (-k 5) which is file size
> 3. `head -n 5` shows only the first 5 results
>
> The pipe symbols connect these three commands to achieve a complex operation with a simple one-liner.
## Summary: Unix Command Line Fundamentals
The Unix command line gives you powerful control through a consistent philosophy:
- Programs are files that can be executed
- Everything is controlled through permissions
- Arguments modify command behavior
- Environment variables configure your shell experience
- Wildcards let you work with multiple files efficiently
- Pipes connect commands into powerful workflows
Understanding these fundamentals allows you to build complex operations from simple components, following the Unix philosophy of "do one thing well" and combining small tools to solve bigger problems.
> [!important] Most Important Takeaway
>
> Unix power comes from combining simple tools in creative ways. Master the basics of file management, understand how commands communicate with each other through pipes, and learn to use wildcards safely. With these skills, you can accomplish complex tasks efficiently and avoid dangerous mistakes.
--
Reference:
- Data Science, HarvardX