Introduction to Sysadmin: Basic Shell and Command Line Utilities

2020-08-15 13:50:05 | #sysadmin

Tested On

  • Linux Ubuntu 20.04
  • Windows 10
  • macOS Catalina

The shell is software that allows us to interact with the computer/server through the command line interface. There will be no Graphical User Interface (GUI) with clickable buttons or mouse input- just a black screen and a blinking cursor, waiting for your text input.

It may seem daunting to have to memorize commands just to be able to navigate folders or move files around but, I assure you, it's very intuitive and will become second nature. Plus, it makes you feel like hackerman.

How to Use the Shell and the Command Line Interface

On Windows, the two most popular shell programs are DOS(command.com) and PowerShell. On Unix systems like Linux and macOS, we have bash(Bourne again shell"), and modern shells like zsh and fish shell. Commands vary depending on the operating system, so skip to the next section if you're on Windows 10.

Just keep in mind that 96.3% of the world's top 1 million servers run on Linux. Regardless of your laptop's operation system, knowing how to SSH in and find your way around a Linux server is quite invaluable.

Introduction to the Linux Shell

First, we'll cover the basic command line syntax. Start by opening Terminal in either Linux or macOS. The Command Line Interface (CLI) expects you to enter a command and then a series of arguments. For example, if you wanted to navigate to your home directory, you would type cd ~/ and hit enter. cd stands for "change directory". Then, to list the files in your current directory, you would type ls and hit enter. ls is short for "list". You might then see something similar to the following folders, printed to your console:

ls
  Desktop  Documents  Downloads   Music   Pictures   Public

If you pass the -l argument to the ls command, you'll get a long listing with more detailed information. Try it yourself by typing ls -l and hitting the enter key. You should see the same folders, formatted like so:

ls -l
total 52
drwxr-xr-x  2 user group 4096 Jun 28 16:17 Desktop
drwxr-xr-x  2 user group 4096 May 22 20:17 Documents
drwxr-xr-x 34 user group 4096 Aug 11 15:18 Downloads
drwxr-xr-x  2 user group 4096 May 22 20:17 Music
drwxr-xr-x  3 user group 4096 Jul  3 22:04 Pictures
drwxr-xr-x  2 user group 4096 May 22 20:17 Public

You can even pass a path to ls directly to get information about a specific folder. Try it yourself by typing ls Documents -l. And you aren't limited to paths in the current directory. You can pass paths at any level, like so: ls ~/Documents. As you type the path, hit tab, and the CLI will autocomplete with existing folders at that level. Hit tab as many times as you like to reorient yourself. This is the basic loop for navigation in the shell, so get used to relying on the tab key.

Another useful argument to pass into ls is -a which prints "all" files and folders, including hidden ones and folders that start with ".". Whenever you want to see a full listing of the arguments a command accepts, type in the command and pass in --help or -h. For example, type ls --help and hit enter. Then, try experimenting with those arguments. There's no need to memorize them all. You will naturally remember the commands and arguments you use most often. For others, rely on the --help argument.

For an even deeper dive into a command, such as ls you can use man to print its manual. For example, type man ls. You can even type man man to see the manual for the man command.

Introduction to Windows PowerShell

Windows PowerShell is a object-oriented scripting language, designed primarily for systems administrators, allowing them to automate tasks and functions. It is tightly integrated with the command line and easy to utilize with its simple syntax. Being OO, PowerShell returns objects with properties, not just text.

To launch PowerShell, search for "PowerShell" in the Windows start menu, and click the icon. When you see the following prompt, you can begin scripting.

Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

PS C:\Users\User>

PowerShell employs Cmdlets (Command lets) are lightweight commands that allows us to invoke various functions. They are formatted in the Verb-Noun syntax. For example:

Verb Description
Get Used to get/retrieve/list something
Set Used to define/update something
New Used to create something
Start Used to start/run/boot something
Stop Used to stop/terminate/kill something
Out Used to output/display something

So to print the contents of our current directory, we would type Get-ChildItem or gci or ls. That last command comes from Unix shell syntax, is short for "list", and makes it more intuitive for sysadmins that come from Linux or macOS. After typing in any of those commands, you should see something similar to the following (if you're in your home directory):

Get-ChildItem

Directory: C:\Users\User

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-r---        2/18/2019     09:29                Desktop
d-r---        3/12/2019     10:25                Documents
d-r---        2/02/2019     13:03                Downloads
d-r---         4/1/2019     23:33                Favorites
-ar---        2/12/2019     04:44             27 File.txt

You can even pass arguments to commands, to perform more complex functions. For example, if you were interested in listing the contents of a different directory, you could indicate the path like so: Get-ChildItem -Path C:\Users\User\Documents. This would instead, list the content of the "Documents" directory. And don't worry about having to know the precise path. As you type in the path, you can hit tab, which will autocomplete your input with contents relative to that location. Hit tab as many times as needed to get a bearing on your location.

To navigate directories, we can invoke Set-Location or sl or cd. That last command, again, carrying over from Unix systems. Type in Set-Location .. to navigate to your home directory, and then Set-Location Document to dive in. You can even type in a full path like Set-Location C:\Users\User\Documents and remember to hit tab as you go to autocomplete which directories are accessible to you.

How to Pipe Commands with the Shell

Did you know you could chain together commands with the | key? To explain this in simple terms, as you already know Unix commands operate with a series of inputs and outputs. When you type ls -l to the command line, that is an input. When the shell prints files and folders to the screen, that is an output. And this output can be fed into another command as an input. Let's elaborate on that concept with an example.

How to Pipe Commands with the Unix Shell on Linux and macOS

Before we can begin piping one command's output into another command, we need to get familiar with a command that can handle said output. Let's learn about grep, which stands for Global Regular Expression Print. In plain English, this command allows us to search through text. So if we wanted ot search for the word "Hello" in some text, we would type grep "Hello". If you were to type that now and hit enter, without feeding it any text, it would wait for you to type or copy and paste something for it to search through. Go ahead and do that. Type grep "Hello", hit enter, and then start typing phrases like "Hello world!" and notice how it highlights occurrences of the word "Hello". When you are done, hold ctrl and c to exit/quit.

Now let's feed some text input into grep using |. A real world example would be to search for files/folders inside a directory. We're going to list out files and folders, and then pipe that output into grep as input, which will filter the results down to what we want to search for. Type ls ~/ -l | grep "D" and you should see something similar to the following:

ls ~/ -l | grep "D"
4 Desktop
4 Documents
4 Downloads

You can even search, recursively, many folders deep by adding -r to grep. For example grep "D" -r. Try the full example with ls ~/ -l | grep "D" -r. Or try searching through a text file with grep -r "Some text" /path/to/file.txt.

Now that you know how piping two commands works, keep it in mind as you learn more commands. One day, you'll be able to pipe 3, 4 or an even higher number of inputs and outputs this way. For now, let's learn about some more real world commands.

How to Pipe Commands with PowerShell on Windows

We can understand how piping works by having PowerShell list files that contain text we want to search for. Since we're already familiar with Get-ChildItem, we will use the Cmdlet to scan the files of a directory, and pipe its output into Select-String as input. First, create a file, by typing Set-Content -Path C:\hello.txt -Value 'Hello, World!'. Then, type Get-ChildItem C:\*.txt -Recurse | Select-String -Pattern 'he'. For each instance of the word 'he', PowerShell will list the filename, line number, and line of text. We can pipe even further with the following command: Get-ChildItem C:\*txt -Recurse | Select-String -Pattern 'he' -List | Select Path, which just gives us the filepath of each file.

Most Useful Shell Commands

Most Useful Unix Shell Commands Every Developer Should Know

Command Description Example
pwd Stands for Print Working Directory pwd
cd Stands for Change Directory cd /path/to/folder
cp Short for Copy; Copies files or folders cp file1.txt file2.txt
touch Creates an empty file touch newfile.txt
mv Short for Move; Moves file to another location or renames a file mv file1.txt file2.txt
cat Short for Concatenate (used for printing file contents, creating or combining files) cat existing_file.txt
grep Short for Global Regular Expression Print (used for searching) grep -r "text" /path/to/file.txt
less Efficiently reads a text/file in chunks less file.txt
cmp Short for Compare; Compares 2 files, byte-by-byte cmp file1.txt file2.txt
history Prints a past shell commands used history
tail Reads the last 10 lines of a text file; Logs the output of a running process tail file.txt
mkdir Short for Make Directory mkdir new_directory
rmdir Short for Remove Directory rmdir directory
df Prints information about disk space df
ps Visualizes running processes ps aux
kill Terminates a running process kill -9 PID
service Invoke system-wide services sudo service restart nginx
man Short for Manual man ls

Most Useful Windows PowerShell Commands Every Developer Should Know

Command Description Aliases Example
Set-Location Changes the working location to a new location cd Set-Location -Path C:\Users\path\to\folder
Get-ChildItem Retrieves the items in one or more locations gci, ls, dir Get-ChildItem -Path C:\Users\path\to\folder
Get-Content Retrieves item content Get-Content -Path C:\Users\path\to\file.txt
Set-Content Writes new content or replaces existing file content sc Set-Content -Path C:\Users\path\to\file.txt -Value 'Hello, World'
Get-Process Retrieve process details gps, ps Get-Process
Get-Service Retrieve service details gsv Get-Service
Get-Help Displays details about PowerShell commands and concepts help, man Get-Help Get-Content
New-Item Creates new items ni New-Item -Path . -Name "file1.txt" -ItemType "file" -Value "Hello, world!"
Copy-Item Copies items cp Copy-Item "C:\Users\User\Document\file.txt" -Destination "C:\Users\User\Documents"
Move-Item Moves or renames items mv Move-Item -Path C:\file.txt -Destination C:\Users\User\new_file.txt
Remove-Item Deletes items rm Remove-Item C:\Users\*.*
Where-Object Filters items that meet a specified condition, based on property values Get-Service | Where-Object {$_.Status -eq "Stopped"}
Select-String Searches for text and patterns sls Select-String -Path .\*.txt -Pattern 'Hello'
Set-Execution Policy Changes PowerShell execution policies Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope LocalMachine

Exercises

Try to solve the following problems, using everything you've learned up to this point. Feel free to share solutions in the comments. Optimize each solution, as much as possible.

  1. Write a Linux shell command to copy all of the .txt files from one folder to another or the PowerShell equivalent

    Using Linux Terminal or macOS Terminal:

    cp /path/to/source_folder/*.txt /path/to/desitnation_folder

    Windows PowerShell equivalent of the cp command:

    Copy-item -Recurse "C:\OldFolder" -File -include "*.txt" -Destination "C:\NewFolder"
  2. Write a Linux shell command to create multiple empty files, or the PowerShell equivalent

    Expected Output: file1.txt file2.txt file3.txt file4.txt

    Using Linux Terminal or macOS Terminal:

    touch file{1..4}.txt

    Windows PowerShell Equivalent of the touch command:

    1..4 | ForEach-Object { New-Item -ItemType File -Name "file$_.txt" }
  3. Write a Linux shell command to append the contents of one file to another, or the PowerShell equivalent

    Input #1 (file1.txt):

    The quick brown fox jumped over the lazy dog.

    Input #1 (file2.txt):

    The early bird catches the worm.

    Expected Output (file2.txt):

    The early bird catches the worm.
    The quick brown fox jumped over the lazy dog.

    Using Linux Terminal or macOS Terminal:

    cat file1.tt >> file2.txt

    Windows PowerShell Equivalent of the cat >> command:

    Get-Content file1.txt | Add-Content file2.txt
  4. Write a Linux shell command to recursively delete all of the .txt files in a folder, or the PowerShell equivalent

    Using Linux Terminal or macOS Terminal:

    find . -type f -name '*.txt' -delete

    Windows PowerShell Equivalent:

    Get-ChildItem * -Include *.txt -Recurse | Remove-Item

Want To See More Exercises?

View Exercises

Comments

You must log in to comment. Don't have an account? Sign up for free.

Subscribe to comments for this post

Want To Receive More Free Content?

Would you like to receive free resources, tailored to help you reach your IT goals? Get started now, by leaving your email address below. We promise not to spam. You can also sign up for a free account and follow us on and engage with the community. You may opt out at any time.



Tell Us About Your Project









Contact Us

Do you have a specific IT problem that needs solving or just have a general IT question? Use the contact form to get in touch with us and an IT professional will be with you, momentarily.

Hire Us

We offer web development, enterprise software development, QA & testing, google analytics, domains and hosting, databases, security, IT consulting, and other IT-related services.

Free IT Tutorials

Head over to our tutorials section to learn all about working with various IT solutions.

Contact