Navigating and searching the Linux filesystem is one of the most essential skills the developers, system administrators, and penetration testers will need to master in order to realize the full potential of their bash consoles and utilities. To properly master this skill, you will need a good understanding of the organization of your host operating system though it is a little out of context of this book to have a thorough discussion of the Kali Linux operating system's inner workings and organization.
Navigating a filesystem requires the use of a sample collection of the tools and utilities. Here's a breakdown of these tools:
Command name |
Common name |
Purpose |
---|---|---|
|
Change Directory | |
|
List | |
|
Print Working Directory | |
|
Find |
This locates or verifies the existence of a file based on a the values of certain attributes |
Navigating directories is popularly done by using the cd
command, which is probably one of the simplest commands to use. All you need to do is supply the directory you wish to change to and cd
will do the rest. It also has very useful shorthands to speed up the most common tasks users perform when navigating their filesystems.
The following is what the command usage specification looks like:
cd [ -L | -P ] [directory]
In the syntax specification, [directory]
is the directory you wish to change your current working directory to and [-L|-P]
may be any one of the following:
-L
: When changing directory, symbolic links should not be respected. The current directory will be changed to include the name of the symbolic link and not its target. This is described in documentation as making the symbolic link logical, since it forces the name of the symbolic link to be treated as logical element in the path being set as the working directory.Note
Symbolic links are constructs on a filesystem that allow one file or directory to act purely as a reference to another file. These links affect the way path resolution occurs, since in some situations when a symbolic link is followed, it will allow one path to direct the current directory to a file represented by another name, as opposed to a pathname resolving strictly as it is named.
-P
: This is the opposite of the-L
command. This specifies that should the file being set as the current directory be symbolic link, it should be resolved completely before being set as the current directory. This means if you visit a symbolic link, your current path will not reflect the name of the symbolic link you used to reach it, unless of course if the link has the same name as its target.
The following is a typical usage example of the cd
command:
cd /
The preceding command will change your current directory to the root directory, which is named /
; everything hosted on your filesystem is usually reachable from this directory.
The following are some more examples:
cd ~
: This command is used to navigate to the current user's home directorycd ../
: This command is used to navigate to the directory directly above the current one
In the preceding command, one can have cd
navigate an arbitrary number of directories above the current one, for instance, by supplying it a command as follows:
cd ../../../../../
The following are some other commands that can be used to navigate to different directories:
cd .
: This command is used to navigate to the current directorycd –
: This command is used to navigate to the previous directorycd --
: This command is used to navigate to the second-last directory
To see whether you have indeed changed your current working directory to the one you've specified, you can invoke the pwd
command that will print your working directory. The syntax for the pwd
command is as follows:
pwd [-L|-P] [--help] [--version] pwd [--logical | --physical ]
The –L
or --logical
and –P
or --physical
invocation options serve the same purpose as in the cd
command.
It's not enough to just move between directories. You will eventually want to find out what's inside these directories. You can do this by using the ls
command.
The following is the usage specification for the ls
command—adapted from its man page:
ls [-aAlbBCdDfFghHiIklLmNopqQrRsStTuvwxXZ1] [FILE/DIRECTORY]
The previous command specification is another popular Linux/Unix convention. It's a shorthand to specify that any of the letters appearing in the brackets can be specified as part of the command invocation. Also, any number of them may be specified at the same time. For instance, consider the following commands:
ls –Ham ls –and ls –Rotti
According to the command specification, they are all acceptable ways to use the ls
command. Whether or not any of these will actually do something useful depends on how each switch affects the ls
command's behavior. You should keep in mind that some options may have opposing effects or certain combinations may have no effect, like a general note when reading usage specifications such as the one for ls
.
The [FILE]
or [DIRECTORY]
argument would be any path or file at which you wish to fire ls
. Without any arguments, ls
will list the current working directory's entries.
Note
A switch is a popular jargon for the options, that is, anything directly following the hyphen, specified as part of the command invocation. For example, –l
is a switch.
Here's what some of the switches do—we will only discuss some of the most important switches here for the sake of brevity. Keep in mind that the ls
command lists directory contents, so all its options will be focused on organizing and presenting a given directory's contents in a specified way.
The following are some of the ls
command's invocation options:
-a –-all
: This displays all the directory entries and does not omit directories or file starting with "." in their names.-d –directory
: This lists the directory entries and not their contents. This will also forcels
not to dereference symbolic links.-h
: This prints sizes in human-readable format, for instance, instead of the number of bytes only it will display file sizes in gigabytes, kilobytes, or megabytes where applicable.-i
: This prints the inode number of each file.Note
Inodes or
i-nodes
are data structures assigned to files that represent detailed information about their access rights, access times, sizes, owners, and the location of the file on the actual block devices—the physical medium hosting the file—as well as other important housekeeping-orientated details.-R –-recursive
: This recursively lists directory contents. This tellsls
to nest down all the levels of the specified path and enumerate all the reachable file paths, instead of stopping once the working directory is listed—as is the default.-x
: This sorts entries alphabetically by extension, for example, all PDFs after MP3s.
The following are some examples of these options in action. For instance, if you'd like to say sort a bunch of files by their size, while displaying human-readable file sizes and all the access rights and creation times—which seems like a lot of work—you would run the following command:
ls –alSh
You're output could look something like the following screenshot:
Another very useful example would be checking the volume of logins to the system. This can be done by looking at the output of the following command:
ls –alSh /var/log/auth*
Generally, keeping track of the contents of the /var/log/
directory will always be a good way to grab a good synopsis of the activity on a system.
Another important skill is being able to find resources on your filesystem in a compact yet powerful way. One of the ways you can do this is by using the aptly named find
command. The following command is how find
works:
find [-H] [-L] [-P] [-D debugopts] [-0level] [path…] [expression]
You can find out more about the find
command by checking out the man file on it. This can be done by executing the following command:
man 1 find.
This was discussed in the Getting help from the man pages section earlier in this chapter.
Moving on, the first three switches, namely, -H
, -L
, and –P
, all control the way symbolic links are treated. The following list tells what they do:
-H
: This tellsfind
not to follow symbolic links. Symbolic links will be treated as normal files and will not resolve them to their targets. Putting it simply, if a directory contains a symbolic link, the symbolic link will be treated as any other file. This does not affect symbolic links that form part of the selection criteria; these will be resolved.-L
: This forcesfind
to follow symbolic links in the directories being processed.-P
: This forcesfind
to treat symbolic links as normal files. If a symbolic link is encountered during execution,find
will inspect the properties of the symbolic link itself and not its target.
The –D
switch is used to allow find
to print debug information if you need to know a little about what find
is up to while it's searching for the files you want. -0level
controls how find
optimizes tests and it also allows you to reorder some tests. The level
part can be specified as any number between 0 and 3 (inclusive).
The [path...]
part of the argument is used to tell find where to look for files. You can also use the .
and ..
shorthands to specify the current and directory one level up respectively, as with the cd
command.
The next argument, or rather group of arguments, is quite an important one: the [expression]
. It consists of all the arguments that control the following:
Options: This tells what kind of files
find
should look forTests: This tells how to identify the files it is looking for
Actions: This tells what
find
should do with the files once they are found
The following is the structural breakdown of the find
expression:
[expression] := [options][[test][OPERATOR][test][OPERATOR]...][actions] [options] := [-d][-daystart][-depth][-follow][-help]... [tests] := [-amin n][-atime file][-cmin n][-cnewer file]... [OPERATOR] := [()][!][-not][-a][-and][-or]... [actions] := [-delete][-exec command [;|{} +]][-execdir command]...
Note
The previous code only serves as information about the structure of the expression, to let you know which options go where. Many of the switches for each section have been omitted for brevity. The :=
characters mean that whatever is on the left-hand side is defined by whatever is defined on the right-hand side.
So now that you know where everything goes, let's look at what some of these arguments do. The find
command has quite a number of very powerful options and operational modes, and one could quite literally write an entire book about find
itself. So to make sure you don't get short changed—buying a book about "command line hacking" and instead learning only about find
—we will only discuss some of the most common options and arguments penetration testers, system administrators, and developers use. The rest of the find
command's power can be learned from the Linux manual files.
The following is a summary of some of the find
command's possible arguments for options, tests, and actions.
The following are some of the options arguments you can use with find
:
-maxdepth n
: This specifies that tests must only be applied to entries in directories at mostn
levels below the current directory. This option is useful if you're searching through directories that have a similar structure. For instance, if each directory below the one you're searching has something like alib
directory that contains uninteresting files, you can skip all such directories by specifying this option.-mindepth n
: This specifies that tests should only be applied to files at depth of at leastn
directories lower than the specified path.-daystart
: This forces any–amin
,-atime
,-cmin
,-ctime
, or equivalent time-related tests to use the time starting from the beginning of the current day, rather than 24 hours ago—as is the default behavior.-mount
: This forbidsfind
from traveling into other filesystems.+n
: This indicates the specified argument is to be compared as greater than, or at leastn
-n
: This indicates the specified argument is to be compared as less than or at mostn
n
: This forces find to comparen
as is, and the attribute must have the exact value ofn
Tests are applied to a file and either return true
or false
: either the file being tested has the desired attribute or it doesn't. More than one test can also be supplied, in which case a logical combination—which can also be specified—is applied. By default, if no Boolean is supplied to combined to tests, a logical AND is assumed. This means both tests must be true for the file to be found or reported. The following are some of the file testing options:
-amin n
: This specifies that the last access time of the file should be n minutes ago. For example:-amin 20
: This means the file must have been accessed exactly 20 minutes ago-amin +35
: This means the file must have been accessed at most 35 minutes ago
-atime n
: This specifies that the file should have been access n*24 hours ago, meaning n days. Any fractional part of this number is ignored.-mmin n
: This specifies that the file should have been modified n minutes ago.-mtime n
: This is the same as–atime
, except it matches against the files modified time.-executable | -readable | -writable
: This matches any file that has access rights indicating that the file is executable, readable, or writable, respectively.-perm
: This mode specifies that the file group should be name. The–perm
option offers a myriad of different ways to specify the access mode being tested, here's how it works.Note
The access mode bits can be prefixed with anyone of the following:
mode
: This means no prefix and the mode must be matched exactly.-mode
: This means the file's mode must have at least the specified bits set. This will match files with other bits set as long as the specified bits are set as well./mode
: This means that any of the specified bits must be set for the file.
The mode itself can also be specified in two different ways, symbolically using characters to indicate user types and access modes or the octal decimal mode specification.
-iname nAmE
: This specifies that the name of the file should matchnAmE
if the case is ignored; in other words, case-insensitive name matching.-regex pattern
: This matches the specified pattern as a regular expression against the file's pathname. Your regular expression must describe the entire pathname.Note
Regular expressions are merely ways to describe a set of strings with a specified number of properties in common. If you want to describe a string, you must be able to detail all the properties of the string from beginning to the end. If you don't describe a single character in some or other way, the regular expression won't match!
Regular expression are in themselves a language, for instance, you could write a regular expressions to describe regular expressions! This means you will need to know how to speak this language in order to use regular expressions properly. To find out how to do this, see the Further reading section at the end of this chapter.
The following are a few simple examples of the –regex
option's usage:
Find all the files directly under the
/etc/
directory that start with the letterp
and end in anything using the following command:find / -regex '^/etc/p[a-z]*$'
Find all the files on the filesystem that are called configuration, ignoring case, and accommodating abbreviations such as
confg
,cnfg
, andcnfig
using the following command:find / -regex '^[/a-z_]*[cC]+[Oo]*[nN]+[fF]+[iI]*[gF]+$'
See the following screenshot for a practical example of the previous command:
The regular expression used here must describe the entire file's path! For instance, consider the difference in results between the following two regular expressions:
find / -regex '^[/a-z_]*/$' #matches only the / directory find / -regex '^[/a-z_]*/*$' #matches everything reachable from the / directory!
The following are some of the action arguments you can use with find
:
-delete
: This action forcesfind
to delete any file for which the specified test returnstrue
. For instance, consider the following command:find / -regex '^/[a-z_\-]*/[Vv][iI][rR][uS]*$' –delete
This command will find and delete anything reachable one level from the root that has a name such as 'virus'—case-insensitive.
-exec
: This allows you to specify an arbitrary command to execute on all files that match.The way this argument works is to build a command line—which is probably passed to some
exec*
type system call—using the results of thefind
operation for every result. Thefind
command will use any argument after the–exec
switch as a literal argument to the command being executed and any instance of the{}
chars as a placeholder for the name of the file, until a;
character is encountered.For instance, consider the following as the
–exec
argument:find /etc/ -maxdepth 1 -name passwd -exec stat {} \;
The actual command line(s) that will be run will look something like the following command, since the only file that will match will be
/etc/passwd
:stat /etc/passwd
See the following screenshot for a comparison of the
stat
andfind –exec
commands:-execdir
: This works the same way–exec
does, except it will isolate execution of the specified command to the directory of the match file. This works great if you'd like to execute commands based on the contents of a directory that has certain files. For instance, you may want to edit all the.bashrc
files for users that don't have.vimrc
, which is a configuration script for the VIM text editor. We will discuss more about the.bashrc
code later.-print0
: This prints the file's full name to standard output. This argument also has the added benefit of terminating filenames with a NULL character, or0x0
character, so as to allow filenames to contain newlines. It also helps make sure that any program interpreting the output of find will be able to determine the separation between filenames, as they will be strictly separated by NULL characters.-ls
: This lists the current file by executingls –dils
, and the output is printed to standard output. The–dils
option makes sure that the directory entries are printed. If the matched file is a directory, then inode is printed, and the entry appears in thels
command's long listing format as well as the size of the file.
There are a couple more actions you can specify. For the rest of them, please see the manual file on the find
command, which you can access using the man find
command.
So as far as searching your filesystem for files, directories, or generally any other interesting things, that's pretty much it. The next fundamental skill you'll need to master is redirecting output from one command to another.