Book Image

Mastering Python Scripting for System Administrators

By : Ganesh Sanjiv Naik

Book Image

Mastering Python Scripting for System Administrators

By: Ganesh Sanjiv Naik

Overview of this book

Python has evolved over time and extended its features in relation to every possible IT operation. Python is simple to learn, yet has powerful libraries that can be used to build powerful Python scripts for solving real-world problems and automating administrators' routine activities. The objective of this book is to walk through a series of projects that will teach readers Python scripting with each project. This book will initially cover Python installation and quickly revise basic to advanced programming fundamentals. The book will then focus on the development process as a whole, from setup to planning to building different tools. It will include IT administrators' routine activities (text processing, regular expressions, file archiving, and encryption), network administration (socket programming, email handling, the remote controlling of devices using telnet/ssh, and protocols such as SNMP/DHCP), building graphical user interface, working with websites (Apache log file processing, SOAP and REST APIs communication, and web scraping), and database administration (MySQL and similar database data administration, data analytics, and reporting). By the end of this book, you will be able to use the latest features of Python and be able to build powerful tools that will solve challenging, real-world tasks

Preface

Who this book is for

What this book covers

To get the most out of this book

Free Chapter

Python Scripting Overview

Python Scripting Overview

Technical requirements

Python syntax compared to other programming languages

Python installation

Python interpreter

Understanding lists

Parsing command-line arguments

Decision making

Further reading

Debugging and Profiling Python Scripts

Debugging and Profiling Python Scripts

What is debugging?

Error handling (exception handling)

Debuggers tools

Debugging basic program crashes

Profiling and timing programs

Making programs run faster

Further reading

Unit Testing - Introduction to the Unit Testing Framework

Unit Testing - Introduction to the Unit Testing Framework

What is unittest?

Creating unit tests

Methods used in unit testing

Further reading

Automating Regular Administrative Activities

Automating Regular Administrative Activities

Accepting input by redirection, pipe, and input files

Handling passwords at runtime in scripts

Executing external commands and getting their output

Capturing output using the subprocess module

Prompting for passwords during runtime and validation

Reading configuration files

Adding logging and warning code to scripts

Putting limits on CPU and memory usage

Launching webbrowser

Using the os module for handling directory and files

Making backups (with rsync)

Further reading

Handling Files, Directories, and Data

Handling Files, Directories, and Data

Using the os module to work with directories

Copying, moving, renaming, and deleting data

Working with paths

Pattern matching files and directories

Metadata: data about data

Compressing and restoring

Using the tarfile module to create TAR archives

Using a tarfile module to examine the contents of TAR files

Further reading

File Archiving, Encrypting, and Decrypting

File Archiving, Encrypting, and Decrypting

Creating and unpacking archives

File encryption and decryption

Further reading

Text Processing and Regular Expressions

Text Processing and Regular Expressions

Regular expressions

Unicode strings

Further reading

Documentation and Reporting

Documentation and Reporting

Standard input and output

Information formatting

Further reading

Working with Various Files

Working with Various Files

Working with PDF files

Working with Excel files

Working with CSV files

Working with txt files

Further reading

Basic Networking - Socket Programming

Basic Networking - Socket Programming

The http package

The ftplib module

The urllib package

Further reading

Handling Emails Using Python Scripting

Handling Emails Using Python Scripting

Email message format

Adding HTML and multimedia content

POP3 and IMAP servers

Remote Monitoring of Hosts Over Telnet and SSH

Remote Monitoring of Hosts Over Telnet and SSH

The telnetlib() module

The subprocess.Popen() module

SSH using fabric module

SSH using the Paramiko library

SSH using the Netmiko library

Further reading

Building Graphical User Interfaces

Building Graphical User Interfaces

Introduction to GUI

Using a library to create a GUI-based application

Installing and using the Apache Log Viewer app

Further reading

Working with Apache and Other Log Files

Working with Apache and Other Log Files

Parsing complex log files

The need for exceptions

Tricks for parsing different files

Parsing other log files

Further reading

SOAP and REST API Communication

SOAP and REST API Communication

What is a RESTful API?

Working with JSON data

Further reading

Web Scraping - Extracting Useful Data from Websites

Web Scraping - Extracting Useful Data from Websites

What is web scraping?

Data extraction

Extracting information from Wikipedia

Further reading

Statistics Gathering and Reporting

Statistics Gathering and Reporting

Data visualization

Further reading

MySQL and SQLite Database Administrations

MySQL and SQLite Database Administrations

MySQL database administration

SQLite database administration

Further reading

Assessments

Chapter 1, Python Scripting Overview

Chapter 2, Debugging and Profiling Python Scripts

Chapter 3, Unit Testing – Introduction to the Unit Testing Framework

Chapter 4, Automating Regular Administrative Activities

Chapter 5, Handling Files, Directories, and Data

Chapter 6, File Archiving, Encrypting, and Decrypting

Chapter 7, Text Processing and Regular Expressions

Chapter 8, Documentation and Reporting

Chapter 9, Working with Various Files

Chapter 10, Basic Networking – Socket Programming

Chapter 11, Handling Emails Using Python Scripting

Chapter 12, Remote Monitoring of Hosts Over Telnet and SSH

Chapter 13, Building Graphical User Interfaces

Chapter 14, Working with Apache and Other Log Files

Chapter 15, SOAP and REST API Communication

Chapter 16, Web Scraping – Extracting Useful Data from Websites

Chapter 17, Statistics Gathering and Reporting

Chapter 18, MySQL and SQLite Database Administrations

Other Books You May Enjoy

Other Books You May Enjoy

Leave a review - let other readers know what you think

Customer Reviews

5 star

0

4 star

0

3 star

0

2 star

0

1 star

0

What is web scraping?

Web scraping is the technique used to extract information from websites. This technique is used to transform unstructured data into structured data.

The use of web scraping is to extract the data from the websites. The extracted information is saved as a local file on your system, and you can store it to database in a table format as well. The web scraping software accesses the World Wide Web (WWW) directly using HTTP or a web browser. This is an automated process implemented using a web crawler or a bot.

Scraping a web page involves fetching the page and then extracting the data. A web crawler fetches a web page. A web crawler is a mandatory component in web scraping. After fetching, extraction takes place. You can search, parse, save the data into tables, and reformat the page.