Index

A

• Allinea MAP / The big picture
  • reference link / Debugging and monitoring
• alternatives, Celery
  • Python-RQ / Celery alternatives – Python-RQ
  • Pyro / Celery alternatives – Pyro
• Amazon EC2 / Establishing a multimachine environment
• Amazon Elastic Beanstalk (EB)
  • about / Cloud computing and AWS, Amazon elastic beanstalk
  • reference link / Amazon elastic beanstalk
• Amazon Elastic Block Store (EBS) / Cloud computing and AWS
• Amazon Elastic Compute Cloud (EC2) / Cloud computing and AWS
• Amazon Machine Image (AMI) / Creating an EC2 instance
• Amazon S3
  • about / Cloud computing and AWS
  • data, storing / Storing data in Amazon S3
• Amdahl's law / Amdahl's law, The first two chapters
• asynchronous programming
  • example / An asynchronous example
  • reference link / An asynchronous example
• AWS
  • and cloud computing / Cloud computing and AWS
• AWS account
  • creating / Creating an AWS account
  • reference link / Creating an AWS account
• AWS management console
  • reference link / Creating an AWS account

C

• Celery
  • installation link / Installing Celery
  • installing / Installing Celery
  • installation, testing / Testing the installation
  • about / A tour of Celery
  • complex applications / More complex Celery applications
  • using, in production / Celery in production
• challenges, distributed system developers
  • time / Common problems – clocks and time
  • clock synchronization / Common problems – clocks and time
  • software environments / Common problems – software environments
  • permissions / Common problems – permissions and environments
  • environments / Common problems – permissions and environments
  • hardware resources, availability / Common problems – the availability of hardware resources
  • development environment / Challenges – the development environment
• cloud
  • and HPC world / The cloud and the HPC world
• cloud computing
  • and AWS / Cloud computing and AWS
• CloudStack
  • reference link / Creating a private cloud
• complain_about / Coroutines
• complex applications, Celery / More complex Celery applications
• compute nodes
  • about / Your typical HPC cluster
• computer farms / Distributed computing
• Coordinated Universal Time (UTC) / Common problems – clocks and time
• coroutines
  • about / Coroutines
• cProfile / The big picture
• CPython / Multiple threads
• Cray systems
  • about / Your typical HPC cluster
• CUDA / Distributed computing
• Cython / The tools

D

• DAGMan (Directed Acyclic Graph Manager)
  • about / Running a Python job using HTCondor
• Darshan
  • reference link / Debugging and monitoring
• data trains / The first two chapters
• DDT / The big picture
• debugging
  • defining / Debugging
  / Debugging and monitoring
• distributed applications
  • debugging / The big picture
• distributed computing
  • about / Distributed computing
• Distributed Resource Management Application API (DRMAA)
  • about / Running a Python job using HTCondor
• distributed system developers
  • challenges / Common problems – clocks and time
• Docker
  • reference link / Common problems – software environments

E

• EC2 instance
  • creating / Creating an EC2 instance
• embarrassingly parallel / Amdahl's law
• Eucalyptus
  • reference link / Creating a private cloud
  / The cloud and the HPC world
• event loop / Parallel computing

F

• flower
  • reference link / Celery in production, A useful strategy – logging everything

G

• Ganglia
  • reference link / Debugging and monitoring
• Global Interpreter Lock (GIL) / Multiple threads
• Grid Engine
  • about / Job schedulers
• Gustafson's law / Amdahl's law, The first two chapters

H

• homebrew
  • reference link / Installing Celery
• HPC cluster
  • defining / Your typical HPC cluster
• HPC world
  • and cloud / The cloud and the HPC world
• HTCondor
  • about / Your typical HPC cluster
  • used, for running Python job / Running a Python job using HTCondor
  / Common problems – the availability of hardware resources
• HyperThreading / Testing the installation

I

• Intel VTune / The big picture
• iteration protocol / Coroutines

J

• job file
  • about / Running a Python job using HTCondor
• job scheduler
  • about / Your typical HPC cluster
• Job schedulers
  • defining / Job schedulers
• Jython / Multiple threads

L

• line_profile / The big picture
• LoadLeveler
  • about / Your typical HPC cluster
• logging strategy / A useful strategy – logging everything
• LSF
  • about / Job schedulers
• Luigi
  • reference link / The tools

M

• manual, HTCondor
  • URL / Running a Python job using HTCondor
• mixed paradigm / The mixed paradigm
• monitoring / Debugging and monitoring
• multimachine environment
  • establishing / Establishing a multimachine environment
• Multiple Application, Multiple User (MAMU) nodes
  • about / Your typical HPC cluster
• multiple processes / Multiple processes
• multiple threads
  • about / Multiple threads
• multiprocessing / Parallel computing, Multiple processes
• multiprocess queues / Multiprocess queues
• multithreaded programming / Parallel computing

N

• negotiation cycle
  • about / Job schedulers
• Netflix Chaos Monkey
  • reference link / A useful strategy – simulating components
• Network Time Protocol (NTP) / Common problems – clocks and time

O

• OpenAc / Distributed computing
• OpenCL / Distributed computing
• OpenStack
  • reference link / Creating a private cloud

P

• parallel applications
  • developing, issues / Closing thoughts
• parallel computing / Parallel computing
• PBS Pro
  • about / Your typical HPC cluster
  / Common problems – the availability of hardware resources
• pdb debugger / The big picture
• Portable Batch System (PBS)
  • used, for running Python job / Running a Python job using PBS
  • URL / Running a Python job using PBS
• POSIX / Multiple threads
• Principles of Distributed Computing
  • reference link / Where to go next
• private cloud
  • creating / Creating a private cloud
• PyPI / Common problems – software environments
• Pyro
  • reference link / Celery alternatives – Pyro
  • about / Celery alternatives – Pyro
• Python-RQ
  • reference link / Celery alternatives – Python-RQ
  • about / Celery alternatives – Python-RQ
• Python-specific tools
  • reference links / The tools
• Python job
  • running, HTCondor used / Running a Python job using HTCondor
  • running, Portable Batch System (PBS) used / Running a Python job using PBS

R

• RabbitMQ
  • reference link / Installing Celery
  • administration guide, reference link / Installing Celery
• race condition / Multiple threads
• Raspberry Pi
  • reference link / Establishing a multimachine environment
• Redis
  • reference link / Installing Celery

S

• Sentry
  • reference link / A useful strategy – logging everything, Debugging and monitoring
• service nodes
  • about / Your typical HPC cluster
• share-nothing architecture / Multiple processes
• shared memory
  • versus distributed memory / Shared memory versus distributed memory
• SLURM
  • about / Job schedulers
• software components
  • simulating / A useful strategy – simulating components
• SQLAlchemy
  • reference link / Installing Celery
• submission file
  • about / Running a Python job using HTCondor
• supervisord
  • reference link / Celery in production

T

• tools / The tools
• Torque
  • about / Your typical HPC cluster

V

• VirtualBox
  • reference link / Establishing a multimachine environment