Name: Learning Concurrency in Python
Author: Elliot Forbes
ISBN: 9781787283169

微信读书书城Learning Concurrency in Python

首页我的书架登录

翻译完成

切换

Learning Concurrency in Python

Elliot Forbes

扉页
cover
+书签
Title Page
Copyright
Learning Concurrency in Python
Credits
About the Author
About the Reviewer
www.PacktPub.com
Why subscribe?
Customer Feedback
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Downloading the example code
Errata
Piracy
Questions
Speed It Up!
History of concurrency
Threads and multithreading
What is a thread?
Types of threads
What is multithreading?
Processes
Properties of processes
Multiprocessing
Event-driven programming
Turtle
Breaking it down
Reactive programming
ReactiveX - RxPy
Breaking it down
GPU programming
PyCUDA
OpenCL
Theano
The limitations of Python
Jython
IronPython
Why should we use Python?
Concurrent image download
Sequential download
Breaking it down
Concurrent download
Breaking it down
Improving number crunching with multiprocessing
Sequential prime factorization
Breaking it down
Concurrent prime factorization
Breaking it down
Summary
Parallelize It
Understanding concurrency
Properties of concurrent systems
I/O bottlenecks
Understanding parallelism
CPU-bound bottlenecks
How do they work on a CPU?
Single-core CPUs
Clock rate
Martelli model of scalability
Time-sharing - the task scheduler
Multi-core processors
System architecture styles
SISD
SIMD
MISD
MIMD
Computer memory architecture styles
UMA
NUMA
Summary
Life of a Thread
Threads in Python
Thread state
State flow chart
Python example of thread state
Breaking it down
Different types of threads
POSIX threads
Windows threads
The ways to start a thread
Starting a thread
Inheriting from the thread class
Breaking it down
Forking
Example
Breaking it down
Daemonizing a thread
Example
Breaking it down
Handling threads in Python
Starting loads of threads
Example
Breaking it down
Slowing down programs using threads
Example
Breaking it down
Getting the total number of active threads
Example
Breaking it down
Getting the current thread
Example
Breaking it down
Main thread
Example
Breaking it down
Enumerating all threads
Example
Breaking it down
Identifying threads
Example
Breakdown
Ending a thread
Best practice in stopping threads
Example
Output
Orphan processes
How does the operating system handle threads
Creating processes versus threads
Example
Breaking it down
Multithreading models
One-to-one thread mapping
Many-to-one
Many-to-many
Summary
Synchronization between Threads
Synchronization between threads
The Dining Philosophers
Example
Output
Race conditions
Process execution sequence
The solution
Critical sections
Filesystem
Life-critical systems
Shared resources and data races
The join method
Breaking it down
Putting it together
Locks
Example
Breaking it down
RLocks
Example
Breaking it down
Output
RLocks versus regular locks
Condition
Definition
Example
Our publisher
Our subscriber
Kicking it off
The results
Semaphores
Class definition
Example
The TicketSeller class
Output
Thread race
Bounded semaphores
Events
Example
Breaking it down
Barriers
Example
Breaking it down
Output
Summary
Communication between Threads
Standard data structures
Sets
Extending the class
Exercise - extending other primitives
Decorator
Class decorator
Lists
Queues
FIFO queues
Example
Breaking it down
Output
LIFO queues
Example
Breaking it down
Output
PriorityQueue
Example
Breakdown
Output
Queue objects
Full/empty queues
Example
Output
The join() function
Example
Breakdown
Output
Deque objects
Example
Breakdown
Output
Appending elements
Example
Breaking it down
Output
Popping elements
Example
Breaking it down
Output
Inserting elements
Example
Breaking it down
Output
Rotation
Example
Breaking it down
Output
Defining your own thread-safe communication structures
A web Crawler example
Requirements
Design
Our Crawler class
Our starting point
Extending the queue object
Breaking it down
Output
Future enhancements
Conclusion
Exercise - testing your skills
Summary
Debug and Benchmark
Testing strategies
Why do we test?
Testing concurrent software systems
What should we test?
Unit tests
PyUnit
Example
Output
Expanding our test suite
Unit testing concurrent code
Integration tests
Debugging
Make it work as a single thread
Pdb
An interactive example
Catching exceptions in child threads
Benchmarking
The timeit module
Timeit versus time
Command-line example
Importing timeit into your code
Utilizing decorators
Timing context manager
Output
Profiling
cProfile
Simple profile example
The line_profiler tool
Kernprof
Memory profiling
Memory profile graphs
Summary
Executors and Pools
Concurrent futures
Executor objects
Creating a ThreadPoolExecutor
Example
Output
Context manager
Example
Output
Maps
Example
Output
Shutdown of executor objects
Example
Output
Future objects
Methods in future objects
The result() method
The add_done_callback() method
The .running() method
The cancel() method
The .exception() method
The .done() method
Unit testing future objects
The set_running_or_notify_cancel() method
The set_result() method
The set_exception() method
Cancelling callable
Example
Output
Getting the result
Example
Output
Using as_completed
Example
Output
Setting callbacks
Example
Output
Chaining callbacks
Exception classes
Example
Output
ProcessPoolExecutor
Creating a ProcessPoolExecutor
Example
Output
Context Manager
Example
Output
Exercise
Getting started
Improving the speed of computationally bound problems
Full code sample
Output
Improving our crawler
The plan
New improvements
Refactoring our code
Storing the results in a CSV file
Exercise - capture more info from each page crawl
concurrent.futures in Python 2.7
Summary
Multiprocessing
Working around the GIL
Utilizing sub-processes
Example
Output
The life of a process
Starting a process using fork
Spawning a process
Forkserver
Daemon processes
Example
Breaking it down
Output
Identifying processes using PIDs
Example
Output
Terminating a process
Example
Getting the current process
Subclassing processes
Example
Output
Multiprocessing pools
The difference between concurrent.futures.ProcessPoolExecutor and Pool
Context manager
Example
Output
Submitting tasks to a process pool
Apply
Apply_async
Map
Map_async
Imap
Imap_unordered
Starmap
Starmap_async
Maxtasksperchild
Communication between processes
Pipes
Anonymous pipes
Named pipes
Working with pipes
Example
Handling Exceptions
Using pipes
Multiprocessing managers
Namespaces
Example
Queues
Example
Output
Listeners and clients
Example
The Listener class
The Client class
Output
Logging
Example
Communicating sequential processes
PyCSP
Processes in PyCSP
Output
Summary
Event-Driven Programming
Event-driven programming
The event loop
Asyncio
Getting started
Event loops
The run_forever() method
The run_until_complete() method
The stop() method
The is_closed() method
The close() function
Tasks
Example
The all_tasks(loop=None) method
The current_tasks() function
The cancel() function
Task functions
The as_completed(fs, *, loop=None, timeout=None) function
The ensure_future(coro_or_future, *, loop=None) function
The wrap_future(future, *, loop=None) function
The gather(*coroes_or_futures, loop=None, return_exceptions=False) function
The wait() function
Futures
Example
Output
Coroutines
Chaining coroutines
Output
Transports
Protocols
Synchronization between coroutines
Locks
Queues
Events and conditions
Semaphores and BoundedSemaphores
Sub-processes
Debugging asyncio programs
Debug mode
Twisted
A simple web server example
Gevent
Event loops
Greenlets
Simple example-hostnames
Output
Monkey patching
Summary
Reactive Programming
Basic reactive programming
Maintaining purity
ReactiveX, or RX
Installing RxPY
Observables
Creating observers
Example
Example 2
Breaking it down
Output
Lambda functions
Example
Breaking it down
On_next, on_completed, and on_error in lambda form
Output
Operators and chaining
Filter example
Breaking it down
Chained operators
The different operators
Creating observables
Transforming observables
Filtering observables
Error-handling observables
Hot and cold observables
Emitting events
Example
Breaking it down
Output
Multicasting
Example
Output
Combining observables
Zip() example
Output
The merge_all() operator
Output
Concurrency
Example
Output
PyFunctional
Installation and official docs
Simple example
Output
Streams, transformations, and actions
Filtering lists
Output
Reading/writing SQLite3
Compressed files
Parallel execution
Summary
Using the GPU
Basic reactive programming
Maintaining purity
ReactiveX, or RX
Installing RxPY
Observables
Creating observers
Example
Example 2
Breaking it down
Output
Lambda functions
Example
Breaking it down
On_next, on_completed, and on_error in lambda form
Output
Operators and chaining
Filter example
Breaking it down
Chained operators
The different operators
Creating observables
Transforming observables
Filtering observables
Error-handling observables
Hot and cold observables
Emitting events
Example
Breaking it down
Output
Multicasting
Example
Output
Combining observables
Zip() example
Output
The merge_all() operator
Output
Concurrency
Example
Output
PyFunctional
Installation and official docs
Simple example
Output
Streams, transformations, and actions
Filtering lists
Output
Reading/writing SQLite3
Compressed files
Parallel execution
Summary
Choosing a Solution
Libraries not covered in this book
GPU
PyGPU
Event-driven and reactive libraries
Tornado
Flask
Celery
Data science
Pandas
Matplotlib
TensorFlow
Designing your systems
Requirements
Functional requirements
Non-functional requirements
Design
Computationally expensive
Event-heavy applications
I/O-heavy applications
Recommended design books
Software Architecture with Python
Python: Master the Art of Design Patterns
Research
Summary

简介

版权

Learning Concurrency in Python