Concurrent Programming Paradigms and Socket Programming in Python, Essays (high school) of History

Download Concurrent Programming Paradigms and Socket Programming in Python and more Essays (high school) History in PDF only on Docsity!

18CSC207J – Advanced Programming Practice

Parallel &

Concurrent

Programming

Paradigm

Explicit parallelism

• (^) Explicit Parallelism is characterized by the presence of explicit constructs in the programming language, aimed at describing (to a certain degree of detail) the way in which the parallel computation will take place.
• (^) A wide range of solutions exists within this framework. One extreme is represented by the ``ancient'' use of basic, low level mechanisms to deal with parallelism--like fork/join primitives, semaphores, etc--eventually added to existing programming languages. Although this allows the highest degree of flexibility (any form of parallel control can be implemented in terms of the basic low level primitives gif), it leaves the additional layer of complexity completely on the shoulders of the programmer, making his task extremely complicate.

Implicit Parallelism

• (^) Allows programmers to write their programs without any concern about the exploitation of parallelism. Exploitation of parallelism is instead automatically performed by the compiler and/or the runtime system. In this way the parallelism is transparent to the programmer maintaining the complexity of software development at the same level of standard sequential programming.
• (^) Extracting parallelism implicitly is not an easy task. For imperative programming languages, the complexity of the problem is almost prohibitively and allows positive results only for restricted sets of applications (e.g., applications which perform intensive operations on arrays.
• Declarative Programming languages, and in particular Functional and Logic languages, are characterized by a very high level of abstraction, allowing the programmer to focus on what the problem is and leaving implicit many details of how the problem should be solved.
• (^) Declarative languages have opened new doors to automatic exploitation of parallelism. Their focusing on a high level description of the problem and their mathematical nature turned into positive properties for implicit exploitation of parallelism.

Methods for parallelism Message Passing:

• (^) Each Processor has direct access only to its local memory
• (^) Processors are connected via high-speed interconnect
• (^) Data exchange is done via explicit processor-to-processor communication i.e processes communicate by sending and receiving messages : send/receive messages
• (^) Data transfer requires cooperative operations to be performed by each process (a send operation must have matching receive) Data Parallel:
• (^) Each process works on a different part of the same data structure
• (^) Processors have direct access to global memory and I/O through bus or fast switching network
• (^) Each processor also has its own memory (cache)
• (^) Data structures are shared in global address space
• Concurrent access to shared memory must be coordinate
• (^) All message passing is done invisibly to the programmer

Steps in Parallelism

• (^) Independently from the specific paradigm considered, in order to execute a program which exploits parallelism, the programming language must supply the means to: - (^) Identify parallelism, by recognizing the components of the program execution that will be (potentially) performed by different processors; - (^) Start and stop parallel executions; - (^) Coordinate the parallel executions (e.g., specify and implement interactions between concurrent components).

Parallel Programming Paradigm

• (^) Phase parallel
• (^) Divide and conquer
• (^) Pipeline
• (^) Process farm
• Work pool Note:
• (^) The parallel program consists of number of super steps, and each super step has two phases : computation phase and interaction phase

Phase Parallel Model

• (^) The phase-parallel model offers a paradigm that is widely used in parallel programming.
• (^) The parallel program consists of a number of supersteps, and each has two phases. - (^) In a computation phase, multiple processes each perform an independent computation C. - (^) In the subsequent interaction phase, the processes perform one or more synchronous interaction operations, such as a barrier or a blocking communication. - (^) Then next superstep is executed.

Process Farm & Work Pool Model

• (^) This paradigm is also known as the master-slave paradigm.
• (^) A master process executes the essentially sequential part of the parallel program and spawns a number of slave processes to execute the parallel workload.
• (^) When a slave finishes its workload, it informs the master which assigns a new workload to the slave.
• (^) This is a very simple paradigm, where the coordination is done by the master.
• (^) This paradigm is often used in a shared variable model.
• (^) A pool of works is realized in a global data structure.
• (^) A number of processes are created. Initially, there may be just one piece of work in the pool.
• (^) Any free process fetches a piece of work from the pool and executes it, producing zero, one, or more new work pieces put into the pool. The parallel program ends when the work pool becomes empty.
• (^) This paradigm facilitates load balancing, as the workload is dynamically allocated to free processes.

Parallel Program using Python

• (^) A thread is basically an independent flow of execution. A single process can consist of multiple threads. Each thread in a program performs a particular task. For Example, when you are playing a game say FIFA on your PC, the game as a whole is a single process, but it consists of several threads responsible for playing the music, taking input from the user, running the opponent synchronously, etc.
• Threading is that it allows a user to run different parts of the program in a concurrent manner and make the design of the program simpler.
• (^) Multithreading in Python can be achieved by importing the threading module. Example: import threading from threading import *

Parallel program using Process in Python

import multiprocessing def worker(num): print('Worker:', num) for i in range(num): print(i) return jobs = [] for i in range(1,5): p = multiprocessing.Process(target=worker, args=(i+10,)) jobs.append(p) p.start()

Concurrent Programming Paradigm

• (^) Computing systems model the world, and the world contains actors that execute independently of, but communicate with, each other. In modelling the world, many (possibly) parallel executions have to be composed and coordinated, and that's where the study of concurrency comes in.
• (^) There are two common models for concurrent programming: shared memory and message passing.
  • Shared memory. In the shared memory model of concurrency, concurrent modules interact by reading and writing shared objects in memory.
  • (^) Message passing. In the message-passing model, concurrent modules interact by sending messages to each other through a communication channel. Modules send off messages, and incoming messages to each module are queued up for handling

Issues Concurrent Programming Paradigm

Race Condition

import threading x = 0 # A shared value COUNT = 100 def incr(): global x for i in range(COUNT): x += 1 print(x) def decr(): global x for i in range(COUNT): x -= 1 print(x) t1 = threading.Thread(target=incr) t2 = threading.Thread(target=decr) t1.start() t2.start() t1.join() t2.join() print(x)