What is the difference between concurrent programming and parallel programming?

difference between sequential and concurrent programming
concurrent and parallel programming material
concurrent and parallel programming ppt
concurrent vs parallel vs distributed
concurrent programming in os
concurrent execution in os
parallel programming in ppl
concurrency is not parallelism

What is the difference between concurrent programming and parallel programing? I asked google but didn't find anything that helped me to understand that difference. Could you give me an example for both?

For now I found this explanation: http://www.linux-mag.com/id/7411 - but "concurrency is a property of the program" vs "parallel execution is a property of the machine" isn't enough for me - still I can't say what is what.

If you program is using threads (concurrent programming), it's not necessarily going to be executed as such (parallel execution), since it depends on whether the machine can handle several threads.

Here's a visual example. Threads on a non-threaded machine:

        --  --  --
     /              \
>---- --  --  --  -- ---->>

Threads on a threaded machine:

     ------
    /      \
>-------------->>

The dashes represent executed code. As you can see, they both split up and execute separately, but the threaded machine can execute several separate pieces at once.

General concepts: concurrency, parallelism, threads and processes , In programming, concurrency is the composition of independently executing processes, while parallelism is the simultaneous execution of (possibly related) computations. On the other hand, a well-written concurrent program might run efficiently in parallel on a multiprocessor. Concurrent programming is in a general sense to refer to environments in which the tasks we define can occur in any order. One task can occur before or after another, and some or all tasks can be performed at the same time. Parallel programming is to specifically refer to the simultaneous execution of concurrent tasks on different processors. Thus, all parallel programming is concurrent, but not all concurrent programming is parallel.

Concurrent programming regards operations that appear to overlap and is primarily concerned with the complexity that arises due to non-deterministic control flow. The quantitative costs associated with concurrent programs are typically both throughput and latency. Concurrent programs are often IO bound but not always, e.g. concurrent garbage collectors are entirely on-CPU. The pedagogical example of a concurrent program is a web crawler. This program initiates requests for web pages and accepts the responses concurrently as the results of the downloads become available, accumulating a set of pages that have already been visited. Control flow is non-deterministic because the responses are not necessarily received in the same order each time the program is run. This characteristic can make it very hard to debug concurrent programs. Some applications are fundamentally concurrent, e.g. web servers must handle client connections concurrently. Erlang, F# asynchronous workflows and Scala's Akka library are perhaps the most promising approaches to highly concurrent programming.

Multicore programming is a special case of parallel programming. Parallel programming concerns operations that are overlapped for the specific goal of improving throughput. The difficulties of concurrent programming are evaded by making control flow deterministic. Typically, programs spawn sets of child tasks that run in parallel and the parent task only continues once every subtask has finished. This makes parallel programs much easier to debug than concurrent programs. The hard part of parallel programming is performance optimization with respect to issues such as granularity and communication. The latter is still an issue in the context of multicores because there is a considerable cost associated with transferring data from one cache to another. Dense matrix-matrix multiply is a pedagogical example of parallel programming and it can be solved efficiently by using Straasen's divide-and-conquer algorithm and attacking the sub-problems in parallel. Cilk is perhaps the most promising approach for high-performance parallel programming on multicores and it has been adopted in both Intel's Threaded Building Blocks and Microsoft's Task Parallel Library (in .NET 4).

List of concurrent and parallel programming languages, A system is said to be parallel if it can support two or more actions executing simultaneously. The key concept and difference between these  Is there a difference between parallel programming and concurrent Programming? Answer may need to explain how exactly the processors are utilized and probably how parts of a program are executed

https://joearms.github.io/published/2013-04-05-concurrent-and-parallel-programming.html

Concurrent = Two queues and one coffee machine.

Parallel = Two queues and two coffee machines.

What is concurrent processing?, One of them is parallelism--having multiple CPUs working on the different tasks at For details read this research paper Concepts of Concurrent Programming. Concurrency relates to an application that is making progress more than one task at the same time. Concurrency is a approach that is used for decreasing the response time of the system by using the single processing unit. Concurrency is that the illusion of parallelism, however in actual the chunks of a task aren’t parallelly processed, but inside the application, there are more than one task is being processed at a time.

Interpreting the original question as parallel/concurrent computation instead of programming.

In concurrent computation two computations both advance independently of each other. The second computation doesn't have to wait until the first is finished for it to advance. It doesn't state however, the mechanism how this is achieved. In single-core setup, suspending and alternating between threads is required (also called pre-emptive multithreading).

In parallel computation two computations both advance simultaneously - that is literally at the same time. This is not possible with single CPU and requires multi-core setup instead.

Images from article: "Parallel vs Concurrent in Node.js"

versus

Multithreading and Concurrency - Java Programming Tutorial, Distinguish parallelism (using extra computational units to do more work per unit time) from concurrency (managing access to shared resources). Parallel programming is to specifically refer to the simultaneous execution of concurrent tasks on different processors. Thus, all parallel programming is concurrent, but not all concurrent

In the view from a processor, It can be described by this pic

In the view from a processor, It can be described by this pic

What is the difference between concurrent programming and , Concurrency and parallelism are two terms often used in relation to multithreaded applications. This tutorial explains the difference between  Concurrent programming is in a general sense to refer to environments in which the tasks we define can occur in any order. One task can occur before or after another, and some or all tasks can be performed at the same time. Parallel programming is to specifically refer to the simultaneous execution of concurrent tasks on different processors.

Parallel Programming vs. Concurrent Programming, Parallelism is about speeding things up, whereas concurrency is about dealing with simultaneity or nondeterminism. When we talk about parallel programming,  The difference comes from the sets of topics the two areas cover. For example, concurrent programming includes topic like signal handling, while parallel programming includes topic like memory

The difference between "concurrent" and "parallel" execution , Differences between concurrency vs. parallelism. Now let's list down Love computers, programming and solving everyday problems. Find me on Facebook​  Parallel programming carries out many algorithms or processes simultaneously. One of these is multithreading (multithreaded programming), which is the ability of a processor to execute on multiple threads at the same time. Learn what is parallel programming, multithreading, and concurrent vs parallel.

Difference between Parallel and Concurrent programming , Concurrent programming basics and concurrent programming in iOS. The difference between a condition and a mutex lock is that multiple 

Comments
  • possible duplicate of Concurrency vs Parallelism - What is the difference?
  • Parallel execution and parallel programming are not the same thing. The answer from Jon Harrop is correct. But it seems that the question itself confuses parallel execution and parallel programming.
  • The ability to execute threads in parallel depends upon more than just the machine. For example, OCaml (and Python?) executes threads concurrently but not in parallel due to a global lock for the garbage collector.
  • Parallel programming is not a subset of concurrent programming, according to this blog; you're answer doesn't take that into account, what do you think about this statement?
  • @Kevin: I think "more general" means superset. I agree that it is wrong.
  • This answer is good for visualizing the difference between concurrent & parallel executing, but not for poster's original question about programming.
  • "The hard part of parallel programming ... such as granularity and communication." If parallel tasks need to communicate, doesn't that make them concurrent?
  • "If parallel tasks need to communicate, doesn't that make them concurrent?". Wow, great question! Not necessarily, no. Supercomputers are often programmed with bulk parallel operations followed by global redistribution of data and more bulk parallelism. So there is parallelism and communication but no real concurrency to speak of. In this context, I was thinking more of multicore parallelism where communication means cache complexity, e.g. communication required for cache coherency. Although that is concurrent it is also not directly visible.
  • @BoppityBop Just because I can say in a drawing what he said in a novel doesn't make my answer less correct. Just easier to read for those who actually don't know the answer. Which I guess is the point of coming here. You can write a book in the language used by this post, but that's going to be absolutely jibberish to most readers, since you probably didn't google this question if you already know half of what jon wrote.
  • The picture was very helpful for me, someone pretty new to the topic, and the description from @JonHarrop was useful to me, someone who appreciates correct, even if technical, language. Both answers contributed to my more complete understanding. We all win! (although I do appreciate the distinction made between parallel execution and parallel programming)
  • "Erlang is perhaps the most promising upcoming language...". Interesting choice of words, since Erlang is ~30 years old and was open sourced in 1998.
  • Incorrect and misleading. Concurrent = allowing one or more queues (nondeterministic composition). Parallel = having more than one queues to make any of them shorter than the original one if not empty (asymptotic efficiency).
  • Concurrent code requires two or more processors (or "coffee machines"). Thus this answer is essentially wrong.