How do I manage ruby threads so they finish all their work?

working with ruby threads
ruby nested threads
ruby thread start do
ruby sleep
ruby thread pool
rails thread
ruby thread break
thread wakeup ruby

I have a computation that can be divided into independent units and the way I'm dealing with it now is by creating a fixed number of threads and then handing off chunks of work to be done in each thread. So in pseudo code here's what it looks like

# main thread
work_units.take(10).each {|work_unit| spawn_thread_for work_unit}

def spawn_thread_for(work)
  Thread.new do
    do_some work
    more_work = work_units.pop
    spawn_thread_for more_work unless more_work.nil?
  end
end

Basically once the initial number of threads is created each one does some work and then keeps taking stuff to be done from the work stack until nothing is left. Everything works fine when I run things in irb but when I execute the script using the interpreter things don't work out so well. I'm not sure how to make the main thread wait until all the work is finished. Is there a nice way of doing this or am I stuck with executing sleep 10 until work_units.empty? in the main thread


If you modify spawn_thread_for to save a reference to your created Thread, then you can call Thread#join on the thread to wait for completion:

x = Thread.new { sleep 0.1; print "x"; print "y"; print "z" }
a = Thread.new { print "a"; print "b"; sleep 0.2; print "c" }
x.join # Let the threads finish before
a.join # main thread exits...

produces:

abxyzc

(Stolen from the ri Thread.new documentation. See the ri Thread.join documentation for some more details.)

So, if you amend spawn_thread_for to save the Thread references, you can join on them all:

(Untested, but ought to give the flavor)

# main thread
work_units = Queue.new # and fill the queue...

threads = []
10.downto(1) do
  threads << Thread.new do
    loop do
      w = work_units.pop
      Thread::exit() if w.nil?
      do_some_work(w)
    end
  end
end

# main thread continues while work threads devour work

threads.each(&:join)

Ruby wait for all threads finish · GitHub, You'll learn about Ruby threads so you can make your Ruby applications faster. Threads make your Ruby programs do multiple things at the same time. The problem is that Ruby doesn't wait for threads to finish. you go out sprinkling threads all over your code you must know that there are some problems associated  If you have the following code you will notice that there is no output from the thread: The problem is that Ruby doesn’t wait for threads to finish. You need to call the join method on your thread to fix the code above: If you want to create multiple threads you can put them inside an array & call join on every thread.


In ruby 1.9 (and 2.0), you can use ThreadsWait from the stdlib for this purpose:

require 'thread'
require 'thwait'

threads = []
threads << Thread.new { }
threads << Thread.new { }
ThreadsWait.all_waits(*threads)

How to Use Ruby Threads: An Easy To Understand Tutorial, Basically the GIL prevents multiple Ruby threads from executing at the same time​. how many threads you spawn, and how many cores you have at your disposal, We can take advantage of this and perform other work, which doesn't We still need them all to complete before returning the results to the  When a Ruby program terminates, all threads are killed, regardless of their states. However, you can wait for a particular thread to finish by calling that thread's Thread.join method. The calling thread will block until the given thread is finished. By calling join on each of the requestor threads, you can make sure that all two requests have completed before you terminate the main program.


It seems like you are replicating what the Parallel Each (Peach) library provides.

Untangling Ruby Threads, If you manage to get your threads deadlocked, the whole process may grind to a halt. And if some thread happens to make a call to the operating system that takes time to complete, all threads will hang until the interpreter gets control back. You can wait for a particular thread to finish by calling that thread's Thread.join method. The calling thread will block until the given thread is finished. Threads and Exceptions. If an exception is raised in the main thread, and is not handled anywhere, the Ruby interpreter prints a message and exits.


Thread.list.each{ |t| t.join unless t == Thread.current }

Ruby Threads: Ruby Study Notes, Let's start by clearing up an all-too-common point of confusion among Ruby In particular, Ruby concurrency is when two tasks can start, run, and complete in though, that they'll ever both be running at the same instant (e.g., multiple threads on a The key point here is that concurrent threads and/or processes will not  Ruby concurrency without parallelism can still be very useful, though, for tasks that are IO-heavy (e.g., tasks that need to frequently wait on the network). So threads can still be useful in the MRI, for IO-heavy tasks. There is a reason threads were, after all, invented nd used even before multi-core servers were common.


You can use Thread#join

join(p1 = v1) public

The calling thread will suspend execution and run thr. Does not return until thr exits or until limit seconds have passed. If the time limit expires, nil will be returned, otherwise thr is returned.

Also you can use Enumerable#each_slice to iterate over the work units in batches

work_units.each_slice(10) do |batch|
  # handle each work unit in a thread
  threads = batch.map do |work_unit|
    spawn_thread_for work_unit
  end

  # wait until current batch work units finish before handling the next batch
  threads.each(&:join)
end

Ruby Concurrency and Parallelism in Multithreaded Apps: A Tutorial , If you manage to get your threads deadlocked, the whole process may grind to a halt. And time to complete, all threads will hang until the interpreter gets control back. And when you're working with shared data, race conditions pretty much  Threads are the Ruby implementation for a concurrent programming model. Programs that require multiple threads of execution are a perfect candidate for Ruby's Thread class. For example, we can create a new thread separate from the main thread's execution using ::new. thr = Thread. new { puts "What's the big deal"}


Threads and Processes @ Programming Ruby, After creating a few threads we wait for them all to finish consecutively. You can also use alive? to tell if the thread is running or sleeping, and stop? if the thread is With the addition of the class method ::handle_interrupt, you can now handle not work as dynamic scope if the methods are thread-local and a given block  Ruby: parallel processing made simple and fast. Contribute to grosser/parallel development by creating an account on GitHub.


Class: Thread (Ruby 2.5.1), Our items arrays don't depend upon each other, so it would be ideal to our data should be OK (meaning, not corrupt) after all threads finish execution. to control the program so that when one thread is doing work another  A blocking I/O blocks all threads; As of Ruby 1.9, Ruby uses native threads. Native threads means that each thread created by Ruby is directly mapped to a thread generated at the Operating System


Threads in Ruby, Thread.fork { while true puts 'forked thread' end } while true puts 'main thread' end Threads are supposed to “run all together”, but actually they are running for a little time in There are roughly two ways to do it: kernel-level threads and user-​level threads. Let's look at the variables and the data types to manage threads. When a Ruby program terminates, all running threads are killed, regardless of their states. However, you can wait for a particular thread to finish by calling that thread's Thread # join method. The calling thread will block until the given thread is finished.