dispy sample program hangs
TL;DR: I can't get the most basic
dispy sample code to run properly. Why not?
I'm trying to get into distributed processing in python, and thought the dispy library sounded interesting, due to the comprehensive feature set.
However, I've been trying to follow their basic canonical program example, and I'm getting nowhere.
- I've installed dispy (
python -m pip install dispy)
- I went on to another machine with the same subnet address and ran
python dispynode.py. It seems to work, as I get the following output:
2016-06-14 10:33:38 dispynode - dispynode version 4.6.14 2016-06-14 10:33:38 asyncoro - version 4.1 with epoll I/O notifier 2016-06-14 10:33:38 dispynode - serving 8 cpus at 10.0.48.54:51348
Enter "quit" or "exit" to terminate dispynode, "stop" to stop service, "start" to restart service, "cpus" to change CPUs used, anything else to get status:
- Back on my client machine, I run the sample code downloaded from http://dispy.sourceforge.net/_downloads/sample.py, copied here:
# function 'compute' is distributed and executed with arguments # supplied with 'cluster.submit' below def compute(n): import time, socket time.sleep(n) host = socket.gethostname() return (host, n) if __name__ == '__main__': # executed on client only; variables created below, including modules imported, # are not available in job computations import dispy, random # distribute 'compute' to nodes; 'compute' does not have any dependencies (needed from client) cluster = dispy.JobCluster(compute) # run 'compute' with 20 random numbers on available CPUs jobs =  for i in range(20): job = cluster.submit(random.randint(5,20)) job.id = i # associate an ID to identify jobs (if needed later) jobs.append(job) # cluster.wait() # waits until all jobs finish for job in jobs: host, n = job() # waits for job to finish and returns results print('%s executed job %s at %s with %s' % (host, job.id, job.start_time, n)) # other fields of 'job' that may be useful: # job.stdout, job.stderr, job.exception, job.ip_addr, job.end_time cluster.print_status() # shows which nodes executed how many jobs etc.
When I run this (
python sample.py), it just hangs. Debugging through pdb, I see it eventually is hanging at
dispy/__init__.py(117)__call__(). The line reads
self.finish.wait(). finish is just a python thread, as
wait() then goes into
lib/python3.5/threading.py(531)wait(). It hangs once it hits the wait.
I've tried running dispynode on the client machine and gotten the same results. I've tried a lot of variants of passing nodes into the creation of the cluster, e.g:
cluster = dispy.JobCluster(compute, nodes=['localhost']) cluster = dispy.JobCluster(compute, nodes=['*']) cluster = dispy.JobCluster(compute, nodes=[<hostname of the remote node running the client>])
I've tried running with the
cluster.wait() line uncommented, and got the same results.
When I added logging (
cluster = dispy.JobCluster(compute, loglevel = 10)), I got the following output on the client side:
2016-06-14 10:27:01 asyncoro - version 4.1 with epoll I/O notifier 2016-06-14 10:27:01 dispy - dispy client at :51347 2016-06-14 10:27:01 dispy - Storing fault recovery information in "_dispy_20160614102701" 2016-06-14 10:27:01 dispy - Pending jobs: 0 2016-06-14 10:27:01 dispy - Pending jobs: 1 2016-06-14 10:27:01 dispy - Pending jobs: 2 2016-06-14 10:27:01 dispy - Pending jobs: 3 2016-06-14 10:27:01 dispy - Pending jobs: 4 2016-06-14 10:27:01 dispy - Pending jobs: 5 2016-06-14 10:27:01 dispy - Pending jobs: 6 2016-06-14 10:27:01 dispy - Pending jobs: 7 2016-06-14 10:27:01 dispy - Pending jobs: 8 2016-06-14 10:27:01 dispy - Pending jobs: 9 2016-06-14 10:27:01 dispy - Pending jobs: 10
This doesn't seem unexpected, but doesn't help me figure out why the jobs aren't running.
For what it's worth, here's _dispy_20160614102701.bak:
'_cluster', (0, 207) 'compute_1465918021755', (512, 85)
and similarly, _dispy_20160614102701.dir:
'_cluster', (0, 207) 'compute_1465918021755', (512, 85)
I'm out of guesses, unless I'm using an unstable version.
When first setting up and using dispy on a network, I found that I had to specify the client node IP when creating the job cluster, see below:
cluster = dispy.JobCluster(compute, ip_addr=your_ip_address_here)
See if that helps.
Simple job hangs � Issue #134 � pgiri/dispy � GitHub, Hello, This is my first time using dispy and it hangs with this simple to host and review code, manage projects, and build software together. Figure out why an application hangs. Find the source of mysterious program freeze-ups by searching the Web and running free diagnostic utilities.
If you're just running sample.py on your client, change the following in your main statement:
cluster = dispy.JobCluster(compute, nodes=['nodeip_1','nodeip_2',.....,'nodeip_n])
Then run it in your IDE, or via shell.
I hope that helps.
7. Examples — dispy 4.12.0 documentation, A simple client program that distributes a program (say, '/path/to/program'), executes client crashes or loses network connection to nodes after submitting jobs. The program hangs because there is an infinite loop, very long loop, or recursive calls. When WhatIsHang detects the first hang problem, the report will contain only one system call snapshot. When WhatIsHang detects the second hang problem, the report will contain 3 snapshots of 3 random execution points.
1. dispy (Client) — dispy 4.12.0 documentation, For example, starting dispynode (Server) program on each of the nodes on a local results of jobs that were scheduled but not yet finished at the time of crash . • Certain software can cause a random freeze on Windows 10. Users have reported that software such as Speccy, Acronis True Image, Privatefirewall, McAfee and Office Hub App can cause problems with Windows 10. If you have any of these programs installed on your computer, we advise that you remove them by following these steps: 1.
dispy: Distributed and Parallel Computing with/for Python — dispy , The results of the scheduled (but unfinished at the time of crash) jobs for that This information may be useful to clients, for example, to analyze application� dispy is a generic, comprehensive, yet easy to use framework for creating and using compute clusters to execute computations in parallel across multiple processors in a single machine (SMP), among many machines in a cluster, grid or cloud. dispy is well suited for data parallel (SIMD) paradigm where a computation (Python function or standalone program) is evaluated with different (large
Tips and Ideas for Hanging Pictures and Gallery Wall Layouts, Creating a gallery wall or art display wall is a stylish way to add personality to a home. When done Those variations do add up and will show in the execution. But what if Perfect example of the versatility of a Hanging System. Easy and� This post is an introduction to the Nextion display with the Arduino. We're going to show you how to configure the display for the first time, download the needed resources, and how to integrate it with the Arduino UNO board. We'll also make a simple graphical user interface to control the Arduino pins.
How to hang pictures on walls, Hanging pictures on walls is a balancing act of spacing, colour and proportion. Alongside inspiring examples from our archive of how to hang, display and Gary Hume prints at a Louis Vuitton fashion show and immediately bought the lot. After completing an installation of 3ds Max or finishing a Windows update, the program will not start up and hangs on the splash screen at "Initializing". In some instances a crash may also occur. Includes, but is not limited to: Old or unsupported graphics card driver. Network proxy settings. FlexNet Licensing Service 64 disabled in Windows services. Corrupted 3ds Max user preferences