Hot questions for Using ZeroMQ in matlab

Question:

I am using a python-matlab-bridge that calls MATLAB from python by starting it on a ZMQ socket. On my own computer, I hand the bridge the location of the executable (in this case MATLAB 2014B):

executable='/Applications/MATLAB_R2014b.app/bin/matlab'

and everything works as required and the printed statement is:

Starting MATLAB on ZMQ socket ipc:///tmp/pymatbridge-49ce56ed-f5b4-43c4-8d53-8ae0cd30136d

Now I want to do the same on a cluster. Through module avail I find there are two MATLAB versions (2015a and 2016b) available and located at the following path:

/opt/ud/LOCAL/etc/modulefiles/matlab.

When I now call MATLAB using:

executable='/opt/ud/LOCAL/etc/modulefiles/matlab/MATLAB_R2015a.app/bin/matlab'

the error:

 Starting MATLAB on ZMQ socket ipc:///tmp/pymatbridge-95775445-359d-441f-803a-7193eedbf215    
Send 'exit' command to kill the server
............................................................MATLAB session timed out after 60 seconds

is returned. It cannot find the MATLAB executable. How to proceed?


Answer:

Step 0:

Check whether the code-segment responsible for "remote" launch of MATLAB goes well - for details about params, check pymatbridge.py:

    ### class _Session( object ): _____________________________
    ...

    def _run_server(self):
        code = self._preamble_code()
        code.extend([
            "matlabserver('%s')" % self.socket_addr
        ])
        command = '%s %s %s "%s"' % ( self.executable,
                                      self.startup_options,
                                      self._execute_flag(),
                                      ','.join( code )
                                      )
        subprocess.Popen( command,
                          shell  = True,
                          stdin  = subprocess.PIPE,
                          stdout = subprocess.PIPE
                          )

If such invocation works on any Cluster-Node you will try to harness in a massively distributed MATLAB scenario, the problem goes into ZeroMQ messaging part, if not, the remote server simply cannot launch a _Session, while the messaging ( and all it's { ipc:// | tcp:// | pgm:// | epgm:// | vmci:// } transport-classes available for a massively-distributed computing ) is not to be blamed.

Step 1:

If Step 0 ( a subprocess.Popen(...) invocation ) works well on all remote nodes, check all the ZeroMQ / pymatbridge pre-requisites ( dynamic loader settings - lines added to all remote cluster nodes' .bash_profile ( or similar file for respective shell ).

Yes, a tedious task to prove all remote nodes on the cluster meet these, but without checking these, there is no warranty a remote node can operate the remote MATLAB process connected via otherwise smart and powerful tool, like the pymatbridge is.

Question:

Using JeroMQ in Matlab, my subscriber connection drops when I try to recv a message. I have verified this with Wireshark. I also note that the closing of the TCP connection is initiated by my subscribing port, not the publishing port.

javaclasspath('jeromq-0.5.1.jar')
import org.zeromq.*;

ctx = zmq.Ctx();
socket = ctx.createSocket(ZMQ.SUB);
socket.connect('tcp://127.0.0.1:5996')
message = socket.recv(1) %this is when the connection gets dropped

Wireshark screenshot showing connection being closed

I don't know for sure if it would help, but I have investigated using this,

socket.setSocketOpt(ZMQ.ZMQ_TCP_KEEPALIVE,1)

but Matlab reports that ZMQ_TCP_KEEPALIVE is an unrecognized function or variable.

I am indebted to the folks on this thread for getting me started on the right foot with JeroMQ in Matlab.


Answer:

I did some more research and factored in suggestions from some friends over at Github. Turns out that my socket wasn't closing until later in the code at my close() function. Here is my working code for others looking to get started using JeroMQ in Matlab. Seems that some of the code on other posts is antiquated.

javaclasspath('jeromq-0.5.1.jar')
import org.zeromq.*;

%subscribe to ZMQ feed
context = ZContext();
socket = context.createSocket(ZMQ.SUB); 
success = false;
while(~success)
    success = socket.connect('tcp://127.0.0.1:5996');
end
socket.subscribe("");
socket.setTCPKeepAlive(1);

%receive a message
message = socket.recv(0); %nonblocking receive uses argument (1)

%when done
socket.close();

Question:

I want to send and receive data from Matlab and MetaTrader 5 on Windows 10. According to this post, JeroMQ is the easiest route to go:

  • I cloned the repo from https://github.com/zeromq/jeromq
  • I installed maven
  • Then I went into the repository root and start building it using: mvn package, which produced the following error message:

 [ERROR] Failures:
 [ERROR] TestEvents.testEventConnectRetried:85 No event was received
 [ERROR] Errors:
 [ERROR] PollTest.testPollUdp:100 ยป Bind Cannot assign requested address: 
 connect
 [INFO]
 [ERROR] Tests run: 588, Failures: 1, Errors: 1, Skipped: 17
 [INFO]
 [INFO] ------------------------------------------------------------------------
 [INFO] BUILD FAILURE
 [INFO] ------------------------------------------------------------------------
 [INFO] Total time: 04:48 min
 [INFO] Finished at: 2019-09-12T18:51:01+02:00
 [INFO] ------------------------------------------------------------------------
 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.1:test (default-test) on project jeromq: There are test failures.    

I tried to fix this by setting the IP configuration for the localhost to 127.0.0.1 according to these posts:

which did not fix the problem (same error message).

I think it is related to some IP settings, but I am new to socket communication. I have no experience in Java programming/debugging. Please help me to fix this, so the .jar file is built successfully and I can add it to my javaclasspath in Matlab.

Otherwise: Is there an (easy) alternative way to establish Matlab socket communication with other programs (e.g. via ZeroMQ)?


Answer:

I had the same problem, to bypass the tests, you need to add some code to the pom.xml file which is at the main folder jeromq-masters.

So the part that I changed is below. You can compare with your own pom and change it according to it. Since a couple of days passed, I dont exactly remember the part that I added,but starting with "includes" must be the part that I added.

<groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.22.1</version>
    <configuration>
      <useSystemClassLoader>false</useSystemClassLoader>
      <includes>
        <include>TestFail.java</include>
      </includes>
      <testFailureIgnore>true</testFailureIgnore>
    </configuration>

Then you need to recompile it with mvn package command.

This allowed me to compile the JAR file.