Hot questions for Using ZeroMQ in nanomsg

Question:

I'm using a PUB/SUB design and my question is:

Can I .bind() to a port after another socket has .connect()-ed to it, or should I .bind() before another socket tries to .connect() to the same address?

In other words:

Does the order of .bind() and .connect() matter?

( I think this question is not specific to PUB/SUB but is relevant for any design ).


Answer:

A driving principle behind ZMQ is not worrying whether the socket you're trying to connect to is already there or not. These are details that ZMQ tries to abstract away from you, the developer. So, no, the order of bind() and connect() does not matter for any socket type.

I suggest you read the zmq guide if you plan to do extensive work with it, relevant portion here:

Remember that ZeroMQ does asynchronous I/O, i.e., in the background. Say you have two nodes doing this, in this order:

  • Subscriber connects to an endpoint and receives and counts messages.
  • Publisher binds to an endpoint and immediately sends 1,000 messages.

Then the subscriber will most likely not receive anything. You'll blink, check that you set a correct filter and try again, and the subscriber will still not receive anything.

... there's an important point to note here, for PUB/SUB - even if you connect() first with your subscriber, that connection doesn't actually occur until after the publisher has bind()-ed, so if you attempt to send messages with your publisher without waiting for your subscriber to finish its connection, those messages will never make it to your subscriber.

Question:

How can I achieve an IP Multicasting in nanomsg?

I know that in ZeroMQ messaging, the IP Multicasting is achieved through a Pragmatic General Multicast ( PGM ) protocol.

Is there any way in nanomsg to also achieve the IP Multicasting?


Answer:

No, not yet.

The currently latest release of the nanomsg ( being 1.1.2. in EoY 2017 ) does not yet have a confirmed support for either a pgm:// or epgm:// transport classes, as you might have known from ZeroMQ toolkit.

Question:

I want to implement data distribution application using zeromq or nanomsg or aeron.net or OpenDDS or OpenMAMA, I am little confused , which one is suitable as my requirement is to support 500 to 1000 users and each user can subscribe for atleast 100 topics

this can be handled by using pub/sub pattern or I have to use UDP multicast ?

Also I need to deploy this application on Amazon EC2


Answer:

You mix apples with oranges :

"Can ( this ) be handled by using pub/sub pattern or ( do ) I have to use UDP multicast?"

The PUB/SUB Scalable Formal Communication Pattern tells how the agents behave each to the other.

Whereas,the UDP multicast tells what technical transport class could be preferred to be used, i.e. either of { pgm:// | epgm:// } from those of all the technically available ones { inproc:// | ipc:// | tcp:// | pgm:// | epgm:// | vmci:// }

Finalle,the PUB/SUB can work over { pgm:// | epgm:// }, so the "or"-choice constructor is principally not an adequate one.


Back to Scaling :

Neither ZeroMQ nor nanomsg ( as of implementations available in 2018-Q1 ) will have issues with the PUB/SUB mechanics over ~ 1000 users x 100+ topics.

For details, one may read wonderfull Martin Sustrik's technical details how and why topic-filters work in either of these platforms. He, being a father of both these signalling / messaging systems, one could hardly ask a better or more insightfull person on this subject and its limits.

Having expressed your prior exposure to FinTech / FIX-Protocol / Trading platforms, you ought have experience enough to decide further on principal issues.


Also I need to deploy this application on Amazon EC2 :

Easy. Go get it done. If in problems, re-read platform documentation and ask the service provider first.

Question:

I'm using nanomsg to transfer pieces of my data from Windows to CentOS.

At the moment, I'm using Req/Rep mode.

The CentOS will send a Req-request, and "Windows" will respond with a Rep to it.

It looks alright.

But the pieces of data I respond to have mixed several structs. A data head following with several data frames.

I want to send them one by one.

But according to nanomsg, Req/Rep should just send a request, and receive a response one at a time. That's all.

So if I send the head, the nn_socket will forbid me to send the data frame.

What I can do is to make a big buffer, and cement the head and the data frames, and send them together.

Is there a way to send them one by one instead of sending together?

Thanks!


Answer:

Something like Google Protocol Buffers will be your friend here.

nanomsg sends complete messages. It's then simply a matter of what a "message" is. Essentially, it's just a set of bytes that it guarantees to deliver. It's up to you what those bytes actually represent.

This is where Google Protocol Buffers (or JSON, or XML, or ASN.1 [my personal favourite], and there's many others to choose from) comes in. It allows you to define message structures that meet your needs, and it then generates code that serialises and deserialises those messages. You write your code to serialise / deserialise to buffers (byte arrays, strings, whatever), which you pass as messages using nanomsg.

The nice thing is that you then don't have to worry about endianness, structure packing in different compilers, etc, or what programming language is being used at the other end, or whether the message in its deserialised form is contiguous in memory.

I mention GPB because it is free, fairly straightforward