Hot questions for Using ZeroMQ in rabbitmq

Question:

I have built a WebSockets server that acts as a chat message router (i.e. receiving messages from clients and pushing them to other clients according to a client ID).

It is a requirement that the service be able to scale to handle many millions of concurrent open socket connections, and I wish to be able to horizontally scale the server.

The architecture I have had in mind is to put the websocket server nodes behind a load balancer, which will create a problem because clients connected to different nodes won't know about each other. While both clients A and B enter via the LoadBalancer, client A might have an open connection with node 1 while client B is connected to node 2 - each node holds it's own dictionary of open socket connections.

To solve this problem, I was thinking of using some MQ system like ZeroMQ or RabbitMQ. All of the websocket server nodes will be subscribers of the MQ server, and when a node gets a request to route a message to a client which is not in the local connections dictionary, it will pub-lish a message to the MQ server, which will tell all the sub-scriber nodes to look for this client and issue the message if it's connected to that node.

Q1: Does this architecture make sense?

Q2: Is the pub-sub pattern described here really what I am looking for?


Answer:

ZeroMQ would be my option - both architecture-wise & performance-wise

-- fast & low latency ( can measure your implementation performance & overheads, down to sub [usec] scale )

-- broker-less ( does not introduce another point-of-failure, while itself can have { N+1 | N+M } self-healing architecture )

-- smart Formal Communication Pattern primitives ready to be used ( PUB / SUB is the least cardinal one )

-- fair-queue & load balancing architectures built-in ( invisible for external observer )

-- many transport Classes for server-side internal multi-process / multi-threading distributed / parallel processing

-- ready to almost linear scaleability

Adaptive node re-discovery

This is a bit more complex subject. Your intention to create a feasible architecture will have to drill down into more details to solve.

  • Node authentication vs. peer-to-peer messaging
  • Node (re)-discovery vs. legal & privacy issues
  • Node based autonomous self-organising Agents vs. needs for central policy enforcement

Question:

I'm working on monitoring time-series data as realtime (DCIM).

I'm using the Influxdb for the store and using ZMQ for queueing data and transfer (pub/sub) between server and clients in the local network.

Is there a better approach to managing it (i.e. using API or Celery or some things like this)?


[NOTE]:

  • Python/Django and influxdb used on this project.
  • Transfer speed is important for me.

Answer:

The best choice can only be selected if we know what are the features you are looking for.

If you are open for suggestions and looking for very stable and very light communication protocol I would suggest to take a good look at nanomsg and next generation protocol nng. For me nanomsg is more stable then ZMQ and requires zero configuration. nng added secured web sockets and has a chance to be next big thing in light communication protocols.

See Why ZeroMQ Shouldn’t Be Your First Choice.

In a long run you will have less problems with nanomsg than with ZMQ.


Quote:

If I were in your shoes, and having for years my hands dirty with ZeroMQ v2.1.1 -- right due to cross-compatibility issues in heterogeneous uncontrollable code-execution environment -- I would investigate what POSIX-compliant nanomsg can bring to the table ( C-based, C++ & .NET bindings ready ) and indeed a very lightweight tool. Link

What library can I use to do simple, lightweight message passing?