Hot questions for Using ZeroMQ in websocket

Question:

Im am trying to implement push integration using php and native zmq. I have successfully send send my message to server, but my problem is I cannot push the message to browser using js Websocket(). I says WebSocket connection to 'ws://127.0.0.1:8080/' failed: Error during WebSocket handshake: Invalid status line

here is my code for client:

<?php
 try {
   function send($data) {
      $context = new ZMQContext();
      $push = new ZMQSocket($context, ZMQ::SOCKET_PUSH);
      $push->connect("tcp://localhost:5555");

      $push->send($data);
    }    

    if(isset($_POST["username"])) {
      $envelope = array(
        "from" => "client",
        "to" => "owner",
        "msg" => $_POST["username"]
      );
      send(json_encode($envelope)); # send the data to server
    }
 }
 catch( Exception $e ) {
   echo $e->getMessage();
 }

?>

Client

here is my server:

$context = new ZMQContext();

$pull = new ZMQSocket($context, ZMQ::SOCKET_PULL);
$pull->bind("tcp://*:5555"); #this will be my pull socket from client

$push = new ZMQSocket($context, ZMQ::SOCKET_PUSH);
$push->bind("tcp://127.0.0.1:8080"); # this will be the push socket to owner

while(true) {
    $data = $pull->recv(); # when I receive the data decode it
    $parse_data = json_decode($parse_data);

    if($parse_data["to"] == "owner") {
        $push->send($parse_data["msg"]); # forward the data to the owner
    }
    printf("Recieve: %s.\n", $data);
}

and here is my owner.php i'm expecting the data to be send thru Websocket in browser:

<html>
<head>
    <meta charset="UTF-8">
    <title></title>
</head>
<body>
    <h2>Message</h2>
    <ul id="messagelog">
    </ul>
    <script>
        var logger = document.getElementById("messagelog");
        var conn = new WebSocket("ws://127.0.0.1:8080"); # the error is pointing here.

        conn.onOpen = function(e) {
            console.log("connection established");
        }
        conn.onMessage = function(data) {
            console.log("recieved: ", data);
        }

        conn.onError = function(e) {
            console.log("connection error:", e);
        }
        conn.onClose = function(e) {
            console.log("connection closed~");
        }
    </script>
</body>

Please do tell me what I am missing. thank you.


Answer:

You didn't establish a protocol communication at all. You managed to receive the message, but you never confirmed, by parsing it and sending appropriate response, that your server is indeed a WebSocket server.

Since you are already using PHP and ZeroMQ, the easiest way out is to use Mongrel2 which is, among other things, capable of understanding WebSocket protocol and deliver it to a ZeroMQ endpoint encoded as a tnetstring (a json-like encoding format, trivial to parse).

The other solution is to fully support the WebSocket protocol in your code - something that's outside of the scope of this question and answer.

Question:

So, I am running a Ratchet (php) websocket server with multiple routes that connect do multiple Ratchet apps (MessageComponentInterfaces):

//loop 
$loop   = \React\EventLoop\Factory::create();

//websocket app
$app = new Ratchet\App('ws://www.websocketserver.com', 8080, '0.0.0.0', $loop);

/*
 * load routes
 */
$routeOne = '/example/route';
$routeOneApp = new RouteOneApp();
$app->route($routeOne, $routeOneApp, array('*'));

$routeTwo = '/another/route';
$routeTwoApp = new AnotherApp();
$app->route($routeTwo, $routeTwoApp, array('*'));

From here I am binding a ZMQ socket, in order to be able to receive messages sent from php scripts run on the normal apache server.

// Listen for the web server to make a ZeroMQ push after an ajax request
$context = new \React\ZMQ\Context($loop);
$pull = $context->getSocket(\ZMQ::SOCKET_PULL);
$pull->bind('tcp://127.0.0.1:5050'); // Binding to 127.0.0.1 means the only client that can connect is itself
$pull->on('message', array($routeOneApp, 'onServerMessage'));

Finally, the server is started:

//run
$loop->run();

This works perfectly fine as long as i am binding only one of the ratchet apps to the ZMQ socket. However, i would like to be able to separately push messages to both of the Ratchet apps. For this purpose i thought of binding two ZMQ sockets to different routes like:

$pullOne->bind('tcp://127.0.0.1:5050' . $routeOne); // Binding to 127.0.0.1 means the only client that can connect is itself
$pullOne->on('message', array($routeOneApp, 'onServerMessage'));

and

$pullTwo->bind('tcp://127.0.0.1:5050' . $routeTwo); // Binding to 127.0.0.1 means the only client that can connect is itself
$pullTwo->on('message', array($routeTwoApp, 'onServerMessage'));

However, this leads to an error message from ZMQ when binding the second socket, saying the given address is already in use.

So the question is, is there any other way to use routes over a ZMQ socket? Or should i use other means to distinguish between messages for the separate Ratchet apps, and if so, what would be a good solution? I thought about binding to 2 different ports, but figured that would be a pretty ugly solution?!


Answer:

In general in TCP packets are identified by the 4 tuple (sender ip, sender port, receiver ip, receiver port).

When a incoming packet reaches the network layer, it is forwarded to the appropriate application by looking at the receiver ip and port. If you use the same pair for both the apps, it will be impossible for the layer to decide whom to send it to when a connection comes in.

One solution would be to bind a single connection and the write a common handler that looks at the incoming content and then decides (I assume you have some logic) to differentiate the incoming connections to the different instances and then invokes the corresponding handler. The handler can get the connection object and can handle the connection hence forth.

If both your instances are identical and it doesn't matter who gets the request then you can just randomly forward the new connection to any of the handler.

Edit: I have tried to answer the question irrespective of the application type (Racket/ZMQ etc) because the issue you are trying to address is a fundamental one common to any network application.

For this case since you have two apps running and want to listen on the same port, you can have a common handler which can look at the request URL and forward the connection to the appropriate handler.

The request URL can be obtained using

$querystring = $conn->WebSocket->request->getQuery();

Now the clients can connect using

ws://localhost:5050/app1 
and
ws://localhost:5050/app2

Your different apps can now handle these connections separately.

Question:

I have been developing a websocket server until recently I have encountered a 413 Entity too large error. I am using ratchetphp on my server. Have anyone encountered this? Is there anything I can do so that this won't occur? If there isn't anything I can do please help me recreate this error, the current solution I have on fixing this is clearing my browser's cache. I only find this error out by checking network tab on developer tools using chrome, but if clients connect to the server without knowledge of developer tools how can I let them know that they need to clear their cache?


Answer:

After hours of reading codes inside ratchetphp library I have found out that http connection upgrade header is limited to 4kb of data. So all I did to solve this is to increase the limit.

If you also encounter this error, see file

Ratchet/Http/HttpRequestParser.php

Hope this helps

Question:

I am writing a code to run a pi-based robot. It has several sensors and communicates with external computers over wifi as well. Some of the sensor and computer communication data is done over sockets (to other scripts) to keep things modular and simple (for me). As long as I stay at less than 10 sockets, will I come across any problems? I am mostly wondering if there is anything inherently wrong with this coding strategy.


Answer:

Q : As long as I stay at less than 10 sockets, will I come across any problems?

Maybe yes, maybe not. The number of ZeroMQ Socket-instances does not matter in this, the configuration of the ZeroMQ Context()-instances and thereof of the underlying operating system's resources do matter.

Q : anything inherently wrong with this coding strategy?

No, unless you do right things in a wrong way. Particularly "this coding" remains magically hidden as an unseen, so far, code, the less the MCVE-formulated code-example and the general answer for such a problem thus suffers from a principal undecidability, until an MCVE-code is let to run in the problem-specific eco-system, with known limits for the control-loops and the robot physical properties ( which is left to the kind readers to review ad-hoc ).

ZeroMQ will have Zero-problems in running on the RPi-platform, given due configuration does not suffocate the Context()-instance(s) by not providing adequate resources.

Question:

I'm a beginner in PHP Websockets and I'm trying to create real-time chat with database storage. I was doing pretty good, but now I'm standing at one problem. There's problem, when user1 sends message to user2 and user2 came to the site first (reload first on localhoste), it won't be "real-time".

Let me explain it further.

Here's my server.php. It is practicly the same as ratchet tutorial:

$loop = React\EventLoop\Factory::create();    
$pusher = new \Pusher();
$context = new React\ZMQ\Context($loop);
$pull = $context->getSocket(ZMQ::SOCKET_PULL);
$pull->bind('tcp://127.0.0.1:5555'); // Binding to 127.0.0.1 means the only client that can connect is itself
$pull->on('message', array($pusher, 'onBlogEntry'));
$webSock = new React\Socket\Server($loop);
$webSock->listen(8080, '0.0.0.0'); // Binding to 0.0.0.0 means remotes can connect
$webServer = new Ratchet\Server\IoServer(
        new Ratchet\Http\HttpServer(
        new Ratchet\WebSocket\WsServer(
        new Ratchet\Wamp\WampServer($pusher ))), $webSock);
$loop->run();

In pusher.php are most important these methods (I omitted other non-important stuff):

protected $subscribedTopics = array();
protected $myID = array();

public function onSubscribe(ConnectionInterface $conn, $data) { 
    $this->subscribedTopics[json_decode($data)->teamID] = $data;
    $this->myID[json_decode($data)->userID] = $data;    
}

public function onBlogEntry($entry) {
    $entryData = json_decode($entry, true);

    if ((!array_key_exists($entryData['team_id'], $this->subscribedTopics)) ||
            (!array_key_exists($entryData['to_user_id'], $this->myID))
    ) {
        return;
    }

    $teamID = $this->subscribedTopics[$entryData['team_id']];
    $teamID->broadcast($entryData);   
}

In my presenter Class I have simple form. When user submits this form, this code follows:

$this->chatPartner = $values['to_user_id'];       //this I get from the form
$this->redrawControl('msg');                      //here I redraw my layout
$this->messages_model->addMessage($values);       //here I send data to database
$context = new \ZMQContext();
$socket = $context->getSocket(\ZMQ::SOCKET_PUSH, 'my pusher');
$socket->connect("tcp://localhost:5555");
$socket->send(json_encode($values));

Then, in view I have this JavaScript code:

var myJSON = '{'
            + '"teamID" : {$teamId},'     //this I get from the presenter
            + '"userID"  : {$userId}'     //this I get from the presenter
            + '}'; 
var conn = new ab.Session('ws://localhost:8080',
       function() {
            conn.subscribe(myJSON, function(topic, data) { 
             if (data.from_user_id == mypartnerIdA) {                     
                         //here I edit the DOM
                     }  
                });
            },
            function() {
                console.warn('WebSocket connection closed');
            },
            {'skipSubprotocolCheck': true}
    );

So, back to my problem. I simulate 2 users. User1 reloads this page, where is javascript connection first. User2 reloads this page after him. When User1 sends a message to user2, message appers immediatly (real-time). But when user2 sends a message to user1, this message doesn't appear immediatly - it appears only after next reload of the page.

And my question is - How to fix this? How to make user2's message real-time, too? How can I fix this my code?


Answer:

You probably have a misunderstanding of what the data yous end to subscribe is. It is meant to use for ID's of chat sessions.

for example:

  • A has a chat with B (chatId = 1)
  • B has a chat with C (chatId = 2)
  • C has a chat with A (chatId = 3)
  • A, B and C are in one chat (chatId = 4)

    var chatId = 2; //this chat is only between users B and C
    conn.subscribe( chatId , function(topic, data) {
        ...
    }
    

The easiest way for me to understand it was by comparing it to a hashtag on twitter. Each hashtag is in your case a chatId. And for each subscription to a hashtag/chatId. You will have a WebSocket connection so that you receive all the updates for it.

This will be an easier way to do this in the long run then by having subdivided connections for a userId parameter. It can also easily be stored in a database so that you know whom to send the messages to and who not.