Hot questions for Using ZeroMQ in encryption

Question:

Context
  • I am trying to implement properly the encryption with ZMQ 4.2.1 (this commit actually)
  • I read the very interesting article from Pieter Hintjens (here), where he describe the handshake procedure between a client and a server.
  • I have:
    • a client, who knows the server's public key + its own key pair
    • a server, who knows only its own key pair
Use case

I want to know when my client is using wrong keys (one or more of the three he knows are wrong).

When playing this use case, today the connection fails as expected but I have no way to know if the connection failed because of the encryption handshake failed.

Nb.: when I set the good keys, the connection works just fine as well

Settings

The server:

  • Socket: ZMQ_ROUTER, binds on a TCP endpoint.
  • Parameters:
    • ZMQ_CURVE_SERVER = 1
    • ZMQ_CURVE_SECRETKEY = the private key of the server
    • ZMQ_CURVE_PUBLICKEY = the public key of the server
  • A monitoring socket is listening to the events of the router socket.

The client:

  • Socket: ZMQ_DEALER, connects to the server's TCP endpoint.
  • Parameters:
    • ZMQ_CURVE_SERVER = 0
    • ZMQ_CURVE_SERVERKEY = the public key of the server
    • ZMQ_CURVE_SECRETKEY = the private key of the client
    • ZMQ_CURVE_PUBLICKEY = the public key of the client
  • A monitoring socket is listening to the events of the dealer socket.
Behaviour

I am running my server and client in two simple console applications, the client uses a wrong server's publioc key.

Here are the log traces of the monitoring socket of the server:

Router monitoring event: MONITOR_STARTED - 
Router monitoring event: LISTENING - tcp://0.0.0.0:20100
Router monitoring event: ACCEPTED - tcp://0.0.0.0:20100
Router monitoring event: DISCONNECTED - tcp://0.0.0.0:20100
Router monitoring event: ACCEPTED - tcp://0.0.0.0:20100
Router monitoring event: DISCONNECTED - tcp://0.0.0.0:20100
Router monitoring event: ACCEPTED - tcp://0.0.0.0:20100
Router monitoring event: DISCONNECTED - tcp://0.0.0.0:20100
Router monitoring event: ACCEPTED - tcp://0.0.0.0:20100
Router monitoring event: DISCONNECTED - tcp://0.0.0.0:20100
Router monitoring event: ACCEPTED - tcp://0.0.0.0:20100

And so on...

Here is the console traces that just happened when executing this use case:

CURVE I: cannot open client HELLO -- wrong server key?
CURVE I: cannot open client HELLO -- wrong server key?
CURVE I: cannot open client HELLO -- wrong server key?
CURVE I: cannot open client HELLO -- wrong server key?
CURVE I: cannot open client HELLO -- wrong server key?

And so on...

Here are the log traces of the client's monitoring socket:

Dealer monitoring event: MONITOR_STARTED - 
Dealer monitoring event: CONNECT_DELAYED - tcp://127.0.0.1:20100
Dealer monitoring event: CONNECTED - tcp://127.0.0.1:20100
Dealer monitoring event: DISCONNECTED - tcp://127.0.0.1:20100
Dealer monitoring event: CONNECT_RETRIED - tcp://127.0.0.1:20100
Dealer monitoring event: CONNECT_DELAYED - tcp://127.0.0.1:20100
Dealer monitoring event: CONNECTED - tcp://127.0.0.1:20100
Dealer monitoring event: DISCONNECTED - tcp://127.0.0.1:20100
Dealer monitoring event: CONNECT_RETRIED - tcp://127.0.0.1:20100
Dealer monitoring event: CONNECT_DELAYED - tcp://127.0.0.1:20100
Dealer monitoring event: CONNECTED - tcp://127.0.0.1:20100
Dealer monitoring event: DISCONNECTED - tcp://127.0.0.1:20100
Dealer monitoring event: CONNECT_RETRIED - tcp://127.0.0.1:20100
Dealer monitoring event: CONNECT_DELAYED - tcp://127.0.0.1:20100
Dealer monitoring event: CONNECTED - tcp://127.0.0.1:20100
Dealer monitoring event: DISCONNECTED - tcp://127.0.0.1:20100

And so on...

I tried very quickly to follow the code of ZMQ from the trace "cannot open client HELLO -- wrong server key" (see this file), but it doesn't looks like there is a specific handling for handshake fails, or maybe I didn't went far enough in the code to find it out...

Does any one has already faced that case and found out how to know if the keys we use are good or not? To me this information seems kind of important, but maybe it's not provided by ZMQ for security reason? I'm really not an expert in security...


Answer:

Edit 2018-02-05:

The feature is stable and available since version 4.2.1, still in the DRAFT section of the API.

See the documentation:

ZMQ_EVENT_HANDSHAKE_FAILED

The ZMTP security mechanism handshake failed. The event value is unspecified. NOTE: in DRAFT state, not yet available in stable releases.

ZMQ_EVENT_HANDSHAKE_SUCCEED

The ZMTP security mechanism handshake succeeded. The event value is unspecified. NOTE: in DRAFT state, not yet available in stable releases.


Edit 2017-01-01:

The pull request has been merged into the master branch of libzmq. It is now possible to get the handshake status, using the monitoring events:

  • ZMQ_EVENT_HANDSHAKE_SUCCEED is raised, once the encryption handshake succeed
  • ZMQ_EVENT_HANDSHAKE_FAILED is raised, when it failed

However this feature is still not stable, you need to compile libzmq using the preprocessor directive ZMQ_BUILD_DRAFT_API.


Original answer (2016-12-29):

There is currently no proper feature for this purpose.

There is an open feature request on libzmq github.

Question:

At the moment, I am investigating the possibilities to use ZeroMQ with Curve to secure traffic between my publishers and subscribers.

I have successfully implemented a pub sub system which is using CZMQ.

At the moment, my publisher is encrypting the data he wants to send with his private key and subscribers can decrypt this data using the public key of the publisher. This is more 'authenticating' data than 'encrypting' data. Because when there is a Man In The Middle he can still decrypt all the data because the public key of the publisher is public.

I'm coding in C with the latest version of ZeroMQ and CZMQ.

My publisher.c

zctx_t* zmq_context = zctx_new();

zauth_t* auth = zauth_new (zmq_context);
zauth_set_verbose (auth, true);
zauth_configure_curve (auth, "*", CURVE_ALLOW_ANY);

zcert_t* pub_cert = zcert_load("cert.key"); // private key of publisher
void* socket = zsocket_new(zmq_context, ZMQ_PUB);
zcert_apply(pub_cert, socket);
zsocket_set_curve_server(socket, true);

//start publishing from here

My subscriber.c

zctx_t* zmq_context = zctx_new();

zcert_t* sub_cert = zcert_new();
zcert_t* pub_cert = zcert_load("cert.pub"); // public key of publisher
char* pub_key = zcert_public_txt(pub_cert);

void* zmq_s = zsocket_new(zmq_context, ZMQ_SUB);
zcert_apply(sub_cert, zmq_s);
zsocket_set_curve_serverkey(zmq_s, pub_key);

//start subscribing to topics and receiving messages from here

From this point, the publisher is encrypting all the data with his private key and the subscribing is decrypting all the data with the public key of the publisher. I would like to swap this system.

So, I would like to encrypt all the data with the public key of the publisher and decrypt all the data with the private key of the publisher.

I have tested it and changed the zcert_load("cert.key") to zcert_load("cert.pub") in my publisher.c.

I also changed this code in my subscriber.c:

zcert_t* pub_cert = zcert_load("cert.pub"); // public key of publisher
char* pub_key = zcert_public_txt(pub_cert);

to this code:

zcert_t* pub_cert = zcert_load("cert.key"); // private key of publisher
char* pub_key = zcert_secret_txt(pub_cert);

When I run my publisher and subscriber with these code changes, the publisher is constantly giving me the message: CURVE I: cannot open client HELLO -- wrong server key?

My question: Is it possibile to use a public key for encrypting data (publisher socket) and the private key for decrypting data (subscriber socket) with the architecture of ZeroMQ and CZMQ ?

Many thanks in advance, Roy


Answer:

I think you're misunderstanding the ZeroMQ CURVE mechanism. There are two great articles written about it by Pieter Hintjens, one is theoretical, where you can find this:

Clients and servers have long-term permanent keys, and for each connection, they create and securely exchange short-term transient keys. Each key is a public/secret keypair, following the elliptic curve security model.

To start a secure connection the client needs the server permanent public key. It then generates a transient key pair and sends a HELLO command to the server that contains its short term public key. The HELLO command is worthless to an attacker; it doesn't identify the client.

The server, when it gets a HELLO, generates its own short term key pair (one connection uses four keys in total), and encodes this new private key in a "cookie", which it sends back to the client as a WELCOME command. It also sends its short term public key, encrypted so only the client can read it. It then discards this short term key pair.

At this stage, the server hasn't stored any state for the client. It has generated a keypair, sent that back to the client in a way only the client can read, and thrown it away.

The client comes back with an INITIATE command that provides the server with its cookie back, and the client permanent public key, encrypted as a "vouch" so only the server can read it. As far as the client is concerned, the server is now authenticated, so it can also send back metadata in the command.

The server reads the INITIATE and can now authenticate the client permanent public key. It also unpacks the cookie and gets its short term key pair for the connection. As far as the server is now concerned, the client is now authenticated, so the server can send its metadata safely. Both sides can then send messages and commands.

So the keys you generate here are not used directly to encrypt data, they are only used to authenticate parties. The handshake process between client and server produces real encryption (and authentication as in MAC) keys that are used to encrypt the data. So if the only concern you have is MITM, you're already protected.

But you can also be concerned about rogue clients, your current scheme allows anyone to connect. This is where the second article by Pieter Hintjens can help you with "The Ironhouse Pattern". The critical part is to disable CURVE_ALLOW_ANY and tell server where to look for client public certificates:

//  Tell authenticator to use the certificate store in .curve
zauth_configure_curve (auth, "*", ".curve");

Then you generate client key on the client (once! not calling zcert_new() for every connection), transfer its public part to the ".curve" directory of your server and load (on the client) that key instead of calling zcert_new().

If you want to make life a bit easier (not needing to transfer public keys from client to server for each client), you can create a "golden" client key pair, get its public part into the ".curve" store and then (via a secure channel, of course) copy the secret key on every client that needs to connect to your publisher.

Question:

I'm exercising with ØMQ and have developed a simple publisher/subscribers using cppzmq with reference to zguide. Please find code snippet of the same as below, I'm able to successfully publish and subscribe data and memory footprint also quite impressive. Since this is the distributed messaging system I would like to use it in my project.

publisher.cpp
int main (int argc, char * argv[]) {
//  Prepare our context and publisher
zmq::context_t context(1);
zmq::socket_t publisher(context, ZMQ_PUB);

std::string addr("tcp://*:");
addr += ((argc > 1) ? std::string(argv[1]) : "5563");
std::string tappend((argc > 2) ? std::string(argv[2]) : "");
publisher.bind(addr.c_str());

struct timeval timeofday;

for(unsigned int idx = 0; idx < Total_Topics; ++idx)
{
    Topics.emplace(std::make_pair(idx, std::string("temperature/celsius" + tappend + std::to_string(idx))));
}

std::this_thread::sleep_for (std::chrono::seconds(2));

for(unsigned int NoOfTimes = 0; NoOfTimes < 20000; ++NoOfTimes)
{
    for(unsigned int count = 0; count < Topics.size(); ++count)
    {
      {
         Quote quote = {};
         quote.ticker = "BHELL";
         gettimeofday(&timeofday,NULL);
         quote.timestampus = timeofday.tv_usec;
         quote.timestamps = timeofday.tv_sec;

         std::stringstream ss;

         quote.value *= 0.1;
         quote.data = std::to_string(NoOfTimes) + std::to_string(count);

         ss << quote;

         s_sendmore(publisher, Topics[count]);
         std::cout << "Publish "<< Topics[count] << std::endl;
         s_send(publisher, ss.str());
      }
    }

    std::this_thread::sleep_for (std::chrono::seconds(8));
    }

    return 0;
}
subscriber.cpp
int main (int argc, char * argv[]) {
//  Prepare our context and subscriber
zmq::context_t context(1);
zmq::socket_t subscriber (context, ZMQ_SUB);

std::string addr("tcp://localhost:");
addr += ((argc > 1) ? std::string(argv[1]) : "5563");
std::string tappend((argc > 2) ? std::string(argv[2]) : "");
subscriber.connect(addr.c_str());

struct timeval timeofday;

std::string topic("");

for(unsigned int idx = 0; idx < Total_Topics; ++idx)
{
    topic = std::string("temperature/celsius" + tappend + std::to_string(idx));
    subscriber.setsockopt( ZMQ_SUBSCRIBE, topic.c_str(), topic.length());
}

while(1)
{
    {
        Quote quote;
        std::stringstream ss;

        std::string address = s_recv (subscriber);
        std::cout<<"Msg Rcvd" << std::endl;
        std::cout<<"Topic: " << address << std::endl;

        std::string contents = s_recv (subscriber);

        ss << contents;
        ss >> quote;

        gettimeofday(&timeofday,NULL);
        std::cout << "us diff  "  << (timeofday.tv_usec - quote.timestampus) << std::endl;
        std::cout << "Sec diff "  << (timeofday.tv_sec - quote.timestamps) << std::endl;
    }
}
    return 0;
}

Now I would like to know which security technique should be used with zeroMQ. I understand there are curveZMQ and czmq. But I'm not sure which one should I use. Anybody already has used any of these or you have anyother libraries for ZeroMQ security. Please advise.


Answer:

After digging up further on the zeroMQ encryption, I found that libcurve (C version) and libzmq (C++ version) supports the curve security. By default libzmq supports the curve and libcurve requires libsodium, libzmq, and czmq.

To find out what the difference between libcurve and libzmq curve implementation, you can through the discussion here.

Please find for sample PUB/SUB implementation with libcurve here.