Redis and escaping binary data

redis list
redis-cli
redis commands
redis http
python redis
node js redis binary data
redis data model
redis-cli binary

i am having a tough time understanding how to use binary datatypes with redis. I want to use the command

   set '{binary data}' 'Alex'

what if the binary data actually includes a quote symbol or /r/n? I know I can escape characters but is there an official list of characters I need to escape?


There's no need to do anything special with the data itself. All Redis strings are binary safe.

Your problem relates to redis-cli (which is a very nice redis client for getting to know Redis, but almost never what you want in production, because of usage and performance issues).

Your problem also relates to common (bash/sh/other) terminal escaping. Here's a nice explanation.

I suggest you use python for this, or any other language you are comfortable with.

Example:

import redis
cli=redis.Redis('localhost', 6379)
with open('data.txt','rb') as f:
  for d in f:
    t = d.partition('\t')
    cli.set(t[0], t[2].rstrip())
#EOF

Storing Binary Data in Redis, what if the binary data actually includes a quote symbol or /r/n? I know I can escape characters but is there an official list of characters I need to escape? Redis is a in-memory key-value storage system. Primary Memory storage size tends to be much smaller than hard disk storage size. So its never a good idea to populate primary memory with binary data/files as you will soon run out of memory and cause redis/other applications unexpected shutdowns.


You can send the command as an array of bulk strings to Redis, no need to escape characters or Base64 encode. Since bulk strings begin with the data length, Redis doesn't try to parse the data bytes and instead just jumps to the end to verify the terminating CR/LF pair:

*3<crlf>
$3<crlf>SET<crlf>
${binary_key_length}<crlf>{binary_key_data}<crlf>
${binary_data_length}<crlf>{binary_data}<crlf>

Redis Protocol specification – Redis, keys and values don't have to be text strings. Redis, which stands for Remote Dictionary Server, is a fast, open-source, in-memory key-value data store for use as a database, cache, message broker, and queue.The project started when Salvatore Sanfilippo, the original developer of Redis, was trying to improve the scalability of his Italian startup.


Arbitrary bytes can be input in redis-cli using hexadecimal notation, e.g.

set "\x00\xAB\x20" "some value"

Redis and the speed of key-value data stores - Dace, , it can be used for other client-server software projects. RESP is a compromise between the following things: Simple to implement. I am trying to save a binary object in redis and then serve it back as an image. Here is the code I am using to save the data: var buff=new Buffer(data.data,'base64'); client.set(key,new Buffer(data.data,'base64')); Here is the code to dump the data out:


I found it is best to use the Redis protocol to do this as the boundaries can be defined before the datatype.

ReJSON, Redis replies with a command-specific data type. RESP is binary-safe and does not require processing of bulk data transferred from one process to another,  Redis keys are binary safe, this means that you can use any binary sequence as a key, from a string like "foo" to the content of a JPEG file. The empty string is also a valid key. A few other rules about keys:


FastoNoSQL and FastoRedis can handle binary keys without escaping.

Secondary indexing with Redis – Redis, Redis is not exactly a key-value store, since values can be complex data structures. they are sorted lexicographically, comparing the strings as binary data with is better to use some form of escaping in order to guarantee that the separator  Since lexicographical ranges in Redis are binary safe you can use any byte or any sequence of bytes. However if you receive untrusted user input, it is better to use some form of escaping in order to guarantee that the separator will never happen to be part of the key.


An introduction to Redis data types and abstractions – Redis, The following is the list of all the data structures supported by Redis, which will be covered separately in this tutorial: Binary-safe strings. Lists: collections of string  *Data types *Strings. Strings are the most basic kind of Redis value. Redis Strings are binary safe, this means that a Redis string can contain any kind of data, for instance a JPEG image or a serialized Ruby object. A String value can be at max 512 Megabytes in length.


Command line redis-cli dump, redis-cli dump lsa_data_html_hide > /tmp/data.dmp When it is displayed on screen, redis-cli escapes non-printable characters using the \xHH But when you redirect it to the file, it is written in the raw binary format. Requests are sent from the client to the Redis server as arrays of strings representing the arguments of the command to execute. Redis replies with a command-specific data type. RESP is binary-safe and does not require processing of bulk data transferred from one process to another, because it uses prefixed-length to transfer bulk data.


Redis and escaping binary data - Stack Overflow, Redis protocol is binary safe, so if you say that 9 bytes are to follow, they can all be Standardize a tabular, line-oriented data format with some escape  The Redis itself can be used as a sole data store of an application as well. Redis Keys: Keys are central to Redis data model. Everything that is stored on a Redis instance is stored against a key. All the keys are of type binary safe strings. Keys supported by Redis can have their size up to 512 MB.