Use multithreading with redis , using jedis connection pool

Use multithreading with redis , using jedis connection pool

jedis pool configuration
redis connection pool
redis multi thread
jedis async
jedis vs lettuce
redis best practices
redis multiprocessing
redis thread safe

I have read that redis is single threaded per user/client.

Is it possible to use jedis connection pool to achieve multithreaded behaviour?

Basically different jedis client request would be using different connection obatined from jedis connection pool , to fire their commands to one redis server.

Since client requests are using different connections hence one redis should be server multiple requests parallely right ?


I answered similar question here, you may check it for details.

TL/DR: Redis itself will still be single threaded, but in the grand scheme of things time used by Redis to answer is order of magnitude less than time used by networking. You will get benefit from multithreading, yes.

EDIT: Along the request pipeline you will have 3 points where requests are processed sequentially. Its when the data packets are sent through a wire (you only have one wire, but the packets from different requests will be mixed) forward and backward, and when Redis serves your request. But Redis itself is no dumb either. Your requests will come from a network socket and will be pre-processed and written into client buffers. This part is processed in parallel. Then Redis's main loop will pick your commands from a queue, process and write response into client buffers. This part is done sequentially. Then responses from client buffers will be packed and sent through a wire back to you. This is also being done in parallel.

Use multithreading with redis , using jedis connection pool - php, I have read that redis is single threaded per user/client. Is it possible to use jedis connection pool to achieve multithreaded behaviour? Basically different jedis  In the actual development, we need to use the Redis connection tool to connect to Redis and then operate Redis, For mainstream languages, redis provides corresponding clients. You can view the corresponding clients on the redis official website. Click on Clients You can see that clients are provided for those languages.


Is it possible to use jedis connection pool to achieve multithreaded behaviour?

Though you use multiple connection or threading or processing to communicating with redis, These commands are all put on one queue, and redis will get one by one and execute one by one.And every single command is atomic. So this is still a single thread behaviour at the aspect of the redis.

Since client requests are using different connections hence one redis should be server multiple requests parallely right ?

Not always, the commands from different clients may arrive at redis at arbitrary order, so if these command has some data relation, dangerous things will happen.

consider this simple scenario, if redis has a counter S, if all client do the INCR command, then it is ok, because every command is executed atomicly, But if some client's code is like something that:

s = get S
t = s + 10
set S t

Then things would be wrong. And you should use the Multi/exec command to ensure that multi commands would be atomic no just only one command. More things about transactions in redis, you could refer this page. It is detailed.

Getting started · xetorthio/jedis Wiki · GitHub, using Jedis in a multithreaded environment You can use the pool to reliably create several Jedis instances, given you return the JedisPoolConfig includes a number of helpful Redis-specific connection pooling defaults. As Jedis documentation state that Jedis client is not thread-safe. A single Jedis instance is not threadsafe! So I am using JedisPool. I want to push data to browser's WebSocket client from server. For this I am using Redis's PubSub mechanism.


This works fine in my (web based) multi-threaded environment.

Make sure that RedisInterface instance be static in your class.

public class RedisInterface implements Closeable
{   
/*      
<dependency>
<groupId>redis.clients</groupId>
<artifactId>jedis</artifactId>
<version>2.9.0</version>
</dependency>   
*/

private JedisPool jedisPool =  null;

private synchronized void initializePool()
{
    if(jedisPool!=null) return;

    JedisPoolConfig poolConfig = new JedisPoolConfig();
    poolConfig.setMaxTotal(Integer.parseInt(AppConfig.REDIS_MAX_CONN)); //max conn count
    poolConfig.setMaxIdle(Integer.parseInt(AppConfig.REDIS_MAX_IDLE_CONN)); //max idle conn count
    poolConfig.setMaxWaitMillis(Long.parseLong(AppConfig.REDIS_MAX_WAIT_TIME)); // max wait time for new connection (before throwing exception)

    jedisPool = new JedisPool(poolConfig, 
            AppConfig.REDIS_IP, 
            Integer.parseInt(AppConfig.REDIS_PORT),
            Integer.parseInt(AppConfig.REDIS_CONN_TIMEOUT));

}

//not synchronized after testing thread safety of jedisPool.getResource()
protected Jedis getJedis()
{               
    if(jedisPool==null)
        initializePool();

      Jedis jedis = jedisPool.getResource();

      return jedis;     
}


public Long set(final byte[] key, final byte[] field, final byte[] value)
{
    Jedis redis = null; 
    Long ret =0L;

    try
    {
        redis = getJedis();

        ret = redis.hset(key, field, value);            
        redis.expire(key, Integer.parseInt(AppConfig.REDIS_EXPIRE_MIN)*60); //      
    }
    finally
    {
        if(redis!=null)         
            redis.close();
    }

    return ret;
}

public byte[] get(final byte[] key, final byte[] field) {

    Jedis redis = null ;
    byte[] valueBytes = null;

    try
    {
        redis = getJedis();

        valueBytes = redis.hget(key, field);
    }
    finally
    {       
        if(redis!=null)             
            redis.close();
    }

    return valueBytes;

}

@Override
public void close() throws IOException {

    if(jedisPool!=null)     
        jedisPool.close();      
}

}

Spring data redis, multi-threading issue with Jedis · Issue #918 , redis integration. I am setting the configuration to use pool, testOnBorrow as well as testOnReturn true. I am using Jedis pool, and have verified that spring ensures th at org.springframework.data.redis.connection.jedis. In a real-world scenario, you do not want to use a single instance in a multi-threaded environment as a single instance is not thread-safe. Luckily enough we can easily create a pool of connections to Redis for us to reuse on demand, a pool that is thread safe and reliable as long as you return the resource to the pool when you are done with it.


Multithreaded Jedis, I have multiple threads that connect to Redis, if I use a single instance it works is important to return the jedis instance to the pool once you've finished using it. does your application release the connection to the pool after each use? /// Jedis implements Closeable. Hence, the jedis instance will be auto-closed after the last statement. try (Jedis jedis = pool.getResource()) { // use your jedis jedis.set("foo", "bar"); } // the resource will be released back to the pool If not using the try with the


Use Connection pool with Jedis : redis, I am using Jedis to connect with redis server in rest service. when I am calling that's titled "using Jedis in a multithreaded environment", and mentions using  the mod does not have a connection pool, it does use redis connection pipeline. This means that a socket connection is always active to the server and commands are pushed in, since redis also does not block this is the recommended way to use the server.


Using Jedis as a Redis client - Thanuja Kularathna, JedisPool also has different constructors where you can use according to the Redis server installation. Using Jedis Connection Pool Resource to connect Redis  We can connect to Redis by creating a single object but if we are connecting to Redis again and again then we need a connection pool from where we can fetch an object and use it. Using connection pool we can reuse the object which is contained in the pool. Here is an example of creating a connection pool with Redis.