how to configure pooled connection idle timeout in reactor-netty

Related searches

I am using reactor-netty http client (0.7.X series) with connection pooling and would like to configure pooled connection's idle timeout but don't know where.

More precisely, I need to configure reactor-netty http client connection pool in such a way that it will automatically close connections that did not see any activity within configurable timeout. These connections are open but no bytes were transferred in or out for some (configurable) amount of time.

How can I configure reactory-netty http client to close idle connections preemptively?

I was able to accomplish this on the 0.7.x branch by adding netty write and read time-out handlers to the channel pipeline. However, on 0.8.x, this approach no longer works.

HttpClient httpClient = HttpClient
    .create((HttpClientOptions.Builder builder) -> builder
    .host(endpointUrl.getHost())
    .port(endpointUrl.getPort())
    .poolResources(PoolResources.fixed(connectionPoolName, maxConnections, timeoutPool))
    .afterChannelInit(channel -> {
        channel.pipeline()
                // The write and read timeouts are serving as generic socket idle state handlers.
                .addFirst("write_timeout", new WriteTimeoutHandler(timeoutIdle, TimeUnit.MILLISECONDS))
                .addFirst("read_timeout", new ReadTimeoutHandler(timeoutIdle, TimeUnit.MILLISECONDS));
    })
    .build());

how to configure pooled connection idle timeout in reactor-netty , I was able to accomplish this on the 0.7.x branch by adding netty write and read time-out handlers to the channel pipeline. However, on 0.8.x,� I am using reactor-netty http client (0.7.X series) with connection pooling and would like to configure pooled connection's idle timeout but don't know where. More precisely, I need to configure reactor-netty's connection pool in such a way that it will automatically close connections that did not see any activity within configurable timeout.

I managed to configure WebClient (via underlying TcpClient) to remove idle connections on timeout from connection pool in reactor-netty 0.8.9

My solution is partially based on the official documentation about IdleStateHandler extended with my research on how to properly apply it when creating an instance of HttpClient.

Here is how I did that:

public class IdleCleanupHandler extends ChannelDuplexHandler {
    @Override
    public void userEventTriggered(final ChannelHandlerContext ctx, final Object evt) throws Exception {
        if (evt instanceof IdleStateEvent) {
            final IdleState state = ((IdleStateEvent) evt).state();
            if (state == IdleState.ALL_IDLE) { // or READER_IDLE / WRITER_IDLE
                // close idling channel
                ctx.close();
            }
        } else {
            super.userEventTriggered(ctx, evt);
        }
    }
}

...

public static WebClient createWebClient(final String baseUrl, final int idleTimeoutSec) {
    final TcpClient tcpClient = TcpClient.create(ConnectionProvider.fixed("fixed-pool"))
        .bootstrap(bootstrap -> BootstrapHandlers.updateConfiguration(bootstrap, "idleTimeoutConfig",
            (connectionObserver, channel) -> {
                channel.pipeline()
                    .addLast("idleStateHandler", new IdleStateHandler(0, 0, idleTimeoutSec))
                    .addLast("idleCleanupHandler", new IdleCleanupHandler());
            }));

    return WebClient.builder()
        .clientConnector(new ReactorClientHttpConnector(HttpClient.from(tcpClient)))
        .baseUrl(baseUrl)
        .build();
}


IMPORTANT UPDATE:

My further testing has indicated that adding handlers during bootstrap hook distructs the pool and sockets (channels) are not reused by Connection.

The right way to add the handlers is:

public static WebClient createWebClient(final String baseUrl, final int idleTimeoutSec) {
    final TcpClient tcpClient = TcpClient.create(ConnectionProvider.fixed("fixed-pool"))
        .doOnConnected(conn -> {
            final ChannelPipeline pipeline = conn.channel().pipeline();
            if (pipeline.context("idleStateHandler") == null) {
                pipeline.addLast("idleStateHandler", new IdleStateHandler(0, 0, idleTimeoutSec))
                        .addLast("idleCleanupHandler", new IdleCleanupHandler());
            }
        });

    return WebClient.builder()
        .clientConnector(new ReactorClientHttpConnector(HttpClient.from(tcpClient)))
        .baseUrl(baseUrl)
        .build();
}

Note: in reactor-netty 0.9.x there will be a standard way to configure idle timeout for connections in the connection pool, see this commit: https://github.com/reactor/reactor-netty/pull/792

Reactor Netty Reference Guide, I am using reactor-netty http client (0.7.X series) with connection pooling and would like to configure pooled connection's idle timeout but don't know where. Fields inherited from interface reactor.netty.resources.ConnectionProvider DEFAULT_POOL_ACQUIRE_TIMEOUT, DEFAULT_POOL_LEASING_STRATEGY, DEFAULT_POOL_MAX_CONNECTIONS, DEFAULT_POOL_MAX_IDLE_TIME, LEASING_STRATEGY_FIFO, LEASING_STRATEGY_LIFO; Constructor Summary

The easiest way to do this in reactor-netty 0.9.x with TCP client is by using the below approach, I got this from the link referred by @Vladimir-L. Configure "maxIdleTime" for your question.

TcpClient timeoutClient = TcpClient.create(ConnectionProvider.fixed(onnectionPoolName, maxConnections, acquireTimeout,maxIdleTime));

how to configure pooled connection idle timeout, I am using reactor-netty http client (0.7.X series) with connection pooling and would like to configure pooled connection's idle timeout but don't� The server.connection-timeout configuration key is not supported for Netty servers (yet), I've raised spring-boot#15368 to fix that. The connection timeout is about the maximum amount of time we should wait to for a connection to be established. If you're looking to customize the read/write timeouts, those are different options.

I am currently at reactor-netty 0.8.2 because of spring-boot-starter-webflux and faced the same issue, the connection pool kept connections open for 60 seconds after they were finished.

With this approach you can't configure the timeout, but you can disable it:

WebClient.builder()
    .clientConnector(new ReactorClientHttpConnector(
        HttpClient.from(TcpClient.create()).keepAlive(false)))
    .build()
    .get()
    .uri("someurl")
    .retrieve()
    .bodyToMono(String.class)

I am using reactor-netty http client (0.7.X series) with connection pooling and would like to configure pooled connection's idle timeout but don't know where. The implementation uses FIFO order for channels in the pool. By default, there is no idle time specified for the channels in the pool. active or idle. reactor

If the Connection Timeout time has been reached, and there are still no connections available in the pool, the connection pooling service raises an exception indicating that the connection pool request has timed-out. Upon a connection timeout, ODP.NET distinguishes whether the timeout occurred due to the database server failing to deliver a

Issue in this case was that the firewalls and target servers idle timeout were exactly equal to the check interval. Therefore it came to concurrency issues from time to time as the target server or firewall closed the connection at the same time as the request was made. After changing firewall timeouts and check interval the errors are gone.

On the production server sometimes randomly the connection fails to the ORacle database. I get a lot of . Oracle.DataAccess.Client.OracleException Pooled connection request timed out at Oracle.DataAccess.Client.OracleException.HandleErrorHelper(Int32 errCode, OracleConnection conn, IntPtr opsErrCtx, OpoSqlValCtx* pOpoSqlValCtx, Object src, String procedure, Boolean bCheck, Int32 isRecoverable

Comments
  • Thank you for your response.