Hot questions for Spring Retry

Top 10 Java Open Source / Spring / Spring Retry

Question:

I have this piece of code

@Retryable(maxAttempts = 3, stateful = true, include = ServiceUnavailableException.class,
        exclude = URISyntaxException.class, backoff = @Backoff(delay = 1000, multiplier = 2) )
public void testThatService(String serviceAccountId)
        throws ServiceUnavailableException, URISyntaxException {

//some implementation here }

Is there a way I can make the maxAttempts , delay and multiplier configurable using @Value? Or is there any other approach to make such fields inside annotations configurable?


Answer:

with the release of Spring-retry version 1.2, it's possible. @Retryable can be configured using SPEL.

@Retryable(
    value = { SomeException.class,AnotherException.class },
    maxAttemptsExpression = "#{@myBean.getMyProperties('retryCount')}",
    backoff = @Backoff(delayExpression = "#{@myBean.getMyProperties('retryInitalInterval')}"))
public void doJob(){
    //your code here
}

For more details refer: https://github.com/spring-projects/spring-retry/blob/master/README.md

Question:

In spring boot application, I define some config properties in yaml file as below.

my.app.maxAttempts = 10
my.app.backOffDelay = 500L

And an example bean

@ConfigurationProperties(prefix = "my.app")
public class ConfigProperties {
  private int maxAttempts;
  private long backOffDelay;

  public int getMaxAttempts() {
    return maxAttempts;
  }

  public void setMaxAttempts(int maxAttempts) {
    this.maxAttempts = maxAttempts;
  }

  public void setBackOffDelay(long backOffDelay) {
    this.backOffDelay = backOffDelay;
  }

  public long getBackOffDelay() {
    return backOffDelay;
  }

How can I inject the values of my.app.maxAttempts and my.app.backOffdelay to Spring Retry annotation? In the example below, I want to replace the value 10 of maxAttempts and 500Lof backoff value with the corresponding references of config properties.

@Retryable(maxAttempts=10, include=TimeoutException.class, backoff=@Backoff(value = 500L))

Answer:

Staring from spring-retry-1.2.0 we can use configurable properties in @Retryable annotation.

Use "maxAttemptsExpression", Refer the below code for usage,

 @Retryable(maxAttemptsExpression = "#{${my.app.maxAttempts}}",
 backoff = @Backoff(delayExpression = "#{${my.app. backOffDelay}}"))

It will not work if you use any version less than 1.2.0.Also you don't require any configurable property classes.

Question:

Is Spring Retry guaranteed to work with Spring's @Transactional annotation?

Specifically, I'm trying to use @Retryable for optimistic locking. It seems like it would be dependent on the ordering of the AOP proxies that were created. For example, if the calls look like this:

Calling code -> Retry Proxy -> Transaction Proxy -> Actual DB Code

Then it would work correctly, but if the proxies were structured like this:

Calling code -> Transaction Proxy -> Retry Proxy -> Actual DB Code

Then the retry wouldn't work, because the act of closing the transaction is what throws the optmistic locking exception.

In testing, it appeared to generate the first case (retry, then transaction), but I couldn't tell if this was a guaranteed behavior or just lucky.


Answer:

Found the answer here: https://docs.spring.io/spring/docs/5.0.6.BUILD-SNAPSHOT/spring-framework-reference/data-access.html#transaction-declarative-annotations Table 2 indicates that the advice for the Transactional annotation has an order of Ordered.LOWEST_PRECEDENCE, which means that it is safe to combine Retryable with Transactional as long as you aren't overriding the order of the advice for either of those annotations. In other words, you can safely use this form:

@Retryable(StaleStateException.class)
@Transactional
public void performDatabaseActions() {
    //Database updates here that may cause an optimistic locking failure 
    //when the transaction closes
}

Question:

I use compile 'org.springframework.retry:spring-retry:1.2.2.RELEASE'with Spring Boot 1.5.9.RELEASE.

Configured to retry my method and it works well:

@Retryable(value = { IOException.class }, maxAttempts = 5, backoff = @Backoff(delay = 500))
public void someMethod(){...}

How to output some specific message when retry occurs?


Answer:

You can register a RetryListener:

@Bean
public List<RetryListener> retryListeners() {
    Logger log = LoggerFactory.getLogger(getClass());

    return Collections.singletonList(new RetryListener() {

        @Override
        public <T, E extends Throwable> boolean open(RetryContext context, RetryCallback<T, E> callback) {
            // The 'context.name' attribute has not been set on the context yet. So we have to use reflection.
            Field labelField = ReflectionUtils.findField(callback.getClass(), "val$label");
            ReflectionUtils.makeAccessible(labelField);
            String label = (String) ReflectionUtils.getField(labelField, callback);
            log.trace("Starting retryable method {}", label);
            return true;
        }

        @Override
        public <T, E extends Throwable> void onError(RetryContext context, RetryCallback<T, E> callback, Throwable throwable) {
            log.warn("Retryable method {} threw {}th exception {}",
                    context.getAttribute("context.name"), context.getRetryCount(), throwable.toString());
        }

        @Override
        public <T, E extends Throwable> void close(RetryContext context, RetryCallback<T, E> callback, Throwable throwable) {
            log.trace("Finished retryable method {}", context.getAttribute("context.name"));
        }
    });

If you don't need to log from all 3 interception points, you can override RetryListenerSupport instead. For example:

@Bean
public List<RetryListener> retryListeners() {
    Logger log = LoggerFactory.getLogger(getClass());

    return Collections.singletonList(new RetryListenerSupport() {

        @Override
        public <T, E extends Throwable> void onError(
                RetryContext context, RetryCallback<T, E> callback, Throwable throwable) {
            log.warn("Retryable method {} threw {}th exception {}",
                    context.getAttribute("context.name"), 
                    context.getRetryCount(), throwable.toString());
        }
    });
}

Question:

Is it possible to set RetryPolicy in spring retry (https://github.com/spring-projects/spring-retry) based on error status code? e.g. I want to retry on HttpServerErrorException with HttpStatus.INTERNAL_SERVER_ERROR status code, which is 503. Therefore it should ignore all other error codes -- [500 - 502] and [504 - 511].


Answer:

The RestTemplate has setErrorHandler option and DefaultResponseErrorHandler is the default one.

Its code looks like:

public void handleError(ClientHttpResponse response) throws IOException {
    HttpStatus statusCode = getHttpStatusCode(response);
    switch (statusCode.series()) {
        case CLIENT_ERROR:
            throw new HttpClientErrorException(statusCode, response.getStatusText(),
                    response.getHeaders(), getResponseBody(response), getCharset(response));
        case SERVER_ERROR:
            throw new HttpServerErrorException(statusCode, response.getStatusText(),
                    response.getHeaders(), getResponseBody(response), getCharset(response));
        default:
            throw new RestClientException("Unknown status code [" + statusCode + "]");
    }
}

So, you can provide your own implementation for that method to simplify your RetryPolicy around desired status codes.

Question:

I have to work with a project where I cannot use Java-Config for Spring, but I have to use XML-Config. Now I am looking for an XML-Config equivalent to @EnableRetry from Java-Config.

I want this line to work.

@Retryable(SQLException.class)
public void saveOrUpdate(Entity entity) 

Answer:

Yes, @EnableRetry on an empty @Configuration will work (comment by OP).

Just for completeness, here's the equivalent in XML.

<context:annotation-config />

<aop:aspectj-autoproxy />

<bean class="org.springframework.retry.annotation.RetryConfiguration" />

EDIT

Complete example using XML to enable retry:

@SpringBootApplication
@ImportResource("context.xml")
public class So31923175Application {

    public static void main(String[] args) {
        SpringApplication.run(So31923175Application.class, args);
    }

    @Bean
    public ApplicationRunner runner(Foo app) {
        return args -> {
            try {
                app.retry("foo");
            }
            catch (Exception e) {
                e.printStackTrace();
            }
        };
    }

    @Component
    public static class Foo {

        @Retryable
        public void retry(String param) {
            System.out.println(param);
            throw new RuntimeException("test retry");
        }

    }

}

and

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:aop="http://www.springframework.org/schema/aop"
    xmlns:context="http://www.springframework.org/schema/context"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
        http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop.xsd">

    <context:annotation-config />

    <aop:aspectj-autoproxy />

    <bean class="org.springframework.retry.annotation.RetryConfiguration" />

</beans>

and

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.1.1.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>net.gprussell.filemailer</groupId>
    <artifactId>so31923175</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>jar</packaging>
    <name>so31923175</name>
    <description>Mail non-empty files passed on the command line</description>

    <properties>
        <java.version>1.8</java.version>
    </properties>

    <dependencies>

        <dependency>
            <groupId>org.springframework.retry</groupId>
            <artifactId>spring-retry</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-aop</artifactId>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>


</project>

and

foo
foo
foo
java.lang.RuntimeException: test retry

Question:

I am using Spring-Retry for some database operations. On a SQLRecoverableException I retry three times (this assumes that whatever is causing the exception is non-transient if it fails three times), on a SQLTransientException I retry indefinitely (the program can't do anything without access to the database, so it may as well keep retrying until the user decides to reboot the server), and on any other exception I don't retry. I use an exponential backoff policy with a base retry of 100ms and a max retry of 30,000ms.

private static final int MAX_RECOVERABLE_RETRIES = 3;
private static final long INITIAL_INTERVAL = 100;
private static final long MAX_INTERVAL = 30 * 1000;
private static final double MULTIPLIER = 2.0;

public static RetryTemplate databaseTemplate() {
    RetryTemplate template = new RetryTemplate();
    ExceptionClassifierRetryPolicy retryPolicy = new ExceptionClassifierRetryPolicy();
    Map<Class<? extends Throwable>, RetryPolicy> policyMap = new HashMap<>();
    NeverRetryPolicy baseException = new NeverRetryPolicy();
    SimpleRetryPolicy recoverablePolicy = new SimpleRetryPolicy();
    recoverablePolicy.setMaxAttempts(MAX_RECOVERABLE_RETRIES);
    AlwaysRetryPolicy transientPolicy = new AlwaysRetryPolicy();
    policyMap.put(Exception.class, baseException);
    policyMap.put(SQLRecoverableException.class, recoverablePolicy);
    policyMap.put(SQLTransientException.class, transientPolicy);
    retryPolicy.setPolicyMap(policyMap);
    template.setRetryPolicy(retryPolicy);
    ExponentialBackOffPolicy backOffPolicy = new ExponentialBackOffPolicy();
    backOffPolicy.setInitialInterval(INITIAL_INTERVAL);
    backOffPolicy.setMaxInterval(MAX_INTERVAL);
    backOffPolicy.setMultiplier(MULTIPLIER);
    template.setBackOffPolicy(backOffPolicy);
    return template;
}

Ideally, I would like to use a fixed backoff of 100ms for all SQLRecoverableExceptions, and only apply the exponential backoff policy to SQLTransientExceptions. I could accomplish this with nested retries, but that will greatly increase the code complexity - given no other option I would prefer to simply apply the exponential backoff to both SQLRecoverableException and SQLTransientException exceptions.

Is there a way for me to apply different backoff policies to different exceptions using a single retry template?


Answer:

Indeed, ExceptionClassifierRetryPolicy is the way to go. I didn't manage to make it work with the policyMap though.

Here is how I've used it:

@Component("yourRetryPolicy")
public class YourRetryPolicy extends ExceptionClassifierRetryPolicy
{
    @PostConstruct
    public void init()
    {
        final SimpleRetryPolicy simpleRetryPolicy = new SimpleRetryPolicy();
        simpleRetryPolicy.setMaxAttempts( 3 );

        this.setExceptionClassifier( new Classifier<Throwable, RetryPolicy>()
        {
            @Override
            public RetryPolicy classify( Throwable classifiable )
            {
                    if ( classifiable instanceof YourException )
                    {
                            return new NeverRetryPolicy();
                    }
                    // etc...
                    return simpleRetryPolicy;
            }
        });
    }
}

Then, you just have to set it on the retry template :

@Autowired
@Qualifier("yourRetryPolicy")
private YourRetryPolicy yourRetryPolicy;

//...

RetryTemplate retryTemplate = new RetryTemplate();
retryTemplate.setRetryPolicy( yourRetryPolicy );

Question:

I have a restful service calling an external service using Spring Cloud Feign client

@FeignClient(name = "external-service", configuration = FeignClientConfig.class)
public interface ServiceClient {

    @RequestMapping(value = "/test/payments", method = RequestMethod.POST)
    public void addPayment(@Valid @RequestBody AddPaymentRequest addPaymentRequest);

    @RequestMapping(value = "/test/payments/{paymentId}", method = RequestMethod.PUT)
    public ChangePaymentStatusResponse updatePaymentStatus(@PathVariable("paymentId") String paymentId,
            @Valid @RequestBody PaymentStatusUpdateRequest paymentStatusUpdateRequest);

}

I noticed the following failure 3-4 times in the last 3 months in my log file:

json.ERROR_RESPONSE_BODY:Connection refused executing POST http://external-service/external/payments json.message:Send Payment Add Payment Failure For other reason: {ERROR_RESPONSE_BODY=Connection refused executing POST http://external-service/external/payments, EVENT=ADD_PAYMENT_FAILURE, TRANSACTION_ID=XXXXXXX} {} json.EVENT:ADD_PAYMENT_FAILURE json.stack_trace:feign.RetryableException: Connection refused executing POST http://external-service/external/payments at feign.FeignException.errorExecuting(FeignException.java:67) at feign.SynchronousMethodHandler.executeAndDecode(SynchronousMethodHandler.java:104) at feign.SynchronousMethodHandler.invoke(SynchronousMethodHandler.java:76) at feign.ReflectiveFeign$FeignInvocationHandler.invoke(ReflectiveFeign.java:103)

Is it possible to add Spring Retry on a Feign client. What I wanted to annotate the addPayment operation with

@Retryable(value = {feign.RetryableException.class }, maxAttempts = 3, backoff = @Backoff(delay = 2000, multiplier=2))

But this is not possible, what other options do I have?


Answer:

You can add a Retryer in the FeignClientConfig

@Configuration
public class FeignClientConfig {

    @Bean
    public Retryer retryer() {
        return new Custom();
    }

}

class Custom implements Retryer {

    private final int maxAttempts;
    private final long backoff;
    int attempt;

    public Custom() {
        this(2000, 3);
    }

    public Custom(long backoff, int maxAttempts) {
        this.backoff = backoff;
        this.maxAttempts = maxAttempts;
        this.attempt = 1;
    }

    public void continueOrPropagate(RetryableException e) {
        if (attempt++ >= maxAttempts) {
            throw e;
        }

        try {
            Thread.sleep(backoff);
        } catch (InterruptedException ignored) {
            Thread.currentThread().interrupt();
        }
    }

    @Override
    public Retryer clone() {
        return new Custom(backoff, maxAttempts);
    }
}

Updated with sample Retryer example config based on the Retryer.Default.