Programmatically changing Hystrix properties

hystrix command properties example
spring boot hystrix configuration properties
hystrix configuration properties file
hystrix allowmaximumsizetodivergefromcoresize
hystrix throttling
hystrix in microservices
hystrix-javanica properties
hystrix threadpool key

I have a circuit breaker set up that I would like to change parameters for runtime. Things like threads and timeout needs to be tuned at customer site.

I create a HystrixCommandProperties.Setter like this:

HystrixCommandProperties.Setter hystrixProps = 
    HystrixCommandProperties.defaultSetter()
        .withCircuitBreakerSleepWindowInMilliseconds(myconf.sleepWindow);
HystrixThreadPoolProperties.Setter threadPoolSettings = 
    HystrixThreadPoolProperties.Setter()
        .withCoreSize(myconf.threadPoolSize);

new MyCommand(HystrixCommand.Setter.withGroupKey("mygroup")
    .andCommandPropertiesDefaults(hystrixProps)
    .andThreadPoolPropertiesDefaults(threadPoolSettings));

MyCommand implements standard HystrixCommand and calls super(hystrixProps).

This works the first time, but when I try to change the properties at runtime (same group name) nothing happens. Is there another way to programmatically change this?

I don't want to go through the property files or specifying an URL to Archaius.

There are also answers that tells me to go through Archaius with ConfigurationManager.getConfigInstance().setProperty("...") . But surely there has to be a way that is similar to the original setters I create? Doing it completely different because it's the second time around just feels awkward.

Late answer, but today i struggled with the same thing and found a way.

The way the default property manager is implemented is that it uses a cache of HystrixCommandProperties based on the name of the command you run. On the first use of the command, it caches what it gets out of the HystrixCommandProperties.Setter passed to the Command's constructor and that's it.

However, using the custom HystrixPropertiesStrategy as a Plugin you can override the cache key (and hence force Hystrix to re-evaluate the Setter passed to the new Command instance, because the cache key is new, so it thinks it's a new Command).

The code would then look similar to this:

public HystrixCommandFactory(....) {
    HystrixPlugins.getInstance().registerPropertiesStrategy(new HystrixPropertiesStrategyWithReloadableCache());
    updateHystrixSettings();        
}

//configurable attributes
private volatile int commandTimeoutMillis;
private volatile long lastSettingsUpdatedTimestamp;
private volatile HystrixCommand.Setter setterForNewCommands;

private void updateHystrixSettings() {
    lastSettingsUpdatedTimestamp = LocalDateTime.now().toDateTime().getMillis();
    HystrixCommandProperties.Setter propertiesSetter = HystrixCommandProperties.Setter()
        .withExecutionTimeoutInMilliseconds(commandTimeoutMillis)
        .withExecutionTimeoutEnabled(true);

    this.setterForNewCommands = HystrixCommand.Setter
        .withGroupKey(HystrixCommandGroupKey.Factory.asKey(GROUP_NAME))
        .andCommandPropertiesDefaults(propertiesSetter);

}

public void setCommandTimeoutMillis(int commandTimeoutMillis) {     
    this.commandTimeoutMillis = commandTimeoutMillis;
    updateHystrixSettings();        
}

private class HystrixPropertiesStrategyWithReloadableCache extends HystrixPropertiesStrategy {

    @Override
    public String getCommandPropertiesCacheKey(HystrixCommandKey commandKey, HystrixCommandProperties.Setter builder) {
        return String.format("%s-%d", commandKey.name(), lastSettingsUpdatedTimestamp);
    }
}

Alternatively, you could always return null from the getCommandPropertiesCacheKey method (that completely turns off caching), but then you have the overhead of Hystrix having to reconstruct the HystrixCommandProperties each time a Command is called

PS: Be sure to use proper thread synchronization for reading and updating these properties, because those will be called from different threads. I omitted that in this sample for simplicity, but I actually use a ReentrantReadWriteLock in my code to guard accesses to these variables

Dynamically update properties for a particular HystixCommandKey , timeoutInMillisecond dynamically but it always got ignored if /40109374/​programmatically-changing-hystrix-properties/50769207#50769207  Hystrix uses Archaius for the default implementation of properties for configuration. The documentation below describes the default HystrixPropertiesStrategy implementation that is used unless you override it by using a plugin. Each property has four levels of precedence: 1. Global default from code. This is the default if none of the following 3 are set.

For future reference: I ended up using the settings through ConfigurationManager and a string property.

ConfigurationManager.getConfigInstance().setProperty("...")

It let's me change things, but in a less type-safe way than the original code. I did struggle for some time with a typo in the string which is why I'd like to avoid that.

I now use this for all the properties I need to change runtime. Creating a new Hystrix circuit breaker every time something changes (new command key) could be an option as well, but would break using properties files later on.

How do I change the configuration dynamically??? · Issue #1717 , I wanted to change the timeout period dynamically when the program and lookup the property names used in the source com.netflix.hystrix. The Hystrix Circuit breaker's thread pool properties can be changed by one of the followings . By using @HystrixCommand#threadPoolProperties; By defining @HystrixCommand#threadPoolKey and referencing that in application.properties via hystrix.threadpool.<myThreadPoolKey>.<propertyName>=value

There is a very simple way of doing this. It just needs 2 steps: a. registering the right plugin b. Adding the correct strategy with the required override.

Use-case: Override ExecutionTimeoutInMilliseconds to 30000 ms from 1000 ms

HystrixPlugins.getInstance().registerPropertiesStrategy(new HystrixPropertiesStrategy() {
            @Override
            public HystrixCommandProperties getCommandProperties(HystrixCommandKey commandKey, HystrixCommandProperties.Setter builder) {
                HystrixCommandProperties.Setter timeout
                        = HystrixCommandProperties.Setter().withExecutionTimeoutInMilliseconds(30000);
                return super.getCommandProperties(commandKey, timeout);
            }
        });

Here I am just overriding the required property. When you run your application you can see this in the DEBUG mode:

2018-06-08 23:18:32 [main] DEBUG c.n.h.s.p.HystrixPropertiesChainedProperty - Flipping property: hystrix.command.Client#getAllData().execution.isolation.thread.timeoutInMilliseconds to use its current value: 30000

Dynamically Setting Hystrix configuration for Unit Tests, So in lieu of this I thought that maybe setting a system property would do the trick. This has worked well in my main application when passed in  HystrixProperty<java.lang.Integer> () The time in milliseconds after a HystrixCircuitBreaker trips open that it should wait before trying requests again. () Factory method to retrieve the default Setter. HystrixProperty<java.lang.Integer> () Number of concurrent requests permitted to HystrixCommand.run().

Hystrix properties can also be set in our service class inside @HystrixCommand annotation, for this we use the Hystrix-Javanica project which is used for implementing the annotations in our project. For that we need to add the dependency of hystrix-javanica into our classpath.

Dependency for Maven:

<dependency>
    <groupId>com.netflix.hystrix</groupId>
    <artifactId>hystrix-javanica</artifactId>
    <version>x.y.z</version>
</dependency>

Inside the @HystrixCommand annotation we can use @HystrixProperty to set the properties for hystrix.

sample @HystrixCommand properties setting:

@HystrixCommand(groupKey = "StoreSubmission", commandKey = "StoreSubmission", threadPoolKey = "StoreSubmission", commandProperties = {
        @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "30000"),
        @HystrixProperty(name = "circuitBreaker.requestVolumeThreshold", value = "4"),
        @HystrixProperty(name = "circuitBreaker.sleepWindowInMilliseconds", value = "60000"),
        @HystrixProperty(name = "metrics.rollingStats.timeInMilliseconds", value = "180000") }, threadPoolProperties = {
        @HystrixProperty(name = "coreSize", value = "30"),
        @HystrixProperty(name = "metrics.rollingStats.timeInMilliseconds", value = "180000") })
public String storeSubmission(ReturnType returnType, InputStream is, String id) {
}

The best way to define these properties is in externalized application.yaml, that way you can control it better & change them for different environments.

Here is the sample hystrix config in my application.yaml

hystrix:
   command.StoreSubmission.execution.isolation.thread.timeoutInMilliseconds: 30000
   command.StoreSubmission.circuitBreaker.requestVolumeThreshold: 4
   command.StoreSubmission.circuitBreaker.sleepWindowInMilliseconds: 60000
   command.StoreSubmission.metrics.rollingStats.timeInMilliseconds: 180000
   collapser.StoreSubmission.maxRequestsInBatch: 1
   collapser.StoreSubmission.requestCache.enabled: FALSE
   threadpool.StoreSubmission.coreSize: 30
   threadpool.StoreSubmission.metrics.rollingStats.timeInMilliseconds: 180000

The Exact format for the application.yml file is

hystrix:
    command:
        findAllProducts:
            execution:
                isolation:
                    thread:
                        timeoutInMilliseconds: 1000
            circuitBreaker:
                requestVolumeThreshold: 20
                errorThresholdPercentage: 50
            metrics:
                rollingStats:
                    timeInMilliseconds: 10000
                    numBuckets: 10
    threadpool:
        ProductService:
            coreSize: 10

For further information on Hystrix-Javanica visit here

HystrixCommandProperties (Hystrix Javadoc 1.5.3), Fluent interface that allows chained setting of properties that can be passed HystrixThreadPoolKey that will dynamically change which HystrixThreadPool a  The Hystrix is Khora 's signature sidearm that fires metallic quills. Sporting a high critical chance , it is also a very versatile weapon, with its quills applying Toxin, Heat, Electricity, or Cold elemental status procs that can be cycled at will with Alternate Fire .

For me HystrixPropertiesStrategy and ConfigurationManager.getConfigInstance().setProperty(..) did not help , I ended up creating my own HystrixDynamicProperty implementation as follows,

public CustomHystrixDynamicProperty extends HystrixDynamicPropertiesArchaius {

    @Override
    public HystrixDynamicProperty<Integer> getInteger(final String name, final Integer fallback) {
         if(name.equals("Your hystrix Property")){
            return new HystrixDynamicProperty<Integer>() {
            public Integer get() {
               // Place your logic here...
              return yourValue;
             }
           public String getName(){ return name;};
           public void addCallback(Runnable callback) {};

              }
           } else {
                return super.getInteger(name,fallback);
                 }


    }
}

and add below Property before you start the spring boot application.

System.setProperty("hystrix.plugin.HystrixDynamicProperties.implementation","YourPackage.CustomHystrixDynamicProperty");

For Curious developers: : getDynamicProperty method from the HystrixPropertiesChainedProperty is overriding the overridden value via HystrixPropertiesStrategy, which is being called by "AbstractCommand" everytime before setting up the HystrixCommand for execution, So HystrixPropertiesStrategy idea did not work for me.

Index (Hystrix Javadoc 1.5.3), Static method in class com.netflix.hystrix.strategy.properties. HystrixThreadPoolKey that will dynamically change which HystrixThreadPool a HystrixCommand  In order to programmatically change a property for an individual element in a cluster, you need to get access to the control reference for that element. There are three ways of gaining access to the control reference of an object inside a cluster in LabVIEW.

Getting started with Hystrix - Chirpy_me, Programmatically changing Hystrix properties. I have a circuit breaker set up that I would like to change parameters for runtime. Things like threads and timeout  Hystrix properties can also be set in our service class inside @HystrixCommand annotation, for this we use the Hystrix-Javanica project which is used for implementing the annotations in our project. For that we need to add the dependency of hystrix-javanica into our classpath.

Introduction to Netflix Archaius with Spring Cloud, You can change it using necessary @HystrixCommand properties: Javanica dynamically sets properties using Hystrix ConfigurationManager. For the example  Is there a way to programmatically change a Property's Type? If so, I could solve this by keeping my single WebContainer1 property and change its Type to either DesktopWebContainer or MobileWebContainer, as needed. Or perhaps there's a more obvious way to solve this problem without having to rewrite my Methods and Handlers.

Spring Cloud, On top of this, the library allows properties to change dynamically at runtime, configurations, so why bother setting up a different mechanism? Re: How to Programmatically Change Linked Picture Equation I'm not exactly sure what you mean. Really, all I want to do is somehow just iterate the column portion of the range reference used by the equation defining what is displayed in the image by 1 each time I click the button.

Comments
  • I am facing a similar issue - once i create a first HystrixCommand with a Setter in a certain group, i can't change the properties later by creating a new HystrixCommand with a different Setter (in the same group). So is it indeed the case that hystrix is somehow "caching" the properties from the first Setter it gets and, except for using ConfigurationManager, there really isn't a way to change the properties?
  • I need to change the properties at runtime programmatically, so you're not really answering my quesiton. Javanica is nice, but we're not having people change params through files. I could theoretically write to a file from code, but that's even more of a hassle and indirect way than what I'm doing now: ConfigurationManager.getConfigInstance().setProperty("...") . I just would like to use a strongly typed API for changing the properties after first initialisation like I did in the first place. Strings and possible typos/misunderstandings just gets in the way. As it did when i switched to it...