Hot questions for Spring Cloud Bus

Top 10 Java Open Source / Spring / Spring Cloud Bus

Question:

I'm trying to use Spring cloud bus with Kafka in my microservices application, and indeed I could use it, but only data which is controlled by Spring cloud config server got refreshed!

I'm using jdbc back-end with my config server, and in order to simulate my need, I'm changing some value in properties file in one of my services, beside the properties table, and call the /monintor end point again (mentioned here section 4.3 https://www.baeldung.com/spring-cloud-bus); as a result, only data coming from properties table is changed.

This is the yml file for my Config server

spring:
  cloud:
    config:
      server:
        jdbc:
          sql: SELECT KEY,VALUE from PROPERTIES where APPLICATION=? and PROFILE=? and LABEL=?
          order: 1
    stream:
      kafka:
        binder:
          brokers: localhost:9092
  datasource:
    url: jdbc:mysql://localhost:3306/sweprofile?zeroDateTimeBehavior=convertToNull
    username: 123
    password: 123ertbnm
    hikari:
      maximum-pool-size: 10
      connection-timeout: 5000
  profiles:
    active:
      - jdbc
  application:
    name: configServer

These are yml files for One of my Miscroservices and its propeties file respectively

spring:
  datasource:
    username: 123
    password: 123ertbnm
    url: jdbc:mysql://localhost:3306/sweprofile?zeroDateTimeBehavior=convertToNull
    jpa:
      properties:
        hibernate:
          format_sql: true
          ddl-auto: none
  application:
    name: auth-service
  cloud:
    config:
      discovery:             
        enabled: true
        service-id: configServer
    bus:
      refresh:
        enabled: true
    profiles:
      active: jdbc

management:
  endpoints:
    web:
      exposure:
        include: ["health","info","refresh", "bus-refresh"]
# This line is dummy data for testing purpose 
ali.man = " Ola 12333"

This is snapshot from rest controller

@RestController
@RequestMapping("/user")
@RefreshScope
public class AuthController {
    private UserAuthService userAuthService;

    @Value("${name}")
    private String name;   // changed normally

    // Calling the key value mentioned in properties file after changing
    @Value("${ali.man}")
    private String k;      // -> not changed

    public AuthController(UserAuthService userAuthService) {
        this.userAuthService = userAuthService;
    }

    @GetMapping("authTest")
    public String getAuth() {
        return name + k;
    }
}

What did I miss? Why value from Properties file is not changed? hopefully I can use Spring cloud bus with Kafka to refresh these external data.


Answer:

After some hours of investigation, I found that there is some recommended way. Cloud bus can send Refresh Event and Spring boot can listen to that event; this what I build my solution on, I used this snippet

@Configuration
public class ReloadLookupEvent implements ApplicationListener<RefreshScopeRefreshedEvent> {
    @Autowired
    private CacheService cacheService;

    @Override
    public void onApplicationEvent(RefreshScopeRefreshedEvent event) {
        cacheService.refreshLookUp();          // The target load 
    }

}

I could refresh all other configuration on demand, maybe it is a workaround, but applicable and acceptable.

Question:

This is simple code in my Spring boot application:

package com.maxxton.SpringBootHelloWorld;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class SpringBootHelloWorldApplication {

  public static void main(String[] args) {
    SpringApplication.run(SpringBootHelloWorldApplication.class, args);
  }
}

And a ApplicationListener class to listen to ApplicationEvent:

package com.maxxton.SpringBootHelloWorld;

import org.springframework.context.ApplicationEvent;
import org.springframework.context.ApplicationListener;
import org.springframework.stereotype.Component;

@Component
public class Test implements ApplicationListener {

  @Override
  public void onApplicationEvent(ApplicationEvent event) {
    if (event.getClass().getSimpleName().equals("ApplicationReadyEvent")) {
      System.out.println("-------------------------------------");
      System.out.println(event.getClass().getSimpleName());
      System.out.println("-------------------------------------");
    }
  }
}

build.gradle contains these dependencies:

dependencies {

    compile("org.springframework.boot:spring-boot-starter-amqp")
    compile("org.springframework.cloud:spring-cloud-starter-bus-amqp")

    compile('org.springframework.boot:spring-boot-starter-web')
    compile('org.springframework.boot:spring-boot-starter')
    compile("org.springframework.cloud:spring-cloud-starter")
    compile("org.springframework.cloud:spring-cloud-starter-security")
    compile("org.springframework.cloud:spring-cloud-starter-eureka")
    testCompile('org.springframework.boot:spring-boot-starter-test')
}

Now, when I run this spring boot application, I see this log printed twice:

[main] c.m.S.SpringBootHelloWorldApplication : Started SpringBootHelloWorldApplication in ... seconds (JVM running for ...)

Usually, this log get printed only once, but it get printed twice if I add these dependencies:

compile("org.springframework.boot:spring-boot-starter-amqp")
compile("org.springframework.cloud:spring-cloud-starter-bus-amqp")

This is complete log:

2017-11-17 15:44:07.372  INFO 5976 --- [           main] o.s.c.support.GenericApplicationContext  : Refreshing org.springframework.context.support.GenericApplicationContext@31c7c281: startup date [Fri Nov 17 15:44:07 IST 2017]; root of context hierarchy
-------------------------------------
ApplicationReadyEvent
-------------------------------------
2017-11-17 15:44:07.403  INFO 5976 --- [           main] c.m.S.SpringBootHelloWorldApplication    : Started SpringBootHelloWorldApplication in 1.19 seconds (JVM running for 10.231)
2017-11-17 15:44:09.483  WARN 5976 --- [           main] o.s.amqp.rabbit.core.RabbitAdmin         : Failed to declare exchange: Exchange [name=springCloudBus, type=topic, durable=true, autoDelete=false, internal=false, arguments={}], continuing... org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection refused: connect
2017-11-17 15:44:09.492  INFO 5976 --- [           main] o.s.integration.channel.DirectChannel    : Channel 'a-bootiful-client.springCloudBusOutput' has 1 subscriber(s).
2017-11-17 15:44:09.493  INFO 5976 --- [           main] o.s.c.support.DefaultLifecycleProcessor  : Starting beans in phase 0
2017-11-17 15:44:09.530  INFO 5976 --- [           main] o.s.i.endpoint.EventDrivenConsumer       : Adding {logging-channel-adapter:_org.springframework.integration.errorLogger} as a subscriber to the 'errorChannel' channel
2017-11-17 15:44:09.530  INFO 5976 --- [           main] o.s.i.channel.PublishSubscribeChannel    : Channel 'a-bootiful-client.errorChannel' has 1 subscriber(s).
2017-11-17 15:44:09.530  INFO 5976 --- [           main] o.s.i.endpoint.EventDrivenConsumer       : started _org.springframework.integration.errorLogger
2017-11-17 15:44:09.530  INFO 5976 --- [           main] o.s.c.support.DefaultLifecycleProcessor  : Starting beans in phase 2147482647
2017-11-17 15:44:09.539  INFO 5976 --- [           main] c.s.b.r.p.RabbitExchangeQueueProvisioner : declaring queue for inbound: springCloudBus.anonymous.kZ1vvxHaRfChKe1TncH-MQ, bound to: springCloudBus
2017-11-17 15:44:11.562  WARN 5976 --- [           main] o.s.amqp.rabbit.core.RabbitAdmin         : Failed to declare exchange: Exchange [name=springCloudBus, type=topic, durable=true, autoDelete=false, internal=false, arguments={}], continuing... org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection refused: connect
2017-11-17 15:44:13.587  WARN 5976 --- [           main] o.s.amqp.rabbit.core.RabbitAdmin         : Failed to declare queue: Queue [name=springCloudBus.anonymous.kZ1vvxHaRfChKe1TncH-MQ, durable=false, autoDelete=true, exclusive=true, arguments={}], continuing... org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection refused: connect
2017-11-17 15:44:15.611  WARN 5976 --- [           main] o.s.amqp.rabbit.core.RabbitAdmin         : Failed to declare binding: Binding [destination=springCloudBus.anonymous.kZ1vvxHaRfChKe1TncH-MQ, exchange=springCloudBus, routingKey=#], continuing... org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection refused: connect
2017-11-17 15:44:17.662  INFO 5976 --- [           main] o.s.i.a.i.AmqpInboundChannelAdapter      : started inbound.springCloudBus.anonymous.kZ1vvxHaRfChKe1TncH-MQ
2017-11-17 15:44:17.662  INFO 5976 --- [           main] o.s.i.endpoint.EventDrivenConsumer       : Adding {message-handler:inbound.springCloudBus.default} as a subscriber to the 'bridge.springCloudBus' channel
2017-11-17 15:44:17.662  INFO 5976 --- [           main] o.s.i.endpoint.EventDrivenConsumer       : started inbound.springCloudBus.default
2017-11-17 15:44:17.663  INFO 5976 --- [           main] o.s.c.support.DefaultLifecycleProcessor  : Starting beans in phase 2147483647
2017-11-17 15:44:17.714  INFO 5976 --- [           main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat started on port(s): 8080 (http)
-------------------------------------
ApplicationReadyEvent
-------------------------------------
2017-11-17 15:44:17.717  INFO 5976 --- [           main] c.m.S.SpringBootHelloWorldApplication    : Started SpringBootHelloWorldApplication in 20.131 seconds (JVM running for 20.545)

As you can see, ApplicationReadyEvent is happening twice.

Why is this happening? Is there any way to avoid this?


Answer:

spring-cloud-bus uses spring-cloud-stream which puts the binder in a separate boot child application context.

You should make your event listener aware of the application context it is running in. You can also use generics to select the event type you are interested in...

@Component
public class Test implements ApplicationListener<ApplicationReadyEvent>, 
                             ApplicationContextAware {

    private ApplicationContext applicationContext;

    @Override
    public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
        this.applicationContext = applicationContext;
    }

    @Override
    public void onApplicationEvent(ApplicationReadyEvent event) {
        if (event.getApplicationContext().equals(this.applicationContext)) {
            System.out.println("-------------------------------------");
            System.out.println(event.getClass().getSimpleName());
            System.out.println("-------------------------------------");
        }
    }

}

Question:

We use Spring Cloud Config (Dalston.SR5), with Cloud clients using Spring Boot 2.x, Spring Cloud Bus, and Finchley.SR1.

I understand from this answer why a Cloud Client application bootstraps with Config for the parent SpringBootApplication and then again once the Cloud Bus is bound. I'm fine with that.

My question is whether there is any way to distinguish the two bootstrap requests?

The reason I ask is that our Config server generates credentials and returns them to the client to authenticate with. Two bootstraps means two sets of credentials, only one of which gets used, and this is wasteful.

As far as I can tell the same bootstrap payload is sent each time by ConfigServicePropertySourceLocator, which gives Config no chance.

Is there an override / hook so that I can let Config know not to generate credentials second time around?

(I could tackle from the Config/server side, but that would be a bit desperate, and I'm reluctant to try to manage state - across two otherwise identical requests that just happen to be ~ 20 seconds apart.)


Best idea I have at the moment is to subclass PropertySourceBootstrapConfiguration and update spring.factories as per:

# Bootstrap components
org.springframework.cloud.bootstrap.BootstrapConfiguration=\
org.springframework.cloud.bootstrap.config.MyCountingPropertySourceBootstrapConfiguration,\

Before making any requests, I should be able to examine the PropertySources and look for any of the properties that the first successful bootstrap would have returned. If present, I'd try to get an additional label or profile into ConfigServicePropertySourceLocator for my Config server to pick up second time around.

I guess that could work, but is there a cleaner / more Spring Boot-y way?


Answer:

This solution seems to be both simple and very effective.

New autoconfiguration to take control over ConfigServicePropertySourceLocator:

@Configuration
@AutoConfigureBefore(ConfigServiceBootstrapConfiguration.class)
public class ConfigPropertyLocatorConfiguration {

    @Bean
    @ConditionalOnProperty(value = "spring.cloud.config.enabled", matchIfMissing = true)
    public ConfigServicePropertySourceLocator configServicePropertySource(ConfigClientProperties properties) {
        return new CachingConfigServicePropertySourceLocator(properties);
    }
}

spring.factories:

org.springframework.cloud.bootstrap.BootstrapConfiguration=\
  autoconfigure.ConfigPropertyLocatorConfiguration

Caching locator implementation:

public class CachingConfigServicePropertySourceLocator extends
                                           ConfigServicePropertySourceLocator {

    private final static Logger LOG = getLogger("...");

    private PropertySource<?> cachedProperties;

    public CachingConfigServicePropertySourceLocator(ConfigClientProperties props) {
        super(props);
    }

    public PropertySource<?> locate(final Environment env) {
        if (cachedProperties == null) {
            cachedProperties = super.locate(env);
        }
        else {
            LOG.debug("Returning cached PropertySource for second bootstrap");
        }

        return cachedProperties;
    }
}

Having been presented with a second opportunity to bootstrap, it seems a little rude to ignore it completely and return the same PropertySource once again - but in our situation this is fine. It prevents us hitting the Config Server twice and wastefully generating credentials.

No changes to PropertySourceBootstrapConfiguration required. No changes to the Cloud Config Server.

Question:

My Spring Cloud Config Client has dependency to spring.cloud.starter.bus.amqp, but it is still not enabling /bus/refresh endpoint

build.gradle    
compile("org.springframework.cloud:spring-cloud-starter-stream-rabbit:1.1.3.RELEASE")    
compile("org.springframework.cloud:spring-cloud-starter-bus-amqp:1.2.2.RELEASE")

I have these dependencies in my config client application, but still not enabling /bus/refresh, /bus/env.

Please let me know what am I missing in my client application.

Note:

spring.cloud.bus.refresh.enabled: true
spring.cloud.bus.env.enabled: true
endpoints.spring.cloud.bus.refresh.enabled: true
endpoints.spring.cloud.bus.env.enabled: true

I have tried setting up these indicators in application.yml or application.properties as these are used by BusAutoConfiguration, to enable /bus/* endpoints.

@ConditionalOnProperty(value = "endpoints.spring.cloud.bus.refresh.enabled", matchIfMissing = true)

In my Spring Cloud Config Server application I have disabled these endpoints, i.e., set to false

endpoints.spring.cloud.bus.refresh.enabled: false
endpoints.spring.cloud.bus.env.enabled: false

and observed that during Spring Boot startup /bus/* endpoints are not being enabled.


Answer:

Did you map client's url to /bus/refresh? I believe it's mapped to /refresh by default.

You could also try sending a POST request to client app at:

curl -X POST http://server:port/refresh

I also believe you might not need spring-cloud-starter-stream-rabbit dependency, just spring-cloud-starter-bus-amqp.

BTW I published a detailed post with working demo at: Refreshable Configuration using Spring Cloud Config Server, Spring Cloud Bus, RabbitMQ and Git that might help you as a starting point.