Hot questions for Spring Data for Cassandra

Hot questions for Spring Data for Cassandra

Top 10 Java Open Source / Spring / Spring Data for Cassandra

Question:

I am building a Command Line java application using Spring Boot to get it working quickly.

The application loads different types of files (for example CSV) and loads them into a Cassandra Database. It does NOT use any web components, it is not a web application.

The problem I am having is to stop the application when the work is done. I am using the Spring CommandLineRunner interface with a @Component to run the tasks, as shown below, but when the work is completed the application does not stop, it keeps running for some reason and I can't find a way to stop it.

@Component
public class OneTimeRunner implements CommandLineRunner {

    @Autowired
    private CassandraOperations cassandra;

    @Autowired
    private ConfigurableApplicationContext context;

    @Override
    public void run(String... args) throws Exception {
        // do some work here and then quit
        context.close();
    }
}

UPDATE: the problem seems to be spring-cassandra, since there is nothing else in the project. Does anyone know why it keeps threads running in the background that prevent the application from stopping?

UPDATE: the problem disappeared by updating to the latest spring boot version.


Answer:

I found a solution. You can use this:

public static void main(String[] args) {
    SpringApplication.run(RsscollectorApplication.class, args).close();
    System.out.println("done");
}

Just use .close on run.

Question:

Using Cassandra, I want to create keyspace and tables dynamically using Spring Boot application. I am using Java based configuration.

I have an entity annotated with @Table whose schema I want to be created before application starts up since it has fixed fields that are known beforehand.

However depending on the logged in user, I also want to create additional tables for those user dynamically and be able to insert entries to those tables.

Can somebody guide me to some resources that I can make use of or point me in right direction in how to go about solving these issues. Thanks a lot for help!


Answer:

The easiest thing to do would be to add the Spring Boot Starter Data Cassandra dependency to your Spring Boot application, like so...

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-data-cassandra</artifactId>
  <version>1.3.5.RELEASE</version>
</dependency>

In addition, this will add the Spring Data Cassandra dependency to your application.

With Spring Data Cassandra, you can configure your application's Keyspace(s) using the CassandraClusterFactoryBean (or more precisely, the subclass... CassandraCqlClusterFactoryBean) by calling the setKeyspaceCreations(:Set) method.

The KeyspaceActionSpecification class is pretty self-explanatory. You can even create one with the KeyspaceActionSpecificationFactoryBean, add it to a Set and then pass that to the setKeyspaceCreations(..) method on the CassandraClusterFactoryBean.

For generating the application's Tables, you essentially just need to annotate your application domain object(s) (entities) using the SD Cassandra @Table annotation, and make sure your domain objects/entities can be found on the application's CLASSPATH.

Specifically, you can have your application @Configuration class extend the SD Cassandra AbstractClusterConfiguration class. There, you will find the getEntityBasePackages():String[] method that you can override to provide the package locations containing your application domain object/entity classes, which SD Cassandra will then use to scan for @Table domain object/entities.

With your application @Table domain object/entities properly identified, you set the SD Cassandra SchemaAction to CREATE using the CassandraSessionFactoryBean method, setSchemaAction(:SchemaAction). This will create Tables in your Keyspace for all domain object/entities found during the scan, providing you identified the proper Keyspace on your CassandraSessionFactoryBean appropriately.

Obviously, if your application creates/uses multiple Keyspaces, you will need to create a separate CassandraSessionFactoryBean for each Keyspace, with the entityBasePackages configuration property set appropriately for the entities that belong to a particular Keyspace, so that the associated Tables are created in that Keyspace.

Now...

For the "additional" Tables per user, that is quite a bit more complicated and tricky.

You might be able to leverage Spring Profiles here, however, profiles are generally only applied on startup. If a different user logs into an already running application, you need a way to supply additional @Configuration classes to the Spring ApplicationContext at runtime.

Your Spring Boot application could inject a reference to a AnnotationConfigApplicationContext, and then use it on a login event to programmatically register additional @Configuration classes based on the user who logged into the application. You need to follow your register(Class...) call(s) with an ApplicationContext.refresh().

You also need to appropriately handle the situation where the Tables already exist.

This is not currently supported in SD Cassandra, but see DATACASS-219 for further details.

Technically, it would be far simpler to create all the possible Tables needed by the application for all users at runtime and use Cassandra's security settings to restrict individual user access by role and assigned permissions.

Another option might be just to create temporary Keyspaces and/or Tables as needed when a user logs in into the application, drop them when the user logs out.

Clearly, there are a lot of different choices here, and it boils down more to architectural decisions, tradeoffs and considerations then it does technical feasibility, so be careful.

Hope this helps.

Cheers!

Question:

I updated my project to spring-boot Version 2.1.0.RELEASE. Now i get the following error:

Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.data.cassandra.ReactiveSession]: Factory method 'reactiveSession' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'session' defined in class path resource [ch/sbb/kat/fc/config/CassandraConfig.class]: Invocation of init method failed; nested exception is java.lang.NoClassDefFoundError: com/codahale/metrics/JmxReporter
    at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185)
    at org.springframework.beans.factory.support.ConstructorResolver.lambda$instantiate$2(ConstructorResolver.java:615)
    at java.security.AccessController.doPrivileged(Native Method)
    at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:614)
    ... 120 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'session' defined in class path resource [ch/sbb/kat/fc/config/CassandraConfig.class]: Invocation of init method failed; nested exception is java.lang.NoClassDefFoundError: com/codahale/metrics/JmxReporter
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1745)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:576)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:498)
    at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
    at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:339)
    at ch.sbb.kat.fc.config.CassandraConfig$$EnhancerBySpringCGLIB$$a22674d7.session(<generated>)
    at org.springframework.data.cassandra.config.AbstractCassandraConfiguration.getRequiredSession(AbstractCassandraConfiguration.java:66)
    at org.springframework.data.cassandra.config.AbstractReactiveCassandraConfiguration.reactiveSession(AbstractReactiveCassandraConfiguration.java:47)
    at ch.sbb.kat.fc.config.CassandraConfig$$EnhancerBySpringCGLIB$$a22674d7.CGLIB$reactiveSession$7(<generated>)
    at ch.sbb.kat.fc.config.CassandraConfig$$EnhancerBySpringCGLIB$$a22674d7$$FastClassBySpringCGLIB$$7973a63.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244)
    at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:363)
    at ch.sbb.kat.fc.config.CassandraConfig$$EnhancerBySpringCGLIB$$a22674d7.reactiveSession(<generated>)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154)
    ... 123 more
Caused by: java.lang.NoClassDefFoundError: com/codahale/metrics/JmxReporter
    at com.datastax.driver.core.Metrics.<init>(Metrics.java:146)
    at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1501)
    at com.datastax.driver.core.Cluster.init(Cluster.java:208)
    at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:376)
    at com.datastax.driver.core.Cluster.connect(Cluster.java:332)
    at org.springframework.data.cassandra.config.CassandraCqlSessionFactoryBean.connect(CassandraCqlSessionFactoryBean.java:89)
    at org.springframework.data.cassandra.config.CassandraCqlSessionFactoryBean.afterPropertiesSet(CassandraCqlSessionFactoryBean.java:82)
    at org.springframework.data.cassandra.config.CassandraSessionFactoryBean.afterPropertiesSet(CassandraSessionFactoryBean.java:59)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.lambda$invokeInitMethods$5(AbstractAutowireCapableBeanFactory.java:1795)
    at java.security.AccessController.doPrivileged(Native Method)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1794)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1741)
    ... 143 more
Caused by: java.lang.ClassNotFoundException: com.codahale.metrics.JmxReporter
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 155 more

Using the property introduced in this issue https://github.com/spring-projects/spring-boot/issues/14778 seems to have no effect to solve my issue.

How can i disable jmx for cassandra?

My current cassandra config looks like this:

@Configuration
@EnableReactiveCassandraRepositories({"repository"})
public class CassandraConfig extends AbstractReactiveCassandraConfiguration {

    @Value("${cassandra.host}")
    private String host;

    @Override
    protected String getKeyspaceName() {
        return "keyspace";
    }

    @Override
    public String[] getEntityBasePackages() {
        return new String[]{"model"};
    }

    @Override
    public SchemaAction getSchemaAction() {
        return SchemaAction.CREATE_IF_NOT_EXISTS;
    }

    @Override
    public String getContactPoints() {
        return host;
    }
}

Answer:

The spring.data.cassandra.jmx-enabled property is used when Spring Boot is auto-configuring a Cassandra Cluster bean. By extending AbstractReactiveCassandraConfiguration, you are switching off this auto-configuration in favour of the Cluster bean that's created by AbstractClusterConfiguration which is a super-class of AbstractReactiveCassandraConfiguration. As a result, the property has no effect.

There are two ways that you can fix your problem:

  1. Remove your AbstractReactiveCassandraConfiguration sub-class and use the various spring.data.cassandra.* properties to configure things instead.
  2. Override cluster on AbstractClusterConfiguration in CassandraConfig, call super.cluster() to get the CassandraClusterFactoryBean and then call setJmxReportingEnabled(false) on the factory bean before returning it.

Alternatively, if you are not using Dropwizard elsewhere in your application, you may be able to downgrade to an older version that is compatible with Cassandra's JMX reporting by overriding the dropwizard-metrics.version property in your pom.xml or build.gradle.

Question:

Picture the following senario: A Spark application (Java implementation) is using Cassandra database to load, convert to RDD and process the data. Also the application is steaming new data from the database which are also processed by a custom receiver. The output of the streaming process is stored in the database. The implementation is using Spring Data Cassandra from the integration with the database.

CassandraConfig:

@Configuration
@ComponentScan(basePackages = {"org.foo"})
@PropertySource(value = { "classpath:cassandra.properties" })
public class CassandraConfig {

    @Autowired
    private Environment env;

    @Bean
    public CassandraClusterFactoryBean cluster() {
        CassandraClusterFactoryBean cluster = new CassandraClusterFactoryBean();
        cluster.setContactPoints(env.getProperty("cassandra.contactpoints"));
        cluster.setPort(Integer.parseInt(env.getProperty("cassandra.port")));

        return cluster;
    }

    @Bean
    public CassandraMappingContext mappingContext() {
        return new BasicCassandraMappingContext();
    }

    @Bean
    public CassandraConverter converter() {
        return new MappingCassandraConverter(mappingContext());
    }

    @Bean
    public CassandraSessionFactoryBean session() throws Exception {
        CassandraSessionFactoryBean session = new CassandraSessionFactoryBean();
        session.setCluster(cluster().getObject());
        session.setKeyspaceName(env.getProperty("cassandra.keyspace"));
        session.setConverter(converter());
        session.setSchemaAction(SchemaAction.NONE);

        return session;
    }

    @Bean
    public CassandraOperations cassandraTemplate() throws Exception {
        return new CassandraTemplate(session().getObject());
    }

}

DataProcessor.main method:

// Initialize spring application context
ApplicationContext applicationContext = new AnnotationConfigApplicationContext(CassandraConfig.class);
ApplicationContextHolder.setApplicationContext(applicationContext);
CassandraOperations cassandraOperations = applicationContext.getBean(CassandraOperations.class);
// Initialize spark context
SparkConf conf = new SparkConf().setAppName("test-spark").setMaster("local[2]");
JavaSparkContext sc = new JavaSparkContext(conf);

// Load data pages
List<Event> pagingResults = cassandraOperations.select("select * from event where event_type = 'event_type1' order by creation_time desc limit " + DATA_PAGE_SIZE, Event.class);
// Parallelize the first page
JavaRDD<Event> rddBuffer = sc.parallelize(pagingResults);

while(pagingResults != null && !pagingResults.isEmpty()) {
    Event lastEvent = pagingResults.get(pagingResults.size() - 1);
    pagingResults = cassandraOperations.select("select * from event where event_type = 'event_type1' and creation_time < " + lastEvent.getPk().getCreationTime() + " order by creation_time desc limit " + DATA_PAGE_SIZE, Event.class);
    // Parallelize page and add to the existing
    rddBuffer = rddBuffer.union(sc.parallelize(pagingResults));
}

// data processing
...

It is expected to have a big amount of data for the initial loading. For this reason the data are paginated, loaded and distributed in rddBuffer.

There are also the following options available:

  1. The Spark-Cassandra example (https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/CassandraCQLTest.scala), although there is minimum amount of documentation for this example.
  2. The Calliope project (http://tuplejump.github.io/calliope/)

I would like to know what is the best practice for the integration of Spark with Cassandra. What would be the best option to follow in my implementation?

Apache Spark 1.0.0, Apache Cassandra 2.0.8


Answer:

The easiest way to work with Cassandra and Spark is to use the official open source Cassandra driver for Spark developed by DataStax: https://github.com/datastax/spark-cassandra-connector

This driver has been built on top of Cassandra Java Driver and provides a direct bridge between Cassandra and Spark. Unlike Calliope, it does not use the Hadoop interface. Additionally it offers the following unique features:

  • support for all Cassandra data types, including collections, out of the box
  • lightweight mapping of Cassandra rows to custom classes or tuples without the need to use any implicits or other advanced features in Scala
  • saving any RDDs to Cassandra
  • full support for Cassandra Virtual Nodes
  • ability to filter / select on the server side, e.g. leveraging Cassandra clustering columns or secondary indexes

Question:

In my Spring Boot 1.5.1 application I'm going to write unit/integration tests for Cassandra related logic.

I have added folowing Maven dependency:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-cassandra</artifactId>
</dependency>

The default Spring Boot Cassandra configuration is going to connect with a real Cassandra DB server.

Is there any options in Spring/Spring Boot in order to configure my tests to use embedded Cassandra server ? If so, could you please show the required configuration.


Answer:

We use on the project Cassandra + Spring Boot. Here are the steps which worked for us:

a) Configure you test like this

import org.cassandraunit.spring.CassandraDataSet;
import org.cassandraunit.spring.CassandraUnitDependencyInjectionTestExecutionListener;
import org.cassandraunit.spring.CassandraUnitTestExecutionListener;
import org.cassandraunit.spring.EmbeddedCassandra;

@RunWith(SpringRunner.class)
@SpringBootTest(classes = TestConfiguration.class)
@TestExecutionListeners(listeners = {
    CassandraUnitDependencyInjectionTestExecutionListener.class,
    CassandraUnitTestExecutionListener.class,
    ServletTestExecutionListener.class,
    DependencyInjectionTestExecutionListener.class,
    DirtiesContextTestExecutionListener.class
})
@EmbeddedCassandra(timeout = 60000)
@CassandraDataSet(value = {"bootstrap_test.cql"}, keyspace = "local_test")
public abstract class BaseTest {

b) in your src/test/resources/application.properties, add this (please note, embedded cassandra starts on port 9142, but not on default 9042)

spring.data.cassandra.port=9142
spring.data.cassandra.keyspace-name=local_test

c) Create empty file bootstrap_test.cql in src/test/resources

d) Add to your pom.xml

    <dependency>
        <groupId>org.cassandraunit</groupId>
        <artifactId>cassandra-unit-spring</artifactId>
        <version>${cassandra-unit.version}</version>
        <scope>test</scope>
        <exclusions>
            <exclusion>
                <groupId>org.cassandraunit</groupId>
                <artifactId>cassandra-unit</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>org.cassandraunit</groupId>
        <artifactId>cassandra-unit-spring</artifactId>
        <version>${cassandra-unit.version}</version>
    </dependency>

This should be enough to run your tests with Embedded Cassandra. Hope it helps.

Question:

I'm working on a project where we have an entity we want to persist with a field of type LocalDateTime, we know cassandra does not have native support for this type conversion, we've created our own custom converter using Spring's support for converters, however it seems Spring-Data-Cassandra is unable to either recognise them or understand that the field should be mapped to a column.

This is how we've registered our converters with Spring's conversion service.

<bean id="conversionService" class="org.springframework.context.support.ConversionServiceFactoryBean">
        <property name="converters">
            <set>
                <!-- list of converters-->
                <bean class="com.klappo.userservice.api.persistence.converters.DateToLocalDateTimeConverter" />
                <bean class="com.klappo.userservice.api.persistence.converters.LocalDateTimeToDateConverter" />
                <bean class="com.klappo.userservice.api.persistence.converters.DateToLocalDateConverter" />
                <bean class="com.klappo.userservice.api.persistence.converters.LocalDateToDateConverter" />
            </set>
        </property>
    </bean>

The end result is the following exception thrown on startup of our application:

    Caused by: 
org.springframework.data.cassandra.mapping.VerifierMappingExceptions: java.time.LocalDate:|Cassandra entities must have the @Table, @Persistent or @PrimaryKeyClass Annotation|
    at org.springframework.data.cassandra.mapping.BasicCassandraPersistentEntityMetadataVerifier.verify(BasicCassandraPersistentEntityMetadataVerifier.java:45)
    at org.springframework.data.cassandra.mapping.BasicCassandraPersistentEntity.verify(BasicCassandraPersistentEntity.java:198)
    at org.springframework.data.mapping.context.AbstractMappingContext.addPersistentEntity(AbstractMappingContext.java:297)
    at org.springframework.data.mapping.context.AbstractMappingContext$PersistentPropertyCreator.createAndRegisterProperty(AbstractMappingContext.java:469)
    at org.springframework.data.mapping.context.AbstractMappingContext$PersistentPropertyCreator.doWith(AbstractMappingContext.java:426)
    at org.springframework.util.ReflectionUtils.doWithFields(ReflectionUtils.java:607)
    at org.springframework.data.mapping.context.AbstractMappingContext.addPersistentEntity(AbstractMappingContext.java:294)
    at org.springframework.data.mapping.context.AbstractMappingContext$PersistentPropertyCreator.createAndRegisterProperty(AbstractMappingContext.java:469)
    at org.springframework.data.mapping.context.AbstractMappingContext$PersistentPropertyCreator.doWith(AbstractMappingContext.java:426)
    at org.springframework.util.ReflectionUtils.doWithFields(ReflectionUtils.java:607)
    at org.springframework.data.mapping.context.AbstractMappingContext.addPersistentEntity(AbstractMappingContext.java:294)
    at org.springframework.data.mapping.context.AbstractMappingContext.addPersistentEntity(AbstractMappingContext.java:256)
    at org.springframework.data.mapping.context.AbstractMappingContext.initialize(AbstractMappingContext.java:372)
    at org.springframework.data.cassandra.mapping.BasicCassandraMappingContext.initialize(BasicCassandraMappingContext.java:79)
    at org.springframework.data.mapping.context.AbstractMappingContext.afterPropertiesSet(AbstractMappingContext.java:362)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1612)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1549)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:539)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:302)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:298)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:328)
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:108)
    at org.springframework.beans.factory.support.ConstructorResolver.resolveConstructorArguments(ConstructorResolver.java:632)
    at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:140)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1114)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1017)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:302)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:298)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:328)
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:108)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1456)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1197)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:537)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:302)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:298)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:328)
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:108)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1456)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1197)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:537)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:302)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:298)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:328)
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:108)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1456)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1197)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:537)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:302)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:298)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1017)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:960)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:858)
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:480)
    at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:87)
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:289)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1185)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:537)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:302)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:298)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:703)
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:760)
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
    at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:403)
    at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:306)
    at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:106)
    at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800)
    at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)
    at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791)
    at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)
    at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
    at org.eclipse.jetty.maven.plugin.JettyWebAppContext.startWebapp(JettyWebAppContext.java:296)
    at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
    at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
    at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
    at org.eclipse.jetty.maven.plugin.JettyWebAppContext.doStart(JettyWebAppContext.java:365)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:163)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
    at org.eclipse.jetty.server.Server.start(Server.java:387)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
    at org.eclipse.jetty.server.Server.doStart(Server.java:354)
    at org.eclipse.jetty.maven.plugin.JettyServer.doStart(JettyServer.java:73)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.eclipse.jetty.maven.plugin.AbstractJettyMojo.startJetty(AbstractJettyMojo.java:534)
    at org.eclipse.jetty.maven.plugin.AbstractJettyMojo.execute(AbstractJettyMojo.java:357)
    at org.eclipse.jetty.maven.plugin.JettyRunMojo.execute(JettyRunMojo.java:167)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:132)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:120)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:347)
    at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:154)
    at org.apache.maven.cli.MavenCli.execute(MavenCli.java:582)
    at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
    at org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:483)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
    at org.codehaus.classworlds.Launcher.main(Launcher.java:46)

Spring's data documentation on this is lacking quite a bit, so we're wondering if anyone has attempted to use custom type converters with spring-data-cassandra and if so, we'd like to know what's the right way of doing it.

Thanks in advance.


Answer:

Unfortunatey the spring-data-cassandra ver. <= 1.2.x could be used only with beans which could be directly mapped to datatypes from this list: http://www.datastax.com/documentation/developer/java-driver/1.0/java-driver/reference/javaClass2Cql3Datatypes_r.html

I met the same problem, because wanted to map my classes into TEXT fields in Cassandra, but couldn't do it. After several days of trying I came to the conclusion that it is not possible to use custom type converters in Spring-Data for Cassandra (tested with version 1.2.0.M1).

CassandraTemplate uses MappingCassandraConverter which has embedded DefaultConversionService, but this DefaultConversionService is not used anywhere in the spring-data-cassandra module. It was introduced only for integration with SpringData API.

You could find in the code of library that conversionService is using only in

BeanWrapper<S> wrapper = BeanWrapper.create(instance, conversionService);

which just skips conversionService:

@Deprecated
public static <T> BeanWrapper<T> create(T bean, ConversionService conversionService) {
    return new BeanWrapper<T>(bean);
}

You could see in MappingCassandraConverter.readPropertyFromRow(...) which uses ColumnReader.get(int i) where supported only Cassandra's MAP, LIST, SET and primary data types (Text, Numbers, UUID, java.util.Date, Boolean, ByteBuffer).

In addition BasicCassandraMappingContext.verifier (BasicCassandraPersistentEntityMetadataVerifier) throw exception on attempt to write any field which could not be mapped to supported types of Cassandra or is not @Table.

That's why you had the the exception

// it was attempt to check conversion of an embedded field 'date' in LocalDateTime    
org.springframework.data.cassandra.mapping.VerifierMappingExceptions: java.time.LocalDate

So the only way to solve it is using ConversionService on top of spring-data-cassandra for your needs.

Sample code for conversion service part:

import org.springframework.data.convert.Jsr310Converters;

...    

@Bean
public ConversionService getConversionService() {
    // creates DefaultConversionService
    ConversionServiceFactoryBean bean = new ConversionServiceFactoryBean();
    bean.setConverters(new HashSet<>(getConverters()));
    bean.afterPropertiesSet();
    ConversionService service = bean.getObject();
    return service;
}

private Set<Converter<?, ?>> getConverters() {
    Set<Converter<?,?>> converters = new HashSet<Converter<?,?>>();
    converters.addAll(Jsr310Converters.getConvertersToRegister());
    converters.add(new TimeWriteConverter());
    converters.add(new TimeReadConverter());

    return converters;
}

public static class TimeWriteConverter implements Converter<LocalDateTime, Long> {
    @Override
    public Long convert(LocalDateTime source) {
        return source.atZone(ZoneId.systemDefault()).toInstant().toEpochMilli();
    }
}
public static class TimeReadConverter implements Converter<Long, LocalDateTime> {
    @Override
    public LocalDateTime convert(Long source) {
        return LocalDateTime.ofInstant(Instant.ofEpochMilli(source), ZoneId.systemDefault());
    }
}

Question:

I am working on Cassandra Spring-data module using Scala. I have created domain model & setup crud repository.

Spring data repository is setup as below:

@Repository
trait SampleRepository extends CassandraRepository[Sample, SampleKey]

Table domain model is setup as below:

@Table
 case class Sample(@PrimaryKey
              sampleKey: SampleKey,
              @Column("surName")
              surName: String)

Primary Key is composite key. Id column as Partition key. name column as Cluster key

@PrimaryKeyClass
case class SampleKey(
                  @PrimaryKeyColumn( `type` = PrimaryKeyType.PARTITIONED, ordinal = 0, name = "id")
                  id: Int,
                  @PrimaryKeyColumn( `type` = PrimaryKeyType.CLUSTERED, ordinal = 1,  name = "name")
                  name: String
                )

In scala type is a valid keyword hence used backward quotes. I am expecting this to work but when I start the application but I get

Suppressed: org.springframework.data.mapping.MappingException: Composite primary key type [com.barclays.decisionstore.trac.model.SampleKey] has no fields annotated with @PrimaryKeyColumn
    at org.springframework.data.cassandra.core.mapping.PrimaryKeyClassEntityMetadataVerifier.verify(PrimaryKeyClassEntityMetadataVerifier.java:91)
    ... 262 common frames omitted
Suppressed: org.springframework.data.mapping.MappingException: At least one of the @PrimaryKeyColumn annotations must have a type of PARTITIONED
    at org.springframework.data.cassandra.core.mapping.PrimaryKeyClassEntityMetadataVerifier.verify(PrimaryKeyClassEntityMetadataVerifier.java:98)
    ... 262 common frames omitted

This means type parameter to annotation is not recognized by Spring-Data. The same code with java model classes works without issue.

Would appreciate any pointers or substitute to solve this.


Answer:

Actually java bean properties not recognized other than annotations.

Scala does not bind instance variables as bean properties by default at case classes. For this you need to add @BeanProperty annotations to each properties.

import scala.beans.BeanProperty

@PrimaryKeyClass
case class SampleKey(@BeanProperty
                     @PrimaryKeyColumn(`type` = PrimaryKeyType.PARTITIONED, ordinal = 0, name = "id")
                     id: Int,
                     @BeanProperty
                     @PrimaryKeyColumn(`type` = PrimaryKeyType.CLUSTERED, ordinal = 1, name = "name")
                     name: String)

Also if you get an object construction error, add a constructor with default values (this requires to make variables var):

@PrimaryKeyClass
case class SampleKey(@BeanProperty
                     @PrimaryKeyColumn(`type` = PrimaryKeyType.PARTITIONED, ordinal = 0, name = "id")
                     var id: Int,
                     @BeanProperty
                     @PrimaryKeyColumn(`type` = PrimaryKeyType.CLUSTERED, ordinal = 1, name = "name")
                     var name: String) {
  def this() = this(0, "")
}

Hope this helps!