Hot questions for Cache in Guava

What is the best way to cache single object within fixed timeout?

Question: Google Guava has CacheBuilder that allows to create ConcurrentHash with expiring keys that allow to remove entries after the fixed tiemout. However I need to cache only one instance of certain type.

What is the best way to cache single object within fixed timeout using Google Guava?

Answer: use Guava's Suppliers.memoizeWithExpiration(Supplier delegate, long duration, TimeUnit unit)

public class JdkVersionService {

    @Inject
    private JdkVersionWebService jdkVersionWebService;

    // No need to check too often. Once a year will be good :) 
    private final Supplier<JdkVersion> latestJdkVersionCache
            = Suppliers.memoizeWithExpiration(jdkVersionSupplier(), 365, TimeUnit.DAYS);


    public JdkVersion getLatestJdkVersion() {
        return latestJdkVersionCache.get();
    }

    private Supplier<JdkVersion> jdkVersionSupplier() {
        return new Supplier<JdkVersion>() {
            public JdkVersion get() {
                return jdkVersionWebService.checkLatestJdkVersion();
            }
        };
    }
}

using JDK 8 method references and constructor injection for cleaner code:

import java.util.concurrent.TimeUnit;
import java.util.function.Supplier;
import javax.inject.Inject;
import org.springframework.stereotype.Service;
import com.google.common.base.Suppliers;

@Service
public class JdkVersionService {

    private final Supplier<JdkVersion> latestJdkVersionCache;

    @Inject
    public JdkVersionService(JdkVersionWebService jdkVersionWebService) {
        this.latestJdkVersionCache = Suppliers.memoizeWithExpiration(
                jdkVersionWebService::checkLatestJdkVersion,
                365, TimeUnit.DAYS
        );
    }

    public JdkVersion getLatestJdkVersion() {
        return latestJdkVersionCache.get();
    }
}

Bad performance with Guava Cache on Android

Question: We use a loading Google Guava LoadingCache for bitmaps in an Android application. In the application I am running a drawing Thread, that paints the bitmaps in the cache to a Canvas. If a specific bitmap is not in the cache, it does not get drawn so that no loading will ever block the drawing Thread.

However, the painting results in visual stuttering and the frames per second rate is not how we would like it. I nailed it down to the getIfPresent() method of the cache. That alone takes over 20% of the applications total CPU time. In getIfPresent() LocalCache$Segment.get() takes over 80% of the time

Bear in mind, this is only a lookup of an already present bitmap. There will never happen a load in get(). I figured there would be a bookkeeping overhead in get() for the LRU queue that decides which eviction takes place if the segment is full. But this is at least an order of magnitude slower of what a Key-Lookup in LRU-LinkedHashmap.get() would give me.

We use a cache to get fast lookups if an element is in the cache, if the lookup is slow, there is no point in caching it. I also tried getAllPresent(a) and asMap() but it gives equal performance.

LoadingCache is defined as follows:

LoadingCache<TileKey, Bitmap> tiles = CacheBuilder.newBuilder().maximumSize(100).build(new CacheLoader<TileKey,Bitmap>() {
            @Override
            public Bitmap load(TileKey tileKey) {
            System.out.println("Loading in " + Thread.currentThread().getName() + " "
                + tileKey.x + "-" + tileKey.y);

            final File[][] tileFiles = surfaceState.mapFile.getBuilding()
                .getFloors().get(tileKey.floorid)
                .getBackground(tileKey.zoomid).getTileFiles();
            String tilePath = tileFiles[tileKey.y][tileKey.x].getAbsolutePath();

            Options options = new BitmapFactory.Options();
            options.inPreferredConfig = Bitmap.Config.RGB_565;

            return BitmapFactory.decodeFile(tilePath, options);
            }
        });

Answer: CacheBuilder was designed for server-side caching, where concurrency was a primary concern. It therefore trades off single-threaded and memory overhead in exchange for better multi-threaded behavior. Android developers should use LruCache, LinkedHashMap, or similar where single-threaded performance and memory are the primary concerns. In the future there may be a concurrencyLevel=0 to indicate that a lightweight, non-concurrent cache is required.


Why is Cache.asMap() not consistent with Cache.size()?

Question: In the Guava Library, I am confused about why Cache.asMap() is not consistent with Cache.size(), unless Cache.cleanUp() is called.

Cache<Object, Object> cache = CacheBuilder.newBuilder()
           .expireAfterWrite(1, TimeUnit.SECONDS)
           .build();
cache.get(...);
...
//After some seconds, all entries are expired.
//cache.asMap() is EMPTY Map, but cache.size() != 0

So my question: Is it bug that Cache.asMap() is not consistent to Cache.size()? Although I notice the javadoc of Cache.size() is:

 /**
   * Returns the **approximate** number of entries in this cache.
   */

I can just guess it's related to a concurrent environment. And what does Cache.cleanUp() do exactly?

Answer: Guava's cache is designed around lock amortization and the cleanUp method forces the cache to arrive at a consistent state. The Map.size() method is an approximation, but may count entries pending removal due to expiration or reference eviction. The visibility of the approximations in Guava's cache are rarely of much interest to an application, which tends to think of a cache as a transient data store. The different expectations of a cache from a Map led to the asMap method to allow the cache to be viewed as a map, but disfavor developers perceiving it that way.

asMap() views have the luxury of traversing each element in the cache, and thus can skip invalid entries which are pending cleanup. On the other hand, size() is expected to be a quick operation, and it would be silly to traverse the entire cache just to get a more accurate size estimate.

The CacheBuilder javadocs go into more detail regarding the cleanup that needs to happen in various situations (such as expireAfterWrite, in your case).


Google Guava's CacheLoader loadAll() method Implementation Issues

Question: I am interested in knowing what is the effective way of loadAll method implementation introduced in google guava library.

Here is the following code that describes load all method implementation extended as per the example from achesExplained

LoadingCache<Key, Graph> graphs = CacheBuilder.newBuilder().maximumSize(1000)
.refreshAfterWrite(1, TimeUnit.MINUTES)
.build(
   new CacheLoader<Key, Graph>() {
     public Graph load(Key key) { // no checked exception
       return getGraphFromDatabase(key);
     }

     public Map<Key, Graph> loadAll(Iterable<? extends K> keys) {
         return getAllGraphsFromDatabase(keys);
     }
   }
);

private Map<Key, Graph> getAllGraphsFromDatabase(Iterable<? extends key> keys)
{
  lListOfGraph = //resultset got from DB Call
  for (lCount = 0; lCount < lListOfGraph.size(); lCount++)
  {
     lGraph = (Graph)lListOfGraph.get(lCount).get(0);
     graphs.asMap().put((key , lGraph);
  }
  return (Map<key, Graph>) graphs;
}

Here return type that is Map throws error

java.lang.ClassCastException:com.google.common.cache.LocalCache$LocalLoadingCache cannot be cast to java.util.Map (Knowing the fact that Loading Cache object can not be of type Map)

If this is not the correct way of implementation of using LoadingCache then How is the data injected in LoadingCache's Component so that it can be used as Cache.

Answer: Your getAllGraphsFromDatabase method should be fetching the values from the underlying data store. The LoadingCache implementation handles adding the returned values to the map for you.

I think your loading method should look like:

private Map<Key, Graph> getAllGraphsFromDatabase(Iterable<? extends key> keys)
{
  final List<Graph> lListOfGraph = //resultset got from DB Call

  final Map<Key, Graph> map = new HashMap<Key, Graph>(listOfGraph.size());
  for (final Graph graph : lListOfGraph)
    map.put(graph.getKey(), graph);

  return map;
}

Google Guava Cache - Change the eviction timeout values during run time

Question: I am using the following:

LoadingCache<String, Long> inQueueLoadingCache = CacheBuilder.newBuilder()
    .expireAfterWrite(120, TimeUnit.SECONDS)
    .removalListener(inQueueRemovalListener)
    .build(inQueueCacheLoader);

After every 120 seconds, the cache entries are evicted and it works as expected.

My question is: How do I change the timeout value, say from 120 to 60 seconds, for the current cache? What will happen to the cache entries during this change?

Answer: you can't change eviction timeout value, or any property of Cache / LoadingCache created by CacheBuilder.

Anyway, why would you want to change the timeout? (Also bare in mind that Guava Caches are quite simple.) If you really do want to change the timeout, you have two choices:

1. create new Cache with target semantics and copy old cache contents, ex.

LoadingCache<String, Long> newCache = CacheBuilder.newBuilder()
    .expireAfterWrite(60, TimeUnit.SECONDS)
    .removalListener(inQueueRemovalListener)
    .build(inQueueCacheLoader);
newCache.putAll(inQueueLoadingCache.asMap());

but you'll loose original access times etc./p>

2. don't use CacheBuilder at all and implement LoadingCache yourself, for example using AbstractLoadingCache skeletal implementation with your own policy for changing timeouts. It's not easy though, because you have nice LoadingCache's API but you have to implement whole thing by yourself (I tried it once but ended using more advanced cache than Guava's one).


How to get all values from guava LoadingCache without passing any keys

Question: I am using Guava LoadingCache to cache some of the results. Using load method I fetch results from other source and put into cache using 'put(key,value)'. Now the problem I am trying to solve is: I want to get all the available results in that cache with out passing any keys. Because I am interested in taking all the values presented in the cache at that time regardless of any specific keys.

getall(Iterable<?> keys) or getAllPresent(Iterable<?> keys) methods are there but those are expecting the keys to be passed.

Answer: You can use Cache#asMap view and operate on returned ConcurrentMap. There's nice description on Guava wiki page:

You can view any Cache as a ConcurrentMap using its asMap view, but how the asMap view interacts with the Cache requires some explanation.

cache.asMap() contains all entries that are currently loaded in the cache. So, for example, cache.asMap().keySet() contains all the currently loaded keys.