How to iterate a big list to make it smaller for a REST call using Java streams?

Related searches

I have logic something like this to process in chunks.

List<String> bigList = getList(); // Returns a big list of 1000 recs
int startIndex=0;
boolean entireListNotProcessed=false;
while(entireListNotProcessed) {
   int endIndex = startIndex + 100;
   if(endIndex > myList.size()-1) {
      endIndex=myList.size()-1;
   }
   List<String> subList= bigList.subList(startIndex, endIndex);
   //call Rest API with subList and update entireListNotProcessed flag...
}

Is there a better way of doing this iteration using java streams?


You can do similar to this answer, by creating range with stepping and then subList:

int step = 100;
IntStream
  .iterate(0, o -> o < bigList.size(), o -> o + step)
  .mapToObj(i -> bigList.subList(i, Math.min(i + step, bigList.size()))
  .forEach(subList -> callRestApi(subList));

Or you can extract method:

private static <T>  Stream<List<T>> partition(List<T> list, int step) {
    return IntStream
        .iterate(0, o -> o < list.size(), o -> o + step)
        .mapToObj(i -> list
            .subList(i, Math.min(i + step, list.size()))
        );
}

and then

partition(bigList, 100).forEach(subList -> callRestApi(subList));

Partition a List in Java, How to Partition a List using Guava or Apache Commons REST. The guides on building REST APIs with Spring Learn how to shuffle various collections in Java. List<Integer> lastPartition = subSets.get(2); <Integer> newArrayList(7, 8); List<Integer> firstPartition = subSets.iterator().next();. I'm relatively new to REST calls and am probably a beginner to intermediate in C#. I am trying to make a REST call to Qualys to obtain appliance information. My problem is that when I do a foreach loop to iterate through the List, the console app fails by just exiting after I input my credentials.


You could use ListUtils from apache commons to do the splitting for you (if you really need to since 1000 is not that muuch)

List<List<Integer>> output = ListUtils.partition(largeList, targetSize);

Filtering a Java Collection by a List, Learn different ways to filter a Collection in Java based on the values of Filtering a Collection by a List is a common business logic scenario. We'll begin with the most classic syntax, a for-each loop. If we call the size of the employee list n, then nameFilter will be on Using Streams and List#contains. 5 Checking list of enums with Java 8 Stream API Feb 11 4 Java Stream: Filter with multiple ranges Jan 29 4 How to iterate a big list to make it smaller for a REST call using Java streams?


You can use Guava's Iterables.partition or Lists.partition

Lists.partition(bigList, 100).forEach(subList -> callRestApi(subList));

Faster parallel processing in Java using Streams and a fixed-batch , I can't remember a single for-loop that I had to write with Java 8 (well, except next big thing (seamless multicore computing), concise lambda syntax with A bit of explanation is present in this message by Paul Sandoz on the lambda-dev list. framework decides to consume a spliterator in full, it can make a single call to� PayPal Java SDK Complete Example – How to Invoke PayPal Authorization REST API using Java Client? How to Iterate Through Map and List in Java? Example attached (Total 5 Different Ways) How to Iterate through LinkedList Instance in Java? In Java8 – How to Convert Array to Stream using Arrays.stream() and Stream.of() Operations


You can batch your processing into chucks of a fixed size (or more advance dynamically computed batch size).

 final List<Integer> numbers = Arrays.asList(1,2,3,4,5,6,7,); // any range of items.
    final int batchSize = 3;
    final AtomicInteger counter = new AtomicInteger();

    final Collection<List<Integer>> result = numbers.stream()
        .collect(Collectors.groupingBy(it -> counter.getAndIncrement() / batchSize))
        .values();

You have now a collection of List of N element each (N=3)

Java 8 streams API and parallelism, Prior to the feature updates made available in Java 8, running bulky processing due to the inherent explicit iteration that was being used. of fetching out the biggest trade from a set collection using Java 7: Now that we have all the pieces in play, a client can invoke the filterTrades method as follows:. Few Java 8 examples to execute streams in parallel. 1. BaseStream.parallel() A simple parallel example to print 1 to 10.


10 Examples of forEach() method in Java 8, From Java 8 onward, you can iterate over a List or any Collection without using any If you not started with Java 8 yet then you should make it one of your new year once calling forEach() method on stream, you cannot call another method. In Python, list is a type of container in Data Structures, which is used to store multiple data at the same time. Unlike Sets, the list in Python are ordered and have a definite count. There are multiple ways to iterate over a list in Python. Let’s see all different ways to iterate over a list in Python, and a performance comparison between them.


We created an infinite stream using an iterate() method. Then we called a limit() transformation and a collect() terminal operation. Then in our resulting List, we will have first 10 elements of an infinite sequence due to a laziness of a Stream. 4. Infinite Stream of a Custom Type of Elements


Java Streams Improvements In Java 9. Java 8 brought Java streams to the world. However, the following version of the language also contributed to the feature. So, we’ll now give a brief overview of the improvements that Java 9 brought to the Streams API. Let’s do it. takeWhile. The takeWhile method is one of the new additions to the Streams