Swift: Lazily encapsulating chains of map, filter, flatMap

swift map
swift flatmap
swift compactmap
swift map array of objects
swift map dictionary
swift map array to dictionary
swift map array to set
swift map index

I have a list of animals:

let animals = ["bear", "dog", "cat"]

And some ways to transform that list:

typealias Transform = (String) -> [String]

let containsA: Transform = { $0.contains("a") ? [$0] : [] }
let plural:    Transform = { [$0 + "s"] }
let double:    Transform = { [$0, $0] }

As a slight aside, these are analogous to filter (outputs 0 or 1 element), map (exactly 1 element) and flatmap (more than 1 element) respectively but defined in a uniform way so that they can be handled consistently.

I want to create a lazy iterator which applies an array of these transforms to the list of animals:

extension Array where Element == String {
  func transform(_ transforms: [Transform]) -> AnySequence<String> {

    return AnySequence<String> { () -> AnyIterator<String> in
      var iterator = self
        .lazy
        .flatMap(transforms[0])
        .flatMap(transforms[1])
        .flatMap(transforms[2])
        .makeIterator()

      return AnyIterator {
        return iterator.next()
      }
    }
  }
}

which means I can lazily do:

let transformed = animals.transform([containsA, plural, double])

and to check the result:

print(Array(transformed))

I'm pleased with how succinct this is but clearly:

        .flatMap(transforms[0])
        .flatMap(transforms[1])
        .flatMap(transforms[2])

is an issue as it means the transform function will only work with an array of 3 transforms.

Edit: I tried:

  var lazyCollection = self.lazy
  for transform in transforms {
    lazyCollection = lazyCollection.flatMap(transform) //Error
  }
  var iterator = lazyCollection.makeIterator()

but on the marked row I get error:

Cannot assign value of type 'LazyCollection< FlattenCollection< LazyMapCollection< Array< String>, [String]>>>' to type 'LazyCollection< Array< String>>'

which I understand because each time around the loop another flatmap is being added, so the type is changing.

How can I make the transform function work with an array of any number of transforms?

One WET solution for a limited number of transforms would be (but YUK!)

  switch transforms.count {
  case 1:
    var iterator = self
      .lazy
      .flatMap(transforms[0])
      .makeIterator()
    return AnyIterator {
      return iterator.next()
    }
  case 2:
    var iterator = self
      .lazy
      .flatMap(transforms[0])
      .flatMap(transforms[1])
      .makeIterator()
    return AnyIterator {
      return iterator.next()
    }
  case 3:
    var iterator = self
      .lazy
      .flatMap(transforms[0])
      .flatMap(transforms[1])
      .flatMap(transforms[2])
      .makeIterator()
    return AnyIterator {
      return iterator.next()
    }
  default:
    fatalError(" Too many transforms!")
  }

Whole code:

let animals = ["bear", "dog", "cat"]

typealias Transform = (String) -> [String]

let containsA: Transform = { $0.contains("a") ? [$0] : [] }
let plural:    Transform = { [$0 + "s"] }
let double:    Transform = { [$0, $0] }

extension Array where Element == String {
  func transform(_ transforms: [Transform]) -> AnySequence<String> {

    return AnySequence<String> { () -> AnyIterator<String> in
      var iterator = self
        .lazy
        .flatMap(transforms[0])
        .flatMap(transforms[1])
        .flatMap(transforms[2])
        .makeIterator()

      return AnyIterator {
        return iterator.next()
      }
    }
  }
}

let transformed = animals.transform([containsA, plural, double])

print(Array(transformed))

Another approach to achieve what you want:

Edit: I tried:

var lazyCollection = self.lazy
for transform in transforms {
    lazyCollection = lazyCollection.flatMap(transform) //Error
}
var iterator = lazyCollection.makeIterator()

You were very near to your goal, if the both types in the Error line was assignable, your code would have worked.

A little modification:

var lazySequence = AnySequence(self.lazy)
for transform in transforms {
    lazySequence = AnySequence(lazySequence.flatMap(transform))
}
var iterator = lazySequence.makeIterator()

Or you can use reduce here:

var transformedSequence = transforms.reduce(AnySequence(self.lazy)) {sequence, transform in
    AnySequence(sequence.flatMap(transform))
}
var iterator = transformedSequence.makeIterator()

Whole code would be:

(EDIT Modified to include the suggestions from Martin R.)

let animals = ["bear", "dog", "cat"]

typealias Transform<Element> = (Element) -> [Element]

let containsA: Transform<String> = { $0.contains("a") ? [$0] : [] }
let plural:    Transform<String> = { [$0 + "s"] }
let double:    Transform<String> = { [$0, $0] }

extension Sequence {
    func transform(_ transforms: [Transform<Element>]) -> AnySequence<Element> {
        return transforms.reduce(AnySequence(self)) {sequence, transform in
            AnySequence(sequence.lazy.flatMap(transform))
        }
    }
}

let transformed = animals.transform([containsA, plural, double])

print(Array(transformed))

Swift, In swift, Sequence is the heart of the collection. Collections are responsible for holding the elements sequentially in most of the cases. Swift has  Use this method to receive a single-level collection when your transformation produces a sequence or collection for each element. In this example, note the difference in the result of using map and flat Map with a transformation that returns an array.

You can apply the transformations recursively if you define the method on the Sequence protocol (instead of Array). Also the constraint where Element == String is not needed if the transformations parameter is defined as an array of (Element) -> [Element].

extension Sequence {
    func transform(_ transforms: [(Element) -> [Element]]) -> AnySequence<Element> {
        if transforms.isEmpty {
            return AnySequence(self)
        } else {
            return lazy.flatMap(transforms[0]).transform(Array(transforms[1...]))
        }
    }
}

Experimenting with Swift 3 Sequences and Iterators, In this article, part of a series on Swift and the functional approach, we'll explore the sequence's iteration interface and /// encapsulates its iteration state. article on map and flatMap), filter, reduce, subsequence functions and others. Making your infinite sequences lazy is a required step, since Swift  Swift: Lazily encapsulating chains of map, filter, flatMap; Different behavior between lambda expression and method reference by instantiation; C# 7.3 Enum constraint: Why can't I use the enum keyword? Is binary equality comparison of floats correct? Should I throw exceptions in an if-else block?

How about fully taking this into the functional world? For example using (dynamic) chains of function calls, like filter(containsA) | map(plural) | flatMap(double).

With a little bit of reusable generic code we can achieve some nice stuff.

Let's start with promoting some sequence and lazy sequence operations to free functions:

func lazy<S: Sequence>(_ arr: S) -> LazySequence<S> {
    return arr.lazy
}

func filter<S: Sequence>(_ isIncluded: @escaping (S.Element) throws -> Bool) -> (S) throws -> [S.Element] {
    return { try $0.filter(isIncluded) }
}

func filter<L: LazySequenceProtocol>(_ isIncluded: @escaping (L.Elements.Element) -> Bool) -> (L) -> LazyFilterSequence<L.Elements> {
    return { $0.filter(isIncluded) }
}

func map<S: Sequence, T>(_ transform: @escaping (S.Element) throws -> T) -> (S) throws -> [T] {
    return { try $0.map(transform) }
}

func map<L: LazySequenceProtocol, T>(_ transform: @escaping (L.Elements.Element) -> T) -> (L) -> LazyMapSequence<L.Elements, T> {
    return { $0.map(transform) }
}

func flatMap<S: Sequence, T: Sequence>(_ transform: @escaping (S.Element) throws -> T) -> (S) throws -> [T.Element] {
    return { try $0.flatMap(transform) }
}

func flatMap<L: LazySequenceProtocol, S: Sequence>(_ transform: @escaping (L.Elements.Element) -> S) -> (L) -> LazySequence<FlattenSequence<LazyMapSequence<L.Elements, S>>> {
    return { $0.flatMap(transform) }
}

Note that the lazy sequences counterparts are more verbose that the regular Sequence ones, but this is due to the verbosity of LazySequenceProtocol methods.

With the above we can create generic functions that receive arrays and return arrays, and this type of functions are extremely fitted for pipelining, so let's define a pipeline operator:

func |<T, U>(_ arg: T, _ f: (T) -> U) -> U {
    return f(arg)
}

Now all we need is to feed something to these functions, but to achieve this we'll need a little bit of tweaking over the Transform type:

typealias Transform<T, U> = (T) -> U

let containsA: Transform<String, Bool> = { $0.contains("a") }
let plural:    Transform<String, String> = { $0 + "s" }
let double:    Transform<String, [String]> = { [$0, $0] }

With all the above in place, things get easy and clear:

let animals = ["bear", "dog", "cat"]
let newAnimals = lazy(animals) | filter(containsA) | map(plural) | flatMap(double)
print(Array(newAnimals)) // ["bears", "bears", "cats", "cats"]

RxSwift: Transforming Operators in Practice, In the process, you will learn more about map and flatMap , and in which With the filter operator above, you easily discard all error response codes. It's one of the lesser-known Swift operators, and when used with a range on It's really cool how RxSwift forces you to encapsulate these discrete pieces  Swift: Lazily encapsulating chains of map, filter, flatMap How do I merge two maps in STL and apply a function for conflicts? Update rows depending on the value in following rows in multiple columns

ReversedCollection, Swift documentation for 'ReversedCollection': A collection that presents the c.​lazy.reversed().map(f) maps lazily and returns a LazyMapCollection A type that provides the sequence's iteration interface and encapsulates its iteration state. func filter(_:) func flatMap<ElementOfResult>(_: (ReversedCollection<Base>. It does sound like a lot of work, and you might be planning on using a filter, map, another filter, or more. In this section, you will use a single flatMap to easily filter the sequence. You can use flatMap to filter responses that don’t feature a Last-Modified header. Append this to the operator chain from above:

LazyRandomAccessCollection, Swift documentation for 'LazyRandomAccessCollection': A collection but on which some operations such as map and filter are implemented lazily. A type that provides the collection's iteration interface and encapsulates its iteration state. func flatMap<ElementOfResult>(_: (LazyRandomAccessCollection<Base​>. Combinators are methods used to construct a serialized chain of FutureStreams that perform asynchronous requests and apply mappings and filters. map. Apply a mapping: (T) throws -> M to the result of a successful FutureStream<T> to produce a new FutureStream<M> of a different type.

Lightning Read #1: Lazy Collections in Swift, Problem: Intermediate clutter that comes with map, flatMap and filter functions. Let's say we have the following requirements in an imaginary  Retrieving.lazy from the original sequence, we’ll get a new LazySequenceType on which operations such as map, flatMap, reduce or filter will be executed lazily and the real evaluation will be performed on demand only when a terminal operation (other languages call them this way) down the chain such as next or something that needs the whole sequence content will be performed.

Comments
  • can't you send the number of iterations ?
  • transforms.count ?
  • As you say I know the number of transforms as transforms.count but I can't see how to use that to make a loop of ".flatMap" operations because the type changes at each iteration.
  • i can see, the problem is this type as far as i know "iterator" can't take bounds to as it needs to implement directly the way you did, but some workaround can be applied by sending a flag to use another iterator or something.
  • I'm wondering about changing the title of the question to: "Swift: Lazily encapsulating chains of map, filter, flatMap" as I think that would describe the essence of what this question is about much better but wondering if it's considered bad form to change the title of a question?
  • Very nice! – Minor remarks: The initial value in reduce can be AnySequence(self), without the lazy. On the other hand, AnySequence(sequence.lazy.flatMap(transform)) would avoid the creation of intermediate arrays.
  • Nice suggestion, frankly I had been forgotten that flatMap on Sequence generates an intermediate array. Thanks so much.
  • Thanks so much Martin and OOPer. Your combined thought have given me an elegant solution. Much appreciated.
  • Wow! Very nice @Cristik. When I first started thinking about this I did wonder if there might be a way to create an operator to combine filter, map, flatMap but have only been exploring functional thinking for a couple of months so my brain couldn't fathom it - so great to see your methods. One thing that isn't clear to me... how would I access the result lazily? let operation = filter(containsA) | map(plural) | flatMap(double) for x in operation(animals) { print(x) } is not lazy?
  • @Adahus good question, the laziness in this approach doesn't seem to function that well so I updated the answer and removed the lazy calls since they seem to help with almost nothing. I'll try and get back with a really "lazy" solution :)
  • I'm wondering if the free functions need to be passing AnySequence?
  • @Adahus actually, I was able to solve the problem via protocols only, I updated the answer and now if you provide a lazy sequence as a starting point then the whole pipeline will be lazy.
  • Wow again! This is awesome. Thanks for putting in the time to write this. Even though this isn't many lines of code it represents quite a significant set of methods. My next challenge is to ponder how to allow a lazy cartesian product to be added to the pipeline...