Stopping a Reduce() operation mid way. Functional way of doing partial running sum
stop reduce js
end reduce early
return early from reduce
how to break a reduce
I have been doing some functional programming and had a question. Perhaps I might be missing something but is there any way to stop a "reduce()" function midway? Lets say when I reach a certain condition? The idea somehow seems anti functional. I haven't seen any such option in python or F#,
As an example, lets say I have a list such as [1,2,3,4,5]. I want to sum the elements in this list until the sum is not greater than some number (lets say 8), and return/mark/store/identify somehow, the number of elements I have actually added.
If we looked at python for example for I might try something like
reduce(lambda a,b : a if a + b > 8 else a + b, input)
This gives me the right answer 6, but how do I find that I had added 3 elements to get here. There is no counter as such. I can't do assignments inside lambdas. I think F# has the same situation.
I know I can use a for loop or use a function that can store state etc. But what would be the functional way of doing/thinking about this. Reduce() wants to run until the end, but somewhere along this line of processing, we either want to stop it (because we don't care about processing the rest of the elements) or at least make a note of the place where we stopped caring.
Reduce is often used in combination with map. Google for example has developed a map-reduce framework for querying their databases and this map-reduce pattern is now used in several other projects (e.g. CouchDB, Hadoop, etc).
First, you need to map the
[2, 1, 3, 4, 5] to something like:
[(1, 2), (1, 1), (1, 3), (1, 4), (1, 5)]
In that case,
x will represent the number of the elements to get the sum
x. Of course, the number of elements is
1 at the beginning for each single element.
The next thing then, is to operate on those tuples:
reduce( lambda a, b: a if a + b > 8 else (a + b, a + b), map(lambda x: (1, x), input))
This will return
(3, 6), meaning the partial sum is
I hope you got the idea behind map-reduce-algorithms.
I agree with JaredPar that writing your own recursive function that behaves similarly to
fold, but allows you to stop the computation earlier is the best approach. The way I would write it is a bit more general (so that you can use the function for any situation where you need folding that can stop earlier):
// Generalized 'fold' function that allws you to stop the execution earlier // The function 'f' has a type 'State -> 'T -> Option<'State> // By returning 'None' we can stop the execution (and return the // current state), by returning Some(newState), we continue folding let rec foldStop f state input = match input with | x::xs -> match f state x with | None -> state | Some(newState) -> foldStop f newState xs |  -> state // Example that stops folding after state is larger than 10 foldStop (fun st n -> if st > 10 then None else Some(st + n)) 0 [ 1 .. 10 ]
This is a very general function and you can use it for all similar scenarios. The nice thing about writing it is that you will never need to write similar explicit recursion again (because you can just use
foldStop once you have it).
Note that you can use
foldStop to implement
fold by always wrapping the result of the accumulation function in 'Some' (so it is more general):
let fold f state input = foldStop (fun st n -> Some(f st n)) state input
Bulletin of the Atomic Scientists, The scientific revolution of the 1950s and early 1960s has given way to a for curtailing the exploration of the moon simply to save the cost of operating rockets 18 Stopping a Reduce() operation mid way. Functional way of doing partial running sum Functional way of doing partial running sum 14 Connecting to LDAP from C# using DirectoryServices
Let's imagine Python had two functions, ireduce (similar to reduce but it would yield intermediate values; it's called scanl in some languages) and ilast (get last item of an iterable):
from itertools import takewhile from operator import add xs = [1, 2, 3, 4, 5] pair = ilast(enumerate(takewhile(lambda x: x < 8, ireduce(add, xs, 0)))) # (3, 6)
last $ zip [0..] (takeWhile (< 8) (scanl (+) 0 xs))
I think that the 'most functional' way to do this is probably via lazy evaluation. If you're in a lazy language like Haskell, or in an eager language but using a lazy list data structure (like
LazyList in the F# PowerPack), you can create e.g. a 'scan' of the running sums, and then leave it in the hands of the consumer of the list to decide how much she wants/needs to evaluate.
Or, you know, write a simple recursive function, like @JaredPar's answer. For some reason I often get a mental block on that, preventing me from noticing that "not everything has to be a
fold, you can in fact write your own recursive functions" :)
The Advocate, New Jersey governor James E. McGreevey signs a bill creating a domestic-partnership registry, making his state the fifth one to extend at least some Sum over partition and sum over order as running total : Sum « Analytical Functions « Oracle PL / SQL
Try the following
let sumUntil list stopAfter = let rec inner list sum = if sum >= stopAfter then sum else match list with |  -> sum | h::t-> inner t (sum + h) inner list 0
F# interactive result
> sumUntil [1;2;3;4;5] 8;; val it : int = 10
Interstate Commerce Commission Reports: Reports and Decisions of , Account) and others are virtually fixed in the short-run (e.g., overhead be developed for the various types of inputs for each functional category. Hence, the approach taken will be to develop equations to forecast total expenses both ways of operating expenses with transportation accounting for about one-half and the The WBS's from those past projects can be a great way to be sure that you are thinking of all of the work that you will need to do from the very beginning. Your project will never match the old WBS exactly, but there could be work packages listed there that you might not have thought of on your own but really are necessary in your project.
Congressional Record: Proceedings and Debates of the Congress, The cost of the Federal civilian payroll is now running at an annual average of as is now being exerted upon the Congress through the medium of Federal employees now that there will be a 10-percent reduction in the total of these two bills. is advocating socialized housing which, in partial form, is already under way. The SUMIFS function provides a powerful and relatively easy way to calculate a running total in a table of data where you want the total to reflect a dynamic date range, such as "all sales in the last 10 days" or "all expected expenditure in the next 30 days".
Water and Gas Review, a consumption sufficient to prevent all possible dangers to public health from a of that city, in which a reduction ofthe schedule rates for water used for domestic is recommended and their eventual total abolition favored, will be found in full its way into sewers, flushing them, thereby performing an important function Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.
Dearborn Independent, That sum is nearly three times the cost to the ultimate consumer of the same amount But the citizen is affected in two ways: first, by the extent to which he must buy are in reality making huge profits, the extent of which is only partially _hidden by most valuable function is to keep down prices and to stabilize the industry.
- What's important to you, the 3 or the 6? Or both? How would you want to use this function? Return a tuple -
(num_items, result)? It's a neat idea, but I think a loop is the most straightforward code.
- They are both important. I want to know I can take 3 elements and that the closest I can get to my limit is 6. Yes, a loop would be pretty straight forward, but I wanted to see how a functional programmer would attack it / think about it. I can't return a tuple, because reduce needs another int from the function to add to the next element in the list.
- Regarding Python, it could be possible to write a
filtered_reducefunction, but Python remains to be an imperative language whose functional-like features should not be overestimated. In Python the call to
reduceis translated into a simple loop anyway, so you gain nothing from it.
- Oooohhhh.... niiice. I had read about map reduce, but I guess I didn't fully grok it. Very nicely done.
- Here are two links which might interest you: Google's Map-Reduce paper (labs.google.com/papers/mapreduce.html) and a course Map Reduce in a Week (code.google.com/edu/submissions/mapreduce/listing.html).
- And a Python framework (based on Erlang) for doing efficient map-reduce computing is Disco. With that you can use multiple cores / computers and work with (nearly) unlimited data sets... discoproject.org
- I'm not downvoting, but this can hardly be idiomatic FP..? Chaitanya has picked his golden hammer, and you're helping him/her use it to bash a square peg into a round hole.
- Nice description of map/reduce, but if the input contained a million values and we hit the exit condition after three of them that's a lot of empty work being done. When you hit the exit condition use an exception to exit the loop.
- But I want to return the final state when I stopped as well as place where I stopped as well. My F# isn't fluent enough, but this would require changing the state and the input function as follows : foldStop (fun (st,i) n -> if st > 10 then None else Some(st + n, i + 1)) (0,0) [ 1 .. 10 ]
- @Chaitanya: Yes, that would require changing the code a little bit (or you would need to update the condition to stop on the next state). Alternatively, you could use
Option(that allows you to return the state, but still break the computation by returning a speical case).
- Hmmm ... Haskell one of those languages I keep wanting to learn but never get around to it