Why element order is not preserved when converting a list of pairs to Map in Scala?

convert dataframe to key-value pair spark scala
spark rdd get value by key
scala map
scala convert map to listmap
spark pair rdd
list.map scala
reduce by key spark
spark groupbykey

Example:

val occ = List(('l', 2), ('n', 1), ('r', 1), ('u', 2), ('x', 1))
occ.toMap
// Map(x -> 1, n -> 1, u -> 2, l -> 2, r -> 1)

The elements are no longer sorted in alphabetical order. Why does this happen?

Maps are not sorted and the keys may be in any order depending on implementation.

However a ListMap preserves the order of addition. You can build a ListMap using the normal Map operations, or you can create one from a List like this:

ListMap(occ:_*)

Immutable maps have inconsistent behaviour on update · Issue , Why element order is not preserved when converting a list of pairs to Map in Scala? - scala. A Map is an Iterable consisting of pairs of keys and values (also named mappings or associations).Scala’s Predef object offers an implicit conversion that lets you write key -> value as an alternate syntax for the pair (key, value).

Edit: there's a LinkedMap coming in 2.13 with different trade-offs.

Edit: the question is really that you might expect list.toMap to produce a ListMap. Why doesn't it?

The doc says "inserting or removing entries, are also O(n), which makes this collection suitable only for a small number of elements."

So that's another sensitivity to size. Often you create a collection as pairs resulting from other operations and you just want to dump them into a map, often a large one.

--

There is sensitivity to size due to specialization. There are custom maps for the smallest sizes. This isn't guaranteed, but is confusing if you notice the pattern.

scala $ ~/scala-2.12.6/bin/scala -Dscala.repl.info
Welcome to Scala 2.12.6 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_144).
Type in expressions for evaluation. Or try :help.
[info] started at Wed Jun 20 00:17:33 PDT 2018

scala 2.12.6> val m = List(1->10,2->20,3->30,4->40).toMap
m: scala.collection.immutable.Map[Int,Int] = Map(1 -> 10, 2 -> 20, 3 -> 30, 4 -> 40)

scala 2.12.6> val m = List(1->10,2->20,3->30,4->40,5->50).toMap
m: scala.collection.immutable.Map[Int,Int] = Map(5 -> 50, 1 -> 10, 2 -> 20, 3 -> 30, 4 -> 40)

scala 2.12.6> :quit
scala $ ~/scala-2.13.0-M4/bin/scala -Dscala.repl.info
[info] started at Wed Jun 20 00:18:41 PDT 2018
Welcome to Scala 2.13.0-M4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_144).
Type in expressions for evaluation. Or try :help.

scala 2.13.0-M4> val m = List(1->10,2->20,3->30,4->40).toMap
m: scala.collection.immutable.Map[Int,Int] = ChampHashMap(1 -> 10, 2 -> 20, 3 -> 30, 4 -> 40)

scala 2.13.0-M4> val m = List(1->10,2->20,3->30,4->40,5->50).toMap
m: scala.collection.immutable.Map[Int,Int] = ChampHashMap(5 -> 50, 1 -> 10, 2 -> 20, 3 -> 30, 4 -> 40)

Learning Scala Programming: Object-oriented programming meets , I would be happy to work on this change after confirmation of the semantics required List map iterators and; traversal methods visit key-value pairs in the order whey were first inserted. I personally think that the scaladoc should not include the ordering Existing key elements must not be preserved. Program to convert Java list to Set in Scala Scala | reduce() Function The reduce() method is a higher-order function that takes all the elements in a collection (Array, List, etc) and combines them using a binary operation to produce a single value.

You can use ListMap instead of Map, it keeps items in the order they were inserted. To convert List to ListMap:

val occ = List(('l', 2), ('n', 1), ('r', 1), ('u', 2), ('x', 1))
ListMap(occ: _*)

In case you don't know what _* means - repeated parameters

scala.collection.immutable.ListMap, Almost every collection has higher order functions to operate with. In the majority of situations, we come up with Map, Set, List, ArrayBuffer, or a Vector. is of no importance, and ListMap when we want to store the key-value pairs in a sequence. elements, so to remove duplicates we may choose to use a Set, or convert  * This function takes the first two elements of the list `trees` and combines * them into a single `Fork` node. This node is then added back into the * remaining elements of `trees` at a position such that the ordering by weights * is preserved. * * If `trees` is a list of less than two elements, that list should be returned * unchanged. */

4. Working with Key/Value Pairs - Learning Spark [Book], [use case] Returns a new immutable list map containing the elements from A syntactic sugar for out of order folding. The order of the elements is preserved. Returns a new map with all key/value pairs for which the predicate p returns true . and must not change the result (e.g., Nil for list concatenation, 0 for addition,  Scala Lists are quite similar to arrays which means, all the elements of a list have the same type but there are two important differences. First, lists are immutable, which means elements of a list cannot be changed by assignment. Second, lists represent a linked list whereas arrays are flat. The type of a list that has elements of type T is

How to choose a collection class in Scala, We can do this by running a map() function that returns key/value pairs. An implicit conversion on RDDs of tuples exists to provide the additional key/value functions. When creating a pair RDD from an in-memory collection in Scala and Python, as the aggregation function (appending to a list) does not save any space. a pair of lists: the list of all elements which satisfy p and the list of all elements which do not. The relative order of the elements in the sub-lists is the same as in the original list. The relative order of the elements in the sub-lists is the same as in the original list.

Scala List class examples: range, fill, tabulate, appending, foreach , A map contains a collection of key/value pairs, like a Java Map , Ruby Hash , or Python dictionary. A set is a collection that contains no duplicate elements. The documentation states, “If you plan to convert the buffer to a list, use Map classes, a SortedMap trait to keep elements in sorted order by key,  Iterating over a Scala Map - Summary. In summary, I hope these examples of iterating a Scala Map have been helpful. As you can see, there are many ways to loop over a Map, using for, foreach, tuples, and key/value approaches.

Comments
  • @CannedMoose if we use sorted map then only it will sort the map by key it's not the case always.
  • Possible duplicate of Scala Map implementation keeping entries in insertion order?
  • The question asked about a 'toMap' method in List, I see it is different with ListMap. And I do not find an answer in the question recommended.
  • I know there are ways to preserve the order. I am just curious why converting to map do not preserve the order. What is reason/concern in implementation the "toMap" method.
  • @EmmaLiu toMap just converts the List to a Map. It is the implementation of Map that matters. This is typically done with a table indexed by a hash of the key value (hence HashMap). The order of the keys will be the order of the hashed values, not the order of the keys themselves.
  • @EmmaLiu: A map has no order. It doesn't preserve the order because there is no order to preserve.
  • @Tim Your explanation answers my question. Thanks a lot.
  • Yes, I do noticed this. For very small maps, the conversion keeps the element in order. But for slightly larger map, the order is not longer preserved, but determined.