Reproducible results in Tensorflow with tf.set_random_seed

I am trying to generate N sets of independent random numbers. I have a simple code that shows the problem for 3 sets of 10 random numbers. I notice that even though I use the tf.set_random_seed to set the seed, the results of different runs do not look alike. Any help or comments are greatly appreciated.

(py3p6) bash-3.2$ cat test.py 
import tensorflow as tf
for i in range(3):
  tf.set_random_seed(1234)
  generate = tf.random_uniform((10,), 0, 10)
  with tf.Session() as sess:
    b = sess.run(generate)
    print(b)

This is the output of the code:

# output :
[9.604688  5.811516  6.4159    9.621765  0.5434954 4.1893444 5.8865128
 7.9785547 8.296125  8.388672 ]
[8.559105  3.2390785 6.447526  8.316823  1.6297233 1.4103293 2.647568
 2.954973  6.5975866 7.494894 ]
[2.0277488 6.6134906 0.7579422 4.6359386 6.97507   3.3192968 2.866236
 2.2205782 6.7940736 7.2391043]

I want something like

[9.604688  5.811516  6.4159    9.621765  0.5434954 4.1893444 5.8865128
 7.9785547 8.296125  8.388672 ]
[9.604688  5.811516  6.4159    9.621765  0.5434954 4.1893444 5.8865128
 7.9785547 8.296125  8.388672 ]
[9.604688  5.811516  6.4159    9.621765  0.5434954 4.1893444 5.8865128
 7.9785547 8.296125  8.388672 ]

Update 1: Indeed the reason I had put the seed initializer within the for loop, was because I want to set them differently (think of it as for different MCMC runs, for instance). This is my code which does the job but I am not sure if it's efficient. Basically I generate a couple random seeds between 0 and 2^32-1, and change the seed in each run. Any help or comments to make it more memory/RAM efficient are greatly appreciated.

import numpy as np
import tensorflow as tf
global_seed = 42
N_chains = 5
np.random.seed(global_seed)
seeds = np.random.randint(0, 4294967295, size=N_chains)

for i in range(N_chains):
    tf.set_random_seed(seeds[i])
    .... some stuff ....
    kernel_initializer = tf.random_normal_initializer(seed=seeds[i])
    .... some stuff
    with tf.Session() as sess:
         .... some stuff .....
 .
 .
 .

In tensorflow, a random operation relies on two different seeds: a global seed, set by tf.set_random_seed, and an operation seed, provided as an argument to the operation. You will find more details on how they relate in the docs.

You have a different seed for each random op because each random op maintains its own internal state for pseudo-random number generation. The reason for having each random generator maintaining its own state is to be robust to change: if they shared the same state, then adding a new random generator somewhere in your graph would change the values produced by all the other generators, defeating the purpose of using a seed.

Now, why do we have this dual system of global and per-op seeds? Well, actually the global seed is not necessary. It is there for convenience: It allows to set all random op seeds to a different and deterministic (if unknown) value at once, without having to go exhaustively through all of them.

Now when a global seed is set but not the op seed, according to the docs,

The system deterministically picks an operation seed in conjunction with the graph-level seed so that it gets a unique random sequence.

To be more precise, the seed that is provided is the id of the last operation that has been created in the current graph. Consequently, globally-seeded random operation are extremely sensitive to change in the graph, in particular to those created before itself.

For example,

import tensorflow as tf
tf.set_random_seed(1234)
generate = tf.random_uniform(())
with tf.Session() as sess:
  print(generate.eval())
  # 0.96046877

Now if we create a node before, the result changes:

import tensorflow as tf
tf.set_random_seed(1234)
tf.zeros(()) # new op added before 
generate = tf.random_uniform(())
with tf.Session() as sess:
  print(generate.eval())
  # 0.29252338

If a node is create after however, it does not affect the op seed:

import tensorflow as tf
tf.set_random_seed(1234)
generate = tf.random_uniform(())
tf.zeros(()) # new op added after
with tf.Session() as sess:
  print(generate.eval())
  # 0.96046877

Obviously, as in your case, if you generate several operations, they will have different seeds:

import tensorflow as tf
tf.set_random_seed(1234)
gen1 = tf.random_uniform(())
gen2 = tf.random_uniform(())
with tf.Session() as sess:
  print(gen1.eval())
  print(gen2.eval())
  # 0.96046877
  # 0.85591054

As a curiosity, and to validate the fact that seeds are simply the last used id in the graph, you could align the seed of gen2 to gen1 with

import tensorflow as tf
tf.set_random_seed(1234)
gen1 = tf.random_uniform(())
# 4 operations seems to be created after seed has been picked
seed = tf.get_default_graph()._last_id - 4
gen2 = tf.random_uniform((), seed=seed)
with tf.Session() as sess:
  print(gen1.eval())
  print(gen2.eval())
  # 0.96046877
  # 0.96046877

Obviously though, this should not pass code review.

tf.random.set_seed, the operation seed is set, we get different results for every call to the random op and every re-run of the program: print(tf.random.uniform([1])) # generates 'A1' tf.random.set_seed (1234) print (tf.random.uniform ()) # generates 'A1' print (tf.random.uniform ()) # generates 'A2' The reason we get 'A2' instead 'A1' on the second call of tf.random.uniform above is because the second call uses a different operation seed. Note that tf.function acts like a re-run of a program in this case.

Unable to get reproducible results using Keras with TF backend on , Unable to get reproducible results using Keras with TF backend on GPU #12800. Closed tf.set_random_seed(seed_value). sess = tf. jvishnuvardhan added type:support backend:tensorflow labels on May 8, 2019. A common mistake is not realizing that tf.set_random_seed() is only a graph-level seed and that running the script multiple times will alter the graph, explaining the non-repeatable results. I used the following statement to print out the entire graph and verified (via diff) that the graph is the same even when the results are different.

There is a related GitHub issue. But in your case, please refer to the documentation of tf.set_random_seed:

Sets the graph-level random seed.

You probably want to use the same graph and same operation to get the same random numbers in different sessions.

import tensorflow as tf

tf.set_random_seed(1234)
generate = tf.random_uniform((10,), 0, 10)
tf.get_default_graph().finalize() # something everybody tends to forget

for i in range(3):
    with tf.Session() as sess:
        b = sess.run(generate)
        print(b)

gives

[9.604688  5.811516  6.4159    9.621765  0.5434954 4.1893444 5.8865128
 7.9785547 8.296125  8.388672 ]
[9.604688  5.811516  6.4159    9.621765  0.5434954 4.1893444 5.8865128
 7.9785547 8.296125  8.388672 ]
[9.604688  5.811516  6.4159    9.621765  0.5434954 4.1893444 5.8865128
 7.9785547 8.296125  8.388672 ]

In your case, you created different operations within the same graph.

No reproducible using tensorflow backend � Issue #2280 � keras , import numpy as np np.random.seed(123) import tensorflow as tf tf. set_random_seed(123). The results now look sufficiently reproducible to me. how to get reproducible result in Tensorflow (4) Deterministic behaviour can be obtained either by supplying a graph-level or an operation-level seed. Both worked for me. A graph-level seed can be placed with tf.set_random_seed. An operation-level seed can be placed e.g, in a variable intializer as in:

Reproducible results using Keras, I am trying to obtain reproducible results from run to run and wasn't able to do it. import tensorflow as tf import random as rn import os os.environ[' PYTHONHASHSEED'] = '0' from keras import backend as K tf. set_random_seed(1) sess = tf. np.random.seed(42) # The below is necessary for starting core Python generated random numbers # in a well-defined state. rn.seed(12345) # Force TensorFlow to use single thread. # Multiple threads are a potential source of # non-reproducible results.

I noticed that you want to have 3 different vectors containing random numbers. Every time you want to run the code you want these three vectors containing random numbers to be the same as the first time. This approach is completely explainable, why need four the same random vectors. You want to have 4 random vectors each other.

There are two types of seeds that you can set when defining chart operations: Grain at the chart level, which is set by tf.set_random_seed, and seeds at the operational level, which are placed in the initializer variable As the grain is at the chart level, each time the result is different. You must use tf.InteractiveSession ()

tf.set_random_seed(1234)

sess = tf.InteractiveSession()
print(sess.run(tf.random_uniform((10,), 0, 10, seed=1)))
print(sess.run(tf.random_uniform((10,), 0, 10, seed=2)))
print(sess.run(tf.random_uniform((10,), 0, 10, seed=3)))
print(sess.run(tf.random_uniform((10,), 0, 10, seed=4)))

You get 4 random number vectors containing numbers from 0 to 10.

Reproducible results using Keras with TensorFlow backend, keras tensorflow reproducible results Set `tensorflow` pseudo-random generator at a fixed value import tensorflow as tf tf.set_random_seed(seed_value ) # 5. tf.set_random_seed(seed_value) sess = tf.Session(graph=tf.get_default_graph(), config=session_conf) K.set_session(sess) import torch torch.manual_seed(seed_value) What more I have to do to get the reproducible results? I'm running the code on Google Colab GPU. 👍

How to Get Reproducible Results with Keras, How to Get Reproducible Results from Neural Networks with Keras Photo by Samuel 1. 2. from tensorflow import set_random_seed. set_random_seed(2) Constants, Sequences, and Random Values � tf.set_random_seed. import numpy as np np.random.seed(123) import tensorflow as tf tf.set_random_seed(123) The results now look sufficiently reproducible to me. The small differences I assume are due to the use of cuDNN. I tried running without cuDNN: $ TF_USE_CUDNN=0 python mnist_cnn.py but it seems it's not possible:

Getting Reproducible Results in TensorFlow, Machine learning requires a good deal of random initialization of your weights, biases and other variables. As a result, when you develop� That means by setting the seed for numpy, the random seed for TensorFlow and additionally the seed for your initialization variables, you can make your TensorFlow training values reproducible.

Properly Setting the Random Seed in ML Experiments. Not as , It may be clear that reproducibility in machine learning is important, but how do we Data preparation — in the case of a neural network, the shuffled batches will import tensorflow as tf tf.set_random_seed(seed_value)# 5. Set `tensorflow` pseudo-random generator at a fixed value import tensorflow as tf tf.set_random_seed(seed_value) # 5. Configure a new global `tensorflow` session from keras import backend as K session_conf = tf.ConfigProto(intra_op_parallelism_threads=1, inter_op_parallelism_threads=1) sess = tf.Session(graph=tf.get_default_graph(), config

Comments
  • I could generate the same result by setting the seed in the operation as well: tf.set_random_seed(1234) and generate = tf.random_uniform((10,), 0, 10, seed=1234). I wonder if there is any other way without requiring to set the seed in the operation level.
  • Thanks so much for your thorough answer. I really appreciate it. However in my work, I need to run multiple chains with different randoms seeds (although in the question I mentioned the same seeds. That was just to make my question look simpler). I also want it to be reproducible, that is why I wanted to set the global seed too. Does it make it clear why I am trying to set the seeds within the for loop?
  • Thanks a lot for your answer! Yes, this works just perfectly. However indeed, what I want to do in the next step, is to set the seeds differently within the for loop, in order to have different running chains with different seeds. Does it make sense?
  • Thanks for your answer. I really appreciate it. The reason I am trying to set the random seeds within the for loop, is because I want to change the seed for each loop (Sorry! my ultimate goal is to do a task for N times while the seeds are set differently and I want everything to be reproducible). So far I could make it reproducible, by using the seting the global seed and the operation seed within the loop, but I am not sure if that is the efficient way.