How can I print the values of Keras tensors?
I am implementing own Keras loss function. How can I access tensor values?
What I've tried
def loss_fn(y_true, y_pred): print y_true
Tensor("target:0", shape=(?, ?), dtype=float32)
Is there any Keras function to access
Keras' backend has
print_tensor which enables you to do this. You can use it this way:
import keras.backend as K def loss_fn(y_true, y_pred): y_true = K.print_tensor(y_true, message='y_true = ') y_pred = K.print_tensor(y_pred, message='y_pred = ') ...
The function returns an identical tensor. When that tensor is evaluated, it will print its content, preceded by
From the Keras docs:
Note that print_tensor returns a new tensor identical to x which should be used in the following code. Otherwise the print operation is not taken into account during evaluation.
So, make sure to use the tensor afterwards.
TensorFlow's Print is not printing, Tensor. A tf.Tensor object represents a partially defined computation that import tensorflow as tf from keras import backend as K x= K.abs(-2.0) with tf. use the print or tensorflow.print() to print out the value of your object. Keras expects tensors that are passed to other layers to have specific attributes such as _keras_shape. Values processed by the backend, i.e. through Print , usually do not have that attribute. To solve this, you can wrap debug statements in a Lambda layer for example.
y_true you know in advance - during preparation of your train corpora...
However, there's one trick to see the values inside
y_pred. Keras gives you an opportunity to write respective callback for printing the neural network's output.
It will look something like this:
def loss_fn(y_true, y_pred): return y_true # or y_pred ... import keras.callbacks as cbks class CustomMetrics(cbks.Callback): def on_epoch_end(self, epoch, logs=None): for k in logs: if k.endswith('loss_fn'): print logs[k]
loss_fn is name of your loss function when you pass it into the
model.compile(...,metrics=[loss_fn],) function during model's compilation.
So, finally, you have to pass this
CustomMetrics callback as the argument into the
model.fit(x=train_X, y=train_Y, ... , callbacks=[CustomMetrics()])
P.S.: If you use Theano (or TensorFlow) like here in Keras, you write a python program, and then you compile it and execute. So, in your example
y_true - is just a tensor variable which is used for further compilation and loss function counting.
It means that there's no way to see the values inside it. In Theano, for example, you can look inside the only so-called shared variable after the execution of respective
eval() function. See this question for more info.
tf.keras.backend.print_tensor, Prints message and the tensor value when evaluated. View aliases. Pre-trained models and datasets built by Google and the community
If you are using TensorFlow's keras, you can enable Eager Execution:
import tensorflow as tf tf.enable_eager_execution()
Afterwards you can print the tensors in your loss function.
In case you get the error message "ValueError: Only TF native optimizers are supported in Eager mode." and you have used 'adam' as an optimizer for example, you can change the model's compile arguments to
model.compile(optimizer = tf.train.AdamOptimizer(), loss = loss_fn, ...)
How to print the value of a Tensor object in TensorFlow?, () operator, as Andrzej suggests in another answer. According to the official documentation: To make sure the operator runs, users need to pass the produced op to tf. The point of using tf.Print is to add another print node so that when actual values flow through the tensors they will be printed to console at every epoch. Thank you for your answer though. – Leo Appleseed Aug 26 '18 at 8:50
You could redefine your loss function to return the value instead:
def loss_fn(y_true, y_pred): return y_true
Let's create some tensors:
from keras import backend as K a = K.constant([1,2,3]) b = K.constant([4,5,6])
And use the
keras.backend.eval() API to evaluate your loss function:
loss = loss_fn(a,b) K.eval(loss) # array([1., 2., 3.], dtype=float32)
Understanding Keras tensors - Emmanuel Caradec, How do I create a custom loss function in keras? The way this manifests in code is to pass the Print call, as its first parameter, the node that is its ‘input’, and then assign the return value of tf.Print to a variable that will serve as an input in a later node in the graph, thus linking the Print statement serially into the flow of operations.
You can't get the values from the tensor symbolic variable directly. Yo need to write a theano function to extract the value. Don't forget to choose theano as backend of Keras.
Check the notebook link to get some basic of theano variables and functions : get tensor value in call function of own layers
How to create a custom loss function in Keras, Returns the value of a variable. Keras custom callbacks K = tf.keras.backend # Common keras convention v = K.variable(1.) print(K.get_value(v)) 2.0. @xyzhang16: You can't get the values from the tensor symbolic variable directly. Yo need to write a theano function to extract the value. Yo need to write a theano function to extract the value. Check the notebook link below to get some basic of theano variables and functions :
Introduction to Tensor with Tensorflow, I heard you wanted to print some tensors. the Print call, as its first parameter, the node that is its 'input', and then assign the return value of tf. Basically, I'm saying that my mental model of tensors and how computations flow through keras/theano is still pretty fuzzy and I need all the help I can get :) If I print a TensorVariable.shape attribute I get the string "Shape.0", this seems to be an lvector of subtensors but I can't seem to access the values.
tf.keras.backend.get_value, Dense(10) >>> print(new_layer.name) dense_10 tf.keras.backend.set_floatx(value) Returns the value of the fuzz factor used in numeric expressions. A "Keras tensor" is a tensor that was returned by a Keras layer, ( Layer class) or by *inputs: Positional arguments that are the inputs to print. Inputs in the printed output will be separated by spaces. Inputs may be python primitives, tensors, data structures such as dicts and lists that may contain tensors (with the data structures possibly nested in arbitrary ways), and printable python objects.
Using tf.Print() in TensorFlow, I want to print intermediate tensor in model constructed by tensorflow value is printed once(just like bellow code tf.print("outputs",outputs) ) In keras: R Interface to 'Keras' Description Usage Arguments Value Keras Backend. View source: R/backend.R. Description. Note that print_tensor returns a new tensor identical to x which should be used in the following code. Otherwise the print operation is not taken into account during evaluation. Usage