What does the `order` argument mean in `tf.keras.utils.normalize()`?

Consider the following code:

import numpy as np

A = np.array([[.8, .6], [.1, 0]])
B1 = tf.keras.utils.normalize(A, axis=0, order=1)
B2 = tf.keras.utils.normalize(A, axis=0, order=2)

print('A:')
print(A)
print('B1:')
print(B1)
print('B2:')
print(B2)

which returns

A:
[[0.8 0.6]
 [0.1 0. ]]
B1:
[[0.88888889 1.        ]
 [0.11111111 0.        ]]
B2:
[[0.99227788 1.        ]
 [0.12403473 0.        ]]

I understand how B1 is computed via order=1 such that each entry in A is divided by the sum of the elements in its column. For example, 0.8 becomes 0.8/(0.8+0.1) = 0.888. However, I just can't figure out how order=2 produces B2 nor can I find any documentation about it.

However, I just can't figure out how order=2 produces B2 nor can I find any documentation about it.

order=1 means L1 norm while order=2 means L2 norm. For L2 norm, You need to take the square root after summing the individual squares. Which elements to square depends on the axis.

Keras

A = np.array([[.8, .6], [.1, 0]])
B2 = tf.keras.utils.normalize(A, axis=0, order=2)
print(B2)

array([[0.99227788, 1.        ],
       [0.12403473, 0.        ]])

Manual

B2_manual = np.zeros((2,2))
B2_manual[0][0] = 0.8/np.sqrt(0.8 ** 2 + 0.1 ** 2)
B2_manual[1][0] = 0.1/np.sqrt(0.8 ** 2 + 0.1 ** 2)
B2_manual[0][1] = 0.6/np.sqrt(0.6 ** 2 + 0 ** 2)
B2_manual[1][1] =  0 /np.sqrt(0.6 ** 2 + 0 ** 2)
print(B2_manual)

array([[0.99227788, 1.        ],
       [0.12403473, 0.        ]])

You can look up the different types of Norm here: https://en.wikipedia.org/wiki/Norm_(mathematics) Worked examples: https://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.norm.html

What does the `order` argument mean in `tf.keras.utils.normalize()`?, Consider the following code: import numpy as np A = np.array([[.8, .6], [.1, 0]]) B1 = tf.keras.utils.normalize(A, axis=0, order=1) B2 = tf.keras.utils.normalize(A,  Normalizes a Numpy array. View aliases. Compat aliases for migration. See Migration guide for more details. tf.compat.v1.keras.utils.normalize. tf.keras.utils

Order 1 normalize the input such that the sum of the absolute value of all element is 1 (L1 norm of the input equals 1). Order 2 normalize the input such that the sum of the squared value of all element is 1 (L2 norm of the input equals 1).

tf.keras.utils.normalize, false_positives · false_positives_at_thresholds · mean · mean_absolute_error tf.keras.utils.normalize. Contents; Arguments; Returns x, axis=-1, order=2 axis, axis along which to normalize. order, Normalization order (e.g. 2 for L2 norm​). Except as otherwise noted, the content of this page is licensed under the  Pre-trained models and datasets built by Google and the community

Passing order 2 in the order parameter, means you will be applying Tikhonov regularization commonly known as L2 or Ridge. L1 and L2 are different regularization techniques, both with pros and cons you can read in detail here in wikipedia and here in kaggle. The approach for L2 is to solve the standard equation for regresison, when calculating residual sum of squares adding an extra term λβTβ which is the square of the transposed matrix Beta (that's why it is called L2, because of the sqaure).

What does the `order` argument mean in `tf. keras. utils. normalize , What does the `order` argument mean in `tf. keras. utils. normalize()`? Consider the following code: import numpy as np A = np.array([[.8, .6], [.1, 0]]) B1  While Batch Normalization is also available in the tf.nn module, it requires extra bookkeeping, as the mean and variance are required arguments for the function. Thus the user has to manually compute mean and variance at both the batch level and training set level.

tf.keras.utils.normalize, Arguments: x : Numpy array to normalize. axis : axis along which to normalize. order : Normalization order (e.g. 2 for L2 norm). If the user passes in dictionaries for the inputs or outputs argument during tf.keras.Model initialization, preserve the dictionary datatype in tf.keras.Model.inputs and tf.keras.Model.outputs (i.e. avoid normalizing these arguments to lists). This would allow us to refer to inputs and outputs by name rather than by indices in a list.

How to normalize features in TensorFlow, TL;DR When using tf.estimator, use the normalizer_fn argument in tf.​feature_column.numeric_feature to normalize using the same parameters (mean, std, etc.) Batch normalization is the most comprehensive approach for Normalize using input_fn · Batch Normalization using tf.keras; Standard score  Teams. Q&A for Work. Stack Overflow for Teams is a private, secure spot for you and your coworkers to find and share information.

tf.keras.utils.normalize, tf.keras.utils.normalize. tf.keras.utils.normalize( x, axis=-1, order=2 ). Defined in tensorflow/python/keras/utils/np_utils.py . Normalizes a Numpy array. Arguments:​. Argument definition is - the act or process of arguing, reasoning, or discussing : argumentation. How to use argument in a sentence.

Comments
  • According to this answer it calls numpy linalg.norm() under the hood, you can see the "notes" section of that page for a description of the order parameter