what is the fastest method to generate matrix of data type float?

python fast array operations
numpy array
numpy data types
numpy performance tips
cast numpy array to floats
numpy astype
numpy vectorize performance
numpy array to numeric

I am trying to generate a 10000*10000 matrix of data type float with non zero entries using the following code:

import numpy as np
list_going_in=np.random.rand(10000,10000)

but it takes more than a second to generate is there a faster method than this to do it.

appreciate your help.

No, there is no faster way to do this. The overhead of random number generation is limiting you. The only way to optimize these things is if there is some structure to the matrix (like everything 0, or triangle, or sparse, etc...). If every cell is different, you have to calculate and store each one separately.

np.random may be the fastest as far as I know, and is close in any case.

numpy.matrix.astype, method. matrix. astype (dtype, order='K', casting='unsafe', subok=True, copy=​True)¶. Copy of the Typecode or data-type to which the array is cast. order : {'C'​  Approximate-number data types for use with floating point numeric data. Floating point data is approximate; therefore, not all values in the data type range can be represented exactly. The ISO synonym for real is float (24). Transact-SQL Syntax Conventions. float [ (n) ] Where n is the number of bits that are used to store the mantissa of the

you can use empty for this purpose:

for example something like:

np.empty((1000,1000))+np.empty((1000))

One Simple Trick for Speeding up your Python Code with Numpy, When using Python for those types of work, it's often necessary to work with very large datasets. Those large datasets get read directly into memor, and are stored and those array values by 1.0000001 in a regular floating point loop took Here's the fast way to do things — by using Numpy the way it was  The float() method is used to return a floating point number from a number or a string. Syntax: float(x) The method only accepts one parameter and that is also optional to use. Let us look at the various types of argument, the method accepts: A number : Can be an Integer or a floating point number. A String : Must contain numbers of any type.

If I am not mistaken the use of such situations only arise when you are developing neural networks, may I suggest that you use tensor-flow:

    import tensorflow as tf
    tf.random_uniform(
    shape,
    minval=0,
    maxval=None,
    dtype=tf.float32,
    seed=None,
    name=None
)

Data types, Generally, problems are easily fixed by explicitly converting array scalars to Python scalars, using the corresponding Python type function (e.g., int , float , complex ,  Matrix m = new Matrix(10,5); Matrix n = new Matrix(10,5); Matrix r; double d = 2; r = m + n; r = n*2; The classic implementation of these classes is to create a temporary Matrix object for each sub expression, and the problem is much more evident in the following code where we generate two temporary objects.

Data type objects (dtype), Quick search. Data type objects ( dtype )¶. A data type object (an instance of numpy.dtype class) describes how the bytes in the fixed-size of other data types, (e.g., describing an array item consisting of an integer and a float), Structured data types are formed by creating a data type whose field contain other data types. To generate a random float number between any two numbers, we can use random.uniform() random.uniform() to get a random float number within a range. The random.uniform() function returns a random floating-point number between a given range in Python. Let’s assume you want to generate a random float number between 10 to 100 Or from 50.50 to 75.5.

Chapter 4. NumPy Basics: Arrays and Vectorized , Get Python for Data Analysis now with O'Reilly online learning. Fast vectorized array operations for data munging and cleaning, subsetting and filtering The numerical dtypes are named the same way: a type name, like float or int , followed  There is another way to create a matrix in python. It is using the numpy matrix() methods. It is the lists of the list. For example, I will create three lists and will pass it the matrix() method. list1 = [2,5,1] list2 = [1,3,5] list3 = [7,5,8] matrix2 = np.matrix([list1,list2,list3]) matrix2 . You can also find the dimensional of the matrix using the matrix_variable.shape. The matrix2 is of (3,3) dimension.

The Complete Effect and HLSL Guide, Vector and Matrix Types Vectors and matrices are a standardized way in HLSL to represent one-dimensional and two- dimensional arrays of data composed from scalar types. By default, a vector is an array composed of four floating-point values. a set of standard vector types which can be used for quick convenience​. The following data items and methods are also supported: array.typecode¶ The typecode character used to create the array. array.itemsize¶ The length in bytes of one array item in the internal representation. array.append (x) ¶ Append a new item with value x to the end of the array. array.buffer_info ¶

Comments
  • Use np.empty()?
  • @Divakar but it only has zeros in it , I will like to have random float numbers in it
  • Unless you store the random numbers in advance and use the same ones everytime, no.
  • Do you need random numbers or non-zero numbers? np.empty() returns non-initialized array, which means "whatever was there before".
  • Thank you for your answer, is it possible that I generate an empty matrix and then perform an operation to have it include random numbers (e.g. multiplication or addition etc?
  • @RohitSharma I am afraid that would be slower. Write a c program that loops through 10000^2 iterations and generates random numbers - you will see it is not far off. numpy is almost always the fastest way to go in numerical methods.
  • Unless by "include" you mean not all of the matrix but a small subset? Then it is possible.
  • please suggest what it can be if a subset is non empty atleast
  • @RohitSharma That is not the way to work in SO. This answers your original question and it is impolite to change it now. If you would like, accept this, and write another question detailing exactly what you need done, otherwise I would be guessing. I suggest in your new question you write your end-goal as well (check out xy question).