Hot questions for Using Neural networks in ensemble learning
Is it possible to have an example on how to create an ensamble learner with Encog for time series forecasting?
I was thinking at a iRPROP+ ensamble learner.
Ensemble learning is still in the process of being added to Encog. More examples will be added when it is finalized. You can add a feature request on the issues page if you would like to suggest a specific example. https://github.com/encog/encog-java-core/issues
I have two matrices X
(122 x 125973) and Y
(1 x 125973). I want to do bootstrapping on my dataset in which I want to create
B observations (say
B = 3). As I understand it, the observations of size
B should be randomly drawn with replacement. How can I split in the same way
Y into smaller observation bootstrap samples?
randi() gives you the ability to drawn pseudorandom integers, including duplicate entries. These can then be used as indices to your observations. Thus:
X = rand(122,125973); Y = rand(1,125973); m = 3; % Your desired number of observations; maximum 125973 idx = randi(numel(Y),m,1) % Generate an mx1 vector BX = X(:,idx); % 122xm matrix BY = Y(:,idx); % 1xm matrix
You can remove entries from
Y as well, but since you said duplicate entries are explicitly allowed its usage may not be relevant:
X(:,idx) = ; %  sets to empty array, thus removes the entry
If you want multiple times
m observations, simply loop over it:
N = 100; % Number of observation matrices to be generated m = 3; % Number of observations per matrix X = rand(122,125973); Y = rand(1,125973); BX = rand(size(X,1),m,N); % 3D matrix for collection BY = rand(size(X,1),m,N); for ii = 1:N % Loop over all matrices to be generated idx = randi(numel(Y),m,1) % Generate an mx1 vector BX(:,:,ii) = X(:, idx); % 122xmxN matrix BY(:,:,ii) = Y(:, idx); % 1xmxN matrix end
BY are now 3D matrices containing
N matrices with
m observations each. Calling
BX(:,:,n) selects the
nthe matrix with observations. For reading on various indexing ways I suggest to read this post.
I have created two different Neural Networks that both predict if a team is going to WIN/LOSE a hockey game. The first NN has been trained on 82 features from games that has been played. The other has been trained on 115 features from player stats in all the games that has been played.
Some features are the same in both sets, but the important metrics differ or are represented differently (aggregated/individual)
Is there anyway to ensemble these NNs so that it is possible to just feed it a set containing only two features like this.
h_team1 h_team_2 h_team3 a_team1 a_team2 a_team3 WIN/LOSE 1 0 0 1 0 0 1 0 0 0 1 0 0 1 0 0 0 1
And the new NN's task will be to classify the probability of the home team winning the game.
If I'm really off with what I'm trying to do, please correct me.
I've added an image to help visualize the problem I'm describing.
https://imgur.com/a/ZKnGT8N (The arrows point to the model I want to build, have not built it, because I do not know how)
Example data from game_DATA-set h=home a=away
h_Won/Lost h_powerPlayGoals h_powerPlayPercentage a_powerPlayGoals a_powerPlayPercentage h_team1 h_team 2 a_team1 a_team2 gameID 1 2 0.4 1 0.5 1 0 1 0 1
Example data from player_DATA-set
Won/Lost playerID team1 team2 metric1 opponent1 opponent2 gameID 1 5678 1 0 10 1 0 1
In principle it is possible to feed that into another NN to get a new output but it makes absolutely no sense from a statistics point of view. You predict that your team is gonna win with x probability so the other team wins with 1-x probability. Therefore why would you need another network for this.
By using the softmax on the last layer your output is essentially x and 1-x for your first network and y and 1-y in the second network. Therefore, the only reasonable thing to do for the third network is to check if x > y and output a probability based on the margin between those two, but you can also just do this with a difference instead of a NN and then you do not have to learn anything.