Hot questions for Using Neural networks in ensemble learning

Question:

Is it possible to have an example on how to create an ensamble learner with Encog for time series forecasting?

I was thinking at a iRPROP+ ensamble learner.

Thank you


Answer:

Ensemble learning is still in the process of being added to Encog. More examples will be added when it is finalized. You can add a feature request on the issues page if you would like to suggest a specific example. https://github.com/encog/encog-java-core/issues

Question:

I have two matrices X (122 x 125973) and Y (1 x 125973). I want to do bootstrapping on my dataset in which I want to create B observations (say B = 3). As I understand it, the observations of size B should be randomly drawn with replacement. How can I split in the same way X and Y into smaller observation bootstrap samples?


Answer:

randi() gives you the ability to drawn pseudorandom integers, including duplicate entries. These can then be used as indices to your observations. Thus:

X = rand(122,125973);
Y = rand(1,125973);
m = 3; % Your desired number of observations; maximum 125973
idx = randi(numel(Y),m,1) % Generate an mx1 vector
BX = X(:,idx); % 122xm matrix
BY = Y(:,idx);  % 1xm matrix

You can remove entries from X and Y as well, but since you said duplicate entries are explicitly allowed its usage may not be relevant:

X(:,idx) = [];  % [] sets to empty array, thus removes the entry

If you want multiple times m observations, simply loop over it:

N = 100;  % Number of observation matrices to be generated
m = 3; % Number of observations per matrix
X = rand(122,125973);
Y = rand(1,125973);
BX = rand(size(X,1),m,N);  % 3D matrix for collection
BY = rand(size(X,1),m,N);

for ii = 1:N  % Loop over all matrices to be generated
    idx = randi(numel(Y),m,1) % Generate an mx1 vector
    BX(:,:,ii) = X(:, idx); % 122xmxN matrix
    BY(:,:,ii) = Y(:, idx);  % 1xmxN matrix
end

BX and BY are now 3D matrices containing N matrices with m observations each. Calling BX(:,:,n) selects the nthe matrix with observations. For reading on various indexing ways I suggest to read this post.

Question:

I have created two different Neural Networks that both predict if a team is going to WIN/LOSE a hockey game. The first NN has been trained on 82 features from games that has been played. The other has been trained on 115 features from player stats in all the games that has been played.

Some features are the same in both sets, but the important metrics differ or are represented differently (aggregated/individual)

Is there anyway to ensemble these NNs so that it is possible to just feed it a set containing only two features like this.

h_team1   h_team_2    h_team3    a_team1   a_team2    a_team3     WIN/LOSE
   1         0           0          1          0         0
   1         0           0          0          1         0
   0         1           0          0          0         1

And the new NN's task will be to classify the probability of the home team winning the game.

If I'm really off with what I'm trying to do, please correct me.

I've added an image to help visualize the problem I'm describing.

https://imgur.com/a/ZKnGT8N (The arrows point to the model I want to build, have not built it, because I do not know how)

EDIT:

Example data from game_DATA-set h=home a=away

h_Won/Lost h_powerPlayGoals h_powerPlayPercentage a_powerPlayGoals a_powerPlayPercentage h_team1    h_team 2     a_team1     a_team2     gameID
   1            2                    0.4                1              0.5                      1          0            1          0      1

Example data from player_DATA-set

Won/Lost   playerID    team1  team2   metric1   opponent1     opponent2    gameID
   1          5678       1       0     10       1              0          1

Answer:

In principle it is possible to feed that into another NN to get a new output but it makes absolutely no sense from a statistics point of view. You predict that your team is gonna win with x probability so the other team wins with 1-x probability. Therefore why would you need another network for this.

By using the softmax on the last layer your output is essentially x and 1-x for your first network and y and 1-y in the second network. Therefore, the only reasonable thing to do for the third network is to check if x > y and output a probability based on the margin between those two, but you can also just do this with a difference instead of a NN and then you do not have to learn anything.