Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster

Contents of mapred-site.xml :
<configuration>
<property>
 <name>mapreduce.framework.name</name>
 <value>yarn</value>
</property>

<property>
 <name>yarn.app.mapreduce.am.env</name>
 <value>HADOOP_MAPRED_HOME=/home/admin/hadoop-3.1.0</value>
</property>

<property>
 <name>mapreduce.map.env</name>
 <value>HADOOP_MAPRED_HOME=/home/admin/hadoop-3.1.0</value>
</property>

<property>
 <name>mapreduce.reduce.env</name>
 <value>HADOOP_MAPRED_HOME=/home/admin/hadoop-3.1.0</value>
</property>

<property> 
    <name>mapreduce.application.classpath</name>
    <value>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*,$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*</value>
</property>

</configuration>

Though I have set yarn.app.mapreduce.am.env and other parameters; I am getting Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster error. I am trying to run map reduce program remotely where hadoop is installed on linux machine and I am running it from windows machine. Following are my configuration settings for job.

public class WordCount {
  public static void main(String[] args)
      throws IOException, ClassNotFoundException, InterruptedException {
    //
    UserGroupInformation ugi = UserGroupInformation.createRemoteUser("admin");
    ugi.doAs(new PrivilegedExceptionAction<Void>() {

      public Void run() throws Exception {
        try {
          Configuration configuration = new Configuration();

          configuration.set("yarn.resourcemanager.address", "192.168.33.75:50001"); // see step 3
          configuration.set("mapreduce.framework.name", "yarn");
          // configuration.set("yarn.app.mapreduce.am.env",
          // "HADOOP_MAPRED_HOME=/home/admin/hadoop-3.1.0");
          // configuration.set("mapreduce.map.env", "HADOOP_MAPRED_HOME=/home/admin/hadoop-3.1.0");
          // configuration.set("mapreduce.reduce.env",
          // "HADOOP_MAPRED_HOME=/home/admin/hadoop-3.1.0");
          configuration.set("fs.defaultFS", "hdfs://192.168.33.75:54310"); // see step 2
          configuration.set("mapreduce.app-submission.cross-platform", "true");
          configuration.set("mapred.remote.os", "Linux");
          configuration.set("yarn.application.classpath",
              "$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/*:$HADOOP_COMMON_HOME/lib/*:"
                  + "$HADOOP_HDFS_HOME/*:$HADOOP_HDFS_HOME/lib/*:"
                  + "$HADOOP_YARN_HOME/*:$HADOOP_YARN_HOME/lib/*:"
                  + "$HADOOP_MAPRED_HOME/*:$HADOOP_MAPRED_HOME/lib/*");

          Job job = Job.getInstance(configuration);

          job.setJarByClass(WordCount.class); // use this when uploaded the Jar to the server and
                                              // running the job directly and locally on the server
          job.setOutputKeyClass(Text.class);
          job.setOutputValueClass(IntWritable.class);
          job.setMapperClass(MapForWordCount.class);
          job.setReducerClass(ReduceForWordCount.class);

          Path input = new Path("/user/admin/wordCountInput.txt");
          Path output = new Path("/user/admin/output");
          FileInputFormat.addInputPath(job, input);
          FileOutputFormat.setOutputPath(job, output);
          System.exit(job.waitForCompletion(true) ? 0 : 1);
        } catch (Exception e) {
          e.printStackTrace();
        }
        return null;
      }

    });


  }

Please help me. I am stuck on this problem from last 6 days. Thanks a lot in advance. Hadoop version : 3.1.0


I had the same problem and resolved by adding this in mapred-site.xml (so edit your mapreduce.application.classpath property)

<property> 
    <name>mapreduce.application.classpath</name>
    <value>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*,$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*,$HADOOP_MAPRED_HOME/share/hadoop/common/*,$HADOOP_MAPRED_HOME/share/hadoop/common/lib/*,$HADOOP_MAPRED_HOME/share/hadoop/yarn/*,$HADOOP_MAPRED_HOME/share/hadoop/yarn/lib/*,$HADOOP_MAPRED_HOME/share/hadoop/hdfs/*,$HADOOP_MAPRED_HOME/share/hadoop/hdfs/lib/*</value>
</property>

Could not find or load main class org.apache.hadoo, After checking the YARN logs it is showing the Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster. It is observed that MR jobs are failing with *Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster" even though HADOOP_MAPRED_HOME is set in mapred-site.xml. Tried building tar.gz in branch-3.0 and seems works fine with same configurations. But in branch-3.1 and trunk, it is failing.


I added following property in yarn-site.xml

<property>
  <name>yarn.application.classpath</name>
  <value> $HADOOP_CONF_DIR,$HADOOP_COMMON_HOME/share/hadoop/common/*,$HADOOP_COMMON_HOME/share/hadoop/common/lib/*,$HADOOP_HDFS_HOME/share/hadoop/hdfs/*,$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*,$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*,$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*,
    $HADOOP_YARN_HOME/share/hadoop/yarn/*,$HADOOP_YARN_HOME/share/hadoop/yarn/lib/* 
  </value>
</property>

And did same changes in my map reduce program.

  configuration.set("yarn.application.classpath",
              "{{HADOOP_CONF_DIR}},{{HADOOP_COMMON_HOME}}/share/hadoop/common/*,{{HADOOP_COMMON_HOME}}/share/hadoop/common/lib/*,"
                  + " {{HADOOP_HDFS_HOME}}/share/hadoop/hdfs/*,{{HADOOP_HDFS_HOME}}/share/hadoop/hdfs/lib/*,"
                  + "{{HADOOP_MAPRED_HOME}}/share/hadoop/mapreduce/*,{{HADOOP_MAPRED_HOME}}/share/hadoop/mapreduce/lib/*,"
                  + "{{HADOOP_YARN_HOME}}/share/hadoop/yarn/*,{{HADOOP_YARN_HOME}}/share/hadoop/yarn/lib/*");

My program is running smoothly now. Feel free to ask me details.

Oozie job failed with "Could not find or load main class org.apache , main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster". Article Number: 3249 □ Publication Date: April 15, 2018 □ Author: Bhuvnesh Chaudhary. Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster Please check whether your etc/hadoop/mapred-site.xml contains the below configuration: <property>


The issue is that Your Resource Manager(yarn) is not able to load Hadoop Libraries(jars). I solved this by updating the configurations. Added this to yarn-site.xml :

<property>
<name>yarn.application.classpath</name>
<value>C:/hadoop-2.8.0/share/hadoop/mapreduce/*,C:/hadoop-2.8.0/share/hadoop/mapreduce/lib/*,C:/Hadoop-2.8.0/share/hadoop/common/*,C:/Hadoop-2.8.0/share/hadoop/common/lib/*,
    C:/hadoop-2.8.0/share/hadoop/hdfs/*,C:/hadoop-2.8.0/share/hadoop/hdfs/lib/*,C:/hadoop-2.8.0/share/hadoop/yarn/*,C:/hadoop-2.8.0/share/hadoop/yarn/lib/*</value>
</property>

Please note that the paths used here can be relative according to your system.

[#MAPREDUCE-7055] MR jobs are failing with Could not find or , It is observed that MR jobs are failing with *Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster" even though  When I'm running distcp to move data from s3 to my local hdfs i get this exception during the map reduce job launched to copy the data: Error: Could not find or load main class org.apache.hadoop.


[#YARN-6999] Add log about how to solve Error: Could not find or , Add log about how to solve Error: Could not find or load main class org.apache.​hadoop.mapreduce.v2.app.MRAppMaster. Status: Assignee:. So we need to add the related config in etc/hadoop/mapred-site.xml manually. Currently the log only tells there is an Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster, without suggestion on how to solve it. I want to add the useful suggestion in log.


Hadoop 3.1.1 Could not find or load main class org.apache.hadoop , I installed Hadoop on Windows using Cygwin terminal. When I run hadoop version command, I get apache.hadoop.util.VersionInfo How to  Oozie job failed with "Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster" Article Number: 3249 Publication Date: April 15, 2018 Author: Bhuvnesh Chaudhary


Hadoop: Error: Could not find or load main class org.apache.hadoop , during the map reduce job launched to copy the data: Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster I checked​