Hot questions for Using Mockito in jacoco

Question:

I am using mockito 1.8.3, jacoco 0.72 and maven 3.0.5 surefire plugin (2.12.4) to execute unit test and generating coverage report, it was working fine.

With more and more tests are added, it starts not working. I continuously encounter out of memory error during test execution, and cannot find out a way to figure out what is wrong.

I have about 1800+ test cases with mockito as the mocking tool. It is working fine if I do not run jacoco during maven test with "org.jacoco:jacoco-maven-plugin:prepare-agent " before test phase, but as long as I add jacoco agent, I get OOO issue with PermGen full.

I already added the PermGen to 2GB by modifying MAVEN_OPTS (which should not work since surefire will fork a new process) and surefire argline argument in pom, but it does not help a lot.

I try to get a core dump when OOO occurs by adding parameter to surefire plugin, but never saw a dump file in any folder. I am suspicious that my JVM setting does not work for the surefire plugin, but not sure what is wrong. Anyone could do me a favor? Thanks.

            <plugin>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>${surefire.version}</version>
                <inherited>true</inherited>
                <configuration>
                    <properties>
                        <property>
                            <name>argLine</name>                                    <value>-server -ea -XX:-UseSplitVerifier -XX:MaxPermSize=2g -Xmx3g -XX:+HeapDumpOnOutOfMemoryError </value>
                        </property>
                        <property>
                            <name>forkMode</name>
                            <value>once</value>
                        </property>
                        <property>
                            <name>reportFormat</name>
                            <value>plain</value>
                        </property>
                        <property>
                            <name>skipTests</name>
                            <value>${maven.test.skip}</value>
                        </property>
                    </properties>
                </configuration>
            </plugin>

Answer:

You need to set the memory for maven-surefire-plugin like the following:

<plugins>
[...]
  <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.18.1</version>
    <configuration>
        <forkCount>3</forkCount>
        <reuseForks>true</reuseForks>
        <argLine>-Xmx1024m -XX:MaxPermSize=256m</argLine>
        <systemPropertyVariables>
            <databaseSchema>MY_TEST_SCHEMA_${surefire.forkNumber}</databaseSchema>
        </systemPropertyVariables>
    </configuration>
  </plugin>
[...]
</plugins>

Question:

I need to test a method which calls another private method, the problem is not to try to mock the result that the private method returns, but to do so not to obtain the coverage of jacoco or sonar. I tested with Powermock, with EasyMock but when using a spy, when use an spy the test is not covered by sonar or jacoco. The private method and the public method that make the call are at the same class.

Situation to test:

public class ClassOne {

    private Object methodOne () {
        .....
        return object;
    }

    private Object mehodTwo () {
        return Object name = mehtodOne ();
    }
}

Test example:

package foo.bar;

import static org.junit.Assert.*;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.Mockito;
import org.powermock.api.mockito.PowerMockito;
import org.powermock.core.classloader.annotations.PrepareForTest;
import org.powermock.modules.junit4.PowerMockRunner;

@RunWith(PowerMockRunner.class)
@PrepareForTest(VlcPlayerMinimal.class)
public class VlcPlayerMinimalTest {

  @Test
  public void getVlcRcStatusTest() {
    VlcPlayerMinimal vlcPlayerSpy = PowerMockito.spy(new VlcPlayerMinimal());
    try {
      PowerMockito.doReturn("{status: stopped, id: 2}").when(vlcPlayerSpy, "executeGetRequest", Mockito.any(), Mockito.any());
      String vlcRcStatus = vlcPlayerSpy.getVlcRcStatus();
      System.out.println(vlcRcStatus);
    } catch (Exception e) {
      e.printStackTrace();
      fail("Unexpected exception thrown.");
    }
  }
}

Answer:

No, it's not possible. Pick one or the other.

What would such "coverage" even prove? You are not actually covering it. Code coverage is a metric to help you identify spots that are not well tested, not one that you should try to trick into appearing higher than it really is.

Question:

I'm unit testing my web application using JUnit and Mockito. At the moment i'm trying to write testcases for Service layer after completing the DAO level. This is my test case:

package it.********.testService;

import java.util.ArrayList;
import java.util.Date;
import java.util.List;
import java.util.TimeZone;

import static org.mockito.Mockito.*;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.equalTo;
import static org.assertj.core.api.Assertions.*;

import org.junit.Before;
import org.junit.Test;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.Mockito;
import org.mockito.MockitoAnnotations;
import org.mockito.Spy;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.AbstractTransactionalJUnit4SpringContextTests;

import it.********.constants.EasycareEnumTypes.ActivityType;
import it.********.dao.interf.AccountDAOInterface;
import it.********.dao.interf.CareTeamDAOInterface;
import it.********.dao.interf.DeviceDAOInterface;
import it.********.dao.interf.ObservationDAOInterface;
import it.********.dto.DeviceDTO;
import it.********.dto.ObservationDTO;
import it.********.exception.ECForbiddenException;
import it.********.mapper.DeviceMapper;
import it.********.mapper.ObservationMapper;
import it.********.model.Account;
import it.********.model.Device;
import it.********.model.Observation;
import it.********.service.DeviceService;
import it.********.service.ObservationService;
import it.********.testConf.MockDAOUtils;
import it.********.testConf.MockDTOUtils;
import it.********.testConf.TestContext;
import it.********.utilities.DatetimeUtils;
import it.********.utilities.SessionVariable;

@ContextConfiguration(classes = {TestContext.class})
@ComponentScan({"it.********.service"})
public class ObservationServiceTest extends AbstractTransactionalJUnit4SpringContextTests {

    @Autowired
    private MockDAOUtils mockDAOUtils;

    @Mock
    SessionVariable sessionVariable;

    @Mock
    private ObservationDAOInterface observationDAOMock;

    @Mock
    private AccountDAOInterface accountDAOMock;

    @Mock
    private CareTeamDAOInterface careTeamDAOMock;

    @Spy
    private ObservationMapper mapper;

    @InjectMocks
    private ObservationService service;

    @Before
    public void setup() {
        MockitoAnnotations.initMocks(this);
    }

    /**
     * {getObservationList}
     * 
     * [account]
     * -------------------------------------------------------------------------------------------------------
     * | id   | enabled | locked | account_expired | password_expired | data_handling | role                 |
     * -------------------------------------------------------------------------------------------------------
     * | 1000 | 1       | 0      | 0               | 0                | 1             | patient              |  targetAccount : valid patient account
     * | 1001 | 1       | 0      | 0               | 0                | 1             | general_practitioner |  callerAccount : medic in patient careteam
     * | 1002 | 1       | 0      | 0               | 0                | 0             | patient              |  targetAccount : invalid patient account
     * | 1003 | 1       | 0      | 0               | 0                | 1             | patient              |  callerAccount : patient with ID different from patientID
     * -------------------------------------------------------------------------------------------------------
     * 
     * [Observation]
     * ------------------------------------------------------------------------
     * | id | activity_type | medical_record_id | execution_date      | Note  |
     * ------------------------------------------------------------------------
     * | 1  | act_pressure  | 1                 | 2018-06-01 11:00:00 | Note1 |
     * | 2  | act_oximetry  | 1                 | 2018-06-01 14:00:00 | Note2 |
     * ------------------------------------------------------------------------
     * 
     */
    @Test
    public void test_case_u0048() throws Exception {

        // Taking method name
        String methodName = new Object() {
        }.getClass().getEnclosingMethod().getName();
        // Building test case file path
        String path = mockDAOUtils.getTc_path() + methodName + mockDAOUtils.getXml_ext();

        // Setting the test environment
        List<Account> accountList = mockDAOUtils.mockAccount(path);
        List<Observation> observationList = mockDAOUtils.mockObservation(path);
        List<ActivityType> activityTypeList = new ArrayList<ActivityType>();
        activityTypeList.add(ActivityType.act_pressure);
        activityTypeList.add(ActivityType.act_oximetry);
        Date startDate = mockDAOUtils.getFormat().parse("2018-06-01 00:00:00");
        Date endDate = mockDAOUtils.getFormat().parse("2018-06-01 23:59:59");


        // [RESULT CASE - VALID TARGET ACCOUNT / MEDIC CALLER ACCOUNT IN PATIENT CARE TEAM]
        Mockito.when(accountDAOMock.findByPersonId(1000L)).thenReturn(accountList.get(0));
        Mockito.when(careTeamDAOMock.isCareTeam(1000L, 1001L)).thenReturn(true);
        Mockito.when(sessionVariable.getTimezone()).thenReturn("GMT-8:00");
        System.out.println("[SV] = " + sessionVariable.getTimezone());
        Mockito.when(observationDAOMock.getListByMedRecIdAndActivityTypeAndDateRange(1L, activityTypeList, startDate, endDate)).thenReturn(observationList);
        List<ObservationDTO> observationDTOList = service.getObservationList(1000L, accountList.get(1), activityTypeList, startDate, endDate);

    }

}

Service Layer :

package it.********.service;

import java.util.ArrayList;
import java.util.Date;
import java.util.List;
import java.util.TimeZone;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

import it.********.constants.EasycareEnumTypes.ActivityType;
import it.********.constants.EasycareEnumTypes.ApplicationRole;
import it.********.dao.interf.AccountDAOInterface;
import it.********.dao.interf.CareTeamDAOInterface;
import it.********.dao.interf.ObservationDAOInterface;
import it.********.dto.ObservationDTO;
import it.********.exception.ECForbiddenException;
import it.********.exception.ECNotFoundException;
import it.********.mapper.ObservationMapper;
import it.********.model.Account;
import it.********.model.Observation;
import it.********.utilities.DatetimeUtils;
import it.********.utilities.SessionVariable;

@ComponentScan({"it.********.mapper","it.********.dao.impl"})
@Service
@Transactional
public class ObservationService {

    @Autowired
    ObservationDAOInterface observationDAO;

    @Autowired
    SessionVariable sessionVariable;

    @Autowired
    ObservationMapper observationMapper;

    @Autowired
    AccountDAOInterface accountDAO;

    @Autowired
    CareTeamDAOInterface careTeamDAO;

    /**
     * Provide the list of ObservationDTO of the patient included in the period and matching the list of types
     * 
     * @param patientId id of the patient
     * @param activityTypes list of activity types
     * @param startDate beginning of the period
     * @param endDate end of the period
     * @return the list of observations, eventually empty
     * 
     */

    @Transactional(readOnly=true)
    public List<ObservationDTO> getObservationList(long patientId, Account callerAccount, List<ActivityType> activityTypes, Date startDate, Date endDate){  

        // target permission check      
        Account targetAccount = accountDAO.findByPersonId(patientId);
        if(targetAccount == null) {
            throw new ECNotFoundException();
        } else if (!targetAccount.isValid()){
            throw new ECForbiddenException();
        } else if (targetAccount.getFirstApplicationRole() != ApplicationRole.patient) {
            throw new ECForbiddenException();
        }

        // caller permission check
        switch (callerAccount.getFirstApplicationRole()) {
            case general_practitioner:
            case medical_specialist:
                if(!careTeamDAO.isCareTeam(targetAccount.getId(), callerAccount.getId())) {
                    throw new ECForbiddenException();
                }
                break;
            case patient:
                if(targetAccount.getId() != callerAccount.getId()){
                    throw new ECForbiddenException();
                }
                break;
            default:
                throw new ECForbiddenException();       
        }

        List<Observation> observationList = observationDAO.getListByMedRecIdAndActivityTypeAndDateRange(targetAccount.getMedicalRecord().getId(), activityTypes, startDate, endDate);
        List<ObservationDTO> observationDTOList = new ArrayList<>();

        for(Observation item : observationList){
            observationDTOList.add(observationMapper.convertToDTO(item));
        }

        return observationDTOList;      
    }

}

Mapper:

package it.********.mapper;

import java.util.Map;
import java.util.TimeZone;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import it.********.constants.EasycareEnumTypes.ObservationExamType;
import it.********.constants.EasycareEnumTypes.ObservationParameterType;
import it.********.dto.ObservationDTO;
import it.********.dto.ScalarObservationDTO;
import it.********.dto.SignalObservationDTO;
import it.********.model.Observation;
import it.********.model.ObservationScalar;
import it.********.model.ObservationSignal;
import it.********.utilities.DatetimeUtils;
import it.********.utilities.SessionVariable;

@Component
public class ObservationMapper extends Mapper<ObservationDTO,Observation> {

    @Autowired
    SessionVariable sessionVariable;

    @Autowired
    ScalarObservationMapper scalarMapper;

    @Override
    public ObservationDTO convertToDTO(Observation entity) {
        ObservationDTO observationDTO = new ObservationDTO();
        observationDTO.setActivityType(entity.getActivityType());
        observationDTO.setActivityLabel(entity.getActivityType().getLabelCode());
        //observationDTO.setScheduleActivityId(entity.getScheduleActivity() != null ? entity.getScheduleActivity().getId() : null);
        observationDTO.setMedicalRecordId(entity.getMedicalRecord().getId());
        System.out.println("[SV] = " + sessionVariable.getTimezone());
        observationDTO.setExecutionDate(DatetimeUtils.dateToDatetimeISO8601(entity.getExecutionDate(),TimeZone.getTimeZone(sessionVariable.getTimezone())));
        observationDTO.setId(entity.getId());
        observationDTO.setNote(entity.getNote());

        for(ObservationScalar entityScalarObs : entity.getScalarObservationList()){
            ScalarObservationDTO scalarObsDTO = scalarMapper.convertToDTO(entityScalarObs);         
            observationDTO.getScalarObservationList().put(scalarObsDTO.getParameterType(), scalarObsDTO);
        }

        SignalObservationMapper signalMapper = new SignalObservationMapper();
        for(ObservationSignal entitySignalObs : entity.getSignalObservationList()){
            SignalObservationDTO signalObsDTO = signalMapper.convertToDTO(entitySignalObs);         
            observationDTO.getSignalObservationList().put(signalObsDTO.getType(), signalObsDTO);
        }

//      Hibernate.initialize(observationDTO.getScalarObservationList());
//      Hibernate.initialize(observationDTO.getSignalObservationList());
        return observationDTO;
    }

    @Override
    public Observation convertToEntity(ObservationDTO dto) {
        Observation observation = new Observation();
        observation.setActivityType(dto.getActivityType());
        observation.setExecutionDate(DatetimeUtils.stringToDate(dto.getExecutionDate()));
        observation.setNote(dto.getNote());

        for(Map.Entry<ObservationParameterType,ScalarObservationDTO> scalarObsDTO : dto.getScalarObservationList().entrySet()){
            if (scalarObsDTO.getValue() != null){//empty field
                observation.getScalarObservationList().add(scalarMapper.convertToEntity(scalarObsDTO.getValue()));
            }
        }

        SignalObservationMapper signalMapper = new SignalObservationMapper();
        for(Map.Entry<ObservationExamType,SignalObservationDTO> signalObsDTO : dto.getSignalObservationList().entrySet()){
            if (signalObsDTO.getValue() != null){
                observation.getSignalObservationList().add(signalMapper.convertToEntity(signalObsDTO.getValue()));
            }
        }       

        return observation;
    }

    @Override
    public void updateEntity(ObservationDTO dto, Observation entity) {
        // TODO Auto-generated method stub

    }

}

My problem comes while trying to test the "getObservationList" method of the Service : everything I mocked works good until I go to the mapper, where i got NullPointerException here

observationDTO.setExecutionDate(DatetimeUtils.dateToDatetimeISO8601(entity.getExecutionDate(),TimeZone.getTimeZone(sessionVariable.getTimezone())));

I think that the sessioVariable is null due to the fact that it isn't initialized in my test context, so I'm trying to mock o do some stuff to let me have a valid sessionVariable (the content is not important at this moment) not modifying nothing in Service and Mapper.

Thanks in advance! Bye!


Answer:

I believe the reason sessionVariable is null, even though you are clearly mocking it, is because ObservationMapper mapper is mocked using @Spy. Unlike @Mock, @Spy creates an instance of the object being mocked but everything referenced inside is null unless you explicitly inject the mocks in it.

Solution 1:

@Mock
SessionVariable sessionVariable;

@Mock
ScalarObservationMapper scalarMapper;

@Spy
@InjectMocks
private ObservationMapper mapper;

If that doesn't work you might want to try using @Mock, instead of @Spy and then mocking both services (sessionVariable and scalarMapper) inside ObservationMapper class. Regardless, you should mock both services as you are using both of them inside method convertToDTO.

Solution 2:

@Mock
SessionVariable sessionVariable;

@Mock
ScalarObservationMapper scalarMapper;

@Mock
private ObservationMapper mapper;

Question:

I'm running PowerMock 1.6.4 and all the latest (JUnit 4.11 though).

  • I use the Jacoco Ant task to only instrument the classes, not the test classes. I also use the Jacoco ant task to run the Junit tests, then generate the reports.

Now I'm hitting a problem that I can't figure out...

  • I have a test class that tests one member function of class Foo.
  • One of the members of Foo is static, so I've wrapped that in a static function so I can control the execution via mock but the side effect is that I need to mockStatic now.

What I've noticed is that PowerMockito.mockStatic(Foo.class) ... all tests fail with instrumentation problems.

I have another test class that tests another member function of Foo. This test class works fine, but as soon as I introduce a mockStatic the test class fails with instrumentation failures.

Has anyone see this failure and know of any workarounds? I can't change the static member variable.


Answer:

I finally figured out what I believe is the issue. Jacoco instrumentation injects data into your byte code so does PowerMock when it attempts to mock statics. This wrecks havoc since they are stepping on each other and you will get really odd behavior due to them messing with each other. I was getting an assorted bunch of NPE's in code that shouldn't throw NPE's.

The easy solution was to refactor out the unnecessary statics and know that if you plan to use statics to control data flow, probably should rethink the architecture for testing if you plan to use Jacoco for coverage.

You can still run Jacoco instrumentation on statics but you can't mock statics at the same time; at least not with the way PowerMock with Mockito does it. I'm not sure if EasyMock would result in a different behavior, so ymmv.