org.apache.hadoop.hbase.mapreduce

Class TestImportTsv

java.lang.Object

org.apache.hadoop.hbase.mapreduce.TestImportTsv

public class TestImportTsv
extends java.lang.Object

Field Summary

Fields

Modifier and Type	Field and Description
static HBaseClassTestRule	CLASS_RULE
protected static java.lang.String	DELETE_AFTER_LOAD_CONF
ExpectedException	exception
protected static java.lang.String	FORCE_COMBINER_CONF Force use of combiner in doMROnTableTest.
protected static java.lang.String	NAME
protected static HBaseTestingUtility	util

Constructor Summary

Constructors

Constructor and Description
TestImportTsv()

Method Summary

Modifier and Type	Method and Description
protected static Tool	doMROnTableTest(HBaseTestingUtility util, TableName table, java.lang.String family, java.lang.String data, java.util.Map<java.lang.String,java.lang.String> args)
protected static Tool	doMROnTableTest(HBaseTestingUtility util, TableName table, java.lang.String family, java.lang.String data, java.util.Map<java.lang.String,java.lang.String> args, int valueMultiplier, int expectedKVCount) Run an ImportTsv job and perform basic validation on the results.
Configuration	getConf()
static void	provisionCluster()
static void	releaseCluster()
void	setConf(Configuration conf)
void	setup()
void	testBulkOutputWithAnExistingTable()
void	testBulkOutputWithAnExistingTableNoStrictTrue()
void	testBulkOutputWithoutAnExistingTable()
void	testBulkOutputWithTsvImporterTextMapper()
void	testDryModeWithBulkModeAndTableDoesNotExistsCreateTableSetToYes()
void	testDryModeWithBulkOutputAndTableDoesNotExistsCreateTableSetToNo() If table is not present in bulk mode and create.table is not set to yes, import should fail with TableNotFoundException.
void	testDryModeWithBulkOutputAndTableExists()
void	testDryModeWithoutBulkOutputAndTableDoesNotExists() If table is not present in non-bulk mode, dry run should fail just like normal mode.
void	testDryModeWithoutBulkOutputAndTableExists()
void	testJobConfigurationsWithDryMode()
void	testJobConfigurationsWithTsvImporterTextMapper()
void	testMROnTable()
void	testMROnTableWithCustomMapper()
void	testMROnTableWithTimestamp()
void	testMRWithoutAnExistingTable()
void	testSkipEmptyColumns()
void	testTsvImporterTextMapperWithInvalidData() If there are invalid data rows as inputs, then only those rows should be ignored.
void	testWithoutAnExistingTableAndCreateTableSetToNo()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CLASS_RULE

public static final HBaseClassTestRule CLASS_RULE

NAME

protected static final java.lang.String NAME

util

protected static HBaseTestingUtility util

DELETE_AFTER_LOAD_CONF

protected static final java.lang.String DELETE_AFTER_LOAD_CONF

FORCE_COMBINER_CONF

protected static final java.lang.String FORCE_COMBINER_CONF

Force use of combiner in doMROnTableTest. Boolean. Default is true.

exception

public ExpectedException exception

Constructor Detail

TestImportTsv

public TestImportTsv()

Method Detail

getConf

public Configuration getConf()

setConf

public void setConf(Configuration conf)

provisionCluster

public static void provisionCluster()
                             throws java.lang.Exception

Throws:

java.lang.Exception

releaseCluster

public static void releaseCluster()
                           throws java.lang.Exception

Throws:

java.lang.Exception

setup

public void setup()
           throws java.lang.Exception

Throws:

java.lang.Exception

testMROnTable

public void testMROnTable()
                   throws java.lang.Exception

Throws:

java.lang.Exception

testMROnTableWithTimestamp

public void testMROnTableWithTimestamp()
                                throws java.lang.Exception

Throws:

java.lang.Exception

testMROnTableWithCustomMapper

public void testMROnTableWithCustomMapper()
                                   throws java.lang.Exception

Throws:

java.lang.Exception

testBulkOutputWithoutAnExistingTable

public void testBulkOutputWithoutAnExistingTable()
                                          throws java.lang.Exception

Throws:

java.lang.Exception

testBulkOutputWithAnExistingTable

public void testBulkOutputWithAnExistingTable()
                                       throws java.lang.Exception

Throws:

java.lang.Exception

testBulkOutputWithAnExistingTableNoStrictTrue

public void testBulkOutputWithAnExistingTableNoStrictTrue()
                                                   throws java.lang.Exception

Throws:

java.lang.Exception

testJobConfigurationsWithTsvImporterTextMapper

public void testJobConfigurationsWithTsvImporterTextMapper()
                                                    throws java.lang.Exception

Throws:

java.lang.Exception

testBulkOutputWithTsvImporterTextMapper

public void testBulkOutputWithTsvImporterTextMapper()
                                             throws java.lang.Exception

Throws:

java.lang.Exception

testWithoutAnExistingTableAndCreateTableSetToNo

public void testWithoutAnExistingTableAndCreateTableSetToNo()
                                                     throws java.lang.Exception

Throws:

java.lang.Exception

testMRWithoutAnExistingTable

public void testMRWithoutAnExistingTable()
                                  throws java.lang.Exception

Throws:

java.lang.Exception

testJobConfigurationsWithDryMode

public void testJobConfigurationsWithDryMode()
                                      throws java.lang.Exception

Throws:

java.lang.Exception

testDryModeWithoutBulkOutputAndTableExists

public void testDryModeWithoutBulkOutputAndTableExists()
                                                throws java.lang.Exception

Throws:

java.lang.Exception

testDryModeWithoutBulkOutputAndTableDoesNotExists

public void testDryModeWithoutBulkOutputAndTableDoesNotExists()
                                                       throws java.lang.Exception

If table is not present in non-bulk mode, dry run should fail just like normal mode.

Throws:

java.lang.Exception

testDryModeWithBulkOutputAndTableExists

public void testDryModeWithBulkOutputAndTableExists()
                                             throws java.lang.Exception

Throws:

java.lang.Exception

testDryModeWithBulkOutputAndTableDoesNotExistsCreateTableSetToNo

public void testDryModeWithBulkOutputAndTableDoesNotExistsCreateTableSetToNo()
                                                                      throws java.lang.Exception

If table is not present in bulk mode and create.table is not set to yes, import should fail with TableNotFoundException.

Throws:

java.lang.Exception

testDryModeWithBulkModeAndTableDoesNotExistsCreateTableSetToYes

public void testDryModeWithBulkModeAndTableDoesNotExistsCreateTableSetToYes()
                                                                     throws java.lang.Exception

Throws:

java.lang.Exception

testTsvImporterTextMapperWithInvalidData

public void testTsvImporterTextMapperWithInvalidData()
                                              throws java.lang.Exception

If there are invalid data rows as inputs, then only those rows should be ignored.

Throws:

java.lang.Exception

testSkipEmptyColumns

public void testSkipEmptyColumns()
                          throws java.lang.Exception

Throws:

java.lang.Exception

doMROnTableTest

protected static Tool doMROnTableTest(HBaseTestingUtility util,
                                      TableName table,
                                      java.lang.String family,
                                      java.lang.String data,
                                      java.util.Map<java.lang.String,java.lang.String> args)
                               throws java.lang.Exception

Throws:

java.lang.Exception

doMROnTableTest

protected static Tool doMROnTableTest(HBaseTestingUtility util,
                                      TableName table,
                                      java.lang.String family,
                                      java.lang.String data,
                                      java.util.Map<java.lang.String,java.lang.String> args,
                                      int valueMultiplier,
                                      int expectedKVCount)
                               throws java.lang.Exception

Run an ImportTsv job and perform basic validation on the results. Returns the ImportTsv Tool instance so that other tests can inspect it for further validation as necessary. This method is static to insure non-reliance on instance's util/conf facilities.

Parameters:

args - Any arguments to pass BEFORE inputFile path is appended.

Returns:

The Tool instance used to run the test.

Throws:

java.lang.Exception