HDInsight Fixed Issues
The fixed issues listed here are issues that were specifically addressed to support HDInsight. For fixed issues in Ambari, see the Ambari HDInsight Release Notes.
Celtic-HDI-M20-HDP 3.1.6.4 (October 2020)
Other
| Bug ID | Apache JIRA | Apache Component | Summary |
|---|---|---|---|
| BUG-124016 | HIVE-20052 HIVE-20679 HIVE-21281 HIVE-21290 HIVE-21291 HIVE-21706 HIVE-21729 HIVE-21862 HIVE-22094 HIVE-22105 HIVE-22227 HIVE-22405 HIVE-22572 HIVE-22589 HIVE-22593 HIVE-22595 HIVE-22661 HIVE-22736 HIVE-22840 HIVE-22841 HIVE-22842 HIVE-22856 HIVE-22860 HIVE-22890 HIVE-22903 HIVE-22941 HIVE-22973 HIVE-22998 HIVE-23022 HIVE-23023 HIVE-23034 HIVE-23044 ORC-27 ORC-399 ORC-437 ORC-475 ORC-498 ORC-578 | Hive, ORC | Multiple Hive and ORC fixes |
| BUG-124338 | ATLAS-3405 ATLAS-3417 ATLAS-3459 ATLAS-3492 ATLAS-3545 ATLAS-3592 ATLAS-3606 ATLAS-3614 ATLAS-3621 ATLAS-3658 ATLAS-3701 ATLAS-3711 ATLAS-3732 ATLAS-3734 ATLAS-3778 HDFS-15446 HIVE-22055 HIVE-22210 HIVE-22255 HIVE-22298 HIVE-22538 HIVE-22941 HIVE-23011 HIVE-23023 HIVE-23044 HIVE-23058 HIVE-23436 YARN-9685 YARN-9903 | Atlas, YARN, Hive | Multiple Atlas, YARN, and Hive fixes |
| HOTFIX-3259 | HIVE-22094, HIVE-22227, HIVE-22405, HIVE-22572, HIVE-22589, HIVE-22593, HIVE-22595, HIVE-22661, HIVE-22736, HIVE-22840, HIVE-22841, HIVE-22842, HIVE-22856, HIVE-22860, HIVE-22890, HIVE-22903, HIVE-22941, HIVE-22973, HIVE-22998, HIVE-23022, HIVE-23023, HIVE-23034, HIVE-23044, HIVE-20052, HIVE-21706, HIVE-21281, HIVE-20679, HIVE-21290, HIVE-21291, HIVE-22105, HIVE-21729, HIVE-21862 ORC-27, ORC-578, ORC-498, ORC-475, ORC-399 and ORC-437 | Hive, ORC | Multiple Hive and ORC fixes |
Celtic-HDI-M20-HDP 3.1.6.3 (June 2020)
Other
| Bug ID | Apache JIRA | Apache Component | Summary |
|---|---|---|---|
| HIVE-1232 | HIVE-23230 | Hive | Fixes an issue with Hive WareHouseConnector where the limit constraint is ignored and duplicate rows returned to user. |
| HIVE-1229 | HIVE-22221 | Hive | Fixes an error in Spark driver and HiverServer2 when you run queries on large tables from the Hive Warehouse Connector. |
| HIVE-20707 | HIVE-20707 | Hive | Automatic partition management |
| HIVE-23178 | HIVE-23178 | Hive/Tez | Add Tez Total Order Partitioner |
| HADOOP-1100 | HADOOP-16068 | ABFS | ABFS driver change to support getCanonicalServiceName override |
Celtic-HDI-M20-HDP 3.1.6 (April 2020)
Incorrect-Results
| Bug ID | Apache JIRA | Apache Component | Summary |
|---|---|---|---|
| BUG-122386 | HIVE-21924 | Hive | Backport HIVE-21924: Hive doesn't skip header for text file for queries running in Tez/LLAP tasks |
Other
| Bug ID | Apache JIRA | Apache Component | Summary |
|---|---|---|---|
| BUG-121844 | HADOOP-16404 | Hadoop Common | ABFS default blocksize change(256MB from 512MB) |
| BUG-118772 | HADOOP-16169 | Hadoop Common | ABFS: Bug fix for getPathProperties |
| BUG-118672 | HADOOP-16182 | Hadoop Common | Update abfs storage back-end with "close" flag when application is done writing to a file |
| BUG-121935 | HBASE-22380 | HBase | break circle replication when doing bulkload |
| BUG-121788 | HBASE-20662 | HBase | Fix SpaceViolationPolicy.DISABLE enforcement and SpaceQuota policy state issues |
| BUG-121607 | N/A | HBase | Upgrade HBase to branch-2.1 for next HDI release |
| BUG-121776 | N/A | HDP / Stack | Upgrade pack is missing for HOU (3.0->3.0, 3.1->3.1) |
| BUG-120341 | N/A | HDP / Stack | Disable hive-site/hive.server2.enable.doAs property by default and don't advertise it on the UI |
| BUG-123346 | HIVE-23033 | Hive | MSSQL metastore schema init script doesn't initialize NOTIFICATION_SEQUENCE |
| BUG-122386 | HIVE-21924 | Hive | Backport HIVE-21924: Hive doesn't skip header for text file for queries running in Tez/LLAP tasks |
| BUG-119402 | N/A | Hive | Disable HIVE_SERVER 1+ cardinality requirement for MSFT Celtic |
| BUG-119332 | HIVE-21507 | Hive | Hive swallows NPE if no delegation token found |
| BUG-118846 | N/A | Hive | Webhcat Hive calls fail due to missing HS2 URL |
| BUG-118189 | N/A | Hive | Invalid TaskLaunchCmdOpts defined for Vertex Map : Invalid/conflicting GC options found |
| BUG-118417 | N/A | Hive | Hive Metastore can't convert nvarchar to CLOB on SQL Server in Celtic HDI |
| BUG-121829 | N/A | Hive2 | One of the hive server2 is found to be down post U11/U12->U13 EU/HOU upgrade. |
| BUG-119971 | N/A | Kafka | NPE on missing Ranger conf - due to missing symlink |
| BUG-117414 | AMBARI-25168 | Kafka | Leader for a topic is assigned 'None' |
| BUG-122231 | N/A | Phoenix | HiveTezIT UTs are failing on HDP-3.1.6.0 |
| BUG-122702 | RANGER-2691 | Ranger | Ranger upgrade from AtlanticM05 to Celtic fails with MsSQL |
| BUG-118666 | RANGER-2691 | Ranger | External user's email address can be edited |
| BUG-118138 | N/A | Ranger | Incorrect foreign key in MSSQL |
| BUG-121954 | N/A | Spark | Upgrade Spark version to 2.4.4 |
| BUG-113248 | N/A | Spark | Spark2 test failures (Spark 2.4): HBase test failures, test_spark_hbase and test_hbasesource |
| HADOOP-1095 | N/A | Hadoop Common | port hadoop-16640 to celtic m20 |
| HADOOP-1098 | N/A | Hadoop Common | Backport fixes for double close on wasb and abfs which affect spark streaming |
| HBASE-555 | N/A | HBase | On pre-split HBase table with empty regions creating phoenix view causes infinite loop of region transition when WAL is on different filesystem than data |
| HIVE-1132 | N/A | Hive | HMS `create_table` generate `hive` owned tables on unsecure Atlantic environments |
| HIVE-1168 | N/A | Hive | MSSQL upgrade scripts have some issue (2.1.2000 to 3.1.0) |
| HIVE-1167 | N/A | Hive | WebHCat doesn't autostart |
| HIVE-1169 | N/A | Hive | Turn on point lookup optimization |
| HIVE-1180 | N/A | Hive | Hive Warehouse Connector select() issue after count() |
| HIVE-1176 | N/A | Hive | Wrong file owner when Hive table is created with CTAS. |
| HIVE-1186 | N/A | Hive | Text file headers can get included in results for insert queries or other queries that run via Tez/LLAP tasks |
| HIVE-1184 | N/A | Hive | Do as doesn't work as expected for drop view |
| HIVE-1198 | N/A | Hive/Ambari | Ambari Hive check uses the wrong connection string in HSI HA |
| HIVE-1188 | N/A | Hive | Yarn-Service is not launched with higher priority by HS2 |
| SPARK-332 | N/A | Spark | Spark Thrift Server goes down when YARN is restarted |
| SPARK-331 | N/A | Spark | Spark catalog in Hive metastore is not available at first attempt |
| SPARK-335 | N/A | Spark | Unauthorized 401 Exception to Spark |
| SPARK-337 | N/A | Spark | Need SPARK-28699 in Celtic and U9 |
| SQOOP-121 | N/A | Sqoop | Enable Sqoop import as PARQUET Format for WASB, ADL Gen1 and Gen2 |
| SPARK-341 | N/A | Spark | backport https://issues.apache.org/jira/browse/LIVY-642 & LIVY-576 to celtic as hotfix |
| WASB-140 | N/A | WASB | wildfly logs to the console |
| WASB-147 | N/A | WASB | ABFS: Backport updates to Hadoop-16548 and Hadoop-16578 |
| WASB-149 | N/A | WASB | ABFS: Backport HADOOP-16660 |
| WASB-150 | N/A | WASB | ABFS: Backport SAS Token provider Interface from HADOOP-16730 |
Performance
| Bug ID | Apache JIRA | Apache Component | Summary |
|---|---|---|---|
| HIVE-1217 | N/A | Hive | Request to include Hive perf improvement fixes in next drop for HDI |
| OOZIE-383 | OOZIE-3578 | Oozie | MapReduce counters cannot be used over 120 |
| WASB-147 | N/A | WASB | ABFS: Backport updates to Hadoop-16548 and Hadoop-16578 |
| WASB-146 | N/A | WASB | Backport HADOOP-16404: ABFS default blocksize change |
| BUG-122154 | HADOOP-16548 | Hadoop Common | ABFS: Config to enable/disable flush operation |
| BUG-122156 | HADOOP-16578 | Hadoop Common | ABFS: fileSystemExists() should not call container level apis |
Security
| Bug ID | Apache JIRA | Apache Component | Summary |
|---|---|---|---|
| OOZIE-379 | N/A | Oozie | Oozie hive jobs are failing in secure env with mapreduce.job.map.class is incompatible with map compatibility mode |
| SPARK-336 | N/A | Spark | Ranger Policies do not get applied to Spark Thrift Server |
| WASB-150 | N/A | WASB | ABFS: Backport SAS Token provider Interface from HADOOP-16730 |
| BUG-122386 | HIVE-21924 | Hive | Backport HIVE-21924: Hive doesn't skip header for text file for queries running in Tez/LLAP tasks |
Usability
| Bug ID | Apache JIRA | Apache Component | Summary |
|---|---|---|---|
| OOZIE-383 | OOZIE-3578 | Oozie | MapReduce counters cannot be used over 120 |
| HADOOP-1096 | N/A | Hadoop Common | ABFS: Make RetryCount in ExponentialRetryPolicy Configurable |
| RANGER-179 | RANGER-2356 | Ranger | Admin: external user's email can be edited |
| BUG-122051 | PHOENIX-5506 | Phoenix | Psql load fails with lower table name |
Celtic-HDI-M10-HDP 3.1.3 (September 2019)
Other
| Bug ID | Apache JIRA | Apache Component | Summary |
|---|---|---|---|
| BUG-121448 | N/A | Druid | Upgrade jackson to 2.9.9 and jackson-databind to 2.9.9.1 |
| BUG-121449 | N/A | Druid | Upgrade druid dependencies with reported CVEs in blackduck scan for Celtic-HDI M10 |
| BUG-121450 | N/A | Druid | Upgrade to guava 28.0-jre in Druid |
| BUG-121466 | N/A | Druid | Upgrade jackson-databind to 2.9.9.3 |
| BUG-121357 | HADOOP-16460 | Hadoop Common | ABFS: fix for Sever Name Indication (SNI) |
| BUG-121434 | N/A | Hadoop Common | Upgrade jackson to 2.9.9 and jackson-databind to 2.9.9.3 and Guava to 28.0-jre in Hadoop |
| BUG-118772 | HADOOP-16169 | Hadoop Common | ABFS: Bug fix for getPathProperties |
| BUG-118672 | HADOOP-16182 | Hadoop Common | Update abfs storage back-end with "close" flag when application is done writing to a file |
| BUG-121447 | N/A | HBase | Upgrade jackson-databind to 2.9.9.2 in HBase |
| BUG-121468 | N/A | HBase | Upgrade jackson-databind to 2.9.9.3 in HBase |
| BUG-120843 | HBASE-22617 | HBase | Backport HBASE-22617 "Recovered WAL directories not getting cleaned up" |
| BUG-120341 | N/A | HDP / Stack | Disable hive-site/hive.server2.enable.doAs property by default and don't advertise it on the UI |
| BUG-113948 | N/A | Hive | Spark2LLAP test failures (Spark 2.4): jvm related errors, test_tpcds[query*.sql] and test_etl_scala[*] |
| BUG-119142 | N/A | Hive | Hive ptest results - Celtic-HDI |
| BUG-119402 | N/A | Hive | Disable HIVE_SERVER 1+ cardinality requirement for MSFT Celtic |
| BUG-119418 | N/A | Hive | Update HiveKafkaProducer to be compatible with Kafka v2.2 |
| BUG-119332 | HIVE-21507 | Hive | Hive swallows NPE if no delegation token found |
| BUG-119334 | HIVE-21573 | Hive | Binary transport shall ignore principal if auth is set to delegationToken |
| BUG-118846 | N/A | Hive | Webhcat Hive calls fail due to missing HS2 URL |
| BUG-118417 | N/A | Hive | Hive Metastore can't convert nvarchar to CLOB on SQL Server in Celtic HDI |
| BUG-121467 | N/A | HWC | Upgrade to guava 28.0-jre and thrift to 0.9.3-1 in HWC |
| BUG-119971 | N/A | Kafka | NPE on missing Ranger conf - due to missing symlink |
| BUG-117414 | AMBARI-25168 | Kafka | Leader for a topic is assigned 'None' |
| BUG-119492 | N/A | N/A | Spark2 Mawo job fails due to required mvn version |
| BUG-119433 | N/A | N/A | Pig Mawo job tries to build against SNAPSHOTs |
| BUG-121445 | N/A | Pig | Upgrade to guava 28.0-jre in Pig |
| BUG-121446 | N/A | Pig | Update groovy to 2.4.8 in Pig (maven artifact generation) |
| BUG-120957 | N/A | Ranger | Ranger unit test failure on HDP-3.1.3.0 |
| BUG-121444 | N/A | Spark | Upgrade Livy jackson-databind to 2.9.9.2 |
| BUG-121441 | N/A | Spark | Upgrade Spark2 jackson-databind to 2.9.9.2 |
| BUG-121455 | N/A | Spark | Upgrade SparkHive2 jackson-databind to 2.9.9.3 |
| BUG-121456 | N/A | Spark | Upgrade Spark2 jackson-databind to 2.9.9.3 |
| BUG-121457 | N/A | Spark | Upgrade Livy jackson-databind to 2.9.9.3 |
| BUG-113248 | N/A | Spark | Spark2 test failures (Spark 2.4): HBase test failures, test_spark_hbase and test_hbasesource |
| BUG-121440 | N/A | Spark | Upgrade SparkHive2 jackson-databind to 2.9.9.2 |
| BUG-120388 | N/A | Sqoop | Sqoop fails to copy Table to hive using Hcat |
| BUG-121475 | N/A | Superset | Superset CVE fixes for Celtic HDI |
| BUG-121453 | TEZ-4004 | Tez | Upgrade jetty to 9.3.24 in Tez for Celtic HDI |
| BUG-121452 | N/A | Tez | Upgrade to guava 28.0-jre in Tez |
| BUG-121442 | N/A | Zeppelin | Upgrade Zeppelin com.google.guava:guava 28.0-jre |
| BUG-121459 | N/A | Zeppelin | Port CVE fixes to Zeppelin |
| WASB-144 | N/A | N/A | ABFS fix for Sever Name Indication (SNI) |
| WASB-143 | N/A | N/A | ABFS short-term fix to Create Hive tables |
| WASB-140 | N/A | N/A | wildfly logs to the console |
| SQOOP-120 | N/A | N/A | Sqoop fails to copy Table to hive using Hcat |
| SQOOP-118 | N/A | N/A | Sqoop: test_copyParquetTableToHive[mysql] fails |
| SPARK-335 | N/A | N/A | Unauthorized 401 Exception to Spark |
| SPARK-333 | N/A | N/A | Celtic-HDI UT test failures |
| SPARK-332 | N/A | N/A | Spark Thrift Server goes down when Yarn is restarted |
| SPARK-331 | N/A | N/A | Spark catalog in Hive metastore is not available at first attempt |
| OOZIE-382 | N/A | N/A | Oozie-HA tests fail because of missing hive-ha-large-data.tar.gz |
| OOZIE-381 | N/A | N/A | Oozie fails to connect to Hive |
| HIVE-1190 | N/A | HIVE | Hive tests fail because persistence manager is closing in Metastore |
| HIVE-1185 | N/A | HIVE | Metastore cache update jiras taken into hdp-msft release? |
| HIVE-1184 | N/A | N/A | Do as doesn't work as expected for drop view |
| HIVE-1181 | N/A | HIVE | Decimal scale difference between HSI and HWC |
| HIVE-1180 | N/A | HIVE | Hive Warehouse Connector select() issue after count() |
| HIVE-1177 | N/A | HIVE | Group by rollup returns with empty result when the grouped dataset is empty |
| HIVE-1176 | N/A | N/A | Wrong file owner when Hive table is created with CTAS. |
| HIVE-1174 | N/A | HIVE | CTAS ends with wrong data types |
| HIVE-1173 | N/A | HIVE | TPCDS tests fails in setup phase on ABFS |
| HIVE-1172 | N/A | HIVE | Varying result in certain cases |
| HIVE-1170 | N/A | N/A | EOFException in ABFS unsecure Hive tests |
| HIVE-1169 | N/A | N/A | Turn on point lookup optimization |
| HIVE-1168 | N/A | N/A | MSSQL upgrade scripts have some issue (2.1.2000 to 3.1.0) |
| HIVE-1167 | N/A | HIVE | WebHCat doesn't autostart |
| HIVE-1132 | N/A | N/A | HMS `create_table` generate `hive` owned tables on unsecure Atlantic environments |
| HIVE-1092 | N/A | N/A | HA for HIVE_SERVER_INTERACTIVE |
| HBASE-555 | N/A | N/A | On pre-split HBase table with empty regions creating phoenix view causes infinite loop of region transition when WAL is on different filesystem than data |
| HBASE-548 | N/A | N/A | test_runIntegrationTestZKAndFSPermissions fails on secure environments |
Performance
| Bug ID | Apache JIRA | Apache Component | Summary |
|---|---|---|---|
| BUG-119395 | AMBARI-25235 | Ambari | Add a sysprep configurations to run conf-selects only a single time |
Security
| Bug ID | Apache JIRA | Apache Component | Summary |
|---|---|---|---|
| BUG-118138 | N/A | Ranger | Incorrect foreign key in MSSQL |
| OOZIE-379 | N/A | OOZIE | Oozie hive jobs are failing in secure env with mapreduce.job.map.class is incompatible with map compatibility mode |
Usability
| Bug ID | Apache JIRA | Apache Component | Summary |
|---|---|---|---|
| BUG-119999 | HIVE-21831 | Hive | Import of acid table doesn't update row count metadata - import_export.q |
| BUG-121200 | N/A | Hive | Backport HIVE-20707 and related patches to Celtic HDI |
| RANGER-179 | RANGER-2356 | N/A | Admin: external user's email can be edited |
Celtic-HDP 3.1.2 (April 2019)
Other
| Bug ID | Apache JIRA | Component | Summary |
|---|---|---|---|
| BUG-118672 | HADOOP-16182 | Hadoop Common | Update abfs storage back-end with "close" flag when application is done writing to a file |
| BUG-118762 | HADOOP-16157 | Hadoop Common | [Clean-up] Remove NULL check before instanceof in AzureNativeFileSystemStore |
| BUG-118763 | HADOOP-15813 | Hadoop Common | Enable more reliable SSL connection reuse |
| BUG-118764 | HADOOP-16163 | Hadoop Common | NPE in setup/teardown of ITestAbfsDelegationTokens |
| BUG-118770 | HADOOP-16068 | Hadoop Common | ABFS Authentication and Delegation Token plugins to optionally be bound to specific URI of the store |
| BUG-118772 | HADOOP-16169 | Hadoop Common | ABFS: Bug fix for getPathProperties |
| BUG-118774 | HADOOP-16105 | Hadoop Common | WASB in secure mode does not set connectingUsingSAS |
| BUG-118779 | HADOOP-16104 | Hadoop Common | WASB tests to downgrade to skip when test a/c is namespace enabled |
| BUG-118780 | HADOOP-15582 | Hadoop Common | Document ABFS |
| BUG-118846 | N/A | Hadoop Common | WebHcat Hive calls fail due to missing HS2 URL |
| BUG-118965 | HADOOP-15994 | Hadoop Common | Upgrade Jackson2 to 2.9.8 |
| WASB-140 | N/A | Hadoop Common | wildfly logs to the console |
| HBASE-548 | N/A | HBase | test_runIntegrationTestZKAndFSPermissions fails on secure environments |
| BUG-118417 | N/A | Hive | Hive Metastore can't convert nvarchar to CLOB on SQL Server in Celtic HDInsight |
| HIVE-1167 | N/A | Hive | WebHCat doesn't autostart |
| HIVE-1169 | N/A | Hive | Turn-on point lookup optimization |
| HIVE-1173 | N/A | Hive | TPCDS tests fails in setup phase on ABFS |
| HIVE-1168 | N/A | Hive | MSSQL upgrade scripts have some issue (2.1.2000 to 3.1.0) |
| HIVE-1176 | N/A | Hive | Wrong file owner when Hive table is created with CTAS. |
| HIVE-1180 | N/A | Hive | Hive Warehouse Connector select() issue after count() |
| BUG-119402 | N/A | Hive | Disable HIVE_SERVER 1+ cardinality requirement for HDP 3.1.2 for HDInsight |
| BUG-119332 | HIVE-21507 | Hive | Hive swallows NPE if no delegation token is found |
| BUG-118846 | N/A | Hive | WebHCat Hive calls fail due to missing HiveServer2 URL |
| BUG-113948 | N/A | Hive, Spark | Spark2LLAP test failures (Spark 2.4): JVM related errors, test_tpcds[query*.sql] and test_etl_scala[*] |
| BUG-117414 | AMBARI-25168 | Kafka | Leader for a topic is assigned 'None' |
| RMP-12966 | N/A | Spark | Upgrade Spark to V2.4 |
| SPARK-331 | N/A | Spark | Spark catalog in Hive metastore is not available at first attempt |
| SPARK-332 | N/A | Spark | Spark Thrift Server goes down when Yarn is restarted |
| BUG-113248 | N/A | Spark | Spark2 test failures (Spark 2.4): HBase test failures, test_spark_hbase and test_hbasesource |
| SQOOP-118 | N/A | Sqoop | Sqoop: test_copyParquetTableToHive[mysql] fails |
Security
| Bug ID | Apache JIRA | Component | Summary |
|---|---|---|---|
| OOZIE-379 | N/A | Oozie | Oozie Hive jobs are failing in secure environment with mapreduce.job.map.class is incompatible with map compatibility mode |
| BUG-118138 | N/A | Ranger | Incorrect foreign key in MSSQL |
| BUG-118666 | RANGER-2356 | Ranger | External user's email address can be edited |
Usability
| Bug ID | Apache JIRA | Component | Summary |
|---|---|---|---|
| RANGER-179 | N/A | Ranger | Admin: External user's email can be edited |
Atlantic- HDP 3.0.2 (March 2019)
Incorrect Results
| Bug ID | Apache JIRA | Component | Summary |
|---|---|---|---|
| HIVE-1112 | N/A | Hive | Incorrect results with query result cache |
| HIVE-1118 | N/A | Hive | Show tables not listing the table when queries on the table work |
| HIVE-1119 | N/A | Hive | TPCDS - query91 returns 0 rows (incorrect results) |
| SPARK-300 | N/A | Spark | Spark history server UI doesnt show all applications in secure spark 2.2 |
Other
| Bug ID | Apache JIRA | Component | Summary |
|---|---|---|---|
| BUG-114180 | HBASE-21466 | HBase | WALProcedureStore uses wrong FileSystem if wal.dir is not under rootdir |
| BUG-115734 | HBASE-21523, HBASE-21524 | HBase | Chatty DEBUG logging in Master log |
| HBASE-536 | N/A | HBase | System table hbase:namepace left in transition after upgrade from HBase 1.1 to HBase 2.0 |
| HBASE-540 | N/A | HBase | HMaster fails in HDP 3.0 with HBase 2.0 when using Azure managed disk for WAL logs |
| HBASE-544 | N/A | HBase | Regression on split_2 on WASB |
| HDFS-197 | N/A | Hadoop Common | Bug in hadoop-httpfs systemd service scripts |
| HDFS-199 | N/A | Hadoop Common | WASB2 java.lang.NoSuchMethodError: org.apache.commons.codec.binary.Base64.isBase64 |
| HDFS-200 | N/A | Hadoop Common | HDP 3.0.2.0-13 cluster fails with AzureException: No credentials found for account mycluster in the configuration |
| HDFS-202 | N/A | Hadoop Common | hadoop-httpsfs.service file doesn't exist on 3.0 cluster |
| HIVE-1017 | N/A | Hive | The property StatusFolder that unicode content will be changed when we sumit job with powershell CLI (status folder isn't generated if the statusfolder only unicode content) |
| HIVE-1110 | N/A | Hive | Bucket file name issues on HDI atlantic |
| HIVE-1114 | N/A | Hive | Hive CLI does not work on the LLAP cluster for 3.0 |
| HIVE-1120 | N/A | Hive | HiveServer2 Interactive stops during deployment phase |
| HIVE-1136 | N/A | Hive | Hive throws OutOfMemoryError on ABFS during QueryResultsCache operation |
| HIVE-1154 | N/A | Hive | Permission Denied for Create External Table with fallback authorization |
| HIVE-1165 | HIVE-21214 | Hive | HiveServer2 split 2 empty bucket tests fail due to missing data from output |
| OOZIE-371 | N/A | Oozie | Oozie fails to get job information from Hadoop on ABFS |
| PHOENIX-134 | N/A | Phoenix | Phoenix IOException while using ABFS |
| RANGER-174 | N/A | Ranger | Ranger start may be prevented by leftover DB entry |
| SPARK-307 | N/A | Spark | Error in custom script action |
| SPARK-310 | N/A | Spark | STS shows ambari alerts |
| SPARK-316 | N/A | Spark | Backport BUG-110970 into HDI |
| SPARK-319 | N/A | Spark | Secure Spark cluster creation failure with WASB |
| WASB-136 | N/A | Spark | enable use of HDP nameservice filesystem paths in WASB2 (abfs and abfss protocols) |
| YARN-320 | N/A | Spark | METRICS_COLLECTOR is not starting on HDInsight Atlantic ABFS unsecure runs |
| YARN-322 | N/A | Spark | Spark UI failed due to certificate verification error |
| ZOOK-67 | N/A | Spark | zoo.cfg does not exist in /usr/hdp/current/zookeeper-client/conf dir |
Performance
| Bug ID | Apache JIRA | Component | Summary |
|---|---|---|---|
| WASB-132 | N/A | N/A | WASB: improve listStatus performance (Port HADOOP-15547) |
Query Failure
| Bug ID | Apache JIRA | Component | Summary |
|---|---|---|---|
| HDFS-201 | N/A | Hadoop Common | Hive jobs on ABFS 3.0 clusters fail with NoSuchMethodError from commons.codec |
| BUG-96841 | HIVE-20510, HIVE-20540 | Hive | Bucketing: _bucket_number is marked as a constant, not a virtual column |
Security
| Bug ID | Apache JIRA | Component | Summary |
|---|---|---|---|
| HIVE-1113 | N/A | Hive | Incorrect property value for hive.security.metastore.authorization.manager - Atlantic HDI |
| HIVE-1140 | N/A | Hive | HiveServer2 cannot start in WASB Secure for Atlantic clusters |
| RANGER-118 | N/A | Hive | Maintain parity with Ranger-HDFS policy-spec wrt recursive and wildcard |
| SPARK-317 | N/A | Spark | Livy broken in secure spark clusters |
| QE-21187 | N/A | HDFS, MapReduce, YARN | YARN and HDFS test failures due to OpenSSL logging |
| BUG-118138 | N/A | Ranger | Incorrect foreign key in MSSQL |
Stability
| Bug ID | Apache JIRA | Component | Summary |
|---|---|---|---|
| BUG-113464 | HIVE-20834 | Hive | Hive QueryResultCache entries keeping reference to SemanticAnalyzer from cached query |
| BUG-114044 | RANGER-2286 | Ranger | Ranger start may be prevented by leftover database entry |
| ZEP-65 | N/A | Zeppelin | Zeppelin modifies the authentication cache on logout, which breaks behavior for HDInsight |
Supportability
| Bug ID | Apache JIRA | Component | Summary |
|---|---|---|---|
| HIVE-1116 | N/A | Hive | Kill API support on WebHCat for Hive |
| HIVE-1129 | N/A | Hive | Hive JDBC jars for Beeline not available for download in public URL |
| HIVE-1149 | N/A | Hive | AzureDB schema no longer packaged with Hive |
| ZEP-80 | N/A | Zeppelin | Kerberos (SPNEGO) Support |
| ZEP-81 | N/A | Zeppelin | Zeppelin UI doesn't load fully on Chrome (Works on Edge) |
Usability
| Bug ID | Apache JIRA | Component | Summary |
|---|---|---|---|
| SPARK-309 | N/A | Spark | beeline on port 10002 doesnt work |