CHALLENGE

• Perform in-place upgrades of Hortonworks Data Platform (HDP) 2.x to Cloudera Data Platform (CDP) Data Center version 7.x. with minimum downtime

• Perform an in-place upgrade of Hortonworks Data Flow (HDF) to the current version, and ensure NiFi workloads are tested and functional

• Ensure that all SparkQL and Hive QL data and workloads continue to function properly and efficiently in the upgraded environment.

OUTCOME

• Successfully upgraded HDP 2.6.5 to CDP 7

• Completed a fresh install of CDP 7 for the Development Cluster and migrated the data from HDP 2.6.5

• Spark and HiveQL workloads migrated successfully

• Enabled job Orchestration and DevOps of workloads

APPROACH

• Convert Hive 2 Managed (non-transactional) tables to Hive 3 external

• Refactor the existing Hive, Spark, and Oozie workload code to properly function with Hive 3 tables

• Worked with Cloudera team members, in partnership, to execute HDP to CDP utilities expediting the in-place upgrade process.

SOLUTION

• CDP 7 DC Optimization and Management

• Hive 2 to Hive 3 SQL code migration accelerator and Hive 3 and Spark code conversion

• HQL and Spark/SQL
  Bitbucket

• Microsoft Azure

• HDP2CDP Converter

• Kerberos and Security Configuration

• Hashmap Data & Cloud Consulting and Migration Services including planning, architecture, design, migration build, testing, and delivery