Job Description:
Minimum 7+ years of Informatica experience with 3 years of Hands on experience with Informatica , Hadoop Programming(Sqoop, Hive, Spark) and Big data technologies.
• Performs analysis, design and development of ETL processes to support project requirements.
• Should have strong knowledge and hands-on few of HDFS, Map reduce, Hive, Impala, Tej, Sqoop, Pig, Oozie, HBase, Cassandra, Mongo DB, Kafka, Spark, Storm, Knox, Ranger, Flume, Nifi, Falcon, Kerberos, Sentry, Cloudera Manager, Cloudera Navigator.
• Experience working with Java and Scala are mandatory.
• Experience with data ingestion frameworks covering variety of data source types (RDBMS, Files, API calls, Web Services / Message Queues) and nature of data (structured, semi-structured and unstructured).
• Should be able to write Map-Reduce programs.
• Familiarity with data loading tools like Flume, Sqoop.
• Hands on experience in Hive DDL and Hive QL.
• Develop Sqoop scripts to extract data to/from RDBMS to Hadoop.
• Develop Hive tables and queries.
• Develop Spark jobs in (Scala/Python/Java) in order to stream / publish or consume data from Hadoop.
• Performs unit testing, QA, and work with business partners to resolve any issues discovered during UAT.
• Responsible for peer-review of mappings and workflows when required.
• Maintains development and test data environments by populating the data based on project requirements.
• Works with production control and operations as needed to promote mappings/workflows, implement schedules and resolve the issues.
• Reviews ETL performance and conducts performance tuning as required on mappings / workflows or SQL.
• Good knowledge of database structures, theories, principles, and practices.
• Should be familiar with Serialization and de-serialization in Hadoop.
• Familiarity with Cloud is desirable and knowledge of Machine Learning/Analytics is good to have.
• Should be familiar in scheduling Map-Reduce jobs using OOZIE and other enterprise schedulers.
• Familiarity with AGILE development process is desired.