BIG DATA DEVELOPER

Job Summary

Company
Cognizant Technology Solutions Asia Pacific Pte. Ltd
Industry
IT/Computers - Software
Location
Kuala Lumpur
Years of Experience
7 - 17 years
Function
IT
Posted On
12th Feb 2021
Skills
Bigdata,Hadoop

Job Description:

Minimum 7+ years of Informatica experience with 3 years of Hands on experience with Informatica , Hadoop Programming(Sqoop, Hive, Spark) and Big data technologies.

• Performs analysis, design and development of ETL processes to support project requirements.

• Should have strong knowledge and hands-on few of HDFS, Map reduce, Hive, Impala, Tej, Sqoop, Pig, Oozie, HBase, Cassandra, Mongo DB, Kafka, Spark, Storm, Knox, Ranger, Flume, Nifi, Falcon, Kerberos, Sentry, Cloudera Manager, Cloudera Navigator.

• Experience working with Java and Scala are mandatory.

• Experience with data ingestion frameworks covering variety of data source types (RDBMS, Files, API calls, Web Services / Message Queues) and nature of data (structured, semi-structured and unstructured).

• Should be able to write Map-Reduce programs.

• Familiarity with data loading tools like Flume, Sqoop.

• Hands on experience in Hive DDL and Hive QL.

• Develop Sqoop scripts to extract data to/from RDBMS to Hadoop.

• Develop Hive tables and queries.

• Develop Spark jobs in (Scala/Python/Java) in order to stream / publish or consume data from Hadoop.

• Performs unit testing, QA, and work with business partners to resolve any issues discovered during UAT.

• Responsible for peer-review of mappings and workflows when required.

• Maintains development and test data environments by populating the data based on project requirements.

• Works with production control and operations as needed to promote mappings/workflows, implement schedules and resolve the issues.

• Reviews ETL performance and conducts performance tuning as required on mappings / workflows or SQL.

• Good knowledge of database structures, theories, principles, and practices.

• Should be familiar with Serialization and de-serialization in Hadoop.

• Familiarity with Cloud is desirable and knowledge of Machine Learning/Analytics is good to have.

• Should be familiar in scheduling Map-Reduce jobs using OOZIE and other enterprise schedulers.

• Familiarity with AGILE development process is desired.

APPLY NOW
 
 

About Us:

Cognizant is a global leader in business and technology services that helps Clients bring the future of work to life – today – in a business environment that is being transformed by accelerating globalization, virtualization, and the shift toward cloud technologies.

We continuously seek the “best and the brightest” when we recruit new employees, and we’re consistently an “employer of choice” at leading engineering and business schools around the world. Our domain industry specialists have well-established backgrounds working with companies in the industries we serve.

Our teams and their talent are a principle reason that we consistently outperform the competition.


Disclaimer: