Job Description:
- CDL Support Engineer
- Linux expertise
- Strong Shell scripting skills
- Experience with Hive, Python, and Hadoop
- Experience with GCP (Google Cloud Platform), including BigQuery and SQL
- Excellent understanding of Hadoop architecture and components such as HDFS, YARN, High Availability, and the MapReduce paradigm
- Expertise in setting up processes for Hadoop based application design and implementation
- Strong experience with GCP services including BigQuery and DataProc
- Experience tuning Spark jobs handling heavy data loads with optimized resource allocation
- Strong experience with AWS big data services such as EMR, Redshift, Glue, Lambda, Athena, and S3 for cost effective, optimized applications
- Strong proficiency in developing and automating ETL processes using Python, Shell scripting, and PySpark