Looking for a senior Data Engineer with extensive knowledge and expertise in “big data” technologies and frameworks that works with Python/Scala/Java.
Experienced in writing spark code with Java or python or Scala, Sql queries, Linux/Unix scripting, deploying and maintaining databases.
? Microsoft Certified: Azure Data Engineer Associate
? Knowledge of HDFS, ADLS and S3(Simple Storage service).
? Query tuning, server optimization
? Technical expertise related to data.
? Expert in writing Spark code with Python or Scala.
? Expert in writing SQL queries.
? Specialist in data coming from a database management system.
? Technical expertise on building ETL data pipeline, extraction from different source like Azure storage, HDFS, Kafka topics, structured and unstructured files, Hive.
? A Data Engineer understands how to apply technologies to solve big data problems and to develop innovative big data solutions.
In order to be able to do this, the Data Engineer should have extensive knowledge in different programming or scripting languages like Java, Python, Scala and Shell scripting.
? Good understanding in streaming (Kafka/Storm/Kinesis)
? Good understanding on Big data components (HDFS, YARN, Map Reduce, Spark, Oozie)
? Good understanding on Azure components (ADF, ADB, ADLS)
? Good understanding Version controlling (Git, GitHub, azure DevOps).
Skills:
- Big Data
- Java
- Linux
- Python
- Scala
- SQL
