Senior Data Engineer (Python+Pyspark+Palantir)

Job Overview

Location
Gurgaon, Haryana
Job Type
Full Time
Date Posted
2 years ago

Additional Details

Job ID
537
Job Views
351
Work from home ?
No
Educational Qualification
Graduation
Educational Major / Department
BE / B.Tech / M.Tech / MCA / MSc (IT) or equivalent

Job Description

Major accountabilities:

  • Responsible for Data Engineering, Foundry Data Pipeline Creation, Foundry Analysis & Reporting, Slate Application development, re-usable code development & management and Integrating Internal or External System with Foundry for data ingestion with high quality.
  • Have good understanding on Foundry Platform landscape and it’s capabilities
  • Performs data analysis required to troubleshoot data related issues and assist in the resolution of data issues.
  • Defines company data assets (data models), Pyspark, spark SQL, jobs to populate data models.
  • Designs data integrations and data quality framework.
  • Design & Implement integration with Internal, External Systems, F1 AWS platform using Foundry Data Connector or Magritte Agent
  • Collaboration with data scientists, data analyst and technology teams to document and leverage their understanding of the Foundry integration with different data sources - Actively participate in agile work practices.
  • Coordinating with Quality Engineer to ensure all quality controls, naming convention & best practices have been followed.


Desired Candidate Profile:

  • Strong data engineering background
  • Experience with Clinical Data Model is preferred Experience in SQL Server, Postgres, Cassandra, Hadoop, and Spark for distributed data storage and parallel computing.
  • Java and Groovy for our back-end applications and data integration tools
  • Python for data processing and analysis
  • Cloud infrastructure based on AWS EC2 and S3


  • 7+ years IT experience, 2+ years’ experience in Palantir Foundry Platform, 4+ years’ experience in Big Data platform
  • 5+ years of Python and Pyspark development experience.
  • Strong troubleshooting and problem solving skills
  • BTech or master's degree in computer science or a related technical field
  • Experience designing, building, and maintaining big data pipelines systems
  • Hands-on experience on Palantir Foundry Platform and Foundry custom Apps development
  • Able to design and implement data integration between Palantir Foundry and external Apps based on Foundry data connector framework
  • Hands-on in programming languages primarily Python, R, Java, Unix shell scripts
  • Hand-on experience in AWS / Azure cloud platform and stack
  • Strong in API based architecture and concept, able to do quick PoC using API integration and development.
  • Knowledge of machine learning and AI
  • Skill and comfort working in a rapidly changing environment with dynamic objectives and interation with users.
  • Demonstrated ability to continuously learn, work independently, and make decisions with minimal supervision.

Location

Similar Jobs

Full Time

Kissflow

React Developer

Full Time

Cookies

This website uses cookies to ensure you get the best experience on our website.

Accept