DataBricks Architect Job at Abidi Group LLC, Plainsboro, NJ

TUV0RlV1YXhaY2tuaGFyaDlFSnhsY0dsZUE9PQ==
  • Abidi Group LLC
  • Plainsboro, NJ

Job Description

Job Description

Job Description

Technical Expertise:

  • Data bricks Mastery: Deep, expert-level knowledge of the Data bricks Platform, including:
    • Unity Catalog: Designing and implementing data governance and security.
    • Delta Lake & Delta Live Tables: Architecture and building reliable, scalable data pipelines.
    • Performance & Cost Optimization: Expertise in tuning Spark jobs, optimizing cluster usage, and managing platform costs.
    • MLOps: Strong, practical understanding of the machine learning lifecycle on Data bricks using tools like MLflow.
    • Data bricks SQL: Knowledge of designing and optimizing analytical workloads.
    • Mosaic AI: Knowledge of designing and optimizing AI Agents.
  • Cloud & Infrastructure: Deep knowledge of cloud architecture and services on AWS. Strong command of Infrastructure as Code (Terraform, YAML).
  • Data Engineering & Programming: Strong background in data modeling, ETL/ELT development, and advanced, hands-on programming skills in Python and SQL.
  • CI/CD & Automation: Experience with designing and implementing CI/CD pipelines (preferably with GitHub Actions) for data and ML workloads.
  • Severability: Familiarity with implementing monitoring, logging, and alerting for data platforms.
  • Automation: The platform is ephemeral, and all changes are implemented using Terraform and Python. Expertise in Terraform and Python is a must.

Job Tags

Similar Jobs

Confidential

Vice President, Regulatory Science Communications Job at Confidential

 ...Vice President, Regulatory Science Communications About the Company Accomplished biopharmaceutical company Industry Biotechnology Type Public Company About the Role The Company is seeking a Vice President for Regulatory Science Communications to... 

UPMC Kane

Ophthalmology - General Job at UPMC Kane

 ...Details Client Name UPMC Kane Job Type Permanent Offering Physician Profession Ophthalmology Specialty General Job ID 15369279 Shift Details Shift Full Time Days Scheduled Hours... 

Providence Health and Services

Senior Pharmacy Technician Job at Providence Health and Services

 ...Senior Pharmacy Technician at Providence Health and Services summary: The Senior Pharmacy Technician is responsible for accurately filling...  ...and will report onsite daily as scheduled Flexibility to work a rotating schedule, including weekends & holidays as scheduled... 

annual National Military Fish and Wildlife Association meeti...

Field GIS Analyst: Army Geospatial Compliance & QA Job at annual National Military Fish and Wildlife Association meeti...

 ...An environmental management organization is seeking a contractor to provide GIS support and field verification. The ideal candidate has a relevant degree and at least three years of experience in GIS and GPS technologies. Job responsibilities include supporting geospatial... 

Pyramid Consulting, Inc

Data Analyst Job at Pyramid Consulting, Inc

 ...Job Description Immediate need for a talented Data Analyst. This is a 06+months contract opportunity with long-term potential and is located in Dallas TX (Hybrid). Please review the job description below and contact me ASAP if you are interested. Job ID:...