This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.
← Back to Open Jobs
Lead Data Engineer | Job ID: Job #1855
- Lead Data Engineer
- Summary:
- Build High level technical design both for Streaming and batch processing systems
- Design and build reusable components, frameworks and libraries at scale to support analytics data products
- Perform POCs on new technology, architecture patterns
- Design and implement product features in collaboration with business and Technology stakeholders
- Anticipate, identify, and solve issues concerning data management to improve data quality
- Clean, prepare and optimize data at scale for ingestion and consumption
- Drive the implementation of new data management projects and re-structure of the current data architecture
- Implement complex automated workflows and routines using workflow scheduling tools
- Build continuous integration, test-driven development and production deployment frameworks
- Drive collaborative reviews of design, code, test plans and dataset implementation performed by other data engineers in support of maintaining data engineering standards
- Analyze and profile data for the purpose of designing scalable solutions
- Troubleshoot complex data issues and perform root cause analysis to proactively resolve product and operational issues
- Lead, Mentor and develop offshore Data Engineers in adopting best practices and deliver data products.
- Partner closely with product management to understand business requirements, breakdown Epics,
- Partner with Engineering Managers to define technology roadmaps, align on design, architecture, and enterprise strategy
Requirements:
- Minimum of 8+ years experience with the following:
- Snowflake (Columnar MPP Cloud data warehouse)
- DBT (ETL tool)
- Python
- Experience designing and implementing Data Warehouse
Preferred Skills:
- Azure/AWS cloud technology
- SQL objects (procedures, triggers, views, functions) in SQL Server. SQL query optimizations
- Understanding of T-SQL, indexes, stored procedures, triggers, functions, views, etc.
- Design and development of Azure/AWS Data Factory Pipelines preferred.
- Design and development of data marts in Snowflake preferred
- Working knowledge of Azure/AWS Architecture, Data Lake, Data Factory
- Business analysis experience to analyze data to write code and drive solutions
- Knowledge of: Git, Azure DevOps, Agile, Jira and Confluence
Independence/ Accountability:
- Requires minimal daily supervision
- Receives detailed instruction on new assignments and determines next steps with guidance
- Regularly reviews goals and objectives with supervisor
- Demonstrates competence in relevant job responsibilities which allows for increasing level of independence
- Ability to manage and prioritize multiple tasks
- Ability to work under pressure and meet deadlines
Problem Solving:
- Makes logical suggestions of likely causes of problems and independently suggests solutions
- Excellent organizational skills are required to prioritize responsibilities, thus completing work in a timely fashion
- Outstanding ability to multiplex tasks as required
- Excellent project management and/or business analysis skills.
- Attention to detail and concern for impact is essential