Data Analytics Engineer
Data Analytics Engineer
Primary Objective of Position:
As a member of the technology team, you will work closely with our business counterparts, software engineers, Solutions Architect, and CRM Admin to rapidly design, secure, build, test, and release data analytics capabilities. You will use your technical, organizational and leadership skills to enhance the data pipeline, make it “future proof” by driving scalability and resiliency while respecting architectural guidelines.
Primary Duties and Responsibilities:
- Responsible for the engineering data pipelines for enterprise data platform, integrating data from systems and applications, and enabling analytics capabilities.
- Continually transform, aggregate and clean data and provide rapid access for multiple software applications
- Report on maintenance, monitoring, performance, and problem resolution of all ETL processes
- Identify data discrepancies and data quality issues and work to ensure data consistency and integrity
- Optimize performance and create automated self-testing of ETL processes
- Support the organization in their use of analytics reporting and visualization tools
- Model the data architecture and data structures to meet the organizations analytics needs
- Coordinate with all levels of organization to design and deliver technical solutions to business problems.
- Survey markets for industry and technology trends and opportunities that may support our client's goals and objectives.
- Keep management informed of important developments, risk areas, potential problems, and related information necessary for decision making.
- Perform related work as apparent or assigned.
Experience and Qualifications:
- Experience with AWS Data Lake methodologies, SQL, data integration & data modeling
- Experience in AWS Cloud technologies: S3, CloudFormation, Glue, Athena, Redshift, DMS, Appflow, RDS, Quicksight.
- Strong experience in working with heterogeneous datasets in building and optimizing data pipelines, pipeline architectures and integrated datasets using various data integration technologies: ETL, data replication/CDC, etc
- Pipeline management with AWS Step functions or Airflow
- Experience with data analytics reporting and visualization tools
- Experience with scripting languages, preferable Python with Pyspark and Pyspark.sql
- Experience with one of notebooks: jupyter, sagemaker notebook, zeppelin)
- Experience with Agile software development methodology
- Experience with git/github
- Experience of relational and NoSQL databases. MongoDB is preferred.
- Experience with EMR or Databricks is preferred
- Experience with Delta Lake is preferred
- Knowledge of Salesforce data is preferred
- Self-starter with a record of success.
- Strong collaboration and teambuilding skills.
- Strong organizational and planning skills.
- Ability to influence without authority
- Ability to cope with the rapid pace and constant change associated with the industry.
- Ability to successfully manage numerous projects simultaneously.
- Ability to communicate effectively, both orally and in writing with personnel and outside contacts.