AWS Data Engineer at Vaco LLC
South San Francisco, CA 94080
About the Job
One of our client's is adding a fully remote Data Engineer (AWS) to an existing team of Engineers supporting a data lake (RDL - Renewables Data Lake). They are supporting an AWS stack as listed below which is the platform that houses all the data and intelligence collected from all their sites in North America for Wind, Solar and Storage. This is a 9-month contract with the possibility to extend.
Scope of Work
The Renewable Data Lake (RDL) is focused on building renewable asset optimization solutions around the world and transformingthe data into meaningful metrics for our customers (internal stakeholders).
Our team ingests massive volumes of data and generates a series of KPIs that help us and our customers understand how our sites are performing, where and why they are under-performing, what the most common problems are, etc.
The number of sites and the volume of data is growing fast. We'll need your help building smart, reliable, and performant solutions to make the most of the data.
Our current stack includes Python Redshift, RDS, Glue, PySpark, Lambdas, S3, the AWS developer tools, and other AWS technologies.
If you have good ideas for alternative solutions, we want to hear about it!
Responsibilities
- 60%: Design, develop, and maintain data applications and solutions to enable efficient data processing, analysis, and visualization
- 5%: Collaborate with data scientists, analysts, and business stakeholders to understand requirements and translate them into scalable and efficient data solutions
- 5%: Optimize and tune data applications for performance and scalability
- 5%: Implement data quality checks and ensure data integrity and consistency
- 5%: Lead projects and initiatives, providing technical guidance and ensuring successful delivery
- 5%: Manage project timelines, resources, and deliverables to meet business objectives
- 5%: Mentor and coach junior team members, fostering their growth and development
- 10% - Other Duties as Assigned
Education/Experience
- 10+ years of experience working in software engineering
- 6+ years working with Big Data in a cloud environment, AWS strongly preferred
- Bachelor’s degree in Computer Science or other related technical field or relevant experience
- Experience developing multi-platform software utilizing various programmingenvironments and tools
- Knowledge of agile methodologies
- Knowledge of QA processes and test-driven development
- Strong experience with database design, specifically geared towards Big Data
- Familiar with serverless approach, micro-service-oriented architecture, event-driven architecture
- Strong experience working in a AWS cloud technology environment: Lambda functions, Step functions, SQS, DynamoDB, Spark with Glue, along with EMR, EMR serverless, Spark streaming
- Experience building and maintaining critical, reliable ETL pipelines
- Experience writing high-quality, production-ready code, preferably in Python
- Comfortable with Git (AWS CodeCommit) and Unix command lines
- DevOps experience: strive for scalability and automation via CI/CD and Infrastructure as Code. A plus if knowledge of CDK
- Proactive communicator who can translate between technical and non-technical stakeholders
- Self-starter and autonomous, strong bias for action in high ambiguity environment
- Explore and experiment with the limits of cloud technologies
- Help mentor and educate your teammates (and vice versa) to expand everyone's skillsets
- Displays personal and corporate integrity
Vaco by Highspring values a diverse workplace and strongly encourages women, people of color, LGBTQ+ individuals, people with disabilities, members of ethnic minorities, foreign-born residents, and veterans to apply.
EEO Notice
Vaco by Highspring is an Equal Opportunity Employer and does not discriminate against any employee or applicant for employment because of race (including but not limited to traits historically associated with race such as hair texture and hair style), color, sex (includes pregnancy or related conditions), religion or creed, national origin, citizenship, age, disability, status as a veteran, union membership, ethnicity, gender, gender identity, gender expression, sexual orientation, marital status, political affiliation, or any other protected characteristics as required by federal, state or local law.
Vaco by Highspring and its parents, affiliates, and subsidiaries are committed to the full inclusion of all qualified individuals. As part of this commitment, Vaco by Highspring and its parents, affiliates, and subsidiaries will ensure that persons with disabilities are provided reasonable accommodations. If reasonable accommodation is needed to participate in the job application or interview process, to perform essential job functions, and/or to receive other benefits and privileges of employment, please contact HR@vaco.com .
Vaco by Highspring also wants all applicants to know their rights that workplace discrimination is illegal.
By submitting to this position, you agree that you will be giving Vaco by Highspring the exclusive right to present your as a candidate for the foregoing employment opportunity. You further agree that you have represented information about yourself accurately and have not affirmatively misrepresented your qualifications. You also agree to maintain as confidential, to the fullest extent permitted by law, any information you learn from Vaco by Highspring about the position and you will limit disclosure of information about the position only to the extent necessary to perform any obligations in furtherance of your application. In exchange, Vaco by Highspring agrees to exercise reasonable efforts to represent you through all solicitation, job screening and resume dispersal.
Privacy Notice
Vaco by Highspring and its parents, affiliates, and subsidiaries (“we,” “our,” or “Vaco by Highspring”) respects your privacy and are committed to providing transparent notice of our policies.
- California residentsmay access Vaco by Highspring HR Notice at Collection for California Applicants and Employees here.
- Virginia residents may access our state specific policies here.
- Residents of all other states may access our policies here.
- Canadian residents may access our policies in English here and in French here.
- Residents of countries governed by GDPR may access our policies here.
Pay Transparency Notice
Determining compensation for this role (and others) at Vaco by Highspring depends upon a wide array of factors including but not limited to:
- the individual’s skill sets, experience and training;
- licensure and certification requirements;
- office location and other geographic considerations;
- other business and organizational needs.
With that said, as required by local law, Vaco by Highspring believes that the following salary range referenced above reasonably estimates the base compensation for an individual hired into this position in geographies that require salary range disclosure. The individual may also be eligible for discretionary bonuses.