Staff Data Engineer (SQL, Hadoop, Big Data, ETL)
Company Description
Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable, and secure payments network, enabling individuals, businesses, and economies to thrive while driven by a common purpose – to uplift everyone, everywhere by being the best way to pay and be paid.
Make an impact with a purpose-driven industry leader. Join us today and experience #LifeAtVisa
Job Description
- Hands-on SQL Programming
- Build and maintain high performing ETL processes, including data quality and testing aligned across technology, internal reporting and other functional teams.
- Execute and manage large scale ETL processes to support development and publishing of reports, Datamart’s and predictive models.
- Build ETL pipelines in Spark, Python, HIVE or SAS that process transaction and account level data and standardize data fields across various data sources
- Collaborate with Data Engineering teams in North America to production and maintenance of key data assets.
- Should have strong problem-solving capabilities and ability to quickly propose feasible solutions and effectively communicate strategy and risk mitigation approaches to leadership
- Create data dictionaries, setup/monitor data validation alerts and execute periodic jobs like performance dashboards, predictive models scoring for client’s deliverables
- Works independently with minimal insight to develop software that conforms to Visa’s high standards of security, quality, performance, resiliency, and compliance.
- Design, develop, and unit test applications in accordance with established standards.
- Participate in design and code review sessions as appropriate.
- Keep up with current and developing engineering trends.
- Work with product owners, systems analysts, project managers and other developers to efficiently implement business requirements while applying the latest available tools and technology.
- Work with managers and clients to fully understand business requirements and desired business outcomes.
- Build and maintain a robust data engineering process to develop and implement self-serve data and tools for Front end team.
- Find opportunities to create, automate and scale repeatable analyses or build self-service tools for business users.
- Execute data engineering projects ranging from small to large either individually or as part of a project team.
- Provide coaching and mentoring to junior team members.
This is a hybrid position. Hybrid employees can alternate time between both remote and office. Employees in hybrid roles are expected to work from the office 2-3 set days a week (determined by leadership/site), with a general guidepost of being in the office 50% or more of the time based on business needs.
Qualifications
Basic Qualifications:
• 6+ years of work experience with an B.S/M. S/Ph.D. in an engineering/quantitative field such as Mathematics, Computer Science, Statistics, Artificial Intelligence, Machine Learning, etc.
Preferred Qualifications
• Minimum of 4 years of software development experience (with a concentration in data centric initiatives), with demonstrated expertise in leveraging standard development best practice methodologies.
• Minimum of 4 years of experience in building large-scale applications using open source technologies. Design and coding skills with Big Data technologies like Hadoop, Spark, Hive, and Map Reduce
• Experience with Big Data and Real Time platforms in general, leveraging technologies like Hadoop, and Spark, Tuber etc.
• Experience with highly distributed, scalable, concurrent and low latency systems working with one or more of the following database technologies: DB2, MySQL
• Experience working in an Agile and Test-Driven Development environment.
• Experience with Continuous Integration and Automated Test tools such as Jenkins, Artifactory, Git, Selenium, Chef desirable.
• A fast learner with strong sense of ownership.
• Strong interpersonal and leadership skills with effective communication (both written and verbal) skills and the ability to present sophisticated ideas concisely.
· A background in the payment / fintech domain is a plus.
• 6+ years of work experience with an B.S/M. S/Ph.D. in an engineering/quantitative field such as Mathematics, Computer Science, Statistics, Artificial Intelligence, Machine Learning, etc.
Preferred Qualifications
• Minimum of 4 years of software development experience (with a concentration in data centric initiatives), with demonstrated expertise in leveraging standard development best practice methodologies.
• Minimum of 4 years of experience in building large-scale applications using open source technologies. Design and coding skills with Big Data technologies like Hadoop, Spark, Hive, and Map Reduce
• Experience with Big Data and Real Time platforms in general, leveraging technologies like Hadoop, and Spark, Tuber etc.
• Experience with highly distributed, scalable, concurrent and low latency systems working with one or more of the following database technologies: DB2, MySQL
• Experience working in an Agile and Test-Driven Development environment.
• Experience with Continuous Integration and Automated Test tools such as Jenkins, Artifactory, Git, Selenium, Chef desirable.
• A fast learner with strong sense of ownership.
• Strong interpersonal and leadership skills with effective communication (both written and verbal) skills and the ability to present sophisticated ideas concisely.
· A background in the payment / fintech domain is a plus.
Additional Information
Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.
Related Jobs
Replicant
Senior Software Engineer II, ML team
- Engineering
- Full Time
- architecture
- chatbots
- chatgpt
- conversational ai
- engineering
- gpt
- llms
- machine learning
- nlg
- python
Octopus Energy
Data Engineer (m/w/d)
- Engineering
- Full Time
- aws
- data pipelines
- pipelines
- python
- spark
- sql
Hostinger
AI Engineer
- Engineering
- Full Time
- apis
- azure
- devops
- docker
- engineering
- generative ai
- git
- jira
- langchain
- linux
Land your dream job
Get a weekly email with the latest startup jobs.