Design and maintain web scrapers for alternative datasets, clean and transform large data sets using Python and Pandas, and ingest data into databases or warehouses. Build scheduled workflows with Airflow, create quality-control checks, maintain alerting systems, investigate data incidents, and develop tools for the web-scraping platform. Collaborate with analysts and cross-functional teams to deliver reliable datasets and data products.
This is a remote position.
Alternative Path is seeking skilled software developers to collaborate on client projects with an asset management firm. In this role, you will collaborate with individuals across various company departments, to shape and innovate new products and features for our platform, enhancing existing ones. You will have a large degree of independence and trust, but you won't be isolated; the support of the Engineering team leads, the Product team leads, and every other technology team member is behind you. This is an opportunity to join a team-first meritocracy and help grow an entrepreneurial group inside Alternative Path. You will be asked to contribute, given ownership, and will be expected to make your voice heard.
Role Summary
Performing Web Scraping using various scraping techniques and then utilizing Python’s Pandas library for data cleaning and manipulation. Then ingesting the data into a Database/Warehouse, and scheduling the scrapers using Airflow or other tools.
Requirements
Role Overview
The Web Scraping Team at Alternative Path is seeking a creative and detail-oriented developer to contribute to client projects. The team develops essential applications, datasets, and alerts for various teams within the client's organization, supporting their daily investment decisions. The mission is to maintain operational excellence by delivering high-quality proprietary datasets, timely notifications, and exceptional service. We are seeking someone who is self-motivated, self-sufficient, with a passion for tinkering and a love for automation.
In this role, you will,
- Collaborate with analysts to understand and anticipate requirements.
- Design, implement, and maintain Web scrapers for a wide variety of alternative datasets.
- Perform data cleaning, exploration, transformation, etc., on scraped data.
- Collaborate with cross-functional teams to understand data requirements and implement efficient data processing workflows.
- Author QC checks to validate data availability and integrity.
- Maintain alerting systems and investigate time-sensitive data incidents to ensure smooth day-to-day operations.
- Design and implement products and tools to enhance the Web scraping Platform.
Must Have
- Bachelor's/master’s degree in computer science or in any related field.
- 2-4 years of software development experience
- Strong Python and SQL/Database skills
- Strong expertise in using the Pandas library (Python) is a must
- Experience with web technologies (HTML/JS, APIs, etc.)Proven work experience in working with large data sets for Data cleaning, Data transformation, Data manipulation, and Data replacements.
- Excellent verbal and written communication skills
- Aptitude for designing infrastructure, data products, and tools for Data Scientists
Preferred - Familiarity with scraping and common scraping tools (Selenium, Scrappy, Fiddler, Postman, XPath)
- Experience containerizing workloads with Docker (Kubernetes a plus)
- Experience with build automation (Jenkins, GitLab CI/CD)
- Experience with AWS technologies like S3, RDS, SNS, SQS, Lambda, etc.
Similar Jobs
Information Technology • Consulting
Design, maintain, and improve web scrapers for alternative datasets; clean, transform, validate, and load data into databases or warehouses; orchestrate pipelines with Airflow; and build quality checks, alerts, and data tools. Collaborate with analysts and cross-functional teams to define data requirements, investigate time-sensitive incidents, and support investment decision-making through reliable datasets and notifications.
Top Skills:
Apache AirflowAPIsAws LambdaAws RdsAws S3Aws SnsAws SqsCi/CdDockerFiddlerGithub ActionsHTMLJavaScriptJenkinsKubernetesPandasPostmanPythonScrapySeleniumSQLXpath
Artificial Intelligence • Fintech • Hardware • Information Technology • Sales • Software • Transportation
Manage end-to-end digital marketing campaigns and executions including builds, audience segmentation, testing, deployment, and optimization. Administer marketing automation stack, support integrations, track KPIs with channel managers, and improve lead lifecycle processes, nurturing, data hygiene, and workflow automation. Maintain documentation, templates, and share campaign best practices across stakeholders.
Top Skills:
6SenseAi Automation ToolsMarketoPendoSalesforceZuddl
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Support regulatory liquidity reporting by applying predefined rules and mappings, performing variance analysis and reconciliations, documenting business/functional requirements, coordinating BA/IT/testing streams, and contributing to governance, controls and audit readiness within regulatory change programmes.
What you need to know about the Kolkata Tech Scene
When considering the industries shaping India's tech scene, gaming might not immediately come to mind. However, in the last decade, increased internet usage and greater access to mobile devices have catapulted the industry to new heights, with Kolkata-based companies like Virtualinfocom, Red Apple Technologies and Digitoonz, at the forefront, driving the design and animation of new gaming titles for players.
