Job Description

As a Python Developer, your role is to apply your knowledge set to fetch data from multiple online sources, cleanse it and build APIs on top of it.

• Develop a deep understanding of our vast data sources on the web and know exactly how, when, and which data to scrape, parse and store this data through API.
• Work on to store data in SQL and NoSQL database.
• Work independently with little supervision to research and test innovative solutions.
• Should build and lead the web extraction team.
• Complete Data Extraction/Collection within the committed timeline.
• Will be responsible for tool selection, design, setting up the processes, and ensuring quality of the data provided by the web extraction group.
• Developing highly reliable web crawlers and parsers across various websites.
• Extract structured unstructured data and store them into SQLNo SQL datastore.

Skills and Qualification:

• Very good understanding of Object Orientation principles, design patterns and analytical thinking, data structures, and algorithms.
• Strong coding experience in Python (experience of PHP MVC framework, Javascript is a must).
• Should have a good knowledge of Regular expression, File Handling.
• Experience with SQL and NoSQL databases.
• Experience with multi-processing, multi-threading, and AWS/Azure/Linux/Windows server.
• Strong knowledge of scraping frameworks such as Python with (Request, Scrapy, spiders), and HTML.
• Strong Knowledge of PHP MVC framework - Codeigniter, CakePHP with oops concept.
• Strong experience in server handling like Linux/azure/AWS.
• Experience of svn tools - git/Jira/bitbucket.
• Good in verbal and written communication.
• Experience with data parsing - Strong knowledge of Regular expression, HTML, CSS, DOM, XPATH. Knowledge of Javascript would be a plus.
• Must be able to develop reusable code-based scraper products that can be used by others.