Resume - Vijay Anand Pandian (Vijay)

Summary

Who you are?

Hi, my name is Vijay Anand Pandian, but you can call me Veejaai ;-)

What you do?

I'm having 9+ years of IT experience in solving data problems and building Big Data pipelines, ML pipelines and deploying Machine Learning models in Azure, AWS and GCP.

Areas of Expertise

Domain Sales, Marketing, Product Analytics, Cyber Security

Interests Big Data pipelines, DevOps, MLOPS

Programming Python, Scala, Java

Bigdata Hadoop, Spark, Hive, Databricks

SQL Postgres, Oracle, MS-SQL, MongoDB, Elasticsearch

Data Ware House Snowflake, Redshift, Hive, BigQuery

Web Django, Flask, React

Schedulers Apache Airflow, cron, Windows Scheduler

Cloud Infra AWS, GCP, Azure

CI/CD, Tools & Documentations Git, Jenkins, Bamboo, Terraform, Agile (Jira, Rally), Splunk, Sonar, Confluence

CERTIFICATIONS

Apache Airflow Fundamentals
Apache Airflow DAG Authoring

Working Experiences

Data Engineer| Sky UK

Dec 2022 – Current | London, United Kingdom

AWS S3 (Parquet) files batch load to spectrum tables and create a star-schema tables and views top of it.
Smart solutions on cloud monitoring

Data Engineer| Channel4

May 2022 – Dec 2022 | London, United Kingdom

AWS S3 (Parquet) files batch load to spectrum tables and create a star-schema tables and views top of it.
Smart solutions on cloud monitoring (martech, advertisement, targetted audience)

Data Engineer| Eli Lilly and Company

May 2021 – April 2022 | Bangalore, India

Re-architected legacy queue application with cutting edge technologies using IBM MQ, AWS - MQ,Lambda,DynamoDB,Fargate.
Built Data pipelines using Azure Databricks to push data to Data Lake.

Software Engineer | Data Engineer | United Health Group ‑ Optum

July 2019 – April 2021 | Chennai, India

Worked as a full stack developer. Develop and enhance the product features listed for GA. Worked in backend, forntend and infrastructure tasks.
Migrated the data from Oracle DB to AWS S3 using Spark, Scala and EMR.
Automated the data quality checks. Converted complex Oracle SQL to Spark SQL.
Built ETL data pipeline to extract data from Azure Blob to AWS S3 using rclone and pulsar.Used Airflow for workflow management and Terraform for cluster creation.

Data Analytics Engineer | Gartner

September 2015 ‑ July 2019 | Chennai, India

Built Email Bot product that extracts data from outlook mail box and generates sales leads and missing clients info based on auto reply emails using python and MongoDB
Built data pipelines for a marketing and insights team to extract google analytics data from GCP BigQuery to AWS S3.
Designed competitor analytics dashboard for events organizing team to understand the competitor past and future event details using python, Elastic Search and Kibana
Migrated data from Oracle to Datalake using Sqoop. Orchestrated workflow with Apache Airflow. Initiated CICD pipeline in AWS to deploy machine learning models

Associate Software Engineer | Symantec ‑ Norton

Jan 2014 ‑ Aug 2015 | Chennai, India

Built automation framework using Python for Norton Antivirus for Mac product to test the various security scenarios.
Contributed and maintained the common libraries used for different team usages.

Education

Amrita Vishwa Vidyapeetham Master's in Cyber Security

July 2014 | Coimbatore, TN, India

University College of Engineering Tindivanam Bachelor's in Information Technology

June 2012 | Tindivanam, TN, India

What others need to know?

Studies & Job (How you have been transformed from various past roles to present ?)

I did Bachelors in Information Technology and continued Master's in Cyber Security.

While doing Masters, I started an Internship program with Symantec.
Initially I worked with Managed Security Services Team develops and maintains SIEM tool. There I learned more about data architecture include multiple network devices, linux servers, SQL servers, log parsers, large data flows from network to on-premise servers and application end point. There I was helping team with linux, shell scripting and python skills. I was writing pythons scripts to do a log files transfer from one server to another server based on watchdog events(on create, on delete, ssh). Generated more device based mock logs files for the development and testing.
After 6 months of Internship, I moved to Norton Product Team, builds the Antivirus product for Mac OS and iOS products. There I was developing and contributing to Python automation frameworks for different teams uses like simulating security scenario includes bruteforce password attack, DOS attack, scan virus files, port scans to list open ports and vulnerability scanner to list outdated packages or software in the host machine.
Then I moved to CEB (Corporate Executive Board), later acquired by Gartner. First, I was hired for maintain and support NLP Web Product that predicts the best hire for the job using multiple resumes as input. It was built using Python in backend and Java in Front End, runs on Tomcat on-premise Server. Then I moved data analytics team inside Gartner. There I have started working as individual contributor as well as a team player. My roles includes collaborating with Stake Holders from Sales/Marketing/Events team understand their problem and provide data solutions from initial stage to final deployment including maintaining code repo and performing CI/CD. I have developed products like
- Email Bot (classifies the emails into 4 categories - OOO, Left, Bad, Others and sends weekly and monthly reports. This report helped Sales team to manage maintain contacts status easily. Continues the membership company even the people left the company. - Python(nltk, pandas), MongoDB)
- Competitive Dashboard (Scraps the data of competitor from their websites, stored and creates the dynamic chart to understand more about competitors - Python(Scrapy, requests), Elasticsearch, Kibana)
- Database - Table values cleaner (Lots of survey data withe bad characters and format. Created a machine learning prototype to cleanse the values based on right values. - Python(scikit, pandas))
- Universal Data Scrapper (created a dynamic tool to scrap the data about any company from the popular Websites, news channel and blogs)
- GCP to AWS data migration (Moved BigQuery Datasets to AWS S3 - Python, Pyspark, Shell, API)
- Worked with Data Science Team in building Recommendation Engine, does ETL (From On-premise Oracle Server to Data lake using sqoop, transform data with spark and created tables in Hive)
In Gartner, gained lots of experience in terms data, cloud knowledge and building data end-to-end solutions
Then I moved to ABCO - Optum - United Health Group, I worked in two solid Projects in 1.10 years span
- Initiative Manager Product - Acted a sole Product owner and Full Stack and DevOps Engineer, managed an end to end operations, develop and enhance the product features listed for GA. Worked in backend (Django python), frontend (ReactJs), PostgresSQl, infrastructure tasks (Sonar, Splunk, New relic and Nagios) and CI/CD (Bamboo).
- Data Engineer convert Oracle Queries to Spark queries. Written UDF for the custom procedures/functions available in Oracle. Written spark solution on top of large JSON data and create flat tables using case classes. - AWS EMR, Spark, Scala, Bash, HDFS, Github, Jenkins, Sonar, Terraform
In Optum, learned more about different technologies and stacks, how are they relying on one and another? adopting them while working in Agile is more challenging.
Then worked as Senior Associate Manager at Eli Lilly
- App Modernization - eCSP - Convert the Clinical Supply Platform legacy Application (Fuse and java application) to a cloud based solution (AWS Lambda, Cloudwatch, SNS, Fargate, RabbitMQ, S3, DynamoDB). Developed the Python framework that has pre-commit hooks to maintain the coding standards, unit test cases and code coverages.

Last updated on Feb 2022