About

More about me and myself

avatar

Hi I'm Vijay Anand Pandian

A brief timeline of my career:

  • May 2022 - Present
    Senior Data Engineer at Channel4 - London, United Kingdom
    Key-works:
    - Built batch load process from AWS S3 to Redshift (star-schema), Spectrum (external tables) using lambda, glue
    - Built generic monitoring and alerting tools in cloud services
  • May 2021 - April 2022
    Senior Data Engineer at Eli Lilly and Company - Bengaluru, India
    Key-works:
    - Re-architected legacy queue application with cutting edge technologies using IBM MQ, AWS - MQ,Lambda,DynamoDB,Fargate.
    - Built Data pipelines using Azure Databricks to push data to Data Lake.
  • Jul 2019 - April 2021
    Data Engineer at United Health Group - Optum - Chennai, India
    Key-works:
    - Migrated the data from Oracle DB to AWS S3 using Spark, Scala and EMR.
    - Automated the data quality checks. Converted complex Oracle SQL to Spark SQL.
    - Built ETL data pipeline to extract data from Azure Blob to AWS S3 using rclone and pulsar.Used Airflow for workflow management and Terraform for cluster creation.
  • Sep 2015 - Jul 2019
    Data Analytics Engineer at Gartner - Chennai, India
    Key-works:
    - Built Email Bot product that extracts data from outlook mail box and generates sales leads and missing clients info based on auto reply emails using python and MongoDB
    - Built data pipelines for a marketing and insights team to extract google analytics data from GCP BigQuery to AWS S3.
    - Designed competitor analytics dashboard for events organizing team to understand the competitor past and future event details using python, Elastic Search and Kibana
    - Migrated data from Oracle to Datalake using Sqoop. Orchestrated workflow with Apache Airflow. Initiated CICD pipeline in AWS to deploy machine learning models
  • Jul 2014 - Aug 2015
    Associate SQA Engineer at Symantec - Chennai, India
    Key-works:
    - Built automation framework using Python for Norton Antivirus for Mac product to test the various security scenarios.
    - Contributed and maintained the common libraries used for different team usages.
  • Jan 2014 - Jun 2014
    Masters Internship at Symantec - Chennai, India
    Key-works:
    - Worked in Managed Cyber Security Services (SIEM) Team
  • Aug 2012 - Jun 2014
    Masters in Cyber Security at Amrita Vishwa Vidhyapeetham - Coimbatore, India
  • Aug 2008 - Jun 2012
    Bachelors in Information Technology at University College of Engineering - Tindivanam, India
  • Jun 2006 - Mar 2008
    High School at Veveaham Higher Secondary School - Dharapuram, India
  • Before 2006
    Grown up in Karur District

See more about my working experiences at my resume page.

Why have this blog?

Because sharing is learning

I believe noting down and sharing is the best way to learn! So I created this blog, where I keep things that I learned, find useful, and necessary to save for later.

More than just technical stuff, there are personal thoughts about work and life from the perspective of a Software Engineer.

It would be highly appreciated if I could read your comment or hear your thoughts on what I wrote

Tech stack

This blog is hosted on Vercel, built with Next.js (Preact for production) and Tailwind CSS using Tailwind Nextjs Starter Blog.

A huge thanks to Timothy Lin for the minimal, lightweight, and super easy-to-customize blog starter.

A hearty 💖 Thanks to Tuan Anh Huynh

Images

All images in this blog are from Unsplash, imgur and GIPHY - Thanks for the free resources!