About
More about me and myself
Vijay Anand Pandian
Learner | Solver | Builder
Employed - building products
London, United Kingdom
Hi I'm Vijay Anand Pandian
A brief timeline of my career:
- May 2022 - PresentSenior Data Engineer at Channel4 - London, United KingdomKey-works:- Built batch load process from AWS S3 to Redshift (star-schema), Spectrum (external tables) using lambda, glue- Built generic monitoring and alerting tools in cloud services
- May 2021 - April 2022Senior Data Engineer at Eli Lilly and Company - Bengaluru, IndiaKey-works:- Re-architected legacy queue application with cutting edge technologies using IBM MQ, AWS - MQ,Lambda,DynamoDB,Fargate.- Built Data pipelines using Azure Databricks to push data to Data Lake.
- Jul 2019 - April 2021Data Engineer at United Health Group - Optum - Chennai, IndiaKey-works:- Migrated the data from Oracle DB to AWS S3 using Spark, Scala and EMR.- Automated the data quality checks. Converted complex Oracle SQL to Spark SQL.- Built ETL data pipeline to extract data from Azure Blob to AWS S3 using rclone and pulsar.Used Airflow for workflow management and Terraform for cluster creation.
- Sep 2015 - Jul 2019Data Analytics Engineer at Gartner - Chennai, IndiaKey-works:- Built Email Bot product that extracts data from outlook mail box and generates sales leads and missing clients info based on auto reply emails using python and MongoDB- Built data pipelines for a marketing and insights team to extract google analytics data from GCP BigQuery to AWS S3.- Designed competitor analytics dashboard for events organizing team to understand the competitor past and future event details using python, Elastic Search and Kibana- Migrated data from Oracle to Datalake using Sqoop. Orchestrated workflow with Apache Airflow. Initiated CICD pipeline in AWS to deploy machine learning models
- Jul 2014 - Aug 2015Associate SQA Engineer at Symantec - Chennai, IndiaKey-works:- Built automation framework using Python for Norton Antivirus for Mac product to test the various security scenarios.- Contributed and maintained the common libraries used for different team usages.
- Jan 2014 - Jun 2014Masters Internship at Symantec - Chennai, IndiaKey-works:- Worked in Managed Cyber Security Services (SIEM) Team
- Aug 2012 - Jun 2014Masters in Cyber Security at Amrita Vishwa Vidhyapeetham - Coimbatore, India
- Aug 2008 - Jun 2012Bachelors in Information Technology at University College of Engineering - Tindivanam, India
- Jun 2006 - Mar 2008High School at Veveaham Higher Secondary School - Dharapuram, India
- Before 2006Grown up in Karur District
See more about my working experiences at my resume page.
Why have this blog?
Because sharing is learning
I believe noting down and sharing is the best way to learn! So I created this blog, where I keep things that I learned, find useful, and necessary to save for later.
More than just technical stuff, there are personal thoughts about work and life from the perspective of a Software Engineer.
It would be highly appreciated if I could read your comment or hear your thoughts on what I wrote
Tech stack
This blog is hosted on Vercel, built with Next.js (Preact for production) and Tailwind CSS using Tailwind Nextjs Starter Blog.
A huge thanks to Timothy Lin for the minimal, lightweight, and super easy-to-customize blog starter.
A hearty 💖 Thanks to Tuan Anh Huynh
- See source repo repository for this blog.
Images
All images in this blog are from Unsplash, imgur and GIPHY - Thanks for the free resources!