Harish Kesava Rao

Hello, I am Harish, a Data Engineering & Cloud Data Infrastructure professional.

I hold a Masters Degree from the University of Arizona’s Eller College of Management, majoring in Management Information Systems. It is the #2 ranked best public graduate information systems program per U.S. News & World Report 2025.

Although I am located in India for the past year, almost all of my professional work experience (2012-2024) and my advanced education has been from the United States.

Currently, I am a Principal Data Engineer & Data Architect for the Data Engineering organization at Altassian India.

My most recent role, prior to Atlassian’s was Staff Software Engineer and Data Engineering Team Lead for Security at Databricks, where I was involved in overseeing the execution of multi-quarter Data Engineering projects, including design, automation and scaling of multi-TB/day data in AWS and Azure.

Earlier, I was in Senior Data Engineering(Individual Contributor) roles at Salesforce’s Tableau, Amazon’s Prime Video and Indeed.com. In Amazon, I was hired as the first Data Engineer to help build a Data Lake for Prime Video Search. At Indeed, I worked for Marketing, Sales and Finance to help build their datawarehouses.

Prior to Indeed, I was in various roles are Informatica, focusing mainly on building ETL pipelines between relational databases and MPP databases.

My interests and key areas of work include:

• Understanding short-term & long-term data engineering requirements, converting them into high-level data architecture roadmaps for execution.
• Designing, building and scaling big data pipelines to ingest large volume data via Spark and AWS, Azure. I also have experience deploying big data pipelines on-premise.
• Performance tuning for high volume ingestion pipelines with SparkStreaming / AWS Kinesis, Azure Eventhubs on Databricks.
• Automation of workflows (such as Apache Airflow and other equivalent tooling on the cloud), automated infrastructure management on the cloud using Terraform (HCL).
• Experience in configuring and deploying Docker containers for specialized use cases.
• Extensive experience in writing Data Engineering code (primarily Python and Scala), adhering to good software engineering practices.
• I also contribute to Open Source projects on a regular basis. I developed the first DAG Sensors for the Databricks Provider in Apache Airflow.