field notes
Quick observations and lessons learned in the field.
Reading List & Resources
Blogs
- A collection of data engineering blogs from across organizations
- Semantic Search and RAG for Hacker News and a vector search application example
- Open Source Data Engineering
- A good read about LTAP by Reynold Xin
Papers & Research
- BAIR blog on Compound AI systems and what are they?
- Some good posts on text retrieval and LLMs
- Readings in Databases from rxin.
- Zerrow: True Zero-Copy Arrow Pipelines in Bauplan: I was playing around with Polars for a side project recently and this is an interesting related read on zero-copy.
Conference Talks
- Data-Driven Detection using PySpark - I was part of this during my time at Databricks