I am an Expert Cloud&Blockchain Data Engineer with over 7 years of production experience, as an Architect for 5 most recent years, designed 25+ projects for various domains including finance, marketing, retail, blockchain. On PyChain I am going to share with you a brief summary of my original Medium article "How to Export a Full History of Ethereum Blockchain to S3"
Most recent solutions I have designed and implemented:
— AWS-based serverless Lakehouse (using S3, EMR, Glue, ECS, Athena) for a full history of Ethereum data.
— GCP-based ML-oriented ETL infrastructure (using Airflow, Dataflow); domain: foodtech.
— Real-time events tracking system (using ClickHouse, DataLens, serverless); domain: media, marketing.
— Adopted Druid+Superset BI toolset; domain: adtech.
Skills:
— Proficient in Python.
— Major remote experience: I build an efficient work process for a distributed team.
— Airflow contributor and plugin creator: pypi.org/project/airflow-clickhouse-plugin
— Extensive experience with MapReduce (incl. Spark) and BI tools.
— Effective communicator, responsible, team-oriented.
I have a Master's degree in Computer Science and have authored a scientific paper in Big Data.