Data Engineer

We are seeking a detail-oriented and proactive Data Engineer to join our team as a contractor, with the potential to transition into a full-time role based on performance and available funding. This position will play a key role in developing scalable and reliable data infrastructure, including the construction of simulation pipelines that simulate millions of agents at real-time scales. 

This role is ideal for candidates with strong Python and SQL skills, a solid foundation in data engineering best practices, and a passion for building reliable systems to support analytical and modeling workflows. The ideal candidate is also passionate about making an impact by solving complex problems related to the energy system and its role in climate change. 

As a Data Engineer, you will be responsible for gathering and cleaning data from a wide variety of sources, designing efficient data pipelines, managing cloud-based data storage, and ensuring data quality across workflows. This is a high-impact role directly supporting the development of cutting-edge simulations that provide guidance to decision-makers. 

This role requires that you both live in the United States and have authorization to work in the United States.

About us:

You will be joining Macrocosm, an early stage startup that is using complexity economics to revolutionize economic modeling. Our purpose is simple: better economics for a better world.

We are building the world's only empirically validated digital twin of the global economy, and our initial focus is on the energy system. You will be creating a robust data pipeline for a product platform that provides foresight into a number of key economic indicators (e.g., energy prices, energy supply, firms’ profits, technology costs, GDP, inflation, unemployment, etc.). This decision platform will be used to guide decisions for a range of clients, including asset managers, energy firms, institutional investors, policymakers, central banks, and others.

Key Responsibilities:

• Design, build, and maintain scalable data pipelines for ingesting, transforming, and storing structured and unstructured data.

• Optimize cloud-based data pipelines to support large-scale agent-based modeling simulations, including input/output data management, configuration tools like Hydra, and visualization in cloud environments.

• Integrate data from APIs, flat files, databases, and other sources into a centralized infrastructure.

• Implement data quality checks, monitoring systems, and validation tools to ensure accuracy and reliability.

• Collaborate with modeling teams to support simulation and analytics workflows while also ensuring data availability and consistency.

• Maintain documentation of data architecture, processes, and workflows.

• Contribute to the continuous improvement of our data infrastructure and tooling.
  

Required Qualifications:

• Bachelor’s degree (BA/BS or equivalent) in Computer Science, Engineering, Data Science, Information Systems, or a related field.

• Must reside and be authorized to work in the United States.

• Proficiency in Python, with experience using data-related libraries (e.g., pandas, SQLAlchemy, requests).

• Proficiency in SQL, including experience large-scale data querying and managing relational databases.

• Experience building and maintaining production-grade ETL pipeline development.

• Experience working with the Google Cloud Platform (GCP) ecosystem (e.g., BigQuery, Cloud Storage, Cloud Functions, Pub/Sub).

• Familiarity with Git and collaborative coding practices.

• Strong analytical, quantitative, and critical thinking skills.

• Ability to work independently, manage time effectively, and meet project deadlines.

Preferred Qualifications:

• Experience in one or more of the following domains: energy markets, financial markets, or economic modeling.

• Familiarity with data orchestration tools such as Airflow, Prefect, or Cloud Composer.

• Familiarity with data warehousing solutions (e.g., Snowflake, BigQuery, Redshift).

• Understanding and knowledge of data governance, metadata management, and best practices for data quality and security.

• Experience in complexity economics or agent-based modeling