Data Engineer - Build the Data Pipelines Fueling AI-Powered Real Estate

Propcorn AI is seeking a skilled and passionate Data Engineer to join our growing startup team. We're building the next generation of AI-driven real estate solutions, and we need your expertise to build and maintain the robust data infrastructure that powers our insights and models. This includes building and maintaining systems for collecting data from various sources, including web crawling.

About the Role:

As a Data Engineer at Propcorn AI, you'll be a key player in designing, building, and maintaining the data pipelines that collect, process, and store the vast amounts of data we use. You'll work closely with data scientists and engineers to ensure data quality, reliability, and accessibility. This role requires a strong understanding of data engineering principles, experience with various data processing technologies, and the ability to thrive in a fast-paced, collaborative environment.

What We Offer:

• Growth Potential: We're invested in your professional development. You'll have ample opportunities to learn new technologies, expand your skillset, and advance your career within a rapidly growing company.

• Impactful Work: Contribute directly to a product that's transforming the real estate landscape. Your work will be critical to our success.

• Collaborative Environment: Work alongside a passionate and supportive team in a flat organizational structure.

• "Hacker Mentality" Encouraged: We value creativity, problem-solving, and a proactive approach to development. We encourage you to experiment, explore new ideas, and contribute to our innovative culture.

What We're Looking For:

• A highly motivated Data Engineer with a passion for building robust and scalable data infrastructure.

• Strong experience with at least one of the following programming languages: Python (essential), Java, or Scala.

• Experience with data warehousing solutions and ETL processes. • Proficiency with SQL and NoSQL databases (e.g., PostgreSQL, MongoDB). • Experience with data processing frameworks like Apache Spark. • Familiarity with cloud computing platforms (e.g., AWS, GCP, Azure) and related data engineering services.

• Experience with data pipeline orchestration tools (e.g., Airflow). • Experience with web scraping and crawling techniques and tools. • Experience with containerization and orchestration technologies like Docker and Kubernetes is a plus.

• A strong understanding of data quality and data governance principles. • Excellent communication and collaboration skills.

• A proactive and collaborative approach to problem-solving.

• A "hacker mentality" – a passion for exploring new technologies and finding creative solutions.

• A desire to learn and grow in a fast-paced startup environment.

Our Tech Stack (Relevant to Data Engineering):

• Python (essential), Java, or Scala

• PostgreSQL(required), MongoDB (or other relevant databases) • Apache Spark(required)

• AWS, GCP, or Azure (and related data services)

• Airflow (or similar tools)

• Web scraping/crawling libraries and frameworks (e.g., Scrapy, Beautiful Soup) • Docker, Kubernetes (desirable)

Responsibilities (Examples):

• Design, build, and maintain data pipelines for collecting, processing, and storing data from various sources, including web data through scraping and crawling.

• Develop and implement ETL processes to transform and load data into data warehouses and data lakes.

• Ensure data quality, reliability, and consistency.

• Monitor data pipelines and troubleshoot issues.

• Collaborate with data scientists and engineers to understand data requirements and provide support.

• Optimize data pipelines for performance and scalability.

• Implement data governance and security measures.

• Develop and maintain web scraping and crawling systems, including handling website structure changes, rate limiting, and data extraction.

• Stay up-to-date with the latest advancements in data engineering technologies.

Interested?

If you're ready to tackle challenging and rewarding projects, and you're excited about the prospect of working on a cutting-edge AI-powered product that's transforming the real estate industry, we encourage you to apply! Please send your resume and a brief introduction to hiring@propcorn.ai. We look forward to hearing from you!