Software Engineer

itD is seeking a Software Engineer to design and scale the data pipelines that power next-generation foundation models for machine-generated data, including time series, logs, and large-scale event streams. This role contributes directly to the success of model training and production systems by enabling reliable, high-performance data infrastructure at scale. The ideal candidate will bring deep experience in distributed systems and data engineering, along with a proven track record of delivering scalable, production-ready data pipelines that support machine learning workflows. Location: Remote (U.S.-based; time zone alignment with Pacific or Central preferred) We provide comprehensive medical benefits, a 401(k) plan, paid holidays, and more. Please note that we are only considering direct W2 candidates at this time, as we are unable to offer sponsorship. Responsibilities: • Build and scale distributed data pipelines for large-scale time series, log data, and high-volume event streams. • Design and maintain reliable, high-performance Spark and Python workflows to support model training datasets. • Analyze and resolve performance bottlenecks related to latency, memory utilization, data skew, and throughput. • Improve data quality, validation processes, and reproducibility for machine learning workloads. • Partner with machine learning engineers and researchers to