Build and operate production machine learning training systems on Amazon SageMaker using AWS Trainium. The role focuses on optimizing PyTorch code for Trainium hardware, understanding NeuronCore compilation, diagnosing device-level issues, tuning distributed training for cost and throughput, and delivering production pipelines. The engineer will collaborate directly with client and internal teams in a forward-deployed engineering environment.
Robots & Pencils is an AWS Partner building production AI systems for enterprise clients who need real engineering, not proofs of concept that never ship. We work forward-deployed, embedded directly with client teams, solving the problems that are too new or too specialized for a typical vendor relationship to handle.
The Role
We're looking for an engineer who can operate and train models on Amazon SageMaker running on AWS Trainium, AWS's custom silicon built specifically for large-scale model training. This isn't a role where you call an API and wait. You'll be walking up the stack: understanding what a training request actually looks like at the Trainium hardware and compiler level, then carrying that understanding all the way up through PyTorch training code and into a production SageMaker pipeline.
PyTorch is the backbone of this work. If you know the framework deeply and you're comfortable reasoning about how your code actually behaves on custom accelerator hardware rather than treating it as a black box, this role is built around that skill set specifically.
What You'll Do
- Train and operate models on Amazon SageMaker with AWS Trainium as the underlying compute
- Write and optimize PyTorch training code with a real understanding of how it compiles and executes on Trainium (NeuronCore architecture, compiler behavior, memory and throughput tradeoffs)
- Diagnose training run issues that show up specifically because of the hardware, not just the model, distinguishing a data or code problem from a compiler or device-level one
- Translate a request for "a Trainium job" into an actual working, cost-aware training pipeline, end to end
- Tune distributed training runs for throughput and cost on SageMaker's training infrastructure
- Work directly with client and internal engineering teams to scope and deliver real production training workloads, not experiments that stay in a notebook
What You'll Bring
- Strong, hands-on PyTorch experience, ideally including distributed or multi-device training
- Production experience with Amazon SageMaker for training and/or inference
- Comfort working close to the hardware layer: you understand device-specific compilation and can debug issues that are actually about the accelerator, not just the model
- AWS Trainium or Inferentia (Neuron SDK) experience is a strong plus; if you don't have it yet but have deep PyTorch and a track record of picking up new hardware targets fast, we want to talk to you
- Solid Python fundamentals and comfort operating in a client-facing, production engineering environment
Robots & Pencils Austin, Texas, USA Office
119 Nueces St, Suite 215, Austin, Texas, United States, 78701
Similar Jobs
Automotive • Hardware • Internet of Things • Mobile • Software • App development • PropTech
Lead development and production deployment of machine learning systems for time-series anomaly detection in connected-device environments. Build streaming and MLOps pipelines, tune alerts, monitor model drift, and integrate AI systems with LLM-based assistants. Guide architecture, tooling, observability, and production readiness while mentoring and managing applied AI and full-stack engineers. Collaborate across engineering, product, data, and research teams.
Top Skills:
ArimaCi/CdDartsFlinkGluontsGruIsolation ForestKafkaKatsLlmsLstmMerlionNumpyOne-Class SvmPandasProphetPythonPyTorchScikit-LearnSparkTcnTensorFlowTransformersVector Databases
Digital Media • Information Technology • Professional Services • Consulting
Leads client-facing discovery and technical product development for AI initiatives. Designs, prototypes, deploys, and operates agentic systems, retrieval pipelines, enterprise integrations, and production-grade generative AI applications. Establishes evaluation, observability, safety, governance, and reliability frameworks while translating business needs into technical specifications. Requires strong Python and software engineering expertise, cloud and DevOps experience, retrieval architecture knowledge, and at least seven years of engineering experience, including production generative AI experience.
Top Skills:
AWSAzureCi/CdClaude CodeCodexCursorDockerGCPGithub CopilotKubernetesLoraMcpOpencodePeftPythonRagRestWebsockets
Social Media
Lead technical direction for Pinterest’s foundational machine learning capabilities in visual representation learning, multimodal content understanding, and LLM-based personalization. Drive initiatives from research and experimentation through productionization and adoption. Build reusable embeddings, models, semantic signals, and ML infrastructure using distributed training and data pipelines. Collaborate across product, engineering, data science, and applied science while mentoring technical teams and guiding multi-quarter roadmaps.
Top Skills:
Distributed Machine LearningGenerative AiHiveKnowledge DistillationLlmsMapreduceMultimodal LearningRecommender SystemsRetrievalSelf-Supervised LearningSequence ModelingSparkVlms
What you need to know about the Austin Tech Scene
Austin has a diverse and thriving tech ecosystem thanks to home-grown companies like Dell and major campuses for IBM, AMD and Apple. The state’s flagship university, the University of Texas at Austin, is known for its engineering school, and the city is known for its annual South by Southwest tech and media conference. Austin’s tech scene spans many verticals, but it’s particularly known for hardware, including semiconductors, as well as AI, biotechnology and cloud computing. And its food and music scene, low taxes and favorable climate has made the city a destination for tech workers from across the country.
Key Facts About Austin Tech
- Number of Tech Workers: 180,500; 13.7% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Dell, IBM, AMD, Apple, Alphabet
- Key Industries: Artificial intelligence, hardware, cloud computing, software, healthtech
- Funding Landscape: $4.5 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Live Oak Ventures, Austin Ventures, Hinge Capital, Gigafund, KdT Ventures, Next Coast Ventures, Silverton Partners
- Research Centers and Universities: University of Texas, Southwestern University, Texas State University, Center for Complex Quantum Systems, Oden Institute for Computational Engineering and Sciences, Texas Advanced Computing Center
