Design and implement AI model inference solutions, optimize performance, own production systems, and collaborate in engineering best practices.
DeepInfra is looking for strong Software Engineers to join our team. You’ll work on designing, building, and scaling infrastructure for serving top open-source AI models in production. This role is ideal for engineers who are already comfortable owning problems end-to-end and want to deepen their experience working on high-impact AI systems.
If you’re excited about AI/ML and are looking to work on real systems at scale — we’d love to meet you.
- Design, develop, and test inference solutions for state-of-the-art AI models
- Implement, optimize, and evaluate AI models using Python, C++, CUDA, and NCCL
- Own and operate production model-serving systems, including monitoring and debugging
- Build new features, improve system performance, and contribute to overall system design
- Participate in code reviews and technical discussions to maintain high engineering standards
- Explore and apply new AI/ML techniques to improve model performance and efficiency
- Take ideas from concept to production
- Bachelor’s or Master’s degree in Computer Science, Computer Engineering, or a related field
- 3+ years of relevant experience
- Strong fundamentals in data structures, algorithms, and software design
- Proficiency in Python and experience working with AI/ML frameworks (e.g., PyTorch, TensorFlow)
- Hands-on experience building, shipping, and maintaining software systems
- Familiarity with AI models, Transformers, and Diffusers
- Experience working with version control (Git) and collaborative development workflows
- Ability to debug, optimize, and improve existing systems
- Strong communication skills and ability to work independently in a fast-paced environment
- Experience with C++, CUDA, or AI inference
- Contributions to open-source ML projects
- Work on cutting-edge AI model serving - the systems that power the next generation of LLMs and multimodal models.
- Small team, huge impact: your work ships directly to customers.
- Opportunity to learn from engineers building high-performance inference at scale.
- Fast-paced environment with ownership, autonomy, and end-to-end responsibility.
Similar Jobs
Artificial Intelligence • Machine Learning • Software
Assist the engineering team in designing, developing, and deploying scalable AI models. Responsibilities include implementing models, optimizing code, and participating in collaboration activities.
Top Skills:
C++CudaGitNcclNumpyPandasPythonPyTorchScipyTensorFlow
Artificial Intelligence • Machine Learning • Software
DeepInfra seeks early-career Software Engineers to design and scale infrastructure for AI models. Responsibilities include collaboration on AI solutions, coding, testing, and maintaining production systems, alongside learning and growth opportunities.
Top Skills:
C++CudaGitNcclNumpyPandasPythonPyTorchScipyTensorFlow
Artificial Intelligence • Machine Learning • Natural Language Processing • Software • Conversational AI
The Pre-Sales Solutions Engineer leads technical discovery, designs and executes POCs, advises on architectural needs, triages issues, and collaborates with teams to ensure customer success and continuous engagement throughout the project lifecycle.
Top Skills:
DockerHTTPJavaScriptKubernetesPythonSipTypescriptWebrtc
What you need to know about the Austin Tech Scene
Austin has a diverse and thriving tech ecosystem thanks to home-grown companies like Dell and major campuses for IBM, AMD and Apple. The state’s flagship university, the University of Texas at Austin, is known for its engineering school, and the city is known for its annual South by Southwest tech and media conference. Austin’s tech scene spans many verticals, but it’s particularly known for hardware, including semiconductors, as well as AI, biotechnology and cloud computing. And its food and music scene, low taxes and favorable climate has made the city a destination for tech workers from across the country.
Key Facts About Austin Tech
- Number of Tech Workers: 180,500; 13.7% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Dell, IBM, AMD, Apple, Alphabet
- Key Industries: Artificial intelligence, hardware, cloud computing, software, healthtech
- Funding Landscape: $4.5 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Live Oak Ventures, Austin Ventures, Hinge Capital, Gigafund, KdT Ventures, Next Coast Ventures, Silverton Partners
- Research Centers and Universities: University of Texas, Southwestern University, Texas State University, Center for Complex Quantum Systems, Oden Institute for Computational Engineering and Sciences, Texas Advanced Computing Center
