CentML Logo

CentML

Senior Software Engineer - LLM Inference

Sorry, this job was removed at 06:08 p.m. (CST) on Wednesday, Aug 06, 2025
Remote or Hybrid
10 Locations
Remote or Hybrid
10 Locations

Similar Jobs

5 Minutes Ago
Remote
Canada
Senior level
Senior level
Software
The Cloud Platform Engineer will automate AWS infrastructure deployment, maintain CI/CD pipelines, enhance observability tools, and support developer operations.
Top Skills: AWSAzureBashDockerGoogle Cloud PlatformJavaJavaScriptKafkaKubernetesMySQLPostgresPythonRubyTerraformVpcs
7 Minutes Ago
Easy Apply
Remote
2 Locations
Easy Apply
161K-226K Annually
Mid level
161K-226K Annually
Mid level
Artificial Intelligence • Cloud • eCommerce • Enterprise Web • Software • Design • Generative AI
As a Partner Manager at Webflow, you'll develop strategies for partner growth, establish relationships with agencies, and drive revenue through collaboration with various teams.
Top Skills: Ai-Driven PersonalizationContent Management SystemsSaaS
7 Minutes Ago
Remote or Hybrid
Canada
Senior level
Senior level
Cloud • Fintech • Information Technology • Machine Learning • Software • App development • Generative AI
The Senior Territory Manager will drive BlackLine's SaaS adoption in Central Canada, focusing on generating leads, managing the sales cycle, and developing relationships with C-level executives to close significant deals.
Top Skills: ClariCRMErpGongHcmSales NavigatorSalesforce
About Us
We believe AI will fundamentally transform how people live and work. CentML's mission is to massively reduce the cost of developing and deploying ML models so we can enable anyone to harness the power of AI and everyone to benefit from its potential.

Our founding team is made up of experts in AI, compilers, and ML hardware and has led efforts at companies like Amazon, Google, Microsoft Research, Nvidia, Intel, Qualcomm, and IBM. Our co-founder and CEO, Gennady Pekhimenko, is a world-renowned expert in ML systems who holds multiple academic and industry research awards from Google, Amazon, Facebook, and VMware.

About the Position:  
As a member of the LLM inference team, you will help build state-of-the-art software with the goal of enabling LLM inference to become more efficient, scalable, and accessible. Are you interested in architecting and implementing the best inference stacks in the LLM world? Work and collaborate with a diverse set of teams involving resource orchestration, distributed systems, inference engine optimization, and writing high-performance GPU kernels.

Come join our team and contribute towards democratizing Machine Learning for the world!

Responsibilities:

  • Write safe, scalable, modular, and high-quality (C++/Python) code for our core backend software.
  • Perform benchmarking, profiling, and system-level programming for GPU applications. 
  • Provide code reviews, design docs, and tutorials to facilitate collaboration among the team.  
  • Conduct unit tests and performance tests for different stages of the inference pipeline.

Who you are:

  • Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience. 
  • Strong coding skills in Python and C/C++.
  • 5+ years of industry experience in software engineering. 
  • Knowledgeable and passionate about machine learning and performance engineering.

Nice to haves:

  • Solid fundamentals in machine learning and deep learning.
  • Solid fundamentals in operating systems, computer architecture, and parallel programming.
  • Research experience in systems or machine learning.
  • Industry experience in building enterprise-scale large distributed systems.
  • Experience with training, deploying, or optimizing the inference of LLMs in production is a plus.
  • Experience with performance modeling, profiling, debugging, and code optimization or architectural knowledge of CPU and GPU is a plus.

We strongly encourage you to include sample projects (e.g. Github) that demonstrate the qualifications above. 

For recent graduates, you can optionally submit your unofficial transcripts.

Benefits & Perks
- An open and inclusive work environment
- Employee stock options
- Best-in-class medical and dental benefits
- Parental Leave top-up
- Professional development budget
- Flexible vacation time to promote a healthy work-life blend

We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, disability, and any other protected ground of discrimination under applicable human rights legislation. 

CentML strives to respect the dignity and ‎‎independence of people with disabilities and is committed to giving them the same ‎‎opportunity to succeed as all other employees. 

Inclusiveness is core to our culture at CentML, and we strive to ensure you get the most from your interview experience. CentML makes reasonable accommodations for applicants with disabilities. If a reasonable accommodation is needed to participate in the job application or interview process, please reach out to the Talent team.

What you need to know about the Austin Tech Scene

Austin has a diverse and thriving tech ecosystem thanks to home-grown companies like Dell and major campuses for IBM, AMD and Apple. The state’s flagship university, the University of Texas at Austin, is known for its engineering school, and the city is known for its annual South by Southwest tech and media conference. Austin’s tech scene spans many verticals, but it’s particularly known for hardware, including semiconductors, as well as AI, biotechnology and cloud computing. And its food and music scene, low taxes and favorable climate has made the city a destination for tech workers from across the country.

Key Facts About Austin Tech

  • Number of Tech Workers: 180,500; 13.7% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Dell, IBM, AMD, Apple, Alphabet
  • Key Industries: Artificial intelligence, hardware, cloud computing, software, healthtech
  • Funding Landscape: $4.5 billion in VC funding in 2024 (Pitchbook)
  • Notable Investors: Live Oak Ventures, Austin Ventures, Hinge Capital, Gigafund, KdT Ventures, Next Coast Ventures, Silverton Partners
  • Research Centers and Universities: University of Texas, Southwestern University, Texas State University, Center for Complex Quantum Systems, Oden Institute for Computational Engineering and Sciences, Texas Advanced Computing Center

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account