Member of Technical Staff, Pre/Mid-Training at Inception – San Mateo, California
Inception
San Mateo, California, United States
Posted on
Updated on
Employment Type:Full-Time
Explore Related Opportunities
Miscellaneous Computer Occupations jobs in CaliforniaJobs in CaliforniaMiscellaneous Computer Occupations jobs
About This Position
The Role
We seek experienced scientists and engineers with deep expertise in pre- and mid-training large language models. You will advance our diffusion-based LLM models, developing novel training techniques and pushing the boundaries of parallel token generation.
Key Responsibilities
- Design, develop, and optimize architectures for diffusion-based language models.
- Implement innovative training objectives and loss functions for discrete diffusion LLMs.
- Research and implement techniques for controlled text generation and constraint satisfaction.
- Develop methods for multi-modal integration within the diffusion framework.
- Improve model efficiency, reduce training time, and optimize inference throughput.
Qualifications
- BS/MS/PhD in Computer Science or a related field (or equivalent experience).
- At least 2 years of experience working on ML projects in PyTorch (or equivalent), preferably in a research lab or engineering role.
- Excellent familiarity with transformers and core LLM concepts (autoregressive pretraining, instruction tuning, in-context learning, KV caching).
- Familiarity with training and inference in diffusion models.
- Experience training deep learning models at scale in distributed computing environments.
Preferred Skills
- Extensive experience training transformer-based language models from scratch.
- Knowledge of advanced training techniques (mixed precision, gradient accumulation, etc.).
- Experience with multi-modal learning and cross-modal architectures.
- Background in optimization theory and neural network architecture design.
- Experience with LLM serving frameworks like vLLM, SGLang, or TensorRT.
Scan to Apply
Just scan this QR code to apply from your phone.
Job Location
San Mateo, California, United States
Frequently asked questions about this position
Continue to apply
Enter your email to continue. You’ll be redirected to the employer’s application.By clicking Continue, you understand and agree to JobTarget's Terms of Use and Privacy Policy.