ModelVault is hiring a
ModelVault is building the world’s first fully-integrated, on-prem AI appliance — a plug-and-play AI server preloaded with software, models, and tools that let businesses (and eventually consumers) run powerful AI models locally, privately, and securely.
We’re looking for a Senior Software Engineer to help us build the core software stack that powers our on-prem systems — from model integration and system orchestration to user interfaces and remote updates. You’ll be a key technical partner in defining our architecture, tools, and technical culture from day one.
🚀 What You’ll Do
-
Design and build the software stack that ships with ModelVault’s on-prem AI systems — everything from containerized inference engines to local dashboards and command-line tools.
-
Set up and manage integration with modern LLMs and vision models (e.g. LLaMA, Mistral, SDXL), including support for fine-tuning and local deployment.
-
Build and maintain user-facing tooling: CLIs, lightweight GUIs, and APIs that make the device intuitive for developers and AI teams.
-
Make architecture decisions for system orchestration, hardware interfacing, remote update pipelines, and telemetry — all optimized for stability and security.
-
Help set up GPU drivers, OS configuration (e.g., Ubuntu, NVIDIA stack), and optimized inference runtimes as needed.
-
Collaborate with the founding team to test and iterate with early business customers.
-
Contribute to hiring and scaling the engineering team over the next 12 months.
🧠 You Might Be a Fit If You
-
Have 5+ years of software engineering experience, preferably at early-stage startups or in fast-paced environments.
-
Are comfortable working across the stack — from OS-level setup and deployment to user-facing tools.
-
Have experience with containerization, system packaging, deployment tooling, and managing model runtimes.
-
Have worked with, or integrated, LLMs and ML models in real-world systems (e.g., inference, fine-tuning, embeddings).
-
Can independently drive architecture decisions and own large pieces of product infrastructure.
-
Thrive in ambiguity, learn fast, and are passionate about building great tools from scratch.
⭐️ Bonus Points
-
Experience with GPU computing, NVIDIA drivers, CUDA, or model acceleration tools.
-
Familiarity with ML tooling: vLLM, Ollama, Triton Inference Server, LangChain, etc.
-
Comfort with Linux system internals, remote access tooling (e.g., SSH, Ansible), or auto-update agents.
-
Previous experience with AI developer tooling, embedded systems, or secure local apps.
🌍 Why Join ModelVault?
-
Help define a new category: secure, local AI for everyone — starting with businesses, expanding to the world.
-
Work on real, shipped hardware + software that users will touch.
-
Be part of the founding engineering team with high autonomy and deep ownership.
-
Fully remote, async-friendly, with strong product velocity and clear vision.
Please mention the word COMPREHENSIVE when applying to show you read the job post completely (#RMmEwMjo0NzgwOjExOjE0Mzg6MDozNzdjOjM2ZjQ6MQ==). This is a feature to avoid fake spam applicants. Companies can search these words to find applicants that read this and instantly see they’re human.
Salary and compensation
$60,000 — $100,000/year
Benefits
🌎 Distributed team
⏰ Async
🏖 Paid time off
📚 Learning budget
💰 Profit sharing
🚫 No politics at work
How do you apply?
Please submit your application using the link below. We aim to respond to you within one week.
Apply for this job
👉 Please reference you found the job on Remote OK, this helps us get more companies to post here, thanks!
When applying for jobs, you should NEVER have to pay to apply. You should also NEVER have to pay to buy equipment which they then pay you back for later. Also never pay for trainings you have to do. Those are scams! NEVER PAY FOR ANYTHING! Posts that link to pages with “how to work online” are also scams. Don’t use them or pay for them. Also always verify you’re actually talking to the company in the job post and not an imposter. A good idea is to check the domain name for the site/email and see if it’s the actual company’s main domain name. Scams in remote work are rampant, be careful! Read more to avoid scams. When clicking on the button to apply above, you will leave Remote OK and go to the job application page for that company outside this site. Remote OK accepts no liability or responsibility as a consequence of any reliance upon information on there (external sites) or here.
|
