Luke Marsden
@lmarsden.bsky.social
Hacker & entrepreneur. Founder helix.ml, private GenAI stack, getting business value out of local open source LLMs
Bootstrapped a Private GenAI Startup to $1M revenue, AMA blog.helix.ml/p/bootstrapp...
Bootstrapped Private GenAI Startup Hits $1M Annual Revenue, Launches Helix 2.0
The people behind the story, how agentic AI is changing and why we don't want a sales call with you
blog.helix.ml
Good morning #KubeCon! Come find me if you want to talk running LLM systems like vision RAG in production on Kubernetes today. Email luke@helix.ml or reply here!
Phil setting up a vLLM vision model provider on helix on his nerd phone on the Elizabeth line #kubecon
Catch my talk with Priya Samuel on running LLMs in production on k8s today at #KubeCon at S10A at 3:15pm ✨
Luke Marsden@lmarsden.bsky.social · last yr.Trying (but failing) to be as colourful as @hannahfoxwell.net today!
Oh hello there BlueSky! We've arrived here just in time to share all the highlights of #KubeCon #CloudNativeCon in sunny London! Don't miss @hannahfoxwell.net and @lmarsden.bsky.social at 15:20 today talking about Platform Engineering and Developer Experience For Your On Prem LLM
I looked to define a term that’s being thrown around — #AI-native developer — with the help of @guypo.com @lmarsden.bsky.social and Thoughtworks’ Mike Mason so you can prepare your software teams for the near future. thenewstack.io/what-is-an-a... What’s your definition? Only on @thenewstack.io
What Is an AI Native Developer?
We explain what an AI native developer is, how the role will evolve, and why spec-driven AI development is driving this trend.
thenewstack.io
Deepseek are clever fuckers. I wrote this about how Deepseek is pushing decision makers in large financial institutions to seriously consider running their own models instead of calling out to Microsoft, Amazon & Google Link below 👇
Speaker @lmarsden.bsky.social of MLOps Consulting will be in our SOOCon25 AI Openness track! 🎤 Join to hear about Production-Ready LLMs on Kubernetes: Patterns, Pitfalls, and Performance.💡https://buff.ly/3HjeQxq #opensource #opensourceai #ai #stateofopencon #soocon25 #opensourcelondon #openuk
I enjoyed Stories of Your Life and Others so much that I'm moving straight to Chiang's other collection: Exhalation.
MS Teams in the UK this morning is the most grim experience imaginable, tons of jitter and bandwidth seems fucked - like trying to do a video call over a highly contended LTE connection - my network connection is fine (per mtr) - is there an outage in some MS data center somewhere?
We shipped a lot over Christmas and I came here to release it: github.com/helixml/heli... - You can now drag'n'drop files directly into knowledge for Helix apps (rather than having to go via the filestore). - Initial support for MCP (more on this coming soon!)
Release 1.4.12 - apps drag'n'drop filestore, initial MCP support · helixml/helix
What's Changed make frontend work better with filestore knowledge by @nessie993 in #668 Model Context Protocol (MCP) support by @nessie993 in #680 security/refactoring/quality changes: Revive vo...
github.com
Cancelled my X subscription over all the stupid political interference
Thank you to @dciangot.com for doing the heavy lifting getting HelixML GPU runners running on Slurm HPC infra to take advantage of hundreds of thousands of GPUs running on Slurm infrastructure and transform them into multi-tenant GenAI systems! Read all about it here: blog.helix.ml/p/running-ge...
Running GenAI on Supercomputers: Bridging HPC and Modern AI Infrastructure
Transform Your Supercomputer into a Private OpenAI: A Look at How Modern HPC Infrastructure Can Power Enterprise AI
blog.helix.ml
This is so awesome! Imagine being able to combine the worlds of cloud-native scalable web services for LLMOps with the raw power of Slurm-powered supercomputers with some of the biggest compute power, networking and GPUs! Check out the writeup here: blog.helix.ml/p/running-ge...
Running GenAI on Supercomputers: Bridging HPC and Modern AI Infrastructure
Transform Your Supercomputer into a Private OpenAI: A Look at How Modern HPC Infrastructure Can Power Enterprise AI
blog.helix.ml
Diego Ciangottini@dciangot.com · 2y agoWhat if you could have a #GenAI platform to develop and deploy your apps, while offloading the model inference into a huge #EuroHPC supercomputer? The power of #HelixML and #interlink exploiting VEGA #HPC center is crazy :) Take a look at this demo! #AI #DevOps www.youtube.com/watch?v=0OTI...
So @cybernetist.com got gptscript working with llama3.3:70b! Check out the detailed writeup here: blog.helix.ml/p/gptscript-...
GPTScript Helix Apps: For Fun and Profit
Making GPTScript Shine with Open Source LLMs: How Llama 3's 70B Model Finally Makes Natural Language Programming Reliable
blog.helix.ml
Fucking dope helix.ml release from @cybernetist.com if you like function calling in open source LLMs and @GPTScript_ai github.com/helixml/heli... (Excuse my French)
Release 1.4.9 - support function calling in helix runners, add gptscript runner · helixml/helix
What's Changed Add gptscript_runner service to helix stack by @milosgajdos in #635 Bump ollama and update LLM inference with Tool calling by @milosgajdos in #637 Full Changelog: 1.4.8...1.4.9
github.com
Test Driven Development (TDD) for your LLMs? Yes please, more of that please! Back to basics - write a test, see the test fail, improve the prompt, see it pass, check it in - just like you would with any other code 😄
Luke, this post, including the video, was very helpful. I loved the conference talk. I took my graduate students through a somewhat similar exercise last week for their last lab of the semester. Well done. Thank you.
This is a wonderful post by @lmarsden.bsky.social Great presentation and video as well.
Luke Marsden@lmarsden.bsky.social · 2y agoICYMI, here's my 2-part blog series on how we can all be AI engineers with open source models – and how to apply testing best practices to LLM apps 1. blog.helix.ml/p/we-can-all... 2. blog.helix.ml/p/from-click...
ICYMI, here's my 2-part blog series on how we can all be AI engineers with open source models – and how to apply testing best practices to LLM apps 1. blog.helix.ml/p/we-can-all... 2. blog.helix.ml/p/from-click...
We can all be AI engineers – and we can do it with open source models
The barriers to AI engineering are crumbling fast
blog.helix.ml
@llm360.bsky.social please consider adding me to the open source LLMs starter pack ☺️
So how can we _actually_ do AI on Kubernetes? Here's my humble proposal: LLM apps and tests as data (CRDs). Deeply integrated into the existing CI/CD pipeline e.g GitLab and Flux. Self hosted LLMs on GPUs if you want to. Runnable code: github.com/helixml/gena...
GitHub - helixml/genai-cicd-ref: Cloud-Native GenAI Reference Architecture
Cloud-Native GenAI Reference Architecture. Contribute to helixml/genai-cicd-ref development by creating an account on GitHub.
github.com
Just followed all 1400 people who followed me. If you see this, like this post to say hi! 🦋💚🦋
I wrote a quick summary of the latest MLOps Community podcast: blog.helix.ml/p/from-click...
From ClickOps to GitOps: The Evolution of AI App Development
How we bridge the gap between no-code AI prototypes and production-ready applications
blog.helix.ml
One year ago, I promised to build helix.ml in the open, so here's an update on our commercial progress: blog.helix.ml/p/revenue-an...
Building in the open: Revenue and Pipeline for Private GenAI
National data regulation, repatriation of data, and return to the private cloud is combining with the demand to stay competitive to drive interesting pipeline for our Private GenAI business
blog.helix.ml
Anyone got a good system in place for running automated evals against your LLM-driven applications, to help test if tweaks to prompts get better results? I'd love to hear about approaches that are working for people on this
Now that loads more people are here I guess I should run rerun this thing, right?
📷 Daniel Hückmann@danielhuckmann.com · 2y ago
