Tessera
Folgen

CNCF containerd Team

@containerd.dev

The Cloud Native Computing Foundation (CNCF) graduated container runtime project, containerd. Check us out at https://containerd.io

5 Posts1016 Follower22 Folgt
Samuel Karp@samuelkarp.com · last yr.

This is pretty cool: a @containerd.dev shim to run LLMs with ollama. The shim augments the image by mounting in the ollama binary, generating a Modelfile as input to ollama (based on the model bundled in the image), and setting an entry point.

Mount Ollama binary • Mount the Ollama binary from the host into the container config. Mounts append(config. Mounts, specs. Mount Source: "/usr/bin/ollama" Destination: "/usr/bin/ollama", Type: "bind". Options: []string("rbind", "ro"}, })Mount OCI as a Modelfile • Ollama needs to read the GGUF from a Modelfile file modelPath: getTaskEnv ("OLLAMA_MODEL PATH", config.Process.Env) modelfile: fmt.Sprintf("FROM %s", modelPath) err = os.WriteFile(filepath.Join(tempDir, "Modelfile"), []byte (modelfile), 0644) if err != nil { return nil, err } config.Mounts append(config.Mounts, specs. Mount( Source: filepath.Join(tempDir, "Modelfile"), Destination: "/tmp/Modelfile", Type: "bind", Options: []string("rbind", "ro"}, })Serve the Model Set the container args to serve the model using Ollama config.Process.Args = [] string( /bin/bash", "-c". strings.Join([]string{ "ollama serve &", "while ollama ps> /dev/null 2>&1; do sleep 1; done", "ollama create modelName + --file/tmp/Modelfile", "tail -f /dev/null", }, "\n"),
Samuel Karp@samuelkarp.com · 2y ago

We released containerd 2.0 this week, with new features, now-stable features, and deprecations. Join me next Friday at KubeCon to learn how to prepare for 2.0 in your production environments. We'll cover what's new and discuss how to detect/remediate any impact of removed features. sched.co/1i7qL

Promotional card for "What containerd 2.0 means for you" at KubeCon NA 2024 on November 15, 2024.
CNCF containerd Team@containerd.dev · 2y ago

Good things come in threes! One of our maintainers, Akihiro Suda, also leads the Lima and nerdctl projects, and blogged about the containerd 2.0.0 release alongside nerdctl 2.0 and Lima 1.0 releases, also released yesterday! Check his blog post out here: medium.com/nttlabs/cont...

containerd v2.0, nerdctl v2.0, and Lima v1.0

Ahead of the KubeCon North America 2024 (November 12–15), this week saw the releases of containerd v2.0, nerdctl (contaiNERD CTL) v2.0, and…

medium.com

CNCF containerd Team@containerd.dev · 2y ago

Hi 👋 Bluesky! We're joining our friends like @kubernetes.io here and look forward to interacting with our cloud native community here. Let us know what you think of containerd 2.0 and catch us next week at KubeCon/CloudNativeCon North America!

CNCF containerd Team@containerd.dev · 2y ago

Yes, the @containerd.dev team is excited to announce our 2.0.0 release, just in time for KubeCon NA in Salt Lake City next week 😅 We're thankful to our amazing set of contributors who have collaborated to make this release a reality. We will have a project pavilion booth as well! Come chat with us!

Aurélie Vache@aurelievache.bsky.social · 2y ago

Containerd 2.0 (and KubeCon NA 2024)! samuel.karp.dev/blog/2024/11... The first major release of containerd since 1.0 was released in 2017, and represents a commitment both to the evolution of the containerd project and continued investment in stability, reliability, and efficiency.