Yuan Tang

@terrytangyuan.xyz

Senior Principal Software Engineer at Red Hat AI | Open Source Leader at KServe, Argo, Kubeflow, Kubernetes, CNCF | Maintainer of XGBoost, TensorFlow | Keynote Speaker | Author | Technical Advisor More info: http://terrytangyuan.xyz We are hiring!

๐Ÿ“ข ๐—ง๐—ต๐—ฒ ๐—ฆ๐˜๐—ฎ๐˜๐—ฒ ๐—ผ๐—ณ ๐— ๐—ผ๐—ฑ๐—ฒ๐—น ๐—ฆ๐—ฒ๐—ฟ๐˜ƒ๐—ถ๐—ป๐—ด ๐—–๐—ผ๐—บ๐—บ๐˜‚๐—ป๐—ถ๐˜๐—ถ๐—ฒ๐˜€: ๐—”๐˜‚๐—ด๐˜‚๐˜€๐˜ ๐—˜๐—ฑ๐—ถ๐˜๐—ถ๐—ผ๐—ป ๐—ถ๐˜€ ๐—ผ๐˜‚๐˜!

I'm thrilled to share that I've been invited to become Omnigent's first external maintainer! It's remarkable that the project is only two months old and I feel incredibly fortunate to contribute at such an early stage and collaborate with the Databricks team and broader community.

Part 2 of our ๐——๐—ถ๐˜€๐˜๐—ฟ๐—ถ๐—ฏ๐˜‚๐˜๐—ฒ๐—ฑ ๐—”๐—œ ๐—œ๐—ป๐—ณ๐—ฒ๐—ฟ๐—ฒ๐—ป๐—ฐ๐—ฒ series is now live on Red Hat Developer: ๐˜–๐˜ฑ๐˜ต๐˜ช๐˜ฎ๐˜ช๐˜ป๐˜ช๐˜ฏ๐˜จ ๐˜‹๐˜ช๐˜ด๐˜ต๐˜ณ๐˜ช๐˜ฃ๐˜ถ๐˜ต๐˜ฆ๐˜ฅ ๐˜ˆ๐˜ ๐˜๐˜ฏ๐˜ง๐˜ฆ๐˜ณ๐˜ฆ๐˜ฏ๐˜ค๐˜ฆ: ๐˜ˆ๐˜ฅ๐˜ท๐˜ข๐˜ฏ๐˜ค๐˜ฆ๐˜ฅ ๐˜‹๐˜ฆ๐˜ฑ๐˜ญ๐˜ฐ๐˜บ๐˜ฎ๐˜ฆ๐˜ฏ๐˜ต ๐˜—๐˜ข๐˜ต๐˜ต๐˜ฆ๐˜ณ๐˜ฏ๐˜ด. In Part 1, we covered prefill/decode phases and the 5D parallelism framework.

Excited to share Part 1 of our blog series on Red Hat Developer: ๐˜‹๐˜ฆ๐˜ด๐˜ช๐˜จ๐˜ฏ๐˜ช๐˜ฏ๐˜จ ๐˜‹๐˜ช๐˜ด๐˜ต๐˜ณ๐˜ช๐˜ฃ๐˜ถ๐˜ต๐˜ฆ๐˜ฅ ๐˜ˆ๐˜ ๐˜๐˜ฏ๐˜ง๐˜ฆ๐˜ณ๐˜ฆ๐˜ฏ๐˜ค๐˜ฆ: ๐˜Š๐˜ฐ๐˜ณ๐˜ฆ ๐˜Š๐˜ฐ๐˜ฏ๐˜ค๐˜ฆ๐˜ฑ๐˜ต๐˜ด ๐˜ข๐˜ฏ๐˜ฅ ๐˜š๐˜ค๐˜ข๐˜ญ๐˜ช๐˜ฏ๐˜จ ๐˜‹๐˜ช๐˜ฎ๐˜ฆ๐˜ฏ๐˜ด๐˜ช๐˜ฐ๐˜ฏ๐˜ด. LLM inference is two workloads pretending to be one. The prefill phase is compute-bound, processing entire prompts in parallel to populate the KV cache.

๐Ÿ“ข ๐—ง๐—ต๐—ฒ ๐—ฆ๐˜๐—ฎ๐˜๐—ฒ ๐—ผ๐—ณ ๐— ๐—ผ๐—ฑ๐—ฒ๐—น ๐—ฆ๐—ฒ๐—ฟ๐˜ƒ๐—ถ๐—ป๐—ด ๐—–๐—ผ๐—บ๐—บ๐˜‚๐—ป๐—ถ๐˜๐—ถ๐—ฒ๐˜€: ๐—๐˜‚๐—ป๐—ฒ ๐—˜๐—ฑ๐—ถ๐˜๐—ถ๐—ผ๐—ป ๐—ถ๐˜€ ๐—ผ๐˜‚๐˜! We recently launched our newsletter publicly after sharing it internally at Red Hat AI for over a year.

Iโ€™ve had a great experience using Omnigent and have been collaborating with the Databricks team to land a few features and fixes, including: - Kubernetes & OpenShift deployment - NVIDIA OpenShell sandbox integration - Claude on Vertex AI auto-detection - Podman support

I recently received a LinkedIn Premium Business subscription that I'll be able to use indefinitely. More than the subscription itself, I'm grateful for the connections I've made through this platform over the years.

The job market is tough right now, especially for new graduates and talented people impacted by layoffs. I have several free 2-month LinkedIn Premium invites available, and Iโ€™d love to share them with people who could benefit from the extra visibility, networking tools, and learning resources.

Excited to share our latest blog post on how we're solving real-world LLM inference challenges at production scale, a collaboration between Red Hat AI and Tesla engineering teams.

Bild

๐Ÿ“ข ๐—ง๐—ต๐—ฒ ๐—ฆ๐˜๐—ฎ๐˜๐—ฒ ๐—ผ๐—ณ ๐— ๐—ผ๐—ฑ๐—ฒ๐—น ๐—ฆ๐—ฒ๐—ฟ๐˜ƒ๐—ถ๐—ป๐—ด ๐—–๐—ผ๐—บ๐—บ๐˜‚๐—ป๐—ถ๐˜๐—ถ๐—ฒ๐˜€: ๐—”๐—ฝ๐—ฟ๐—ถ๐—น ๐—˜๐—ฑ๐—ถ๐˜๐—ถ๐—ผ๐—ป ๐—ถ๐˜€ ๐—ผ๐˜‚๐˜! Our goal with this newsletter is to give a clear, community-driven view of whatโ€™s happening across the model serving ecosystem, including updates from projects like vLLM, KServe, @llm-d.ai, @kubernetes.io, Llama Stack, and more.

Another fruitful KubeCon โ€” this time in Amsterdam! I had a wonderful time presenting across ๐—ณ๐—ผ๐˜‚๐—ฟ ๐˜€๐—ฒ๐˜€๐˜€๐—ถ๐—ผ๐—ป๐˜€: - Opening remarks for Cloud Native AI Day - KServe project lightning session - @kubernetes.io AI Conformance Program updates - @cncf.io TAG Workloads Foundation updates

BildBildBildBild

๐ŸŽค ๐—ฆ๐—น๐—ถ๐—ฑ๐—ฒ๐˜€ ๐—ฎ๐—ฟ๐—ฒ ๐—ฎ๐˜ƒ๐—ฎ๐—ถ๐—น๐—ฎ๐—ฏ๐—น๐—ฒ! At #KubeCon Europe 2026 in Amsterdam, I had the pleasure of joining Stephen Rust, Kante Yin, and Marlow Warnicke for our session: ๐˜Œ๐˜น๐˜ฑ๐˜ญ๐˜ฐ๐˜ณ๐˜ฆ ๐˜›๐˜ˆ๐˜Ž ๐˜ž๐˜ฐ๐˜ณ๐˜ฌ๐˜ญ๐˜ฐ๐˜ข๐˜ฅ๐˜ด ๐˜๐˜ฐ๐˜ถ๐˜ฏ๐˜ฅ๐˜ข๐˜ต๐˜ช๐˜ฐ๐˜ฏ: ๐˜ˆ๐˜ฅ๐˜ท๐˜ข๐˜ฏ๐˜ค๐˜ช๐˜ฏ๐˜จ ๐˜Š๐˜ญ๐˜ฐ๐˜ถ๐˜ฅ ๐˜•๐˜ข๐˜ต๐˜ช๐˜ท๐˜ฆ ๐˜Œ๐˜น๐˜ฆ๐˜ค๐˜ถ๐˜ต๐˜ช๐˜ฐ๐˜ฏ ๐˜๐˜ณ๐˜ฐ๐˜ฎ ๐˜Š๐˜ฐ๐˜ณ๐˜ฆ ๐˜™๐˜ถ๐˜ฏ๐˜ต๐˜ช๐˜ฎ๐˜ฆ ๐˜›๐˜ฐ ๐˜ˆ๐˜ฑ๐˜ฑ๐˜ญ๐˜ช๐˜ค๐˜ข๐˜ต๐˜ช๐˜ฐ๐˜ฏ๐˜ด

I was fortunate to get my hands on ๐˜Ž๐˜ฆ๐˜ฏ๐˜ฆ๐˜ณ๐˜ข๐˜ต๐˜ช๐˜ท๐˜ฆ ๐˜ˆ๐˜ ๐˜ฐ๐˜ฏ ๐˜’๐˜ถ๐˜ฃ๐˜ฆ๐˜ณ๐˜ฏ๐˜ฆ๐˜ต๐˜ฆ๐˜ด before itโ€™s even available for everyone to order online! It was also great to meet the co-authors and my wonderful colleagues - Daniele Zonca and @ro14nd.bsky.social at KubeCon Europe. And look whose quote showed up on the first page!

BildBild