Cloud Security Alliance News Clipping Site

Tag: cache compression

Hacker News: Tencent drops a 389B MoE model(Open-source and free for commercial use))

Nov 5, 2024

—

by

system automation

in Uncategorized

Source URL: https://github.com/Tencent/Tencent-Hunyuan-Large Source: Hacker News Title: Tencent drops a 389B MoE model(Open-source and free for commercial use)) Feedly Summary: Comments AI Summary and Description: Yes **Summary:** The text introduces the Hunyuan-Large model, the largest open-source Transformer-based Mixture of Experts (MoE) model, developed by Tencent, which boasts 389 billion parameters, optimizing performance while managing resource…
The Cloudflare Blog: Making Workers AI faster and more efficient: Performance optimization with KV cache compression and speculative decoding

Sep 27, 2024

—

by

system automation

in Uncategorized

Source URL: https://blog.cloudflare.com/making-workers-ai-faster Source: The Cloudflare Blog Title: Making Workers AI faster and more efficient: Performance optimization with KV cache compression and speculative decoding Feedly Summary: With a new generation of data center accelerator hardware and using optimization techniques such as KV cache compression and speculative decoding, we’ve made large language model (LLM) inference lightning-fast…