合作机构:阿里云 / 腾讯云 / 亚马逊云 / DreamHost / NameSilo / INWX / GODADDY / 百度统计
内存优化是一个经典问题,在看具体 K8S 做了哪些工作之前,可以先抽象一些这个过程,思考一下如果是我们的话,会如何来优化。这个过程可以简单抽象为外部并发请求从服务端获取数据,如何在不影响吞吐的前提下降低服务端内存消耗?一般有几种方式:
数据压缩在这个场景可能不适用,压缩确实可以降低网络传输带宽,从而提升请求响应速度,但对服务端内存的优化没有太大的作用。kube-apiserver 已经支持基于 gzip 的数据压缩,只需要设置 Accept-Encoding 为 gzip 即可,详情可以参考官网[1]介绍。
当然缓存序列化的结果适用于客户端请求较多的场景,尤其是服务端需要同时把数据发送给多个客户时,缓存序列化的结果收益会比较明显,因为只需要一次序列化的过程即可,只要完成一次序列化,后续给其他客户端直接发送数据时直接使用之前的结果即可,省去了不必要的 CPU 和内存的开销。当然缓存序列化的结果这个操作本身来说也是会占用一些内存的,如果客户端数量较少,那么这个操作可能收益不大甚至可能带来额外的内存消耗。kube-apiserver watch 请求就与这个场景非常吻合。
下文会就 kube-apiserver 中是如何就这两点进行的优化做一个介绍。
下文列出的时间线中的各种问题和优化可能而且有很大可能只是众多问题和优化中的一部分。
所以如果你不是在以 WebSocket 形式(默认使用 Http Transfer-Encoding: chunked)使用 watch,那么升级到 1.17 之后理论上就可以了。
图片
新增了 CacheableObject 接口,同时在所有 Encoder 中支持对 CacheableObject 的支持,如下
// Identifier represents an identifier.
// Identitier of two different objects should be equal if and only if for every
// input the output they produce is exactly the same.
type Identifier string
type Encoder interface {
...
// Identifier returns an identifier of the encoder.
// Identifiers of two different encoders should be equal if and only if for every input
// object it will be encoded to the same representation by both of them.
Identifier() Identifier
}
// CacheableObject allows an object to cache its different serializations
// to avoid performing the same serialization multiple times.
type CacheableObject interface {
// CacheEncode writes an object to a stream. The <encode> function will
// be used in case of cache miss. The <encode> function takes ownership
// of the object.
// If CacheableObject is a wrapper, then deep-copy of the wrapped object
// should be passed to <encode> function.
// CacheEncode assumes that for two different calls with the same <id>,
// <encode> function will also be the same.
CacheEncode(id Identifier, encode func(Object, io.Writer) error, w io.Writer) error
// GetObject returns a deep-copy of an object to be encoded - the caller of
// GetObject() is the owner of returned object. The reason for making a copy
// is to avoid bugs, where caller modifies the object and forgets to copy it,
// thus modifying the object for everyone.
// The object returned by GetObject should be the same as the one that is supposed
// to be passed to <encode> function in CacheEncode method.
// If CacheableObject is a wrapper, the copy of wrapped object should be returned.
GetObject() Object
}
func (e *Encoder) Encode(obj Object, stream io.Writer) error {
if co, ok := obj.(CacheableObject); ok {
return co.CacheEncode(s.Identifier(), s.doEncode, stream)
}
return s.doEncode(obj, stream)
}
func (e *Encoder) doEncode(obj Object, stream io.Writer) error {
// Existing encoder logic.
}
// serializationResult captures a result of serialization.
type serializationResult struct {
// once should be used to ensure serialization is computed once.
once sync.Once
// raw is serialized object.
raw []byte
// err is error from serialization.
err error
}
// metaRuntimeInterface implements runtime.Object and
// metav1.Object interfaces.
type metaRuntimeInterface interface {
runtime.Object
metav1.Object
}
// cachingObject is an object that is able to cache its serializations
// so that each of those is computed exactly once.
//
// cachingObject implements the metav1.Object interface (accessors for
// all metadata fields). However, setters for all fields except from
// SelfLink (which is set lately in the path) are ignored.
type cachingObject struct {
lock sync.RWMutex
// Object for which serializations are cached.
object metaRuntimeInterface
// serializations is a cache containing object`s serializations.
// The value stored in atomic.Value is of type serializationsCache.
// The atomic.Value type is used to allow fast-path.
serializations atomic.Value
}
TOP