TPE全称Tree-structured Parzen Estimator,是用GMM(Gaussian Mixture Model)来学习超参模型的一种方法。 首先把 Bayes 引入进来,p(x|y) 即模型 loss 为 y 的时候超参为 x 的条件概率。第一步,我们根据已有的数据选取一个 loss 的阈值 y*,比如按照中位数。对大于阈值和小于阈值的数据,分别学习两个概率密度 ...
TPE gets its name from two main ideas: 1. using Parzen Estimation to model our beliefs about the best hyperparameters (more on this later) and 2. using a tree-like data structure called a posterior-inference graph to optimize the algorithm runtime. In this example we will ignore the “tr...
A widely-used versatile HPO method is a variant of Bayesian optimization called tree-structured Parzen estimator (TPE), which splits data into good and bad groups and uses the density ratio of those groups as an acquisition function (AF). However, real-world applications often have some ...
Tree-structured Parzen estimator (TPE) is one of the most advanced BO algorithms based on tree-structured Parzen density estimation. First, the TPE algorithm defines two probability density functions (Equations (5) and (6)) based on 𝑦∗y* (i.e., the value at the quantile r in the ...