首页 > itarticle > pytorch日常

pytorch日常

admin 11月 11, 2020 0

记录一些pytorch使用注意事项

BCE Loss

Binary Cross Entropy Loss，可用于多分类多标签任务。网络输出经过Sigmoid激活函数，得到预测值h（$mathbf{h}inmathbb{R}^d$），真实标签为y（$mathbf{y}inlbrace 0, 1rbrace^d$），那么BCELoss表示为：
$$
L_i=-left(y_ilog(h_i)+(1-y_i)log(1-h_i)right)
$$
pytorch中用torch.nn.BCELoss或torch.nn.BCEWithLogitsLoss函数计算。

1	torch.nn.BCELoss(weight=None, size_average=None, reduce=None, reduction='elementwise_mean')

Example

import torch
import torch.nn as nn
sigm = nn.Sigmoid()
loss = nn.BCELoss()
x = torch.FloatTensor([-2, 4, 0, 1])
>>> hyp = sigm(x)
>>> hyp
tensor([0.1192, 0.9820, 0.5000, 0.7311])
>>> y = torch.FloatTensor([0, 1, 0, 1])
>>> loss(hyp, y).item()
0.28787168860435486


>>> loss_no_reduc = nn.BCELoss(reduction='none')
>>> loss_no_reduc(hyp, y)
tensor([0.1269, 0.0181, 0.6931, 0.3133])
>>> loss_sum_reduc = nn.BCELoss(reduction='sum')
>>> loss_sum_reduc(hyp, y)
tensor(1.1515)

torch.nn.BCEWithLogitsLoss将Sigmoid层与BCELoss层合并起来，类似于CrossEntropyLoss与NLLLoss的差异。

1	torch.nn.BCEWithLogitsLoss(weight=None, size_average=None, reduce=None, reduction='elementwise_mean', pos_weight=None)

其中pos_weight给定正例的权重$p_i$，如下
$$
L_i=-left(p_icdot y_ilog(h_i)+(1-y_i)log(1-h_i)right)
$$