Data & Data Layers
Data Type
- Image Data - read raw images.
- Database - read data from LEVELDB or LMDB.
- HDF5 Input - read HDF5 data, allows data of arbitrary dimensions.
- HDF5 Output - write data as HDF5.
- Input - typically used for networks that are being deployed.
- Window Data - read window data file.
- Memory Data - read data directly from memory.
- Dummy Data - for static data and debugging.
Data Layers defination
Tops and Bottoms for data layers, only have top, no bottom.
Data and Label Data layer has two tops: data, label.
Transformations data preprocessing.
Prefetching: for throughput data layers fetch the next batch of data and prepare it in the background while the Net computes the current batch.
Deployment Input For on-the-fly computation deployment Nets define their inputs by input fields: these Nets then accept direct assignment of data for online or interactive computation.
生产部署，使用input参数。
Vison Layers
- Convolution Layer - convolves the input image with a set of learnable filters, each producing one feature map in the output image.
- Pooling Layer - max, average, or stochastic pooling.
Spatial Pyramid Pooling (SPP) - Crop - perform cropping transformation.
- Deconvolution Layer - transposed convolution.
- Im2Col - relic helper layer that is not used much anymore.
Recurrent Layers
- Recurrent
- RNN
- Long-Short Term Memory (LSTM)
Common Layers
- Inner Product - fully connected layer.
Dropout - Embed - for learning embeddings of one-hot encoded vector (takes index as input).
Normalization Layers
- Local Response Normalization (LRN) - performs a kind of “lateral inhibition” by normalizing over local input regions.
- Mean Variance Normalization (MVN) - performs contrast normalization / instance normalization.
- Batch Normalization - performs normalization over mini-batches.
The bias and scale layers can be helpful in combination with normalization.
Activation / Neuron Layers
- ReLU / Rectified-Linear and Leaky-ReLU - ReLU and Leaky-ReLU rectification.
- PReLU - parametric ReLU.
- ELU - exponential linear rectification.
- Sigmoid
- TanH
- Absolute Value
- Power - f(x) = (shift + scale * x) ^ power.
- Exp - f(x) = base ^ (shift + scale * x).
- Log - f(x) = log(x).
- BNLL - f(x) = log(1 + exp(x)).
- Threshold - performs step function at user defined threshold.
- Bias - adds a bias to a blob that can either be learned or fixed.
- Scale - scales a blob by an amount that can either be learned or fixed.
Utility Layers
Loss Layers
Solver
The solver orchestrates model optimization by coordinating the network’s forward inference and backward gradients to form parameter updates that attempt to improve the loss. The responsibilities of learning are divided between the Solver for overseeing the optimization and generating parameter updates and the Net for yielding loss and gradients.
The Caffe solvers are:
- Stochastic Gradient Descent (type: "SGD"),
- AdaDelta (type: "AdaDelta"),
- Adaptive Gradient (type: "AdaGrad"),
- Adam (type: "Adam"),
- Nesterov’s Accelerated Gradient (type: "Nesterov") and
- RMSprop (type: "RMSProp")
近期评论