Latte.Net — Type.
ensembles– a vector containing each
Ensemblein the network.
ensembles_map– a mapping between ensemble names in the network to the instance stored in
buffers– the internal buffers used by the network
curr_buffer_set– the current buffer set (used for double buffering)
forward_tasks– a set of tasks for performing forward propogation of the network
backward_tasks– a set of tasks for performing back propogation of the network
update_tasks– a set of tasks for performing parameter updates
params– a vector of
Paraminstances corresponding to network parameters
batch_size– the batch size of the network TODO: support different batch sizes for train/test
train_epoch– number of epochs completed for training
test_epoch– number of epochs completed for testing
curr_time_step– the current time step (for RNNs)
time_steps– total number of time steps to unroll (for RNNs)
num_subgroups– number of partitions in network (for model parallelism)
ensemble_send_list– a mapping between ensembles and a list of subgroups to send values to, used internally when synthesizing code for model parallelism
Latte.Net — Method.
Net constructor that should be used. Params -
batch_size – batch size of the network -
time_steps – number of time steps to unroll the network (for RNNs) -
num_subgroups – number of subgroups in the network (for model parallelism)
Latte.Ensemble — Type.
- name – name of the ensemble
- neurons – an array of neurons of type
- connections – a list of
Ensembles connected to this ensemble
- batch_fields – a vector of
- arg_dim_info –
- params – a vector of
Params associated with the ensemble
- phase – phase(s) in which this ensemble is active
- net_subgroup – the net subgroup the ensemble is a member of (use for model parallelism)
Latte.Connection — Type.
A connection between two ensembles.
- source – the source
- mapping – a mapping function to a range of neurons in
- shape – shape of the connected neurons returned by
- size – length of the connected neurons returned by
- copy – whether the connection requires input values to be copied
- is_dim_fixed – vector of booleans that are true if the connection is fixed along a dimension
- is_one_to_one – whether the connection is one to one
- padding – amount of padding used for the connection
- recurrent – whether the connection is recurrent
Latte.Param — Type.
A parameter in a
Net (learned during training)
name– the name of the parameter
gradient_name– the name of the gradient (should be ∇
hist_name– the name of the history buffer (should be
- learning_rate – local learning rate for the parameter
- regu_coef – local regularization coefficient
- clip_gradients – NOT IMPLEMENTED, gradient clipping parameter
- value – buffer containing the value of the parameter
- gradient – buffer containing the gradient of the parameter
- hist – buffer containing the history of the parameter
- request – request id, used for MPI data parallelism
Latte.get_buffer — Method.
Get a buffer at the time_step
net– network to get buffer
name– name of the buffer
t– time step
Latte.add_connections — Method.
Connect neurons in
source to neurons in
sink using the function
mapping should be a function with a parameter for the index in each dimension of sink. For example, if sink is a 3-d ensemble, mapping = f(i, j, k) -> ...
mapping should return a tuple of continuous ranges corresponding to the indices of neurons in source that should be connected to the neuron at the current index
Latte.add_ensemble — Method.
Add an ensemble to the network
Latte.init — Method.
Initialize the network
This function begins by initializing each
Ensemble in the Network. This is done by calling
init(ens) which will be dispatched to the appropriate initialization routine. These initialization routines are responsible for adding buffers to the network to contain neuron fields and output values. See the specific initialization functions for different Ensemble types for more information (TODO: Reference these).
After initializaing the local fields and output values for each ensemble, we then initialize the input buffers for each ensemble. This is done after all output values have been initialized so that we can analyze recurrent connections. This is done by calling the
Latte.load_snapshot — Method.
Load a network snapshot from
TODO: Can we save the structure of
net in the snapshot?
Latte.save_snapshot — Method.
Save a snapshot of
Latte.test — Method.
net for one epoch