--emb_layer_aggregation

Switch

--emb_layer_aggregation aggregation_method

Description

Specifies the aggregation method to apply over transformer layers to produce a single vector representation. The layers to aggregate over are specified with --emb_layers.

Argument and Default Value

Default: concatenate. Currently accepts only one aggregation method.

Details

This option is interpreted as a numpy method name, applied to the 0th axis.

Other Switches

Required Switches:

• -d, -t , -g

Optional Switches:

• --emb_model

• --emb_msg_aggregation

• --emb_layers

Example Commands

Creates a Transformers feature table summed over the last two layers.

dlatkInterface.py -d dla_tutorial -t msgs -c user_id --add_emb_feat --emb_model bert-base-uncased --emb_layers 11 12 --emb_layer_aggregation sum