DiscReT: Discourse Relation tagging

The MELODI team submission for Task 3.

Contents

• mappings: a folder with the label conversions we implemented, and specifications on which test results are created from which of our models.
• pytorch_classifier.py: the bare classifier using mBERT-base-cased and built on Pytorch
• make_adapter.py: code to create a classifier adapter, based on AdapterHub
• adapter_classifier.py: classifier using one of the trained adapters (training the adapter beforehand is required)
• requirements.txt: list of dependencies
• train_classifiers.sh: shell script to train all classifiers
• configure.py: list of training arguments
• utils.py: various functions

Installation

• Pull data from the DISRPT Shared Task repository:

   git clone https://github.com/disrpt/sharedtask2023

• Install requirements, either:

   pip install -r requirements.txt

  or by making a conda environment:

   conda env create -f environment.yml
   conda activate discret 

Running classifiers

The results are created by three different models:

• the bare classifier: an mBERT-base-cased model (max. 6 epochs)
• the classifier with A1 adapter: an mBERT-base-cased model trained for 3 epochs with an adapter trained with mBERT-base-cased, for 15 epochs, with frozen layer 1
• the classifier with A1-3 adapter: an mBERT-base-cased model trained for 4 epochs with an adapter trained with mBERT-base-cased, for 15 epochs, with frozen layers 1-3

Run either the train_classifiers.sh script or each script individually (adapters must be trained beforehand):

Bare classifier

python pytorch_classifier.py \
		--num_epochs 6 \
		--data_path [PATH_TO_DATA]

Adapter training

A 1:

python make_adapter.py \
		--num_epochs 15 \
		--freeze_layers 'layer.1'
		--data_path [PATH_TO_DATA]

A 1-3:

python make_adapter.py \
		--num_epochs 15 \
		--freeze_layers 'layer.1;layer.2;layer.3'
		--data_path [PATH_TO_DATA]

Classifiers with adapter

with A 1:

python adapter_classifier.py \
		--num_epochs 3 \
		--data_path [PATH_TO_DATA] \
		--adapter_name 'adapter_15-epochs_frozen-1'

with A 1-3:

python adapter_classifier.py \
		--num_epochs 4 \
		--data_path [PATH_TO_DATA] \
		--adapter_name 'adapter_15-epochs_frozen-1-2-3'