This repository contains the code for the experiments and algorithm from the paper Erasure of Unaligned Attributes from Neural Representations (appears at TACL 2023).

We present the Assignment-Maximization Spectral Attribute removaL (AMSAL) algorithm, which aims at removing information from neural representations when the information to be erased is implicit rather than directly being aligned to each input example.

It first loops between two Alignment and Maximization steps, during which it finds an alignment (A) based on existing projections and then projects the representations and guarded attributes into a joint space based on an existing alignment (M). After these two steps are iteratively repeated and an alignment is identified, the algorithm takes another step to erase information from the input representations based on the projections identified.

We use the Dataset and Experimental Settings from the paper

The external datasets required by the projects above

Each dataset should be downloaded to the specified path, relative to the root directory of the project.

This dataset and model size

Suppose each dataset are downloaded or created to the specified path.

Move the script file under the folder of script_glove to the root of matlab. Then call the script file to run AMSAL with specified hyperparameters, or call another script file to queue the experimental scripts automatically by the following code.

./src/matlab/matscript.sh

Please note: you need to change the start index and end index in matscript.sh, depends on how many matlab jobs you want to submit at once.

Check the spreadsheet created under the folder data, named full.xlsx and epoch.xlsx.

Move the script files under the folder of biography to the root of matlab, then run AMSAL with hyperparameters specified in the scripts by calling the scripts one by one or automatically queueu the jobs by calling the following code.

./src/matlab/matscript.sh

Check the result in the spreadsheet created under the folder of data, named full.xlsx and epoch.xlsx.

Please also check the script files under the folder of BertModel_PARTIAL and FastTextModel_PARTIAL to run the partially supervised assignment of AMSAL.

Run the script files to debias the neural representations of the biographies on genders, and perform the profession classifications.

./src/assignment/tpr-gap_biasbios.sh

./src/assignment/export_tpr-gap_biography.sh

The tables will be stored in the folder of tables and the overleaf code will be stored in the folder of biography.

./src/bias-bench/batch_jobs/create_data.sh

Move the script files under the folder of biasbench to the root of matlab, then run AMSAL with hyperparameters specified in the scripts by calling the scripts one by one or automatically queueu the jobs by calling the following code.

./src/matlab/matscript.sh

Check the result in the spreadsheet created under the folder of data, named full.xlsx and epoch.xlsx.

# Compute the projection matrix by SAL
./src/bias-bench/batch_jobs/sal_eigenvector.sh

# Compute the projection matrix by INLP
./src/bias-bench/batch_jobs/inlp_projection_matrix.sh

# Evaluates non-debiased models against StereoSet
./src/bias-bench/batch_jobs/stereoset.sh

# Evaluates debiased models against StereoSet
./src/bias-bench/batch_jobs/stereoset_debias.sh

# Summary the results
./src/bias-bench/batch_jobs/stereoset_evaluation.sh

# Export to LaTeX code
./src/bias-bench/batch_jobs/export_stereoset.sh

Please check the spreadsheet and results json file under the folder of stereoset, also the LaTex code stored in the folder of tables.

# Evaluates non-debiased models against CrowS-Pairs
./src/bias-bench/batch_jobs/crows.sh

# Evaluates debiased models against CrowS-Pairs
./src/bias-bench/batch_jobs/crows_debias.sh

# Export to LaTeX code
./src/bias-bench/batch_jobs/export_crows.sh

Please check the results saved in the folder of CrowS-Pairs and LaTex code stored in the folder of tables.

# Evaluates non-debiased models against SEAT
./src/bias-bench/batch_jobs/seat.sh

# Evaluates debiased models against SEAT
./src/bias-bench/batch_jobs/seat_debias.sh

# Export to LaTeX code
./src/bias-bench/batch_jobs/export_seat.sh

Please check the results saved in the folder of SEAT and LaTex code stored in the folder of tables.

Very Complex, will be explained in a separate page

# Save the Twitter Sentiment (Political) datasets in npz and matlab readable format
# Twitter dataset with original labelling
./src/assignment/create_twitter_dataset_.sh

# Twitter dataset labelled by word overlap
./src/assignment/create_twitter_dataset_labelled_by_overlap_.sh

Deepmoji experiments consider the stereotypes of race in the twitters.

Move the script files under the folder of deepmoji to the root of matlab, then run AMSAL with hyperparameters specified in the scripts by calling the scripts one by one or automatically queueu the jobs by calling the following code.

./src/matlab/matscript.sh

Check the result in the spreadsheet created under the folder of data, named full.xlsx and epoch.xlsx.

Please also check the script files under the folders of R5S5, R5S8 and R8S5 to run the partially supervised assignment of AMSAL.

Run the script files to debias the neural representations of the biographies on genders, and perform the profession classifications.

./src/assignment/tpr-gap_deepmoji.sh

./src/assignment/export_tpr-gap_table-deepmoji.sh

The tables will be stored in the folder of tables and the overleaf code will be stored in the folder of deepmoji.