What does this PR do?

This PR lets you save test_step predictions across multiple GPUs via a new function EvalResult.write(name, values, filename='predictions.txt').

Here's what it looks like:

class MyModel(pl.LightningModule):
    ...
    def test_step(self, batch, batch_idx):
        x, y = batch
        logits, loss = self(x)
        preds = torch.argmax(logits, dim=1)
        result = pl.EvalResult()
        result.log('test_loss', loss)
        result.write('preds', preds, filename='./predictions.txt')
        return result

• If you don't call .write() on your EvalResult objects, nothing should happen.
• If you do, it'll save at the files you specified.
• If on multiple GPUs, it should prepend your filename with the rank, leaving you with n_gpus number of prediction files.
  • For example, if running on two GPUs, test dataset size of 10,000, and your prediction file is the default of predictions.txt, you'll end up with:
    • predictions_rank_0.txt --> 5000 predictions
    • predictions_rank_1.txt --> 5000 predictions

Its basically going to take whatever you send to .write() and torch.cat/list.extend it with existing data with matching filename/name keys. In the background, the dictionary we're creating looks like this right now:

_predictions = {
    # The default file that predictions are saved to when you call .write
    'predictions.txt': {
        'preds': ['cat', 'dog', 'horse'],
        'idxs': [0, 1, 2]
    },

    # Optionally, if the predictions you want to save have different lengths, you can add more files
    'others.txt': {
        'something': ['wow', 'so', 'cool', 'dont', 'you', 'think?'],
        'idxs': [0, 1, 2, 3, 4, 5]
    }
}

Fixes # (issue)

Before submitting

• Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure your PR does only one thing, instead of bundling different changes together? Otherwise, we ask you to create a separate PR for every change.
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?
• Did you verify new and existing tests pass locally with your changes?
• If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃