Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/Multiple trainers for MA-DDPG #253

Merged
merged 164 commits into from
Oct 25, 2021
Merged

Feature/Multiple trainers for MA-DDPG #253

Changes from 1 commit
Commits
Show all changes
164 commits
Select commit Hold shift + click to select a range
f543f5f
feature: Save work on seperate variable source.
DriesSmit Jun 15, 2021
5333e74
fix: A general inter-node variable communicator module is now impleme…
DriesSmit Jun 16, 2021
ab10d87
fix: Cleanup variable_utils and some other files.
DriesSmit Jun 16, 2021
a84946e
Merge remote-tracking branch 'origin' into feature/mava-scaling
DriesSmit Jun 16, 2021
329f95f
Merge remote-tracking branch 'origin/develop' into feature/mava-scaling
DriesSmit Jun 18, 2021
5d93108
Add scaled mad4pg example.
DriesSmit Jun 18, 2021
58926f5
Merge remote-tracking branch 'origin/develop' into feature/mava-scaling
DriesSmit Jun 24, 2021
0a1119b
feature: Save latest code.
DriesSmit Jun 25, 2021
e259dd1
feature: Save latest code.
DriesSmit Jun 25, 2021
e9d6a55
fix: A lot of bugfixes.
DriesSmit Jun 25, 2021
9f2f2cd
fix: A lot of bugfixes.
DriesSmit Jun 25, 2021
6648245
fix: Last save for today.
DriesSmit Jun 25, 2021
6cda7fc
fix: Fix code so that other algorithms can still run in their normal …
DriesSmit Jun 25, 2021
bc2a013
Fix some bugs in the debugging 3 trainer example.
DriesSmit Jun 28, 2021
acbfd4e
fix: Fix some more bugs.
DriesSmit Jun 28, 2021
9aa7834
Fix makefile.
DriesSmit Jun 28, 2021
a678bf3
fix: First attempt running.
DriesSmit Jun 28, 2021
e5037f9
fix: Fix bug where trainers did not update variable source.
DriesSmit Jun 29, 2021
ad76e16
Merge remote-tracking branch 'origin/develop' into feature/mava-scaling
DriesSmit Jun 29, 2021
ad3a184
fix: Small fix.
DriesSmit Jun 29, 2021
8c159b9
fix: Resolve merge conflict.
DriesSmit Jun 30, 2021
fb29faa
fix: Fix environment_loop for when no variable_client is presented.
DriesSmit Jun 30, 2021
0607dca
fix: Fix error where no count variable crashes the environment_loop.
DriesSmit Jun 30, 2021
efec9d4
fix: Small fix.
DriesSmit Jun 30, 2021
ce939e7
fix: Small fix.
DriesSmit Jun 30, 2021
2bf2e46
Fix: Small fix.
DriesSmit Jun 30, 2021
28cb0e9
Merge remote-tracking branch 'origin/develop' into feature/mava-scaling
DriesSmit Jun 30, 2021
8b24be2
Merge branch 'bugfix/logging-running-stats' into feature/mava-scaling
DriesSmit Jun 30, 2021
e9fc43e
Merge remote-tracking branch 'origin/develop' into feature/mava-scaling
DriesSmit Jun 30, 2021
7a44219
fix: Fix environment loop.
DriesSmit Jun 30, 2021
2be9993
fix: Remove comment.
DriesSmit Jun 30, 2021
e235a66
fix: Small fixes.
DriesSmit Jun 30, 2021
d318c8c
fix: Small fixes.
DriesSmit Jun 30, 2021
6832841
Start with network_key conversion
DriesSmit Jul 1, 2021
6d25b66
fix: Update scaled example networks.
DriesSmit Jul 1, 2021
bc87cdc
fix: Fix scaled maddpg system.
DriesSmit Jul 2, 2021
1a06216
Merge network-keys branch.
DriesSmit Jul 2, 2021
0263b54
fix: Net key bugfixes in scaled maddpg system.
DriesSmit Jul 2, 2021
c7f8d72
Start with setting up population based training example.
DriesSmit Jul 2, 2021
37fa724
Merge feature/network-keys.
DriesSmit Jul 2, 2021
d1e8d89
Merge feature/mava-scaling.
DriesSmit Jul 2, 2021
a49e84e
Merge branch feature/network-keys.
DriesSmit Jul 3, 2021
8b468e0
Merge branch 'feature/mava-scaling' into feature/population-based-tra…
DriesSmit Jul 3, 2021
07ead24
Small changes.
DriesSmit Jul 5, 2021
f09e2c1
Save changes.
DriesSmit Jul 5, 2021
bd4aafc
Merge branch 'feature/network-keys' into feature/population-based-tra…
DriesSmit Jul 5, 2021
9dcbb8f
feature: Update decentralised architecture to work for both methods o…
DriesSmit Jul 6, 2021
3d1c2dd
block: PBT is blocked for the moment as the reverb version must first…
DriesSmit Jul 7, 2021
0424702
Merge remote-tracking branch 'origin/develop' into feature/population…
DriesSmit Jul 7, 2021
9791529
Merge remote-tracking branch 'origin/develop' into feature/mava-scaling
DriesSmit Jul 7, 2021
ab1586f
Merge branch 'feature/network-keys' into feature/mava-scaling
DriesSmit Jul 7, 2021
fc63367
fix: Delete shared weights for network_factory in examples.
DriesSmit Jul 7, 2021
ee4f97d
Merge remote-tracking branch 'origin/feature/network-keys' into featu…
DriesSmit Jul 7, 2021
8c04e74
feature: Add better function descriptions.
DriesSmit Jul 7, 2021
7e82914
Merge feature/mava-scaling.
DriesSmit Jul 7, 2021
0ad3cc1
Merge acme adder update branch.
DriesSmit Jul 12, 2021
8ac2ab2
Merge remote-tracking branch 'origin/develop' into feature/mava-scaling
DriesSmit Jul 12, 2021
2ee0bf2
Merge feature/network-keys.
DriesSmit Jul 12, 2021
19aaa98
feature: Delete original maddpg and rename maddpg_scaled to maddpg. f…
DriesSmit Jul 13, 2021
c222bf9
fix: Add deleted README back again.
DriesSmit Jul 13, 2021
c12cead
Merge branch 'feature/network-keys' into feature/mava-scaling
DriesSmit Jul 13, 2021
bf12cdf
fix: Small fix to MAD4PG executor.
DriesSmit Jul 13, 2021
dbedde8
Merge feature/mava-scaling.
DriesSmit Jul 13, 2021
ac5dff3
Merge remote-tracking branch 'origin/feature/upgrade-acme-version' in…
DriesSmit Jul 13, 2021
6706068
fix: General updates to adders for PBT.
DriesSmit Jul 13, 2021
6ecae40
fix: Fix trainer for PBT.
DriesSmit Jul 14, 2021
62b1a37
Merge branch 'develop' into feature/mava-scaling
DriesSmit Jul 14, 2021
e656ef0
fix: Correct table names in builder and system.
DriesSmit Jul 14, 2021
1f8c422
fix: A few small bugfixes. The PBT general frameworks seems to be wor…
DriesSmit Jul 15, 2021
e0ae92a
fix: Small fixes to mad4pg.
DriesSmit Jul 16, 2021
26a284e
Save code
DriesSmit Jul 19, 2021
b7ab9c6
Save code
DriesSmit Jul 19, 2021
cb7c6b1
Save work
DriesSmit Jul 19, 2021
633774c
Merge adders branch.
DriesSmit Jul 19, 2021
5d363b2
Save changes
DriesSmit Jul 20, 2021
96f2bb1
Merge uprade-acme-version.
DriesSmit Jul 20, 2021
25ca993
Save changes.
DriesSmit Jul 20, 2021
401dfbc
fix: Small fixes to PBT setup.
DriesSmit Jul 20, 2021
3ff3ffb
fix: Small fixes to PBT setup.
DriesSmit Jul 20, 2021
bb91b79
Merge branch 'develop' into feature/mava-scaling
DriesSmit Jul 20, 2021
8b55ac9
fix: Implement custom sort algorithm.
DriesSmit Jul 21, 2021
b6e6495
Merge branch 'develop' into feature/mava-scaling
DriesSmit Jul 21, 2021
c51d19b
Start work on scaled recurrent setup.
DriesSmit Jul 22, 2021
bfa15bf
Merge develop.
DriesSmit Jul 22, 2021
1a258f0
Merge upgrade-acme branch.
DriesSmit Jul 22, 2021
5cd4257
Merge branch 'feature/population-based-training' into feature/mava-sc…
DriesSmit Jul 22, 2021
8dbf5a4
Merge upgrade-acme branch.
DriesSmit Jul 22, 2021
d54ee4c
Add old maddpg system for debugging purposes.
DriesSmit Jul 22, 2021
00e35c4
fix: Fix recurrent maddpg/mad4pg.
DriesSmit Jul 22, 2021
542ed65
fix: Fix run_maddpg.py.
DriesSmit Jul 22, 2021
56eeb95
Merge remote-tracking branch 'origin/develop' into feature/mava-scaling
DriesSmit Jul 22, 2021
1da03d7
fix: SMAC imports.
DriesSmit Jul 22, 2021
fd4a255
fix: Fix mypy issues.
DriesSmit Jul 22, 2021
19b1903
Fix MA-DDPG executor to work for non indentity observation networks.
DriesSmit Jul 23, 2021
7b31b55
Reduce variable_server checkpoint waiting time.
DriesSmit Jul 24, 2021
16b8c7b
Fix .keys() and .values() error.
DriesSmit Jul 24, 2021
2bfedce
fix: Small fixes.
DriesSmit Jul 26, 2021
4cb7d99
update: Change supersuit import so that it does not break Mava if the…
DriesSmit Jul 26, 2021
0562341
update: Update sequential and transition adder with better machanics.…
DriesSmit Jul 27, 2021
b86c8a8
fix: Fix adders to be backwards compatible.
DriesSmit Jul 27, 2021
b3331d5
fix: Small fixes.
DriesSmit Jul 27, 2021
c12f85d
fix: Small fixes.
DriesSmit Jul 27, 2021
0084f75
fix: Small fix.
DriesSmit Jul 27, 2021
d6edc46
Small fixes.
DriesSmit Jul 27, 2021
4e8981e
Merge develop.
DriesSmit Jul 28, 2021
df23cb1
fix: is Dict bugs.
DriesSmit Jul 28, 2021
bfe6946
Merge develop.
DriesSmit Aug 3, 2021
167ad44
Address PR comments.
DriesSmit Aug 3, 2021
1a86fd0
Address PR comments.
DriesSmit Aug 3, 2021
b849293
Merge branch 'develop' into feature/mava-scaling
DriesSmit Aug 4, 2021
182f1e9
Merge branch 'develop' into feature/mava-scaling
DriesSmit Aug 6, 2021
c98e460
Adders PR comments.
DriesSmit Aug 6, 2021
12da764
Update mava scalling example.
DriesSmit Aug 6, 2021
3e6a006
Merge branch 'develop' into feature/mava-scaling
DriesSmit Aug 9, 2021
1b70136
Fix slow training times.
DriesSmit Aug 12, 2021
dc9348d
Add some more async methods in the trainer.
DriesSmit Aug 12, 2021
00361fa
Add tf.function to wrapper steps.
DriesSmit Aug 12, 2021
34b0925
fix: Fix the evaluator and executor loggers.
DriesSmit Aug 18, 2021
55e4f2b
fix: Small changes.
DriesSmit Aug 18, 2021
2341d10
Merge develop.
DriesSmit Aug 19, 2021
f0ef242
Add init.
DriesSmit Aug 21, 2021
9842000
Merge Develop. Update MAD4PG vmin and vmax parameters.
DriesSmit Aug 24, 2021
5244bde
Small fixes.
DriesSmit Aug 24, 2021
acb127c
Small fixes.
DriesSmit Aug 24, 2021
7ce2141
Fix tests.
DriesSmit Aug 24, 2021
f7c0bcd
Fix tests.
DriesSmit Aug 24, 2021
16903f8
Fix tests.
DriesSmit Aug 24, 2021
8bf30f3
Merge remote-tracking branch 'origin/develop' into feature/mava-scaling
DriesSmit Aug 26, 2021
8a5681a
Update the base.py adder file with some comments.
DriesSmit Aug 30, 2021
fa59d5b
Update multiple trainer example.
DriesSmit Aug 30, 2021
852da27
Merge branch 'develop' into feature/mava-scaling
DriesSmit Aug 30, 2021
3cc5831
Merge develop.
DriesSmit Aug 31, 2021
a74ab9f
Added enums for the trainer setup to make it easier to setup multiple…
DriesSmit Sep 9, 2021
a6b8e89
Merge remote-tracking branch 'origin/develop' into feature/mava-scaling
DriesSmit Sep 9, 2021
f4fb448
Convert network_sampling_setup enums.
DriesSmit Sep 9, 2021
70717da
Made a lot of corrections to the docstrings.
DriesSmit Sep 9, 2021
7a94465
Minor corrections to docstrings
arnupretorius Sep 9, 2021
4d455b1
Address PR comments.
DriesSmit Sep 10, 2021
6bf7623
Small fix.
DriesSmit Sep 10, 2021
e478f27
fix: Address PR comment.
DriesSmit Sep 10, 2021
4a160c1
Small fix to the checkpointer.
DriesSmit Sep 10, 2021
163a1d7
Small fix to trainer_networks setup.
DriesSmit Sep 10, 2021
4663856
Address PR comments.
DriesSmit Sep 11, 2021
266db23
Address PR comments.
DriesSmit Sep 14, 2021
3bb8a3c
Add termination_conditions.
DriesSmit Sep 14, 2021
41429a4
Change write flush back.
DriesSmit Sep 15, 2021
ed271be
Address PR comments.
DriesSmit Sep 15, 2021
2a0d7c1
Update trainer naming to match dev branch for one trainer.
DriesSmit Sep 16, 2021
91bc297
Quick fix.
DriesSmit Sep 16, 2021
7b0db2a
Small updates to make the trainers and executors faster.
DriesSmit Sep 17, 2021
e478da0
Small fixes.
DriesSmit Sep 17, 2021
df9ea64
Small fix to the get and set async functions.
DriesSmit Sep 18, 2021
bd97bcc
Add a buffer to the async_add function if it is called to often.
DriesSmit Sep 18, 2021
0bdb98b
Fix mypy issue?
DriesSmit Sep 21, 2021
d1a51df
Small changes.
DriesSmit Sep 30, 2021
2a82347
Fix error where get variables was not getting pulled in the trainer.
DriesSmit Oct 1, 2021
b4f664f
Remove sigma.
DriesSmit Oct 1, 2021
62cd1fa
Add state based critic with one one action.
DriesSmit Oct 4, 2021
71e4117
Small fixes.
DriesSmit Oct 5, 2021
19a867e
Fix problem with tf.function and num_step not incrementing.
DriesSmit Oct 7, 2021
d3a272f
Address PR comments.
DriesSmit Oct 7, 2021
e39daca
Small fix.
DriesSmit Oct 7, 2021
fb97273
Add comments on multiple trainer being experimental.
DriesSmit Oct 25, 2021
be2a4df
Merge develop.
DriesSmit Oct 25, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix: Small fix.
  • Loading branch information
@DriesSmit
DriesSmit committed Jun 30, 2021
commit ce939e751143964a93d0f78392a76cd2f63e92f6
2 changes: 1 addition & 1 deletion mava/environment_loop.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -426,7 +426,7 @@ def run_episode(self) -> loggers.LoggingData:
"steps_per_second": steps_per_second,
}

if not hasattr(self, "counts"):
if hasattr(self, "counts"):
result.update(counts)

return result
Expand Down