Add new local private network mode - devmode. devMode allows the developer to deploy a single node network, where every transaction being sent to the node automatically generates a new block.

This feature is focused primarily around 3rd parties that want to test their solution on the Algorand platform, without waiting for the network to make progress.

commit 9f4707617bcaeb991ff5ab3a1aa97595610b8051
Merge: fec01759 9982bb7
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Mon Jul 12 15:21:24 2021 -0400

    Merge branch 'feature/txnsync' into tsachi/grouphash

commit fec0175913accc7364a779339090824ae8a07050
Merge: b9ade6c0 2f2761a
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Mon Jul 12 14:32:58 2021 -0400

    Merge branch 'tsachi/merge15' into tsachi/grouphash

commit b9ade6c061322120584019bbcd45f18f25f84cdf
Merge: 5ad7c519 6fdf336
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Mon Jul 12 14:32:44 2021 -0400

    Merge branch 'feature/txnsync' into tsachi/grouphash

commit 2f2761a
Merge: 6fdf336 10e91da
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Mon Jul 12 12:58:30 2021 -0400

    Merge branch 'master' into tsachi/merge15

commit 10e91da
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Mon Jul 12 12:52:42 2021 -0400

    Improve testing of alloc bounded slices. (algorand#2515)

    This change is needed in order to support codec types such as the following:
    ```golang
    //msgp:allocbound typeA 16
    type typeA []typeB
    ```
    since we want the codec object randomizer to recognize that `typeA` has a allocbound defined, and use that bound.
    ( this would be used in the feature/txnsync branch, but the change doesn't really related to any of the other changes in the feature/txnsync branch and would be a good change regardless )

commit da26ca7
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Mon Jul 12 10:31:14 2021 -0400

    Make TestPeersDownloadFailed and TestHistoricData predictable (algorand#2516)

    Both the `TestPeersDownloadFailed` as well as `TestHistoricData` were randomly failing, as they were depending on a "genuine" random distribution of the random function in order to succeed. When that doesn't happen, they were failing.

    This PR make sure to "bake-in" the random seed so that the tests are repeatable and a success is repeatable as well.

commit 5ad7c519cdb018d95438905b514bde17066006f9
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Sat Jul 10 23:59:37 2021 -0400

    first draft

commit 415d82c
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Fri Jul 9 17:40:21 2021 -0400

    Fix random failure in TestWebsocketNetworkPrioLimit (algorand#2509)

    The peers array is modified when adding/removing entries from it. When that does happen, we increase the peersChangeCounter, so that the broadcast method would know that it's peers list need to be refreshed.
    The said update was missing from prioTracker.setPriority, which was causing the issue.

commit 66fbd60
Author: Brian Olson <brianolson@users.noreply.github.com>
Date:   Fri Jul 9 13:28:22 2021 -0400

    fix regex for matching charset of valid DNS hosts, add test (algorand#2505)

    A regex was trying to match valid DNS names but missed the '-' char. Fix.

commit 398d606
Author: John Jannotti <john.jannotti@algorand.com>
Date:   Fri Jul 9 13:22:53 2021 -0400

    Creator access and app_params_get (algorand#2301)

    Adds the ability to get the creator of apps and assets in teal.

    This is a simple new field for `asset_params_get`, but it introduces a new opcode `app_params_get` to enable access for apps.  Meanwhile `app_params_get` allows access to other global parameters about an app - the schema sizes, extra pages, and the programs themselves.

    This also begins work on LogicVersion=5, which should be AVM 1.0

commit 6cfcbeb
Author: John Jannotti <john.jannotti@algorand.com>
Date:   Fri Jul 9 13:22:21 2021 -0400

    selector pseudo-op in support of ABI (algorand#2358)

    Adds a Teal pseudo-op `selector` that assembles as if it were the `byte` pseudo-op, but stores 4 bytes of hash.

    This allows Teal, like:
    ```
    txn ApplicationArgs 0
    selector "add(uint64,uint64)uint128"
    ==
    bnz add
    ```

    and avoid the need to embed the actual hash in a .teal file (which would also require calculating it)

    Unlike `byte`, the argument *must* be a quoted string - no base64 or hex allowed, as the argument should be a method signature.

commit abc4058
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Thu Jul 8 12:43:34 2021 -0400

    testing: avoid division by zero during TestBasicCatchpointWriter (algorand#2502)

    The `randomFullAccountData` method was dividing by `lastCreatableID` which could be zero.
    The probability for that is pretty slim, but given that it was found during a travis run, we should fix it.

commit af8c2ce
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Thu Jul 8 12:42:58 2021 -0400

    disable TestAgreementSynchronous10 (algorand#2503)

    The test TestAgreementSynchronous10 is failing. We should be fixing it, but until we do that, I'm going to disable it so it won't mask other issues.

commit 3ac481e
Author: Brian Olson <brianolson@users.noreply.github.com>
Date:   Wed Jul 7 17:25:58 2021 -0400

    allow to parse ipv6 localhost "[::]:4601" (algorand#2430)

    Allow parsing of ipv6 localhost -colon- port address.
    "[::]:4601" was failing to parse. This and other forms pass ParseHostOrURL() now.

commit f09d814
Author: Will Winder <wwinder.unh@gmail.com>
Date:   Wed Jul 7 16:08:15 2021 -0400

    Fix 10 minute timeout in travis. (Revert parts of algorand#2324) (algorand#2494)

    We started seeing the 10-minute timeout error on travis after the recent changes to where travis_retry gets called.

    This means travis_wait is still needed. travis_wait and travis_retry don't play well together on the ephemeral build machine, so we basically need to rollback the entire change in algorand#2324

    Note: while making this change I noticed that we don't use travis_retry for all of the build_test.sh / integration_test.sh entries. Not sure why but I left them as they were before algorand#2324

commit d52eef5
Author: algonautshant <55754073+algonautshant@users.noreply.github.com>
Date:   Tue Jul 6 20:53:35 2021 -0400

    enable TestPartkeyOnlyRewards on macos (algorand#2429)

    TestPartkeyOnlyRewards was disabled on darwin.
    It is no longer failing on darwin (5/5 runs passed).

commit c4206f7
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Tue Jul 6 19:22:38 2021 -0400

    optimize txtail memory consumption (algorand#2413)

    The changes in this PR are as follows:

    The unused method Ledger. GetRoundTxIds was removed. As a result, the txTail. getRoundTxIds can be removed as well. This makes the txids map stored in the roundTxMembers structure redundant.
    In the cow.go, avoid adding empty leases to the cb.mods.Txleases map. Since we already not testing for empty leases, we can safely avoid storing them.
    Optimize the txTail. loadFromDisk to generate optimal lastValid map sizes.
    Optimize the txTail. loadFromDisk to avoid storing empty leases in the txleases map.

commit 2169e05
Merge: 6dbfd3c e4c7ecb
Author: John Lee <64482439+algojohnlee@users.noreply.github.com>
Date:   Tue Jul 6 17:17:44 2021 -0400

    Merge pull request algorand#2451 from Algo-devops-service/relstable2.7.1-remerge

    go-algorand relstable2.7.1-remerge

commit 6dbfd3c
Author: Jacob Daitzman <jdtzmn@gmail.com>
Date:   Tue Jul 6 17:16:29 2021 -0400

    Initialize past side effects with correct length during dryrun requests (algorand#2448)

    This PR fixes a typo in the dryrun code that caused certain dryrun requests containing multiple transactions to fail.

commit e4c7ecb
Author: DevOps Service <devops-service@algorand.com>
Date:   Tue Jul 6 21:13:45 2021 +0000

    Bump Version, Remove buildnumber.dat and genesistimestamp.dat files.

commit 3f6a279
Merge: 4f281b0 5e00bcd
Author: DevOps Service <devops-service@algorand.com>
Date:   Tue Jul 6 21:13:45 2021 +0000

    Merge remote-tracking branch 'origin/rel/stable' into relstable2.7.1-remerge

commit 4f281b0
Author: John Lee <john.lee@algorand.com>
Date:   Tue Jul 6 11:04:17 2021 -0400

    Fix docker repo update (algorand#2342)

    The scripts to update the docker repository do not do a full rebuild, and the submitted Dockerfile does not change. This means that if you run the docker image/repo update, it'll re-use the old image. To fix this, we call docker build with --no-cache. Other refactoring helps simplify the code.

    We add a new --cached flag to the docker/releases/build_releases.sh script to explicitly cache. Otherwise the testnet update would issue a full rebuild. We additionally automatically handle the 'latest' tagging in build_releases.sh as well.

commit 8f41556
Author: algonautshant <55754073+algonautshant@users.noreply.github.com>
Date:   Fri Jul 2 19:37:00 2021 -0400

    Fix and enable TestNewAccountCanGoOnlineAndParticipate (algorand#2238)

    * TestNewAccountCanGoOnlineAndParticipate was failing because the test was
    not waiting enough to get to the round where the newly funded account's
    funds will be considered for proposing purposes.

    It was miscalculating the round that it should wait form.

    Moreover, the rounds considered to when the account is funded was prone to
    race conditions.

    In addition, the test was using WaitForRoundWithTimeout which may be
    very slower if the current round is already ahead. Instead, now it is
    using ClientWaitForRound, which does not care about individual rounds delayed.

    * Addressing review commnets:
    - fixing a typo
    - getting exact transaction round for funding the account
    - testing exact blocks for the proposer
    - using a single node network instead of two nodes
    - waiting for exactly one round for the new account to propose and checking that
    - sending the funds and closing the rich account so there will be no possiblity of that proposing a block

commit e38dcff
Author: John Jannotti <john.jannotti@algorand.com>
Date:   Fri Jul 2 12:51:14 2021 -0400

    Specs and unit test to avoid forgetting in the future (algorand#2418)

    Specs and unit test to avoid forgetting in the future

commit 70d15fe
Author: Rakshith G <rakshith.gopalakrishna@algorand.com>
Date:   Fri Jul 2 06:30:56 2021 -0700

    testing: ensure deploy_linux_version generates a linux/amd64 binaries for algonet usage (algorand#2422)

    The deploy_linux_version.sh script currently creates a linux version of the current project tree with the same architecture as the hosting environment.

    When we attempt to builds on M1 Macs, it default to arm64 base images and as a result - makes a linux/arm64 compatible binaries. These resulting binaries, however, would not work correctly on an algonet deployed network, since the hosts there are amd64.

    To rectify that situation, we'll be changing the docker file to ensure linux/amd64 binaries are generated. This would ensure that the existing functionality works as intended. In the future, when we would add arm64 support for algonet, we could accompany that by creating corresponding images on docker as well.

commit fc52ab9
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Thu Jul 1 18:10:09 2021 -0400

    catchup: fix potential nil dereferencing (algorand#2420)

    The handing for returned peer was incorrect in case the getPeerErr is non-nil, as we would attempt to dereference the nil pointer.

commit 29a6fec
Author: bricerisingalgorand <60147418+bricerisingalgorand@users.noreply.github.com>
Date:   Thu Jul 1 17:39:03 2021 -0400

    Implement hello circleci yaml (algorand#2417)

    This adds a hello world circleci yaml. This can be used to run circleci jobs on PRs until the pipeline is fully implemented.

commit b0935e2
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Wed Jun 30 19:20:32 2021 -0400

    Add initial devMode support (algorand#2334)

    Add new local private network mode - devmode. devMode allows the developer to deploy a single node network, where every transaction being sent to the node automatically generates a new block.

    This feature is focused primarily around 3rd parties that want to test their solution on the Algorand platform, without waiting for the network to make progress.

commit 78e311e
Author: Pavel Zbitskiy <65323360+algorandskiy@users.noreply.github.com>
Date:   Wed Jun 30 18:28:25 2021 -0400

    Add missing txna allowed args to doc and langspec (algorand#2336)

    We exposed Assets and Applications in TEAL v3 but did not update doc/langspec. This commit fixes it.

commit 5e00bcd
Merge: 8fe22d4 dc829f6
Author: John Lee <64482439+algojohnlee@users.noreply.github.com>
Date:   Mon Jun 28 13:37:51 2021 -0400

    Merge pull request algorand#2332 from Algo-devops-service/relstable2.7.1

    go-algorand 2.7.1-stable

commit fbae428
Author: pzbitskiy <pavel.zbitskiy@gmail.com>
Date:   Mon Jun 28 09:26:55 2021 -0400

    Add Fedora support into install_linux_deps.sh (algorand#2331)

    Add Fedora deps into install_linux_deps.sh

commit f870265
Author: Will Winder <wwinder.unh@gmail.com>
Date:   Thu Jun 24 12:36:53 2021 -0400

    testing: fix telemetry unit tests (algorand#2321)

    Fix and re-enable async telemetry unit test TestAsyncTelemetryHook_CloseDrop.

commit b55b53f
Author: Will Winder <wwinder.unh@gmail.com>
Date:   Thu Jun 24 12:34:28 2021 -0400

    testing: move travis_retry to skip rebuilding (algorand#2324)

    Remove the top-level retry command. This will allow build / lint failures to terminate the build with no retry, and may speed up test failure retries.

commit 90edab3
Author: Will Winder <wwinder.unh@gmail.com>
Date:   Thu Jun 24 12:32:57 2021 -0400

    testing: fix gotestsum install (algorand#2328)

    testing: fix gotestsum install

    avoid installing swagger when not needed.

commit 798c612
Author: algonautshant <55754073+algonautshant@users.noreply.github.com>
Date:   Wed Jun 23 21:58:16 2021 -0400

    testing: enable previously disabled TestConfigMigrate unit test (algorand#2326)

    testing: enable previously disabled TestConfigMigrate unit test

commit 93f2586
Author: John Lee <john.lee@algorand.com>
Date:   Wed Jun 23 16:57:58 2021 -0400

    Restore TestPeriodicSync and TestBasicCatchpointCatchup tests. (algorand#2315)

commit ed9609d
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Wed Jun 23 16:40:05 2021 -0400

    Improve TestMetricSegment test realibility (algorand#2322)

    mprove TestMetricSegment test reliability by repeating the test with incrementing time delays.
    This implementation would allow faster execution on faster platforms, and allow fallback for slower platforms.

commit 1cd8dd4
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Wed Jun 23 14:32:59 2021 -0400

    Reduce unneeded contention around checking the peers connectivity. (algorand#2319)

    Existing `messageHandlerThread` was checking that all the connected peers are properly communicating by examining their recent message timings. This implementation served us well - however, it was executed redundantly. How much redundantly ? 19 times too many every 3 minutes ( and all of them at the **exact** same time.. ).

    This PR ensures that all the `messageHandlerThread` shares the same ticker for testing the `checkPeersConnectivity`. This is expected to reduce the pressure on the internal `peersLock`.

commit 63b8241
Author: chris erway <51567+cce@users.noreply.github.com>
Date:   Tue Jun 22 21:07:24 2021 -0400

    Run misspell linter with -w flag (algorand#2320)

    This runs the popular misspell linter with the -w flag to automatically correct spelling mistakes in go-algorand.

commit dc829f6
Merge: d57586a 0b18c94
Author: John Lee <64482439+algojohnlee@users.noreply.github.com>
Date:   Mon Jun 21 21:03:29 2021 -0400

    Merge pull request algorand#2314 from onetechnical/onetechnical/relbeta2.7.1

    go-algorand 2.7.1-beta

commit 0b18c94
Author: John Lee <john.lee@algorand.com>
Date:   Mon Jun 21 18:01:51 2021 -0400

    Bump buildnumber.dat

commit 855fac7
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Mon Jun 21 17:18:16 2021 -0400

    Merge pull request algorand#2313 from algorandskiy/pavel/extra-pages-api

    REST API: make extra-program-pages and apps-total-extra-pages optional

commit 708581b
Merge: 2cd04d6 cff7771
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Mon Jun 21 17:18:16 2021 -0400

    Merge pull request algorand#2313 from algorandskiy/pavel/extra-pages-api

    REST API: make extra-program-pages and apps-total-extra-pages optional

commit 2cd04d6
Merge: 5d01664 6edb3f8
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Mon Jun 21 15:05:29 2021 -0400

    Merge pull request algorand#2309 from figurestudios/patch-1

    fix few README typos, grammar and inconsistencies

commit 5d01664
Merge: 2824120 8faf82a
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Mon Jun 21 15:01:09 2021 -0400

    Merge pull request algorand#2308 from fabrice102/patch-3

    Specify truncated division is used in TEAL
    Specify that the truncated division is used (https://en.wikipedia.org/wiki/Modulo_operation#Variants_of_the_definition)

commit 2824120
Merge: 0cb300d 9d8a460
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Mon Jun 21 14:44:47 2021 -0400

    Merge pull request algorand#2285 from algonautshant/shant/fixTestBasicCatchpointCatchup

    <!--
    Thanks for submitting a pull request! We appreciate the time and effort you spent to get this far.

    If you haven't already, please make sure that you've reviewed the CONTRIBUTING guide:
    https://github.com/algorand/go-algorand/blob/master/CONTRIBUTING.md#code-guidelines

    In particular ensure that you've run the following:
    * make generate
    * make sanity (which runs make fmt, make lint, make fix and make vet)

    It is also a good idea to run tests:
    * make test
    * make integration
    -->

    ## Summary
    peerSelector.go: various bug fixes
    - introduce peerSelectorPeer to wrap the network.Peer and add peerClass information, to be able to distinguish between peers of the same address but different classes.
    - keep track of download failures to be able to exponentially increase the cost of each failure when failing more than succeeding. This is to evict the peer faster when constantly failing to download.
    - initialize rankSum and rankSamples to initialRank of the class. Otherwise, the peer rank will have a very long warmup time before relfecting the correct rank.
    - let resetRequestPenalty bound the rank within the class bounds. Otherwise, the penalty calculation pushes the rank out of the class bounds (bug).
    - getNextPeer, rankPeer, and peerDownloadDurationToRank are local to the package, since they are using non-exported peerSelectorPeer
    - getNextPeer, PeerDownloadDurationToRank and RankPeer use peerSelectorPeer instead of network.Peer
    - refreshAvailablePeers distinguishes between peers with the same address but of different peer class
    - findPeer returns the peer given the address and the peer class (instead of just the address)

    catchpointCatchup_test.go:
    - Remove comment about giving the second node all the stake, since it is not the case here.
    - Use the round from the catchpoint instead of guessing the round as 36. In case the following catchpoint was obtained due to race conditions, checking for round 37 will be trivial, since it will also be obtained from the catchpoint.

    catchpointService.go and service.go:
    - Update the code to use peerSelectorPeer instead of network.Peer with peerSelector

    peerSelector_test.go:
    - Add new tests to check the peerSelector fixes in this PR
    - Update the tests to use peerSelectorPeer instead of network.Peer with peerSelector
    - Cleanup debugging printouts.

    <!-- Explain the goal of this change and what problem it is solving. Format this cleanly so that it may be used for a commit message, as your changes will be squash-merged. -->

    ## Test Plan
    Added tests to confirm the fixes.
    - TestClassUpperBound
    - TestClassLowerBound
    - TestEvictionAndUpgrade

    <!-- How did you test these changes? Please provide the exact scenarios you tested in as much detail as possible including commands, output and rationale. -->

commit 0cb300d
Merge: 520e896 b27ab2c
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Mon Jun 21 14:31:25 2021 -0400

    Merge pull request algorand#2312 from algonathan/lruaccts-benchmark

    testing: improve lruaccts benchmark

commit cff7771
Author: Pavel Zbitskiy <pavel@algorand.com>
Date:   Mon Jun 21 14:05:20 2021 -0400

    REST API: make extra-program-pages and apps-total-extra-pages optional

    * This also helps in not exposing them before the protocol switch

commit b27ab2c
Author: Jonathan Weiss <jonathan.weiss@algorand.com>
Date:   Mon Jun 21 18:25:24 2021 +0300

    lruAccounts benchmark: a more controlled distribution between accounts in the benchmark

commit d8f5b33
Merge: 3b85cdb 520e896
Author: algonathan <85506383+algonathan@users.noreply.github.com>
Date:   Mon Jun 21 16:25:54 2021 +0300

    Merge branch 'algorand:master' into lruaccts-benchmark

commit 3b85cdb
Author: Jonathan Weiss <jonathan.weiss@algorand.com>
Date:   Mon Jun 21 16:22:51 2021 +0300

    lruAccounts write fix: benchmark used too much memory

commit 520e896
Merge: 9e07d6a ee3f371
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Mon Jun 21 08:58:19 2021 -0400

    Merge pull request algorand#2311 from algonathan/lruaccts-benchmark

    added benchmark to measure the performance of the `lruAccounts.write` function

commit ee3f371
Author: Jonathan Weiss <jonathan.weiss@algorand.com>
Date:   Sun Jun 20 19:02:57 2021 +0300

    fix: amount of accounts generated in lruAccounts write benchmark

commit b726844
Author: Jonathan Weiss <jonathan@Algo-Weiss-MBP.local>
Date:   Sun Jun 20 16:48:46 2021 +0300

    lruaccounts benchmark: filling the acounts with data before benchmarking

commit f2ca4fe
Author: Jonathan Weiss <jonathan@Algo-Weiss-MBP.local>
Date:   Sun Jun 20 09:05:27 2021 +0300

    added benchmark to lruAccounts write function

commit 8faf82a
Author: John Jannotti <jj@cs.brown.edu>
Date:   Sat Jun 19 17:53:29 2021 -0400

    Spec updates to go with division explanation.

commit 6edb3f8
Author: figurestudios <64747030+figurestudios@users.noreply.github.com>
Date:   Sat Jun 19 03:00:19 2021 +0200

    typos, grammar, inconsistencies

    mostly casing issues, ie, algorand > Algorand || sqlite > SQLite

commit 07154fd
Author: Fabrice Benhamouda <fabrice.benhamouda@normalesup.org>
Date:   Fri Jun 18 15:25:29 2021 -0400

    Specify truncated division is used in TEAL

    Specify that the truncated division is used (https://en.wikipedia.org/wiki/Modulo_operation#Variants_of_the_definition)

commit 9e07d6a
Merge: 91b8ca1 bc473af
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Thu Jun 17 16:44:19 2021 -0400

    Merge pull request algorand#2306 from algorandskiy/pavel/extra-pages-e2e-subs

    testing: move extra page test to e2e_subs

commit bc473af
Author: Pavel Zbitskiy <pavel@algorand.com>
Date:   Thu Jun 17 14:27:48 2021 -0400

    Move extra page test to e2e_subs

commit 91b8ca1
Merge: 9549171 77b6e11
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Thu Jun 17 13:59:06 2021 -0400

    Merge pull request algorand#2303 from cce/read-permutation-tests

    Add enums to player permutation tests:
    This made it easier for me to understand the different 7 initial player states * 14 simulated events when reading through each permutation's expected actions. I hope the enum names I picked are accurate — would appreciate any feedback on better names.

commit 9549171
Merge: 5cb9d3a 19490d4
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Thu Jun 17 13:44:08 2021 -0400

    Merge pull request algorand#2291 from winder/will/test-mac

    testing: enable unit tests on travis mac build

commit 77b6e11
Author: chris erway <chris.erway@algorand.com>
Date:   Thu Jun 17 13:08:28 2021 -0400

    Remove commented code in TestPlayerPermutation

commit 5b80276
Author: chris erway <chris.erway@algorand.com>
Date:   Thu Jun 17 11:08:30 2021 -0400

    make linter happy by removing underscores from enum names

commit 5cb9d3a
Merge: ebfc1fa 96f7b76
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Thu Jun 17 09:48:04 2021 -0400

    Merge pull request algorand#2265 from nicholasguoalgorand/nguo/blockqueue-cover

    testing: add unit tests for ledger BlockQueue

commit ebfc1fa
Merge: c43e226 6997588
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Thu Jun 17 09:46:07 2021 -0400

    Merge pull request algorand#2290 from grakshith/arm64-dev-build

    Fix go-algorand dev builds for darwin-arm64 machines (Apple's M1 CPUs).

commit c43e226
Merge: 65aa0da 957958a
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Thu Jun 17 09:45:09 2021 -0400

    Merge pull request algorand#2284 from nicholasguoalgorand/nguo/ledger-cover

    Add tests to increase coverage for onlineacct roundlru txtail

commit d57586a
Merge: 34f7106 78a221c
Author: John Lee <64482439+algojohnlee@users.noreply.github.com>
Date:   Thu Jun 17 02:43:18 2021 -0400

    Merge pull request algorand#2304 from Algo-devops-service/relbeta2.7.0

    go-algorand 2.7.0-beta

commit 78a221c
Author: DevOps Service <devops-service@algorand.com>
Date:   Thu Jun 17 04:36:40 2021 +0000

    Update the Version, BuildNumber, genesistimestamp.data

commit 65aa0da
Author: Tsachi Herman <tsachi.herman@algorand.com>
Date:   Thu Jun 17 00:31:07 2021 -0400

    testing: fix random failure in TestAppEmptyAccountsLocal (algorand#2302)

    The test had two unrelated bugs:
    1. We need to call `WaitForCommit` before `reloadLedger` to ensure the block is being written to disk before the blockQ getting reinitialized ( and loose its content ).
    2. The calculation of the total rewards unit in `makeNewEmptyBlock` was wrong. I corrected it. For tests that run only one or two rounds, this might be good enough, but for long-running tests, it would start fail pretty quickly.

commit 1e7c43b
Author: chris erway <chris.erway@algorand.com>
Date:   Wed Jun 16 23:46:16 2021 -0400

    rename requireTraceContainsAction to requireTraceContains

commit 11a4e39
Author: chris erway <chris.erway@algorand.com>
Date:   Wed Jun 16 17:29:36 2021 -0400

    add playerPermutation enums to permutation test

commit e8f9c70
Author: chris erway <chris.erway@algorand.com>
Date:   Wed Jun 16 16:48:46 2021 -0400

    use enums for message event permutations, add require trace helpers

commit 5fb7ac8
Author: Jason Paulos <jasonpaulos@users.noreply.github.com>
Date:   Wed Jun 16 13:24:11 2021 -0700

    Expose extra program pages to API (algorand#2294)

    Add the AppsTotalExtraPages account field to the response returned by the /v2/accounts/{addr} endpoint.

commit 0a3d9df
Author: John Jannotti <john.jannotti@algorand.com>
Date:   Wed Jun 16 15:24:46 2021 -0400

    Allow fee to be below minfee, if given explicitly. (algorand#2295)

    This makes other txns accept explicitly low fees.  Sorry for the code
    duplication, but I did not want to change libgoal's existing behavior
    where it increases fee to minfee.

commit 96f7b76
Author: Nicholas Guo <nicholas.guo@algorand.com>
Date:   Wed Jun 16 11:12:38 2021 -0700

    add another case

commit 37b828a
Author: John Jannotti <john.jannotti@algorand.com>
Date:   Wed Jun 16 11:16:37 2021 -0400

    Forgot to run .md generation (algorand#2292)

commit 1ca4ef8
Author: Pavel Zbitskiy <65323360+algorandskiy@users.noreply.github.com>
Date:   Wed Jun 16 11:14:56 2021 -0400

    Introduce V28 consensus version (algorand#2255)

    * TEAL v4
    * Larger programs
    * Larger app/asset lookup limits
    * Longer asset URL
    * Fee pooling within a group
    * Keyreg txn additional checks

    Removed InitialRewardsRateCalculation and PaysetCommit from vFuture since it is already in v26

    Fixed some tests after exposing zero fees and strict keyreg as current consensus

commit 19490d4
Author: Will Winder <wwinder.unh@gmail.com>
Date:   Wed Jun 16 10:54:40 2021 -0400

    More changes.

commit f937799
Author: Will Winder <wwinder.unh@gmail.com>
Date:   Wed Jun 16 06:18:39 2021 -0400

    Enable unit tests on mac build script.

commit f25c951
Author: Nicholas Guo <nicholas.guo@algorand.com>
Date:   Tue Jun 15 22:01:05 2021 -0700

    address comments

commit 6997588
Author: Rakshith Gopala Krishna <rakshith.gopalakrishna@algorand.com>
Date:   Tue Jun 15 15:04:05 2021 -0700

    Fix darwin-arm64 builds

commit 957958a
Author: Nicholas Guo <nicholas.guo@algorand.com>
Date:   Tue Jun 15 11:34:53 2021 -0700

    add check

commit 02eacd1
Author: John Jannotti <john.jannotti@algorand.com>
Date:   Tue Jun 15 14:31:23 2021 -0400

    report substring missing immediates properly (algorand#2287)

    Fixes crash that should be a clean error report for using substring wrong.

    Unit tests added to confirm and prevent regression.

commit f878967
Author: Nicholas Guo <nicholas.guo@algorand.com>
Date:   Tue Jun 15 11:15:32 2021 -0700

    fix tests

commit ff9f69d
Author: John Jannotti <john.jannotti@algorand.com>
Date:   Tue Jun 15 12:48:37 2021 -0400

    final v28 foundation spec (algorand#2286)

    Changes to readmes and specs to reflect the matching foundation spec.

commit 9d8a460
Author: algonautshant <shant@algorand.com>
Date:   Tue Jun 15 12:24:44 2021 -0400

    Fix the test name so it runs

commit 4aa0f48
Author: algonautshant <shant@algorand.com>
Date:   Tue Jun 15 02:08:47 2021 -0400

    test for error first

commit d8b73f8
Author: algonautshant <shant@algorand.com>
Date:   Tue Jun 15 02:06:04 2021 -0400

    Added tests, exponential increase of download failure impact, local functions.

commit eb6604c
Author: Nicholas Guo <nicholas.guo@algorand.com>
Date:   Mon Jun 14 22:28:51 2021 -0700

    rename test

commit 3eed93f
Author: Nicholas Guo <nicholas.guo@algorand.com>
Date:   Mon Jun 14 22:27:36 2021 -0700

    fix import

commit 3dc1512
Author: Nicholas Guo <nicholas.guo@algorand.com>
Date:   Mon Jun 14 22:09:44 2021 -0700

    write tests

commit a2be5f1
Author: algonautshant <shant@algorand.com>
Date:   Mon Jun 14 19:01:04 2021 -0400

    In this chage, fixes to peer selector and the test.

    peerSelector.go: various bug fixes
    - introduce peerSelectorPeer to wrap the network.Peer and add peerClass information, to be able to distinguish between peers of the same address but different classes.
    - keep track of download failures to be able to increase the cost of each failure when failing more than succeeding. This is to evict the peer faster when constantly failing to download.
    - initialize rankSum and rankSamples to initialRank of the class. Otherwise, the peer rank will have a very long warmup time before relfecting the correct rank.
    - let resetRequestPenalty bound the rank within the class bounds. Otherwise, the penalty calculation pushes the rank out of the class bounds (bug).
    - getNextPeer is local to the package
    - getNextPeer, PeerDownloadDurationToRank and RankPeer use peerSelectorPeer instead of network.Peer
    - refreshAvailablePeers distinguishes between peers with the same address but of different peer class
    - findPeer returns the peer given the address and the peer class (instead of just the address)

    catchpointCatchup_test.go:
    - Remove comment about giving the second node all the stake, since it is not the case here.
    - Use the round from the catchpoint instead of guessing the round as 36. In case the following catchpoint was obtained due to race conditions, checking for round 37 will be trivial, since it will also be obtained from the catchpoint.

    catchpointService.go and service.go:
    - Update the code to use peerSelectorPeer instead of network.Peer with peerSelector

    peerSelector_test.go:
    - Update the tests to use peerSelectorPeer instead of network.Peer with peerSelector
    - Cleanup debugging printouts.

commit 7ca6fd1
Author: Nicholas Guo <nicholas.guo@algorand.com>
Date:   Wed Jun 9 19:00:18 2021 -0700

    lint

commit 33885fc
Author: Nicholas Guo <nicholas.guo@algorand.com>
Date:   Wed Jun 9 13:00:51 2021 -0700

    wrote tests

* Update the Version, BuildNumber, genesistimestamp.data

* testing: fix TestRekeyUpgrade when round=0 (algorand#2188)

The previous fix for this test (algorand#2178) was incomplete. It did not included proper handling for the case of round 0.
This PR addresses this corner case.

* Improve fresh node startup time (algorand#2185)

## Summary

This PR contains three different changes that would allow the node to start up faster:
1. It disables the block 0 fix we previously deployed. This fix was intended to address catching up archival nodes, where we would calculate the block 0 hash incorrectly. The fix was long deployed, and the hash calculation is also fixed since.
2. The tracker and blocks databases are now being opened concurrently.
3. The accounts tracker database schema upgrade would now skip some of the steps that aren't required for a fresh database.

Combining all the above, the ledger startup of a new node is now about 2-3 times faster.

## Test Plan

This change has no functional enduser change. The existing tests provide sufficient coverage.

* PeerSelector and Catchup service tests (algorand#2184)

Unit tests to improve the test code coverage in peerSelector and catchup/service
peerSelector.go 100% coverage
catchup/service.go ~69%

* Adding missing test for universalFetcher.go (algorand#2175)

Adding missing test for universalFetcher.go

Code coverage increased from 74%->96%

* testing: fix data race accessing messagesOfInterest during network shutdown (algorand#2181)

<!--
Thanks for submitting a pull request! We appreciate the time and effort you spent to get this far.

If you haven't already, please make sure that you've reviewed the CONTRIBUTING guide:
https://github.com/algorand/go-algorand/blob/master/CONTRIBUTING.md#code-guidelines

In particular ensure that you've run the following:
* make generate
* make sanity (which runs make fmt, make lint, make fix and make vet)

It is also a good idea to run tests:
* make test
* make integration
-->

## Summary

Fix race [reported here](https://github.com/algorand/go-algorand-internal/issues/1268):
```
Read at 0x00c00675ed48 by goroutine 210:
  github.com/algorand/go-algorand/network.(*WebsocketNetwork).ServeHTTP()
      /home/travis/gopath/src/github.com/algorand/go-algorand/network/wsNetwork.go:1114 +0x19ad
  github.com/gorilla/mux.(*Router).ServeHTTP()
      /home/travis/gopath/pkg/mod/github.com/gorilla/mux@v1.6.2/mux.go:162 +0x193
  github.com/algorand/go-algorand/network.(*RequestTracker).ServeHTTP()
      /home/travis/gopath/src/github.com/algorand/go-algorand/network/requestTracker.go:474 +0x77b
  net/http.serverHandler.ServeHTTP()
      /home/travis/.gimme/versions/go1.14.7.linux.amd64/src/net/http/server.go:2836 +0xce
  net/http.(*conn).serve()
      /home/travis/.gimme/versions/go1.14.7.linux.amd64/src/net/http/server.go:1924 +0x837
Previous write at 0x00c00675ed48 by goroutine 79:
  github.com/algorand/go-algorand/network.(*WebsocketNetwork).Stop()
      /home/travis/gopath/src/github.com/algorand/go-algorand/network/wsNetwork.go:855 +0x265
  github.com/algorand/go-algorand/network.TestGetPeers()
      /home/travis/gopath/src/github.com/algorand/go-algorand/network/wsNetwork_test.go:977 +0xe1d
  testing.tRunner()
      /home/travis/.gimme/versions/go1.14.7.linux.amd64/src/testing/testing.go:1039 +0x1eb

```

## Test Plan

The CI should run this test. I don't think the flaky test runs on pull requests right now, but it should run during the nightly builds. 

The race is not reproducible on my system. Running this before and after the test does not report any errors (until a 10 minute timeout):
```
$ gotestsum --format testname -- -tags "sqlite_unlock_notify sqlite_omit_load_extension osusergo netgo static_build" -race github.com/algorand/go-algorand/network -run TestGetPeers -count 1000000000
```

* Add mention of install_buildtools.sh to README (algorand#2195)

Recently the build tooling was updated in algorand#2108 which split out the installation of Go tools golint, stringer, swagger, msgp into a separate script install_buildtools.sh. This updates the README to reflect that in the environment setup instructions.

* Remove buildnumber.dat and genesistimestamp.dat files.

* run some independent commands in parallel, total test 230s -> 170s (algorand#2197)

run some independent commands in parallel, total test 230s -> 170s
`20210526_115456 :179 finished e2e_subs/sectok-app.sh OK in 235.471021 seconds`
down to
`20210526_122739 :179 finished e2e_subs/sectok-app.sh OK in 169.771514 seconds`

* Update version.go

* testing: e2e sub assets-app speedup (algorand#2198)

Optimize assets-app test

* Clean up `goal wallet new` output (algorand#2183)

When you run goal wallet new there is a confusing warning ("One or more non-printable characters ...") presented just before the backup phrase, which is displayed between some sanitized ANSI codes. This is due to security improvements in algorand#1585 to prevent goal from printing control characters. This PR removes the ANSI color formatting altogether from the infoBackupPhrase (already done for Windows in algorand#1942), which is one simple way to clean up this output and remove the warning message.

* testing: fix bug in TestApplicationsUpgradeOverREST (algorand#2196)

The TestApplicationsUpgradeOverREST e2e had a bug in the testing code -
if the upgrade takes place before the broadcast takes place, the broadcast result could be non-err.
This PR ensures that if we get a non-error, we have already upgraded.

* debug tools: improve carpenter by adding timestamp (algorand#2207)

debug tools: improve carpenter by adding timestamp

* Improve testing predicate to provide better info in case of a fail case. (algorand#2206)

The e2e test TestRewardUnitThreshold was failing on this statement:
```golang
r.Truef(latestBalanceNewAccount.PendingRewards >= (initialBalanceNewAccount+amountRichAccountPokesWith)/rewardUnit, "new account should have pending rewards (e2e)")
```

I replaces the `r.Truef` with `r.GreaterOrEqualf` to get the values off when it fails.
( I wan't able to reproduce it, so I figured getting some extra info for the next time would be helpful )

* Re-enable gofmt check during travis build (algorand#2212)

While talking to @tsachiherman we noticed stray fmt'd files in my repo after running make sanity, and he suggested submitting a fix PR. I discovered this shell line generated a list of gofiles that is always empty, probably because the way we set up builds for branches (and vendoring) has changed since it was written.

* A test recommendended by Pavel, an imporvment rec'd by Brian (algorand#2216)

Two small improvements recommended during CR of teal4-math

* "Rent" larger program size (algorand#2157)

Allow paying for more program space.

* Access values from a previous app call's scratch space (algorand#2158)

This change allows contract to contract composability by allowing future app call transactions to read the scratch space of previous transactions. This allows smart contracts to expose side effects for future application calls, such as a price oracle returning the exchange rate of a particular asset pair.

Changes include:

Adding PastSideEffects fields to EvalParams.
gload and gloads docs.

* Refresh wallet handle after waiting for series of rounds, as handle might expire. (algorand#2203)

Refresh wallet handle after waiting for series of rounds, as handle might expire

* Fix proposal propagation (algorand#2079)

When a relay receives a proposal for a future round, the proposal fails to be relayed. This change fixes that by sending it when the agreement advances to the corresponding round.

* Correct path to genesis file in nightly perf testing (algorand#2218)

Fix issue cp: cannot create regular file 'gen/devnet/genesis.json': No such file or directory by creating a directory if it doesn't exist to store the genesis file. Changed the reference genesis.json file to installer/genesis/ path.

* pool fees in a txgroup (algorand#2173)

A fair amount of complexity is introduced in transaction groups when the goal is to let some entity perform an action at the expense of another. For example, a contract account might be willing to perform an exchange, but expects the caller to compensate it to replace the fee that the contract account must pay.

This changes fee accounting to simplify these situations. Rather than check that each txn in a txn group meets the min fee, the txgroup is checked as a whole to ensure the total fee exceeds n*min_fee for an n member txgroup.

* Fixing a bug in eval and TestOnSwitchToUnSupportedProtocol (algorand#2220)

estOnSwitchToUnSupportedProtocol had multiple segments. This test is separated into multiple tests.

This PR fixes two issues:

Fix for a bug in eval: in the event of an error and early termination of the eval, the loading of accounts in a separate go routine was not terminated. loadAccounts will be terminated when eval returns.
TestOnSwitchToUnSupportedProtocol1 had a bug in setting the blocks for protocol switch. The first block was not getting the NextProtocolSwitchOn set to 1. Despite the bug, the test was passing most of the time, and failing some of the time.
TestOnSwitchToUnSupportedProtocol3 had a bug in setting the blocks for protocol switch. AddBlocks starts adding from the next round of the passed block. NextProtocolSwitchOn was getting set one round late.

* Extend catchpoint wait for round timeout from 10 to 60 milliseconds. (algorand#2231)

Extend catchpoint test next round timeout from 10 to 60 seconds to try to avoid test failure.

* bandwidth stats for cluster tests (algorand#2159)

heapWatch.py gains the ability to get a list of hosts from terraform-inventory from algonet tool. Can have --token and --admin-token set for talking to those cluster algod instances. (Cluster tool is separately updated to set one token across all cluster algod.)

metrics_delta.py processes captured /metrics data into stats for each algod followed and also produces one summary line of all data processed, e.g.:
summary: 1091.99 TPS, 26341397 tx B/s, 3490256 rx B/s

* Check setbyte length properly. (algorand#2236)

We we checking the against the length with > instead of >=.  That's
wrong, so we could cause a panic instead of clean error when trying to
set the byte 1 past the array length.  Fortunately, both have the same
effect: the txn fails. But we should do this properly.  Thanks to
@amityadav0 for the report in algorand#2221.

* Use minimal travis container since we install go ourselves. (algorand#2232)

Since we manage go ourselves I want to see if our build works without running the travis go setup step.

* Modify build_release to match build_pr and remove ARM Deploy. (algorand#2237)

This pulls in the cleanups we've made to other testing into the rel/
tests. Additionally, the ARM Deploy builder has not successfully built
in a while, so we'll remove it altogether.

* Fix rel nightly test by extending catchpoint round timeout (algorand#2240)

Fix catchpoint unit test by extending timeout from 10 to 60 seconds.

* Fix the two docker files - avoid make deps step, which is no longer needed. (algorand#2241)

Fix the two docker files - avoid make deps step, which is no longer needed.
The compilation of go-algorand no longer requires the installation ( and therefore the validation ) of the dependencies.

* Use gotestsum instead of logfilter for test formatting. (algorand#2161)

Use `gotestsum` to format go test output, include duration messages in a few spots that take up some time, simplify some small things here and there.

* Ensures that the catchup service have the network library load the DNS records before attempting the first sync. (algorand#2248)

Ensures that the catchup service have the network library load the DNS records before attempting the first sync -

Current code was attempting to sync without any DNS records available, failing, and trying again later on. Since the DNS records were being refreshed every 60 seconds, the third or fourth attempt would be successful. This PR attempt to give the catchup service a good chance of succeeding on the first attempt.

* Combine Clear and Approval Program size limits (algorand#2225)

Apps have been able to use 1k space for each of their programs since apps were introduced in v24. But clear state programs are quite small. It is more useful to provide 2k of space, divided however the app prefers. This PR does that, including giving 2k extra space for each unit of "extra pages" requested at app creation time.

Tests considering before and after consensus updates.

* Remove ci-deps and update GOPROXY in Dockerfiles (algorand#2247)

Recent refactoring has changed dependency installation. ci-deps did not appear to be used in a meaningful way, so it was removed from the Makefile and Dockerfiles. Additionally, downloading from go has become unstable, so updating the GOPROXY options should help.

* codecov integration (algorand#2228)

Integrate with codecov.io to attach coverage reports to PRs

* Disable compact certs and auction tests (algorand#2254)

They need work and shouldn't block our releases since they features aren't currently in use/enabled.

* If the branch is rel/nightly set channel to nightly (algorand#2252)

The way that the new pipeline scripts computed channel did not account for the rel/nightly branch, which should set the channel to nightly. This was preventing rpm and deb binaries from being built. This change identifies if the TRAVIS_BRANCH env is set to rel/nightly, and if so, will set the channel to nightly.

* Optimize constant assembly (algorand#2215)

With this PR, constants introduced by pseudo-ops (int, byte, addr) are now placed into constant blocks or loaded with pushint/pushbytes in the optimal way to save program space, starting with TEAL v4.

This optimization effectively reorders the intcblock and bytecblock that the assembler creates for pseudo-op constants such that the most frequently referenced constants are first (and can thereby take advantage of the space-saving intc_X/bytec_X opcodes). Additionally, any constants referenced only once are taken out of the constant block and instead loaded with pushint or pushbytes to further save space.

Other changes:

Renamed OpStream's noIntcBlock and noBytecBlock to hasIntcBlock and hasBytecBlock to clarify this flag is true only if the input code defines a constant block.
pushbytes now disassembles with a comment containing a parsed representation of its contents, like bytec.

* combine app state key/value size limits (algorand#2172)

Combine the limits enforced on keys and values in Teal such that there is a 128 byte limit on a key/value pair, rather than a 64 byte limit on each. The previous method meant app creators/users were essentially paying for ("renting") space that might be used by keys, but rarely was, and even if it was, could serve little purpose.

* Catchupaccessor coverage (algorand#2235)

testing: improve Catchupaccessor code coverage

* Expose creatable IDs to TEAL code within the same group (algorand#2243)

This PR adds gaid and gaids opcodes (similar to gtxn and gtxns), which will allow smart contracts to access asset IDs of assets/apps which were created earlier in the same transaction group.

Previously, accessing creatable IDs required subsequent app calls with asset/application IDs as app arguments.

* Allow AVM code to access a max number of foreign refs. (algorand#2263)

Along the lines of similar "combine" PRs, this allows AVM code to access a total number of "foreign" references, as opposed to limiting each one individually.

* deprecate auction code (algorand#2261)

RIP Auction code. We should create a tag when this is merged. It is also archived in https://github.com/algorand/auction-tools

* wrote tests

* Temporarily disabled TestBasicCatchpointCatchup and reset test timeouts from 60 to 10 seconds. (algorand#2266)

* Add support for RHEL Centos 8 RPM  (algorand#2190)

* lint

* Disable TestPeriodicSync test (algorand#2269)

Temporarily disable TestPeriodicSync test since it is broken and blocking the release pipeline.

* Fix incorrect `gaid` and `gaids` docs (algorand#2275)

The gaid and gaids docs refer to the deprecated CreatableID transaction field that was used by the txn opcode. This PR fixes the documentation and adds additional information about only being able to access IDs of previous transactions in the current group.

* Regularize access to "foreign" references. (algorand#2264)

All opcodes that take accounts, asas, or apps can use the thing itself (an address, or asa/app id) or an "indirect" reference through the "foreign" arrays of the app transaction. In all cases, the "thing" must appear in those foreign arrays, it's just more convenient sometimes to use the item in code, rather than indirect. (Note this is a new requirement for some opcodes that previously allowed access to any number of local state objects of an account asset_holding_get asset_opted_in and app_local_*)

* fix random failure (algorand#2280)

The unit test had a corner case that would not ensure that all the rounds were flushed to disk.

* Make account endpoint produce deterministic output (algorand#2276)

The JSON return value of the /v2/accounts/{addr} endpoint represents an account with several arrays which are populated from Go maps (such as held assets, created apps, etc.). Currently these arrays are unordered, meaning every time you query this endpoint, you'll likely receive a response with a different ordering of these arrays. This makes SDK testing more difficult than it needs to be, since different responses cannot be directly compared to each other.

This PR sorts the slices in generated.Account so that the /v2/accounts/{addr} endpoint will always produce arrays with deterministic order.

* In this chage, fixes to peer selector and the test.

peerSelector.go: various bug fixes
- introduce peerSelectorPeer to wrap the network.Peer and add peerClass information, to be able to distinguish between peers of the same address but different classes.
- keep track of download failures to be able to increase the cost of each failure when failing more than succeeding. This is to evict the peer faster when constantly failing to download.
- initialize rankSum and rankSamples to initialRank of the class. Otherwise, the peer rank will have a very long warmup time before relfecting the correct rank.
- let resetRequestPenalty bound the rank within the class bounds. Otherwise, the penalty calculation pushes the rank out of the class bounds (bug).
- getNextPeer is local to the package
- getNextPeer, PeerDownloadDurationToRank and RankPeer use peerSelectorPeer instead of network.Peer
- refreshAvailablePeers distinguishes between peers with the same address but of different peer class
- findPeer returns the peer given the address and the peer class (instead of just the address)

catchpointCatchup_test.go:
- Remove comment about giving the second node all the stake, since it is not the case here.
- Use the round from the catchpoint instead of guessing the round as 36. In case the following catchpoint was obtained due to race conditions, checking for round 37 will be trivial, since it will also be obtained from the catchpoint.

catchpointService.go and service.go:
- Update the code to use peerSelectorPeer instead of network.Peer with peerSelector

peerSelector_test.go:
- Update the tests to use peerSelectorPeer instead of network.Peer with peerSelector
- Cleanup debugging printouts.

* write tests

* fix import

* rename test

* Added tests, exponential increase of download failure impact, local functions.

* test for error first

* Fix the test name so it runs

* final v28 foundation spec (algorand#2286)

Changes to readmes and specs to reflect the matching foundation spec.

* fix tests

* report substring missing immediates properly (algorand#2287)

Fixes crash that should be a clean error report for using substring wrong.

Unit tests added to confirm and prevent regression.

* add check

* Fix darwin-arm64 builds

* address comments

* Enable unit tests on mac build script.

* More changes.

* Introduce V28 consensus version (algorand#2255)

* TEAL v4
* Larger programs
* Larger app/asset lookup limits
* Longer asset URL
* Fee pooling within a group
* Keyreg txn additional checks

Removed InitialRewardsRateCalculation and PaysetCommit from vFuture since it is already in v26

Fixed some tests after exposing zero fees and strict keyreg as current consensus

* Forgot to run .md generation (algorand#2292)

* add another case

* Allow fee to be below minfee, if given explicitly. (algorand#2295)

This makes other txns accept explicitly low fees.  Sorry for the code
duplication, but I did not want to change libgoal's existing behavior
where it increases fee to minfee.

* Expose extra program pages to API (algorand#2294)

Add the AppsTotalExtraPages account field to the response returned by the /v2/accounts/{addr} endpoint.

* use enums for message event permutations, add require trace helpers

* add playerPermutation enums to permutation test

* rename requireTraceContainsAction to requireTraceContains

* testing: fix random failure in TestAppEmptyAccountsLocal (algorand#2302)

The test had two unrelated bugs:
1. We need to call `WaitForCommit` before `reloadLedger` to ensure the block is being written to disk before the blockQ getting reinitialized ( and loose its content ).
2. The calculation of the total rewards unit in `makeNewEmptyBlock` was wrong. I corrected it. For tests that run only one or two rounds, this might be good enough, but for long-running tests, it would start fail pretty quickly.

* Update the Version, BuildNumber, genesistimestamp.data

* make linter happy by removing underscores from enum names

* Remove commented code in TestPlayerPermutation

* Move extra page test to e2e_subs

* Specify truncated division is used in TEAL

Specify that the truncated division is used (https://en.wikipedia.org/wiki/Modulo_operation#Variants_of_the_definition)

* typos, grammar, inconsistencies

mostly casing issues, ie, algorand > Algorand || sqlite > SQLite

* Spec updates to go with division explanation.

* added benchmark to lruAccounts write function

* lruaccounts benchmark: filling the acounts with data before benchmarking

* fix: amount of accounts generated in lruAccounts write benchmark

* lruAccounts write fix: benchmark used too much memory

* lruAccounts benchmark: a more controlled distribution between accounts in the benchmark

* REST API: make extra-program-pages and apps-total-extra-pages optional

* This also helps in not exposing them before the protocol switch

* Merge pull request algorand#2313 from algorandskiy/pavel/extra-pages-api

REST API: make extra-program-pages and apps-total-extra-pages optional

* Bump buildnumber.dat

* Run misspell linter with -w flag (algorand#2320)

This runs the popular misspell linter with the -w flag to automatically correct spelling mistakes in go-algorand.

* Reduce unneeded contention around checking the peers connectivity. (algorand#2319)

Existing `messageHandlerThread` was checking that all the connected peers are properly communicating by examining their recent message timings. This implementation served us well - however, it was executed redundantly. How much redundantly ? 19 times too many every 3 minutes ( and all of them at the **exact** same time.. ).

This PR ensures that all the `messageHandlerThread` shares the same ticker for testing the `checkPeersConnectivity`. This is expected to reduce the pressure on the internal `peersLock`.

* Improve TestMetricSegment test realibility (algorand#2322)

mprove TestMetricSegment test reliability by repeating the test with incrementing time delays.
This implementation would allow faster execution on faster platforms, and allow fallback for slower platforms.

* Restore TestPeriodicSync and TestBasicCatchpointCatchup tests. (algorand#2315)

* testing: enable previously disabled TestConfigMigrate unit test (algorand#2326)

testing: enable previously disabled TestConfigMigrate unit test

* testing: fix gotestsum install (algorand#2328)

testing: fix gotestsum install

avoid installing swagger when not needed.

* testing: move travis_retry to skip rebuilding (algorand#2324)

Remove the top-level retry command. This will allow build / lint failures to terminate the build with no retry, and may speed up test failure retries.

* testing: fix telemetry unit tests (algorand#2321)

Fix and re-enable async telemetry unit test TestAsyncTelemetryHook_CloseDrop.

* Add Fedora support into install_linux_deps.sh (algorand#2331)

Add Fedora deps into install_linux_deps.sh

* Add missing txna allowed args to doc and langspec (algorand#2336)

We exposed Assets and Applications in TEAL v3 but did not update doc/langspec. This commit fixes it.

* Add initial devMode support (algorand#2334)

Add new local private network mode - devmode. devMode allows the developer to deploy a single node network, where every transaction being sent to the node automatically generates a new block.

This feature is focused primarily around 3rd parties that want to test their solution on the Algorand platform, without waiting for the network to make progress.

* Implement hello circleci yaml (algorand#2417)

This adds a hello world circleci yaml. This can be used to run circleci jobs on PRs until the pipeline is fully implemented.

* catchup: fix potential nil dereferencing (algorand#2420)

The handing for returned peer was incorrect in case the getPeerErr is non-nil, as we would attempt to dereference the nil pointer.

* testing: ensure deploy_linux_version generates a linux/amd64 binaries for algonet usage (algorand#2422)

The deploy_linux_version.sh script currently creates a linux version of the current project tree with the same architecture as the hosting environment.

When we attempt to builds on M1 Macs, it default to arm64 base images and as a result - makes a linux/arm64 compatible binaries. These resulting binaries, however, would not work correctly on an algonet deployed network, since the hosts there are amd64.

To rectify that situation, we'll be changing the docker file to ensure linux/amd64 binaries are generated. This would ensure that the existing functionality works as intended. In the future, when we would add arm64 support for algonet, we could accompany that by creating corresponding images on docker as well.

* Specs and unit test to avoid forgetting in the future (algorand#2418)

Specs and unit test to avoid forgetting in the future

* Fix and enable TestNewAccountCanGoOnlineAndParticipate (algorand#2238)

* TestNewAccountCanGoOnlineAndParticipate was failing because the test was
not waiting enough to get to the round where the newly funded account's
funds will be considered for proposing purposes.

It was miscalculating the round that it should wait form.

Moreover, the rounds considered to when the account is funded was prone to
race conditions.

In addition, the test was using WaitForRoundWithTimeout which may be
very slower if the current round is already ahead. Instead, now it is
using ClientWaitForRound, which does not care about individual rounds delayed.

* Addressing review commnets:
- fixing a typo
- getting exact transaction round for funding the account
- testing exact blocks for the proposer
- using a single node network instead of two nodes
- waiting for exactly one round for the new account to propose and checking that
- sending the funds and closing the rich account so there will be no possiblity of that proposing a block

* Fix docker repo update (algorand#2342)

The scripts to update the docker repository do not do a full rebuild, and the submitted Dockerfile does not change. This means that if you run the docker image/repo update, it'll re-use the old image. To fix this, we call docker build with --no-cache. Other refactoring helps simplify the code.

We add a new --cached flag to the docker/releases/build_releases.sh script to explicitly cache. Otherwise the testnet update would issue a full rebuild. We additionally automatically handle the 'latest' tagging in build_releases.sh as well.

* Bump Version, Remove buildnumber.dat and genesistimestamp.dat files.

* Initialize past side effects with correct length during dryrun requests (algorand#2448)

This PR fixes a typo in the dryrun code that caused certain dryrun requests containing multiple transactions to fail.

* optimize txtail memory consumption (algorand#2413)

The changes in this PR are as follows:

The unused method Ledger. GetRoundTxIds was removed. As a result, the txTail. getRoundTxIds can be removed as well. This makes the txids map stored in the roundTxMembers structure redundant.
In the cow.go, avoid adding empty leases to the cb.mods.Txleases map. Since we already not testing for empty leases, we can safely avoid storing them.
Optimize the txTail. loadFromDisk to generate optimal lastValid map sizes.
Optimize the txTail. loadFromDisk to avoid storing empty leases in the txleases map.

* enable TestPartkeyOnlyRewards on macos (algorand#2429)

TestPartkeyOnlyRewards was disabled on darwin.
It is no longer failing on darwin (5/5 runs passed).

* Fix 10 minute timeout in travis. (Revert parts of algorand#2324) (algorand#2494)

We started seeing the 10-minute timeout error on travis after the recent changes to where travis_retry gets called.

This means travis_wait is still needed. travis_wait and travis_retry don't play well together on the ephemeral build machine, so we basically need to rollback the entire change in algorand#2324

Note: while making this change I noticed that we don't use travis_retry for all of the build_test.sh / integration_test.sh entries. Not sure why but I left them as they were before algorand#2324

* allow to parse ipv6 localhost "[::]:4601" (algorand#2430)

Allow parsing of ipv6 localhost -colon- port address.
"[::]:4601" was failing to parse. This and other forms pass ParseHostOrURL() now.

* disable TestAgreementSynchronous10 (algorand#2503)

The test TestAgreementSynchronous10 is failing. We should be fixing it, but until we do that, I'm going to disable it so it won't mask other issues.

* testing: avoid division by zero during TestBasicCatchpointWriter (algorand#2502)

The `randomFullAccountData` method was dividing by `lastCreatableID` which could be zero.
The probability for that is pretty slim, but given that it was found during a travis run, we should fix it.

* selector pseudo-op in support of ABI (algorand#2358)

Adds a Teal pseudo-op `selector` that assembles as if it were the `byte` pseudo-op, but stores 4 bytes of hash.

This allows Teal, like:
```
txn ApplicationArgs 0
selector "add(uint64,uint64)uint128"
==
bnz add
```

and avoid the need to embed the actual hash in a .teal file (which would also require calculating it)

Unlike `byte`, the argument *must* be a quoted string - no base64 or hex allowed, as the argument should be a method signature.

* Creator access and app_params_get (algorand#2301)

Adds the ability to get the creator of apps and assets in teal.

This is a simple new field for `asset_params_get`, but it introduces a new opcode `app_params_get` to enable access for apps.  Meanwhile `app_params_get` allows access to other global parameters about an app - the schema sizes, extra pages, and the programs themselves.

This also begins work on LogicVersion=5, which should be AVM 1.0

* fix regex for matching charset of valid DNS hosts, add test (algorand#2505)

A regex was trying to match valid DNS names but missed the '-' char. Fix.

* Fix random failure in TestWebsocketNetworkPrioLimit (algorand#2509)

The peers array is modified when adding/removing entries from it. When that does happen, we increase the peersChangeCounter, so that the broadcast method would know that it's peers list need to be refreshed.
The said update was missing from prioTracker.setPriority, which was causing the issue.

* Make TestPeersDownloadFailed and TestHistoricData predictable (algorand#2516)

Both the `TestPeersDownloadFailed` as well as `TestHistoricData` were randomly failing, as they were depending on a "genuine" random distribution of the random function in order to succeed. When that doesn't happen, they were failing.

This PR make sure to "bake-in" the random seed so that the tests are repeatable and a success is repeatable as well.

* Improve testing of alloc bounded slices. (algorand#2515)

This change is needed in order to support codec types such as the following:
```golang
//msgp:allocbound typeA 16
type typeA []typeB
```
since we want the codec object randomizer to recognize that `typeA` has a allocbound defined, and use that bound.
( this would be used in the feature/txnsync branch, but the change doesn't really related to any of the other changes in the feature/txnsync branch and would be a good change regardless )

* Save Logging Information in Data Directory If Provided (algorand#2415)

Our logic should be as follows:

When Loading:

We first look inside the provided data-directory. If a config file is there, load it and return it
Otherwise, look in the global directory. If a config file is there, load it and return it.
When Saving:

If a data-directory was provided then save the config file there.
Otherwise, save the config file in the global directory

* Lruaccts write speedup (algorand#2329)

Optimized lruAccounts write function. gains: from ~260ns to ~200ns per write op

* Use golangci-lint (algorand#2523)

This adds a golangci-lint configuration file that runs an initial minimal set of linters: golint, govet, and misspell. This makes it easier to integrate with editors and IDEs and also opens the door to using from CI.

* update codec tester (algorand#2527)

This PR improves the checkBoundsLimitingTag method of the message pack random generated object tester.
With this change, we're no longer ignoring data types that has no struct tags, but rather looking for corresponding msgp directives.

This is the same change applied to the feature/txnsync branch. This change is expected to be nop on the master branch, but would be required on the feature branch.

* Fix random failures in TestPeriodicSync (algorand#2535)

The waiting period wasn't long enough. Allow much longer wait period before giving up.

Also - omit unneeded log entries.

* Testing: use periodicSyncLogger for cleaner test runs (algorand#2539)

Use periodicSyncLogger in a test to avoid unnecessary log outputs.

* Add travis wait to compilation step on travis. (algorand#2544)

Travis nightly build failed due to no-output for over 10m from build process.

* test conditional slack alert (algorand#2537)

Travis is currently unable to only notify on failures of a particular branch, however, it can notify on us on the build of a particular branch (both success and failures). Since we are also considering CircleCI, I think it would be best to just add the slack notification into the travis file now for the rel/nightly branch and notify the #devops-jenkins channel (where most of our jobs status activity goes) when it both succeeds/fails. The "successes" will also help us determine whether or not the test ran at all.

* Add Extract opcodes  (algorand#2521)

Adds extract opcodes that allow a substring to be extracted given a start index and a length (algorand#2347).

The specs for the extract and extract3 opcodes are the same as the substring and substring3 opcodes, but takes a length rather than an end index. This also adds extract16bits, extract32bits, extract64bits which extracts 2, 4, and 8 byte strings respectively and converts them into integers.

* Benchmark

* Modify target

Co-authored-by: DevOps Service <devops-service@algorand.com>
Co-authored-by: John Lee <64482439+algojohnlee@users.noreply.github.com>
Co-authored-by: Tsachi Herman <tsachi.herman@algorand.com>
Co-authored-by: algonautshant <55754073+algonautshant@users.noreply.github.com>
Co-authored-by: Will Winder <wwinder.unh@gmail.com>
Co-authored-by: chris erway <51567+cce@users.noreply.github.com>
Co-authored-by: Brian Olson <brianolson@users.noreply.github.com>
Co-authored-by: algobarb <78746954+algobarb@users.noreply.github.com>
Co-authored-by: John Jannotti <john.jannotti@algorand.com>
Co-authored-by: shiqizng <80276844+shiqizng@users.noreply.github.com>
Co-authored-by: Jacob Daitzman <jdtzmn@gmail.com>
Co-authored-by: nicholasguoalgorand <67928479+nicholasguoalgorand@users.noreply.github.com>
Co-authored-by: egieseke <eric_gieseke@yahoo.com>
Co-authored-by: John Lee <john.lee@algorand.com>
Co-authored-by: bricerisingalgorand <60147418+bricerisingalgorand@users.noreply.github.com>
Co-authored-by: Jason Paulos <jasonpaulos@users.noreply.github.com>
Co-authored-by: Jacob Daitzman <jdtzmn@users.noreply.github.com>
Co-authored-by: Nicholas Guo <nicholas.guo@algorand.com>
Co-authored-by: algonautshant <shant@algorand.com>
Co-authored-by: Rakshith Gopala Krishna <rakshith.gopalakrishna@algorand.com>
Co-authored-by: Pavel Zbitskiy <65323360+algorandskiy@users.noreply.github.com>
Co-authored-by: chris erway <chris.erway@algorand.com>
Co-authored-by: Pavel Zbitskiy <pavel@algorand.com>
Co-authored-by: Fabrice Benhamouda <fabrice.benhamouda@normalesup.org>
Co-authored-by: figurestudios <64747030+figurestudios@users.noreply.github.com>
Co-authored-by: John Jannotti <jj@cs.brown.edu>
Co-authored-by: Jonathan Weiss <jonathan@Algo-Weiss-MBP.local>
Co-authored-by: Jonathan Weiss <jonathan.weiss@algorand.com>
Co-authored-by: algonathan <85506383+algonathan@users.noreply.github.com>
Co-authored-by: pzbitskiy <pavel.zbitskiy@gmail.com>
Co-authored-by: AlgoStephenAkiki <85183435+AlgoStephenAkiki@users.noreply.github.com>