Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: Catch decimal.InvalidOperation from TestNodeCLI#send_cli #19632

Merged
merged 1 commit into from
Aug 5, 2020
Merged

test: Catch decimal.InvalidOperation from TestNodeCLI#send_cli #19632

merged 1 commit into from
Aug 5, 2020

Conversation

Empact
Copy link
Contributor

@Empact Empact commented Jul 31, 2020 •
edited
Loading

decimal.InvalidOperation is a special case of a float parsing error, which
presumably should be handled in the same way as a general parsing error,
rather than blow up.

Alternatives include: logging the error, or re-raising with more information.

Example log output:

    File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 603, in sync_all
      self.sync_blocks(nodes)
    File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 568, in sync_blocks
      best_hash = [x.getbestblockhash() for x in rpc_connections]
    File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 568, in <listcomp>
      best_hash = [x.getbestblockhash() for x in rpc_connections]
    File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_node.py", line 571, in __call__
      return self.cli.send_cli(self.command, *args, **kwargs)
    File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_node.py", line 639, in send_cli
      return json.loads(cli_stdout, parse_float=decimal.Decimal)
    File "/usr/lib64/python3.6/json/__init__.py", line 367, in loads
      return cls(**kw).decode(s)
    File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
      obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    File "/usr/lib64/python3.6/json/decoder.py", line 355, in raw_decode
      obj, end = self.scan_once(s, idx)
  decimal.InvalidOperation: [<class 'decimal.InvalidOperation'>]

See: https://travis-ci.org/github/bitcoin/bitcoin/jobs/713502326

@Empact
test: Catch decimal.InvalidOperation from TestNodeCLI#send_cli
82fc401 
decimal.InvalidOperation is a special case of a float parsing error, which
presumably should be handled in the same way as a general parsing error,
rather than blow up.

Alternatives include: logging the error, or re-raising with more information.

Example log output:
    File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 603, in sync_all
      self.sync_blocks(nodes)
    File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 568, in sync_blocks
      best_hash = [x.getbestblockhash() for x in rpc_connections]
    File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 568, in <listcomp>
      best_hash = [x.getbestblockhash() for x in rpc_connections]
    File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_node.py", line 571, in __call__
      return self.cli.send_cli(self.command, *args, **kwargs)
    File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_node.py", line 639, in send_cli
      return json.loads(cli_stdout, parse_float=decimal.Decimal)
    File "/usr/lib64/python3.6/json/__init__.py", line 367, in loads
      return cls(**kw).decode(s)
    File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
      obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    File "/usr/lib64/python3.6/json/decoder.py", line 355, in raw_decode
      obj, end = self.scan_once(s, idx)
  decimal.InvalidOperation: [<class 'decimal.InvalidOperation'>]
@fanquake fanquake added the Tests label Jul 31, 2020
@maflcko
Copy link
Member

maflcko commented Jul 31, 2020

Too bad the stdout isn't available. I presume it is something like "connection reset"?

ACK

@laanwj
Copy link
Member

laanwj commented Aug 5, 2020

ACK 82fc401

I don't understand what happened though. If you psas invalid input to a Decimal, you'll normally get this message:

>>> decimal.Decimal("sdfsdf3f")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
decimal.InvalidOperation: [<class 'decimal.ConversionSyntax'>]

The exception message has a condition "decimal.ConversionSyntax". However in your case, the condition isdecimal.InvalidOperation itself. Too bad we can't see the input.

@maflcko
Copy link
Member

maflcko commented Aug 5, 2020

Looks like this might happen when the quantization is too precise:

>>> decimal.Decimal(3).quantize(decimal.Decimal('0.000000000000000000000000000000000000000000001'), rounding=decimal.ROUND_DOWN)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
decimal.InvalidOperation: [<class 'decimal.InvalidOperation'>]

or

>>> decimal.Decimal('inf')*0
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
decimal.InvalidOperation: [<class 'decimal.InvalidOperation'>]

See https://docs.python.org/3.8/library/decimal.html#decimal.InvalidOperation

Though, still no idea how that could happen in our test framework.

@laanwj laanwj merged commit 4644b13 into bitcoin:master Aug 5, 2020
@laanwj
Copy link
Member

laanwj commented Aug 5, 2020

It looks like it also happens if some limit is exceeded e.g. decimal.Decimal("1e9999999999999999999") (wasn't able to trigger this with just a long number without exponent). This would be possible to trigger though unlikely.

@Empact Empact deleted the 2020-07-send_cli-invalid-op branch August 5, 2020 14:56
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Aug 5, 2020
@laanwj
Merge bitcoin#19632: test: Catch decimal.InvalidOperation from TestNo…
a4ebad2 
…deCLI#send_cli

82fc401 test: Catch decimal.InvalidOperation from TestNodeCLI#send_cli (Ben Woosley)

Pull request description:

  `decimal.InvalidOperation` is a special case of a float parsing error, which
  presumably should be handled in the same way as a general parsing error,
  rather than blow up.

  Alternatives include: logging the error, or re-raising with more information.

  Example log output:
  ```
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 603, in sync_all
        self.sync_blocks(nodes)
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 568, in sync_blocks
        best_hash = [x.getbestblockhash() for x in rpc_connections]
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 568, in <listcomp>
        best_hash = [x.getbestblockhash() for x in rpc_connections]
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_node.py", line 571, in __call__
        return self.cli.send_cli(self.command, *args, **kwargs)
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_node.py", line 639, in send_cli
        return json.loads(cli_stdout, parse_float=decimal.Decimal)
      File "/usr/lib64/python3.6/json/__init__.py", line 367, in loads
        return cls(**kw).decode(s)
      File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
        obj, end = self.raw_decode(s, idx=_w(s, 0).end())
      File "/usr/lib64/python3.6/json/decoder.py", line 355, in raw_decode
        obj, end = self.scan_once(s, idx)
    decimal.InvalidOperation: [<class 'decimal.InvalidOperation'>]
  ```
  See: https://travis-ci.org/github/bitcoin/bitcoin/jobs/713502326

ACKs for top commit:
  laanwj:
    ACK 82fc401

Tree-SHA512: 8c102b8bf831b05c5ca4b2e1feb5574dcbaed8cab0b2f22b013c5dfcb81788a38839a163dd1e2c6470ccbe5874214663b84485f45467738fd850ca38d539ae25
jasonbcox pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Sep 3, 2020
@laanwj @jasonbcox
Merge #19632: test: Catch decimal.InvalidOperation from TestNodeCLI#s…
7cfda4b 
…end_cli

Summary:
82fc4017b774aaff8799c2b6e8ba5370d94dbf4d test: Catch decimal.InvalidOperation from TestNodeCLI#send_cli (Ben Woosley)

Pull request description:

  `decimal.InvalidOperation` is a special case of a float parsing error, which
  presumably should be handled in the same way as a general parsing error,
  rather than blow up.

  Alternatives include: logging the error, or re-raising with more information.

  Example log output:
  ```
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 603, in sync_all
        self.sync_blocks(nodes)
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 568, in sync_blocks
        best_hash = [x.getbestblockhash() for x in rpc_connections]
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 568, in <listcomp>
        best_hash = [x.getbestblockhash() for x in rpc_connections]
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_node.py", line 571, in __call__
        return self.cli.send_cli(self.command, *args, **kwargs)
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_node.py", line 639, in send_cli
        return json.loads(cli_stdout, parse_float=decimal.Decimal)
      File "/usr/lib64/python3.6/json/__init__.py", line 367, in loads
        return cls(**kw).decode(s)
      File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
        obj, end = self.raw_decode(s, idx=_w(s, 0).end())
      File "/usr/lib64/python3.6/json/decoder.py", line 355, in raw_decode
        obj, end = self.scan_once(s, idx)
    decimal.InvalidOperation: [<class 'decimal.InvalidOperation'>]
  ```
  See: https://travis-ci.org/github/bitcoin/bitcoin/jobs/713502326

ACKs for top commit:
  laanwj:
    ACK 82fc4017b774aaff8799c2b6e8ba5370d94dbf4d

Tree-SHA512: 8c102b8bf831b05c5ca4b2e1feb5574dcbaed8cab0b2f22b013c5dfcb81788a38839a163dd1e2c6470ccbe5874214663b84485f45467738fd850ca38d539ae25

Backport of Core [[bitcoin/bitcoin#19632 | PR19632]]
Seen here: https://build.bitcoinabc.org/viewLog.html?tab=buildLog&logTab=tree&filter=debug&expand=all&buildId=47762&guest=1&_focus=4020

Test Plan: `ninja check-functional` for sanity

Reviewers: #bitcoin_abc, majcosta

Reviewed By: #bitcoin_abc, majcosta

Differential Revision: https://reviews.bitcoinabc.org/D7340
PastaPastaPasta pushed a commit to PastaPastaPasta/dash that referenced this pull request Sep 17, 2021
@laanwj @PastaPastaPasta
Merge bitcoin#19632: test: Catch decimal.InvalidOperation from TestNo…
e4db0a0 
…deCLI#send_cli

82fc401 test: Catch decimal.InvalidOperation from TestNodeCLI#send_cli (Ben Woosley)

Pull request description:

  `decimal.InvalidOperation` is a special case of a float parsing error, which
  presumably should be handled in the same way as a general parsing error,
  rather than blow up.

  Alternatives include: logging the error, or re-raising with more information.

  Example log output:
  ```
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 603, in sync_all
        self.sync_blocks(nodes)
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 568, in sync_blocks
        best_hash = [x.getbestblockhash() for x in rpc_connections]
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 568, in <listcomp>
        best_hash = [x.getbestblockhash() for x in rpc_connections]
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_node.py", line 571, in __call__
        return self.cli.send_cli(self.command, *args, **kwargs)
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_node.py", line 639, in send_cli
        return json.loads(cli_stdout, parse_float=decimal.Decimal)
      File "/usr/lib64/python3.6/json/__init__.py", line 367, in loads
        return cls(**kw).decode(s)
      File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
        obj, end = self.raw_decode(s, idx=_w(s, 0).end())
      File "/usr/lib64/python3.6/json/decoder.py", line 355, in raw_decode
        obj, end = self.scan_once(s, idx)
    decimal.InvalidOperation: [<class 'decimal.InvalidOperation'>]
  ```
  See: https://travis-ci.org/github/bitcoin/bitcoin/jobs/713502326

ACKs for top commit:
  laanwj:
    ACK 82fc401

Tree-SHA512: 8c102b8bf831b05c5ca4b2e1feb5574dcbaed8cab0b2f22b013c5dfcb81788a38839a163dd1e2c6470ccbe5874214663b84485f45467738fd850ca38d539ae25
PastaPastaPasta pushed a commit to PastaPastaPasta/dash that referenced this pull request Sep 19, 2021
@laanwj @PastaPastaPasta
Merge bitcoin#19632: test: Catch decimal.InvalidOperation from TestNo…
72f2882 
…deCLI#send_cli

82fc401 test: Catch decimal.InvalidOperation from TestNodeCLI#send_cli (Ben Woosley)

Pull request description:

  `decimal.InvalidOperation` is a special case of a float parsing error, which
  presumably should be handled in the same way as a general parsing error,
  rather than blow up.

  Alternatives include: logging the error, or re-raising with more information.

  Example log output:
  ```
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 603, in sync_all
        self.sync_blocks(nodes)
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 568, in sync_blocks
        best_hash = [x.getbestblockhash() for x in rpc_connections]
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 568, in <listcomp>
        best_hash = [x.getbestblockhash() for x in rpc_connections]
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_node.py", line 571, in __call__
        return self.cli.send_cli(self.command, *args, **kwargs)
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_node.py", line 639, in send_cli
        return json.loads(cli_stdout, parse_float=decimal.Decimal)
      File "/usr/lib64/python3.6/json/__init__.py", line 367, in loads
        return cls(**kw).decode(s)
      File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
        obj, end = self.raw_decode(s, idx=_w(s, 0).end())
      File "/usr/lib64/python3.6/json/decoder.py", line 355, in raw_decode
        obj, end = self.scan_once(s, idx)
    decimal.InvalidOperation: [<class 'decimal.InvalidOperation'>]
  ```
  See: https://travis-ci.org/github/bitcoin/bitcoin/jobs/713502326

ACKs for top commit:
  laanwj:
    ACK 82fc401

Tree-SHA512: 8c102b8bf831b05c5ca4b2e1feb5574dcbaed8cab0b2f22b013c5dfcb81788a38839a163dd1e2c6470ccbe5874214663b84485f45467738fd850ca38d539ae25
thelazier pushed a commit to thelazier/dash that referenced this pull request Sep 25, 2021
@laanwj @thelazier
Merge bitcoin#19632: test: Catch decimal.InvalidOperation from TestNo…
ccc2fbf 
…deCLI#send_cli

82fc401 test: Catch decimal.InvalidOperation from TestNodeCLI#send_cli (Ben Woosley)

Pull request description:

  `decimal.InvalidOperation` is a special case of a float parsing error, which
  presumably should be handled in the same way as a general parsing error,
  rather than blow up.

  Alternatives include: logging the error, or re-raising with more information.

  Example log output:
  ```
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 603, in sync_all
        self.sync_blocks(nodes)
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 568, in sync_blocks
        best_hash = [x.getbestblockhash() for x in rpc_connections]
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 568, in <listcomp>
        best_hash = [x.getbestblockhash() for x in rpc_connections]
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_node.py", line 571, in __call__
        return self.cli.send_cli(self.command, *args, **kwargs)
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_node.py", line 639, in send_cli
        return json.loads(cli_stdout, parse_float=decimal.Decimal)
      File "/usr/lib64/python3.6/json/__init__.py", line 367, in loads
        return cls(**kw).decode(s)
      File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
        obj, end = self.raw_decode(s, idx=_w(s, 0).end())
      File "/usr/lib64/python3.6/json/decoder.py", line 355, in raw_decode
        obj, end = self.scan_once(s, idx)
    decimal.InvalidOperation: [<class 'decimal.InvalidOperation'>]
  ```
  See: https://travis-ci.org/github/bitcoin/bitcoin/jobs/713502326

ACKs for top commit:
  laanwj:
    ACK 82fc401

Tree-SHA512: 8c102b8bf831b05c5ca4b2e1feb5574dcbaed8cab0b2f22b013c5dfcb81788a38839a163dd1e2c6470ccbe5874214663b84485f45467738fd850ca38d539ae25
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Feb 15, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers
No reviews
Assignees
No one assigned
Labels
Tests
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@Empact @maflcko @laanwj @fanquake