Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix issue 9979: false positive: containerOutOfBounds with conditional resize #3136

Merged
merged 27 commits into from
Mar 30, 2021

Conversation

pfultz2
Copy link
Contributor

@pfultz2 pfultz2 commented Feb 18, 2021

No description provided.

lib/token.cpp Outdated
@@ -29,6 +29,7 @@
#include <algorithm>
#include <cassert>
#include <cctype>
#include <cstdio>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems my editor sometimes insert these headers. I'll remove them.

@pfultz2
Copy link
Contributor Author

pfultz2 commented Feb 22, 2021

I see this error on travis:

$ python3 ../misra.py -verify misra/misra-test.c.dump

Checking misra/misra-test.c.dump, config ...

Expected but not seen: 244:7.2

Not expected: 506:12.2

Not expected: 662:12.4

Not expected: 664:12.4

The command "python3 ../misra.py -verify misra/misra-test.c.dump" exited with 1.

What does that mean? Also the scriptcheck stage fails, but there is no errors in the output.

@danmar
Copy link
Owner

danmar commented Feb 23, 2021

What does that mean?

is there unsigned integer overflow or a weird looking shift operation (that would overflow without integer promotion).

@pfultz2
Copy link
Contributor Author

pfultz2 commented Feb 23, 2021

is there unsigned integer overflow or a weird looking shift operation (that would overflow without integer promotion).

No I check that before doing a bitshift calculation here.

@danmar
Copy link
Owner

danmar commented Feb 23, 2021

hmm.. I try..

@danmar
Copy link
Owner

danmar commented Feb 23, 2021

Example code:

int x = 0u - 1u;

debug output before your changes:

  = always -1
  0u always 0
  - always 18446744073709551615
  1u always 1

debug output after your changes:

  = always !<=-1
  0u always 0
  - always !<=18446744073709551615
  1u always 1

Spontanously it seems to me those values should be known?

@danmar
Copy link
Owner

danmar commented Feb 23, 2021

The addons did not handle impossible values very nicely. I made a tweak for that with 22727ee

@pfultz2
Copy link
Contributor Author

pfultz2 commented Feb 24, 2021

Spontanously it seems to me those values should be known?

They're not set since it can't be stored as MathLib::bigint(see here). The problem is that we set impossible values to -1 because its impossible for the value to be -1 because its unsigned. However, this can lead to confusion if we also interpret -1 as (MathLib::biguint)-1.

To avoid confusion, I skip setting a value if its negative and unsigned unless its an impossible value. If we move to making MathLib::bigint in the future to use arbirtary-precision integer or 128-bit integer, then large unsigned values should not be affected.

@pfultz2
Copy link
Contributor Author

pfultz2 commented Feb 24, 2021

The addons did not handle impossible values very nicely. I made a tweak for that with 22727ee

Ok with that fix it still errors with this:

$ python3 ../misra.py -verify misra/misra-test.c.dump

Checking misra/misra-test.c.dump, config ...

Expected but not seen: 244:7.2

Expected but not seen: 737:12.4

Expected but not seen: 738:12.4

I assume there is some misra tests I need to disable. Or perhaps there is some better logic to use to allow for negative values, but I am not sure how.

@danmar
Copy link
Owner

danmar commented Feb 24, 2021

Ideally the value of the - should be always 0xffffffff. And the value of the = should be always -1.

@danmar
Copy link
Owner

danmar commented Feb 24, 2021

I assume there is some misra tests I need to disable.

well we could disable those but then we have regressions. It should be detected that 0u - 1u overflows.

@pfultz2
Copy link
Contributor Author

pfultz2 commented Feb 24, 2021

Ideally the value of the - should be always 0xffffffff. And the value of the = should be always -1.

I see, this is a 32-bit unsigned integer. This should be fine since we truncate the value, I just need to check the size of the type. Although, it will still be somewhat ambigious with if(x == -1) return; since we dont truncate impossible values.

Of course, this wont fix the issue with -1 and 64-bit integers. For now, we can't represent 18446744073709551615 using a 64-bit signed integer. Even though, 18446744073709551615 should be represented as 0xffffffffffffffff, the problem is that this can be interpreted as either 18446744073709551615 or -1. And there are some places that we will interpret it as -1. We can fix that issue by switching to use a 128-bit integer for MathLib::bigint.

@danmar
Copy link
Owner

danmar commented Feb 25, 2021

The misra addon needs to be rewritten a bit also. It does not check for overflows properly. If you fix the ValueFlow so the value is correct (0xffffffff) then the addon will not detect the overflow anymore.

We can fix that issue by switching to use a 128-bit integer for MathLib::bigint.

yes. @IOBYTE had such work in progress. how is the progress on that? did that work in windows also or was it only for gcc?

I am guessing that it is not necessary to use 128-bit integers we could use 64 bits for the integer data and then add a flag or something that indicates the sign. I don't know if we could encapsulate such 64 bit value somehow.. so it will be somewhat straightforward to use. For instance if it has internal knowledge about ValueType it could ensure that the value is truncated according to the type and perform the proper integer operations.

@IOBYTE
Copy link
Contributor

IOBYTE commented Feb 25, 2021

yes. @IOBYTE had such work in progress. how is the progress on that? did that work in windows also or was it only for gcc?

I had it working with gcc. I tried a few 128 bit libraries but they all had issues. Some of the removal of bigint patches were merged but bigint is still being used everywhere inappropriately. This is on hold because I'm trying to get simplifyUsing stable for release.

@danmar
Copy link
Owner

danmar commented Feb 25, 2021

well we could disable those but then we have regressions. It should be detected that 0u - 1u overflows.

I think we should disable the check because it is not implemented properly. And then we should implement the check properly.

So I believe you can ignore that particular problem.

@pfultz2
Copy link
Contributor Author

pfultz2 commented Feb 25, 2021

we could use 64 bits for the integer data and then add a flag or something that indicates the sign.

We could build a simple wrapper class that has:

struct bigint {
    biguint data;
    int8_t sign; // Either 1 or -1
};

Implementing the arithmetic operators should be simpler than implementing large integer math. The only thing is that this won't use twos complement.

@pfultz2
Copy link
Contributor Author

pfultz2 commented Feb 25, 2021

I had it working with gcc. I tried a few 128 bit libraries but they all had issues.

What were some of the issues? I wonder if we need to write a wrapper around these classes.

@pfultz2
Copy link
Contributor Author

pfultz2 commented Feb 26, 2021

I think we should disable the check because it is not implemented properly. And then we should implement the check properly.

It still fails in the CI.

@danmar
Copy link
Owner

danmar commented Feb 26, 2021

We could build a simple wrapper class that has:

yes exactly. Unless the 128-bit integer works well I think that is a good solution..

@danmar
Copy link
Owner

danmar commented Feb 27, 2021

It still fails in the CI.

Example code:

unsigned long long k = 0x8000000000000000; // 7.2

with HEAD Cppcheck says the value is a large positive value both in the debug output and dump output. After your changes this value is removed. This particular behavior is unfortunate we have the proper behavior in HEAD as far as I see but not after your changes..

@danmar
Copy link
Owner

danmar commented Feb 27, 2021

You could add a TODO in the misra-test.c on the corresponding line.

unsigned long long k = 0x8000000000000000; // TODO 7.2

When the ValueFlow is fixed later the verification would fail on this line and then we can remove the TODO.

@pfultz2
Copy link
Contributor Author

pfultz2 commented Mar 1, 2021

So sciptcheck fails, but there is no error message.

@danmar
Copy link
Owner

danmar commented Mar 1, 2021

So sciptcheck fails, but there is no error message.

maybe some test problem.. I restarted that test.

@amai2012
Copy link
Collaborator

amai2012 commented Mar 5, 2021

From what I see the Makefile should be re-generated and updated.

lib/token.cpp Outdated
{
if (x.bound != ValueFlow::Value::Bound::Point && x.bound == y.bound) {
return true;
} else {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This else is redundant

Copy link
Owner

@danmar danmar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit

@pfultz2
Copy link
Contributor Author

pfultz2 commented Mar 30, 2021

It looks like this passes the CI now.

@danmar danmar merged commit 5077663 into danmar:main Mar 30, 2021
@pfultz2 pfultz2 deleted the valueflow-infer-unsigned-compare branch March 30, 2021 21:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants