Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move prepare, lowerNonArrayAggregate and lowerArrayAggregate methods to statementsem #17011

Draft
wants to merge 16 commits into
base: master
Choose a base branch
from

Conversation

dchidindu5
Copy link
Contributor

Is there a need to update frontend.h or? @thewilsonator

@dlang-bot
Copy link
Contributor

Thanks for your pull request and interest in making D better, @dchidindu5! We are looking forward to reviewing it, and you should be hearing from a maintainer soon.
Please verify that your PR follows this checklist:

  • My PR is fully covered with tests (you can see the coverage diff by visiting the details link of the codecov check)
  • My PR is as minimal as possible (smaller, focused PRs are easier to review than big ones)
  • I have provided a detailed rationale explaining my changes
  • New or modified functions have Ddoc comments (with Params: and Returns:)

Please see CONTRIBUTING.md for more information.


If you have addressed all reviews or aren't sure how to proceed, don't hesitate to ping us with a simple comment.

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub run digger -- build "master + dmd#17011"

@dchidindu5 dchidindu5 closed this Oct 17, 2024
@thewilsonator
Copy link
Contributor

No, they are all private (and most of them are extern(D)

@thewilsonator
Copy link
Contributor

Also did you mean to close this?

@dchidindu5
Copy link
Contributor Author

Also did you mean to close this?

I'll reopen it.

@dchidindu5 dchidindu5 reopened this Oct 17, 2024
@dchidindu5 dchidindu5 closed this Oct 17, 2024
@dchidindu5 dchidindu5 reopened this Oct 17, 2024
@dchidindu5
Copy link
Contributor Author

dchidindu5 commented Oct 19, 2024

Just saw this from the test suite @thewilsonator

 fail_compilation/test21939.d(11): Error: invalid `;` aggregate `Object` of type `Object`
  fail_compilation/test21939.d(11):        `foreach` works with input ranges (implementing `front` and `popFront`), aggregates implementing `opApply`, or the result of an aggregate's `.tupleof` property

@thewilsonator
Copy link
Contributor

That looks like you messed up the token passed to en error somewhere, but it is very strange that it only happens on a few platforms. Will take a look tomorrow.

@dchidindu5
Copy link
Contributor Author

That looks like you messed up the token passed to en error somewhere, but it is very strange that it only happens on a few platforms. Will take a look tomorrow.

@thewilsonator I would move the methods one by one and see where the problem is from.

@dchidindu5
Copy link
Contributor Author

@RazvanN7 @thewilsonator

That looks like you messed up the token passed to en error somewhere, but it is very strange that it only happens on a few platforms. Will take a look tomorrow.

What can be done to correct it? How do I figure it out?

@BorisCarvajal
Copy link
Member

BorisCarvajal commented Dec 9, 2024

I took some time to investigate this, since I committed the tests that are failing.
I was able to reproduce it by building dmd with ENABLE_RELEASE=1 HOST_DMD=dmd on linux x64. (*)

This is a backend bug, one of the createForeach() inline expansions inside lowerNonArrayAggregate() will have the op field of the new ForeachStatement instance clobbered with unrelated data.

On the second createForeach() call (s2.push(lstf.createForeach(aloc, pparams[1], null));)
inside void lowerNonArrayAggregate(StaticForeach lstf, Scope* sc):

  65eb1a:	48 8d 15 9f bd 42 00 	lea    rdx,[rip+0x42bd9f]
# a8a8c0 <_D3dmd9statement16ForeachStatement7__ClassZ>
...
  65eb28:	e8 33 6c 2b 00       	call   915760 <gc_malloc>
...
  65eb38:	f3 48 a5             	rep movs QWORD PTR es:[rdi],QWORD PTR ds:[rsi]
  65eb3b:	48 8b 55 f8          	mov    rdx,QWORD PTR [rbp-0x8]
  65eb3f:	4c 8b 42 18          	mov    r8,QWORD PTR [rdx+0x18]
  65eb43:	41 0f b6 48 15       	movzx  ecx,BYTE PTR [r8+0x15]
  65eb48:	88 8d 54 fe ff ff    	mov    BYTE PTR [rbp-0x1ac],cl ; 'aggrfe.op' is saved on stack,
; value = 0xa5 (TOK.foreach_)
  65eb4e:	4d 8b 78 20          	mov    r15,QWORD PTR [r8+0x20]
  65eb52:	48 8b b5 38 fa ff ff 	mov    rsi,QWORD PTR [rbp-0x5c8]
  65eb59:	48 89 b5 48 fe ff ff 	mov    QWORD PTR [rbp-0x1b8],rsi ; first 8 bytes of 'loc' param
  65eb60:	48 8b bd 40 fa ff ff 	mov    rdi,QWORD PTR [rbp-0x5c0]
  65eb67:	48 89 bd 50 fe ff ff 	mov    QWORD PTR [rbp-0x1b0],rdi ; last 4 bytes of 'loc' are saved,
; note that 'Loc' size is 12 bytes and a qword mov, but the problem is that there are only 4 bytes
; between [rbp-0x1ac] and [rbp-0x1b0] so 'op' gets overwritten.
...
  65eb80:	8a 8d 54 fe ff ff    	mov    cl,BYTE PTR [rbp-0x1ac] ; 'op' with garbage is read
  65eb86:	88 48 15             	mov    BYTE PTR [rax+0x15],cl ; and finally assigned

(*) Only DFLAGS=-inline is required though, but IIRC it changed the bug to the first occurrence of createForeach() in my case.

@dchidindu5
Copy link
Contributor Author

@BorisCarvajal is it something that can be fixed?

@BorisCarvajal
Copy link
Member

Yes and it should be fixed, bad codegen that manifests when DMD compile itself feels like a nasty bug,
but I think only @WalterBright knows exactly where to look in the backend.

However, you can workaround this bug by adding pragma(inline, false) to createForeach() signature, it seems to generate good code and the test suite should pass.
I would recommend doing it in a new branch and keep this one available for anyone attempting to fix it.

@dchidindu5
Copy link
Contributor Author

@BorisCarvajal is it something that can be fixed?

d recommend doing it in a new branch and keep this one available for anyone attempting

Alright, I would play around it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants