Create document on safely executing repros Fixes #45341 #45456
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dotnet-issue-labeler
bot
added
Area-NetSDK
untriaged
| If none of the above methods work, it may be viable to read all customer code carefully to ensure that no part of it is risky or malicious. | ||
|
|
||
| Some examples of specific patterns to watch out for that can cause issues include: | ||
| 1. PackageReferences. These download code from the internet and permit its execution locally. If you are unfamiliar with any package in a PackageReference, this is insecure. |
There was a problem hiding this comment.
NuGet.Config could point to a malicious site, even if the package name is reserved at nuget.org.
dsplaisted
approved these changes
Dec 20, 2024
|
|
||
| The other disadvantage of this approach is that it can take a lot of time to select appropriate parameters for your VM, start it, connect to it, set it up properly for your scenario, and execute the scenario. Unlike Sandbox, it can retain certain information across restarts, but it may lose active work just as your computer loses active work when you restart it. It's important to keep in mind that many VMs restart automatically at a fixed time every day when not in use to save resources. | ||
|
|
||
| That said, this is a secure means for executing a customer's scenario, and it works for any operating system and can even be configured to work for other architectures. Note that it does cost real money to use. |
There was a problem hiding this comment.
Note that it does cost real money to use.
I think you're referring to VMs in the cloud. It's also possible to create a local VM (for example using Hyper-V).
Forgind
commented
Dec 20, 2024
Thanks for the review! I set a reminder for myself to ping them in January when they're back from vacations 🙂 |
JanKrivanek
approved these changes
Dec 23, 2024
|
|
||
| When customers discover issues with the .NET SDK, we often need more information to see how exactly their scenarios differ from the (presumably working) mainline scenarios. This additional information often takes the form of a 'repro' or set of steps by which we can see the error ourselves and walk through what is happening as their scenario plays out and how it ultimately diverged from our expectations. | ||
|
|
||
| Blindly executing unvetted customer code can be a security hazard, however, not just for the machine executing the code but for any machine on the same network and any service accessible using credentials they can access through those machines. In this way, a malicious actor can exfiltrate sensitive Microsoft data, including information about other Microsoft employees, proprietary code, or private customer data. They may even be able to take down a service or introduce further security bugs in shipping products. Indeed, the most common vector hackers use to gain access is through compromising one or more individual users with employee credentials. At Microsoft where security is paramount, we want to prevent such hacks. |
There was a problem hiding this comment.
Suggested change
| Blindly executing unvetted customer code can be a security hazard, however, not just for the machine executing the code but for any machine on the same network and any service accessible using credentials they can access through those machines. In this way, a malicious actor can exfiltrate sensitive Microsoft data, including information about other Microsoft employees, proprietary code, or private customer data. They may even be able to take down a service or introduce further security bugs in shipping products. Indeed, the most common vector hackers use to gain access is through compromising one or more individual users with employee credentials. At Microsoft where security is paramount, we want to prevent such hacks. | |
| Blindly executing, or even just opening in IDE without intention to build, unvetted customer code can be a security hazard, however, not just for the machine executing the code but for any machine on the same network and any service accessible using credentials they can access through those machines. In this way, a malicious actor can exfiltrate sensitive Microsoft data, including information about other Microsoft employees, proprietary code, or private customer data. They may even be able to take down a service or introduce further security bugs in shipping products. Indeed, the most common vector hackers use to gain access is through compromising one or more individual users with employee credentials. At Microsoft where security is paramount, we want to prevent such hacks. |
| To provide information to customers about collecting binlogs, direct them to: | ||
| https://aka.ms/msbuild/binlog | ||
|
|
||
| It is important to note, however, that binlogs often collect secrets as part of the information relevant to the build. A binlog of a simplified repro can work as well, but it may be difficult to create a simple repro. Customers are sometimes concerned (with good reason) about sharing binlogs of their real (or simplified) builds. If that is the case, or if the reported issue does not involve MSBuild, another option may be more relevant. |
There was a problem hiding this comment.
Note there are currently options for redact secrets from the binlog - either via viewer (https://msbuildlog.com/#redaction) or with binlogtool (binlogtool redact)
|
|
||
| Blindly executing unvetted customer code can be a security hazard, however, not just for the machine executing the code but for any machine on the same network and any service accessible using credentials they can access through those machines. In this way, a malicious actor can exfiltrate sensitive Microsoft data, including information about other Microsoft employees, proprietary code, or private customer data. They may even be able to take down a service or introduce further security bugs in shipping products. Indeed, the most common vector hackers use to gain access is through compromising one or more individual users with employee credentials. At Microsoft where security is paramount, we want to prevent such hacks. | ||
|
|
||
| This document contains recommended best practices on how to securely test users' code. They are arranged in order or security with the most secure at the top. This should also be the priority you should use to stay secure when executing code. |
There was a problem hiding this comment.
Suggested change
| This document contains recommended best practices on how to securely test users' code. They are arranged in order or security with the most secure at the top. This should also be the priority you should use to stay secure when executing code. | |
| This document contains recommended best practices on how to securely test users' code. They are arranged in order of security with the most secure at the top. This should also be the priority you should use to stay secure when executing code. |
|
|
||
| Depending on how well you understand the customer's scenario, you may or may not understand ahead of time the most likely problem, but if it's possible to create your own solution that reproduces the issue the customer alluded to in their report, and you understand all the steps it took to create that solution, that is a safe method for obtaining a project to test and debug. Note that this is only a secure method if you understand all steps involved in creating the test project. "Copy project file provided by customer to disk and build it" does not count as a well-understood pair of steps. | ||
|
|
||
| ### Create a Virtual Machine Without Your Credentials |
There was a problem hiding this comment.
Should we add an option for running within docker?
There area readily available images from various channels of sdk (nightly/preview/release) https://hub.docker.com/r/microsoft/dotnet
It feels more lightweight then full blown VM
There was a problem hiding this comment.
Docker doesn't really make isolation and security promises, so no.
|
|
||
| ### Read Code | ||
|
|
||
| If none of the above methods work, it may be viable to read all customer code carefully to ensure that no part of it is risky or malicious. |
There was a problem hiding this comment.
It should be noted here that a basic editor should be prefered. Any IDE (VS, Raider etc.) or extensions (DevKit) that run design time build can cause unintended and unnoticed execution of untrusted code.
| 4. Typos. NuGet packages starting with Microsoft. and System. can only be pushed to nuget.org by Microsoft accounts, but Mircosoft and Sysstem are unregulated. | ||
| 5. Malicious NuGet.Config files. Even if a PackageReference points to a package you're familiar with, perhaps even a Microsoft. or System. package, a NuGet.Config file can dictate searching in a different package source first that may not be secure and may include a malicious version of the package. | ||
|
|
||
| More examples of patterns that can easily be exploited by malicious users can be found [here](https://aka.ms/msbuild-security-documentation) |
There was a problem hiding this comment.
Other things to watch for:
- .rsp files injecting the custom extensibility
- autoinclude files - Directory.Build.[props|target], .user project files - as additional logic can be injected via those
- location where we open the projects from - as e.g. opening from 'Downloads' folder is supectible to injecting logic from unrelated auto-include files that can be placed to Downloads with relatively low privileges
- attempts to write to locations outside of the root of current repo/solution (especially to locations like drive roots, msbuild install location, nuget cache, etc.) - this can inject logic to subsequent unrelated builds
- Some additional tasks of interest: [internal link] https://microsoft.sharepoint.com/:w:/t/ManagedLanguagesIDE/EX0mgFi9TGJEkAKi3JYd7_EBZxyaXX1-tIze9Hbi2megLw?e=QUnpME - this however doesn't feel like it should be in scope of this doc (especially on public repo)
There was a problem hiding this comment.
These are good additions, but I don't want to overload this section with too many things to watch out for. I'd rather people following this option go to the security documentation you're working on and see a more complete set there 🙂
| 1. Most importantly, do not keep any vestige of the repro on your machine. Do not follow their steps. | ||
| 2. Add a note to the issue that it was malicious. Include this information in the title if possible. | ||
| 3. Delete the contents of the repro steps. (On GitHub, it can still be found by looking at your edit.) | ||
| 4. Report your finding to your manager and your security team. |
blowdart
requested changes
Jan 9, 2025
| https://aka.ms/msbuild/binlog | ||
|
|
||
| It is important to note, however, that binlogs often collect secrets as part of the information relevant to the build. A binlog of a simplified repro can work as well, but it may be difficult to create a simple repro. Customers are sometimes concerned (with good reason) about sharing binlogs of their real (or simplified) builds. If so, they can attempt to redact secrets from their binlog, either via the [MSBuild Structured Log Viewer](https://msbuildlog.com/#redaction) or with [binlogtool](https://www.nuget.org/packages/binlogtool). If that isn't sufficient, or if the reported issue does not involve MSBuild, another option may be more relevant. | ||
|
|
There was a problem hiding this comment.
There is an assumption here that the binlog viewer is safe to use on untrusted files. Is that assumption correct?
There was a problem hiding this comment.
I asked KirillOsenkov:
"Is it possible to craft a malicious binlog?"
and he said:
"I really don't think so"
The structured log viewer has an assumption of what the type is of everything it's going to read. If it sees something that's in the right form, it initializes it. If not, it throws an error. The types are fairly basic individually, so it isn't a viable attack vector, as far as we could tell.
|
|
||
| Depending on how well you understand the customer's scenario, you may or may not understand ahead of time the most likely problem, but if it's possible to create your own solution that reproduces the issue the customer alluded to in their report, and you understand all the steps it took to create that solution, that is a safe method for obtaining a project to test and debug. Note that this is only a secure method if you understand all steps involved in creating the test project. "Copy project file provided by customer to disk and build it" does not count as a well-understood pair of steps. | ||
|
|
||
| ### Create a Virtual Machine Without Your Credentials |
There was a problem hiding this comment.
Docker doesn't really make isolation and security promises, so no.
| Windows Sandbox permits safely running any operation in a controlled, isolated environment where it will not affect your machine or Microsoft in any detrimental way. More information can be found [here](https://learn.microsoft.com/windows/security/application-security/application-isolation/windows-sandbox/windows-sandbox-overview). If the steps a customer provides for reproducing their scenario can be executed in Windows Sandbox, it is the safest place to do so. Unfortunately, Sandbox does come with a few notable downsides: | ||
| 1. Windows Sandbox only exists for Windows. If a customer's scenario only reproduces on macOS or linux, it will not be able to be reproduced on Windows, and Sandbox is not helpful. | ||
| 2. When Sandbox exits for any reason, all data is lost. This means, for instance, that if the computer hosting Sandbox restarts while Sandbox is open, any progress you might have made in putting together the customer's repro scenario is lost. | ||
| 3. Sandbox is essentially empty when you first create it. It has internet access as long as your host computer has internet access, but it will be missing various programs and tools you may need to reproduce the customer's problem. Such programs and tools need to be downloaded anew every time you want to use Sandbox. |
There was a problem hiding this comment.
You can (and probably should) disable sandbox networking after setup, before you run customer code.
|
|
||
| #### Docker | ||
|
|
||
| As a corollary to using VMs to sequester the customer's scenario away from sensitive parts of your machine, containers (notably Docker containers) are intended to create a small, self-contained environment in which to operate. They're cheaper to create than full VMs, though with more stringent resource limitations and some missing or altered functionality, they may not work for some scenarios. Even so, if they work, they can be a safer and cheaper option than creating a VM. Make sure to follow general best practices such as running in rootless mode if possible and avoiding signing in using your credentials. |
There was a problem hiding this comment.
As above docker doesn't make many security promises, it has local network access by default, etc. General advice would be to isolate from the network as a starting point.
| 1. Most importantly, do not keep any vestige of the repro on your machine. Do not follow their steps. | ||
| 2. Add a note to the issue that it was malicious. Include this information in the title if possible. | ||
| 3. Delete the contents of the repro steps. (On GitHub, it can still be found by looking at your edit.) | ||
| 4. Report the infraction using https://aka.ms/reportitnow |
There was a problem hiding this comment.
Call out this is for Microsoft employees only
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #45341
/cc: @JanKrivanek