Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Moving AWS Gems from O3DE to AWS Repo #1

Merged
merged 467 commits into from
Dec 19, 2024

Conversation

AMZN-Gene
Copy link
Contributor

@AMZN-Gene AMZN-Gene commented Dec 18, 2024

Moving Open 3D Engine's AWS gems out of github.com/o3de/o3de Gems folder and into a dedicated AWS repository.
Note: Github history has been preserved.

Tested by removing the gems out of O3DE engine, registering the gems in this repository, and creating a GameLift Sample project.

lemonade-dm and others added 30 commits November 9, 2021 12:03
* Enable relocation of the Project Game Release Layout

Relocating the Project Game Release Layout to another directory on the file system failed due to the querying of the engine root failing due to the ComponentApplication::m_engineRoot not using the project path stored in the SettingsRegisry if the engine root cannot be detected

Removed the ApplicationRequestBus GetEngineRoot function.
The ComponentApplicationRequestBus has a function of the same name that returns the same path.

Removed the deprecated GetAppRoot function.
The path it returns has no defined value. It was not the engine root or the project root.
Removed unused CFileUtil and CFileUtil_impl functions that were invoking the ApplicationREquestBus GetEngineRoot function.
On the way to update the functions it was discovered that they aren't called

Added a CalculateBranchToken overload that can populate a fixed_string to avoid heap allocations

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Protect against an empty list of artifacts to remove when generating the
engine.pak

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Update AWS gem info content

Signed-off-by: liug <liug@amazon.com>

* Tweak requirement statement based on feedback

Signed-off-by: liug <liug@amazon.com>
…kle/StabToDevNov21

Signed-off-by: Nicholas Van Sickle <nvsickle@amazon.com>
…fault (#5470)

* Add option to cleanup resources that are retained by default

Signed-off-by: Pip Potter <61438964+lmbr-pip@users.noreply.github.com>
Merge `stabilization/2110` as of `8445481e18` into `development`
…eMapping tool schema (#5569)

* [Resource Mapping Tool] make top-level Account id optional in ResourceMapping tool schema

Signed-off-by: junbo75 <68558268+junbo75@users.noreply.github.com>
* Add a name to JobManager, all worker threads use that name to label their threads

Signed-off-by: rgba16f <82187279+rgba16f@users.noreply.github.com>

* update with PR feedback, changed to use AZStd::fixed_string::format

Signed-off-by: rgba16f <82187279+rgba16f@users.noreply.github.com>

* Fix copy paste errors using local variable names

Signed-off-by: rgba16f <82187279+rgba16f@users.noreply.github.com>
* Add basic autogen documentation to headers

Signed-off-by: rppotter <61438964+lmbr-pip@users.noreply.github.com>
…5475)

Signed-off-by: Stanko <stankoa@amazon.com>
…fault (#5470) (#5618)

* Add option to cleanup resources that are retained by default

Signed-off-by: Pip Potter <61438964+lmbr-pip@users.noreply.github.com>
* Add cvar for aws log level

Signed-off-by: onecent1101 <liug@amazon.com>

* Remove old crysystem registered cmd and add safeguard to get cvar from console

Signed-off-by: onecent1101 <liug@amazon.com>

* Suppress error for client auth unit test as logging was routed to warning before

Signed-off-by: onecent1101 <liug@amazon.com>
Signed-off-by: onecent1101 <liug@amazon.com>
Signed-off-by: Tommy Walton <waltont@amazon.com>
… (#5525)

Signed-off-by: Stanko <stankoa@amazon.com>
* Update resource mapping tool schema version

Signed-off-by: allisaurus <34254888+allisaurus@users.noreply.github.com>
* Enable and fix AWS Resource Mapping Tool on Linux
- Enable the trait AWSCORE_EDITOR_RESOURCE_MAPPING_TOOL_ENABLED for Linux
- Add additional traits to handle launching o3de local python on Linux
- Update the resource_mapping_tool.py to support preloading the shared libraries for Pyside (like the QtForPython gem) in order to resolve the local pyside2 libraries

Signed-off-by: Steve Pham <82231385+spham-amzn@users.noreply.github.com>

* Update pyside2 to a RUNTIME dependency

Signed-off-by: Steve Pham <82231385+spham-amzn@users.noreply.github.com>

* Update special linux preloading comments for clarity

Signed-off-by: Steve Pham <82231385+spham-amzn@users.noreply.github.com>

* Add CR at the end of traits files

Signed-off-by: Steve Pham <82231385+spham-amzn@users.noreply.github.com>

* - Remove unnecessary extra argument '--executable-path' and use '--binaries-path' instead
- Add linux only loading of pyside modules to 'setup_qt_environment' instead
- Update README.md

Signed-off-by: Steve Pham <82231385+spham-amzn@users.noreply.github.com>

* Updated unit tests for test_environment_utils.py

Signed-off-by: Steve Pham <82231385+spham-amzn@users.noreply.github.com>

* Fix indentation for Linux specific logic in environment_utils.py

Signed-off-by: Steve Pham <82231385+spham-amzn@users.noreply.github.com>

* Fix (more) indentation for Linux specific logic in environment_utils.py

Signed-off-by: Steve Pham <82231385+spham-amzn@users.noreply.github.com>

* Replace exit with return in setup_qt_environment

Signed-off-by: Steve Pham <82231385+spham-amzn@users.noreply.github.com>
* Add trait to enable/disable the Resource Mapping Tool

Signed-off-by: spham <82231385+spham-amzn@users.noreply.github.com>

* Moving ly_install_directory(DIRECTIORES Tools/ResourceMappingtool) into of the PAL_TRAIT_ENABLE_RESOURCE_MAPPING_TOOL block

Signed-off-by: Steve Pham <82231385+spham-amzn@users.noreply.github.com>
* Reading 3rdparty from manifest if it exists, removnig the need to pass it across the board, updated jenkins paths so 3rdparty is put into the workspace

Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>

* Removes unneded space

Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>

* Changes how we get the NDK folder

Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>

* needs delayed expansion

Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>

* Ninja is in the path, is not needed for Android

Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
* Unify resource mapping json schema across cpp and python usage

Signed-off-by: onecent1101 <liug@amazon.com>

* Update based on feedback

Signed-off-by: onecent1101 <liug@amazon.com>

* Add empty line at end of file

Signed-off-by: onecent1101 <liug@amazon.com>
* Fix clang 13 compilation problems

Re-instated ShowPlayedSequencesDebug logic by introducing no-op lambda
which replaces missing `Draw2dLabel` functionality ( added TODO marker
there )

Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>

* apply review suggestions

Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>

* Fix type in assert message

Signed-off-by: nemerle <96597+nemerle@users.noreply.github.com>
Signed-off-by: Chris Galvan <chgalvan@amazon.com>
…st O3DE guidline (#5028)

Signed-off-by: Junbo Liang <68558268+junbo75@users.noreply.github.com>
AMZN-Gene and others added 25 commits July 6, 2023 16:56
Signed-off-by: AMZN-Gene <genewalt@amazon.com>
… call if the Client Identity Pool was never set

Signed-off-by: AMZN-Gene <genewalt@amazon.com>
Signed-off-by: AMZN-Gene <genewalt@amazon.com>
Signed-off-by: VeryKaylin-player1 <133713531+VeryKaylin-player1@users.noreply.github.com>
Signed-off-by: VeryKaylin-player1 <133713531+VeryKaylin-player1@users.noreply.github.com>
Signed-off-by: Chris Galvan <chgalvan@amazon.com>
…other platforms but still need to test)

Signed-off-by: AMZN-Gene <genewalt@amazon.com>
…. Built windows and updated hash; still need to generate Linux

Signed-off-by: Gene Walters <genewalt@amazon.com>
Signed-off-by: AMZN-Gene <genewalt@amazon.com>
Signed-off-by: AMZN-Gene <genewalt@amazon.com>
Signed-off-by: Chris Galvan <chgalvan@amazon.com>
…005_o3de

Merged `stabilization/2310` to `main`
Co-authored-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
Signed-off-by: Gene Walters <32776221+AMZN-Gene@users.noreply.github.com>
Co-authored-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
Signed-off-by: Gene Walters <32776221+AMZN-Gene@users.noreply.github.com>
Signed-off-by: AMZN-Gene <genewalt@amazon.com>
* O3DE engine Gem rename support

The following changes have been made to all of the Gems that come with
the O3DE engine source code to make it easier to rename a Gem.

Updated all of the Gems CMakeLists.txt to use the "${gem_name}" variable
when referencing the Gem in CMake Targets.

This allows the name of the CMake targets to update whenever the
"gem_name" in the gem.json is changed.

Updated all of the `AZ_DECLARE_MODULE_CLASS` macro invocations to
reference the `O3DE_GEM_NAME` preprocessor token which is use to
substitute the "gem_name" in the gem.json for the extern "C"
`CreateModuleClass_Gem_*` function which the launcher_generator.cmake
invokes in Monolithic builds to create an AZ::Module derived class instance for each active gem.

resolves #17034

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Removed the boilerplate comments from Gems AZ_DECLARE_MODULE_CLASS

All the comments that started with "DO NOT MODIFY THIS LINE UNLESS YOU RENAME THE GEM" have been removed.

The externed "C" function that instantiates an AZ::Module derived class
in monolithic build now automatically adjust account for a rename of the
Gem.

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Replace all O3DE engine Gems and Gem Templates Gem variable boilerplate

The repeat boilerplate logic in each Gem has been replaced with a CMake
macro called "o3de_gem_setup" which uses the directory of the calling
CMakeLists.txt file to locate the nearest ancestor gem.json file.

It uses that directory of that gem.json file as the `${gem_path}` value.
It reads the "gem_name" field from that gem.json and sets that as the
`${gem_name}` variable.
It also reads the "version" field from that gem.json and sets that as
the `${gem_version}` variable.

Finally it sets the `${pal_dir}` variable based to the equivalent
`<CMakeLists.txt directory>/Platform/<configured platform>/` directory
using the directory of the calling CMakeLists.txt and the current
platform that CMake is being configured for(Windows, Linux, Android,
MacOS, iOS, or a "restricted platform").

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Updated the SimpleSchema, ChildAllocatorSchema `NumAllocateBytes`
function to also output the name of the Allocator when the Assert
Triggers

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Fixed memory tracking issue with the `HpAllocator::deallocate` function
returning a different value for the amount of bytes deallocated, than
were allocated.

The issue is that when the HpAllocator is used for aligned allocation of
>=32 bytes, it could find a block that could fulfill the allocation
request, but wasn't aligned.

If the block need to be adjusted for a size that was >0 but less than
the size of block header(16 bytes) + the size of the free node(40
bytes), than instead of being to split the block into two blocks where
the first block would be a free block that gets added to the free list
and the second block would be the actual aligned block, the code would
try to shift the previous block to be large enough such that the next
block would start aligned on the specified alignent using the
`shift_block` function.

This would result in previous block storing a larger size value.

So when the memory associated with the block was deallocated, it would
report a size larger than the size reported when it was allocated.

This fixes an issue where the Tracking the number of allocated bytes in
the ChildAllocatorSchema and SimpleAllocatorSchema would result in a
non-zero value after all of the allocations have been deleted.

This because the `ChildAllocatorSchema::allocate` function relies on
amount of bytes returned from the actual child allocator `allocate()` call
to be matched when the pointer that was allocated is deallocated via the
`ChildAllocatorSchema::deallocate` function which uses the child
allocator `deallocate()` call.

```
20:28:39  [----------] 1 test from MotionSetFixture

20:28:39  [ RUN      ] MotionSetFixture.MeshLoadTest

20:28:39
   D:\workspace\o3de\Code\Framework\AzCore\AzCore/Memory/ChildAllocatorSchema.h(155):
   error: Child Allocator "EMotionFXAllocator": Total allocated bytes is
   less than zero with a value of -16. Was deallocate() invoked with an
   address that is not associated with this allocator? This should never
   occur

   20:28:39
      D:\workspace\o3de\Code\Framework\AzCore\AzCore/UnitTest/TestTypes.h(76):
      error: Expected equality of these values:

      20:28:39    sizeBeforeTestRan

      20:28:39      Which is: 0

      20:28:39    sizeAfterTestRan

      20:28:39      Which is: 18446744073709551600

      20:28:39  for allocator EMotionFXAllocator with 0 allocation
	 records
```

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

* Update Code/Framework/AzCore/AzCore/Memory/HphaAllocator.cpp

Co-authored-by: Steve Pham <82231385+spham-amzn@users.noreply.github.com>
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>

---------

Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
Co-authored-by: Steve Pham <82231385+spham-amzn@users.noreply.github.com>
Signed-off-by: AMZN-Gene <genewalt@amazon.com>
Signed-off-by: AMZN-Gene <genewalt@amazon.com>
Signed-off-by: AMZN-Gene <genewalt@amazon.com>
Python update and relocation

* Relocates the 3rd Party Python Package
  * The Python 3rd Party Binary was originally downloaded to an internal `python/runtime/<package-name>` folder instead of the location of the other 3rd party binaries `${HOME}/.o3de/3rdParty`
  * For immutable installations like SNAP, this makes it impossible to import and additional python modules after the fact
  * Python commands were run directly from the binary folder. This change introduces running from a engine location specific virtual environment (under `${HOME}/.o3de/3rdParty/venv/{id}`) where **id** is computed from the engine path.
* Updates the python 3rd Party Package to 3.10.13
  *  Updates python to the latest 3.10 subversion for the latest security patches.
* Updates to references for the version of Python (3.10.5->3.10.13)
* Update AZ::DynamicModuleHandle to add support for loading general shared libraries with the option of enabling global symbols on systems that support it, and add the ability to disable platform-specific filenaming support.
* Add a new support method 'Get3rdPartyDirectory' in AZ::Utils
* Introducing a new PythonLoader to support runtime python operations
  * Add a PAL Trait gated operation in the constructor to perform loading of the python shared library into the global symbols space (see updates to AZ::DynamicModuleHandle)
  * Provides path resolution for python environment related paths
* Updates to the python bootstrap (get_python) to support the new location and workflow
* Updates to the python script and EditorPythonBindings to use the python venv instead
* Misc python2->python3 fixes in legacy scripts

Signed-off-by: Steve Pham <82231385+spham-amzn@users.noreply.github.com>
Co-authored-by: Alex Peterson <26804013+AMZN-alexpete@users.noreply.github.com>
Co-authored-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
* Add methods for client configiguration in HTTP Requestor
* Add http client configuration methods definitions to awsClientAuthGemMock tests

---------

Signed-off-by: Artur Kamieniecki <artur.kamieniecki@robotec.ai>
…_stabilization_2409_to_main

No diffs between this pr and upstream/stabilization/2409

Signed-off-by: Nicholas Lawson <70027408+nick-l-o3de@users.noreply.github.com>
Signed-off-by: AMZN-Gene <genewalt@amazon.com>
Signed-off-by: Gene Walters <genewalt@amazon.com>
@AMZN-Gene AMZN-Gene marked this pull request as ready for review December 19, 2024 17:21
Copy link

@moudgils moudgils left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@AMZN-Gene AMZN-Gene merged commit b217d3c into aws:main Dec 19, 2024
@AMZN-Gene AMZN-Gene deleted the move_aws_gems_to_aws_repo branch December 19, 2024 21:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.