Skip to content

Commit

Permalink
Apple Silicon support (#7330)
Browse files Browse the repository at this point in the history 
Add support for Apple Silicon architecture. Update dependancies and documentation to support this. 

Co-authored-by: Stefano Bonicatti <stefano.bonicatti@gmail.com>
Co-authored-by: Adam Meily <adam.meily@trailofbits.com>
  • Loading branch information
@alessandrogario @Smjert @ameily
3 people authored Dec 21, 2021
1 parent bd38031 commit 48d510b
Show file tree
Hide file tree
Showing 392 changed files with 124,835 additions and 9,416 deletions.
294 changes: 208 additions & 86 deletions .github/workflows/build_x86.yml → .github/workflows/hosted_runners.yml
View file

Large diffs are not rendered by default.

9 changes: 5 additions & 4 deletions .github/workflows/build_aarch64.yml → .github/workflows/self_hosted_runners.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ on:
env:
PACKAGING_REPO: https://github.com/osquery/osquery-packaging
PACKAGING_COMMIT: 8168fd7213b23447f1bfbe0b54d4e8b61730b08d
SUBMODULE_CACHE_VERSION: 1

# If the initial code sanity checks are passing, then one job
# per [`platform` * `build_type`] will start, building osquery
Expand All @@ -42,7 +43,7 @@ jobs:
runs-on: ubuntu-18.04

container:
image: osquery/builder18.04:2c2b85cbd
image: osquery/builder18.04:a4961d234
options: --privileged --init -v /var/run/docker.sock:/var/run/docker.sock

steps:
Expand Down Expand Up @@ -160,7 +161,7 @@ jobs:
runs-on: ${{ matrix.os }}

container:
image: osquery/builder18.04:2c2b85cbd
image: osquery/builder18.04:a4961d234
options: --privileged --init -v /var/run/docker.sock:/var/run/docker.sock

strategy:
Expand Down Expand Up @@ -266,10 +267,10 @@ jobs:
path: ${{ steps.build_paths.outputs.SOURCE }}/.git/modules

key: |
gitmodules_${{ matrix.cache_key }}_${{ github.sha }}
gitmodules_${{ matrix.cache_key }}_${{env.SUBMODULE_CACHE_VERSION}}_${{ github.sha }}
restore-keys: |
gitmodules_${{ matrix.cache_key }}
gitmodules_${{ matrix.cache_key }}_${{env.SUBMODULE_CACHE_VERSION}}
- name: Update the git submodules
working-directory: ${{ steps.build_paths.outputs.SOURCE }}
Expand Down
57 changes: 42 additions & 15 deletions .gitmodules
View file
Original file line number Diff line number Diff line change
Expand Up @@ -103,24 +103,9 @@
[submodule "libraries/cmake/source/gflags/src"]
path = libraries/cmake/source/gflags/src
url = https://github.com/gflags/gflags
[submodule "libraries/cmake/source/aws-sdk-cpp/aws-c-common_src"]
path = libraries/cmake/source/aws-sdk-cpp/aws-c-common_src
url = https://github.com/awslabs/aws-c-common
[submodule "libraries/cmake/source/aws-sdk-cpp/aws-c-event-stream_src"]
path = libraries/cmake/source/aws-sdk-cpp/aws-c-event-stream_src
url = https://github.com/awslabs/aws-c-event-stream
[submodule "libraries/cmake/source/aws-sdk-cpp/aws-checksums_src"]
path = libraries/cmake/source/aws-sdk-cpp/aws-checksums_src
url = https://github.com/awslabs/aws-checksums
[submodule "libraries/cmake/source/aws-sdk-cpp/aws-sdk-cpp_src"]
path = libraries/cmake/source/aws-sdk-cpp/aws-sdk-cpp_src
url = https://github.com/aws/aws-sdk-cpp
[submodule "libraries/cmake/source/boost/src"]
path = libraries/cmake/source/boost/src
url = https://github.com/boostorg/boost
[submodule "libraries/cmake/source/icu/src"]
path = libraries/cmake/source/icu/src
url = https://github.com/unicode-org/icu
[submodule "libraries/cmake/source/augeas/gnulib/src"]
path = libraries/cmake/source/augeas/gnulib/src
url = https://github.com/osquery/third-party-gnulib
Expand All @@ -136,3 +121,45 @@
[submodule "libraries/cmake/source/libcap/src"]
path = libraries/cmake/source/libcap/src
url = https://kernel.googlesource.com/pub/scm/libs/libcap/libcap.git
[submodule "libraries/cmake/source/aws-sdk-cpp/src/aws-crt-cpp"]
path = libraries/cmake/source/aws-sdk-cpp/src/aws-crt-cpp
url = https://github.com/awslabs/aws-crt-cpp.git
[submodule "libraries/cmake/source/aws-sdk-cpp/src/aws-c-auth"]
path = libraries/cmake/source/aws-sdk-cpp/src/aws-c-auth
url = https://github.com/awslabs/aws-c-auth.git
[submodule "libraries/cmake/source/aws-sdk-cpp/src/aws-c-cal"]
path = libraries/cmake/source/aws-sdk-cpp/src/aws-c-cal
url = https://github.com/awslabs/aws-c-cal.git
[submodule "libraries/cmake/source/aws-sdk-cpp/src/aws-c-common"]
path = libraries/cmake/source/aws-sdk-cpp/src/aws-c-common
url = https://github.com/awslabs/aws-c-common.git
[submodule "libraries/cmake/source/aws-sdk-cpp/src/aws-c-compression"]
path = libraries/cmake/source/aws-sdk-cpp/src/aws-c-compression
url = https://github.com/awslabs/aws-c-compression.git
[submodule "libraries/cmake/source/aws-sdk-cpp/src/aws-c-event-stream"]
path = libraries/cmake/source/aws-sdk-cpp/src/aws-c-event-stream
url = https://github.com/awslabs/aws-c-event-stream.git
[submodule "libraries/cmake/source/aws-sdk-cpp/src/aws-c-http"]
path = libraries/cmake/source/aws-sdk-cpp/src/aws-c-http
url = https://github.com/awslabs/aws-c-http.git
[submodule "libraries/cmake/source/aws-sdk-cpp/src/aws-c-io"]
path = libraries/cmake/source/aws-sdk-cpp/src/aws-c-io
url = https://github.com/awslabs/aws-c-io.git
[submodule "libraries/cmake/source/aws-sdk-cpp/src/aws-c-mqtt"]
path = libraries/cmake/source/aws-sdk-cpp/src/aws-c-mqtt
url = https://github.com/awslabs/aws-c-mqtt.git
[submodule "libraries/cmake/source/aws-sdk-cpp/src/aws-c-s3"]
path = libraries/cmake/source/aws-sdk-cpp/src/aws-c-s3
url = https://github.com/awslabs/aws-c-s3.git
[submodule "libraries/cmake/source/aws-sdk-cpp/src/aws-checksums"]
path = libraries/cmake/source/aws-sdk-cpp/src/aws-checksums
url = https://github.com/awslabs/aws-checksums.git
[submodule "libraries/cmake/source/aws-sdk-cpp/src/aws-lc"]
path = libraries/cmake/source/aws-sdk-cpp/src/aws-lc
url = https://github.com/awslabs/aws-lc.git
[submodule "libraries/cmake/source/aws-sdk-cpp/src/s2n"]
path = libraries/cmake/source/aws-sdk-cpp/src/s2n
url = https://github.com/awslabs/s2n.git
[submodule "libraries/cmake/source/aws-sdk-cpp/src/aws-sdk-cpp"]
path = libraries/cmake/source/aws-sdk-cpp/src/aws-sdk-cpp
url = https://github.com/aws/aws-sdk-cpp
17 changes: 10 additions & 7 deletions CMakeLists.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,7 @@
#
# SPDX-License-Identifier: (Apache-2.0 OR GPL-2.0-only)

cmake_minimum_required(VERSION 3.17.5 FATAL_ERROR)

cmake_policy(SET CMP0083 NEW)
cmake_minimum_required(VERSION 3.21.4 FATAL_ERROR)

# toolchain.cmake needs to be included before project() because the former sets the compiler path for the custom toolchain,
# if the user specify it and the latter does compiler detection.
Expand Down Expand Up @@ -110,22 +108,19 @@ function(importLibraries)
"Linux:libudev"
"Linux,Darwin,Windows:libxml2"
"Linux,Darwin,Windows:linenoise-ng"
"Linux,Darwin:lldpd"
"Linux,Darwin,Windows:lzma"
"Linux,Darwin:popt"
"Linux,Darwin,Windows:rapidjson"
"Linux,Darwin,Windows:rocksdb"
"Linux,Darwin,Windows:sleuthkit"
"Linux,Darwin:smartmontools"
"Linux,Darwin,Windows:sqlite"
"Linux,Darwin:ssdeep-cpp"
"Linux,Darwin:ssdeep"
"Linux,Darwin,Windows:thrift"
"Linux:util-linux"
"Linux,Darwin,Windows:yara"
"Linux,Darwin,Windows:zlib"
"Linux,Darwin,Windows:zstd"
"Linux,Darwin,Windows:openssl"
"Linux,Darwin,Windows:icu"
"Linux:expat"
"Linux:dbus"
"Linux:libcap"
Expand All @@ -149,6 +144,14 @@ function(importLibraries)
)
endif()

# We want to remove support for lldpd and smartmontools, so skip support for M1
if(NOT DEFINED PLATFORM_MACOS OR NOT "${TARGET_PROCESSOR}" STREQUAL "aarch64")
list(APPEND library_descriptor_list
"Linux,Darwin:lldpd"
"Linux,Darwin:smartmontools"
)
endif()

foreach(library_descriptor ${library_descriptor_list})
# Expand the library descriptor
string(REPLACE ":" ";" library_descriptor "${library_descriptor}")
Expand Down
11 changes: 11 additions & 0 deletions cmake/globals.cmake
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,17 @@ elseif(CMAKE_SYSTEM_PROCESSOR STREQUAL "x86_64")
elseif(CMAKE_SYSTEM_PROCESSOR STREQUAL "aarch64")
# *nix AArch64
set(TARGET_PROCESSOR "aarch64")
elseif(CMAKE_SYSTEM_PROCESSOR STREQUAL "arm64")
# Apple Silicon
set(TARGET_PROCESSOR "aarch64")
else()
message(FATAL_ERROR "Unsupported architecture ${CMAKE_SYSTEM_PROCESSOR}")
endif()

if("arm64" IN_LIST CMAKE_OSX_ARCHITECTURES)
set(TARGET_PROCESSOR "aarch64")
endif()

# TODO(alessandro): Add missing defines: PLATFORM_FREEBSD
if("${CMAKE_SYSTEM_NAME}" STREQUAL "Linux")
set(PLATFORM_POSIX 1)
Expand Down Expand Up @@ -77,6 +84,10 @@ if(DEFINED PLATFORM_WINDOWS)
enable_language(ASM_MASM)
endif()

if(DEFINED PLATFORM_POSIX)
enable_language(ASM)
endif()

if(DEFINED PLATFORM_MACOS)
enable_language(OBJCXX)
endif()
8 changes: 8 additions & 0 deletions cmake/options.cmake
View file
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,14 @@ if("${THIRD_PARTY_REPOSITORY_URL}" STREQUAL "")
set(THIRD_PARTY_REPOSITORY_URL "https://s3.amazonaws.com/osquery-packages")
endif()

# When building on macOS, make sure we are only building one architecture at a time
if(PLATFORM_MACOS)
list(LENGTH CMAKE_OSX_ARCHITECTURES osx_arch_count)
if(osx_arch_count GREATER 1)
message(FATAL_ERROR "The CMAKE_OSX_ARCHITECTURES setting can only contain one architecture at a time")
endif()
endif()

detectOsqueryVersion()

message(STATUS "osquery version: ${OSQUERY_VERSION_INTERNAL}")
29 changes: 4 additions & 25 deletions docs/wiki/development/building.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ The supported compilers are: the osquery toolchain (LLVM/Clang 9.0.1) on Linux,

## Prerequisites

Git (>= 2.14.0), CMake (>= 3.17.5), Python 3 are required to build. The rest of the dependencies are downloaded by CMake.
Git (>= 2.14.0), CMake (>= 3.21.4), Python 3 are required to build. The rest of the dependencies are downloaded by CMake.

The default build type is `RelWithDebInfo` (optimizations active + debug symbols) and can be changed in the CMake configure phase by setting the `CMAKE_BUILD_TYPE` flag to `Release` or `Debug`.

Expand Down Expand Up @@ -38,11 +38,8 @@ sudo tar xvf osquery-toolchain-1.1.0-${ARCH}.tar.xz -C /usr/local

# Download and install a newer CMake.
# Afterward, verify that `/usr/local/bin` is in the `PATH` and comes before `/usr/bin`.
# Please see the note below for building CMake on aarch64.
if [[ "${ARCH}" = "x86_64" ]]; then
wget https://cmake.org/files/v3.17/cmake-3.17.5-Linux-${ARCH}.tar.gz
sudo tar xvf cmake-3.17.5-Linux-${ARCH}.tar.gz -C /usr/local --strip 1
fi
wget https://cmake.org/files/v3.21/cmake-3.21.4-linux-${ARCH}.tar.gz
sudo tar xvf cmake-3.21.4-linux-${ARCH}.tar.gz -C /usr/local --strip 1

# Download source
git clone https://github.com/osquery/osquery
Expand All @@ -54,24 +51,6 @@ cmake -DOSQUERY_TOOLCHAIN_SYSROOT=/usr/local/osquery-toolchain ..
cmake --build . -j10 # where 10 is the number of parallel build jobs
```

**CMake on aarch64**

If you are building for aarch64 then please notes CMake > 3.19.3 includes aarch64 Linux binaries,
however it also has a bug that prevents creating RPMs properly.

Prefer to build and install CMake from source:

```bash
wget https://github.com/Kitware/CMake/releases/download/v3.17.5/cmake-3.17.5.tar.gz
sudo apt install gcc g++ libssl-dev
tar zxvf cmake-3.17.5.tar.gz
pushd cmake-3.17.5/
./bootstrap -- -DCMAKE_BUILD_TYPE:STRING=Release
make -j`nproc`
sudo make install
popd
```

## macOS

The current build of osquery supports deployment to the same set of macOS versions (macOS 10.12 and newer). _Building_ osquery from source on macOS now requires 10.15 Catalina.
Expand Down Expand Up @@ -134,7 +113,7 @@ After changing that key, reboot your build machine and re-attempt the build.

Note: It may be easier to install these prerequisites using [Chocolatey](https://chocolatey.org/).

- [CMake](https://cmake.org/) (>= 3.17.5): the MSI installer is recommended. During installation, select the option to add it to the system `PATH` for all users. If there is any older version of CMake installed (e.g., using Chocolatey), uninstall that version first! Do not install CMake using the Visual Studio Installer, because it contains an older version than required.
- [CMake](https://cmake.org/) (>= 3.21.4): the MSI installer is recommended. During installation, select the option to add it to the system `PATH` for all users. If there is any older version of CMake installed (e.g., using Chocolatey), uninstall that version first! Do not install CMake using the Visual Studio Installer, because it contains an older version than required.
- Visual Studio 2019 (2 options)
1. [Visual Studio 2019 Build Tools Installer](https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=BuildTools&rel=16) (without Visual Studio): In the installer choose the "C++ build tools" workload, then on the right, under "Optional", select "MSVC v142 - VS 2019 C++", "Windows 10 SDK", and "C++ Clang tools for Windows".
2. [Visual Studio 2019 Community Installer](https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=Community&rel=16): In the installer choose the "Desktop development with C++" workload, then on the right, under "Optional", select "MSVC v142 - VS 2019 C++", "Windows 10 SDK", and "C++ Clang tools for Windows".
Expand Down
117 changes: 117 additions & 0 deletions libraries/cmake/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
This folder contains all the third party libraries that osquery needs.
All the third party libraries in this folder are built from source.

The ones under `source` use CMake as the build system and they are brought in as submodules.
Their configuration logic is not run, so that their build is always the same, no matter the version of the platform they are built on.
To achieve that, we generate and save their configuration artifacts in these folders, in a way that depends on each platform, but the general idea is that we want to have compatibility with the oldest version of the platform we want to support.

Finally, since we don't want to run their configuration logic and build system when building osquery, we write a CMakeLists.txt which builds the sources for the library and hardcodes additional compiler options that are not included in the generated files, but that the original build system to the compiler invocation when building.

The ones under `formula` (currently OpenSSL only) will still use CMake to build, but it passes through a different build system, which CMake executes. They can be submodules or, as in the case of OpenSSL, archives that CMake takes care to download.
This is because reproducing that build system with CMake has proven to be too complex.
Particular care is needed for these libraries, as with the `source` ones, so that they do not directly depend on features that are present only on the system they are currently built on.

# Linux

Beyond what previously described, we use a custom toolchain ([osquery-toolchain](https://github.com/osquery/osquery-toolchain)) that permits us to build osquery on either new distros or the oldest targeted distro (bar some bugs, see later).

These are the current targeted versions:

## x86

CentOS 6.10

```sh
cat /etc/centos-release
CentOS release 6.10 (Final)
```

```sh
ldd --version
ldd (GNU libc) 2.12
[...]
```

```sh
yum info glibc

[...]
Version: 2.12
Release: 1.212.el6_10.3
[...]
```

```sh
uname -r
2.6.32-754.18.2.el6.x86_64
```

## AArch64

Ubuntu 16.04 on AWS Graviton

```sh
cat /etc/os-release

[...]
VERSION="16.04.7 LTS (Xenial Xerus)"
[...]

```

```sh
ldd --version
ldd (Ubuntu GLIBC 2.23-0ubuntu11.3) 2.23
[...]
```

```sh
apt show libc-bin

[...]
Version: 2.23-0ubuntu11.3
[...]
```

```sh
uname -r
4.15.0-1099-aws
```

## Troubleshooting

There are some issues with the osquery-toolchain 1.1.0 when trying to use it on CentOS 6.10.
Binaries like `as`, `ar`, etc need to be symlinked to their llvm counterpart, since the original ones are fully static and contain a glibc version that won't work on that old distribution, and will throw a `FATAL: kernel too old`.
To fix this, supposing that the osquery-toolchain has been installed under `/usr/local/osquery-toolchain`, run the following commands:

```sh
cd /usr/local/osquery-toolchain/usr/bin

rm as; ln -s llvm-as as
rm ar; ln -s llvm-ar ar
rm objcopy; ln -s llvm-objcopy objcopy
rm ranlib; ln -s llvm-ranlib ranlib
rm objdump; ln -s llvm-objdump objdump
rm nm; ln -s llvm-nm nm
rm strip; ln -s llvm-strip strip
```


# macOS

The system compiler is used on Big Sur 11.6, XCode 13, SDK 11.3.
Both x86 and M1 architectures are built on an x86 machine, using the toolchain ability to cross-compile.

## x86

The deployment target is 10.12

## M1

The deployment target is 10.15


# Windows

The system compiler is used on Windows 10, Visual Studio 2019.
The SDK used currently is not fixed (depends on what's available on the CI or the developer machine), but in general the target we attempt to have is Windows 7.
Loading

0 comments on commit 48d510b

Please sign in to comment.